Continued from the blog post...

Note: For this demonstration, we are using the LTS (long term support) version of Unity 2018.4, which can be downloaded from the Unity Hub. Newer versions may not work as intended. If you are using a version of GameDriver older than October, 2020, please see steps 3 and beyond in the reply to this post below.


Step 1 - Create the 2D Game Kit project

This is a fairly straight forward process. Simply start a new 2D Project in Unity and go to the Asset Store (Window Menu > Asset Store) and search for "2D Game Kit". It should be the first result available from Unity Technologies. From there, simply Import the assets into your project.


Follow the prompts, and be sure to select all assets in the package to import.

Step 2 - Setting up the game to play

Once the assets have been fully imported, which can take several minutes, simply open the Scenes folder in the Project window and open the "Start" scene.


Once the scene is open, you can play the game by pressing the "Play" button. Give the game a try! You can control Ellen, the character, using the WSAD or arrow buttons to move, Spacebar to jump, and O and K keys to attack. The objective of the game is to find 3 keys hidden (poorly) throughout the 4 levels, which unlocks the door to the boss.

We will automate the collection of one key in this tutorial, and give you enough tools to be able to automate tests for the rest if you wish.

Download the GameDriver software by reaching out to us here. Once you have the software and license, follow the installation instructions (we'll wait here for you).

Step 3 - Creating your first automated test

You back? Great. With the GameDriver agent installed, you can start building your automation in Visual Studio. If you followed the installation instructions to the end, you should have a base NUnit test to work from. Just be sure you have added the using gdio.unity_api; declaration and reference to the gdio.unity_api.dll in the project references. 
We're going to start by connecting to the Unity IDE, which is handled with the statement:

Api.WaitForGame("127.0.0.1");

The next few lines will wait 4 seconds for the game to load, then enable the GameDriver agent hooks for keyboard and mouse control.


Api.Wait(4000);
Api.EnableKeybordHooks();
Api.EnableMouseHooks();


Wait for the StartButton object to be enabled, then click it to start the game and wait a few more seconds for the next scene to load. To do this, we replace Api.Wait() with Api.WaitForObject() and wait for the StartButton to load.


Api.WaitForObject("//*[@name='StartButton']");
Api.ClickObject("//*[@name='StartButton']", Api.MouseButtons.LEFT);


When the game plays, inputs are calculated by the number of frames a key is pressed, which can fluctuate depending on the performance of the machine running the game. So rather than hard code the duration of key presses, a best practice is to calculate the number of frames to input using the frames per second (FPS) captured from the game engine. First, we will capture the FPS using the GameDriver API.


var fps = Api.GetLastFPS();


Next, we need a target object for our player character to move towards. In level 1, there is a path down to the next scene that is also the location of an InfoPost object, but not the first one in the scene. So we will find the target InfoPost, and store it's x coordinate for future reference using the Api.GetObjectPosition() function. We then log the value for troubleshooting purposes.


var infoPost = Api.GetObjectPosition("//*[@name='InfoPost'][1]").ToVector3().x;
Console.WriteLine("InfoPost x:" + infoPost);



Now that we have it, we will loop over some actions so long as we're in the Zone1 scene, and until Ellen has reached the post we need to proceed.
while (Api.GetActiveSceneName() == "Zone1")



Zone 1 
Last thing's first. If we're near the target post, as in +/- 1 on the x-coordinate, jump-down to exit the zone. We do this by testing whether Ellen's position is greater than AND less than 1, meaning we're within that range. Next, we log something to indicate the InfoPost was reached, for debugging purposes, then press the down and jump keys at almost the same time. We do this by pressing 'S' for one second using (ulong)Api.GetLastFPS() and using that as an argument for the Api.PressKey() method. Then we wait half a second using Api.Wait(500), before "jumping" for half a second using Api.PressKey(KeyCode.Space, (ulong)Api.GetLastFPS() / 2) which divides the current FPS by 2 as an argument. Which is a long explanation for a very short IF statement:


if ((int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x >= (int)infoPost - 1 && (int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x <= (int)infoPost + 1)
        {
            Console.WriteLine("InfoPost found!");
            Api.PressKey(KeyCode.S, (ulong)Api.GetLastFPS());
            Api.Wait(500);
            Api.PressKey(KeyCode.Space, (ulong)Api.GetLastFPS() / 2);
            Api.CaptureScreen("Zone1");
        }


But since we haven't reached the InfoPost yet, we will need to find a way to it. In this next section, we're going to move to the right while jumping, to try and avoid a rather large hole between the spawn point and the target InfoPost. This time, we test whether Ellen's x-coordinate is less than 10, and y-coordinate is less than -1 which is roughly where the left-side of the gap starts. So as long as we're to the left of that gap, keep trying.


while ((int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x < 10 && (int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().y < -1)
        {
            //Jump the chasm. This is tricky, so it might take a few tries
            Console.WriteLine("We're going to jump!");
            Api.PressKey(KeyCode.D, (ulong)Api.GetLastFPS() * 2); //Move right for ~2 seconds
            Api.Wait(500);
            Api.PressKey(KeyCode.Space, (ulong)Api.GetLastFPS() / 2); //Jump for a 1/2 second
            Api.Wait(500);
        }




If there's a sudden drop in FPS, there's a chance Ellen will fall into the hole. So after each iteration of the previous while loop we will test whether Ellen fell in the hole, and if so to jump-left out of it. This time we check that the x-coordinate is greater than 10 (where the gap begins), and the y-coordinate is less than -3. Note that these jumps are 1/2 - 1/4 of a second, as we don't want to jump too high.


while ((int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x >= 10 && (int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().y <= -3)
        {
            Console.WriteLine("We're stuck in the hole, please stand by!");
            Api.PressKey(KeyCode.A, (ulong)Api.GetLastFPS()); //Move left for 1 second
            Api.Wait(500);
            Api.PressKey(KeyCode.Space, (ulong)Api.GetLastFPS() / 4); //Jump for a 1/4 second
            Api.Wait(250);
        }


Once we have successfully crossed the chasm, we simply move right until we reach the desired position. Since these movements rely on the FPS for input, there's a chance we will go too far. So after each iteration, we will check again and move back to the left if we've gone too far. The movements here are shorter, only 1/3 of a second with 333ms (1/3 of a second) wait in between. You might wonder why we do this after each movement, and that's because inputs are asynchronous. Not all tests will require a wait after each movement, but in this game, it prevents inputs from overlapping and being missed.


while ((int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x > 13 && (int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x < infoPost && (int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().y > -2)
        {
            Console.WriteLine("x:" + Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x + " < " + infoPost + ". Moving Right.");
            Api.PressKey(KeyCode.D, (ulong)Api.GetLastFPS() / 3);
            Api.Wait(333);
        }


This is where we move left if we've gone too far right.


while (Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x > infoPost)
        {
            Console.WriteLine("x:" + Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x + " > " + infoPost + ". Moving Left.");
            Api.PressKey(KeyCode.A, (ulong)Api.GetLastFPS() / 3);
            Api.Wait(333);
        }


Note: Several of the sections above include a Console.WriteLine method which includes Ellen's current position relative to the target. This can be useful for troubleshooting if something doesn't work.

Finally, we test whether we have successfully left Zone1. This is done using an Assert.AreEqual method from the UnitTesting framework. So be sure to add using NUnit.Framework; to the beginning of your test. If this assertion fails, it will also fail your test.


var levelCheck = Api.GetActiveSceneName();
Assert.AreEqual(levelCheck, "Zone2");


Alternatively, we can use the GameDriver Checkpoint method to output the details of the test, and the status of the checkpoint, along with a screenshot to prove the output.


var levelCheck = Api.GetActiveSceneName();
if (levelCheck == "Zone2")
{
    Api.Checkpoint(Api.CheckpointStatus.PASS, "Zone 1 Complete!", true);
}
else Api.Checkpoint(Api.CheckpointStatus.FAIL, "Zone 1 Failed!", true);



Zone 2 
Once we're in Zone2, we need to touch the key in order to obtain it. This is done first by locating the key, then simply moving towards it using the same approach as we did in Zone1. Only the key is on a pedestal and there is no chasm to jump over. Oh, and the only way to know whether we have the key is to check the color value of the key in UI. Initially, these are grayed out but become illuminated once we have found the item.



First, we wait for the Key object to load.


Api.WaitForObject("/*[@name='Key']");


The main loop for this section tests whether the color of the key on the screen has changed. For this, we will use the Api.GetObjectFieldValue method, which can search for any active object in the game and return a field value. In this example, the path to the Object is /KeyCanvas/KeyIcon(Clone)/Key which has an Image component. There are also 3 keys in the UI, so we use //*[@name='KeyIcon(Clone)'][0]/*[@name='Key'] to indicate the 1st instance (or [0]) of the object. When the key is obtained, the color value of that key will change from 1, 1, 1, 0 to 1, 1, 1, 1. The last number refers to the Alpha blending or transparency.


while (Api.GetObjectFieldValue("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)']/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").Equals("RGBA(1.000, 1.000, 1.000, 0.000)")


While the above test is true, we will loop over a set of movements in the same way we did in Zone1, this time checking for the x and y coordinate values relative to Ellen's position. The first action following that test will exit the loop if Ellen gets the key, so as not to waste any more cycles.


while (((int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x != (int)keyVector.x) & ((int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().y != (int)keyVector.y))
        {
            //If we've hit the key, break
            if (Api.GetObjectFieldValue("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)']/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").Equals("RGBA(1.000, 1.000, 1.000, 1.000)"))
            {
                Console.WriteLine("Key Get!");
                break;
            }


Next, if we are to the left of the key, move right. Within that loop, if we are lower than the key, jump.

 else
    {
        //If we're to the left, move right
        while ((int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x < (int)keyVector.x)
        {
            Api.PressKey(KeyCode.D, (ulong)Api.GetLastFPS());
            Api.Wait(500);

            // While moving right, if we're lower than the key, jump
            if (Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().y < keyVector.y)
            {
                Api.PressKey(KeyCode.Space, (ulong)Api.GetLastFPS() / 3);
            }
        }


 If we're to the right of the key, move left. Within that loop, if we are lower than the key, jump.


while ((int)Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().x > (int)keyVector.x)
        {
            Api.PressKey(KeyCode.A, (ulong)Api.GetLastFPS());
            Api.Wait(500);

            // While moving left, if we're lower than the key, jump
            if (Api.GetObjectPosition("//*[@name='Ellen']").ToVector3().y < keyVector.y)
            {
                Api.PressKey(KeyCode.Space, (ulong)Api.GetLastFPS() / 3);
            }
        }


After all of the above, move to the right until we leave the zone. This assumes we have the key and are to the right of it.


while (Api.GetActiveSceneName() != "Zone3")
{
    Api.PressKey(KeyCode.D, (ulong)Api.GetLastFPS());
    Api.Wait(500);
    Api.PressKey(KeyCode.Space, (ulong)Api.GetLastFPS() / 2);
}


If the above was successful, we have the key and are now in Zone3.


var levelCheck = Api.GetActiveSceneName(); Assert.AreEqual(levelCheck, "Zone3");


And that's it! It honestly took 10 times as long to write this tutorial than it did to write the test above, which is attached below for your use. You will need to copy this code into a new NUnit project, as outlined in the installation instructions linked above.


If your test doesn't behave quite the way you expect, try different movements and wait durations between them. It can have a significant impact on the test.

Some games such as puzzles will be much more deterministic in their inputs, meaning there is less variation in the replay and much simpler steps to produce the desired results. Other games, such as FPS or other 3D environments require a little more effort, but we can use a similar approach to what is shown here.

Note: The object identification method used throughout this guide is a core capability of GameDriver, named HierarchyPath. It's a powerful way to identify objects and values similar to XPath. More information on how to use HierarchyPath is available in your GameDriver Documentation directory, within your Unity project.