In our 2D Game Kit Automation Tutorial, which is our introduction to every GameDriver user, we showcase how to use GameDriver to play through a game, and demonstrate some of our capabilities along the way. This is a great intro to the tool but doesn't make a very effective functional test. This is mainly due to the fragile nature of the test design, which requires the player to successfully traverse each level.
This isn't an entirely invalid approach, as it is often desirable and necessary to test gameplay in this way. However, it's important to keep in mind the goal of test automation is to validate that certain events and outcomes take place as a result of our actions and inputs. To take advantage of this approach, it would be necessary to add Assertions to test those outcomes at each phase of the test, and proper error handling to recover from failures in gameplay. To test this approach, our friends at PQA developed an extended version of our tutorial test that traverses multiple levels in the 2D Game Kit demo, including tackling obstacles such as moving platforms and jumping over acid pits in zone 3. That test is available at the bottom of this guide as an example and can be seen in the video below.
However, if we want to test the functionality of this game without the fragility of playing through each level, we need to take a different approach. Instead of moving around as a player would, we can set the transform of the player directly to trigger the events that we would by otherwise moving there manually. We can then test dialogs and events by checking object properties after each step in the test and eliminate the time spent manually navigating the scene. Finally, we can leave the editor or standalone player open at the end of these tests to allow the tester to continue playing from a certain checkpoint, such as fighting the boss or testing a new feature or level.
Additionally, developers often put "shortcuts" in their games to allow testers to load up certain events, bypassing the prerequisites. While this may be an effective approach, there is a risk that those prerequisite actions are not working as intended, leaving the overall experience invalidated. Using our approach, testers can automate the repetitive or mundane tasks leading up to the event they wish to test. This can save a ton of time for the tester who would otherwise need to play to that point manually while ensuring that the functionality is stable. Let's take a closer look at this approach.
Zone 1
First, we start the game as we would in any test, using the typical Connect or Launch methods used for testing in the Unity editor or standalone player, followed by a WaitForObject command to make sure the scene loads before we proceed:
[OneTimeSetUp]
public void Connect()
{
try
{
api = new ApiClient();
if (pathToExe != null)
{
ApiClient.Launch(pathToExe);
api.Connect("localhost", 19734, false, 30);
}
else if (testMode == "IDE")
{
api.Connect("localhost", 19734, true, 30);
}
else api.Connect("localhost", 19734, false, 30);
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
api.EnableHooks(HookingObject.MOUSE);
api.EnableHooks(HookingObject.KEYBOARD);
//Start the Game
api.WaitForObject("//*[@name='StartButton']");
api.ClickObject(MouseButtons.LEFT, "//*[@name='StartButton']", 30);
api.Wait(3000);
}For some explanation of the different connection methods used above, see this article.
Once we have loaded the first scene, Zone1, the objective will be to get to the entrance for Zone2. In the original tutorial, we did this by finding the location of the second InfoPost object and moving towards it with a series of while loops. For example:
while (api.GetSceneName() == "Zone1")
{
if ((int)api.GetObjectPosition("//*[@name='Ellen']").x >= (int)infoPost - 1 && (int)api.GetObjectPosition("//*[@name='Ellen']").x <= (int)infoPost + 1)
{
Console.WriteLine("InfoPost found!");
api.KeyPress(new KeyCode[] { KeyCode.S } , (ulong)api.GetLastFPS());
api.Wait(500);
api.KeyPress(new KeyCode[] { KeyCode.Space } , (ulong)api.GetLastFPS() / 2);
api.CaptureScreenshot("Zone1");
}
while ((int)api.GetObjectPosition("//*[@name='Ellen']").x < 10 && (int)api.GetObjectPosition("//*[@name='Ellen']").y < -1)
{
//Jump the chasm. This is tricky, so it might take a few tries
Console.WriteLine("We're going to jump!");
api.KeyPress(new KeyCode[] { KeyCode.D } , (ulong)api.GetLastFPS() * 2); //Move right for ~2 seconds
api.Wait(500);
api.KeyPress(new KeyCode[] { KeyCode.Space } , (ulong)api.GetLastFPS() / 2); //Jump for a 1/2 second
api.Wait(500);
}
// etc...This works, but depending on the machine that you are playing on the process can be quick or it can take several attempts while the player falls into the hole in the level, and the test attempts to correct itself.
The second problem with this approach is that we're not really testing anything. Sure, we find out whether the player can move around and eventually find the exit, but we're not doing so very efficiently.
The correct approach here would be to move the player to the position of the exit, perform the action to do so, then test that it works. Let's start by moving Ellen to the 2nd InfoPost:
// Move Ellen to the second InfoPost ~~ //*[@name='InfoPost']
Vector3 infoPost1 = api.GetObjectPosition("//*[@name='InfoPost'][1]");
api.Wait(1000);
api.SetObjectFieldValue("//*[@name='Ellen']/fn:component('UnityEngine.Transform')", "position", infoPost1);
api.Wait(1000);Changes to the transform of an object are immediate, so we add `Wait` commands here to give the game a moment to catch up and allow events to register correctly.
We could just exit the scene at this stage by sending the "S + space" commands, but it might make sense to test that the InfoPosts are popping up the dialog boxes they are supposed to while we are here. So for each, we will add the following assertion:
Assert.IsTrue(api.WaitForObjectValue("/*[@name='DialogueCanvas']", "active", true), "InfoPost pop-up failed!");Now we can send the commands to exit the zone and check that we're in Zone2.
// Allow the scene time to load
api.WaitForObject("/Objective[@name='Key' and ./fn:component('UnityEngine.Behaviour')/@isActiveAndEnabled = 'True']");
// Check that we teleported to Zone2
Assert.AreEqual("Zone2", api.GetSceneName(), "Wrong zone!");
Note: As with the previous tutorial, we have tagged the Ellen object prefab with the "Player" tag (located in Zone 1), and the Key prefab with the "Objective" tag in order to demonstrate the usage of this feature. You will need to tag this object yourself to reuse the code above or remove the references to "Player" and "Objective" in the code, and replace these with an asterisk (I.e. "//*[@name=...). For more information on how to add tags to an object, see the Unity manual --> here.
Zone 2
Once we're in Zone2, the object is to get the first key. In the original tutorial, we did this again by finding the location of the key and looping through inputs and checks to move the player to it. For example:
[Test]
public void Test2()
{
api.WaitForObject("/*[@name='Key']");
api.Wait(2000);
var keyVector = api.GetObjectPosition("/*[@name='Key']");
var ellen = api.GetObjectPosition("//*[@name='Ellen']");
Console.WriteLine("Ellen's current coordinates: " + ellen.x + ", " + ellen.y);
Console.WriteLine("Key coordinates: " + keyVector.x + ", " + keyVector.y);
Console.WriteLine(api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color"));
while (api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").ToString().Equals("RGBA(1.000, 1.000, 1.000, 0.000)"))
{
//Check if we're on either side of the key, shaving float precision
while (((int)api.GetObjectPosition("//*[@name='Ellen']").x != (int)keyVector.x) & ((int)api.GetObjectPosition("//*[@name='Ellen']").y != (int)keyVector.y))
{
//If we've hit the key, break
if (api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").ToString().Equals("RGBA(1.000, 1.000, 1.000, 1.000)"))
{
Console.WriteLine("Key Get!");
break;
}
else
{
//If we're to the left, move right
while ((int)api.GetObjectPosition("//*[@name='Ellen']").x < (int)keyVector.x && api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").ToString().Equals("RGBA(1.000, 1.000, 1.000, 0.000)"))
{
api.KeyPress(new KeyCode[] { KeyCode.D } , (ulong)api.GetLastFPS());
api.Wait(500);
// While moving right, if we're lower than the key, jump
if (api.GetObjectPosition("//*[@name='Ellen']").y < keyVector.y)
{
api.KeyPress(new KeyCode[] { KeyCode.Space } , (ulong)api.GetLastFPS() / 3);
}
}
//If we're to the right, move left
while ((int)api.GetObjectPosition("//*[@name='Ellen']").x > (int)keyVector.x && api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").ToString().Equals("RGBA(1.000, 1.000, 1.000, 0.000)"))
{
api.KeyPress(new KeyCode[] { KeyCode.A } , (ulong)api.GetLastFPS());
api.Wait(500);
// While moving left, if we're lower than the key, jump
if (api.GetObjectPosition("//*[@name='Ellen']").y < keyVector.y)
{
api.KeyPress(new KeyCode[] { KeyCode.Space } , (ulong)api.GetLastFPS() / 3);
}
}
}
}
}Once again, this works fairly consistently. But depending on the performance of the machine you are using, the test can be quick or it can fail completely by missing the key repeatedly and moving too far to the right. It's also 50+ lines long, and difficult to maintain.
The simpler approach to this would be to move Ellen directly to the key, and verify we have it by checking the color of the key on-screen has changed. There are likely other ways to verify this, such as checking the character inventory value is correct, but we're using visual verification which is the same as what the player would experience during gameplay.
// Move Ellen to the key, and check that it lands in our inventory
Vector3 key1 = api.GetObjectPosition("//Objective[@name='Key']");
api.SetObjectFieldValue("//*[@name='Ellen']/fn:component('UnityEngine.Transform')", "position", key1);
api.Wait(1000);
Assert.IsTrue(api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").ToString().Equals("RGBA(1.000, 1.000, 1.000, 1.000)"));Once we add the step to move Ellen to the next zone and take a screenshot, we're done with the scene in around 20 lines of code.
The Rest of the Owl (?)
By now you're getting the point about how best to tackle testing a game like this. Instead of playing through the game exactly how a user would, which is likely to be brittle, slow to execute, and difficult to build and maintain, we take a much simpler approach that still touches on the important parts of gameplay. The next few sections will "teleport" the player character to each of the remaining 2 keys, pick up the melee weapon required to break down a door, break that door, and ultimately put the player in front of the final boss in the last zone.
If we were playing this game manually, the run would take 5-10 minutes depending on how capable the player is. If we use the automated proof-of-concept put together by PQA, it can take anywhere from 15 to 30 minutes just to get the keys. But when we test in the way described in this article, we get a complete run-through up to the final boss in around 45 seconds. This touches on all of the core functionality of the game and allows for additional tests to be added with minimal impact on the overall execution. For example, we could easily add tests to check that enemies can be destroyed by moving the player character near one and using a weapon, or that we can take damage by moving Ellen into an enemy and checking the remaining health values - either on-screen or using the health property of the player.
Additionally, this test allows us to manually test that the final boss fight is working as intended without wasting 5-10 minutes each playthrough, simply by disconnecting the GameDriver test at the end while leaving Play mode active, or without terminating the standalone Player. While it would be possible to automate that as well, this is the kind of test that makes sense to test manually as it is a more subjective measure of the game. Is the boss too hard to defeat? Are the mechanics too unforgiving for the target audience of the game? These are the types of qualitative tests better suited to a human tester that we can streamline using automation to reduce the amount of time spent getting to the objective.
Below you will find a portion of the "playthrough" version of the test developed by our friends at PQA, as well as the "Correct" version outlined above. If you have any comments or questions on either approach or would like to see more examples like this, let us know via email at [email protected] or by joining our Slack Community.
Happy Testing!