Lyra is the 3rd person shooter demo game from Unreal. It offers a complex and realistic game for testing and as such it’s a great example to show how Gamedriver works to test your Unreal games.

https://www.unrealengine.com/marketplace/en-US/learn/lyra

In this tutorial, we will show how to simulate clicks on user interface items, how to read our score and health, fire the weapon, change your weapon, move around the level, and check for enemies using Raycasting. 

Before you get started you should follow our basic tutorial for getting gamedriver installed and set up. Once you have the plugin installed and the API downloaded and set up you can follow along with us.

HierarchyPath

We write our tests using C# and use NUnit through Visual Studio although you can use the editor and testing framework of your choice. 

HierarchyPath (HPath) is the query language for selecting objects in a game that is both simple and powerful. You can select objects using names, tags, and classes, as well as relative referencing and tools like regular expressions, all of which let you access dynamic objects created at runtime. Best of all, the exact same framework works with Unity and Godot, creating a universal way of writing reusable tests.

While advanced users will routinely edit and create HPath selectors from scratch, the editor tools provide an easy way to get relative, absolute, and wildcard HPaths that work right away, and we’ll illustrate working from those generated HPaths in our examples. See this guide for more information about HierarchyPath.

Simulate Mouse Click / KeyPress (Shoot the Gun)

Lyra input mappings allow your player to shoot a gun by pressing the mouse button. Therefore making the gun fire is as simple as simulating a click with the Api somewhere on screen that isn’t a button. 

api.Click(MouseButtons.LEFT, new Vector2(100, 100), 20);

Alternatively, there’s a gamepad mapping that also causes the actor to shoot, and gamepad input does not require screen focus so we could use the KeyPress API call with the Gamepad_RightTrigger:

api.KeyPress(new KeyCode[] { KeyCode.Gamepad_RightTrigger }, 10);

There are many potential aspects of gun firing one might want to test. Did the bullet go where it was aimed? Was damage taken? Were shells ejected? Did the ammo count go down?

 
In this example, we will look at whether the ammo count on the screen went down. The function below gets the current ammo count, shoots the gun, and then confirms the count is one smaller. It also makes sure the error message “No Ammo” appears when the user tries to shoot without any ammo. 

public void FireGun(){
   int originalAmmoCount = GetCurrentAmmoCount();

   api.KeyPress(new KeyCode[] { KeyCode.Gamepad_RightTrigger }, 10);//fire

   api.Wait(10);

   if ( oldAmmoCount == 0)    {
      Assert.IsTrue(IsNoAmmoVisible());
   }
   else{
      int newAmmoCount = GetCurrentAmmoCount();
      Assert.AreEqual(oldAmmoCount-1, newAmmoCount);//make sure 1 smaller
      Assert.IsFalse(IsNoAmmoVisible());
  }
}

To write GetCurrentAmmoCount() and IsNoAmmoVisible() we need to extract values from the UI, which we will do in the next section.

Checking UI values

Lyra has a rich User Interface much of which is generated at runtime. To check the amount of ammunition displayed we need to inspect these UI items. GameDriver’s new Slate Inspector provides an easy way to get HPaths out of the Interface while you’re using the Unreal Editor. 

Inspect a String in the User Interface

Using the Slate Explorer that comes with GameDriver we can explore the hierarchy of runtime Slate Widgets while the game is running. Here we can search for the names of widgets and click on them to explore their properties.

In the image above we’ve identified the TotalCountWidget nested in the hierarchy (which is an STextBlock), and note it has a field named Text which contains a string with the ammo count (it says “48”).

By right-clicking on that field we can generate a HPath selector which we can use in our code. In this case, a Relative selector will work since the widget is named uniquely.

//TotalCountWidget/@Text

In our test we can now make a call to get that field value as follows:

string numAmmo = api.GetObjectFieldValue<string>("//TotalCountWidget/@Text");

In this game, there are two displayed values. The total count of ammo and the ammo in the magazine. Thankfully, that second field is equally easy to find and get a reference to. By parsing those returned strings into integers and adding them together we get a numeric value for the total ammo like the completed GetCurrentAmmoCount() method below.

public int GetCurrentAmmoCount()
{
string numAmmo = api.GetObjectFieldValue<string>("//TotalCountWidget/@Text");

int ammoCount = Int32.Parse(numAmmo);

numAmmo =  api.GetObjectFieldValue<string>("//AmmoLeftInMagazineWidget/@Text");

return ammoCount + Int32.Parse(numAmmo);
}

Inspect the visibility of a Slate element. 

The second part of our original FireGun test was to ensure a small error box appears when it is supposed to. This method is essentially one GameDriver API call, where we call the IsVisible method on a Slate Object to determine its current visibility.

public bool IsNoAmmoVisible(){
  return api.CallMethod<bool>("//W_AbilityFailureFeedback", "IsVisible");
}

IsVisible is just one method that belongs to the unreal Slate objects. One of the strengths of GameDriver is that you can call practically any method within Unreal directly.

In the Slate Explorer (or Unreal’s documentation) you can see all methods visible to GameDriver, which includes all reflected methods including those inherited and those marked private! 

You can right-click to generate a sample call to help you get started with calling those methods from your tests.

Moving your AActor

There are several ways to make an AActor move in Unreal. If your level has navMeshBounds defined correctly (required for NPC navigation) you can make the AActor move using the NavAgentMoveToPoint API call. In other scenarios, it makes sense to teleport your AActor to the desired location using the SetObjectPosition API call (or you can modify the location values of the actor directly). 

//Move using Navmesh by making the character walk
api.NavAgentMoveToPoint("//B_Hero_ShooterMannequin_C_0", pos,true);

//Teleport immediately using the utility function.
api.SetObjectPosition(myCurrentPlayer, pos);

To get all the objects of a class we call GetObjectList which returns a List of LiteGameObjects. These read-only client-side objects reflect some key information from the UObjects in the game such as name, position, and a HierarchyPath.

List<LiteGameObject> allAmmoPads  = api.GetObjectList("//*[contains(@class,'WeaponSpawner')]", true);

The method below puts all these parts together, gets all the WeaponSpawners in the world, and makes the Player navigate to the first one that has a weapon available (This should pick up a weapon or replenish the ammo for an existing weapon).

public bool  MoveToAmmo()
 {
     List<LiteGameObject> allAmmoPads  = api.GetObjectList("//*[contains(@class,'WeaponSpawner')]", true);

     foreach (LiteGameObject ammoSpot in allAmmoPads)
     {
         bool IsAvailable = api.GetObjectFieldValue<bool>(ammoSpot.hierarchyPath, "bIsWeaponAvailable",5);

         if (IsAvailable)
         {
             Vector3 pos = api.GetObjectPosition(ammoSpot.hierarchyPath);
             api.NavAgentMoveToPoint(myCurrentPlayer, pos,true);
             Console.WriteLine("Moving to "+pos.ToString());
             return true;
         }
     }
     return false;
 }

Changing Weapon 

In Lyra, the player can switch their weapon by scrolling on the mouse or by clicking the image in the GUI corresponding to the weapon they want.

Simulate a Scroll

To simulate a Middle wheel scroll on a mouse we use the Scroll API command.

api.Scroll(0, 1, 1);

Alternatively, we could use a simulated keypress from a gamepad, which is also mapped to change the weapon.

api.KeyPress(new KeyCode[] { KeyCode.Gamepad_DPad_Left }, 1);

To make a test out of the switching of weapons we should check to see if the weapon the Player is holding after the scroll or keypress is different from the one they had before. To accomplish this we will use a regular expression HPath selector (it’s a simple one).

Use a regular expression to select 

The player character holds either a Shotgun class weapon, a Pistol, or a Rifle. The images below show how the object changes in the Outliner as the Player switches weapons. The instanced name of the weapon is determined at run time.

Since these objects are descendants of the Player Character, we can create an HPath that gets the Player AActor and then gets all its children as follows:

"//B_Hero_ShooterMannequin_C_0/*"

If you enter this into the HPathDebugger tool in the editor, or call GetObjectList from the API, it will return all the Player’s children, in this case, B_Manny0 and B_Pistol2 (or B_Shotgun2). 

Since we know we want either an object that contains the name Shotgun, Pistol, or Rifle we can append [match(@class,'Rifle|Pistol|Shotgun')] to our HPath. This will then only return objects from the children that match the Regular Expression 'Rifle|Pistol|Shotgun'. Put it together to get the following HPath to identify the weapon currently being held.

/*[@name ='B_Hero_ShooterMannequin_C_0']/*[match(@class,'Rifle|Pistol|Shotgun')]

Assuming we have at least 2 weapons the following code will switch between them and confirm that the weapon has changed. 

public void SwitchGunToNext()
{

LiteGameObject currentWeapon = api.GetGameObject("//B_Hero_ShooterMannequin_C_0/*[match(@class,'Rifle|Pistol|Shotgun')]");

api.Scroll(0, 1, 1);

api.Wait(100);

LiteGameObject currentWeaponAfter = api.GetGameObject("//B_Hero_ShooterMannequin_C_0/*[match(@class,'Rifle|Pistol|Shotgun')]");

if (currentWeapon.hierarchyPath == currentWeaponAfter.hierarchyPath)
   {
       Console.WriteLine("Gun not switched");
    }
    else
   {
       Console.WriteLine("Switched gun to " + currentWeapon2.name);
   }
}

Choosing an Enemy - Dynamic testing

Now that we can fire a gun, move to pick up ammo, and change weapons it’s useful to consider how we could test a walkthrough scenario or level playthrough where dynamic enemies and goals make it hard to predetermine user input to achieve success. 

One tool GameDriver has added to the API is the Raycast API call which allows you to leverage Unreal’s collision system to identify objects. In this case, we want to Raycast from the camera behind the Player (which we can grab in the Object Explorer) towards the position of an enemy.

We can check if an Enemy intersects with a Ray shot out from the camera (the camera centers on the crosshairs where the bullet would fire). This tells us that we should fire on the enemy.

 Vector3 EnemyPosition = api.GetObjectPosition(THE_ENEMYS_HPATH);

 RaycastResult[] results = api.Raycast(pos, “//B_Hero_ShooterMannequin_C_0/fn:component('CameraComponent')");

 foreach (RaycastResult r in results)
 {
     //AllEnemies is a list of hpath strings for enemy players 
     if (AllEnemies.Contains(r.name))
     {
         Console.WriteLine("Found him" + THE_ENEMYS_HPATH);
         break;
     }
 }

Tracking dynamic enemies on a real level is more involved than a static list, so we inspect a Struct in each Player object to determine which ones are on our team.

Inspecting a value in a struct

To determine which team a player is on we can Inspect the MyTeamID struct inside of each player object. Again, we can find this object in the Object Explorer and right-click to generate an HPath. 

/*[@name = 'B_Hero_ShooterMannequin_C_0']/@MyTeamID

To examine fields within a struct we need to look at the definition for FGenericTeamId, which is actually an Unreal struct (Source/Runtime/module/Classes/GenericTeamAgentInterface.h). That struct contains a uint8 by the name of TeamId. That means to get the value of the team Id we need to append /@TeamId to our Hpath for the struct.

/*[@name = 'B_Hero_ShooterMannequin_C_0']/@MyTeamID/@TeamId 

When put into the API we call for FieldValue as follows:

int myTeamID = api.GetObjectFieldValue<int>(myCurrentPlayer + "/@MyTeamID/@TeamID"); 

Pointing your weapon at an enemy

To point your gun we will move the mouse to simulate how a user would aim and shoot. The code below will hone in on the users’ position.

Essentially we move the mouse to the crosshairs (which after processing the input will have the mouse at the center of the screen), and then we determine the delta between our position and the position of the enemy. We also use the frame rate in the calculation because unreal can have dynamic frames per second and the resulting movement depends on that value. We send the input, wait briefly, and then send a second mouse input of 0,0 because input events continue to fire until the value changes. Once set to 0,0 it will stop. 

// The cube in the basic level. could be another Aactor
enemyString = "//StaticMeshActor_1";

Vector3 crossHairs = api.GetObjectPosition("//CrossHairs",CoordinateConversion.WorldToScreenPoint);

api.MouseMoveToPoint(new Vector2(crossHairs.x, crossHairs.y), 1);

try{
    api.Wait(10);

    Vector3 enemypos = api.GetObjectPosition(enemyString, CoordinateConversion.WorldToScreenPoint);

    crossHairs = api.GetObjectPosition("//CrossHairs", CoordinateConversion.WorldToScreenPoint);

    float deltaX = enemypos.x - crossHairs.x;
    float deltaY = crossHairs.y - enemypos.y;
    float x = (deltaY * deltaY + deltaX * deltaX);

    float fps = (float)api.GetLastFPS();

    float sqrtfps = (float)Math.Sqrt(Convert.ToDouble(fps));

    api.Vector2InputEvent("Mouse2D", new Vector2(deltaX *0.5f / sqrtfps, deltaY *0.5f / sqrtfps));
    api.Wait(1);

    api.Vector2InputEvent("Mouse2D", new Vector2(0, 0));
    Console.WriteLine("Turning to " + enemyString);
    return true;
}
catch(Exception e) {
    Console.WriteLine("No enemy in sight" + enemyString);

    //spin around wildly looking for another enemy.
    api.Vector2InputEvent("Mouse2D", new Vector2(-100,0));
    api.Wait(10);
    api.Vector2InputEvent("Mouse2D", new Vector2(0, 0));
    return false;
}

Putting it all together

This tutorial has shown you how to move the main player, detect teams, identify weapons in the game, move to pick them up, use Raycasting to select targets, and mouse movement to point at enemies. In isolation, these tests are powerful ways of ensuring the game works as expected. In addition, these fundamental concepts also facilitate writing a full-fledged AI bot to drive your player through the game. However, such a task is beyond the scope of this article and is left as an exercise to the reader.

The source code for the tests described above can be found attached below.