GameDriver

Widget Blueprints allow game developers to use Buttons, sliders, textboxes, lists, and a whole lot more. In this example, we’ve created a widget named “simpleButton” (we should have called it simpleHUD) which contains a full HUD with a variety of UMG widgets displayed to the user when the level starts. 

Within that widget, you can see that we’ve laid out a range of buttons, checkboxes, sliders, images, and text widgets, each with descriptive names like  Button_test, CheckBox_test, Slider_test, editText, and so on.

We can now actuate and press buttons, drag sliders, and do other things with the Gamedriver API. From within a project that contains the gdio.unreal_api you can now reference the UMG widgets using HPath, almost the same way as you do with in-game objects. 

Widget Reflector

Widget Reflector can be accessed by navigating to Tools -> Debug -> Widget Reflector.

The widget reflector can be used to get the class name of a widget blueprint to be used in hpath.  Here simpleButton_C_0 is the class name.

To reference the UMG elements directly, use their name. /Button_test, in our example, is a reference to our UButton. If you have multiple instances of a widget, you could use the fully qualified name that references the object, then the element, as follows:

/simpleButton_C_0/Button_test

Now, button clicking is as simple as:

api.MouseMoveToObject("/simpleButton_C_0/Button_test", 20); // move mouse over button
api.Wait(500); //wait ½ second
api.ClickObject(MouseButtons.LEFT, "/simpleButton_C_0/Button_test", 20); //click

You can change editable text fields as follows:

api.SetInputFieldText("/simpleButton_C_0/editText", "abc");

Drag operations are useful for widgets like sliders. To do a drag on a slider we first determine the drag’s starting screen position (GetObjectPosition of the slider works). We then add an offset to create a drag using api.MouseDrag. 

Vector3 sliderPosition = api.GetObjectPosition("/simpleButton_C_0/Slider_test");
api.MouseDrag(MouseButtons.LEFT,
        new Vector2(sliderPosition.x + 150,
        sliderPosition.y + 150),
        500,
        new Vector2(sliderPosition.x, sliderPosition.y));

Since every reflected method is available on the UMG widgets, there is a lot of Unreal functionality exposed to you because api.CallMethod allows you to call any reflected method of an object!  For instance, to get a button’s visibility you simply use api.CallMethod, with the HierarchyPath, “GetVisibility”, null for parameters, and get an int32 returned (you can look up the Unreal enums to see what the values represent).

int visibility = api.CallMethod<int>("/simpleButton_C_0/Button_test", "GetVisibility", null);

Similarly, you can access the value of a checkbox or slider as follows:

bool isChecked = api.CallMethod<bool>("/simpleButton_C_0/CheckBox_test", "IsChecked", null);
float sliderValue = api.CallMethod<float>("/simpleButton_C_0/Slider_test", "GetValue", null);