Working with Legacy XR and OVR in Unity using GameDriver : GameDriver

Configuration

The 2023.04 release includes support for working with Unity’s Legacy XR implementation. To use this feature, simply enable XR Hooks through the GDIOAgent configuration:

Or using the API:

api.EnableHooks(HookingObject.XRLEGACY);

Note: In mixed input environments (i.e. LegacyXR and Input System), you may need to use the "Force Old Input System" option in the agent. To trigger this via the client API, simply use a SetObjectFieldValue command after the initial connection, such as:

api.SetObjectFieldValue("//*[@name='GDIOAgent']/fn:component('gdio.unity_agent.GDIOAgent')", "m_ForceOldInputSystem", true);

Where "GDIOAgent" is the name of the object holding the GDIOAgent component.

Input Usage

Input calls use the same paths as with the Input System, i.e. using ButtonPress and InputEvent actions, but using paths set by the agent:

GDIOHMD for the Headset
GDIOLeftHand for the Left Controller
GDIORightHand for the Right Controller

Example for setting the location of the HMD:

api.Vector3InputEvent("GDIOHMD/centerEyePosition", new Vector3(0.0f, 1.8f, 0.0f), 0);

Some additional device paths can be used for setting the position and rotation of the simulated devices:

GDIOHMD/centerEyePosition
GDIOHMD/centerEyeRotation
GDIOLeftHand/devicePosition
GDIOLeftHand/deviceRotation
GDIORightHand/devicePosition
GDIORightHand/deviceRotation

Legacy XR is also supported by the GameDriver Recorder and can be used to determine the inputs to be used for testing. If the project contains both old and new input systems, checking the “Force Old Input System“ may be required.

An example test demonstrating the use of LegacyXR inputs is attached below and includes a recording from the Escape Room tutorial project from Unity. You may need to adjust the initial positioning of the XR

Oculus VR (OVR) Support

As of the 2023.10 release, GameDriver's LegacyXR support is extended to include the Oculus VR (OVR) package. This support utilizes the same input methods and general usage as outlined above, with the addition of support for hand gestures.

Hand gesture support is provided via multiple rotations representing each finger, or around 14 rotation calls per gesture. To help with this, we provide a helper class named HandPosePrinter.cs that can print out to a file all the rotations that form the hand gesture.

To use it, run the project without the GDIOAgent enabled, and with the headset on, make the desired
gesture with your hand and press the [P] key on the keyboard. Two files named OVRPoseRightHand and OVRPoseLeftHand will be created on the desktop folder containing the test code necessary. The test code used to support these gestures will include the use of the SkeletonInputEvent method, such as:

api.SkeletonInputEvent("OculusLeftHand/skeleton", new float[]{0.0f,0.0f,0.0f,0.0f,0.0f}, 2); // Set hand skeleton to fully open hand.

You can organize these into methods in your test code to execute the specific captured pose. An example of this approach is shown below.

public void PinchPose()
{
    //Hand_Thumb0
    api.SkeletonInputEvent("GDIORightHand/skeleton", 2, api.EulerToQuat(39.00648f, -297.9758f, -323.5771f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 3, api.EulerToQuat(32.08998f, -341.2277f, -336.2333f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 4, api.EulerToQuat(351.5882f, -7.960947f, -342.8726f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 5, api.EulerToQuat(7.265752f, -353.5541f, -332.6898f), 0);

    //Hand_Index1
    api.SkeletonInputEvent("GDIORightHand/skeleton", 6, api.EulerToQuat(4.145337f, -349.4534f, -307.5352f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 7, api.EulerToQuat(357.1325f, -0.7786186f, -306.1896f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 8, api.EulerToQuat(358.6137f, -2.876725f, -335.6526f), 0);

    //Hand_Middle1
    api.SkeletonInputEvent("GDIORightHand/skeleton", 9, api.EulerToQuat(359.0811f, -1.158072f, -349.7017f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 10, api.EulerToQuat(358.6111f, -0.5548517f, -302.3149f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 11, api.EulerToQuat(352.5197f, -0.6495829f, -333.5562f), 0);

    //Hand_Ring1
    api.SkeletonInputEvent("GDIORightHand/skeleton", 12, api.EulerToQuat(354.2881f, -10.8357f, -10.93568f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 13, api.EulerToQuat(355.0336f, -1.024279f, -297.8333f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 14, api.EulerToQuat(357.1022f, -357.3359f, -327.2591f), 0);

    //Hand_Pinky0
    api.SkeletonInputEvent("GDIORightHand/skeleton", 15, api.EulerToQuat(336.6904f, -17.70523f, -354.1525f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 16, api.EulerToQuat(11.74857f, -13.97097f, -29.30103f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 17, api.EulerToQuat(353.5829f, -5.809196f, -309.7157f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 18, api.EulerToQuat(358.9135f, -354.639f, -333.5012f), 0);
}

public void PinchPoseLeft()
{
    //Hand_Thumb0
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 2, api.EulerToQuat(38.82063f, -297.5854f, -327.9914f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 3, api.EulerToQuat(29.59883f, -338.8843f, -335.5561f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 4, api.EulerToQuat(351.7505f, -7.906457f, -340.3772f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 5, api.EulerToQuat(6.546093f, -354.0137f, -322.5562f), 0);

    //Hand_Index1               
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 6, api.EulerToQuat(4.216051f, -351.8521f, -311.5842f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 7, api.EulerToQuat(357.14f, -0.7670153f, -299.3587f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 8, api.EulerToQuat(358.7017f, -2.832536f, -330.9977f), 0);

    //Hand_Middle1              
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 9, api.EulerToQuat(359.0885f, -2.093114f, -355.8686f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 10, api.EulerToQuat(358.6343f, -0.5280398f, -319.4061f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 11, api.EulerToQuat(353.742f, -0.207874f, -346.9665f), 0);

    //Hand_Ring1                
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 12, api.EulerToQuat(354.183f, -9.713001f, -15.66434f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 13, api.EulerToQuat(355.2855f, -0.6915488f, -316.6288f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 14, api.EulerToQuat(358.1482f, -356.8446f, -342.4605f), 0);

    //Hand_Pinky0               
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 15, api.EulerToQuat(336.6904f, -17.70522f, -354.1525f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 16, api.EulerToQuat(12.01481f, -9.105035f, -32.43835f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 17, api.EulerToQuat(354.1174f, -5.321207f, -325.8288f), 0);
    api.SkeletonInputEvent("GDIOLeftHand/skeleton", 18, api.EulerToQuat(359.5704f, -354.4142f, -348.7165f), 0);
}

public void OpenHandPose()
{
    api.SkeletonInputEvent("GDIORightHand/skeleton", 2, api.EulerToQuat(38.69f, -297.28f, -335.78f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 3, api.EulerToQuat(24.92f, -333.71f, -334.27f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 4, api.EulerToQuat(351.11f, -8.08f, -350.09f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 5, api.EulerToQuat(8.26f, -353.17f, -345.33f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 6, api.EulerToQuat(3.04f, -350.32f, -350.40f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 7, api.EulerToQuat(357.09f, -0.81f, -331.50f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 8, api.EulerToQuat(358.26f, -2.98f, -352.86f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 9, api.EulerToQuat(359.10f, -2.30f, -355.11f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 10, api.EulerToQuat(358.65f, -0.52f, -329.55f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 11, api.EulerToQuat(354.60f, -0.08f, -355.84f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 12, api.EulerToQuat(354.39f, -10.51f, -357.40f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 13, api.EulerToQuat(355.55f, -0.50f, -331.40f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 14, api.EulerToQuat(358.53f, -356.74f, -347.70f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 15, api.EulerToQuat(336.69f, -17.71f, -354.15f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 16, api.EulerToQuat(10.75f, -4.33f, -2.96f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 17, api.EulerToQuat(354.32f, -5.19f, -331.19f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 18, api.EulerToQuat(359.56f, -354.42f, -348.54f), 0);
}

public void PointPose()
{
    //Hand_Thumb0
    api.SkeletonInputEvent("GDIORightHand/skeleton", 2, api.EulerToQuat(38.89811f, -297.7541f, -325.8609f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 3, api.EulerToQuat(35.59741f, -7.550374f, -351.1602f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 4, api.EulerToQuat(352.6481f, -7.44523f, -325.6596f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 5, api.EulerToQuat(5.831383f, -354.6906f, -310.8864f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 6, api.EulerToQuat(3.839882f, -355.4543f, 0f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 7, api.EulerToQuat(357.0401f, -0.8260307f, 0f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 8, api.EulerToQuat(357.9413f, -2.972085f, 0f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 9, api.EulerToQuat(359.4679f, -4.349733f, -289.7907f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 10, api.EulerToQuat(358.5927f, -0.6143429f, -272.0609f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 11, api.EulerToQuat(350.0163f, -3.198704f, -296.062f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 12, api.EulerToQuat(354.7221f, -8.257318f, -289.0827f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 13, api.EulerToQuat(354.8939f, -1.762349f, -263.6889f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 14, api.EulerToQuat(355.621f, -358.9403f, -298.2752f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 15, api.EulerToQuat(336.6904f, -17.70524f, -354.1525f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 16, api.EulerToQuat(8.31627f, -353.5389f, -308.8625f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 17, api.EulerToQuat(353.0914f, -8.18574f, -253.8036f), 0);
    api.SkeletonInputEvent("GDIORightHand/skeleton", 18, api.EulerToQuat(357.8244f, -355.6555f, -300.4726f), 0);
}

By organizing these poses into separate methods, we can call upon them in a test case much more readily. For example, the following test will first position the simulated devices into specific locations in the scene, then call the required methods captured above.

[Test]
public void OVRHandsTest()
{
// LegacyXR Hooking must be enabled for OVR Inputs
    api.DisableHooks(HookingObject.MOUSE);
    api.DisableHooks(HookingObject.KEYBOARD);
    api.EnableHooks(HookingObject.XRLEGACY);

// In this example, we're forcing Legacy Inputs in the agent due to a mixed input method project
    api.SetObjectFieldValue("//*[@name='GDIOAgent']/fn:component('gdio.unity_agent.GDIOAgent')", "m_ForceOldInputSystem", true);

// We're now loading a device description from a JSON file. Refer to Input System usage for more details.
    //string desc = api.LoadDeviceDescription("GDIORightHand.json");
    //api.CreateInputDeviceFromDescription(desc, "GDIORightHand", new string[] { "GDIOInputProvider" });
    api.Wait(2000);

    api.ButtonPress("GDIOHMD/UserPresence", 0, 1f);
    api.ButtonPress("GDIOHMD/IsTracked", 0, 1f);
    api.IntegerInputEvent("GDIOHMD/TrackingState", 63, 0);
    api.IntegerInputEvent("GDIORightHand/TrackingState", 63, 0);
    api.IntegerInputEvent("GDIOLeftHand/TrackingState", 63, 0);

    //api.Vector3InputEvent("GDIOHMD/CenterEyePosition", new Vector3(0.0f, 0.0f, 0.0f), 0);
    api.QuaternionInputEvent("GDIOHMD/CenterEyeRotation", api.EulerToQuat(35f, 0f, 0f), 0);
    api.Vector3InputEvent("GDIOLeftHand/DevicePosition", new Vector3(-0.3f, -1.0f, 0.6f), 0);
    api.Vector3InputEvent("GDIORightHand/DevicePosition", new Vector3(0.3f, -1.0f, 0.6f), 0);
    api.Wait(2000);

    // Position hand on buttons
    PinchPose();
    PinchPoseLeft();

// These are test-specific methods used to position the simulated XR devices at the right place in the scene
    Vector3 position = api.GetObjectPosition("//*[@name='StartStopButtonHousing']");
    Vector3 up = api.GetObjectFieldValue<Vector3>("//*[@name='StartStopButtonHousing']/fn:component('UnityEngine.Transform')", "up");
    position.x += 0.02f;
    position.y += up.y * 0.22f;
    position.z += up.z * 0.22f;
    api.Vector3InputEvent("GDIORightHand/DevicePosition", position, 0);
    api.Wait(1000);
    Quaternion rotation = RotateTo("RightHandAnchor", "StartStopButtonHousing", new Vector3(0.02f, 0.0f, 0.0f));
    api.QuaternionInputEvent("GDIORightHand/DeviceRotation", rotation, 0);
    api.Wait(1500);

    // Point pose and press
    PointPose();
    position.y -= up.y * 0.04f;
    position.z -= up.z * 0.04f;
    api.Vector3InputEvent("GDIORightHand/DevicePosition", position, 0);
    api.Wait(1000);

    // Reposition
    position.y += up.y * 0.04f;
    position.z += up.z * 0.04f;
    api.Vector3InputEvent("GDIORightHand/DevicePosition", position, 0);
    api.Wait(1000);

    position.x = 0f;
    position.y += 0.1f;
    position.z += 0.04f;
    api.Vector3InputEvent("GDIORightHand/DevicePosition", position, 0);
    api.QuaternionInputEvent("GDIORightHand/DeviceRotation", api.EulerToQuat(-20.0f, 0.0f, 180.0f), 0);
    api.Wait(2000);

    position.x += 1.8f;
    position.z -= 2.0f;
    api.Wait(1000);

    OpenHandPose();
    api.Wait(1000);

    api.Vector3InputEvent("GDIORightHand/DevicePosition", new Vector3(position.x, -1.0f, 0.43f), 0);
    api.QuaternionInputEvent("GDIORightHand/DeviceRotation", api.EulerToQuat(0.0f, 0.0f, 0.0f), 0);
    api.Wait(1000);

    PinchPose();
    api.Wait(3000);

    OpenHandPose();
    api.Wait(1000);

    api.QuaternionInputEvent("GDIORightHand/DeviceRotation", api.EulerToQuat(-90.0f, 0.0f, 180.0f), 0);
    api.Wait(1000);

    PinchPose();
    api.Wait(2000);

    OpenHandPose();
    api.Wait(3000);        
}

This example code also makes use of some useful helper functions, such as RotateTo, which can be found in this article.

Working with Legacy XR and Oculus VR (OVR) in Unity Print

Configuration

Input Usage

Oculus VR (OVR) Support

Related Articles