Integrating Devices

Introduction

You can control HPTK hands with almost any device (or even animations) as long as you can script an input data provider (IDP) that translates the output of that device into something that HPTK can understand.

If you want to share your integration, it is highly recommended to package your extension and link it to HPTK's integration manager. Also it's important to include a prefab called DefaultSetup.CustomDevice so other devs can make it to work just doing drag & drop in the scene.

AbstractTsf

Input data providers work with abstract transformations (AbstractTsf).

AbstractTsf is a simple class that stores:

• Position: Generic Vector3.

• Rotation: Generic Quaternion.

• Space: Space in which position and rotation should be applied.

• LocalScale: Generic Vector3.

Input Data Provider

InputDataProvider is just a class with an UpdateData() function, called by an InputController, that is reponsible for translating input data into a list of FingerPoses.

Bone data is usually given at bone level so it's required to:

1. Update InputDataProvider.bones.

2. Call InputDataProvider.UpdateFingerPosesFromBones().

InputDataProvider is also responsible for reading or calculating tracking confidence and real user's hand size.

Bones

InputDataProvider.bones is an array of 19 items of type AbstractTsf. Each of these items represent the transformation that InputController have to apply to each master bone of its hand.

InputDataProvider.bones contains:

0 - wrist
1 - forearm
 
2 - thumb0
3 - thumb1
4 - thumb2
5 - thumb3
 
6 - index1
7 - index2
8 - index3
 
9 - middle1
10 - middle2
11 - middle3

12 - ring1
13 - ring2
14 - ring3
 
15 - pinky0
16 - pinky1
17 - pinky2
18 - pinky3

Confidence

InputDataProvider must read the platform-specific tracking confidence estimation and translate it to a value between 0 and 1.

Some devices don't provide tracking confidence but they can inform when the hand or controller is tracked or not. If this is the case, InputDataProvider must set confidence to 1 when they are being tracked and back to 0 when they are not.

Hand size

Not all devices provide a value for estimated real hand size but some of them provide bone positions. When InputDataProvider.scale is equal to 1, DefaultAvatar hands will use their scale by default (1,1,1). On its default scale, the distance between index1 and index2 in DefaultAvatar hands is about 0.1147 meters. By knowing this, we can extrapolate hand scale by comparing this value with the distance between index1 and index2 given by the device.

If the device provides scale estimation but at a different scale, InputDataProvider should rescale this estimation.

Skeleton

The following code is as a skeleton for a new InputDataProvider:

public class CustomIDP : InputDataProvider
{
    public ThirdPartyData handData;

    public override void InitData()
    {
        base.InitData();
        
        if (!handData)
        {
            // Find handData automatically
        }
    }
    
    public override void UpdateData()
    {
        base.UpdateData();
        
        if (handData.IsTracked)
        {
            for (int b = 0; b < bones.Length; b++)
            {
                bones[b].space = //...
                bones[b].position = //...
                bones[b].rotation = //...
            }

            UpdateFingerPosesFromBones();

            confidence = //...

            scale = //...
        }
        else
        {
            confidence = 0.0f;
        }
    }
}

Integration Manager

An IntegrationExtension checks, in its OnValidate() function, if it's included in the list of available extensions. This list is contained in a unique IntegrationConfiguration inside the HPTK package.

To show your integration package as available in the window HPTK/Integration Manager follow these steps:

1. In project window, right click and navigate to Create/HPTK/IntegrationConfigurationExtension.

2. Add a new item in the list of extensions to add to the Integration Manager. Add more items if needed.

3. Add the reference to the IntegrationConfiguration asset, which is located at path Packages/HPTK/Samples/Integrations/Integrations.

DefaultSetup Prefab

A DefaultSetup prefab must include the following tree of objects:

• Platform specific objects (XRRig, CameraRig, etc.)

• HPTK (singleton)

  • Left IDP

  • Right IDP

DefaultSetup prefab is properly configured if it's possible to:

1. Create a new empty scene.

2. Drag & drop DefaultSetup.CustomDevice in the scene.

3. Drag & drop DefaultAvatar in the scene.

4. Build the game for that scene.

5. Control DefaultAvatar with the new device successfully.