Integrating MRTK

Integrating MRTK (Oculus Quest)

1. Obtain MRTK

MRTK +2.5 can be managed using the Unity Package Manager (UPM). Follow this documentation to install Mixed Reality features using the UPM.

If you're new to MRTK or Mixed Reality development in Unity, take a look to this guide. This guide is the recommended starting point for MRTK, specifically created to walk you through the installation, core concepts, and usage of MRTK in Unity.

For educational purposes, MRTK-Quest-Sample is a ready-to-go Unity project made by provencher that uses MRTK and Oculus Quest hand tracking.

2. Obtain HPTK

In Unity Editor, open Package Manager. Click on Add package from git URL and paste the following URL:

  https://github.com/jorgejgnz/HPTK.git

Click on Add and Unity will download the UPM package. This package has no dependencies.

3. Obtain required extra scripts (Oculus Quest)

InputDataProviders are required to translate raw hand data into something understandable by the HPTK Input module. The InputDataProvider that adds support to Quest hand tracking can downloaded here: OVRSkeletonTracker.cs

MRTK spawns OVRCameraRig and OVRHands on Start. HPTK Core module and InputDataProviders require references to these objects which have to be found and referenced after being created by MRTK. Also, HPTK will need its own Layer to control collisions with MRTK UI. The following script will manage this requirements automatically: OVRFinder.cs

4. Project Settings (!)

HPTK requires a certain configuration to get realistic hand physics.

Set Project Settings according to Getting started guide.

Configure layer collision matrix:

1. Create a new layer called HPTK.

2. Make HPTK layer to ignore Default and UI layers.

5. Scene setup

1. Create a new empty object called HPTK. We will refer to it as the HPTK root object.

2. Drag & drop SpAvatar prefab into this new HPTK root object.

3. Change materials in SpAvatar to support Standard render pipeline (required by shaders included in MRTK).

4. Drag & drop HPTKCore prefab into the HPTK root object.

5. Add OVRFinder component to the HPTK root object. We will set its values later.

6. Create two new empty objects under the HPTK root object. Attach OVRSkeletonTracker component to each of them. Rename these new objects to OVRSkeletonTracker.L and OVRSkeletonTracker.R.

7. Set OVRSkeletonTracker.x as Input Data Provider in SpAvatar>ProxyHand.X>[Modules]>Input.X>InputModel.X (InputModel component)

8. Remove Ray objects from every hand representation in SpAvatar. MRTK has its own implementation for rays.

6. Configure OVRFinder

• Wait For: 3 seconds

InputDataProviders

• Find OVR Hands ForIdps: true

• Ovr Skeleton Tracker L: OVRSkeletonTracker.L

• Ovr Skeleton Tracker R: OVRSkeletonTracker.R

HPTK Slave SMR

• Copy Default SMR Material: any

Default SMR

• Disable Default SMR: any

• Apply Mat To Default SMR: true

• Mat To Apply: Invisible material

Parent

• Tracking Space Is Parent: true

• Root HPTK: HPTK root object

Layer

• Apply Layer Under Root: true

• Layer To Apply: HPTK

7. Configure MRTK pointer's layer mask

1. Find the MixedRealityToolkit component in your Scene.

2. Go to Input > Pointers > Pointer Settings.

3. For each layer mask in the Pointing Raycast Layer Masks array, make sure that HPTK layer is disabled.

8. Add a grabbable object

1. Drag & drop MRTK/Examples/StandardAssets/Prefabs/Cheese in your Scene. From now on, the Object.

2. Remove its NearInteractionGrabbable component (optional).

3. Find its first MeshRenderer component.

4. Add a MeshCollider under that MeshRenderer. Set Convex to true.

5. Set the layer of the Object to Default or UI.

6. Set the layer of the object with the MeshCollider attached, to HPTK.

7. Add a new empty object inside the Object called [Modules].

8. Add InteractableModule prefab as child of [Modules] object. Set InteractableModule > InteractableModel .rigidbodyRef.

9. Add an InteractableRef component to the Object. Set its reference to the Handler of its Interactable module (InteractableModule > InteractableController).