# FinalIK This section explains how to use FinalIK and HPTK on its **most common use case**: * Mixamo avatar. * 5 fingers. * T-pose. * Full body physics. * Input smoothing. * Gesture detection. * Contact/Grabbing detection. {% hint style="info" %} **For** **new scenes**, add the **DefaultSetup** prefab that fits your target platform. More details about initial setup in [Setup](https://jorge-jgnz94.gitbook.io/hptk/master/setup) section. {% endhint %} {% embed url="" %} ## 1. Import HPTK+FinalIK Integration Package Open the **Integration Manager,** click on **FinalIK** and import the built-in integration package. ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZawcQtnL70TJejsCFU%2F-MZbE-ZcMFANOUroGqUL%2Ffinalik-gif.gif?alt=media\&token=24cf51c1-a0bc-412b-b343-a19d56f4f6eb) ## 2. Template Add the prefab **FullBodyAvatar.Template.VRIK** to your scene. Then unpack it, will use it just as a template to setup our avatar. This template contains the following elements: * **FullBody.Mixamo** is the prefab that contains the data model that fits the common armature of a Mixamo avatar. It also includes the HPTK modules that provide the features of input management, gesture detection, hand physics and contact/grabbing detection. * **Representations** contains the actual meshes and transforms that compose our different representations, grouped in the different representations (repr) of our avatar. * **Wizards** contains the different components that guide you in the processes of: * Bone rotations correction. * Special points generation. * Colliders generation. * Transform-PointModel linkage (through ReprModels). * **Effects** contains only one effect that will be the responsible of fading master's mesh when it matches the slave representation. * **Targets** will represent the targets that master's VRIK will try to reach. **Replace the (Mesh)** object by the one that holds the SkinnedMeshRenderer of you avatar. **Replace the (Armature)** object by the root of the armature of your avatar (usually hips). {% hint style="info" %} Apply some **transparent material** to the mesh so its easier to see how bones and colliders are being created and corrected. {% endhint %} The resulting hierarchy should look like this: ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZawcQtnL70TJejsCFU%2F-MZbQdxuT-kkBuZm8wWa%2Ftemplate.PNG?alt=media\&token=758e339e-dd78-420d-ac7e-2d4e988868c2) ## 3. LeftHand/RightHand wizards ### **3.1. HandSearchEngine** **HandSearchEngine will serve as source of bones for the hand wizards**. The population of this source can be done as follows: 1. **Fill the hand bones** references in the **HandSearchEngine** component with the Transforms of each bone in the armature under the object that represents the wrist of the hand. 2. Then, in HandSearchEngine component, **press Search Bones**. {% hint style="warning" %} **Be careful.** Oculus hands have Thumb0 and Pinky0 bones. Mixamo armatures don't have these bones. **If your armature does not have these bones,** **leave the references for Thumb0 and Pinky0 empty in HandSearchEngine.** {% endhint %} ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZawcQtnL70TJejsCFU%2F-MZb_xsLNYad8S_o7phS%2Fhandsearch-gif.gif?alt=media\&token=39896377-86d6-4012-8adb-d4ab924476d0) ### 3.2. RotationWizard HPTK requires hand armatures to have an specific rotation but each armature comes with different rotations for their bones. **RotationWizard will create** **intermediate bones with the correct rotation** and these bones are the ones that will be referenced from HPTK. #### 3.2.1. Axes Rotation Wizard requires to pass two local directions that will use to calculate the right rotation of each bone: **Bone to Interior:** Axis that matches the internal part of the hand, in other words, the part of the bone that will be occluded when the avatar performs a fist gesture. **Wrist to Fingers:** Axis that matches the direction that goes from the wrist to the middle knuckle. As our avatar is in T-pose, front fingers should be aligned with the palm. {% hint style="warning" %} Make sure that Transform axes are being shown in **local space**. {% endhint %} | Bone to Interior **(Positive Z axis)** | Wrist to Fingers **(Positive Y axis)** | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | | #### 3.2.2. The thumb **Some armatures have unaligned rotations for the thumb bones**. In these cases **we need to correct them manually** by aligning the thumb finger with the other fingers. You can know if this is your case by setting **local rotation to (0,0,0) for each thumb bone**. If the thumb is aligned with the other fingers you can skip this correction. If it's not, rotate each finger bone intil the finger is aligned with the other fingers. As you can see in the images bellow, for our avatar, **after setting their local rotations to (0,0,0), the thumb finger bones are almost aligned but not completely so we need to fix that**. If we manage to have the fingers perfeclty aligned, Input module will be able to control hands without unwanted rotational offsets, reflecting the movements more accurately. If you had to align some finger manually, check **Required manual rotation**. | Original | Local rotations set to zero | Manually aligned | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | | | The resulting **RotationWizard** component (for this avatar) would look like this: ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZawcQtnL70TJejsCFU%2F-MZblXTAa8bYKvZYOC8q%2FrotationWizard.PNG?alt=media\&token=0969313e-a6e4-46b9-a629-f8537833bf09) ### 3.3. SpecialPointsWizard For gesture detection and other features, HPTK needs to access some special points in the hand. These special points are: * **Palm center:** Its X will look at hand wrist and its Y axis will look at thumb (ambidextrously). * **Palm normal:** Its Z axis will match hand palm normal (ambidextrously). * **Palm interior:** Estimated position of the index tip when the hand performs a fist gesture. * **Palm exterior:** Estimated position of the pinky tip when the hand performs a fist gesture. * **Pinch center:** Mid point between index tip and thumb tip. * **Throat center:** Mid point between index knuckle and thumb knuckle. If you armature comes with these points, just add their references in the HandSearchEngine component. As this is not common, **we will need to generate them the most of times.** In order to do that, for the SpecialPointsWizard component of the current hand adjust: 1. **Center range:** How close to fingers (0) or to wrist (1) will be the palm center. Default value is good for most armatures. 2. **Center to surface distance:** How thick is the hand of your avatar so palm center matches the skin of the hand of your avatar. The resulting **SpecialPointsWizard** component (for this avatar) would look like this: ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZawcQtnL70TJejsCFU%2F-MZbsFlrMmktrPPDLe1W%2FspecialPointsWizard.PNG?alt=media\&token=80bb9814-20a6-408d-bfe0-d789b38ad4f5) ## 4. Body wizards ### 4.1. Head points We will need Transforms that represent the **eyes** and the **top of the head**. If these points are missing in our armature we will have to create them. **Mixamo armatures** comes with the top head point but not with eyes so, in ou case, **we need to create a Transform for the eyes**. ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZawcQtnL70TJejsCFU%2F-MZbzPvrhd5z1VNZpIqO%2Feyes-gif.gif?alt=media\&token=3e7b4652-e032-4b59-bfbc-7ef982bef32b) ### 4.2. BodySearchEngine **BodySearchEngine will serve as source of bones for the body wizards**. The population of this source can be done as follows: 1. **Fill the body bones** references in the **BodySearchEngine** component with the Transforms of each bone in the armature under the Representations > Repr > Root object. 2. Then, in BodySearchEngine component, **press Search Bones**. ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZawcQtnL70TJejsCFU%2F-MZc-E34RBX4QaaXOQgb%2Fbodysearch-gif.gif?alt=media\&token=e52eb177-1763-4a45-8971-37ede1109315) ## 5. Clone representations **Now we have a generic representation** (Repr object) with all sources populated, hand bones rotations corrected and special points generated. Until this step, our representation is valid for both master and slave so we will duplicate Repr into Master and Slave: 1. **Rename** Repr object into Master. 2. **Clone** Master object as sibling of Master **and rename** it into Slave. 3. **Remove VRIK component from Slave > Root** **object**. Slave will be driven by Puppet modules. 4. **Change materials for master and slave meshes** so they are easily differentiable. {% hint style="info" %} **If you don't need a puppet or slave representation** leave only the Master representation and skip step 7 (Slave representation). These objects are **Puppet** and **ContactDetection** modules, under **\[Modules]** gameobject. {% endhint %} ## 6. Master representation We need actual Transforms to be reachable from the Avatar module. To do this we need each Transform to have a ReprModel that connects the Transform with the Point it represents. For each of the three **ReprWizard** components in the master representation: 1. Select **Master** as target **representation**. 2. Press **Generate missing ReprModels**. 3. Press **Link Point-Reprs**. {% hint style="info" %} **Master** representations will use **ReprModels**. **Slave** representations will use **PuppetReprModel**. {% endhint %} ## 7. Slave representation For each of the three **ReprWizard** components in the master representation: 1. Select **Slave** as target **representation**. 2. Press **Generate missing ReprModels**. 3. Press **Link Point-Reprs**. Puppet module will assign PuppetConfiguration.special values to those bones specified as "special". The default PuppetConfiguration used in full body physics makes special bones stronger than the rest. This is useful to prevent a springy or weak torso for our Slave representation. ReprWizard will set hips, spine, chest, neck and thumb bones as special automatically. ### 7.1. CollidersWizards {% hint style="info" %} Viewing colliders can be helpful for the following steps. You can enable this feature from **Physics Debug** window by clicking on **Show All**. {% endhint %} **Attach a CollidersWizard component for each HandSearchEngine and BodySearchEngine in the Slave representation.** From the Inspector, for each of the three components attached of type CollidersWizard, click on **Update Colliders**. You should see some colliders being generated for its corresponding hand or body. If these colliders does not match the avatar mesh properly, enable the option Update in Editor and modify Bone Density and the Maximum Bone Radius until finding a better result. ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZc4iTxoJCzcD1QYMgR%2F-MZcaIkZ0xidkFzPatQQ%2Fcolliderswizards.gif?alt=media\&token=d03613f4-30c3-4031-9342-e5b094792389) {% hint style="info" %} It is **recommended** to replace some of the generated colliders by MeshColliders that match the avatar mesh. Although, this can impact performance. {% endhint %} ## 8. Fade effect Go to **PuppetMasterFadeEffect** component, in **Effects > FadeEffect**, and **add the mesh of the master representation in the list of SkinnedMeshRenderers**. When the distance between the master and slave representations of the hips bone is lower than Min Error, the alpha channel of the material of each referenced SkinnedMeshRenderer will be 0. When this distance is greater than Max error, the alpha channel of the material of each referenced SkinnedMeshRenderer will be 1. The **PuppetMasterFadeEffect** component should look like this: ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZcgJHc6YmL0knTlQPM%2F-MZcikyNDkpDXv11dCey%2FpuppetMasterFadeEffect.PNG?alt=media\&token=f47dfbf7-6b80-4468-a79f-eb603079ab9b) ### 9. Setup VRIK {% hint style="warning" %} **Slave representation shouldn't be driven by VRIK.** If Representations > Slave > Root has a VRIK component attached, remove it as well as its dependant components. {% endhint %} Go to Representations > Master > Root. In the **SearchEngineToVRIK** component, press the **From Body Points to VRIK** button. Make sure that **HeadTarget, LeftWristTarget and RightWristTargets** are referenced **as targets in VRIK**. ## 10. Targets Make HPTK to move the targets that FinalIK will follow. You can do this by providing Transforms that will be placed where player's head, left wrist and right wrist should be. ### 10.1. Hand targets **Input module** will move **master representation of the wrist** according to its **Input Data Provider**. We can make Input model to **move an arbitrary Transform instead** of moving the actual master representation of the wrist, which is controlled by VRIK. This arbitrary Transform will be **LeftWristTarget** or **RightWristTarget**. For each **InputModel** of the avatar, set its corresponding target in **Move This As Wrist**. | \[Modules] > Hand.L > Input | \[Modules] > Hand.R > Input | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | | {% hint style="info" %} As VRIK does not interfere with finger movements, InputModel can access these fingers and move them directly. {% endhint %} ### 10.2. Head target **Avatar module** will move **master representation of the head** according to **HPTK.trackedCamera** (singleton). We can make Avatar model to **move an arbitrary Transform instead** of moving the actual master representation of the head, which is controlled by VRIK. This arbitrary Transform will be **HeadTarget**. In the **BodyModel** of the avatar, set HeadTarget in **Move This As Head**. ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZcnpHGk5T8ZV8bH_hP%2F-MZd0Mh-OCDCg1Inod_x%2FheadTarget.PNG?alt=media\&token=dae14f4f-3e97-4ad7-988b-c2e673e0dfe4) {% hint style="warning" %} **Make sure that the reference HPTK.trackedCamera is not empty**. This value is platform-dependent so its different on each of the available DefaultSetup prefabs. The following image shows the default tracked camera in DefaultSetup.Oculus and how to find it. {% endhint %} ### 11. Add objects By default, **HPTK singleton applies the layer** named **HPTK** to each avatar that is registeres. **This layer is recommended to not collide with Default layer** so it does not interfere with MRTK. Due to this, if you want your avatar to collide with some object, the **Colliders of this object should be in the HPTK layer too**. Add the prefab **SimpleSample** to your scene. This is a simple demo of the Pheasy script. This prefab uses the HPTK layer. Move the Goal object to define the destination of the capsule. ![](https://2698598769-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYpeL-Cll-FXMj9b3XC%2F-MZdCT_qxA24VJO3UKY-%2F-MZdJ3TEz8z07RLP3WyU%2Ffullbody-gif.gif?alt=media\&token=68340161-0dd2-4fa2-a70d-2d2e3d90cda9)