FinalIK
Last updated
Last updated
This section explains how to use FinalIK and HPTK on its most common use case:
Mixamo avatar.
5 fingers.
T-pose.
Full body physics.
Input smoothing.
Gesture detection.
Contact/Grabbing detection.
For new scenes, add the DefaultSetup prefab that fits your target platform.
More details about initial setup in Setup section.
Open the Integration Manager, click on FinalIK and import the built-in integration package.
Add the prefab FullBodyAvatar.Template.VRIK to your scene. Then unpack it, will use it just as a template to setup our avatar. This template contains the following elements:
FullBody.Mixamo is the prefab that contains the data model that fits the common armature of a Mixamo avatar. It also includes the HPTK modules that provide the features of input management, gesture detection, hand physics and contact/grabbing detection.
Representations contains the actual meshes and transforms that compose our different representations, grouped in the different representations (repr) of our avatar.
Wizards contains the different components that guide you in the processes of:
Bone rotations correction.
Special points generation.
Colliders generation.
Transform-PointModel linkage (through ReprModels).
Effects contains only one effect that will be the responsible of fading master's mesh when it matches the slave representation.
Targets will represent the targets that master's VRIK will try to reach.
Replace the (Mesh) object by the one that holds the SkinnedMeshRenderer of you avatar.
Replace the (Armature) object by the root of the armature of your avatar (usually hips).
Apply some transparent material to the mesh so its easier to see how bones and colliders are being created and corrected.
The resulting hierarchy should look like this:
HandSearchEngine will serve as source of bones for the hand wizards. The population of this source can be done as follows:
Fill the hand bones references in the HandSearchEngine component with the Transforms of each bone in the armature under the object that represents the wrist of the hand.
Then, in HandSearchEngine component, press Search Bones.
Be careful. Oculus hands have Thumb0 and Pinky0 bones. Mixamo armatures don't have these bones. If your armature does not have these bones, leave the references for Thumb0 and Pinky0 empty in HandSearchEngine.
HPTK requires hand armatures to have an specific rotation but each armature comes with different rotations for their bones. RotationWizard will create intermediate bones with the correct rotation and these bones are the ones that will be referenced from HPTK.
Rotation Wizard requires to pass two local directions that will use to calculate the right rotation of each bone:
Bone to Interior: Axis that matches the internal part of the hand, in other words, the part of the bone that will be occluded when the avatar performs a fist gesture.
Wrist to Fingers: Axis that matches the direction that goes from the wrist to the middle knuckle. As our avatar is in T-pose, front fingers should be aligned with the palm.
Make sure that Transform axes are being shown in local space.
Bone to Interior (Positive Z axis) | Wrist to Fingers (Positive Y axis) |
Some armatures have unaligned rotations for the thumb bones. In these cases we need to correct them manually by aligning the thumb finger with the other fingers.
You can know if this is your case by setting local rotation to (0,0,0) for each thumb bone. If the thumb is aligned with the other fingers you can skip this correction. If it's not, rotate each finger bone intil the finger is aligned with the other fingers.
As you can see in the images bellow, for our avatar, after setting their local rotations to (0,0,0), the thumb finger bones are almost aligned but not completely so we need to fix that.
If we manage to have the fingers perfeclty aligned, Input module will be able to control hands without unwanted rotational offsets, reflecting the movements more accurately.
If you had to align some finger manually, check Required manual rotation.
Original | Local rotations set to zero | Manually aligned |
The resulting RotationWizard component (for this avatar) would look like this:
For gesture detection and other features, HPTK needs to access some special points in the hand. These special points are:
Palm center: Its X will look at hand wrist and its Y axis will look at thumb (ambidextrously).
Palm normal: Its Z axis will match hand palm normal (ambidextrously).
Palm interior: Estimated position of the index tip when the hand performs a fist gesture.
Palm exterior: Estimated position of the pinky tip when the hand performs a fist gesture.
Pinch center: Mid point between index tip and thumb tip.
Throat center: Mid point between index knuckle and thumb knuckle.
If you armature comes with these points, just add their references in the HandSearchEngine component. As this is not common, we will need to generate them the most of times. In order to do that, for the SpecialPointsWizard component of the current hand adjust:
Center range: How close to fingers (0) or to wrist (1) will be the palm center. Default value is good for most armatures.
Center to surface distance: How thick is the hand of your avatar so palm center matches the skin of the hand of your avatar.
The resulting SpecialPointsWizard component (for this avatar) would look like this:
We will need Transforms that represent the eyes and the top of the head. If these points are missing in our armature we will have to create them. Mixamo armatures comes with the top head point but not with eyes so, in ou case, we need to create a Transform for the eyes.
BodySearchEngine will serve as source of bones for the body wizards. The population of this source can be done as follows:
Fill the body bones references in the BodySearchEngine component with the Transforms of each bone in the armature under the Representations > Repr > Root object.
Then, in BodySearchEngine component, press Search Bones.
Now we have a generic representation (Repr object) with all sources populated, hand bones rotations corrected and special points generated. Until this step, our representation is valid for both master and slave so we will duplicate Repr into Master and Slave:
Rename Repr object into Master.
Clone Master object as sibling of Master and rename it into Slave.
Remove VRIK component from Slave > Root object. Slave will be driven by Puppet modules.
Change materials for master and slave meshes so they are easily differentiable.
If you don't need a puppet or slave representation leave only the Master representation and skip step 7 (Slave representation).
These objects are Puppet and ContactDetection modules, under [Modules] gameobject.
We need actual Transforms to be reachable from the Avatar module. To do this we need each Transform to have a ReprModel that connects the Transform with the Point it represents.
For each of the three ReprWizard components in the master representation:
Select Master as target representation.
Press Generate missing ReprModels.
Press Link Point-Reprs.
Master representations will use ReprModels. Slave representations will use PuppetReprModel.
For each of the three ReprWizard components in the master representation:
Select Slave as target representation.
Press Generate missing ReprModels.
Press Link Point-Reprs.
Puppet module will assign PuppetConfiguration.special values to those bones specified as "special". The default PuppetConfiguration used in full body physics makes special bones stronger than the rest. This is useful to prevent a springy or weak torso for our Slave representation.
ReprWizard will set hips, spine, chest, neck and thumb bones as special automatically.
Viewing colliders can be helpful for the following steps. You can enable this feature from Physics Debug window by clicking on Show All.
Attach a CollidersWizard component for each HandSearchEngine and BodySearchEngine in the Slave representation.
From the Inspector, for each of the three components attached of type CollidersWizard, click on Update Colliders. You should see some colliders being generated for its corresponding hand or body.
If these colliders does not match the avatar mesh properly, enable the option Update in Editor and modify Bone Density and the Maximum Bone Radius until finding a better result.
It is recommended to replace some of the generated colliders by MeshColliders that match the avatar mesh. Although, this can impact performance.
Go to PuppetMasterFadeEffect component, in Effects > FadeEffect, and add the mesh of the master representation in the list of SkinnedMeshRenderers.
When the distance between the master and slave representations of the hips bone is lower than Min Error, the alpha channel of the material of each referenced SkinnedMeshRenderer will be 0.
When this distance is greater than Max error, the alpha channel of the material of each referenced SkinnedMeshRenderer will be 1.
The PuppetMasterFadeEffect component should look like this:
Slave representation shouldn't be driven by VRIK. If Representations > Slave > Root has a VRIK component attached, remove it as well as its dependant components.
Go to Representations > Master > Root. In the SearchEngineToVRIK component, press the From Body Points to VRIK button.
Make sure that HeadTarget, LeftWristTarget and RightWristTargets are referenced as targets in VRIK.
Make HPTK to move the targets that FinalIK will follow. You can do this by providing Transforms that will be placed where player's head, left wrist and right wrist should be.
Input module will move master representation of the wrist according to its Input Data Provider. We can make Input model to move an arbitrary Transform instead of moving the actual master representation of the wrist, which is controlled by VRIK. This arbitrary Transform will be LeftWristTarget or RightWristTarget.
For each InputModel of the avatar, set its corresponding target in Move This As Wrist.
[Modules] > Hand.L > Input | [Modules] > Hand.R > Input |
As VRIK does not interfere with finger movements, InputModel can access these fingers and move them directly.
Avatar module will move master representation of the head according to HPTK.trackedCamera (singleton). We can make Avatar model to move an arbitrary Transform instead of moving the actual master representation of the head, which is controlled by VRIK. This arbitrary Transform will be HeadTarget.
In the BodyModel of the avatar, set HeadTarget in Move This As Head.
Make sure that the reference HPTK.trackedCamera is not empty. This value is platform-dependent so its different on each of the available DefaultSetup prefabs. The following image shows the default tracked camera in DefaultSetup.Oculus and how to find it.
By default, HPTK singleton applies the layer named HPTK to each avatar that is registeres. This layer is recommended to not collide with Default layer so it does not interfere with MRTK. Due to this, if you want your avatar to collide with some object, the Colliders of this object should be in the HPTK layer too.
Add the prefab SimpleSample to your scene. This is a simple demo of the Pheasy script. This prefab uses the HPTK layer. Move the Goal object to define the destination of the capsule.
