Gestures

Gesture from pose

One of the built-in gestures can receive a HandPose as parameter and provide a lerp value that reflects how similar is the current hand gesture to the referenced HandPose. This gesture is HandPoseMatch.

HandPoseMatch must be referenced in GestureDetectionModel.extra as any other extra gesture. It also needs a reference to a FingerPoseMatch for each finger. Each FingerPoseMatch must also be referenced in its respective FingerGestureModel.extra list so its values are updated by GestureDetectionController.

HandPoseMatch averages the lerp value of each FingerPoseMatch. Each FingerPoseMatch calculates the average angle between the goal bone rotation and the actual bone rotation for each finger bone and translate it to a lerp value. FingerPoseMatch.lerp equal to 0 means that the angle is greater than HandPoseMatch.maxAngle for every finger bone. FingerPoseMatch.lerp equal to 1 means that the angle is 0 for every finger bone so the finger is performing HandPoseMatch.pose for that finger perfeclty.

Variables

• Pose: HandPose that will act as goal pose. Performing this gesture exactly will make HandPoseMatch.lerp to be equal to 1.

• MaxAngle: Angle from which. Big values (80-200) result in flexible gesture detection. Small values (10-60) result in a strict gesture detection.

• Related finger gestures: FingerPoseMatch gestures linked to this HandPoseMatch.

Scripted gestures

Comparing current hand state against an arbitrary hand pose every frame can be unnecessarily demanding and it can be too strict for some use cases. For a more optimized, more flexible or more complex gesture detection, scripted gestures are the best option.

Scripted gestures inherit from HandGesture or FingerGesture which, in turn, ultimately inherit from MonoBehaviour. From their model variable it's possible to get any other gesture at the same level (same hand or same finger) or even accessing their corresponding avatar submodels through _model.hand or _model.finger.

Hand gestures

This is the code skeleton for a scripted hand gesture:

public class YourHandGesture : HandGesture
{
    // GestureDetectionModel HandGesture._model

    public override sealed void InitHandGesture()
    {
        base.InitHandGesture();
        //...
    }

    public override sealed void HandLerpUpdate()
    {
        base.HandLerpUpdate();
        // ...
        _lerp = ...
        if (providesIntention) { //... }
    }

    public override sealed void LateGestureUpdate()
    {
        base.LateGestureUpdate();
        //...
    }
}

Finger gestures

This is the code skeleton for a scripted finger gesture:

public class YourFingerGesture : FingerGesture
{
    // FingerGesturesModel FingerGesture._model

    public override sealed void InitFingerGesture()
    {
        base.InitFingerGesture();      
        //...
    }

    public override sealed void FingerLerpUpdate()
    {
        base.FingerLerpUpdate();      
        //...    
        _lerp = ...    
        if (providesIntention) { //... }
    }

    public override sealed void LateGestureUpdate()
    {
        base.LateGestureUpdate(); 
        //...
    }
}

Connection

Scripted gestures must be added to the scene and connected to a GestureDetection model which will be the responsible of updating (or not) these gestures. By connecting them to a GestureDetection module, it's possible for other scripts to reach them from AvatarModel (or AvatarView) or any of its sub-modules (or sub-views).

This connection can be performed as follows:

1. Attach the scripted gesture as component to a new empty GameObject.

2. Add its reference to its corresponding list of extra gestures:

   • GestureDetectionModel.extra for hand gestures.

   • FingerGesturesModel.extra for finger gestures.