10 Inverse Simulation

10.5 Inverse kinematics

ArtiSynth supplies an inverse kinematic solver, IKSolver, that can be used to compute the configuration of an articulated structure of rigid bodies given the positions of markers attached to those bodies. A primary use of this solver is to transform a marker trajectory, which may not fit the model well due to noise or registration issues, into one that is kinematically feasible given the joint constraints on the bodies. This can be an important preconditioning step for inverse dynamic simulation, as the resulting excitations do not then need to expend effort on infeasible targets.

10.5.1 Inverse kinematics solver

An IKSolver is created with the following constructors:

IKSolver (MechModel mech, Collection<FrameMarker> mkrs)
IKSolver (MechModel mech, Collection<FrameMarker> mkrs, VectorNd weights)

mech is the MechModel containing the bodies and constraints, mkrs is a list of frame markers attached to the bodies, and weights is an optional set of marker weight values. If not specified, markers are assumed to have a weight of 1.

The solver finds all the rigid bodies attached to the specified markers, together with all body connectors (Section 3.4.3) constraining the motion of those bodies, plus any additional bodies connected to the arrangement via connectors.

The solver does not account for the following:

  1. 1.

    Non-rigid bodies, such as FEM models, that might be attached to some of the connectors;

  2. 2.

    Contact constraints.

After the solver is created, the resulting component configuration can be queried with the following methods:

MechModel getMechModel()

Returns the MechModel.

int numMarkers()

Returns the number of markers.

ArrayList<FrameMarker> getMarkers()

Returns the markers.

VectorNd getMarkerWeights()

Returns the marker weights.

void setMarkerWeights(VectorNd wgts)

Sets the marker weights.

int numBodies()

Returns the number of connected bodies.

ArrayList<RigidBody> getBodies()

Returns the connected bodies.

ArrayList<RigidTransform3d> getBodyPoses()

Returns the poses of the connected bodies.

ArrayList<RigidBody> getBodies()

Returns the bodies attached to markers.

ArrayList<BodyConstrainer> getConstrainers()

Returns all constrainers and connectors linking the bodies.

The application can then use the method

int solve (VectorNd mtargs)

to set the body poses (and hence marker positions) so as to minimize the error between the marker target positions specified by mtargs and the feasible marker positions. mtargs supplies the target positions as a composite vector of size 3\times m, where m is the value returned by numMarkers(), and the method returns either the number of iterations required for the solve, or -1 if the solve did not convergence to the required tolerance.

When using the IKSolver, it is sometimes necessary to explicitly identify some bodies as being “grounded”, in the sense that they should not be moved by the solver. This can be done by setting the grounded property of the body to true. (In a dynamic simulation, grounded bodies are often those whose dynamic property is false. However, this property is not meaningful in the context of an IK solve, and so the grounded property substitutes for this.)

Since solve() modifies the positions of the bodies associated with the solver, it changes the state of the MechModel. In order to leave the MechModel state unchanged, an application can call the solver methods saveModelState() and restoreModelState() before and after the use of solve().

An as example, the following code processes a trajectory of marker positions by calling solve() in a loop:

   MechModel mech;
   List<FrameMarker> markers;
   int nframes; // number of frames in the trajectory
   ...
   IKSolver solver = new IKSolver (mech, markers);
   VectorNd mtargs = new VectorNd (3*markers.size());
   solver.saveModelState(); // save the MechModel state
   for (int i=0; i<nframes; i++) {
      ... initialize mtargs for the frame ...
      solver.solve (mtargs);
      ArrayList<RigidTransform3d> poses = solver.getBodyPoses();
      ... do something with the poses ...
   }
   solver.restoreModelState(); // restore the MechModel state

IKSolver exports a number of methods related to the solution process:

double getConvergenceTol()

Returns the convergence tolerance.

void setConvergenceTol (double tol)

Sets the convergence tolerance.

int getMaxIterations()

Returns the maximum number of solve iterations.

void setMaxIterations (int maxi)

Sets the maximum number of solve iterations.

void saveModelState()

Saves the MechModel state internally.

void restoreModelState()

Restores MechModel state to that saved internally.

The default values for the convergence tolerance and maximum number of iterations are 10^{-6} and 30.

10.5.2 Creating probes with IKSolver

IKSolver supplies the following methods that use its solve() method to convert probes of target marker positions into PositionInputProbes of feasible markers positions or body poses:

PositionInputProbe createMarkerPositionProbe (String name, NumericProbeBase targProbe, Mboolean useTargetProps, double interval)
PositionInputProbe createMarkerPositionProbe (String name, String targDataFilePath, Mboolean useTargetProps, double interval)
PositionInputProbe createBodyPoseProbe (String name, NumericProbeBase targProbe, MRotationRep rotRep, boolean useTargetProps, double interval)
PositionInputProbe createBodyPoseProbe (String name, String targDataFilePath, MRotationRep rotRep, boolean useTargetProps, double interval)

In these methods, name is an optional probe name, targProbe is a probe containing the target positions for the markers, targDataFilePath is a path to a probe file (Section 5.4.4) containing target position data, useTargetProps specifies whether the probe should be bound to target properties (Section 5.4.7), rotRep is an instance of RotationRep, which indicates how rotations should be represented (Section 5.4.7), and interval is the time spacing between knots in the created probe. The created probe’s start and stop time is taken from the input probe.

These probe creation methods take care to save and restore the MechModel state.

The only restriction on the targProbe used by these methods is that its data vector size is 3\times m, where m is the number of markers associated with the solver.

These methods transform target marker data into position probes for either markers or body poses that are kinematically feasible. In the following example, a TRCReader (Section 5.5.1) is used to create a marker probe from a TRC file and transform it into a feasible marker position probe:

include artisynth.probes.TRCReader;
include artisynth.probes.IKSolver;
   ...
   MechModel mech;
   File trcFile;
   List<FrameMarker> mkrs;
   ...
   // Create a probe from the TRC file
   PositionInputProbe rawProbe =
      TRCReader.createInputProbe (
         "raw targets", mkrs, trcFile, /*targetProps*/false);
   // Use an IKSolver to transform this into a feasible target probe
   IKSolver solver = new IKSolver (mech, mkrs);
   PositionInputProbe feasibleProbe =
      solver.createMarkerPositionProbe (
         "feasible targets", rawProbe, /*targetProps*/false, /*interval*/0.01);

10.5.3 Using IKSolver in a simulation context

Much of the discussion above entails using IKSolver in a stand-alone manner (perhaps in a model’s build() method) that is independent of an actual simulation. However, there may be situations in which one wishes to use the solver to update body and marker positions as a simulation progresses.

One way to do this would be to define a Controller (Section 5.3) that calls the solve() method inside its apply() method. Marker targets would have to be supplied to solver in some way, and it would necessary to ensure that the bodies being controlled by the solver have their dynamic property set to false. IKSolver supplies some convenience methods to manage this:

void setBodiesDynamic (boolean dynamic)

Sets dynamic property for all bodies.

void resetBodiesDynamic()

Resets dynamic property for bodies to their initial state.

Another way to use IKSolver in a simulation is to use an IKProbe, which embeds an IKSolver inside aNumericControlProbe. An IKProbe can be created with the following constructors:

IKProbe (String name, MechModel mech, Collection<FrameMarker> mkrs, MVectorNd wgts, String fileName)
IKProbe (String name, MechModel mech, Collection<FrameMarker> mkrs, MVectorNd wgts, double startTime, double stopTime)

where name is an optional probe name, fileName is a probe file containing probe data, startTime and stopTime set the probe’s start and stop times, and the other arguments mirror those of the IKSolver constructors (wgts can be null if not needed).

If the second constructor is used, the probe is created with only two data points at its start and end (reflecting the current marker positions), and so the actual trajectory data must be added. One way to do this would be to use one of the probe’s addData() methods:

   MechModel mech;
   List<FrameMarker> markers;
   int nframes; // number of frames in the trajectory
   ...
   IKProbe ikprobe = new IKProbe (
      "inverseK", mech, markers, /*wgts*/null, /*start*/0.0, /*stop*/5.0);
   VectorNd mtargs = new VectorNd (3*markers.size());
   for (int i=0; i<nframes; i++) {
      double t;
      ... initialize t and mtargs for the frame ...
      ikprobe.addData (t, mtargs);
   }
   // bodies must be non-dynamic when solver is applied:
   ikprobe.setBodiesDynamic (false);
   addInputProbe (ikprobe); // add probe to the root model

The method setBodiesDynamic() is one of several IKSolver methods that are passed on by IKProbe, including resetBodiesDynamic(), set/getConvergenceTol(), set/getMaxIterations(), numMarkers(), and getMarkers(). IKProbe also maintains the property bodiesNonDynamicIfActive, which ensures that the probe automatically calls setBodiesDynamic(false) when it becomes active and resetBodiesDynamic() when it becomes inactive.

Another way to set IKProbe data is to simply copy it from another probe using setValues(), as described in Section 5.4.5. For example:

   MechModel mech;
   List<FrameMarker> markers;
   PositionInputProbe targetProbe;
   ...
   IKProbe ikprobe = new IKProbe (
      "inverseK", mech, markers, /*wgts*/null, /*start*/0.0, /*stop*/5.0);
   // copy data from the target probe
   ikprobe.setValues (targetProbe, /*useAbsoluteTime*/false);
   // bodies must be non-dynamic when solver is applied:
   ikprobe.setBodiesDynamic (false);
   addInputProbe (ikprobe); // add probe to the root model

10.5.4 Example: moving MultiJointedArm with an IKProbe

Figure 10.17: IKMultiJointedArm at the end of its inverse kinematic motion.

The example model artisynth.demos.tutorial.IKMultiJointedArm shows how to move the MultiJointedArm model (Section 3.5.16) using an IKProbe and marker target data read from a TRC file. This is essentially the same demo as TRCMultiJointedArm (Section 5.5.3), except that it uses inverse kinematics instead of a crude physical simulation in which the markers are connected to the target points by springs. The source code for the build() method is shown below:

1    public void build (String[] args) throws IOException {
2       super.build(args); // create MultiJointArm
3
4       // read a TRC file containing tracking data for all the markers.
5       File trcfile = new File(getSourceRelativePath ("data/multiJointMkrs.trc"));
6       TRCReader reader = new TRCReader (trcfile);
7       reader.readData();
8
9       // determine probe times from TRC file
10       double startTime = reader.getFrameTime (0);
11       double stopTime = reader.getFrameTime (reader.numFrames()-1);
12
13       // create an IKProbe to drive the bodies from the marker data
14       IKProbe ikprobe = new IKProbe (
15          "inverseK", myMech, myMech.frameMarkers(),
16          /*wgts*/null, startTime, stopTime);
17
18       // load the probe data directly using the TRC data
19       VectorNd mtargs = new VectorNd();
20       for (int fidx=0; fidx<reader.numFrames(); fidx++) {
21          double t = reader.getFrameTime (fidx);
22          reader.getMarkerPositions (mtargs, fidx);
23          ikprobe.addData (t, mtargs);
24       }
25       addInputProbe (ikprobe);
26       // ensure bodies are non-dynamic when probe is active
27       ikprobe.setBodiesNonDynamicIfActive (true);
28    }

Since the model extends MultiJointedArm, build() first calls super.build() to create that model. Next, as in TRCMultiJointedArm, a TRC file containing the marker trajectories is read (lines 5-7). Next, an IKProbe is created to control the markers, with its start and stop times determined directly from the TRC frame times (lines 10-16).

To illustrate a different way of setting probe data, the marker positions for each frame are extracted directly from the TRC reader and added to the probe using addData(t,vec) (lines 19-24). The probe’s bodiesNonDynamicIfActive property is also set (line 27) to ensure that the rigid bodies are set non-dynamic when the probe is active.

To run this example in ArtiSynth, select All demos > tutorial > IKMultiJointedArm from the Models menu.

10.5.5 Example: controlling ToyMuscleArm with inverse kinematics

Figure 10.18: Left: IKInverseMuscleArm, with target positions (cyan) filtered via inverse kinematics. Right: same model with unfiltered targets, and the misregistration between the targets and their markers plainly visible.

The final example in this chapter also involves using the tracking controller to move ToyMuscleArm. Several marker targets are used, driven by data read from a TRC file, an IKSolver transforms the data into a feasible trajectory, and the data is used to create velocity targets that when combined with PD control reduce the tracking error. The model is

  artisynth.demos.tutorial.IKInverseMuscleArm

and its class definition, excluding include statements, is shown below:

1 public class IKInverseMuscleArm extends ToyMuscleArm {
2
3    double startTime = 0; // probe start times
4    double stopTime = 4;  // probe stop times
5
6    public void build (String[] args) throws IOException {
7       super.build(args); // create ToyMuscleArm
8
9       // create a tracking controller
10       TrackingController tcon = new TrackingController (myMech, "tcon");
11       addController (tcon);
12       // for each muscle, reinitialize its rest length for the new
13       // configuration and add it to the controller as an exciter
14       for (AxialSpring spr : myMech.axialSprings()) {
15          spr.setRestLength (spr.getLength());
16          tcon.addExciter ((Muscle)spr);
17       }
18       for (MultiPointSpring spr : myMech.multiPointSprings()) {
19          spr.setRestLength (spr.getLength());
20          tcon.addExciter ((MultiPointMuscle)spr);
21       }
22       // set the base to be ground - useful when using the IK solver
23       myMech.rigidBodies().get("base").setGrounded (true);
24
25       // apply inverse tracking to markers with the following numbers:
26       int[] mkrNums = new int[] { 8, 0, 2 };
27       // use the numbers to build a list of the markers, and also a list of
28       // their labels in the TRC file
29       ArrayList<FrameMarker> markers = new ArrayList<>();
30       ArrayList<String> mkrLabels = new ArrayList<>();
31       for (int num : mkrNums) {
32          markers.add (myMech.frameMarkers().getByNumber (num));
33          mkrLabels.add ("mkr"+num);
34       }
35
36       // add markers to the controller as motion targets
37       for (FrameMarker mkr : markers) {
38          tcon.addPointTarget (mkr);
39       }
40       // add an L-2 regularization term to handle exciter redundancy
41       tcon.setL2Regularization(/*weight=*/0.1);
42
43       // create a probe from TRC data to drive the points
44       File trcFile = new File(getSourceRelativePath("data/offMarkers.trc"));
45       PositionInputProbe targetPos =
46           TRCReader.createInputProbeUsingLabels (
47              "target positions", tcon.getTargetPoints(),
48              mkrLabels, trcFile, /*targetProps*/false);
49       addInputProbe (targetPos);
50
51       // create an IKSolver and use this to create a feasible target probe
52       IKSolver iksolver = new IKSolver (myMech, markers);
53       PositionInputProbe feasible = iksolver.createMarkerPositionProbe (
54          /*name*/null, targetPos, /*useTargetProps*/false, 0.01);
55       // replace data in target probe with feasible data
56       targetPos.setValues (feasible, /*useAbsoluteTime*/false);
57
58       // create a velocity input probe to help with tracking
59       VelocityInputProbe targetVel =
60          VelocityInputProbe.createNumeric (
61             "target velocities", targetPos, /*useTargetProps*/false, 0.01);
62       addInputProbe (targetVel);
63
64       // set tracking controler for PD control so it can use velocity data
65       MotionTargetTerm mterm = tcon.getMotionTargetTerm();
66       mterm.setUsePDControl (true);
67       mterm.setKd (10);
68       mterm.setKp (1000);
69
70       // add an output probe to record the excitations
71       NumericOutputProbe exprobe = InverseManager.createOutputProbe (
72          tcon, ProbeID.COMPUTED_EXCITATIONS, /*fileName=*/null,
73          startTime, stopTime, /*interval=*/-1);
74       addOutputProbe (exprobe);
75
76       // to see lag when velocity probe is inactive, add a probe to record the
77       // target and actual position of the distal marker
78       List<Point> pnts = Arrays.asList (
79          tcon.getTargetPoints().get(0), markers.get(0));
80       PositionOutputProbe tracking = new PositionOutputProbe (
81          "distal tracking", pnts, /*rotRep*/null, startTime, stopTime);
82       addOutputProbe (tracking);
83
84       // add inverse control panel
85       InverseManager.addInversePanel (this, tcon);
86
87       // set point render radius for controller so target points not too large
88       RenderProps.setPointRadius (tcon, 0.02);
89    }
90 }

The build() method starts by setting up a tracking controller in the same manner as for InverseMuscleArm (Section 10.2.4) (lines 10-21). Next, the base body is marked as “ground” by setting its grounded property (line 23); this is necessary to prevent the IKSolver from moving it.

This example tracks three markers with component numbers 8, 0, and 2. These are collected into the list markers and added to the controller as targets (lines 23-39). A target position probe containing the marker trajectory, and attached to the tracking controller target points, is then created directly from the TRC file (lines 44-49). Since this file contains data for all the markers, we specify the markers we want using the label list mkrLabels which was created from their numbers.

To filter the marker data (which for this example has been deliberately misregistered), an IKSolver is created and used to create an input probe containing a feasible trajectory (lines 52-55). The data from this probe is then used to overwrite that of the target position probe (line 56). After this is done, the target position probe is differentiated to create a target velocity probe (lines 59-62). To take advantage of this velocity information, the controller is configured to use PD control (lines 65-68) with constants that were tuned manually. Lastly, an output probe is created to record the excitations, and another is created to observe both the target and actual position of the distal marker (which appears first in markers).

Figure 10.19: Timeline of IKInverseMuscleArm, with the target position and velocity probes expanded, showing the marker trajectories and the velocities that were determined by differentiating them.
Figure 10.20: Left: excitations produced by IKInverseMuscleArm with the marker trajectory filtered through inverse kinematics. Right: excitations produced when there is no filtering, and the model needs to expend effort accommodating the misregistration.

To run this example in ArtiSynth, select All demos > tutorial > IKInverseMuscleArm from the Models menu. A view of the timeline, showing the four probes, is given in Figure 10.19. Figure 10.20 (left) shows the resulting excitations, while Figure 10.20 (right) shows the different excitations produced if the target trajectory is left unfiltered. (To run with an unfiltered trajectory, comment out line 56. The model with then appear as in Figure 10.18, right). Finally, Figure 10.21 shows two outputs of the “distal tracking” probe, the left one with velocity information available to the controller, and the right one with it removed (which can be done interactively by making the target velocity probe inactive). Without velocity information, the marker lags the target by a noticeable amount.

Figure 10.21: Left: target and actual trajectories of the distal marker when velocity information is supplied to the tracking controller. Right: same trajectory pair when the controller is deprived of velocity information by deactivating the target velocity probe. The resulting lag in the tracking is clearly visible.