DataDrivenControl_SCA2012.pdf

(1216 KB) Pobierz
Eurographics/ ACM SIGGRAPH Symposium on Computer Animation (2012)
P. Kry and J. Lee (Editors)
Simple Data-Driven Control for Simulated Bipeds
T. Geijtenbeek
N. Pronost
A. F. van der Stappen
Department of Information and Computing Sciences, Utrecht University, The Netherlands
Abstract
We present a framework for controlling physics-based bipeds in a simulated environment, based on a variety of
reference motions. Unlike existing methods for control based on reference motions, our framework does not re-
quire preprocessing of the reference motion, nor does it rely on inverse dynamics or on-line optimization methods
for torque computation. It consists of three components:
Proportional-Derivative Control
to mimic motion char-
acteristics, a specific form of
Jacobian Transpose Control
for balance control, and
Covariance Matrix Adaption
for off-line parameter optimization, based on a novel high-level reward function. The framework can easily be
implemented using common off-the-shelf physics engines, and generates simulations at approximately
4×
real-
time on a single core of a modern PC. Our framework advances the state-of-the-art by demonstrating motions of
a diversity and dynamic nature previously unseen in comparable methods, including squatting, bowing, kicking,
and dancing motions. We also demonstrate its ability to withstand external perturbations and adapt to changes in
character morphology.
Categories and Subject Descriptors
(according to ACM CCS)
: I.3.7 [Computer Graphics]: Three-Dimensional
Graphics and Realism—Animation I.6.8 [Simulation and Modeling]: Types of Simulation—Animation
1. Introduction
After decades of floundering, recent publications show that
the field of physics-based character animation has matured,
demonstrating a visual quality and robustness that can com-
pete with kinematics-based approaches. However, despite
widespread adoption of physics simulation for lifeless phe-
nomena, such as cloth, water or
rag-doll
characters, produc-
tion games still resort to kinematics-based approaches for
the animation of actively controlled characters.
In an attempt to understand this reservation, we distin-
guish between two different approaches used in physics-
based character animation research. On one hand, there
are methods that compute joint torques based on
kinematic
parameters,
such as joint angles or center-of-mass. These
methods are relatively easy to implement and integrate well
with common physics engines, such as the
Open Dynam-
ics Engine
or
PhysX.
Recent publications that use this ap-
proach have displayed a variety of robust and versatile lo-
comotion controllers (e.g. [CBvdP10,
WFH10]).
However,
these methods allow style control only through
key-framing
or optimization for high-level criteria. Methods that use cap-
tured reference motions have several limitations or require
extensive preprocessing (e.g. [SKL07,
YLvdP07]).
c The Eurographics Association 2012.
Figure 1:
Example results of the control framework.
On the other hand, there are methods that compute joint
torques based on the
equations-of-motion
that describe the
underlying character dynamics. Such methods have demon-
strated impressive results, including the ability to robustly
track recorded running and turning motions, as well as ro-
bust navigation over uneven terrain (e.g. [MLPP09,
WP10,
MdLH10]).
However, these methods are also much more
difficult to implement. They require thorough knowledge
of constrained dynamics and optimization theory and do
not mix trivially with common physics engines. In addition,
computational requirements are often high: publications typ-
ically report around 1× real-time performance for a single
character on a modern PC.
T. Geijtenbeek & N. Pronost & A. F. van der Stappen / Simple Data-Driven Control for Simulated Bipeds
Based on these observations, we expect the former cate-
gory to be more eligible for adaption in industry at this point
in time – especially for the rapidly developing mobile appli-
cation market. Our contribution consists of a control frame-
work that can robustly track unmodified reference motions
based on kinematic properties, without the need to explicitly
invert the equations-of-motion. It advances the state-of-the-
art by allowing robust control based on reference motions
with a diversity and dynamic nature previously unseen in
comparable methods.
Our control framework consists of three components:
Proportional-Derivative Control
to mimic motion character-
istics, a specific form of
Jacobian Transpose Control
for bal-
ance control, and
Covariance Matrix Adaption
for off-line
parameter optimization based on a novel high-level reward
function. It produces simulations at approximately 4× real-
time on a single core of a modern PC. We demonstrate the
capabilities of our framework through a variety of motions,
including squatting, bowing, kicking and dancing, as well
as display its ability to withstand external perturbations and
adapt to changes in character morphology.
2. Related Work
Following our introduction, we distinguish between publica-
tions that compute joint torques based on
kinematic parame-
ters
and those that compute torques based on the
equations-
of-motion
describing the underlying character dynamics.
Torques Computed from Kinematic Properties
Early ex-
amples using this approach include the work of Raibert and
Hodgins [RH91], as well as the milestone human athletics
controller by Hodgins et al. [HWBO95]. Both techniques
control speed and balance through on-line modulation of
target trajectories, and compute torques using Proportional-
Derivative Control. Other key examples are the walking con-
troller of Laszlo et al. [LvdPF96], the various controllers of
and Faloutsos et al. [FvdPT01], and the interactive track-
ing controller of Zordan and Hodgins [ZH02]. More re-
cently, Yin et al. [YLvdP07] introduced SIMBICON, a
framework supporting various types of robust gaits, also us-
ing on-line trajectory modulation for balance and PD Con-
trol, which has been the basis for many subsequent publica-
tions [CBYvdP08,CBvdP09,YCBvdP08,WFH09,WFH10].
Coros et al. incorporate a form of
Jacobian Transpose Con-
trol
[SADM94], resulting in versatile and generic loco-
motion controllers for bipeds [CBvdP10] and quadrupeds
[CKJ
∗
11].
Their control method is based on the
Virtual
Model Control
framework by Pratt et al. [PCTD01] (see also
Section
3.2.1).
To minimize manual tuning, several control frameworks
use off-line parameter optimization based on high-level cri-
teria (a topic famously explored by Sims [Sim94]). Hodgins
and Pollard [HP97] show how optimization can be used to
adapt to changes in character morphology. The parameters of
the SIMBICON framework have been optimized to generate
new behaviors [YCBvdP08], to control style [WFH09], and
to increase robustness [WFH10]. Tan et al. [TGTL11] use
off-line parameter optimization for swimming controllers.
Recent publications typically use Covariance Matrix Adap-
tion (CMA) [Han06] for off-line parameter optimization.
There are not many publications that use kinematics-
based torque computation in combination with captured ref-
erence motions. An early exception is the work of Zor-
dan and Hodgins [ZH02], who track several balanced dual-
stance motions using a balance compensation strategy based
on the work of Wooten and Hodgins [WH00]. Sharon and
Van de Panne [SvdP05] and Sok et al. [SKL07] demonstrate
data-driven controllers that are limited to 2D, both based on
state-action maps. The SIMBICON framework can also be
used in combination with captured reference motions, but
this requires unautomated pre-processing of the motion data
[YLvdP07]. Subsequent work of Lee et al. [LKL10] does
demonstrate robust tracking of unaltered reference motions
using a SIMBICON-like balance strategy, but their method
uses inverse dynamics for torque computation, which re-
quires access to the equations-of-motion.
Torques Computed from Equations-of-Motion
This cat-
egory of approaches is linked to the
spacetime
anima-
tion framework by Witkin and Kass [WK88] and com-
putes torques through constrained optimization based on
the equations-of-motions describing the character dynam-
ics. Stewart and Cremer [SC92] use this method to produce
physically correct animations for climbing and descending
stairs. Abe et al. [AdSP07] use quadratic programming (QP)
to find the set of torques that optimally drive the center-
of-mass over the base of support, demonstrating robust bal-
ance behaviors on moving bases. Macchietto et al. [MZS09]
use inverse dynamics to track trajectories that minimize an-
gular momentum, to which Wu and Zordan [WZ10] add
stepping motions for additional balance correction. Lee et
al. [LKL10] use inverse dynamics in combination with a
SIMBICON-like balance strategy to robustly track captured
reference motions. De Lasa et al. [dLMH10] use on-line
prioritized optimization to construct robust locomotion and
jumping controllers. Muico et al. [MLPP09,MPP11] use off-
line optimization of reference motions and contact forces,
in combination with a nonlinear quadratic regulator to cre-
ate agile walking and running controllers. Wu and Popovi´
c
[WP10] use optimized end-effector trajectories in combina-
tion with QP for navigation over uneven terrain.
Other publications use short-horizon optimization of an
internal model to acquire an on-line look-ahead policy. Ex-
amples are the work of Da Silva et al. [dSAP08], who
demonstrate walking controllers based on reference mo-
tions, and Kwon and Hodgins [KH10], who demonstrate
running controllers based on reference motions. Mordatch
et al. [MdLH10] demonstrate locomotion over constrained
and uneven terrain, using the low-level control framework
c The Eurographics Association 2012.
T. Geijtenbeek & N. Pronost & A. F. van der Stappen / Simple Data-Driven Control for Simulated Bipeds
of [dLMH10]. Jain et al. [JL11] perform motion tracking
using a model based on the principal modes of the charac-
ter, while Ye and Liu [YL10] use an abstract internal model
based on center-of-mass and ground reaction force.
Because of the tight link between control and simulation
in these methods, physics simulation is mostly performed
as part of the control framework. Exceptions of publications
in which such methods have been integrated with common
physics engines are the work of Da Silva et al. [dSAP08]
(Open
Dynamics Engine)
and the uneven terrain controller
of Wu and Popovi´ [WP10] (PhysX).
c
3. Control Framework
The goal of our control framework is to have a simulated
biped follow an unmodified reference motion as faithfully as
possible, while maintaining balance and withstand external
perturbations. It consists of the following elements:
•
Tracking control
(Section
3.1).
To mimic the characteris-
tics of a recorded motion, we track the reference trajec-
tories of the individual degrees of freedom (DOFs) using
Proportional-Derivative (PD) Control.
•
Balance control
(Section
3.2).
To maintain balance, we
use a specific form of Jacobian Transpose (JT) Control,
consisting of a set of
virtual forces
and
virtual torques.
•
Off-line parameter optimization
(Section
3.3)
To find the
right set of control parameters for a specific motion or
character, we perform off-line parameter optimization
based on high-level objectives, using Covariance Matrix
Adaption (CMA).
At any time
t,
we assume availability of the values of all
degrees of freedom and their first order derivatives, both for a
simulated character,
C,
and a reference motion,
A.
For a sim-
ulated character, DOFs and their derivatives are accessible
from the physics simulation engine. For a reference motion,
the first order derivatives can be acquired through low-pass
filtering and differentiation.
Our Work
Our control framework uses a combination
of Proportional-Differential Control for tracking, Jacobian
Transpose Control for balance, and Covariance Matrix
Adaption for off-line parameter optimization. Even though
each of these techniques has been used in several related
publications, they have not been used in this specific com-
bination, and not for the purpose of tracking unaltered refer-
ence motions.
In spirit, our work is most closely related to the motion
capture driven controllers of Zordan and Hodgins [ZH02],
the locomotion controllers of Coros et al. [CBvdP10], which
make extensive use of virtual forces, and to the work of Lee
et al. [LKL10], who perform impressive tracking of a wide
range of unaltered captured walking motions, using a bal-
ance correction algorithm based on kinematic properties.
The main difference between our work and that of Zor-
dan and Hodgins [ZH02] is that their controllers are lim-
ited to dual stance motions with upper-body dynamics, while
we demonstrate motions of both single and double stance,
with significant lower-body dynamics. The main difference
between our work and that of Coros et al. [CBvdP10] is
that in their framework, motion trajectories are constructed
through key poses, while our framework uses unaltered ref-
erence motion trajectories. An important difference between
our work and that of Lee et al. [LKL10] is that their method
requires inverse dynamics for torque computation. Inverse
dynamics is not supported by most common physics engines
and has additional computational requirements. In addition,
inverse dynamics control inefficiently fights the natural dy-
namics of a physics-based character [PCTD01], which may
lead to unnatural joint compliance.
Another key difference between our work and that of both
Coros et al. [CBvdP10] and Lee et al. [LKL10] is that their
balance strategies are largely based on swing foot placement
for balance – inspired by the SIMBICON balance strategy of
Yin et al. [YLvdP07]. This makes their control frameworks
suitable for locomotion, while the static balance algorithms
used in our framework make it more suitable for in-place
motions.
c The Eurographics Association 2012.
3.1. Tracking Control
To mimic the characteristics of a captured motion, our con-
trol framework uses Proportional-Derivative (PD) Control
for tracking reference trajectories of the individual DOFs.
PD Control attempts to minimize the displacement between
a reference joint angle,
θ
A
, and the corresponding joint angle
of the simulated character,
θ
C
, as well as the difference be-
˙
tween reference joint velocity,
θ
A
, and simulated joint veloc-
˙
ity,
θ
C
. The torque produced by PD Control,
τ
pd
, is linearly
proportional to these differences:
˙
˙
τ
pd
=
k
p
(θ
A
−
θ
C
) +
k
d
(
θ
A
−
θ
C
)
(1)
The responsiveness to deviations in position and veloc-
ity is controlled through gains k
p
and k
d
, which are deter-
mined individually for each actuated DOF through off-line
optimization (see Section
3.3).
3.2. Balance Control
Since PD control tracks only actuated degrees of freedom,
additional control is required to prevent errors in global po-
sition and orientation to accumulate, resulting in loss of bal-
ance (see also [SKL07]). Our balance control strategy is
based on the application of
virtual forces
and
virtual torques,
both of which are forms of
Jacobian Transpose Control.
T. Geijtenbeek & N. Pronost & A. F. van der Stappen / Simple Data-Driven Control for Simulated Bipeds
3.2.1. Jacobian Transpose Control
The essence of this control method is that it enables con-
trol in Cartesian space for linked structures with redundant
DOFs. Using the Jacobian Transpose, it is possible to com-
pute the set of torques that emulate the effect of an external
force or torque, applied to a specific body or a virtual point
(such as the center-of-mass). Virtual forces and torques are
applied to a
chain of linked bodies,
starting from a static base
link (such as the stance foot) and moving to a target link (see
Figure
2
for an illustration). The set of joint torques
τ
F
that
emulate a virtual force
F
applied at point
p
corresponds to:
τ
F
=
J(p)
T
F
(2)
Virtual
Force
Virtual
Force
Base
Link
Base
Link
Base
Link
where
J(p)
is the
Jacobian
that represents the rate of change
of a point
p
for each DOF
i
connecting the targeted chain of
bodies. For a chain of bodies connected through
k
rotational
DOFs, each row in
J(p)
T
represents the rate of change of
p
with rotation
α
i
about DOF
i:
∂p
x
∂p
y
∂p
z
∂α
1
Figure 2:
Virtual forces applied to the center of mass,
through a chain of linked bodies, for single stance (left) and
dual stance (right).
3.2.2. Stance State
The selection of the chain of bodies to which we apply our
virtual forces and torques depends on the
stance state
of the
character. A stance state
S
can be one of the following:
S
∈ {left_stance,
right_stance, dual_stance, flight}
(8)
J(p)
T
=
.
.
.
∂α
1
.
.
.
∂α
1
.
.
.
∂p
x
∂α
k
∂p
y
∂α
k
(3)
∂p
z
∂α
k
For a rotational DOF
i
represented by normalized axis
a
i
and anchor position
b
i
(all defined in the world coordinate
∂p
frame), the derivative
∂α
corresponds to the cross product
i
between
a
i
and the relative position of
p
[Jaz10]. Hence,
each row
i
of
J(p)
T
corresponds to:
J(p)
T
=
i
∂p
x
∂α
i
∂p
y
∂α
i
∂p
z
∂α
i
The stance state must be computed separately for sim-
ulated character and reference motion. For the simulated
character, the state can be derived from contact state in the
physics engine, for the reference motion we base the state on
the height of the ankle joint position.
3.2.3. Base of Support
In our framework, the base of support is represented by a
point
p
base
, which is based on the stance state and the pro-
jected of the ankle joint position(s):
⊥p
ankL
if
S
A
=
left_stance
⊥p
ankR
if
S
A
=
right_stance
p
base
=
(9)
⊥p
ankL
+⊥p
ankR
otherwise
2
where
⊥p
ankL
and
⊥p
ankR
are the projected positions of left
and right ankle joint, respectively, on the ground plane. To
allow comparison between the base position of the reference
motion,
p
base,A
, and the base position of the simulated char-
acter,
p
base,C
, both must be computed from using the same
state. We therefore always determine the base of support us-
ing the stance state of the reference motion,
S
A
, even if the
character state,
S
C
, is different.
3.2.4. Balance Strategy Components
Our balance control strategy consists of a combination of
virtual forces and torques, each targeting a different aspect
of balance:
•
A virtual force applied to the center-of-mass (COM) of
the character, to minimize differences in COM position
and velocity between the simulated character and the ref-
erence motion.
c The Eurographics Association 2012.
= (a
i
×
(p
−
b
i
))
T
(4)
A
virtual force F
applied at point
p
can now be emulated
by applying a torque
τ
F,i
to each DOF
i
that is part of the
chain of bodies:
τ
F,i
= (a
i
×
(p
−
b
i
))
T
F
(5)
Similarly, it is possible to apply a virtual torque to a spe-
cific body at the end of a chain. If the orientation of the tar-
geted body is represented using an
exponential map
[Gra98],
e
∈
R
3
, then the rate of change of
e
with rotation
α
i
is iden-
tical to the normalized DOF axis
a
i
:
J(e)
T
=
i
∂e
x
∂α
i
∂e
y
∂α
i
∂e
z
∂α
i
=
a
T
i
(6)
Hence, a
virtual torque T
applied to the body at the end of
the chain can be emulated by applying a torque
τ
T,i
to each
DOF
i
that is part of the chain of bodies:
τ
T,i
=
a
T
T
i
(7)
Note that, since bipeds contain no links that are truly
static, the effect of a virtual force or torque is always an ap-
proximation and can be determined only after simulation.
That said, we have found this approximation to be suffi-
ciently accurate to work well in practice.
T. Geijtenbeek & N. Pronost & A. F. van der Stappen / Simple Data-Driven Control for Simulated Bipeds
•
A virtual torque applied to the pelvis, to maintain the
upper-body posture of the simulated character.
•
A virtual torque applied to the pelvis, to regulate the total
angular momentum of the character.
Each of these components will be described in detail in
the upcoming sections.
Center-of-Mass Position and Velocity
The first compo-
nent of our balance control strategy compensates for differ-
ences in COM position and velocity. It does so by applying
a virtual force at the COM position of the simulated char-
acter, to the chain of bodies from stance foot to pelvis (see
Figure
2).
For a character consisting of
k
bodies, COM posi-
tion
p
com
and COM velocity
v
com
are a weighted average of
individual body positions,
p
body,i
, and velocities
v
body,i
:
m
p
com
=
∑
i
p
body,i
i=1
m
tot
k
corresponding set of individual joint torques,
τ
trunk
, which
are applied to each DOF in
stance ankle, stance knee
and
stance hip
can be acquired through Equation (7).
Angular Momentum
Regulation of the angular momentum
(AM) is an important aspect of biped balance control (see
also [MZS09]). For a character consisting of
k
bodies, the
angular momentum,
L,
corresponds to:
k
L
=
i=1
∑
m
i
(p
body,i
−
p
com
)
×
v
body,i
+
R
world,i
I
i
ω
i
(14)
,
v
com
=
i=1
i
∑
m
tot
v
body,i
k
m
(10)
in which m
i
is the mass of body
i, p
body,i
its position,
v
body,i
its linear velocity, and
ω
i
its angular velocity. I
i
is the 3
×
3
inertia tensor
matrix, describing the mass distribution of
body
i,
while R
world,i
is a rotational matrix from local frame
of body
i
to the world coordinate frame. Mass properties are
derived from body geometry, both for the simulated charac-
ter and the reference motion.
To minimize difference in angular momentum between a
reference motion,
L
A
, and simulation,
L
C
, we apply a virtual
torque
T
am
to the chain of bodies from stance foot to pelvis:
T
am
=
w
am
(L
A
−
L
C
)
(15)
in which m
i
is the mass of body
i
and m
tot
is the total mass.
Mass properties are derived from body geometry, both for
the simulated character and the reference motion.
We attempt to control balance by regarding the COM po-
sition of a character with respect to its base of support:
p
com
=
p
com
−
p
base
ˆ
(11)
The virtual force that minimizes the difference between
relative COM position of the reference motion,
p
com,A
, and
ˆ
simulated character,
p
com,C
, as well as the difference in
ˆ
COM velocity between reference motion,
v
com,A
, and sim-
ulated character,
v
com,C
, now becomes:
F
com
=
w
cp
(
p
com,A
−
p
com,C
) +
w
cv
(v
com,A
−
v
com,C
)
(12)
ˆ
ˆ
in which w
cp
and w
cv
are constants controlling force magni-
tude. These constants are determined through off-line opti-
mization (see Section
3.3).
Different parameter sets are used
depending on the stance state of the character (single stance
or dual stance). The corresponding set of individual joint
torques,
τ
com
, which are applied to each DOF in
stance an-
kle, stance knee
and
stance hip
can be acquired using Equa-
tion (5).
Trunk Orientation
To maintain the posture of the upper
body, we attempt to minimize the difference in trunk orien-
tation. We do so by applying a virtual torque to the chain of
bodies from stance foot to the pelvis (see Figure
2).
If
q
A
is a quaternion describing the trunk orientation of the refer-
ence motion, and
q
C
that of the simulated character, then the
virtual torque
T
trunk
corresponds to:
T
trunk,i
=
w
to
expmap(q
−1
trunk,C
q
trunk,A
)
(13)
in which w
am
is a constant controlling magnitude, deter-
mined through off-line optimization (see Section
3.3).
The
corresponding set of individual joint torques,
τ
am
, which are
applied to each DOF in
stance ankle, stance knee
and
stance
hip
can be acquired through Equation (7).
3.2.5. Combining the Individual Components
After all joint torques are computed using the tracking con-
trol and balance control algorithms, they can be added to-
gether to a single torque vector:
τ
control
=
τ
pd
+
τ
com
+
τ
trunk
+
τ
am
(16)
Following the work of Wang et al. [WFH10], we add
mo-
tor noise
to our control output, in the form of random torque
perturbations,
τ
noise
. The application of motor noise can pro-
vide robustness against unmodeled phenomena, including
numerical errors during simulation. We use a simplified mo-
tor noise model, which consists of individual values ranging
from -5 to 5 Nm, randomly sampled each frame from a uni-
form distribution. Unlike the model of Wang et al., our noise
does not increase with higher torques. However, our noise
range is in the same order of magnitude as theirs for average
torque levels (τ
≈
20Nm).
For each DOF
i,
we also enforce a maximum torque value
τ
max,i
, resulting in the following final torque
τ
i
:
if
τ
control,i
>
τ
max,i
τ
max,i
+
τ
noise,i
−τ
max,i
+
τ
noise,i
if
τ
control,i
<
−τ
max,i
τ
i
=
(17)
τ
control,i
+
τ
noise,i
otherwise
where
expmap
:
R
4
⇒
R
3
is a function that extracts an
ex-
ponential map
from a quaternion (see [Gra98]), and w
to
is
a constant controlling the magnitude of the torque, deter-
mined through off-line optimization (see Section
3.3).
The
c The Eurographics Association 2012.

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika:

Zgłoś jeśli naruszono regulamin