tutorial

CUSTOM MOTION CAPTURE DATA FOR VIRTUAL REALITY SIMULATION
Pipeline for working from Qualisys to Vizard
Virtual Environment Navigation Lab, Brown University
Joanna Lin
2015

Lin 2
Table of Contents
OBJECTIVE:..................................................................................................................................... 3
INTRODUCTION: .............................................................................................................................. 3
TOOLS NEEDED............................................................................................................................ 4
FILES NEEDED.............................................................................................................................. 4
FILES INCLUDED .......................................................................................................................... 5
(a)MODELING IN MAYA + HUMAN-IK RIG .......................................................................................... 6
(b)CAPTURING MOCAP TIPS/QUALISIS TIPS.................................................................................. 11
(c)APPLYING MOCAP TO AVATARS................................................................................................ 12
(d)MODIFYING ANIMATION INTO CYCLES IN MAYA ........................................................................ 21
CHECK BEFORE NEXT STEP ....................................................................................................... 26
(e)EXPORTING ANIMATED CYCLES THROUGH 3DS MAX TO CAL ................................................... 27
(f)CHECKING AVATARS IN VIZARD ................................................................................................ 34

Lin 3
OBJECTIVE:
With the accompanying tutorial files or with one’s own raw motion capture data, one can follow
along through the steps in this document and prepare mocap data for use upon avatars within
simulations in virtual reality.
INTRODUCTION:
This paper presents documentation and instruction on how to capture custom motion capture
data in the Virtual Environment Navigation Lab and then get the data onto avatars for use in
Vizard for the purpose of custom VR simulations.
The method described assumes that one knows general 3D skills such as modeling, rigging,
and keyframe animation, and that one already has a 3D model of an avatar prepared for
applying the data to.
The pipeline starts in Maya with an overview of how to prepare a polygonal model with a
HumanIK rig. *If you are unfamiliar with modeling/character rigging, look online for maya
modeling tutorials (keywords: polygonal modeling, edge loop flow, character modeling,
character skinning)
As the rigging is happening, something that be occurring simultaneously is the data capture
itself on the Qualisys system. There are some notes on how to optimize the capturing
experience.
After these initial steps, the tutorial describes how to clean up the data in MotionBuilder and
get the data applied to the 3D polygonal model. Then that model with the applied data can be
revisited in Maya for some keyframe hygiene in order to get the animation into looping cycles.
The final steps involve taking the files into 3DS Max because 3DS Max has an exporter that
feeds directly into Vizard.
This whole process is full of nuances and steps that all need to be paid attention to. There are
many ways to get motion capture data applied to custom characters and into virtual reality;
depending on the final input and output and software familiarity, there are ways to do this with
more and or less software.

Lin 4
TOOLS NEEDED
You will need the following programs to complete the tutorial:
1) QUALISYS
Installed on Student RA computer in the VENLab control room as well as the Qualisys
PC in the VENLab
2) MAYA 2014+:
Download of student version available here: http://www.autodesk.com/
3) MOTIONBUILDER:
4) 3DS MAX 2014:
5) CAL3D exporter:
Download available here: http://www.worldviz.com/support/download
6) VIZARD 4.0 or 5.0:
Full versions are installed in the VENLab and on ROSE in the VENLab control room. Free
download of a demo version (this is fine for testing purposes) available here:
http://www.worldviz.com/products/vizard
FILES NEEDED/GENERAL WORKFLOW OUTLINE
1) QUALISYS FILE
The raw data/recording from Qualisys needs to exported into the c3d format.
a. Export to .c3d
2) MAYA FILE
An custom model of an avatar needs to be modeled, shaded, and skinned to a HumanIK rig and
then exported as .fbx
a. humanIK rig saved as .fbx
b. UV mapped texture saved as .jpg
3) MOTIONBUILDER file
With the .c3d file from Qualisys, one brings this .c3d into MotionBuilder and combines it to the
.fbx from Maya and reexports it as a new .fbx
a. Import .c3d
b. Export (applied opticals on actor) .fbx
c. Export (applied actor to rig (retargeted)) .fbx
d. Export (baked retargeted to rig) .fbx
4) Open in 3DS MAX
From # 3, that file goes to 3DS Max in order to be processed through cal3D exporter
a. Import baked retargeted .fbx
b. Export skeleton.csf
c. Export mesh.cmf
d. Export animation.caf
e. Export material.xrf
5) VIZARD FILE:
a. With the files exported from #4, one makes a textfile in notepad saved as a .cfg with
code describing what each exported file isCreate text.cfg

Lin 5
FILES INCLUDED
1) Resources_modeling-skinning
a. Lynda tutorial on character modeling
b. How to use graph editor, Maya
2) Start_Files
a. polyModel_humanIK.mb
i. this is a file with a rigged and skinned character
b. polyModel_noRig.mb
i. this is a file of just a polygonal model, one can use this file to practice adding a
humanIK rig and skinning it
c. example_texture.jpg
i. this is a texture that can be applied to the character
d. example_texture.psd
i. this is the working file that painted the texture for the character, you don’t need
to do anything to this file unless you want a try and make a different texture
e. example_UV.png
i. this is a UV snapshot from Maya’s UV Editor of the character
f. rawC3D
i. this folder has some example .c3d files that one can use in MotionBuilder

Lin 6
(a)MODELING IN MAYA + HUMAN-IK RIG
This section provides an overview of how to prepare a polygonal model with a HumanIK rig, please
see the Lynda tutorial if you are unfamiliar with 3D character modeling and skinning. If you are trying
to achieve photographic likeness, this can be difficult for a beginner and may take a long time.
a1_Model the avatar polygonally, keep the model all 1 piece (i.e., do not have separate spheres as
eyeballs, do not have separate shoes, keep the model all 1 flush piece of geometry, this will save you
trouble in the future); Figure1
Figure1 Screenshot of polygonal model
a2_Add skeleton with HumanIK under animation subset; Figure 2
Figure 2 Go to animation submenu, Skeleton, HumanIK

Lin 7
a3_Create skeleton; Figure 3 Screenshot of HumanIK GUI
Figure 3 Screenshot of HumanIK GUI
a4_Resize to bones to match the model, make sure under the definition tab, the circle stays green, if
you do something it dislikes, it’ll turn orange triangle and you won’t be able to bind skin; Figure 4
Screenshot of green circle
Figure 4 Screenshot of green circle
Figure 5 Skeleton customized to match polygonal model

Lin 8
a5_Once its matched, save the edits,Figure 6
Figure 6, Screenshot of ‘save edits’
a6_Then lock definition, Figure 7
Figure 7 Screenshot of Lock Defintion
a7_Bind skin, paint skin weights (see skinning tutorial), Figure 8

Lin 9
Figure 8 Screenshot of Smooth Bind
a8_Once all skinned save rig as FBX, export all, Figure 9, Figure 10
Figure 9 Screenshot of Export All

Lin 10
Figure 10 Screenshot of saving as .FBX

Lin 11
(b)CAPTURING MOCAP TIPS/QUALISIS TIPS
This section has tips and important things to keep in mind during custom mocap capture
Capture
 Maybe add extra optical markers to tip of femur in case some get occluded
 Actor should be wary when putting hand by side, that top of femur or top of iliac crest doesn’t
get occluded
Qualysis (Clean up data)
 Name the opticals in a way that is helpful to you such as, Figure 11:
Figure 11,Screenshot of an example naming convention
 Not necessary to delete the unnecessary opticals
 Connect opticals upon actors body by connecting them to one another with bones shortcut C
or B I don’t remember
 Be wary of when they disappear
 export as c3d
 import c3d into mobu

Lin 12
(c)APPLYING MOCAP TO AVATARS
This section includes details on how to apply custom motion capture data to custom avatars. There
are some other resources that explain parts of my process in more detail
written blog resource for mocap data application:
http://mocappys.com/how-to-map-optical-data-using-motionbuilder-actor/
Playlist of step 1
http://www.youtube.com/watch?v=SQa_Y-vx-mE&list=PL579D27F868BBFE34
Video of important check box to check 2
http://www.youtube.com/watch?v=3Fvd9-sqx4Y
Video of step 3, cleaning up data
http://eat3d.com/free/mocap_cleanup
c1_import marker data, import the .c3d into a new MotionBuilder scene. Ideally this file will contain
the avatar standing in a t-pose with all the markers identified and any errors corrected. Figure 12
Figure 12 Screenshot of Motion File Import
c2_ in the options box, tick create optical segments and un-tick create unused optical segments as
the unused optical segments aren’t required and will only confuse your scene. Figure 13, Figure 14
Figure 13 Screenshot of Import options

Lin 13
Figure 14 Anothr screenshot of Import Options
c3_Create actor, Align the actor and markers, scale actor as necessary, with the keyboard shortcuts
(w)translate, (e)rotate, (r)scale, Figure 15, Figure 16
Figure 15 Screenshot of Create Actor

Lin 14
Figure 16 Screenshot of translate, rotate, scale buttons

Lin 15
c4_Create marker set, and blue circles appear at the body joints, Figure 17
Actor settings>marker set>create
Figure 17 Screenshot of actor settings
c5_Drag opticals to corresponding segments, twirl open the menu and, hopefully you named the
opticals appropriately in Qualisis, and select the ones you want and drag and hover over the
corresponding blue circle, Figure 18
Figure 18 Screenshot of Opitcals drop down menu

Lin 16
c6_ Click active and lock, the actor will now walk and be influenced by mapping, Figure 19
Figure 19 Screenshot of Character and Source settings
c7_Save as avatar_actionActor.fbx
where the avatar is name of your character (person1, Sara, Jason etc), the action is the performance
(walking, running, idle etc)
c8_Then merge fbx rigged model Figure 20
File>merge (uncheck all)
Figure 20 Screenshot of file>Merge

Lin 17
c9_Change from actor to character 1
change applied motion to actor Figure 21
Figure 21 Screenshot of source
c10_Rig will snap to actor, and if you press play, the animation will likely look wonky and distorted.
c11_If this is the case, unlock actor, Figure 22
Figure 22 Screenshot of lock checkbox
Adjust some limbs as necessary, be careful of undoing actions, it will corrupt the whole animation.
Either do not undo, just readjust, or only undo when you’re at the frame at which you made the
adjustment
I.E. shoulders, move up, to fill out shoulders, Figure 23

Lin 18
Figure 23, Left side, orginal shoulder, looking collased, Right side, lifted shoulder after Y-translated
Relock
c12_Under right hand side HUMANIK menu set
Bake to skeleton, Figure 24
Figure 24 Cleck the blue gradient square to find this menu
When its baked, then the character source will be none instead of ‘actor’, Figure 25
Figure 25 Screenshot of Source: None

Lin 19
c13_After its baked….then resave as avatarAction_retargeted.fbx Figure 26
Figure 26 Screenshot of Save File
c14_Open the original rigged model (no animation)

Lin 20
c15_Import motion data by going to file, motion file import (like when you imported .c3d files earlier)
and select avatarAction_retargeted.fbx, Figure 27
Figure 27 Screenshot of Motion File Import
Make sure to unselect all check boxes like in the image below Figure 28
Figure 28 Screenshot of merge settings
c16_Save as avatarAction.fbx

Lin 21
(d)MODIFYING ANIMATION INTO CYCLES IN MAYA
This section is about how to modify an animation into a looping cycle. The tutorial starts in Main Maya,
but then the remaining instruction will be focusing on the GRAPH EDITOR. See graph editor tutorial in
resources The goal of this section is to take all your avatars and make them ready consistent looping
in Vizard
d1_either open the file you just saved or Send to Maya Figure 29
Figure 29 Screenshot of Send to Maya
d2_Pull up graph editor, Figure 30
Figure 30 Screenshot of Graph editor
d3_Pull up the outlinerFigure 31
Figure 31 Screenshot of outliner
In the outliner, select the hips bone which represents the whole skeleton hierarchy, Figure 32

Lin 22
Figure 32 Screenshot of outliner, hips
d4_In the graph editor window, shift+click the root bone to twirl open all the bones, Figure 33
Figure 33 Screenshot of twirled open bones
Shift select all the bones in the sidelines of the graph editor, you can press (f) to frame all selection
Edit the keyframes in graph editor so that you have either a repeating cycle, or make your animation
stay in place, for actions such as walking or running. Also, delete the unnecessary frames, such as
ones that have the t pose that the human performed during data capture

Lin 23
d5_If loop for walk or run, delete translate x and z for hips to have motion in place, Figure 34
Figure 34 Screenshot of x and z translate, drag select with selection tool to highlight all, and delete
Find the loop duration of the walk to the run, walks are approximately 24 frames, runs are
approximately between 14-18 frames, delete all keys that are not the perfect cycle that you need
Shift select all bone nodes and click (f) to frame all
Once selected, shift all to 0/Start of the timeline by (t) translating, make sure you don’t move up and
down, only left
d6_If you need a loop for another action, not walk or run, you need to duplicate all the keyframes on
all of the channels of the first keyframe of the action TO THE last frame on the decided duration of the
cycle Figure 35
Figure 35 Selecting all the channels at Frame 1

Lin 24
In the example below, say you have animation from frames 1-40, if you need this to loop, pick frame
70 for example, and paste all the keys from all the channels to frame 70, then center the remaining
animation so you wont see any jumpiness. Mocap creates a keyframe at every single frame, but maya
is good at interpolating in between. Figure 36
Figure 36 Example of a non-walk loop, the first and last frame are the same
you have shift+clicked the hip bone in the graph editor to twirl open all the bone nodes then shift
select all the bones in the list, drag select all the keyframes at frame 1, and go edit>copy Figure 37
Figure 37 Shift click where the cursor hovers in the Graph Editor
you will want to paste the keys from frame one maybe 30 or so frames after the last frame, so there’s
time for the frame to slowly reset to the first frame in a way that’s not jumpy
then before you paste, do paste special use help images, Figure 38, Figure 39
Figure 38 Clicking the checkbox next to the edit menu takes you to paste Special

Lin 25
Figure 39 Screenshot of past Speical with help images
d7_Line up the model to be in the center of x and z (at origin) and have the model facing frontal Z, an
orientation as similar as possible to when the avatar was modeled in t-pose
This will help you when you’re in vizard, so that all the motions are facing the same way upon first
import
d8_Once keyframes are edited in Maya,
apply texture lambert (no texture), this is important for exporting cal3d
Save the file, treat as if retargeted like in mobu after baking
Repeat steps c14-c16
Back in mobu, load rig
Then merge motion cycle with rig
then resave avatarAction_cycle1-X.fbx
where X is the last frame of the cycle

Lin 26
CHECK BEFORE NEXT STEP
*BEFORE you go on to the next section you should have:
AvatarAction_actor (this is where we bound the opticals to a dummy body)
AvatarAction_retargeted (this is the step where the animation is baked to skeleton)
AvatarAction_1 (this is the step where the retargeted animation is merged to the rig)
AvatarAction_cycle1-X_1 (this is where the animation data has been edited to be looping in Maya)
AvatarAction_cycle1-X_2 (this is the step where the edited looping animation is reMerged with the rig
in MoBu)
texture.jpg
**The text in red represent the files you will need to move on to the final step

Lin 27
(e)EXPORTING ANIMATED CYCLES THROUGH 3DS MAX TO CAL
This section is about the final steps of the process, exporting the work through 3DS Max for Vizard.
It’s a good to make a folder to catch everything, also add the texture.jpg file to this destination. Onc
everything from this section is exported, one must make a notepad file and have code that will
integrate all the separate exports into one file.The official page about this step is at the following web
link:
http://docs.worldviz.com/vizard/#cal3d_introduction.htm%3FTocPath%3DContent%20Creation%7C3D%20Models%7CCal3D%2
0Workflow%7C_____1
e1_ Open file in 3DS Max
change/check the time configuration, Figure 40
Figure 40 Screenshot of time configuration button
Change the frame rate to 24 (maya is in 24 by default) and the animation end time to reflect how much
animation is in the mocap file, Figure 41
Figure 41 Screenshot of time configuration window

Lin 28
e2_Click the material editorFigure 42
Figure 42 Screenshot of where to find material editor
Click the eyedropper, click on the mesh in the main window to pick up that texture(lambert) into the
material slot
Change the name to avatar[0]
Figure 43 Screenshot of material editor

Lin 29
e3_Under the blinn basic parameters, next to diffuse, click the empty square, the map browser pops
up. Select bitmap, then map attributes com up, Figure 43, Figure 44
Click none, select the example jpeg, or your own texture for your custom model, make sure you copy
the jpg to the folder you intend to export all your cal3D assets to, Figure 45
Figure 45 Screenshot of Bitmap
Figure 44 Screenshot of Material Browser

Lin 30
Make sure you click SHOW SHADED MATERIAL IN VIEWPORT, Figure 46
Figure 46 Screenshot of Show Shaded...
e4_Select helpers, Figure 47
Figure 47 Screenshot of Helpers selection
Drag select all from any window, it wont look like anything is selected
e5_Click the top left icon, click export, export, Figure 48
Figure 48 Screenshot of 3DS Max export

Lin 31
e6_Export skeleton file, Figure 49
Figure 49 Screenshot of skeleton export type
e7_Export animation file, Figure 50
Figure 50 Screenshot of file type Cal3D for animation

Lin 32
e8_in the same place where you selected helpers, select Geometry, drag select all from any window,
Figure 51
Figure 51 Screenshot of helpers, select Geometry this time
e9_Export mesh, Figure 52
Figure 52 Screenshot of Cal3D mesh file export

Lin 33
e10_Export material.XRF *if export .crf, vizard will not display the texture, Figure 53
Figure 53 Screenshot of Material file, save as xrf
Sara_idle1 = actor with opticals plus rig, actor unlocked
Sara_idle2RT= baked to skeleton (retargeted)
Sara_idle3=motion imported (ready for 3dsmax)

Lin 34
(f)CHECKING AVATARS IN VIZARD
f1_make the cfg file in notepad, save it in the folder that has all the exported parts (skeleton, mesh,
animation, material). All the names should correspond with what you exported from 3DS Max, looks a
little something like this:
f2_In vizard make sure effects are turned OFF!
################################################
#
# Cal3d cfg File
#
################################################
scale = 0.01
flip_texture = 1
################# Skeleton #################
skeleton = avatar_skel.CSF
################# Meshes #################
mesh = avatar _mesh.CMF
################# Animations #################
########### A0 ###########
#1
animation = avatar _CycleIdle.CAF
animation = avatar _CycleWalk.CAF
animation = avatar _CycleRun.CAF
################# Materials #################
material = avatar _mat.XRF

Lin 35
f3_drag in the CFG file and hopefully everything works. If not contact support@worldviz.com

tutorial

Recomendados

Recomendados

Más contenido relacionado

Similar a tutorial

Similar a tutorial (20)

tutorial