Omni-directional Vision and 3D Animation Based Teleoperation of Hydraulically Actuated Hexapod Robot COMET-IV H. Ohroku and K. Nonami Graduate School of Science and Technology, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba-shi, Chiba, 263-8522, Japan

1. Omni-directional Vision and 3D Robot Animation Based Teleoperation of Hydraulically Actuated Hexpod Robot COMET-IV Hiroshi OHROKU Control and Robotics Lab. CHIBA University

5. Large-scaled legged Robots (Past) Walking Forest Machine MECANT ASV TITAN-XI

6. About autonomous system Fully autonomous system Half autonomous system ＜ Intelligent system ＞ - Task management & Planning - Environment recognition - Self-Localization Upper- layer system ＜ Robust controller ＞ - Force & Position control Lower- layer system ＜ Robust controller ＞ - Force & Position control Lower- layer system - Image of real environment - Sensor data Teleoperation system - Operation - Supervision It is very difficult to implement fully autonomous robot in the stricken area ・・・

7. Tele-robotics researches(1) http://pc.watch.impress.co.jp/docs/2002/1219/hrp.htm Master-slave system + 3D Animation + Open Dynamic Engine （ ODE ） http://const.tokyu.com/topics/1998/topics_03.html Master-slave system + ＦＥＳ （ Functional Electrical Stimulation ） based bilateral control

8. Tele-robotics researches(2) K.Saitoh, T.Machida, K.Kiyokawa, H.Takemura : A Mobile Robot Control Interface Using Omnidirectional Images and 3D Geometric Models, Technical report of IEICE. Multimedia and virtual environment 105-256, 7/12(2005)

9. Focus of my research The research on teleoperation system to assist operator for controlled large-scale legged type robot is little In rough terrain operation, the change of ・・・・ Body height Attitude Leg movement The teleoperation assistant system is indispensable for controlling legged robot in outdoor environment Development of the Teleoperation assistant system ■ Omni-directional vision ■ 3D Animation of robot movement

11. Hardware Specifications Leg 1 Leg 2 Leg 3 Leg 4 Leg 5 Leg 6 L×W×H 2.5×3.3×2.8 （ m ） Weight 2200 （ kg ） Power Source (Max. Output) Gasoline Engine ×2 (25.0ps/3600rpm) Supply Pressure 22 （ Mpa ） Supply Flow 78 × 2 （ L/min ） W:3.3 m H:2.8 m L:2.5 m

12. Hardware Specifications LRF （ Laser Range Finder ） Stereo vision Omni-directional Vision sensor

13. Coordinate System of Leg L2 L3 L4 L1 Link Length (m) Range of Motion [deg] Shoulder L1 0 -180° ≦ θ 1 ≦ 180° Thigh L2 1.13 50° ≦ θ 2 ≦ 142° Shank L3 0.77 36.6° ≦ θ 3 ≦ 150° Foot L4 0.39 -47.9°≦ θ 4 ≦ 103°

14. Basic configuration of hydraulic control system

16. Omni-directional gait Circular gait Rotation center Crab gait Crab gait Circular gait Circular gait includes all gait [*]

17. Definition of coordinate system X c Y c X s_1 Y s_1 <FRONT> <REAR> <LEFT> <RIGHT> Leg1 Leg2 Leg3 Leg4 Leg5 Leg6 O s_1 , Z s_1 O c , Z c X t Y t O t Rotation center coordinate system Body coordinate system Shoulder coordinate system R ct θ ct ： Crab angle

18. Condition of circular angle setting Condition: Arbitrary crab angle walking is achieved by setting as follows. -> ∞ Circular gait ≒ Crab gait

19. Teleoperation assistant system Instruction information Log Omni-directional image Generated image Received sensor data Map 3D Animation of robot

20. Peripheral Device ■ Omni-directional vision sensor - Digital Video Camera ： DCR-HC48 （ SONY ） - Hyperbolic Mirror ■ Joystick: Cyborg Evo Force （ Saitek ） Interface USB1.1 Size （ mm ） 210 × 199 × 240 Buttons 11 Axes 3 Diameter of mirror 82 mm Angle of elevation 15° Angle of depression 50°

21. Instruction information Action value shows the basic movement action. ■ Stand up ■ Sit down ■ Walking start ■ Walking stop Teleoperation Computer Locomotion Computer UDP Socket （ User Datagram Protocol ） Action Action value Cycle_Time Cycle time θ tb Traverse angle of the body center at one cycle [O t ] R ct Position of rotation center [polar display of O c ] θ ct

22. Coordinate system of Joystick The eight directions of basic locomotion angle β can be selected using the joystick.

23. Gait parameters Setting The range of the traverse angle ＜ Crab gait ＞ ＜ Circular gait ＞ -> ∞ is set to the minimal value The circular angle of body center derived from rot_z which is set to

24. Configuration of omni-directional vision sensor a 26.2 [mm] b 35.4 [mm] c 44.1 [mm] Diameter of mirror 55.0 [mm] Angle of elevation 15.0 [deg] Angle of depression 50.0 [deg]

25. Performance of Ambient Environmental Image(1) Central axis of a mirror fits an optical axis of a camera is assumed. The direction where the angle of pan and tilt is set to 0 degree to make X axis and the origin of projection center coordination is (0, 0). （ 7 ） The spherical coordinates was derived by using pan, tilt angle and focal length which is direction projection plane as against the sphere. （ 8 ） （ 9 ） （ 10 ）

26. Performance of Ambient Environmental Image(2) The coordinates that derived from Eq. (7) to Eq. (10) is transformed to rectangular coordinates. （ 11 ） Mirror parameter is in Semi-major axis and mirror parameter b is in Semi-minor axis. Semi-latus rectum is derived by Eq. (12). （ 12 ） （ 13 ）

27. Performance of Ambient Environmental Image(3) Parameters l and m is used in spherical projection was derived from Eq. (14) to Eq. (15) including. （ 14 ） （ 15 ） （ 16 ） On the other hand, projection coordinates of hyperboloidal is calculated as Eq. (16) and (17). （ 17 ）

28. Performance of Ambient Environmental Image(4) Finally, the pixel coordinate system is determined by using matrix K via Eq. (18). Furthermore projection coordinates is calculated by Eq. (19). The values (358.1, 215.1) in the matrix K are XY coordinates of the center point in omni-directional image. （ 18 ） （ 19 ）

29. Texture mapping Time consuming factor on the system processing unit still become an issues because previous mentioned calculation method does similar process to all image pixels. Consequentially, it is difficult to present the smooth image operator. Therefore, the load of the calculation processing is reduced by using texture mapping .

30. Texture mapping Fig.11-(a) Fig.12 Fig.11-(b) The corresponding points in omni-directional image as against lattice points were derived from Eq1 to Eq3. ② ① Texture mapping vertex[0] = P[0] texcoor[0] = T[0] vertex[1] = P[1] texcoor[1] = T[1] vertex[2] = P[2] texcoor[2] = T[2] vertex[3] = P[3] texcoor[3] = T[3] ③

31. Mapping image Fig.12 Calculated points on texture coordinate system

32. Robot animation using 3D geometric models and sensor data The 3D COMET-IV online model is designed to predict the real-time movement of COMET-IV on the reality environment. Each sensor data is used for the robot 3D animation movement reference which is transmitted from target computer unit on COMET-IV to teleoperation computer via wireless serial MODEM with data transfer rate 57600bps at 10Hz. Locomotion Computer Wireless serial Modem (for sensor data) Wireless serial Modem (for sensor data) 2.4GHz

33. Coordinate system of 3D robot Table.6 and Fig. 13 shows the parameters used for robot 3D animation in virtual environment and coordinate system of 3D robot respectively.

34. Initialization of 3D Animation For initialization, received azimuth angle, XY value of GPS, and roll and pitch angle of the body are set up as default values. The robot is rotated using azimuth angle ψ including magnetic variation Δd with Y axis.

35. The homogeneous transformation [1] Each leg (Foot - Shank - Thigh - Shoulder) After initialization, each part is expressed as follows. （ 20 ） （ 21 ） （ 22 ） （ 23 ） [2] Robot body （ 24 ） （ 25 ）

36. Obstacle avoidance walking test Operator Goal Obstacle(2) Obstacle(1) ■ Controller is PID position control ■ Cycle time 16 [s] ■ Omni-directional gait （ crab gait ） ■ Tripod walking Robot Setting: ■ The number of lattices was set as 16×12 ■ virtual obstacle object on the screen System Setting:

37. Experimental Results Fig.15 Walking trajectory acquired by GPS Fig.16 Foot trajectory of walking test (Leg1)

38. Experimental Photos and Images(1) Fig.17 Photos of the obstacle avoidance walking test Fig.18 Images of 3D animation

39. Experimental Photos and Images(2) (a)’ (a) (b)’ (b) (c)’ (c) Fig.19 Omni-directional images and generated images