Nicolas Mansard
Email: Nicolas.Mansard@laas.fr |
Short bio
Since October 2008, I am working as a permanent researcher at LAAS/CNRS, Toulouse. I am involved in the Gepetto research group, with Philippe Souères, Florent Lamiraux, Olivier Stasse and Jean-Paul Laumond. I defended by Habilitation à Diriger des Recherches in Jully 2013.
In 2013, I was an invited researcher at Emo Todorov's lab, University of Washington (Seattle), USA.
I received the 2015 CNRS Bronze Medal for my research in robotics.
My research activities are concerned with sensor-based control, and more specifically the integration of sensor-based schemes into humanoid robot applications. It is an exciting research topic at the intersection of the fields of robotics, automatic control, signal processing and numerical mathematics. My main application field is currently the Humanoid robotics, as it causes serious challenges that are representative of many other robot domains.
Before being in the Gepetto group, I was a post-doc in ISRI Group, AIST Tsukuba, and a post-doc at the IA Lab of Stanford University, in the group led by Prof.O.Khatib. I have done my PhD. with Francois Chaumette in the Lagadic group, at INRIA Rennes. I was a student of the École Normale Supérieure de Bretagne and did both my MSc (Robotics) and BEng (CS) in the INP Grenoble ENSIMAG.
Projects
An overview of my research project is available below (in French only, sorry).
Hierarchical problems
The pseudo-inverse, used in most of the task-function-based works, gives the solution of an implicit least-square quadratic program. When considering only one task, it is then straight-forward to take into account some inequality constraints, such as an obstacle or joint position and velocity limits. The extension to a hierarchy of tasks is not immediate and was proposed by Kanoun in 2009, by using a set of quadratic programs in cascade. This formally defines the hierarchical quadratic problem and gives a naive resolution scheme that is very slow. This paper proposed to find an efficient resolution scheme to the cascade of quadratic problems.
The study first focuses on equality-only quadratic program and proposes an efficient resolution method based on a dedicated extension of the matrix complete decomposition classically used to compute the pseudo inverse. The method is five times faster than the classical concurrent solution to solve a hierarchical quadratic program. An active search dedicated to the hierarchical structure is then proposed to take into account a hierarchy of both equality and inequality. The solution avoids the problem of multiple activation and deactivation generally encountered with cascade of solvers. It is also much faster and straightforward to adapt to real-time constraints. Incidental results are also given to prove the complexity, the continuity and the stability.
This work is performed in collaboration with P-B. Wieber and A. Escande. It is sponsored by the project FP7 Robohow. See the page about the HQP paper. A MATLAB solver is available here (clean code with a lot of documentation, but the algorithm may not be very robust). A more efficient but less clean C++ is available on the LAAS github.
- A Dedicated Quadradic Program for Fast Hierarchical-Inverse-Kinematic Resolution. A. Escande, N. Mansard and P-B. Wieber. In IEEE Int. Conf. on Robotics and Automation (ICRA'10), Anchorage, USA, May 2010.
- Hierarchical quadratic programming: Fast online humanoid-robot motion generation. Adrien Escande, Nicolas Mansard and Pierre-Brice Wieber. {The International Journal of Robotics Research}, 33(7):pp. 1006-1028, June 2014.
- Prioritized Optimal Control : a Hierarchical Differential Dynamic Programming approach. Francesco Romano, Andrea Del Prete, Nicolas Mansard and Francesco Nori. In IEEE Int. Conf. on Robotics and Automation (ICRA'15), Seattle, USA, May 2015 .
Inverse Dynamics
Task-based motion generation enables fast development of whole-body movements. When accounting only for the geometry of the system, the movements are very rough. This is especially true when generating human-plausible motions. The inverse-dynamics solver was developed to generate dynamically-consistent movement from a hierarchy of tasks.
The motion generator is based on our HQP solver. The variables are the contact forces and joint torques and accelerations. These variables must fulll the dynamics equation and satisfy the given contact model (only rigid contact was tested yet). When all the contact points are included in a horizontal plane, the contact constraint is equivalent to the classical ZMP constraint. The tasks are nally introduced in the solver by imposing the acceleration in the operational space to be equal to a specied value.
- A Unified Approach to Integrate Unilateral Constraints in the Stack of Tasks. N. Mansard, O. Khatib and A. Kheddar. IEEE Transaction on Robotics, 25(3):670-685, June 2009.
- Dynamic Whole-Body Motion Generation Under Rigid Contacts and Other Unilateral Constraints. L. Saab, O. Ramos, N. Mansard, P. Souères and J-Y. Fourquet. IEEE Transaction on Robotics, 29(2):346-362, April 2013.
- A Dedicated Solver for Fast Operational-Space Inverse Dynamics. Nicolas Mansard. In {2012 IEEE International Conference on Robotics and Automation}, pages 4943-4949, St Paul, United States, May 2012.
- Partial Force Control of Constrained Floating-Base Robots. Andrea Del Prete, Nicolas Mansard, Francesco Nori, Giorgio Metta and Lorenzo Natale. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS'14), Chicago, USA, September 2014 .
- Whole-body Motion Integrating the Capture Point in the Operational Space Inverse Dynamics Control. O. Ramos, N. Mansard and P. Souères. In IEEE-RAS International Conference on Humanoid Robots (Humanoid'14), Madrid, Spain, November 2014.
Sensor Based control
The camera is able to reconstruct the robot position with respect to previous positions. The map helps to consolidate the accuracy. However, the orientation is subject to drifts and the camera is not able to accurately track alone the robot vertical. Moreover, it is not the proper sensor to estimate the contact model (surface orientation and stiness). Two other sensors can be used to bring the missing data. The inertial measurement unit (IMU) measures the angular velocity on one hand and the linear accelerations and gravitational forces on the other hand of the body it is attached to. When the linear acceleration are perfectly known (for example, when the robot does not move), the weight force gives the gravity orientation. When the acceleration is not known, the six measurements are not sucient to make the gravity orientation observable. A classical association is to use the camera to observe the missing information. The coupling IMU/Camera is very interesting: the camera is slow (<100Hz) whereas the IMU is fast (>1kHz); the IMU quickly drifts whereas the camera does not drift while it stays in already-visited places. Moreover, the system camera+IMU is observable. A first fusion was performed in RT-SLAM.
We now tries to fuse the data coming from the force sensors in the ankles to estimate the elasticity in the robot feet and the ground characteristic. In parallel, Florent Lamiraux is working on the stabilization of the feet elasticity and Olivier Stasse is working on the rough estimation of the oor slope using vision. Coupled with the developments of fast dynamics-based control law, all these works are going toward safe outdoor walking on unknown grounds.
- Visually-Guided Grasping while Walking on a Humanoid Robot. N. Mansard, O. Stasse, F. Chaumette and K. Yokoi. In icra07, Roma, Italia, April 2007.
- Integrating Walking and Vision to increase Humanoid Autonomy. O. Stasse, B. Verrelst, A. Davison, N. Mansard, F. Saidi, B. Vanderborght, C. Esteves and K. Yokoi. International Journal of Humanoid Robotics, 5(9):287-310, June 2008.
- RT-SLAM: a generic and real-time visual SLAM implementation. C. Roussillon, A. Gonzalez, J. Solà, J-M. Codol, N. Mansard, S. Lacroix and M. Devy. In Int. Conf. on Computer Vision Systems (ICVS), Sophia Antipolis, France, September 2011. (Lecture notes in computer science 6962 (2011), pp. 31-40).
- Towards Reactive Vision-guided Walking on Rough Terrain: an Inverse-dynamics Based Approach. O. Ramos, M. García, N. Mansard, O. Stasse, J-B. Hayet and P. Souères. International Journal of Humanoid Robotics, 11(2), July 2014.
Motion signature
When replicating a motion, several spaces can be chosen to compute the distance to the original motion. However, in a given context, only one of them is relevant. The idea of the paper is to automatically recognize the relevant space, which would then be used as the reference task space to replicate.
The task recognition is performed only from the body trajectories without using any contextual information: the input is the configuration trajectory of the demonstrator. The motion is then projected in each task space of a candidate task pool and t with a reference trajectory. The residue of the fitting is used to select the best task among the candidates. The effect of this task is then cancelled from the original motion by projecting it in the space orthogonal to the task. The iterative selection finally ends when the residual motion is null.
- Reverse Control for Humanoid Robot Task Recognition. Sovannara Hak, Nicolas Mansard, Olivier Stasse and Jean-Paul Laumond. {IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics}, 42(6):pp.1524-1537, December 2012. (14 pages).
- An advanced robotics motion generation framework for inferring the organization of human movements. O. Ramos, N. Mansard , O. Stasse and P. Souères. Computer Methods in Biomechanics and Biomedical Engineering, 16(1), September 2013.
Task for planning and performing
We look at a solution to automatically decide the data that are needed to specify a task sequence. When reasoning about a motion (for planning for example), the whole spatio-temporal trajectory should be considered1. The motion planning (both for searching for an initial solution (RRT, PRM) or to optimize a given trajectory) is a very complex problem, subject to the question of the existence of a solution and of many local minima. This is the good mathematical way to build a motion. However, the computational cost is yet much too expensive to hope to use it inside the robot control loop. Moreover, if it is used as a planner (i.e. to guide the controller by computing the desired motion as a much slower rate), it provides a trajectory in the configuration space, that is straight-forward to track with a fixed manipulator robot but can be more complex to follow with a mobile robot. The question behind this chapter is to know how to execute with a closed-loop controller a trajectory computed by a motion planner, or more generally how to bridge the gap between planning and control. In this objective, the task sequence can be used as a trajectory representation that makes the link between these two components: the task sequence can be used for planning, while our experience has shown that it is a very efficient way to control the execution.
- Optimization of tasks warping and scheduling for smooth sequencing of robotic actions. F. Keith, N. Mansard, S. Miossec and A. Kheddar. In iros09, pages 1609-1614, St Louis, USA, October 2009.
Open-source motion generation
A inverse-kinematics based movement on the robot contains many basic routines that are reusable but have to be written on a very efficient manners, and topical implementations that does not require a high computational efficiency but is more difficult to reuse from an application to the other. The software is proposed to maximize the efficiency of the kernel routines and allow an easy integration of these routines to produce an actual robot movement.
The software is built as a mixed between C++ efficient routines, and a scripting language to construct the robotics scenarios. The control scheme is assembled using a graph structure (using a design pattern similar to MATLAB/SIMULINK) that links together the C++ routines. The basic features are then proposed to realize an inverse kinematics scheme: robot model computation (direct geometry and kinematics, Jacobians, generalized inertia matrix, etc), classical task functions (end-eector positioning, COM, gaze, etc) and a hierarchical inverse kinematics solver. The software is then distributed with a LGPL license. Everything is available through the Stack-of-Tasks Github.
- A versatile generalized inverted kinematics implementation for collaborative working humanoid robots: the Stack of Tasks. Nicolas Mansard, Olivier Stasse, Paul Evrard and Abderrahmane Kheddar. In {ICAR'09: International Conference on Advanced Robotics}, pages 1-6, Munich, Germany. IEEE, June 2009.
Dance with a robot
The Dance with HRP-2 project is the opportunity to show the interest of the inverse dynamics approach for the generation of whole-body movements for humanoid robots.
The movements were first demonstrated by T. Benamara and recorded using a vision-based motion-capture system. The human posture is extracted by solving for each pose a non-linear least-square problem. The resulting posture is then dynamically retargeted using the inverse-dynamics solver. When necessary, tasks are added to obtain a more accurate replication of the movement (in particular, for fast motion of the COM). For walking, an inverse-pendulum based pattern generator is used to design the COM trajectory.
Motion capture together with inverse dynamics for reshaping allows a pretty fast development time. Nearly twenty minutes of motion were designed in less than two months by a single programmer. The show was performed live, without the security lifter, in front of more than 1000 in the music-hall of Toulouse city on October 2012.
- Dynamic Motion Capture and Edition using a Stack of Tasks. O. Ramos, L. Saab, S. Hak and N. Mansard. In IEEE-RAS International Conference on Humanoid Robots (Humanoid'11), pages p. 224-230, Bled, Slovenia, October 2011.
- Dynamic Whole Body Motion Generation for the Dance of a Humanoid Robot. O. Ramos, N. Mansard, O. Stasse, S. Hak and L. Saab. IEEE Robotics & Automation Magazine, January 2015. (accepted).
Robust walk
We have proposed a first use of the inverse-dynamics scheme as a robust controller in simulation, to control the walk of HRP-2 on an unknown non-at terrain. The hypotheses of the simulation are the following. The simulated robot is tracking the given reference torques (and accelerations). It also perfectly perceives the collision points that are used in the force part of the dynamics equation. The floor is a random distribution of small objects (2cm) on a horizontal plane. However, the model of the floor is not known from the controller, which uses the approximation that the oor is horizontal. The COM trajectory is computed at each control cycle by MPC using a classical inverse-pendulum linearization. The hierarchy of tasks is composed of the COM task, one task to track the flying-foot placement and one task to emphasize the robot posture and avoid drifts of the chest attitude. Contact points are added in the SOT when a collision occurs between the flying foot and the ground during the foot landing. When three contact points are added, the foot landing ends because the foot movements are completely constrained. The contacts are relaxed when the walking pattern generator decides to take off.
The controller is then able to absorb the unknown rough terrain: the COM tracks the MPC trajectory even if the contact surfaces are not exactly those used by the inverse-pendulum model. The rough horizontal approximation is sufficient to ensure the proper regulation by the inverse-dynamics controller.
- Walking on Non-planar Surfaces using an Inverse Dynamic Stack of Tasks. Oscar Efrain Ramos Ponce, Nicolas Mansard, Olivier Stasse and Philippe Sou{ères. In {IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS 2012)}, page 7p., Osaka, Japan, November 2012.
- Towards Reactive Vision-guided Walking on Rough Terrain: an Inverse-dynamics Based Approach. O. Ramos, M. García, N. Mansard, O. Stasse, J-B. Hayet and P. Souères. International Journal of Humanoid Robotics, 11(2), July 2014.
Selected Publications
- Dynamic Whole Body Motion Generation for the Dance of a Humanoid Robot. O. Ramos, N. Mansard, O. Stasse, S. Hak and L. Saab. IEEE Robotics & Automation Magazine, January 2015. (accepted).
- Optimality in Robot Motion: Optimal Versus Optimized Motion. Jean-Paul Laumond, Nicolas Mansard and Jean-Bernard Bernard Lasserre. Communication of the ACM, 57(9):82-89, September 2014.
- Hierarchical quadratic programming: Fast online humanoid-robot motion generation. Adrien Escande, Nicolas Mansard and Pierre-Brice Wieber. {The International Journal of Robotics Research}, 33(7):pp. 1006-1028, June 2014.
- Dynamic Whole-Body Motion Generation Under Rigid Contacts and Other Unilateral Constraints. L. Saab, O. Ramos, N. Mansard, P. Souères and J-Y. Fourquet. IEEE Transaction on Robotics, 29(2):346-362, April 2013.
- Reverse Control for Humanoid Robot Task Recognition. Sovannara Hak, Nicolas Mansard, Olivier Stasse and Jean-Paul Laumond. {IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics}, 42(6):pp.1524-1537, December 2012. (14 pages).
- Directional redundancy. N. Mansard and F. Chaumette. IEEE Transaction on Automatic Control, 54(6):1179-1192, June 2009.
- A Unified Approach to Integrate Unilateral Constraints in the Stack of Tasks. N. Mansard, O. Khatib and A. Kheddar. IEEE Transaction on Robotics, 25(3):670-685, June 2009.
- Task sequencing for sensor-based control. N. Mansard and F. Chaumette. IEEE Transaction on Robotics, 23(1):60 - 72, February 2007.
- Workshop on Redundancy, inequalities, and the mathematical tools to address them. A. Escande, F. Flacco, L. Righetti and N. Mansard. In IEEE-RAS International Conference on Humanoid Robots (Humanoid'14), Madrid, Spain, November 2014.
- Bounded differential dynamic programming. Y. Tassa, N. Mansard and E. Todorov. In IEEE Int. Conf. on Robotics and Automation (ICRA'14), Hong-Kong, China, June 2014.
- A two-stage suboptimal approximation for variable compliance and torque control. P. Geoffroy, O. Bordron, N. Mansard, O. Stasse and T. Bretl. In IEEE European Conference on Control, Strasbourg, France, June 2014.
- Toward a motion semiotics. N. Mansard. In Human-Friendly robotics (HFR'13), September 2013 \quad\textbf{[keynote speaker]}.
- A Dedicated Solver for Fast Operational-Space Inverse Dynamics. Nicolas Mansard. In {2012 IEEE International Conference on Robotics and Automation}, pages 4943-4949, St Paul, United States, May 2012.
- Generic Dynamic Motion Generation with Multiple Unilateral Constraints. L. Saab, O. Ramos, N. Mansard, J-Y. Fourquet and P. Souères. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS'11), pages 4127-4133, San Fransisco, USA. IEEE, September 2011.
- Analysis of the discontinuities in prioritized tasks-space control under discreet task scheduling operations. F. Keith, P.B. Wieber, N. Mansard and A. Kheddar. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS'11), San Fransisco, USA, September 2011.
- Combining Suppression of the Disturbance and Reactive Stepping for Recovering Balance. Mitsuharu Morisawa, Fumio Kanehiro, Kenji Kaneko, Nicolas Mansard, Joan Solà, Eiichi Yoshida, Kazuhito Yokoi and Jean Paul Laumond. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS'10), Taipei, Taiwan, October 2010.
All the publications are avaiblable here (with pdf files if available).
PhD Students
Two positions are currently open (in optimal control and simultaneous localization and mapping, and in motion planning for versatile factories). Contact me with vitae and reference letters if you are interested.
- Justin Carpentier: Optimality in locomotion (with J-P. Laumond)
- Mathieu Geisert: Experience-based motion generation (with F. Lamiraux)
- Oscar Ramos: dynamics sensor-based robot control, PhD defended in 2013 (with P. Soueres)
- Sovannara Hak: Task recognition by reverse control, PhD defended in 2011 (with O. Stasse)
- Layale Saab: constrained dynamics whole-body motion, PhD defended in 2011 (with P. Souères)
- Francois Keith: Optimization and smoothing of task sequences, PhD defended in 2010 (with A. Kheddar)
Teaching
- Advanced robotics: Supaero, Toulouse (40H) (see the page of the class, along with tutorials #1, #2, #3)
- Mathematics for motion generation: École Normale Supérieure, Paris (20H) (see the page of the class)
- Experimental humanoid robotics: INSA, Toulouse (20H) (see the page of the class)
- Introduction to robotics: Supaéro, Toulouse (6H)
- Humanoid robotics: kinematics and locomotion: practical work in INSA and Supaero, Toulouse (20H). See the description of the tutorial.
I also taught Computer Vision in Rennes (INSA and IFSIC), Numerical Optimisation at EDSYS, Toulouse, and assisted for practical works for the class of O. Khatib at Stanford and J-P. Laumond at Collège de France.
Fundings
I am leading ANR project ENTRACTE (cooperation with INRIA Rennes, start Nov 2013). I am the CNRS coordinator and WP leader of the FP7 EuRoc project (led B. Siciliano, Univ. Napoli, start Jan 2014). I have a PI role in Oseo project Romeo-2 (led by Aldebaran and in the cooperation between LAAS and Airbus (led by O. Stasse). I participate to the ERC Actantrophe (led J-P. Laumond), FP7 Koroibot (led by K. Mombaur, CNRS coordinator P. Soueres) and marginally in FP7 Factory in a Day (led by M. Wisse, CNRS coordinator F. Lamiraux) and FP7 Robohow (led by M. Beetz, CNRS coordinator A. Khedar).
I was also part of the FP7 Robot@CWE, FUI Romeo and of various other projects (project leader of Novela project 2011, workpackage leader of ANR R-Blink 2009, Robea Egocentre 2003 and marginally in FP6 VISORBASE 2002).
Peer-review activities
I am Associate Editor in IEEE TRO since July 2013. I was also Associate Editor in IEEE ICRA from 2011 to 2014, in RSS from 2011 to 2013 and in IEEE ICARCV in 2011. I am a frequent reviewer for IJRR, RAS, IEEE ICRA, IEEE IROS, IEEE Humanoids, IEEE TAC and various other conferences.
I was reviewer for the call-for-proposal CHISTERA 2013, and in the several phases of ECHORD.