Header logo is


2008


no image
Learning Inverse Dynamics: A Comparison

Nguyen-Tuong, D., Peters, J., Seeger, M., Schölkopf, B.

In Advances in Computational Intelligence and Learning: Proceedings of the European Symposium on Artificial Neural Networks, pages: 13-18, (Editors: M Verleysen), d-side, Evere, Belgium, 16th European Symposium on Artificial Neural Networks (ESANN), April 2008 (inproceedings)

Abstract
While it is well-known that model can enhance the control performance in terms of precision or energy efficiency, the practical application has often been limited by the complexities of manually obtaining sufficiently accurate models. In the past, learning has proven a viable alternative to using a combination of rigid-body dynamics and handcrafted approximations of nonlinearities. However, a major open question is what nonparametric learning method is suited best for learning dynamics? Traditionally, locally weighted projection regression (LWPR), has been the standard method as it is capable of online, real-time learning for very complex robots. However, while LWPR has had significant impact on learning in robotics, alternative nonparametric regression methods such as support vector regression (SVR) and Gaussian processes regression (GPR) offer interesting alternatives with fewer open parameters and potentially higher accuracy. In this paper, we evaluate these three alternatives for model learning. Our comparison consists out of the evaluation of learning quality for each regression method using original data from SARCOS robot arm, as well as the robot tracking performance employing learned models. The results show that GPR and SVR achieve a superior learning precision and can be applied for real-time control obtaining higher accuracy. However, for the online learning LWPR presents the better method due to its lower computational requirements.

ei

PDF Web [BibTex]

2008


PDF Web [BibTex]


no image
The role of stimulus correlations for population decoding in the retina

Schwartz, G., Macke, J., Berry, M.

Computational and Systems Neuroscience 2008 (COSYNE 2008), 5, pages: 172, March 2008 (poster)

ei

PDF Web [BibTex]

PDF Web [BibTex]


no image
Effects of Proprioceptive Motion Feedback on Sighted and Unsighted Control of a Virtual Hand Prosthesis

Blank, A., Okamura, A. M., Kuchenbecker, K. J.

In Proc. IEEE Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pages: 141-142, Reno, Nevada, USA, March 2008, Poster presentation given by Blank (inproceedings)

hi

[BibTex]

[BibTex]


no image
The Touch Thimble: Providing Fingertip Contact Feedback During Point-Force Haptic Interaction

Kuchenbecker, K. J., Ferguson, D., Kutzer, M., Moses, M., Okamura, A. M.

In Proc. IEEE Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pages: 239-246, Reno, Nevada, USA, March 2008, Oral presentation given by Kuchenbecker (inproceedings)

hi

[BibTex]

[BibTex]


no image
Haptography: Capturing the Feel of Real Objects to Enable Authentic Haptic Rendering

Kuchenbecker, K. J.

In Proc. Haptic in Ambient Systems (HAS) Workshop, in conjunction with the First International Conference on Ambient Media and Systems, Montreal, Canada, February 2008 (inproceedings)

hi

[BibTex]

[BibTex]


Thumb xl screen shot 2018 02 03 at 5.40.07 pm
Passive compliant quadruped robot using central pattern generators for locomotion control

Rutishauser, S., Spröwitz, A., Righetti, L., Ijspeert, A. J.

In Proceedings of the 2008 2nd Biennial IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, pages: 710-715, IEEE, Scottsdale, AZ, 2008 (inproceedings)

Abstract
We present a new quadruped robot, “Cheetah”, featuring three-segment pantographic legs with passive compliant knee joints. Each leg has two degrees of freedom - knee and hip joint can be actuated using proximal mounted RC servo motors, force transmission to the knee is achieved by means of a Bowden cable mechanism. Simple electronics to command the actuators from a desktop computer have been designed in order to test the robot. A Central Pattern Generator (CPG) network has been implemented to generate different gaits. A parameter space search was performed and tested on the robot to optimize forward velocity.

dlg

DOI [BibTex]

DOI [BibTex]


Thumb xl screen shot 2018 02 03 at 6.34.37 pm
Graph signature for self-reconfiguration planning

Asadpour, M., Spröwitz, A., Billard, A., Dillenbourg, P., Ijspeert, A. J.

In Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages: 863-869, IEEE, Nice, 2008 (inproceedings)

Abstract
This project incorporates modular robots as build- ing blocks for furniture that moves and self-reconfigures. The reconfiguration is done using dynamic connection / disconnection of modules and rotations of the degrees of freedom. This paper introduces a new approach to self-reconfiguration planning for modular robots based on the graph signature and the graph edit-distance. The method has been tested in simulation on two type of modules: YaMoR and M-TRAN. The simulation results shows interesting features of the approach, namely rapidly finding a near-optimal solution.

dlg

DOI [BibTex]

DOI [BibTex]


Thumb xl screen shot 2018 02 03 at 6.36.13 pm
An active connection mechanism for modular self-reconfigurable robotic systems based on physical latching

Spröwitz, A., Asadpour, M., Bourquin, Y., Ijspeert, A. J.

In Proceedings on the 2008 IEEE International Conference on Robotics and Automation (ICRA), 2008, pages: 3508-3513, IEEE, Pasadena, CA, 2008 (inproceedings)

Abstract
This article presents a robust and heavy duty physical latching connection mechanism, which can be actuated with DC motors to actively connect and disconnect modular robot units. The special requirements include a lightweight and simple construction providing an active, strong, hermaphrodite, completely retractable connection mechanism with a 90 degree symmetry and a no-energy consumption in the locked state. The mechanism volume is kept small to fit multiple copies into a single modular robot unit and to be used on as many faces of the robot unit as possible. This way several different lattice like modular robot structures are possible. The large selection for dock-able connection positions will likely simplify self-reconfiguration strategies. Tests with the implemented mechanism demonstrate its applicative potential for self-reconfiguring modular robots.

dlg

DOI [BibTex]

DOI [BibTex]


no image
Simulation and analysis of a passive pitch reversal flapping wing mechanism for an aerial robotic platform

Arabagi, V., Sitti, M.

In Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pages: 1260-1265, 2008 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Human movement generation based on convergent flow fields: A computational model and a behavioral experiment

Hoffmann, H., Schaal, S.

In Advances in Computational Motor Control VII, Symposium at the Society for Neuroscience Meeting, Washington DC, 2008, 2008, clmc (inproceedings)

am

link (url) [BibTex]

link (url) [BibTex]


Thumb xl thumb screen shot 2012 10 06 at 12.28.24 pm
Simultaneous Visual Recognition of Manipulation Actions and Manipulated Objects

Kjellström, H., Romero, J., Martinez, D., Kragic, D.

In European Conference on Computer Vision, ECCV, pages: 336-349, 2008 (inproceedings)

ps

Pdf [BibTex]

Pdf [BibTex]


no image
Tuning analysis of motor cortical neurons in a person with paralysis during performance of visually instructed cursor control tasks

Kim, S., Simeral, J. D., Hochberg, L. R., Truccolo, W., Donoghue, J., Friehs, G. M., Black, M. J.

2008 Abstract Viewer and Itinerary Planner, Society for Neuroscience, Washington, DC, 2008, Online (conference)

ps

[BibTex]

[BibTex]


no image
Pattern generators with sensory feedback for the control of quadruped locomotion

Righetti, L., Ijspeert, A.

In 2008 IEEE International Conference on Robotics and Automation, pages: 819-824, IEEE, Pasadena, USA, 2008 (inproceedings)

Abstract
Central pattern generators (CPGs) are becoming a popular model for the control of locomotion of legged robots. Biological CPGs are neural networks responsible for the generation of rhythmic movements, especially locomotion. In robotics, a systematic way of designing such CPGs as artificial neural networks or systems of coupled oscillators with sensory feedback inclusion is still missing. In this contribution, we present a way of designing CPGs with coupled oscillators in which we can independently control the ascending and descending phases of the oscillations (i.e. the swing and stance phases of the limbs). Using insights from dynamical system theory, we construct generic networks of oscillators able to generate several gaits under simple parameter changes. Then we introduce a systematic way of adding sensory feedback from touch sensors in the CPG such that the controller is strongly coupled with the mechanical system it controls. Finally we control three different simulated robots (iCub, Aibo and Ghostdog) using the same controller to show the effectiveness of the approach. Our simulations prove the importance of independent control of swing and stance duration. The strong mutual coupling between the CPG and the robot allows for more robust locomotion, even under non precise parameters and non-flat environment.

mg

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Experimental Study of Limit Cycle and Chaotic Controllers for the Locomotion of Centipede Robots

Matthey, L., Righetti, L., Ijspeert, A.

In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages: 1860-1865, IEEE, Nice, France, sep 2008 (inproceedings)

Abstract
In this contribution we present a CPG (central pattern generator) controller based on coupled Rossler systems. It is able to generate both limit cycle and chaotic behaviors through bifurcation. We develop an experimental test bench to measure quantitatively the performance of different controllers on unknown terrains of increasing difficulty. First, we show that for flat terrains, open loop limit cycle systems are the most efficient (in terms of speed of locomotion) but that they are quite sensitive to environmental changes. Second, we show that sensory feedback is a crucial addition for unknown terrains. Third, we show that the chaotic controller with sensory feedback outperforms the other controllers in very difficult terrains and actually promotes the emergence of short synchronized movement patterns. All that is done using an unified framework for the generation of limit cycle and chaotic behaviors, where a simple parameter change can switch from one behavior to the other through bifurcation. Such flexibility would allow the automatic adaptation of the robot locomotion strategy to the terrain uncertainty.

mg

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Fabrication and Characterization of Biologically Inspired Mushroom-Shaped Elastomer Microfiber Arrays

Kim, S., Sitti, M.

In ASME 2008 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pages: 839-847, 2008 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Gecko inspired micro-fibrillar adhesives for wall climbing robots on micro/nanoscale rough surfaces

Aksak, B., Murphy, M. P., Sitti, M.

In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages: 3058-3063, 2008 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Miniature Mobile Robots Down to Micron Scale

Sitti, M.

In Micro-NanoMechatronics and Human Science, 2008. MHS 2008. International Symposium on, pages: 525-525, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields

Park, D., Hoffmann, H., Pastor, P., Schaal, S.

In IEEE International Conference on Humanoid Robots, 2008., 2008, clmc (inproceedings)

am

PDF [BibTex]

PDF [BibTex]


Thumb xl screen shot 2012 06 06 at 11.28.04 am
Infinite Kernel Learning

Gehler, P., Nowozin, S.

In Proceedings of NIPS 2008 Workshop on "Kernel Learning: Automatic Selection of Optimal Kernels", 2008 (inproceedings)

ps

project page pdf [BibTex]

project page pdf [BibTex]


Thumb xl thumb screen shot 2012 10 06 at 12.29.08 pm
Visual Recognition of Grasps for Human-to-Robot Mapping

Kjellström, H., Romero, J., Kragic, D.

In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, pages: 3192-3199, 2008 (inproceedings)

ps

Pdf [BibTex]

Pdf [BibTex]


no image
More than two years of intracortically-based cursor control via a neural interface system

Hochberg, L. R., Simeral, J. D., Kim, S., Stein, J., Friehs, G. M., Black, M. J., Donoghue, J. P.

2008 Abstract Viewer and Itinerary Planner, Society for Neuroscience, Washington, DC, 2008, Online (conference)

ps

[BibTex]

[BibTex]


no image
Wetting and premelting of triple junctions and grain boundaries in the Al-Zn alloys

Straumal, B., Kogtenkova, O., Protasova, S., Mazilkin, A., Zieba, P., Czeppe, T., Wojewoda-Budka, J., Faryna, M.

In 495, pages: 126-131, Alicante, Spain, 2008 (inproceedings)

mms

DOI [BibTex]

DOI [BibTex]


no image
The dual role of uncertainty in force field learning

Mistry, M., Theodorou, E., Hoffmann, H., Schaal, S.

In Abstracts of the Eighteenth Annual Meeting of Neural Control of Movement (NCM), Naples, Florida, April 29-May 4, 2008, clmc (inproceedings)

Abstract
Force field experiments have been a successful paradigm for studying the principles of planning, execution, and learning in human arm movements. Subjects have been shown to cope with the disturbances generated by force fields by learning internal models of the underlying dynamics to predict disturbance effects or by increasing arm impedance (via co-contraction) if a predictive approach becomes infeasible. Several studies have addressed the issue uncertainty in force field learning. Scheidt et al. demonstrated that subjects exposed to a viscous force field of fixed structure but varying strength (randomly changing from trial to trial), learn to adapt to the mean disturbance, regardless of the statistical distribution. Takahashi et al. additionally show a decrease in strength of after-effects after learning in the randomly varying environment. Thus they suggest that the nervous system adopts a dual strategy: learning an internal model of the mean of the random environment, while simultaneously increasing arm impedance to minimize the consequence of errors. In this study, we examine what role variance plays in the learning of uncertain force fields. We use a 7 degree-of-freedom exoskeleton robot as a manipulandum (Sarcos Master Arm, Sarcos, Inc.), and apply a 3D viscous force field of fixed structure and strength randomly selected from trial to trial. Additionally, in separate blocks of trials, we alter the variance of the randomly selected strength multiplier (while keeping a constant mean). In each block, after sufficient learning has occurred, we apply catch trials with no force field and measure the strength of after-effects. As expected in higher variance cases, results show increasingly smaller levels of after-effects as the variance is increased, thus implying subjects choose the robust strategy of increasing arm impedance to cope with higher levels of uncertainty. Interestingly, however, subjects show an increase in after-effect strength with a small amount of variance as compared to the deterministic (zero variance) case. This result implies that a small amount of variability aides in internal model formation, presumably a consequence of the additional amount of exploration conducted in the workspace of the task.

am

[BibTex]

[BibTex]


no image
Dynamic movement primitives for movement generation motivated by convergent force fields in frog

Hoffmann, H., Pastor, P., Schaal, S.

In Adaptive Motion of Animals and Machines (AMAM), 2008, clmc (inproceedings)

am

PDF [BibTex]

PDF [BibTex]


no image
Polymeric Micro/Nanofiber Manufacturing and Mechanical Characterization

Nain, A. S., Sitti, M., Amon, C.

In ASME 2008 International Mechanical Engineering Congress and Exposition, pages: 295-303, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
An untethered magnetically actuated micro-robot capable of motion on arbitrary surfaces

Floyd, S., Pawashe, C., Sitti, M.

In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages: 419-424, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Fabrication of bio-inspired elastomer nanofiber arrays with spatulate tips using notching effect

Kim, S., Sitti, M., Jang, J., Thomas, E. L.

In Nanotechnology, 2008. NANO’08. 8th IEEE Conference on, pages: 780-782, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
A motorized anchoring mechanism for a tethered capsule robot using fibrillar adhesives for interventions in the esophagus

Glass, P., Cheung, E., Wang, H., Appasamy, R., Sitti, M.

In Biomedical Robotics and Biomechatronics, 2008. BioRob 2008. 2nd IEEE RAS & EMBS International Conference on, pages: 758-764, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Emergence of Interaction Among Adaptive Agents

Martius, G., Nolfi, S., Herrmann, J. M.

In Proc. From Animals to Animats 10 (SAB 2008), 5040, pages: 457-466, LNCS, Springer, 2008 (inproceedings)

al

DOI [BibTex]

DOI [BibTex]


no image
In-lane Localization in Road Networks using Curbs Detected in Omnidirectional Height Images

Stueckler, J., Schulz, H., Behnke, S.

In Proceedings of Robotik 2008, 2008 (inproceedings)

ev

link (url) [BibTex]

link (url) [BibTex]


no image
Decoding of reach and grasp from MI population spiking activity using a low-dimensional model of hand and arm posture

Yadollahpour, P., Shakhnarovich, G., Vargas-Irwin, C., Donoghue, J. P., Black, M. J.

2008 Abstract Viewer and Itinerary Planner, Society for Neuroscience, Washington, DC, 2008, Online (conference)

ps

[BibTex]

[BibTex]


no image
A Dynamical System for Online Learning of Periodic Movements of Unknown Waveform and Frequency

Gams, A., Righetti, L., Ijspeert, A., Lenarčič, J.

In 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, pages: 85-90, IEEE, Scottsdale, USA, October 2008 (inproceedings)

Abstract
The paper presents a two-layered system for learning and encoding a periodic signal onto a limit cycle without any knowledge on the waveform and the frequency of the signal, and without any signal processing. The first dynamical system is responsible for extracting the main frequency of the input signal. It is based on adaptive frequency phase oscillators in a feedback structure, enabling us to extract separate frequency components without any signal processing, as all of the processing is embedded in the dynamics of the system itself. The second dynamical system is responsible for learning of the waveform. It has a built-in learning algorithm based on locally weighted regression, which adjusts the weights according to the amplitude of the input signal. By combining the output of the first system with the input of the second system we can rapidly teach new trajectories to robots. The systems works online for any periodic signal and can be applied in parallel to multiple dimensions. Furthermore, it can adapt to changes in frequency and shape, e.g. to non-stationary signals, and is computationally inexpensive. Results using simulated and hand-generated input signals, along with applying the algorithm to a HOAP-2 humanoid robot are presented.

mg

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Passive compliant quadruped robot using central pattern generators for locomotion control

Rutishauser, S., Sproewitz, A., Righetti, L., Ijspeert, A.

In 2008 IEEE International Conference on Biomedical Robotics and Biomechatronics, pages: 710-715, IEEE, Scottsdale, USA, October 2008 (inproceedings)

Abstract
We present a new quadruped robot, ldquoCheetahrdquo, featuring three-segment pantographic legs with passive compliant knee joints. Each leg has two degrees of freedom - knee and hip joint can be actuated using proximal mounted RC servo motors, force transmission to the knee is achieved by means of a bowden cable mechanism. Simple electronics to command the actuators from a desktop computer have been designed in order to test the robot. A Central Pattern Generator (CPG) network has been implemented to generate different gaits. A parameter space search was performed and tested on the robot to optimize forward velocity.

mg

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Behavioral experiments on reinforcement learning in human motor control

Hoffmann, H., Theodorou, E., Schaal, S.

In Abstracts of the Eighteenth Annual Meeting of Neural Control of Movement (NCM), Naples, Florida, April 29-May 4, 2008, clmc (inproceedings)

Abstract
Reinforcement learning (RL) - learning solely based on reward or cost feedback - is widespread in robotics control and has been also suggested as computational model for human motor control. In human motor control, however, hardly any experiment studied reinforcement learning. Here, we study learning based on visual cost feedback in a reaching task and did three experiments: (1) to establish a simple enough experiment for RL, (2) to study spatial localization of RL, and (3) to study the dependence of RL on the cost function. In experiment (1), subjects sit in front of a drawing tablet and look at a screen onto which the drawing pen's position is projected. Beginning from a start point, their task is to move with the pen through a target point presented on screen. Visual feedback about the pen's position is given only before movement onset. At the end of a movement, subjects get visual feedback only about the cost of this trial. We choose as cost the squared distance between target and virtual pen position at the target line. Above a threshold value, the cost was fixed at this value. In the mapping of the pen's position onto the screen, we added a bias (unknown to subject) and Gaussian noise. As result, subjects could learn the bias, and thus, showed reinforcement learning. In experiment (2), we randomly altered the target position between three different locations (three different directions from start point: -45, 0, 45). For each direction, we chose a different bias. As result, subjects learned all three bias values simultaneously. Thus, RL can be spatially localized. In experiment (3), we varied the sensitivity of the cost function by multiplying the squared distance with a constant value C, while keeping the same cut-off threshold. As in experiment (2), we had three target locations. We assigned to each location a different C value (this assignment was randomized between subjects). Since subjects learned the three locations simultaneously, we could directly compare the effect of the different cost functions. As result, we found an optimal C value; if C was too small (insensitive cost), learning was slow; if C was too large (narrow cost valley), the exploration time was longer and learning delayed. Thus, reinforcement learning in human motor control appears to be sen

am

[BibTex]

[BibTex]


no image
Movement generation by learning from demonstration and generalization to new targets

Pastor, P., Hoffmann, H., Schaal, S.

In Adaptive Motion of Animals and Machines (AMAM), 2008, clmc (inproceedings)

am

PDF [BibTex]

PDF [BibTex]


no image
Combining dynamic movement primitives and potential fields for online obstacle avoidance

Park, D., Hoffmann, H., Schaal, S.

In Adaptive Motion of Animals and Machines (AMAM), Cleveland, Ohio, 2008, 2008, clmc (inproceedings)

am

link (url) [BibTex]

link (url) [BibTex]


no image
Fabrication of Single and Multi-Layer Fibrous Biomaterial Scaffolds for Tissue Engineering

Nain, A. S., Miller, E., Sitti, M., Campbell, P., Amon, C.

In ASME 2008 International Mechanical Engineering Congress and Exposition, pages: 231-238, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Performance of different foot designs for a water running robot

Floyd, S., Adilak, S., Ramirez, S., Rogman, R., Sitti, M.

In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages: 244-250, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Dynamic modeling of a basilisk lizard inspired quadruped robot running on water

Park, H. S., Floyd, S., Sitti, M.

In Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pages: 3101-3107, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Bacterial propulsion of chemically patterned micro-cylinders

Behkam, B., Sitti, M.

In Biomedical Robotics and Biomechatronics, 2008. BioRob 2008. 2nd IEEE RAS & EMBS International Conference on, pages: 753-757, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Structure from Behavior in Autonomous Agents

Martius, G., Fiedler, K., Herrmann, J.

In Proc. IEEE Intl. Conf. Intelligent Robots and Systems (IROS 2008), pages: 858 - 862, 2008 (inproceedings)

al

DOI [BibTex]

DOI [BibTex]


no image
Orthogonal wall correction for visual motion estimation

Stueckler, J., Behnke, S.

In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), pages: 1-6, May 2008 (inproceedings)

ev

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Neural activity in the motor cortex of humans with tetraplegia

Donoghue, J., Simeral, J., Black, M., Kim, S., Truccolo, W., Hochberg, L.

AREADNE Research in Encoding And Decoding of Neural Ensembles, June, Santorini, Greece, 2008 (conference)

ps

[BibTex]

[BibTex]


no image
Computational model for movement learning under uncertain cost

Theodorou, E., Hoffmann, H., Mistry, M., Schaal, S.

In Abstracts of the Society of Neuroscience Meeting (SFN 2008), Washington, DC 2008, 2008, clmc (inproceedings)

Abstract
Stochastic optimal control is a framework for computing control commands that lead to an optimal behavior under a given cost. Despite the long history of optimal control in engineering, it has been only recently applied to describe human motion. So far, stochastic optimal control has been mainly used in tasks that are already learned, such as reaching to a target. For learning, however, there are only few cases where optimal control has been applied. The main assumptions of stochastic optimal control that restrict its application to tasks after learning are the a priori knowledge of (1) a quadratic cost function (2) a state space model that captures the kinematics and/or dynamics of musculoskeletal system and (3) a measurement equation that models the proprioceptive and/or exteroceptive feedback. Under these assumptions, a sequence of control gains is computed that is optimal with respect to the prespecified cost function. In our work, we relax the assumption of the a priori known cost function and provide a computational framework for modeling tasks that involve learning. Typically, a cost function consists of two parts: one part that models the task constraints, like squared distance to goal at movement endpoint, and one part that integrates over the squared control commands. In learning a task, the first part of this cost function will be adapted. We use an expectation-maximization scheme for learning: the expectation step optimizes the task constraints through gradient descent of a reward function and the maximizing step optimizes the control commands. Our computational model is tested and compared with data given from a behavioral experiment. In this experiment, subjects sit in front of a drawing tablet and look at a screen onto which the drawing-pen's position is projected. Beginning from a start point, their task is to move with the pen through a target point presented on screen. Visual feedback about the pen's position is given only before movement onset. At the end of a movement, subjects get visual feedback only about the cost of this trial. In the mapping of the pen's position onto the screen, we added a bias (unknown to subject) and Gaussian noise. Therefore the cost is a function of this bias. The subjects were asked to reach to the target and minimize this cost over trials. In this behavioral experiment, subjects could learn the bias and thus showed reinforcement learning. With our computational model, we could model the learning process over trials. Particularly, the dependence on parameters of the reward function (Gaussian width) and the modulation of movement variance over time were similar in experiment and model.

am

[BibTex]

[BibTex]


no image
A Bayesian approach to empirical local linearizations for robotics

Ting, J., D’Souza, A., Vijayakumar, S., Schaal, S.

In International Conference on Robotics and Automation (ICRA2008), Pasadena, CA, USA, May 19-23, 2008, 2008, clmc (inproceedings)

Abstract
Local linearizations are ubiquitous in the control of robotic systems. Analytical methods, if available, can be used to obtain the linearization, but in complex robotics systems where the the dynamics and kinematics are often not faithfully obtainable, empirical linearization may be preferable. In this case, it is important to only use data for the local linearization that lies within a ``reasonable'' linear regime of the system, which can be defined from the Hessian at the point of the linearization -- a quantity that is not available without an analytical model. We introduce a Bayesian approach to solve statistically what constitutes a ``reasonable'' local regime. We approach this problem in the context local linear regression. In contrast to previous locally linear methods, we avoid cross-validation or complex statistical hypothesis testing techniques to find the appropriate local regime. Instead, we treat the parameters of the local regime probabilistically and use approximate Bayesian inference for their estimation. This approach results in an analytical set of iterative update equations that are easily implemented on real robotics systems for real-time applications. As in other locally weighted regressions, our algorithm also lends itself to complete nonlinear function approximation for learning empirical internal models. We sketch the derivation of our Bayesian method and provide evaluations on synthetic data and actual robot data where the analytical linearization was known.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Do humans plan continuous trajectories in kinematic coordinates?

Hoffmann, H., Schaal, S.

In Abstracts of the Society of Neuroscience Meeting (SFN 2008), Washington, DC 2008, 2008, clmc (inproceedings)

Abstract
The planning and execution of human arm movements is still unresolved. An ongoing controversy is whether we plan a movement in kinematic coordinates and convert these coordinates with an inverse internal model into motor commands (like muscle activation) or whether we combine a few muscle synergies or equilibrium points to move a hand, e.g., between two targets. The first hypothesis implies that a planner produces a desired end-effector position for all time points; the second relies on the dynamics of the muscular-skeletal system for a given control command to produce a continuous end-effector trajectory. To distinguish between these two possibilities, we use a visuomotor adaptation experiment. Subjects moved a pen on a graphics tablet and observed the pen's mapped position onto a screen (subjects quickly adapted to this mapping). The task was to move a cursor between two points in a given time window. In the adaptation test, we manipulated the velocity profile of the cursor feedback such that the shape of the trajectories remained unchanged (for straight paths). If humans would use a kinematic plan and map at each time the desired end-effector position onto control commands, subjects should adapt to the above manipulation. In a similar experiment, Wolpert et al (1995) showed adaptation to changes in the curvature of trajectories. This result, however, cannot rule out a shift of an equilibrium point or an additional synergy activation between start and end point of a movement. In our experiment, subjects did two sessions, one control without and one with velocity-profile manipulation. To skew the velocity profile of the cursor trajectory, we added to the current velocity, v, the function 0.8*v*cos(pi + pi*x), where x is the projection of the cursor position onto the start-goal line divided by the distance start to goal (x=0 at the start point). As result, subjects did not adapt to this manipulation: for all subjects, the true hand motion was not significantly modified in a direction consistent with adaptation, despite that the visually presented motion differed significantly from the control motion. One may still argue that this difference in motion was insufficient to be processed visually. Thus, as a control experiment, we replayed control and modified motions to the subjects and asked which of the two motions appeared 'more natural'. Subjects chose the unperturbed motion as more natural significantly better than chance. In summary, for a visuomotor transformation task, the hypothesis of a planned continuous end-effector trajectory predicts adaptation to a modified velocity profile. The current experiment found no adaptation under such transformation.

am

[BibTex]

[BibTex]


no image
Design and Numerical Modeling of an On-Board Chemical Release Module for Motion Control of Bacteria-Propelled Swimming Micro-Robots

Behkam, B., Nain, A. S., Amon, C. H., Sitti, M.

In ASME 2008 International Mechanical Engineering Congress and Exposition, pages: 239-244, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Investigation of Calcium Mechanotransduction by Quasi 3-D Microfiber Mechanical Stimulation of Cells

Ruder, W. C., Pratt, E. D., Sitti, M., LeDuc, P. R., Antaki, J. F.

In ASME 2008 Summer Bioengineering Conference, pages: 1049-1050, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Beanbag robotics: Robotic swarms with 1-dof units

Kriesel, D. M., Cheung, E., Sitti, M., Lipson, H.

In International Conference on Ant Colony Optimization and Swarm Intelligence, pages: 267-274, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Particle image velocimetry and thrust of flagellar micro propulsion systems

Danis, U., Sitti, M., Pekkan, K.

In APS Division of Fluid Dynamics Meeting Abstracts, 1, 2008 (inproceedings)

pi

[BibTex]

[BibTex]