Header logo is


2016


A New Perspective and Extension of the Gaussian Filter
A New Perspective and Extension of the Gaussian Filter

Wüthrich, M., Trimpe, S., Garcia Cifuentes, C., Kappler, D., Schaal, S.

The International Journal of Robotics Research, 35(14):1731-1749, December 2016 (article)

Abstract
The Gaussian Filter (GF) is one of the most widely used filtering algorithms; instances are the Extended Kalman Filter, the Unscented Kalman Filter and the Divided Difference Filter. The GF represents the belief of the current state by a Gaussian distribution, whose mean is an affine function of the measurement. We show that this representation can be too restrictive to accurately capture the dependences in systems with nonlinear observation models, and we investigate how the GF can be generalized to alleviate this problem. To this end, we view the GF as the solution to a constrained optimization problem. From this new perspective, the GF is seen as a special case of a much broader class of filters, obtained by relaxing the constraint on the form of the approximate posterior. On this basis, we outline some conditions which potential generalizations have to satisfy in order to maintain the computational efficiency of the GF. We propose one concrete generalization which corresponds to the standard GF using a pseudo measurement instead of the actual measurement. Extending an existing GF implementation in this manner is trivial. Nevertheless, we show that this small change can have a major impact on the estimation accuracy.

am ics

PDF DOI Project Page [BibTex]

2016


PDF DOI Project Page [BibTex]


Wireless actuation with functional acoustic surfaces
Wireless actuation with functional acoustic surfaces

Qiu, T., Palagi, S., Mark, A. G., Melde, K., Adams, F., Fischer, P.

Appl. Phys. Lett., 109(19):191602, November 2016, APL Editor's pick. APL News. (article)

Abstract
Miniaturization calls for micro-actuators that can be powered wirelessly and addressed individually. Here, we develop functional surfaces consisting of arrays of acoustically resonant microcavities, and we demonstrate their application as two-dimensional wireless actuators. When remotely powered by an acoustic field, the surfaces provide highly directional propulsive forces in fluids through acoustic streaming. A maximal force of similar to 0.45mN is measured on a 4 x 4 mm(2) functional surface. The response of the surfaces with bubbles of different sizes is characterized experimentally. This shows a marked peak around the micro-bubbles' resonance frequency, as estimated by both an analytical model and numerical simulations. The strong frequency dependence can be exploited to address different surfaces with different acoustic frequencies, thus achieving wireless actuation with multiple degrees of freedom. The use of the functional surfaces as wireless ready-to-attach actuators is demonstrated by implementing a wireless and bidirectional miniaturized rotary motor, which is 2.6 x 2.6 x 5 mm(3) in size and generates a stall torque of similar to 0.5 mN.mm. The adoption of micro-structured surfaces as wireless actuators opens new possibilities in the development of miniaturized devices and tools for fluidic environments that are accessible by low intensity ultrasound fields.

pf

link (url) DOI Project Page [BibTex]

link (url) DOI Project Page [BibTex]


Nanomotors
Nanomotors

Alarcon-Correa, M., Walker (Schamel), D., Qiu, T., Fischer, P.

Eur. Phys. J.-Special Topics, 225(11-12):2241-2254, November 2016 (article)

Abstract
This minireview discusses whether catalytically active macromolecules and abiotic nanocolloids, that are smaller than motile bacteria, can self-propel. Kinematic reversibility at low Reynolds number demands that self-propelling colloids must break symmetry. Methods that permit the synthesis and fabrication of Janus nanocolloids are therefore briefly surveyed, as well as means that permit the analysis of the nanocolloids' motion. Finally, recent work is reviewed which shows that nanoagents are small enough to penetrate the complex inhomogeneous polymeric network of biological fluids and gels, which exhibit diverse rheological behaviors.

pf

DOI [BibTex]

DOI [BibTex]


Structured light enables biomimetic swimming and versatile locomotion of photoresponsive soft microrobots
Structured light enables biomimetic swimming and versatile locomotion of photoresponsive soft microrobots

Palagi, S., Mark, A. G., Reigh, S. Y., Melde, K., Qiu, T., Zeng, H., Parmeggiani, C., Martella, D., Sanchez-Castillo, A., Kapernaum, N., Giesselmann, F., Wiersma, D. S., Lauga, E., Fischer, P.

Nature Materials, 15(6):647–653, November 2016, Max Planck press release, Nature News & Views. (article)

Abstract
Microorganisms move in challenging environments by periodic changes in body shape. In contrast, current artificial microrobots cannot actively deform, exhibiting at best passive bending under external fields. Here, by taking advantage of the wireless, scalable and spatiotemporally selective capabilities that light allows, we show that soft microrobots consisting of photoactive liquid-crystal elastomers can be driven by structured monochromatic light to perform sophisticated biomimetic motions. We realize continuum yet selectively addressable artificial microswimmers that generate travelling-wave motions to self-propel without external forces or torques, as well as microrobots capable of versatile locomotion behaviours on demand. Both theoretical predictions and experimental results confirm that multiple gaits, mimicking either symplectic or antiplectic metachrony of ciliate protozoa, can be achieved with single microswimmers. The principle of using structured light can be extended to other applications that require microscale actuation with sophisticated spatiotemporal coordination for advanced microrobotic technologies.

pf

Video - Soft photo Micro-Swimmer DOI [BibTex]

Video - Soft photo Micro-Swimmer DOI [BibTex]


Capture of 2D Microparticle Arrays via a UV-Triggered Thiol-yne ``Click{''} Reaction
Capture of 2D Microparticle Arrays via a UV-Triggered Thiol-yne “Click” Reaction

Walker (Schamel), D., Singh, D. P., Fischer, P.

Advanced Materials, 28(44):9846-9850, September 2016 (article)

Abstract
Immobilization of colloidal assemblies onto solid supports via a fast UV-triggered click-reaction is achieved. Transient assemblies of microparticles and colloidal materials can be captured and transferred to solid supports. The technique does not require complex reaction conditions, and is compatible with a variety of particle assembly methods.

pf

DOI [BibTex]


Magnesium plasmonics for UV applications and chiral sensing
Magnesium plasmonics for UV applications and chiral sensing

Jeong, H. H., Mark, A. G., Fischer, P.

Chem. Comm., 52(82):12179-12182, September 2016 (article)

Abstract
We demonstrate that chiral magnesium nanoparticles show remarkable plasmonic extinction- and chiroptical-effects in the ultraviolet region. The Mg nanohelices possess an enhanced local surface plasmon resonance (LSPR) sensitivity due to the strong dispersion of most substances in the UV region.

pf

DOI [BibTex]

DOI [BibTex]


Holograms for acoustics
Holograms for acoustics

Melde, K., Mark, A. G., Qiu, T., Fischer, P.

Nature, 537, pages: 518-522, September 2016, Max Planck press release, Nature News & Views, Nature Video. (article)

Abstract
Holographic techniques are fundamental to applications such as volumetric displays(1), high-density data storage and optical tweezers that require spatial control of intricate optical(2) or acoustic fields(3,4) within a three-dimensional volume. The basis of holography is spatial storage of the phase and/or amplitude profile of the desired wavefront(5,6) in a manner that allows that wavefront to be reconstructed by interference when the hologram is illuminated with a suitable coherent source. Modern computer-generated holography(7) skips the process of recording a hologram from a physical scene, and instead calculates the required phase profile before rendering it for reconstruction. In ultrasound applications, the phase profile is typically generated by discrete and independently driven ultrasound sources(3,4,8-12); however, these can only be used in small numbers, which limits the complexity or degrees of freedom that can be attained in the wavefront. Here we introduce monolithic acoustic holograms, which can reconstruct diffraction-limited acoustic pressure fields and thus arbitrary ultrasound beams. We use rapid fabrication to craft the holograms and achieve reconstruction degrees of freedom two orders of magnitude higher than commercial phased array sources. The technique is inexpensive, appropriate for both transmission and reflection elements, and scales well to higher information content, larger aperture size and higher power. The complex three-dimensional pressure and phase distributions produced by these acoustic holograms allow us to demonstrate new approaches to controlled ultrasonic manipulation of solids in water, and of liquids and solids in air. We expect that acoustic holograms will enable new capabilities in beam-steering and the contactless transfer of power, improve medical imaging, and drive new applications of ultrasound.

pf

Video - Holograms for Sound DOI Project Page [BibTex]

Video - Holograms for Sound DOI Project Page [BibTex]


A loop-gap resonator for chirality-sensitive nuclear magneto-electric resonance (NMER)
A loop-gap resonator for chirality-sensitive nuclear magneto-electric resonance (NMER)

Garbacz, P., Fischer, P., Kraemer, S.

J. Chem. Phys., 145(10):104201, September 2016 (article)

Abstract
Direct detection of molecular chirality is practically impossible by methods of standard nuclear magnetic resonance (NMR) that is based on interactions involving magnetic-dipole and magnetic-field operators. However, theoretical studies provide a possible direct probe of chirality by exploiting an enantiomer selective additional coupling involving magnetic-dipole, magnetic-field, and electric field operators. This offers a way for direct experimental detection of chirality by nuclear magneto-electric resonance (NMER). This method uses both resonant magnetic and electric radiofrequency (RF) fields. The weakness of the chiral interaction though requires a large electric RF field and a small transverse RF magnetic field over the sample volume, which is a non-trivial constraint. In this study, we present a detailed study of the NMER concept and a possible experimental realization based on a loop-gap resonator. For this original device, the basic principle and numerical studies as well as fabrication and measurements of the frequency dependence of the scattering parameter are reported. By simulating the NMER spin dynamics for our device and taking the F-19 NMER signal of enantiomer-pure 1,1,1-trifluoropropan-2-ol, we predict a chirality induced NMER signal that accounts for 1%-5% of the standard achiral NMR signal. Published by AIP Publishing.

pf

DOI [BibTex]

DOI [BibTex]


Active Nanorheology with Plasmonics
Active Nanorheology with Plasmonics

Jeong, H. H., Mark, A. G., Lee, T., Alarcon-Correa, M., Eslami, S., Qiu, T., Gibbs, J. G., Fischer, P.

Nano Letters, 16(8):4887-4894, July 2016 (article)

Abstract
Nanoplasmonic systems are valued for their strong optical response and their small size. Most plasmonic sensors and systems to date have been rigid and passive. However, rendering these structures dynamic opens new possibilities for applications. Here we demonstrate that dynamic plasmonic nanoparticles can be used as mechanical sensors to selectively probe the rheological properties of a fluid in situ at the nanoscale and in microscopic volumes. We fabricate chiral magneto-plasmonic nanocolloids that can be actuated by an external magnetic field, which in turn allows for the direct and fast modulation of their distinct optical response. The method is robust and allows nanorheological measurements with a mechanical sensitivity of similar to 0.1 cP, even in strongly absorbing fluids with an optical density of up to OD similar to 3 (similar to 0.1% light transmittance) and in the presence of scatterers (e.g., 50% v/v red blood cells).

pf

DOI [BibTex]

DOI [BibTex]


Implications of Action-Oriented Paradigm Shifts in Cognitive Science
Implications of Action-Oriented Paradigm Shifts in Cognitive Science

Dominey, P. F., Prescott, T. J., Bohg, J., Engel, A. K., Gallagher, S., Heed, T., Hoffmann, M., Knoblich, G., Prinz, W., Schwartz, A.

In The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science, 18, pages: 333-356, 20, Strüngmann Forum Reports, vol. 18, J. Lupp, series editor, (Editors: Andreas K. Engel and Karl J. Friston and Danica Kragic), The MIT Press, 18th Ernst Strüngmann Forum, May 2016 (incollection) In press

Abstract
An action-oriented perspective changes the role of an individual from a passive observer to an actively engaged agent interacting in a closed loop with the world as well as with others. Cognition exists to serve action within a landscape that contains both. This chapter surveys this landscape and addresses the status of the pragmatic turn. Its potential influence on science and the study of cognition are considered (including perception, social cognition, social interaction, sensorimotor entrainment, and language acquisition) and its impact on how neuroscience is studied is also investigated (with the notion that brains do not passively build models, but instead support the guidance of action). A review of its implications in robotics and engineering includes a discussion of the application of enactive control principles to couple action and perception in robotics as well as the conceptualization of system design in a more holistic, less modular manner. Practical applications that can impact the human condition are reviewed (e.g. educational applications, treatment possibilities for developmental and psychopathological disorders, the development of neural prostheses). All of this foreshadows the potential societal implications of the pragmatic turn. The chapter concludes that an action-oriented approach emphasizes a continuum of interaction between technical aspects of cognitive systems and robotics, biology, psychology, the social sciences, and the humanities, where the individual is part of a grounded cultural system.

am

The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science 18th Ernst Strüngmann Forum Bibliography Chapter link (url) [BibTex]

The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science 18th Ernst Strüngmann Forum Bibliography Chapter link (url) [BibTex]


Learning Action-Perception Cycles in Robotics: A Question of Representations and Embodiment
Learning Action-Perception Cycles in Robotics: A Question of Representations and Embodiment

Bohg, J., Kragic, D.

In The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science, 18, pages: 309-320, 18, Strüngmann Forum Reports, vol. 18, J. Lupp, series editor, (Editors: Andreas K. Engel and Karl J. Friston and Danica Kragic), The MIT Press, 18th Ernst Strüngmann Forum, May 2016 (incollection) In press

Abstract
Since the 1950s, robotics research has sought to build a general-purpose agent capable of autonomous, open-ended interaction with realistic, unconstrained environments. Cognition is perceived to be at the core of this process, yet understanding has been challenged because cognition is referred to differently within and across research areas, and is not clearly defined. The classic robotics approach is decomposition into functional modules which perform planning, reasoning, and problem-solving or provide input to these mechanisms. Although advancements have been made and numerous success stories reported in specific niches, this systems-engineering approach has not succeeded in building such a cognitive agent. The emergence of an action-oriented paradigm offers a new approach: action and perception are no longer separable into functional modules but must be considered in a complete loop. This chapter reviews work on different mechanisms for action- perception learning and discusses the role of embodiment in the design of the underlying representations and learning. It discusses the evaluation of agents and suggests the development of a new embodied Turing Test. Appropriate scenarios need to be devised in addition to current competitions, so that abilities can be tested over long time periods.

am

18th Ernst Strüngmann Forum The Pragmatic Turn- Toward Action-Oriented Views in Cognitive Science Bibliography Chapter link (url) [BibTex]

18th Ernst Strüngmann Forum The Pragmatic Turn- Toward Action-Oriented Views in Cognitive Science Bibliography Chapter link (url) [BibTex]


Dispersion and shape engineered plasmonic nanosensors
Dispersion and shape engineered plasmonic nanosensors

Jeong, H. H., Mark, A. G., Alarcon-Correa, M., Kim, I., Oswald, P., Lee, T. C., Fischer, P.

Nature Communications, 7, pages: 11331, March 2016 (article)

Abstract
Biosensors based on the localized surface plasmon resonance (LSPR) of individual metallic nanoparticles promise to deliver modular, low-cost sensing with high-detection thresholds. However, they continue to suffer from relatively low sensitivity and figures of merit (FOMs). Herein we introduce the idea of sensitivity enhancement of LSPR sensors through engineering of the material dispersion function. Employing dispersion and shape engineering of chiral nanoparticles leads to remarkable refractive index sensitivities (1,091 nmRIU(-1) at lambda = 921 nm) and FOMs (>2,800 RIU-1). A key feature is that the polarization-dependent extinction of the nanoparticles is now characterized by rich spectral features, including bipolar peaks and nulls, suitable for tracking refractive index changes. This sensing modality offers strong optical contrast even in the presence of highly absorbing media, an important consideration for use in complex biological media with limited transmission. The technique is sensitive to surface-specific binding events which we demonstrate through biotin-avidin surface coupling.

pf

link (url) DOI [BibTex]


Magnetic Propulsion of Microswimmers with DNA-Based Flagellar Bundles
Magnetic Propulsion of Microswimmers with DNA-Based Flagellar Bundles

Maier, A. M., Weig, C., Oswald, P., Frey, E., Fischer, P., Liedl, T.

Nano Letters, 16(2):906-910, January 2016 (article)

Abstract
We show that DNA-based self-assembly can serve as a general and flexible tool to construct artificial flagella of several micrometers in length and only tens of nanometers in diameter. By attaching the DNA flagella to biocompatible magnetic microparticles, we provide a proof of concept demonstration of hybrid structures that, when rotated in an external magnetic field, propel by means of a flagellar bundle, similar to self-propelling peritrichous bacteria. Our theoretical analysis predicts that flagellar bundles that possess a length-dependent bending stiffness should exhibit a superior swimming speed compared to swimmers with a single appendage. The DNA self-assembly method permits the realization of these improved flagellar bundles in good agreement with our quantitative model. DNA flagella with well-controlled shape could fundamentally increase the functionality of fully biocompatible nanorobots and extend the scope and complexity of active materials.

pf

DOI [BibTex]

DOI [BibTex]


Gaussian Process-Based Predictive Control for Periodic Error Correction
Gaussian Process-Based Predictive Control for Periodic Error Correction

Klenske, E. D., Zeilinger, M., Schölkopf, B., Hennig, P.

IEEE Transactions on Control Systems Technology , 24(1):110-121, 2016 (article)

ei pn

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
Probabilistic Inference for Determining Options in Reinforcement Learning

Daniel, C., van Hoof, H., Peters, J., Neumann, G.

Machine Learning, Special Issue, 104(2):337-357, (Editors: Gärtner, T., Nanni, M., Passerini, A. and Robardet, C.), European Conference on Machine Learning im Machine Learning, Journal Track, 2016, Best Student Paper Award of ECML-PKDD 2016 (article)

am ei

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Dual Control for Approximate Bayesian Reinforcement Learning
Dual Control for Approximate Bayesian Reinforcement Learning

Klenske, E. D., Hennig, P.

Journal of Machine Learning Research, 17(127):1-30, 2016 (article)

ei pn

PDF link (url) [BibTex]

PDF link (url) [BibTex]


no image
Locally Weighted Regression for Control

Ting, J., Meier, F., Vijayakumar, S., Schaal, S.

In Encyclopedia of Machine Learning and Data Mining, pages: 1-14, Springer US, Boston, MA, 2016 (inbook)

am

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Event-based Sampling for Reducing Communication Load in Realtime Human Motion Analysis by Wireless Inertial Sensor Networks

Laidig, D., Trimpe, S., Seel, T.

Current Directions in Biomedical Engineering, 2(1):711-714, De Gruyter, 2016 (article)

am ics

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
Momentum Control with Hierarchical Inverse Dynamics on a Torque-Controlled Humanoid

Herzog, A., Rotella, N., Mason, S., Grimminger, F., Schaal, S., Righetti, L.

Autonomous Robots, 40(3):473-491, 2016 (article)

Abstract
Hierarchical inverse dynamics based on cascades of quadratic programs have been proposed for the control of legged robots. They have important benefits but to the best of our knowledge have never been implemented on a torque controlled humanoid where model inaccuracies, sensor noise and real-time computation requirements can be problematic. Using a reformulation of existing algorithms, we propose a simplification of the problem that allows to achieve real-time control. Momentum-based control is integrated in the task hierarchy and a LQR design approach is used to compute the desired associated closed-loop behavior and improve performance. Extensive experiments on various balancing and tracking tasks show very robust performance in the face of unknown disturbances, even when the humanoid is standing on one foot. Our results demonstrate that hierarchical inverse dynamics together with momentum control can be efficiently used for feedback control under real robot conditions.

am mg

link (url) DOI [BibTex]


no image
Bioinspired Motor Control for Articulated Robots [From the Guest Editors]

Vitiello, Nicola, Ijspeert, Auke J, Schaal, S.

IEEE Robotics {\&} Automation Magazine, 23(1):20-21, 2016 (article)

am

[BibTex]

[BibTex]

2008


no image
Learning to control in operational space

Peters, J., Schaal, S.

International Journal of Robotics Research, 27, pages: 197-212, 2008, clmc (article)

Abstract
One of the most general frameworks for phrasing control problems for complex, redundant robots is operational space control. However, while this framework is of essential importance for robotics and well-understood from an analytical point of view, it can be prohibitively hard to achieve accurate control in face of modeling errors, which are inevitable in com- plex robots, e.g., humanoid robots. In this paper, we suggest a learning approach for opertional space control as a direct inverse model learning problem. A first important insight for this paper is that a physically cor- rect solution to the inverse problem with redundant degrees-of-freedom does exist when learning of the inverse map is performed in a suitable piecewise linear way. The second crucial component for our work is based on the insight that many operational space controllers can be understood in terms of a constrained optimal control problem. The cost function as- sociated with this optimal control problem allows us to formulate a learn- ing algorithm that automatically synthesizes a globally consistent desired resolution of redundancy while learning the operational space controller. From the machine learning point of view, this learning problem corre- sponds to a reinforcement learning problem that maximizes an immediate reward. We employ an expectation-maximization policy search algorithm in order to solve this problem. Evaluations on a three degrees of freedom robot arm are used to illustrate the suggested approach. The applica- tion to a physically realistic simulator of the anthropomorphic SARCOS Master arm demonstrates feasibility for complex high degree-of-freedom robots. We also show that the proposed method works in the setting of learning resolved motion rate control on real, physical Mitsubishi PA-10 medical robotics arm.

am ei

link (url) DOI [BibTex]

2008


link (url) DOI [BibTex]


Voltage-Controllable Magnetic Composite Based on Multifunctional Polyethylene Microparticles
Voltage-Controllable Magnetic Composite Based on Multifunctional Polyethylene Microparticles

Ghosh, A., Sheridon, N. K., Fischer, P.

SMALL, 4(11):1956-1958, 2008 (article)

pf

DOI [BibTex]


no image
Adaptation to a sub-optimal desired trajectory

M. Mistry, E. A. G. L. T. Y. S. S. M. K.

Advances in Computational Motor Control VII, Symposium at the Society for Neuroscience Meeting, Washington DC, 2008, 2008, clmc (article)

am

PDF [BibTex]

PDF [BibTex]


no image
Operational space control: A theoretical and emprical comparison

Nakanishi, J., Cory, R., Mistry, M., Peters, J., Schaal, S.

International Journal of Robotics Research, 27(6):737-757, 2008, clmc (article)

Abstract
Dexterous manipulation with a highly redundant movement system is one of the hallmarks of hu- man motor skills. From numerous behavioral studies, there is strong evidence that humans employ compliant task space control, i.e., they focus control only on task variables while keeping redundant degrees-of-freedom as compliant as possible. This strategy is robust towards unknown disturbances and simultaneously safe for the operator and the environment. The theory of operational space con- trol in robotics aims to achieve similar performance properties. However, despite various compelling theoretical lines of research, advanced operational space control is hardly found in actual robotics imple- mentations, in particular new kinds of robots like humanoids and service robots, which would strongly profit from compliant dexterous manipulation. To analyze the pros and cons of different approaches to operational space control, this paper focuses on a theoretical and empirical evaluation of different methods that have been suggested in the literature, but also some new variants of operational space controllers. We address formulations at the velocity, acceleration and force levels. First, we formulate all controllers in a common notational framework, including quaternion-based orientation control, and discuss some of their theoretical properties. Second, we present experimental comparisons of these approaches on a seven-degree-of-freedom anthropomorphic robot arm with several benchmark tasks. As an aside, we also introduce a novel parameter estimation algorithm for rigid body dynamics, which ensures physical consistency, as this issue was crucial for our successful robot implementations. Our extensive empirical results demonstrate that one of the simplified acceleration-based approaches can be advantageous in terms of task performance, ease of parameter tuning, and general robustness and compliance in face of inevitable modeling errors.

am

link (url) [BibTex]

link (url) [BibTex]


no image
A library for locally weighted projection regression

Klanke, S., Vijayakumar, S., Schaal, S.

Journal of Machine Learning Research, 9, pages: 623-626, 2008, clmc (article)

Abstract
In this paper we introduce an improved implementation of locally weighted projection regression (LWPR), a supervised learning algorithm that is capable of handling high-dimensional input data. As the key features, our code supports multi-threading, is available for multiple platforms, and provides wrappers for several programming languages.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Adaptive stair-climbing behaviour with a hybrid legged-wheeled robot

Eich, M., Grimminger, F., Kirchner, F.

In Advances In Mobile Robotics, pages: 768-775, World Scientific, August 2008 (incollection)

am

DOI [BibTex]

DOI [BibTex]


no image
Optimization strategies in human reinforcement learning

Hoffmann, H., Theodorou, E., Schaal, S.

Advances in Computational Motor Control VII, Symposium at the Society for Neuroscience Meeting, Washington DC, 2008, 2008, clmc (article)

am

PDF [BibTex]

PDF [BibTex]

2005


no image
Composite adaptive control with locally weighted statistical learning

Nakanishi, J., Farrell, J. A., Schaal, S.

Neural Networks, 18(1):71-90, January 2005, clmc (article)

Abstract
This paper introduces a provably stable learning adaptive control framework with statistical learning. The proposed algorithm employs nonlinear function approximation with automatic growth of the learning network according to the nonlinearities and the working domain of the control system. The unknown function in the dynamical system is approximated by piecewise linear models using a nonparametric regression technique. Local models are allocated as necessary and their parameters are optimized on-line. Inspired by composite adaptive control methods, the proposed learning adaptive control algorithm uses both the tracking error and the estimation error to update the parameters. We first discuss statistical learning of nonlinear functions, and motivate our choice of the locally weighted learning framework. Second, we begin with a class of first order SISO systems for theoretical development of our learning adaptive control framework, and present a stability proof including a parameter projection method that is needed to avoid potential singularities during adaptation. Then, we generalize our adaptive controller to higher order SISO systems, and discuss further extension to MIMO problems. Finally, we evaluate our theoretical control framework in numerical simulations to illustrate the effectiveness of the proposed learning adaptive controller for rapid convergence and high accuracy of control.

am

link (url) [BibTex]

2005


link (url) [BibTex]


Nonlinear optical spectroscopy of chiral molecules
Nonlinear optical spectroscopy of chiral molecules

Fischer, P., Hache, F.

CHIRALITY, 17(8):421-437, 2005 (article)

Abstract
We review nonlinear optical processes that are specific to chiral molecules in solution and on surfaces. In contrast to conventional natural optical activity phenomena, which depend linearly on the electric field strength of the optical field, we discuss how optical processes that are nonlinear (quadratic, cubic, and quartic) functions of the electromagnetic field strength may probe optically active centers and chiral vibrations. We show that nonlinear techniques open entirely new ways of exploring chirality in chemical and biological systems: The cubic processes give rise to nonlinear circular dichroism and nonlinear optical rotation and make it possible to observe dynamic chiral processes at ultrafast time scales. The quadratic second-harmonic and sum-frequency-generation phenomena and the quartic processes may arise entirely in the electric-dipole approximation and do not require the use of circularly polarized light to detect chirality: They provide surface selectivity and their observables can be relatively much larger than in linear optical activity. These processes also give rise to the generation of light at a new color, and in liquids this frequency conversion only occurs if the solution is optically active. We survey recent chiral nonlinear optical experiments and give examples of their application to problems of biophysical interest. (C) 2005 Wiley-Liss, Inc.

pf

DOI [BibTex]

DOI [BibTex]


Negative refraction at optical frequencies in nonmagnetic two-component molecular media
Negative refraction at optical frequencies in nonmagnetic two-component molecular media

Chen, Y., Fischer, P., Wise, F.

PHYSICAL REVIEW LETTERS, 95(6), 2005 (article)

Abstract
There is significant motivation to develop media with negative refractive indices at optical frequencies, but efforts in this direction are hampered by the weakness of the magnetic response at such frequencies. We show theoretically that a nonmagnetic medium with two atomic or molecular constituents can exhibit a negative refractive index. A negative index is possible even when the real parts of both the permittivity and permeability are positive. This surprising result provides a route to isotropic negative-index media at optical frequencies.

pf

DOI [BibTex]

DOI [BibTex]


no image
A model of smooth pursuit based on learning of the target dynamics using only retinal signals

Shibata, T., Tabata, H., Schaal, S., Kawato, M.

Neural Networks, 18, pages: 213-225, 2005, clmc (article)

Abstract
While the predictive nature of the primate smooth pursuit system has been evident through several behavioural and neurophysiological experiments, few models have attempted to explain these results comprehensively. The model we propose in this paper in line with previous models employing optimal control theory; however, we hypothesize two new issues: (1) the medical superior temporal (MST) area in the cerebral cortex implements a recurrent neural network (RNN) in order to predict the current or future target velocity, and (2) a forward model of the target motion is acquired by on-line learning. We use stimulation studies to demonstrate how our new model supports these hypotheses.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Parametric and Non-Parametric approaches for nonlinear tracking of moving objects

Hidaka, Y, Theodorou, E.

Technical Report-2005-1, 2005, clmc (article)

am

PDF [BibTex]

PDF [BibTex]

2004


no image
Discovering optimal imitation strategies

Billard, A., Epars, Y., Calinon, S., Cheng, G., Schaal, S.

Robotics and Autonomous Systems, 47(2-3):68-77, 2004, clmc (article)

Abstract
This paper develops a general policy for learning relevant features of an imitation task. We restrict our study to imitation of manipulative tasks or of gestures. The imitation process is modeled as a hierarchical optimization system, which minimizes the discrepancy between two multi-dimensional datasets. To classify across manipulation strategies, we apply a probabilistic analysis to data in Cartesian and joint spaces. We determine a general metric that optimizes the policy of task reproduction, following strategy determination. The model successfully discovers strategies in six different imitative tasks and controls task reproduction by a full body humanoid robot.

am

[BibTex]

2004


[BibTex]


no image
Rhythmic movement is not discrete

Schaal, S., Sternad, D., Osu, R., Kawato, M.

Nature Neuroscience, 7(10):1137-1144, 2004, clmc (article)

Abstract
Rhythmic movements, like walking, chewing, or scratching, are phylogenetically old mo-tor behaviors found in many organisms, ranging from insects to primates. In contrast, discrete movements, like reaching, grasping, or kicking, are behaviors that have reached sophistication primarily in younger species, particularly in primates. Neurophysiological and computational research on arm motor control has focused almost exclusively on dis-crete movements, essentially assuming similar neural circuitry for rhythmic tasks. In con-trast, many behavioral studies focused on rhythmic models, subsuming discrete move-ment as a special case. Here, using a human functional neuroimaging experiment, we show that in addition to areas activated in rhythmic movement, discrete movement in-volves several higher cortical planning areas, despite both movement conditions were confined to the same single wrist joint. These results provide the first neuroscientific evi-dence that rhythmic arm movement cannot be part of a more general discrete movement system, and may require separate neurophysiological and theoretical treatment.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Learning from demonstration and adaptation of biped locomotion

Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., Kawato, M.

Robotics and Autonomous Systems, 47(2-3):79-91, 2004, clmc (article)

Abstract
In this paper, we introduce a framework for learning biped locomotion using dynamical movement primitives based on non-linear oscillators. Our ultimate goal is to establish a design principle of a controller in order to achieve natural human-like locomotion. We suggest dynamical movement primitives as a central pattern generator (CPG) of a biped robot, an approach we have previously proposed for learning and encoding complex human movements. Demonstrated trajectories are learned through movement primitives by locally weighted regression, and the frequency of the learned trajectories is adjusted automatically by a novel frequency adaptation algorithmbased on phase resetting and entrainment of coupled oscillators. Numerical simulations and experimental implementation on a physical robot demonstrate the effectiveness of the proposed locomotioncontroller.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Towards Tractable Parameter-Free Statistical Learning (Phd Thesis)

D’Souza, A

Department of Computer Science, University of Southern California, Los Angeles, 2004, clmc (phdthesis)

am

link (url) [BibTex]

link (url) [BibTex]


no image
Feedback error learning and nonlinear adaptive control

Nakanishi, J., Schaal, S.

Neural Networks, 17(10):1453-1465, 2004, clmc (article)

Abstract
In this paper, we present our theoretical investigations of the technique of feedback error learning (FEL) from the viewpoint of adaptive control. We first discuss the relationship between FEL and nonlinear adaptive control with adaptive feedback linearization, and show that FEL can be interpreted as a form of nonlinear adaptive control. Second, we present a Lyapunov analysis suggesting that the condition of strictly positive realness (SPR) associated with the tracking error dynamics is a sufficient condition for asymptotic stability of the closed-loop dynamics. Specifically, for a class of second order SISO systems, we show that this condition reduces to KD^2 > KP; where KP and KD are positive position and velocity feedback gains, respectively. Moreover, we provide a ÔpassivityÕ-based stability analysis which suggests that SPR of the tracking error dynamics is a necessary and sufficient condition for asymptotic hyperstability. Thus, the condition KD^2>KP mentioned above is not only a sufficient but also necessary condition to guarantee asymptotic hyperstability of FEL, i.e. the tracking error is bounded and asymptotically converges to zero. As a further point, we explore the adaptive control and FEL framework for feedforward control formulations, and derive an additional sufficient condition for asymptotic stability in the sense of Lyapunov. Finally, we present numerical simulations to illustrate the stability properties of FEL obtained from our mathematical analysis.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Computational approaches to motor learning by imitation

Schaal, S., Ijspeert, A., Billard, A.

In The Neuroscience of Social Interaction, (1431):199-218, (Editors: Frith, C. D.;Wolpert, D.), Oxford University Press, Oxford, 2004, clmc (inbook)

Abstract
Movement imitation requires a complex set of mechanisms that map an observed movement of a teacher onto one's own movement apparatus. Relevant problems include movement recognition, pose estimation, pose tracking, body correspondence, coordinate transformation from external to egocentric space, matching of observed against previously learned movement, resolution of redundant degrees-of-freedom that are unconstrained by the observation, suitable movement representations for imitation, modularization of motor control, etc. All of these topics by themselves are active research problems in computational and neurobiological sciences, such that their combination into a complete imitation system remains a daunting undertaking - indeed, one could argue that we need to understand the complete perception-action loop. As a strategy to untangle the complexity of imitation, this paper will examine imitation purely from a computational point of view, i.e. we will review statistical and mathematical approaches that have been suggested for tackling parts of the imitation problem, and discuss their merits, disadvantages and underlying principles. Given the focus on action recognition of other contributions in this special issue, this paper will primarily emphasize the motor side of imitation, assuming that a perceptual system has already identified important features of a demonstrated movement and created their corresponding spatial information. Based on the formalization of motor control in terms of control policies and their associated performance criteria, useful taxonomies of imitation learning can be generated that clarify different approaches and future research directions.

am

link (url) [BibTex]

link (url) [BibTex]

2002


no image
Forward models in visuomotor control

Mehta, B., Schaal, S.

J Neurophysiol, 88(2):942-53, August 2002, clmc (article)

Abstract
In recent years, an increasing number of research projects investigated whether the central nervous system employs internal models in motor control. While inverse models in the control loop can be identified more readily in both motor behavior and the firing of single neurons, providing direct evidence for the existence of forward models is more complicated. In this paper, we will discuss such an identification of forward models in the context of the visuomotor control of an unstable dynamic system, the balancing of a pole on a finger. Pole balancing imposes stringent constraints on the biological controller, as it needs to cope with the large delays of visual information processing while keeping the pole at an unstable equilibrium. We hypothesize various model-based and non-model-based control schemes of how visuomotor control can be accomplished in this task, including Smith Predictors, predictors with Kalman filters, tapped-delay line control, and delay-uncompensated control. Behavioral experiments with human participants allow exclusion of most of the hypothesized control schemes. In the end, our data support the existence of a forward model in the sensory preprocessing loop of control. As an important part of our research, we will provide a discussion of when and how forward models can be identified and also the possible pitfalls in the search for forward models in control.

am

link (url) [BibTex]

2002


link (url) [BibTex]


Chirality-specific nonlinear spectroscopies in isotropic media
Chirality-specific nonlinear spectroscopies in isotropic media

Fischer, P., Albrecht, A.

BULLETIN OF THE CHEMICAL SOCIETY OF JAPAN, 75(5):1119-1124, 2002, 10th International Conference on Time-Resolved Vibrational Spectroscopy (TRVS 2001), OKAZZAKI, JAPAN, MAY 21-25, 2001 (article)

Abstract
Sum or difference frequency generation (SFG or DFG) in isotropic media is in the electric-dipole approximation only symmetry allowed for optically active systems. The hyperpolarizability giving rise to these three-wave mixing processes features only one isotropic component. It factorizes into two terms, an energy (denominator) factor and a triple product of transition moments. These forbid degenerate SFG, i.e., second harmonic generation, as well as the existence of the linear electrooptic effect (Pockels effect) in isotropic media. This second order response also has no static limit, which leads to particularly strong resonance phenomena that are qualitatively different from those usually seen in the ubiquitous even-wave mixing spectroscopies. In particular, the participation of two (not the usual one) excited states is essential to achieve dramatic resonance enhancement, We report our first efforts to see such resonantly enhanced chirality specific SFG.

pf

DOI [BibTex]

DOI [BibTex]


The chiral specificity of sum-frequency generation in solutions
The chiral specificity of sum-frequency generation in solutions

Fischer, P., Beckwitt, K., Wise, F., Albrecht, A.

CHEMICAL PHYSICS LETTERS, 352(5-6):463-468, 2002 (article)

Abstract
Sum-frequency generation in isotropic media is in the electric-dipole approximation the only symmetry allowed for chiral systems. We demonstrate that the sum-frequency intensity from an optically active liquid depends quadratically on the difference in concentration of the two enantiomers. The dominant contribution to the signal is found to be due to the chirality specific electric-dipolar three-wave mixing nonlinearity. Selecting the polarization of all fields allows the chiral electric-dipolar contributions to the bulk sum-frequency signal to be discerned from any achiral magnetic-dipolar and electric-quadrupolar contributions. (C) 2002 Published by Elsevier Science B.V.

pf

DOI [BibTex]

DOI [BibTex]


On optical rectification in isotropic media
On optical rectification in isotropic media

Fischer, P., Albrecht, A.

LASER PHYSICS, 12(8):1177-1181, 2002 (article)

Abstract
Coherent nonlinear optical processes at second-order are only electric-dipole allowed in isotropic media that are optically active. Sum-frequency generation in chiral liquids has recently been observed, and difference-frequency and optical rectification have been predicted to exist in isotropic chiral media. Both Rayleigh-Schrodinger perturbation theory and the density matrix approach are used to discuss the quantum-chemical basis of optical rectification in optically active liquids. For pinene we compute the corresponding orientationally averaged hyperpolarizability, and estimate the light-induced dc electric polarization and the consequent voltage across a measuring capacitor it may give rise to near resonance.

pf

[BibTex]

[BibTex]


no image
Learning robot control

Schaal, S.

In The handbook of brain theory and neural networks, 2nd Edition, pages: 983-987, 2, (Editors: Arbib, M. A.), MIT Press, Cambridge, MA, 2002, clmc (inbook)

Abstract
This is a review article on learning control in robots.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Arm and hand movement control

Schaal, S.

In The handbook of brain theory and neural networks, 2nd Edition, pages: 110-113, 2, (Editors: Arbib, M. A.), MIT Press, Cambridge, MA, 2002, clmc (inbook)

Abstract
This is a review article on computational and biological research on arm and hand control.

am

link (url) [BibTex]

link (url) [BibTex]


no image
Scalable techniques from nonparameteric statistics for real-time robot learning

Schaal, S., Atkeson, C. G., Vijayakumar, S.

Applied Intelligence, 17(1):49-60, 2002, clmc (article)

Abstract
Locally weighted learning (LWL) is a class of techniques from nonparametric statistics that provides useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of robotic systems. This paper introduces several LWL algorithms that have been tested successfully in real-time learning of complex robot tasks. We discuss two major classes of LWL, memory-based LWL and purely incremental LWL that does not need to remember any data explicitly. In contrast to the traditional belief that LWL methods cannot work well in high-dimensional spaces, we provide new algorithms that have been tested on up to 90 dimensional learning problems. The applicability of our LWL algorithms is demonstrated in various robot learning examples, including the learning of devil-sticking, pole-balancing by a humanoid robot arm, and inverse-dynamics learning for a seven and a 30 degree-of-freedom robot. In all these examples, the application of our statistical neural networks techniques allowed either faster or more accurate acquisition of motor control than classical control engineering.

am

link (url) [BibTex]

link (url) [BibTex]

2001


Isotropic second-order nonlinear optical susceptibilities
Isotropic second-order nonlinear optical susceptibilities

Fischer, P., Buckingham, A., Albrecht, A.

PHYSICAL REVIEW A, 64(5), 2001 (article)

Abstract
The second-order nonlinear optical susceptibility, in the electric dipole approximation, is only nonvanishing for materials that are noncentrosymmetric. Should the medium be isotropic, then only a chiral system. such as an optically active liquid, satisfies this symmetry requirement. We derive the quantum-mechanical form of the isotropic component of the sum- and difference-frequency susceptibility and discuss its unusual spectral properties. We show that any coherent second-order nonlinear optical process in a system of randomly oriented molecules requires the medium to be chiral. and the incident frequencies to be different and nonzero. Furthermore, a minimum of two nondegenerate excited molecular states are needed for the isotropic part of the susceptibility to be nonvanishing. The rotationally invariant susceptibility is zero in the static field limit and shows exceptionally sensitive resonance and dephasing effects that are particular to chiral centers.

pf

DOI [BibTex]

2001


DOI [BibTex]


Reply to ``Comment on `Phenomenological damping in optical response tensors'{''}
Reply to “Comment on ‘Phenomenological damping in optical response tensors’”

Buckingham, A., Fischer, P.

PHYSICAL REVIEW A, 63(4), 2001 (article)

Abstract
We show that damping factors must not be incorporated in the perturbation of the ground state by a static electric field. If they are included, as in the theory of Stedman et al. {[}preceding Comment. Phys. Rev. A 63, 047801 (2001)], then there would be an electric dipole in the y direction induced in a hydrogen atom in the M-s = + 1/2 state by a static electric field in the x direction. Such a dipole is excluded by symmetry.

pf

DOI [BibTex]


no image
Synchronized robot drumming by neural oscillator

Kotosaka, S., Schaal, S.

Journal of the Robotics Society of Japan, 19(1):116-123, 2001, clmc (article)

Abstract
Sensory-motor integration is one of the key issues in robotics. In this paper, we propose an approach to rhythmic arm movement control that is synchronized with an external signal based on exploiting a simple neural oscillator network. Trajectory generation by the neural oscillator is a biologically inspired method that can allow us to generate a smooth and continuous trajectory. The parameter tuning of the oscillators is used to generate a synchronized movement with wide intervals. We adopted the method for the drumming task as an example task. By using this method, the robot can realize synchronized drumming with wide drumming intervals in real time. The paper also shows the experimental results of drumming by a humanoid robot.

am

[BibTex]

[BibTex]


no image
Origins and violations of the 2/3 power law in rhythmic 3D movements

Schaal, S., Sternad, D.

Experimental Brain Research, 136, pages: 60-72, 2001, clmc (article)

Abstract
The 2/3 power law, the nonlinear relationship between tangential velocity and radius of curvature of the endeffector trajectory, has been suggested as a fundamental constraint of the central nervous system in the formation of rhythmic endpoint trajectories. However, studies on the 2/3 power law have largely been confined to planar drawing patterns of relatively small size. With the hypothesis that this strategy overlooks nonlinear effects that are constitutive in movement generation, the present experiments tested the validity of the power law in elliptical patterns which were not confined to a planar surface and which were performed by the unconstrained 7-DOF arm with significant variations in pattern size and workspace orientation. Data were recorded from five human subjects where the seven joint angles and the endpoint trajectories were analyzed. Additionally, an anthropomorphic 7-DOF robot arm served as a "control subject" whose endpoint trajectories were generated on the basis of the human joint angle data, modeled as simple harmonic oscillations. Analyses of the endpoint trajectories demonstrate that the power law is systematically violated with increasing pattern size, in both exponent and the goodness of fit. The origins of these violations can be explained analytically based on smooth rhythmic trajectory formation and the kinematic structure of the human arm. We conclude that in unconstrained rhythmic movements, the power law seems to be a by-product of a movement system that favors smooth trajectories, and that it is unlikely to serve as a primary movement generating principle. Our data rather suggests that subjects employed smooth oscillatory pattern generators in joint space to realize the required movement patterns.

am

link (url) [BibTex]

link (url) [BibTex]