Header logo is


2007


no image
Machine Learning of Motor Skills for Robotics

Peters, J.

University of Southern California, Los Angeles, CA, USA, University of Southern California, Los Angeles, CA, USA, 2007, clmc (phdthesis)

Abstract
Autonomous robots that can assist humans in situations of daily life have been a long standing vision of robotics, artificial intelligence, and cognitive sciences. A first step towards this goal is to create robots that can accomplish a multitude of different tasks, triggered by environmental context or higher level instruction. Early approaches to this goal during the heydays of artificial intelligence research in the late 1980s, however, made it clear that an approach purely based on reasoning and human insights would not be able to model all the perceptuomotor tasks that a robot should fulfill. Instead, new hope was put in the growing wake of machine learning that promised fully adaptive control algorithms which learn both by observation and trial-and-error. However, to date, learning techniques have yet to fulfill this promise as only few methods manage to scale into the high-dimensional domains of manipulator robotics, or even the new upcoming trend of humanoid robotics, and usually scaling was only achieved in precisely pre-structured domains. In this thesis, we investigate the ingredients for a general approach to motor skill learning in order to get one step closer towards human-like performance. For doing so, we study two major components for such an approach, i.e., firstly, a theoretically well-founded general approach to representing the required control structures for task representation and execution and, secondly, appropriate learning algorithms which can be applied in this setting. As a theoretical foundation, we first study a general framework to generate control laws for real robots with a particular focus on skills represented as dynamical systems in differential constraint form. We present a point-wise optimal control framework resulting from a generalization of Gauss' principle and show how various well-known robot control laws can be derived by modifying the metric of the employed cost function. The framework has been successfully applied to task space tracking control for holonomic systems for several different metrics on the anthropomorphic SARCOS Master Arm. In order to overcome the limiting requirement of accurate robot models, we first employ learning methods to find learning controllers for task space control. However, when learning to execute a redundant control problem, we face the general problem of the non-convexity of the solution space which can force the robot to steer into physically impossible configurations if supervised learning methods are employed without further consideration. This problem can be resolved using two major insights, i.e., the learning problem can be treated as locally convex and the cost function of the analytical framework can be used to ensure global consistency. Thus, we derive an immediate reinforcement learning algorithm from the expectation-maximization point of view which leads to a reward-weighted regression technique. This method can be used both for operational space control as well as general immediate reward reinforcement learning problems. We demonstrate the feasibility of the resulting framework on the problem of redundant end-effector tracking for both a simulated 3 degrees of freedom robot arm as well as for a simulated anthropomorphic SARCOS Master Arm. While learning to execute tasks in task space is an essential component to a general framework to motor skill learning, learning the actual task is of even higher importance, particularly as this issue is more frequently beyond the abilities of analytical approaches than execution. We focus on the learning of elemental tasks which can serve as the "building blocks of movement generation", called motor primitives. Motor primitives are parameterized task representations based on splines or nonlinear differential equations with desired attractor properties. While imitation learning of parameterized motor primitives is a relatively well-understood problem, the self-improvement by interaction of the system with the environment remains a challenging problem, tackled in the fourth chapter of this thesis. For pursuing this goal, we highlight the difficulties with current reinforcement learning methods, and outline both established and novel algorithms for the gradient-based improvement of parameterized policies. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm. In conclusion, in this thesis, we have contributed a general framework for analytically computing robot control laws which can be used for deriving various previous control approaches and serves as foundation as well as inspiration for our learning algorithms. We have introduced two classes of novel reinforcement learning methods, i.e., the Natural Actor-Critic and the Reward-Weighted Regression algorithm. These algorithms have been used in order to replace the analytical components of the theoretical framework by learned representations. Evaluations have been performed on both simulated and real robot arms.

am ei

[BibTex]

2007


[BibTex]


Thumb xl toc image
Observation of the Faraday effect via beam deflection in a longitudinal magnetic field

Ghosh, A., Hill, W., Fischer, P.

PHYSICAL REVIEW A, 76(5), 2007 (article)

Abstract
We show that magnetic-field-induced circular differential deflection of light can be observed in reflection or refraction at a single interface. The difference in the reflection or refraction angles between the two circular polarization components is a function of the magnetic-field strength and the Verdet constant, and permits the observation of the Faraday effect not via polarization rotation in transmission, but via changes in the propagation direction. Deflection measurements do not suffer from n-pi ambiguities and are shown to be another means to map magnetic fields with high axial resolution, or to determine the sign and magnitude of magnetic-field pulses in a single measurement.

pf

DOI [BibTex]


Thumb xl toc image
Circular differential double diffraction in chiral media

Ghosh, A., Fazal, F. M., Fischer, P.

OPTICS LETTERS, 32(13):1836-1838, 2007 (article)

Abstract
In an optically active liquid the diffraction angle depends on the circular polarization state of the incident light beam. We report the observation of circular differential diffraction in an isotropic chiral medium, and we demonstrate that double diffraction is an alternate means to determine the handedness (enantiomeric excess) of a solution. (c) 2007 Optical Society of America.

pf

DOI [BibTex]

DOI [BibTex]


no image
Bacterial flagella-based propulsion and on/off motion control of microscale objects

Behkam, B., Sitti, M.

Applied Physics Letters, 90(2):023902, AIP, 2007 (article)

pi

[BibTex]

[BibTex]


no image
Friction of partially embedded vertically aligned carbon nanofibers inside elastomers

Aksak, B., Sitti, M., Cassell, A., Li, J., Meyyappan, M., Callen, P.

Applied Physics Letters, 91(6):061906, AIP, 2007 (article)

pi

[BibTex]

[BibTex]


no image
Enhanced friction of elastomer microfiber adhesives with spatulate tips

Kim, S., Aksak, B., Sitti, M.

Applied Physics Letters, 91(22):221913, AIP, 2007 (article)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
The new robotics - towards human-centered machines

Schaal, S.

HFSP Journal Frontiers of Interdisciplinary Research in the Life Sciences, 1(2):115-126, 2007, clmc (article)

Abstract
Research in robotics has moved away from its primary focus on industrial applications. The New Robotics is a vision that has been developed in past years by our own university and many other national and international research instiutions and addresses how increasingly more human-like robots can live among us and take over tasks where our current society has shortcomings. Elder care, physical therapy, child education, search and rescue, and general assistance in daily life situations are some of the examples that will benefit from the New Robotics in the near future. With these goals in mind, research for the New Robotics has to embrace a broad interdisciplinary approach, ranging from traditional mathematical issues of robotics to novel issues in psychology, neuroscience, and ethics. This paper outlines some of the important research problems that will need to be resolved to make the New Robotics a reality.

am

link (url) [BibTex]

link (url) [BibTex]


Thumb xl ijcvflow2
On the spatial statistics of optical flow

Roth, S., Black, M. J.

International Journal of Computer Vision, 74(1):33-50, 2007 (article)

Abstract
We present an analysis of the spatial and temporal statistics of "natural" optical flow fields and a novel flow algorithm that exploits their spatial statistics. Training flow fields are constructed using range images of natural scenes and 3D camera motions recovered from hand-held and car-mounted video sequences. A detailed analysis of optical flow statistics in natural scenes is presented and machine learning methods are developed to learn a Markov random field model of optical flow. The prior probability of a flow field is formulated as a Field-of-Experts model that captures the spatial statistics in overlapping patches and is trained using contrastive divergence. This new optical flow prior is compared with previous robust priors and is incorporated into a recent, accurate algorithm for dense optical flow computation. Experiments with natural and synthetic sequences illustrate how the learned optical flow prior quantitatively improves flow accuracy and how it captures the rich spatial structure found in natural scene motion.

ps

pdf preprint pdf from publisher [BibTex]

pdf preprint pdf from publisher [BibTex]


no image
On the theory of magnetization dynamics of non-collinear spin systems in the s-d model

De Angeli, L.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Zur ab-initio Elektronentheorie des Magnetismus bei endlichen Temperaturen

Dietermann, F.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Röntgenzirkulardichroische Untersuchungen an ferromagnetischen verdünnten Halbleitersystemen

Tietze, T.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Low-dimensional Fe on vicinal Ir(997): Growth and magnetic properties

Kawwam, M.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Micromagnetic simulations of switching processes and the role of thermal fluctuations

Macke, S.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Hydrogen storage in metal-organic frameworks

Hirscher, M., Panella, B.

{Scripta Materialia}, 56, pages: 809-812, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Substrate-induced current anisotropy in YBa2Cu3O7-δthin films

Djupmyr, M., Albrecht, J.

{Physica C}, 460-462, pages: 1190-1191, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
A micellar approach to magnetic ultrahigh-density data-storage media: extending the limits of current colloidal methods

Ethirajan, A., Wiedwald, U., Boyen, H.-G., Kern, B., Han, L., Klimmer, A., Weigl, F., Kästle, G., Ziemann, P., Fauth, K., Cai, J., Behm, J., Romanyuk, A., Oelhafen, P., Walther, P., Biskupek, J., Kaiser, U.

{Advanced Materials}, 19, pages: 406-410, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Size dependence in the magnetization reversal of Fe/Gd multilayers on self-assembled arrays of nanospheres

Amaladass, E., Ludescher, B., Schütz, G., Tyliszczak, T., Eimüller, T.

{Applied Physics Letters}, 91, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Ma\ssgeschneiderte Wasserstoffspeicher

Hirscher, M., Panella, B.

{Nachrichten aus der Gdch-Energieinitiative}, (Sonderheft April 2007):12-13, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Reconstruction of historical alloys for pipe organs brings true Baroque music back to life.

Baretzky, B., Friesel, M., Straumal, B.

{MRS Bulletin}, 32, pages: 249-255, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Analysis of results from X-ray magnetic reflectometry for magnetic multilayer systems

Fähnle, M., Steiauf, D., Martosiswoyo, L., Goering, E., Brück, S., Schütz, G.

{Physical Review B}, 75, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Dramatic role of critical current anisotropy on flux avalanches in MgB2 films

Albrecht, J., Matveev, A. T., Strempfer, J., Habermeier, H.-U., Shantsev, D. V., Galperin, Y. M., Johansen, T. H.

{Physical Review Letters}, 98, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Transport properties of LCMO/YBCO hybrid structures

Soltan, S., Albrecht, J., Habermeier, H.-U.

{Materials Science and Engineering B}, 144, pages: 15-18, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Microscale and nanoscale robotics systems [grand challenges of robotics]

Sitti, M.

IEEE Robotics \& Automation Magazine, 14(1):53-60, IEEE, 2007 (article)

pi

[BibTex]

[BibTex]


no image
A new biomimetic adhesive for therapeutic capsule endoscope applications in the gastrointestinal tract

Glass, P., Sitti, M., Appasamy, R.

Gastrointestinal Endoscopy, 65(5):AB91, Mosby, 2007 (article)

pi

[BibTex]

[BibTex]


no image
Visual servoing-based autonomous 2-D manipulation of microparticles using a nanoprobe

Onal, C. D., Sitti, M.

IEEE Transactions on control systems technology, 15(5):842-852, IEEE, 2007 (article)

pi

[BibTex]

[BibTex]


Thumb xl arrayhd
Assistive technology and robotic control using MI ensemble-based neural interface systems in humans with tetraplegia

Donoghue, J. P., Nurmikko, A., Black, M. J., Hochberg, L.

Journal of Physiology, Special Issue on Brain Computer Interfaces, 579, pages: 603-611, 2007 (article)

Abstract
This review describes the rationale, early stage development, and initial human application of neural interface systems (NISs) for humans with paralysis. NISs are emerging medical devices designed to allowpersonswith paralysis to operate assistive technologies or to reanimatemuscles based upon a command signal that is obtained directly fromthe brain. Such systems require the development of sensors to detect brain signals, decoders to transformneural activity signals into a useful command, and an interface for the user.We review initial pilot trial results of an NIS that is based on an intracortical microelectrode sensor that derives control signals from the motor cortex.We review recent findings showing, first, that neurons engaged by movement intentions persist in motor cortex years after injury or disease to the motor system, and second, that signals derived from motor cortex can be used by persons with paralysis to operate a range of devices. We suggest that, with further development, this form of NIS holds promise as a useful new neurotechnology for those with limited motor function or communication.We also discuss the additional potential for neural sensors to be used in the diagnosis and management of various neurological conditions and as a new way to learn about human brain function.

ps

pdf preprint pdf from publisher DOI [BibTex]

pdf preprint pdf from publisher DOI [BibTex]


no image
Physisorption von Wasserstoff in neuen Materialien mit gro\sser spezifischer Oberfläche

Schmitz, B.

Universität Bonn, Bonn, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Cluster expansions in multicomponent systems: precise expansions from noisy databases

Diaz-Ortiz, A., Dosch, H., Drautz, R.

{Journal of Physics: Condensed Matter}, 19, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Towards spin injection into silicon

Dash, S. P.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Bestimmung der kritischen Schichtdicken ferromagnetischer Plättchen für Eindomänenverhalten

Soehnle, S.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Unusual propagation of magnetic avalanches in gold covered MgB2

Albrecht, J., Matveev, A. T., Habermeier, H.-U.

{Physica C}, 460-462, pages: 1245-1246, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Lowering of the L10 ordering temperature of FePt nanoparticles by He+ ion irradiation

Wiedwald, U., Klimmer, A., Kern, B., Han, L., Boyen, H.-G., Ziemann, P., Fauth, K.

{Applied Physics Letters}, 90, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Magnetic core shell nanoparticles characterized by X-ray absorption and magnetic circular dichroism

Fauth, K.

{Modern Physics Letters B}, 21(18):1179-1187, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Magnetic moment of Fe in oxide-free FePt nanoparticles

Dmitrieva, O., Spasova, M., Antoniak, C., Acet, M., Dumpich, G., Kästner, J., Farle, M., Fauth, K., Wiedwald, U., Boyen, H.-G., Ziemann, P.

{Physical Review B}, 76, 2007 (article)

mms

[BibTex]

[BibTex]


no image
The effect of bismuth segregation on the faceting of \Sigma3 and \Sigma9 coincidence boundaries in copper bicrystals

Straumal, B. B., Polyakov, S. A., Chang, L.-S., Mittemeijer, E. J.

{International Journal of Materials Research}, 98, pages: 451-456, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Hot isostatic pressing of Cu-Bi polycrystals with liquid-like grain boundary layers

Chang, L.-S., Straumal, B., Rabkin, E., Lojkowski, W., Gust, W.

{Acta Materialia}, 55, pages: 335-343, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Spatially resolved magnetic response in core shell nanoparticles

Fauth, K., Goering, E., Theil Kuhn, L.

{Modern Physics Letters B}, 21(18):1197-1200, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Interfaces in semiconductor/metal radial superlattices

Deneke, C., Sigle, W., Eigenthaler, U., van Aken, P. A., Schütz, G., Schmidt, O. G.

{Applied Physics Letters}, 90, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Live cell adhesion assay with attenuated total reflection infrared spectroscopy

Schmidt, M., Wolfram, T., Rumpler, M., Tripp, C. P., Grunze, M.

{Biointerphases}, 2(1):1-5, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Unusual Co moment reduction in the NiCoO/Co exchange bias system

Brück, S., Goering, E., Tang, Y. J., Schütz, G., Berkowitz, A. E.

{Journal of Magnetism and Magnetic Materials}, 310, pages: 2316-2318, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Adhesion of biologically inspired vertical and angled polymer microfiber arrays

Aksak, B., Murphy, M. P., Sitti, M.

Langmuir, 23(6):3322-3332, ACS Publications, 2007 (article)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Waalbot: An agile small-scale wall-climbing robot utilizing dry elastomer adhesives

Murphy, M. P., Sitti, M.

IEEE/ASME transactions on Mechatronics, 12(3):330-338, IEEE, 2007 (article)

pi

[BibTex]

[BibTex]


no image
Zeitaufgelöste Röntgenmikroskopie an magnetischen Mikrostrukturen

Puzic, A.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Ab initio calculations of adiabatic magnon spectra using the atomic-sphere aproximation for the spin direction

Singer, R., Dietermann, F., Steiauf, D., Fähnle, M.

{Physical Review B}, 76, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Vortex dynamics studied by time-resolved X-ray microscopy

Chou, K. W.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Resonante magnetische Reflektometrie an Ferromagnet/Paramagnet Heterostrukturen

Ferreras Paz, V.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Low-temperature thermal-desorption mass spectroscopy applied to investigate the hydrogen adsorption on porous materials

Panella, B., Hirscher, M., Ludescher, B.

{Microporous and Mesoporous Materials}, 103, pages: 230-234, 2007 (article)

mms

[BibTex]