Header logo is


2007


no image
Machine Learning of Motor Skills for Robotics

Peters, J.

University of Southern California, Los Angeles, CA, USA, University of Southern California, Los Angeles, CA, USA, 2007, clmc (phdthesis)

Abstract
Autonomous robots that can assist humans in situations of daily life have been a long standing vision of robotics, artificial intelligence, and cognitive sciences. A first step towards this goal is to create robots that can accomplish a multitude of different tasks, triggered by environmental context or higher level instruction. Early approaches to this goal during the heydays of artificial intelligence research in the late 1980s, however, made it clear that an approach purely based on reasoning and human insights would not be able to model all the perceptuomotor tasks that a robot should fulfill. Instead, new hope was put in the growing wake of machine learning that promised fully adaptive control algorithms which learn both by observation and trial-and-error. However, to date, learning techniques have yet to fulfill this promise as only few methods manage to scale into the high-dimensional domains of manipulator robotics, or even the new upcoming trend of humanoid robotics, and usually scaling was only achieved in precisely pre-structured domains. In this thesis, we investigate the ingredients for a general approach to motor skill learning in order to get one step closer towards human-like performance. For doing so, we study two major components for such an approach, i.e., firstly, a theoretically well-founded general approach to representing the required control structures for task representation and execution and, secondly, appropriate learning algorithms which can be applied in this setting. As a theoretical foundation, we first study a general framework to generate control laws for real robots with a particular focus on skills represented as dynamical systems in differential constraint form. We present a point-wise optimal control framework resulting from a generalization of Gauss' principle and show how various well-known robot control laws can be derived by modifying the metric of the employed cost function. The framework has been successfully applied to task space tracking control for holonomic systems for several different metrics on the anthropomorphic SARCOS Master Arm. In order to overcome the limiting requirement of accurate robot models, we first employ learning methods to find learning controllers for task space control. However, when learning to execute a redundant control problem, we face the general problem of the non-convexity of the solution space which can force the robot to steer into physically impossible configurations if supervised learning methods are employed without further consideration. This problem can be resolved using two major insights, i.e., the learning problem can be treated as locally convex and the cost function of the analytical framework can be used to ensure global consistency. Thus, we derive an immediate reinforcement learning algorithm from the expectation-maximization point of view which leads to a reward-weighted regression technique. This method can be used both for operational space control as well as general immediate reward reinforcement learning problems. We demonstrate the feasibility of the resulting framework on the problem of redundant end-effector tracking for both a simulated 3 degrees of freedom robot arm as well as for a simulated anthropomorphic SARCOS Master Arm. While learning to execute tasks in task space is an essential component to a general framework to motor skill learning, learning the actual task is of even higher importance, particularly as this issue is more frequently beyond the abilities of analytical approaches than execution. We focus on the learning of elemental tasks which can serve as the "building blocks of movement generation", called motor primitives. Motor primitives are parameterized task representations based on splines or nonlinear differential equations with desired attractor properties. While imitation learning of parameterized motor primitives is a relatively well-understood problem, the self-improvement by interaction of the system with the environment remains a challenging problem, tackled in the fourth chapter of this thesis. For pursuing this goal, we highlight the difficulties with current reinforcement learning methods, and outline both established and novel algorithms for the gradient-based improvement of parameterized policies. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm. In conclusion, in this thesis, we have contributed a general framework for analytically computing robot control laws which can be used for deriving various previous control approaches and serves as foundation as well as inspiration for our learning algorithms. We have introduced two classes of novel reinforcement learning methods, i.e., the Natural Actor-Critic and the Reward-Weighted Regression algorithm. These algorithms have been used in order to replace the analytical components of the theoretical framework by learned representations. Evaluations have been performed on both simulated and real robot arms.

am ei

[BibTex]

2007


[BibTex]


no image
The new robotics - towards human-centered machines

Schaal, S.

HFSP Journal Frontiers of Interdisciplinary Research in the Life Sciences, 1(2):115-126, 2007, clmc (article)

Abstract
Research in robotics has moved away from its primary focus on industrial applications. The New Robotics is a vision that has been developed in past years by our own university and many other national and international research instiutions and addresses how increasingly more human-like robots can live among us and take over tasks where our current society has shortcomings. Elder care, physical therapy, child education, search and rescue, and general assistance in daily life situations are some of the examples that will benefit from the New Robotics in the near future. With these goals in mind, research for the New Robotics has to embrace a broad interdisciplinary approach, ranging from traditional mathematical issues of robotics to novel issues in psychology, neuroscience, and ethics. This paper outlines some of the important research problems that will need to be resolved to make the New Robotics a reality.

am

link (url) [BibTex]

link (url) [BibTex]


no image
On the theory of magnetization dynamics of non-collinear spin systems in the s-d model

De Angeli, L.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Zur ab-initio Elektronentheorie des Magnetismus bei endlichen Temperaturen

Dietermann, F.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Röntgenzirkulardichroische Untersuchungen an ferromagnetischen verdünnten Halbleitersystemen

Tietze, T.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Low-dimensional Fe on vicinal Ir(997): Growth and magnetic properties

Kawwam, M.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Micromagnetic simulations of switching processes and the role of thermal fluctuations

Macke, S.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Hydrogen storage in metal-organic frameworks

Hirscher, M., Panella, B.

{Scripta Materialia}, 56, pages: 809-812, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Substrate-induced current anisotropy in YBa2Cu3O7-δthin films

Djupmyr, M., Albrecht, J.

{Physica C}, 460-462, pages: 1190-1191, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
A micellar approach to magnetic ultrahigh-density data-storage media: extending the limits of current colloidal methods

Ethirajan, A., Wiedwald, U., Boyen, H.-G., Kern, B., Han, L., Klimmer, A., Weigl, F., Kästle, G., Ziemann, P., Fauth, K., Cai, J., Behm, J., Romanyuk, A., Oelhafen, P., Walther, P., Biskupek, J., Kaiser, U.

{Advanced Materials}, 19, pages: 406-410, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Size dependence in the magnetization reversal of Fe/Gd multilayers on self-assembled arrays of nanospheres

Amaladass, E., Ludescher, B., Schütz, G., Tyliszczak, T., Eimüller, T.

{Applied Physics Letters}, 91, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Ma\ssgeschneiderte Wasserstoffspeicher

Hirscher, M., Panella, B.

{Nachrichten aus der Gdch-Energieinitiative}, (Sonderheft April 2007):12-13, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Reconstruction of historical alloys for pipe organs brings true Baroque music back to life.

Baretzky, B., Friesel, M., Straumal, B.

{MRS Bulletin}, 32, pages: 249-255, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Analysis of results from X-ray magnetic reflectometry for magnetic multilayer systems

Fähnle, M., Steiauf, D., Martosiswoyo, L., Goering, E., Brück, S., Schütz, G.

{Physical Review B}, 75, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Dramatic role of critical current anisotropy on flux avalanches in MgB2 films

Albrecht, J., Matveev, A. T., Strempfer, J., Habermeier, H.-U., Shantsev, D. V., Galperin, Y. M., Johansen, T. H.

{Physical Review Letters}, 98, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Transport properties of LCMO/YBCO hybrid structures

Soltan, S., Albrecht, J., Habermeier, H.-U.

{Materials Science and Engineering B}, 144, pages: 15-18, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Physisorption von Wasserstoff in neuen Materialien mit gro\sser spezifischer Oberfläche

Schmitz, B.

Universität Bonn, Bonn, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Cluster expansions in multicomponent systems: precise expansions from noisy databases

Diaz-Ortiz, A., Dosch, H., Drautz, R.

{Journal of Physics: Condensed Matter}, 19, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Towards spin injection into silicon

Dash, S. P.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Bestimmung der kritischen Schichtdicken ferromagnetischer Plättchen für Eindomänenverhalten

Soehnle, S.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Unusual propagation of magnetic avalanches in gold covered MgB2

Albrecht, J., Matveev, A. T., Habermeier, H.-U.

{Physica C}, 460-462, pages: 1245-1246, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Lowering of the L10 ordering temperature of FePt nanoparticles by He+ ion irradiation

Wiedwald, U., Klimmer, A., Kern, B., Han, L., Boyen, H.-G., Ziemann, P., Fauth, K.

{Applied Physics Letters}, 90, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Magnetic core shell nanoparticles characterized by X-ray absorption and magnetic circular dichroism

Fauth, K.

{Modern Physics Letters B}, 21(18):1179-1187, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Magnetic moment of Fe in oxide-free FePt nanoparticles

Dmitrieva, O., Spasova, M., Antoniak, C., Acet, M., Dumpich, G., Kästner, J., Farle, M., Fauth, K., Wiedwald, U., Boyen, H.-G., Ziemann, P.

{Physical Review B}, 76, 2007 (article)

mms

[BibTex]

[BibTex]


no image
The effect of bismuth segregation on the faceting of \Sigma3 and \Sigma9 coincidence boundaries in copper bicrystals

Straumal, B. B., Polyakov, S. A., Chang, L.-S., Mittemeijer, E. J.

{International Journal of Materials Research}, 98, pages: 451-456, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Hot isostatic pressing of Cu-Bi polycrystals with liquid-like grain boundary layers

Chang, L.-S., Straumal, B., Rabkin, E., Lojkowski, W., Gust, W.

{Acta Materialia}, 55, pages: 335-343, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Spatially resolved magnetic response in core shell nanoparticles

Fauth, K., Goering, E., Theil Kuhn, L.

{Modern Physics Letters B}, 21(18):1197-1200, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Interfaces in semiconductor/metal radial superlattices

Deneke, C., Sigle, W., Eigenthaler, U., van Aken, P. A., Schütz, G., Schmidt, O. G.

{Applied Physics Letters}, 90, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Live cell adhesion assay with attenuated total reflection infrared spectroscopy

Schmidt, M., Wolfram, T., Rumpler, M., Tripp, C. P., Grunze, M.

{Biointerphases}, 2(1):1-5, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Unusual Co moment reduction in the NiCoO/Co exchange bias system

Brück, S., Goering, E., Tang, Y. J., Schütz, G., Berkowitz, A. E.

{Journal of Magnetism and Magnetic Materials}, 310, pages: 2316-2318, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Zeitaufgelöste Röntgenmikroskopie an magnetischen Mikrostrukturen

Puzic, A.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Ab initio calculations of adiabatic magnon spectra using the atomic-sphere aproximation for the spin direction

Singer, R., Dietermann, F., Steiauf, D., Fähnle, M.

{Physical Review B}, 76, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Vortex dynamics studied by time-resolved X-ray microscopy

Chou, K. W.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Resonante magnetische Reflektometrie an Ferromagnet/Paramagnet Heterostrukturen

Ferreras Paz, V.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Low-temperature thermal-desorption mass spectroscopy applied to investigate the hydrogen adsorption on porous materials

Panella, B., Hirscher, M., Ludescher, B.

{Microporous and Mesoporous Materials}, 103, pages: 230-234, 2007 (article)

mms

[BibTex]


no image
Dependence of the critical temperature of YBCO thin films on spinpolarized quasiparticle injection

Habermeier, H.-U., Soltan, S., Albrecht, J.

{International Journal of Modern Physics B}, 21(18 \& 19):3303-3306, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Copper alloys for the restoration of reed pipes in historic organs

Straumal, B. B., Baretzky, B., Kalnins, J., Aslund, A., Friesel, M.

{Journal of Functional Materials}, 1, pages: 4-10, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Inhomogeneous vortex distribution and magnetic coupling in oxide superconductor-ferromagnet hybrids

Albrecht, J., Djupmyr, M., Soltan, S., Habermeier, H.-U., Connolly, M. R., Bending, S. J.

{New Journal of Physics}, 9, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Metal hydride materials for solid hydrogen storage: a review

Sakintuna, B., Lamari-Darkrim, F., Hirscher, M.

{International Journal of Hydrogen Energy}, 32, pages: 1121-1140, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Thermal reversal of exchange spring composite media in magnetic fields

Goll, D., Macke, S., Bertram, H. N.

{Applied Physics Letters}, 90, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Magnetism of Co-doped ZnO thin films

Gacic, M., Jakob, G., Herbort, C., Adrian, H., Tietze, T., Brück, S., Goering, E.

{Physical Review B}, 75, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Herstellung und Charakterisierung dünner Niob-Schichten auf verschiedenen Substraten

Mayer, M. W. R.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Formation of hard magnetic L10-FePt/FePd monolayers from elemental multilayers

Goo, N. H.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Universal temperature scaling of flux line pinning in high-temperature superconducting thin films

Albrecht, J., Djupmyr, M., Brück, S.

{Journal of Physics: Condensed Matter}, 19, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Dependence of the critical temperature of YBCO thin films on spin-polarized quasiparticle injection

Habermeier, H.-U., Soltan, S., Albrecht, J.

{Physica C}, 460-462, pages: 32-35, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Interaction of ferromagnetic LCMO layers through a superconducting YBCO spacer

Ravikumar, G., Yashwant, G., Singh, M. R., Gupta, S. K., Bhattacharya, S., Soltan, S., Albrecht, J., Habermeier, H.-U.

{Physica C}, 460-462, pages: 1375-1376, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Vortex dynamics in Permalloy disks with artificial defects: suppression of the gyrotropic mode

Kuepper, K., Bischoff, L., Akhmadaliev, C., Fassbinder, J., Stoll, H., Chou, K., Puzic, A., Fauth, K., Dolgos, D., Schütz, G., Van Waeyenberge, B., Tyliszczak, T., Neudecker, I., Woltersdorf, G., Back, C.

{Appplied Physics Letters}, 90, 2007 (article)

mms

[BibTex]

[BibTex]


no image
Vacancy-interstitial annihilation in titanomagnetite by thermal annealing

Walz, F., Brabers, V. A. M., Brabers, J. H. V. J., Kronmüller, H.

{Physica Status Solidi (A)}, 204(10):3514-3525, 2007 (article)

mms

DOI [BibTex]

DOI [BibTex]