Header logo is


2017


no image
Generalized exploration in policy search

van Hoof, H., Tanneberg, D., Peters, J.

Machine Learning, 106(9-10):1705-1724 , (Editors: Kurt Driessens, Dragi Kocev, Marko Robnik‐Sikonja, and Myra Spiliopoulou), October 2017, Special Issue of the ECML PKDD 2017 Journal Track (article)

ei

DOI Project Page [BibTex]

2017


DOI Project Page [BibTex]


no image
Probabilistic Prioritization of Movement Primitives

Paraschos, A., Lioutikov, R., Peters, J., Neumann, G.

Proceedings of the International Conference on Intelligent Robot Systems, and IEEE Robotics and Automation Letters (RA-L), 2(4):2294-2301, October 2017 (article)

ei

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Learning Movement Primitive Libraries through Probabilistic Segmentation

Lioutikov, R., Neumann, G., Maeda, G., Peters, J.

International Journal of Robotics Research, 36(8):879-894, July 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Guiding Trajectory Optimization by Demonstrated Distributions

Osa, T., Ghalamzan E., A. M., Stolkin, R., Lioutikov, R., Peters, J., Neumann, G.

IEEE Robotics and Automation Letters, 2(2):819-826, April 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Whole-body multi-contact motion in humans and humanoids: Advances of the CoDyCo European project

Padois, V., Ivaldi, S., Babic, J., Mistry, M., Peters, J., Nori, F.

Robotics and Autonomous Systems, 90, pages: 97-117, April 2017, Special Issue on New Research Frontiers for Intelligent Autonomous Systems (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Probabilistic Movement Primitives for Coordination of Multiple Human-Robot Collaborative Tasks

Maeda, G., Neumann, G., Ewerton, M., Lioutikov, R., Kroemer, O., Peters, J.

Autonomous Robots, 41(3):593-612, March 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Bioinspired tactile sensor for surface roughness discrimination

Yi, Z., Zhang, Y., Peters, J.

Sensors and Actuators A: Physical, 255, pages: 46-53, March 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Model-based Contextual Policy Search for Data-Efficient Generalization of Robot Skills

Kupcsik, A., Deisenroth, M., Peters, J., Ai Poh, L., Vadakkepat, V., Neumann, G.

Artificial Intelligence, 247, pages: 415-439, 2017, Special Issue on AI and Robotics (article)

ei

link (url) DOI Project Page [BibTex]

link (url) DOI Project Page [BibTex]


no image
Anticipatory Action Selection for Human-Robot Table Tennis

Wang, Z., Boularias, A., Mülling, K., Schölkopf, B., Peters, J.

Artificial Intelligence, 247, pages: 399-414, 2017, Special Issue on AI and Robotics (article)

Abstract
Abstract Anticipation can enhance the capability of a robot in its interaction with humans, where the robot predicts the humans' intention for selecting its own action. We present a novel framework of anticipatory action selection for human-robot interaction, which is capable to handle nonlinear and stochastic human behaviors such as table tennis strokes and allows the robot to choose the optimal action based on prediction of the human partner's intention with uncertainty. The presented framework is generic and can be used in many human-robot interaction scenarios, for example, in navigation and human-robot co-manipulation. In this article, we conduct a case study on human-robot table tennis. Due to the limited amount of time for executing hitting movements, a robot usually needs to initiate its hitting movement before the opponent hits the ball, which requires the robot to be anticipatory based on visual observation of the opponent's movement. Previous work on Intention-Driven Dynamics Models (IDDM) allowed the robot to predict the intended target of the opponent. In this article, we address the problem of action selection and optimal timing for initiating a chosen action by formulating the anticipatory action selection as a Partially Observable Markov Decision Process (POMDP), where the transition and observation are modeled by the \{IDDM\} framework. We present two approaches to anticipatory action selection based on the \{POMDP\} formulation, i.e., a model-free policy learning method based on Least-Squares Policy Iteration (LSPI) that employs the \{IDDM\} for belief updates, and a model-based Monte-Carlo Planning (MCP) method, which benefits from the transition and observation model by the IDDM. Experimental results using real data in a simulated environment show the importance of anticipatory action selection, and that \{POMDPs\} are suitable to formulate the anticipatory action selection problem by taking into account the uncertainties in prediction. We also show that existing algorithms for POMDPs, such as \{LSPI\} and MCP, can be applied to substantially improve the robot's performance in its interaction with humans.

am ei

DOI [BibTex]

DOI [BibTex]


no image
Kernel Mean Embedding of Distributions: A Review and Beyond

Muandet, K., Fukumizu, K., Sriperumbudur, B., Schölkopf, B.

Foundations and Trends in Machine Learning, 10(1-2):1-141, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Prediction of intention during interaction with iCub with Probabilistic Movement Primitives

Dermy, O., Paraschos, A., Ewerton, M., Charpillet, F., Peters, J., Ivaldi, S.

Frontiers in Robotics and AI, 4, pages: 45, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Manifold-based multi-objective policy search with sample reuse

Parisi, S., Pirotta, M., Peters, J.

Neurocomputing, 263, pages: 3-14, (Editors: Madalina Drugan, Marco Wiering, Peter Vamplew, and Madhu Chetty), 2017, Special Issue on Multi-Objective Reinforcement Learning (article)

ei

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
Minimax Estimation of Kernel Mean Embeddings

Tolstikhin, I., Sriperumbudur, B., Muandet, K.

Journal of Machine Learning Research, 18(86):1-47, 2017 (article)

ei

link (url) [BibTex]

link (url) [BibTex]


no image
easyGWAS: A Cloud-based Platform for Comparing the Results of Genome-wide Association Studies

Grimm, D., Roqueiro, D., Salome, P., Kleeberger, S., Greshake, B., Zhu, W., Liu, C., Lippert, C., Stegle, O., Schölkopf, B., Weigel, D., Borgwardt, K.

The Plant Cell, 29(1):5-19, 2017 (article)

ei

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
A Novel Unsupervised Segmentation Approach Quantifies Tumor Tissue Populations Using Multiparametric MRI: First Results with Histological Validation

Katiyar, P., Divine, M. R., Kohlhofer, U., Quintanilla-Martinez, L., Schölkopf, B., Pichler, B. J., Disselhorst, J. A.

Molecular Imaging and Biology, 19(3):391-397, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Temporal evolution of the central fixation bias in scene viewing

Rothkegel, L. O. M., Trukenbrod, H. A., Schütt, H. H., Wichmann, F. A., Engbert, R.

Journal of Vision, 17(13):3, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Electroencephalographic identifiers of motor adaptation learning

Ozdenizci, O., Yalcin, M., Erdogan, A., Patoglu, V., Grosse-Wentrup, M., Cetin, M.

Journal of Neural Engineering, 14(4):046027, 2017 (article)

ei

link (url) [BibTex]

link (url) [BibTex]


no image
Detecting distortions of peripherally presented letter stimuli under crowded conditions

Wallis, T. S. A., Tobias, S., Bethge, M., Wichmann, F. A.

Attention, Perception, & Psychophysics, 79(3):850-862, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Spectral Clustering predicts tumor tissue heterogeneity using dynamic 18F-FDG PET: a complement to the standard compartmental modeling approach

Katiyar, P., Divine, M. R., Kohlhofer, U., Quintanilla-Martinez, L., Schölkopf, B., Pichler, B. J., Disselhorst, J. A.

Journal of Nuclear Medicine, 58(4):651-657, 2017 (article)

ei

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
A parametric texture model based on deep convolutional features closely matches texture appearance for humans

Wallis, T. S. A., Funke, C. M., Ecker, A. S., Gatys, L. A., Wichmann, F. A., Bethge, M.

Journal of Vision, 17(12), 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
BundleMAP: Anatomically Localized Classification, Regression, and Hypothesis Testing in Diffusion MRI

Khatami, M., Schmidt-Wilcke, T., Sundgren, P. C., Abbasloo, A., Schölkopf, B., Schultz, T.

Pattern Recognition, 63, pages: 593-600, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Model Selection for Gaussian Mixture Models

Huang, T., Peng, H., Zhang, K.

Statistica Sinica, 27(1):147-169, 2017 (article)

ei

link (url) [BibTex]

link (url) [BibTex]


no image
An image-computable psychophysical spatial vision model

Schütt, H. H., Wichmann, F. A.

Journal of Vision, 17(12), 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Methods and measurements to compare men against machines

Wichmann, F. A., Janssen, D. H. J., Geirhos, R., Aguilar, G., Schütt, H. H., Maertens, M., Bethge, M.

Electronic Imaging, pages: 36-45(10), 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
A Comparison of Autoregressive Hidden Markov Models for Multimodal Manipulations With Variable Masses

Kroemer, O., Peters, J.

IEEE Robotics and Automation Letters, 2(2):1101-1108, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Phase Estimation for Fast Action Recognition and Trajectory Generation in Human-Robot Collaboration

Maeda, G., Ewerton, M., Neumann, G., Lioutikov, R., Peters, J.

International Journal of Robotics Research, 36(13-14):1579-1594, 2017, Special Issue on the Seventeenth International Symposium on Robotics Research (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
A Phase-coded Aperture Camera with Programmable Optics

Chen, J., Hirsch, M., Heintzmann, R., Eberhardt, B., Lensch, H. P. A.

Electronic Imaging, 2017(17):70-75, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
On Maximum Entropy and Inference

Gresele, L., Marsili, M.

Entropy, 19(12):article no. 642, 2017 (article)

ei

link (url) [BibTex]

link (url) [BibTex]


no image
Towards Engagement Models that Consider Individual Factors in HRI: On the Relation of Extroversion and Negative Attitude Towards Robots to Gaze and Speech During a Human-Robot Assembly Task

Ivaldi, S., Lefort, S., Peters, J., Chetouani, M., Provasi, J., Zibetti, E.

International Journal of Social Robotics, 9(1):63-86, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Non-parametric Policy Search with Limited Information Loss

van Hoof, H., Neumann, G., Peters, J.

Journal of Machine Learning Research , 18(73):1-46, 2017 (article)

ei

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


no image
Stability of Controllers for Gaussian Process Dynamics

Vinogradska, J., Bischoff, B., Nguyen-Tuong, D., Peters, J.

Journal of Machine Learning Research, 18(100):1-37, 2017 (article)

ei

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


no image
SUV-quantification of physiological lung tissue in an integrated PET/MR-system: Impact of lung density and bone tissue

Seith, F., Schmidt, H., Gatidis, S., Bezrukov, I., Schraml, C., Pfannenberg, C., la Fougère, C., Nikolaou, K., Schwenzer, N.

PLOS ONE, 12(5):1-13, 2017 (article)

ei

DOI [BibTex]

DOI [BibTex]


no image
Absence of EEG correlates of self-referential processing depth in ALS

Fomina, T., Weichwald, S., Synofzik, M., Just, J., Schöls, L., Schölkopf, B., Grosse-Wentrup, M.

PLOS ONE, 12(6):e0180136, 2017 (article)

ei

PDF DOI [BibTex]

2010


no image

no image
Tackling Box-Constrained Optimization via a New Projected Quasi-Newton Approach

Kim, D., Sra, S., Dhillon, I.

SIAM Journal on Scientific Computing, 32(6):3548-3563 , December 2010 (article)

Abstract
Numerous scientific applications across a variety of fields depend on box-constrained convex optimization. Box-constrained problems therefore continue to attract research interest. We address box-constrained (strictly convex) problems by deriving two new quasi-Newton algorithms. Our algorithms are positioned between the projected-gradient [J. B. Rosen, J. SIAM, 8 (1960), pp. 181–217] and projected-Newton [D. P. Bertsekas, SIAM J. Control Optim., 20 (1982), pp. 221–246] methods. We also prove their convergence under a simple Armijo step-size rule. We provide experimental results for two particular box-constrained problems: nonnegative least squares (NNLS), and nonnegative Kullback–Leibler (NNKL) minimization. For both NNLS and NNKL our algorithms perform competitively as compared to well-established methods on medium-sized problems; for larger problems our approach frequently outperforms the competition.

ei

Web DOI Project Page [BibTex]

Web DOI Project Page [BibTex]


no image
Algorithmen zum Automatischen Erlernen von Motorfähigkeiten

Peters, J., Kober, J., Schaal, S.

at - Automatisierungstechnik, 58(12):688-694, December 2010 (article)

Abstract
Robot learning methods which allow autonomous robots to adapt to novel situations have been a long standing vision of robotics, artificial intelligence, and cognitive sciences. However, to date, learning techniques have yet to fulfill this promise as only few methods manage to scale into the high-dimensional domains of manipulator robotics, or even the new upcoming trend of humanoid robotics. If possible, scaling was usually only achieved in precisely pre-structured domains. In this paper, we investigate the ingredients for a general approach policy learning with the goal of an application to motor skill refinement in order to get one step closer towards human-like performance. For doing so, we study two major components for such an approach, i. e., firstly, we study policy learning algorithms which can be applied in the general setting of motor skill learning, and, secondly, we study a theoretically well-founded general approach to representing the required control structures for task representation and execution.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
PAC-Bayesian Analysis of Co-clustering and Beyond

Seldin, Y., Tishby, N.

Journal of Machine Learning Research, 11, pages: 3595-3646, December 2010 (article)

ei

PDF PDF [BibTex]

PDF PDF [BibTex]


no image
Gaussian Processes for Machine Learning (GPML) Toolbox

Rasmussen, C., Nickisch, H.

Journal of Machine Learning Research, 11, pages: 3011-3015, November 2010 (article)

Abstract
The GPML toolbox provides a wide range of functionality for Gaussian process (GP) inference and prediction. GPs are specified by mean and covariance functions; we offer a library of simple mean and covariance functions and mechanisms to compose more complex ones. Several likelihood functions are supported including Gaussian and heavy-tailed for regression as well as others suitable for classification. Finally, a range of inference methods is provided, including exact and variational inference, Expectation Propagation, and Laplace's method dealing with non-Gaussian likelihoods and FITC for dealing with large regression tasks.

ei

Web [BibTex]

Web [BibTex]


no image
Cryo-EM structure and rRNA model of a translating eukaryotic 80S ribosome at 5.5-Å resolution

Armache, J-P., Jarasch, A., Anger, AM., Villa, E., Becker, T., Bhushan, S., Jossinet, F., Habeck, M., Dindar, G., Franckenberg, S., Marquez, V., Mielke, T., Thomm, M., Berninghausen, O., Beatrix, B., Söding, J., Westhof, E., Wilson, DN., Beckmann, R.

Proceedings of the National Academy of Sciences of the United States of America, 107(46):19748-19753, November 2010 (article)

Abstract
Protein biosynthesis, the translation of the genetic code into polypeptides, occurs on ribonucleoprotein particles called ribosomes. Although X-ray structures of bacterial ribosomes are available, high-resolution structures of eukaryotic 80S ribosomes are lacking. Using cryoelectron microscopy and single-particle reconstruction, we have determined the structure of a translating plant (Triticum aestivum) 80S ribosome at 5.5-Å resolution. This map, together with a 6.1-Å map of a Saccharomyces cerevisiae 80S ribosome, has enabled us to model ∼98% of the rRNA. Accurate assignment of the rRNA expansion segments (ES) and variable regions has revealed unique ES–ES and r-protein–ES interactions, providing insight into the structure and evolution of the eukaryotic ribosome.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Policy gradient methods

Peters, J.

Scholarpedia, 5(11):3698, November 2010 (article)

Abstract
Policy gradient methods are a type of reinforcement learning techniques that rely upon optimizing parametrized policies with respect to the expected return (long-term cumulative reward) by gradient descent. They do not suffer from many of the problems that have been marring traditional reinforcement learning approaches such as the lack of guarantees of a value function, the intractability problem resulting from uncertain state information and the complexity arising from continuous states & actions.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Localization of eukaryote-specific ribosomal proteins in a 5.5-Å cryo-EM map of the 80S eukaryotic ribosome

Armache, J-P., Jarasch, A., Anger, AM., Villa, E., Becker, T., Bhushan, S., Jossinet, F., Habeck, M., Dindar, G., Franckenberg, S., Marquez, V., Mielke, T., Thomm, M., Berninghausen, O., Beatrix, B., Söding, J., Westhof, E., Wilson, DN., Beckmann, R.

Proceedings of the National Academy of Sciences of the United States of America, 107(46):19754-19759, November 2010 (article)

Abstract
Protein synthesis in all living organisms occurs on ribonucleoprotein particles, called ribosomes. Despite the universality of this process, eukaryotic ribosomes are significantly larger in size than their bacterial counterparts due in part to the presence of 80 r proteins rather than 54 in bacteria. Using cryoelectron microscopy reconstructions of a translating plant (Triticum aestivum) 80S ribosome at 5.5-Å resolution, together with a 6.1-Å map of a translating Saccharomyces cerevisiae 80S ribosome, we have localized and modeled 74/80 (92.5%) of the ribosomal proteins, encompassing 12 archaeal/eukaryote-specific small subunit proteins as well as the complete complement of the ribosomal proteins of the eukaryotic large subunit. Near-complete atomic models of the 80S ribosome provide insights into the structure, function, and evolution of the eukaryotic translational apparatus.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Spatio-Spectral Remote Sensing Image Classification With Graph Kernels

Camps-Valls, G., Shervashidze, N., Borgwardt, K.

IEEE Geoscience and Remote Sensing Letters, 7(4):741-745, October 2010 (article)

Abstract
This letter presents a graph kernel for spatio-spectral remote sensing image classification with support vector machines (SVMs). The method considers higher order relations in the neighborhood (beyond pairwise spatial relations) to iteratively compute a kernel matrix for SVM learning. The proposed kernel is easy to compute and constitutes a powerful alternative to existing approaches. The capabilities of the method are illustrated in several multi- and hyperspectral remote sensing images acquired over both urban and agricultural areas.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Causal Inference Using the Algorithmic Markov Condition

Janzing, D., Schölkopf, B.

IEEE Transactions on Information Theory, 56(10):5168-5194, October 2010 (article)

Abstract
Inferring the causal structure that links $n$ observables is usually based upon detecting statistical dependences and choosing simple graphs that make the joint measure Markovian. Here we argue why causal inference is also possible when the sample size is one. We develop a theory how to generate causal graphs explaining similarities between single objects. To this end, we replace the notion of conditional stochastic independence in the causal Markov condition with the vanishing of conditional algorithmic mutual information and describe the corresponding causal inference rules. We explain why a consistent reformulation of causal inference in terms of algorithmic complexity implies a new inference principle that takes into account also the complexity of conditional probability densities, making it possible to select among Markov equivalent causal graphs. This insight provides a theoretical foundation of a heuristic principle proposed in earlier work. We also sketch some ideas on how to replace Kolmogorov complexity with decidable complexity criteria. This can be seen as an algorithmic analog of replacing the empirically undecidable question of statistical independence with practical independence tests that are based on implicit or explicit assumptions on the underlying distribution.

ei

PDF Web DOI [BibTex]

PDF Web DOI [BibTex]


no image
Recurrent Policy Gradients

Wierstra, D., Förster, A., Peters, J., Schmidhuber, J.

Logic Journal of the IGPL, 18(5):620-634, October 2010 (article)

Abstract
Reinforcement learning for partially observable Markov decision problems (POMDPs) is a challenge as it requires policies with an internal state. Traditional approaches suffer significantly from this shortcoming and usually make strong assumptions on the problem domain such as perfect system models, state-estimators and a Markovian hidden system. Recurrent neural networks (RNNs) offer a natural framework for dealing with policy learning using hidden state and require only few limiting assumptions. As they can be trained well using gradient descent, they are suited for policy gradient approaches. In this paper, we present a policy gradient method, the Recurrent Policy Gradient which constitutes a model-free reinforcement learning method. It is aimed at training limited-memory stochastic policies on problems which require long-term memories of past observations. The approach involves approximating a policy gradient for a recurrent neural network by backpropagating return-weighted characteristic eligibilities through time. Using a ‘‘Long Short-Term Memory’’ RNN architecture, we are able to outperform previous RL methods on three important benchmark tasks. Furthermore, we show that using history-dependent baselines helps reducing estimation variance significantly, thus enabling our approach to tackle more challenging, highly stochastic environments.

ei

PDF Web DOI [BibTex]

PDF Web DOI [BibTex]


no image
Discriminative frequent subgraph mining with optimality guarantees

Thoma, M., Cheng, H., Gretton, A., Han, J., Kriegel, H., Smola, A., Song, L., Yu, P., Yan, X., Borgwardt, K.

Journal of Statistical Analysis and Data Mining, 3(5):302–318, October 2010 (article)

Abstract
The goal of frequent subgraph mining is to detect subgraphs that frequently occur in a dataset of graphs. In classification settings, one is often interested in discovering discriminative frequent subgraphs, whose presence or absence is indicative of the class membership of a graph. In this article, we propose an approach to feature selection on frequent subgraphs, called CORK, that combines two central advantages. First, it optimizes a submodular quality criterion, which means that we can yield a near-optimal solution using greedy feature selection. Second, our submodular quality function criterion can be integrated into gSpan, the state-of-the-art tool for frequent subgraph mining, and help to prune the search space for discriminative frequent subgraphs even during frequent subgraph mining.

ei

Web DOI [BibTex]

Web DOI [BibTex]