Header logo is


2018


Motion-based Object Segmentation based on Dense RGB-D Scene Flow
Motion-based Object Segmentation based on Dense RGB-D Scene Flow

Shao, L., Shah, P., Dwaracherla, V., Bohg, J.

IEEE Robotics and Automation Letters, 3(4):3797-3804, IEEE, IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2018 (conference)

Abstract
Given two consecutive RGB-D images, we propose a model that estimates a dense 3D motion field, also known as scene flow. We take advantage of the fact that in robot manipulation scenarios, scenes often consist of a set of rigidly moving objects. Our model jointly estimates (i) the segmentation of the scene into an unknown but finite number of objects, (ii) the motion trajectories of these objects and (iii) the object scene flow. We employ an hourglass, deep neural network architecture. In the encoding stage, the RGB and depth images undergo spatial compression and correlation. In the decoding stage, the model outputs three images containing a per-pixel estimate of the corresponding object center as well as object translation and rotation. This forms the basis for inferring the object segmentation and final object scene flow. To evaluate our model, we generated a new and challenging, large-scale, synthetic dataset that is specifically targeted at robotic manipulation: It contains a large number of scenes with a very diverse set of simultaneously moving 3D objects and is recorded with a commonly-used RGB-D camera. In quantitative experiments, we show that we significantly outperform state-of-the-art scene flow and motion-segmentation methods. In qualitative experiments, we show how our learned model transfers to challenging real-world scenes, visually generating significantly better results than existing methods.

am

Project Page arXiv DOI [BibTex]

2018


Project Page arXiv DOI [BibTex]


A Value-Driven Eldercare Robot: Virtual and Physical Instantiations of a Case-Supported Principle-Based Behavior Paradigm
A Value-Driven Eldercare Robot: Virtual and Physical Instantiations of a Case-Supported Principle-Based Behavior Paradigm

Anderson, M., Anderson, S., Berenz, V.

Proceedings of the IEEE, pages: 1,15, October 2018 (article)

Abstract
In this paper, a case-supported principle-based behavior paradigm is proposed to help ensure ethical behavior of autonomous machines. We argue that ethically significant behavior of autonomous systems should be guided by explicit ethical principles determined through a consensus of ethicists. Such a consensus is likely to emerge in many areas in which autonomous systems are apt to be deployed and for the actions they are liable to undertake. We believe that this is the case since we are more likely to agree on how machines ought to treat us than on how human beings ought to treat one another. Given such a consensus, particular cases of ethical dilemmas where ethicists agree on the ethically relevant features and the right course of action can be used to help discover principles that balance these features when they are in conflict. Such principles not only help ensure ethical behavior of complex and dynamic systems but also can serve as a basis for justification of this behavior. The requirements, methods, implementation, and evaluation components of the paradigm are detailed as well as its instantiation in both a simulated and real robot functioning in the domain of eldercare.

am

link (url) DOI [BibTex]


Playful: Reactive Programming for Orchestrating Robotic Behavior
Playful: Reactive Programming for Orchestrating Robotic Behavior

Berenz, V., Schaal, S.

IEEE Robotics Automation Magazine, 25(3):49-60, September 2018 (article) In press

Abstract
For many service robots, reactivity to changes in their surroundings is a must. However, developing software suitable for dynamic environments is difficult. Existing robotic middleware allows engineers to design behavior graphs by organizing communication between components. But because these graphs are structurally inflexible, they hardly support the development of complex reactive behavior. To address this limitation, we propose Playful, a software platform that applies reactive programming to the specification of robotic behavior.

am

playful website playful_IEEE_RAM link (url) DOI [BibTex]


ClusterNet: Instance Segmentation in RGB-D Images
ClusterNet: Instance Segmentation in RGB-D Images

Shao, L., Tian, Y., Bohg, J.

arXiv, September 2018, Submitted to ICRA'19 (article) Submitted

Abstract
We propose a method for instance-level segmentation that uses RGB-D data as input and provides detailed information about the location, geometry and number of {\em individual\/} objects in the scene. This level of understanding is fundamental for autonomous robots. It enables safe and robust decision-making under the large uncertainty of the real-world. In our model, we propose to use the first and second order moments of the object occupancy function to represent an object instance. We train an hourglass Deep Neural Network (DNN) where each pixel in the output votes for the 3D position of the corresponding object center and for the object's size and pose. The final instance segmentation is achieved through clustering in the space of moments. The object-centric training loss is defined on the output of the clustering. Our method outperforms the state-of-the-art instance segmentation method on our synthesized dataset. We show that our method generalizes well on real-world data achieving visually better segmentation results.

am

link (url) [BibTex]

link (url) [BibTex]


Probabilistic Recurrent State-Space Models
Probabilistic Recurrent State-Space Models

Doerr, A., Daniel, C., Schiegg, M., Nguyen-Tuong, D., Schaal, S., Toussaint, M., Trimpe, S.

In Proceedings of the International Conference on Machine Learning (ICML), International Conference on Machine Learning (ICML), July 2018 (inproceedings)

Abstract
State-space models (SSMs) are a highly expressive model class for learning patterns in time series data and for system identification. Deterministic versions of SSMs (e.g., LSTMs) proved extremely successful in modeling complex time-series data. Fully probabilistic SSMs, however, unfortunately often prove hard to train, even for smaller problems. To overcome this limitation, we propose a scalable initialization and training algorithm based on doubly stochastic variational inference and Gaussian processes. In the variational approximation we propose in contrast to related approaches to fully capture the latent state temporal correlations to allow for robust training.

am ics

arXiv pdf Project Page [BibTex]

arXiv pdf Project Page [BibTex]


Real-time Perception meets Reactive Motion Generation
Real-time Perception meets Reactive Motion Generation

(Best Systems Paper Finalists - Amazon Robotics Best Paper Awards in Manipulation)

Kappler, D., Meier, F., Issac, J., Mainprice, J., Garcia Cifuentes, C., Wüthrich, M., Berenz, V., Schaal, S., Ratliff, N., Bohg, J.

IEEE Robotics and Automation Letters, 3(3):1864-1871, July 2018 (article)

Abstract
We address the challenging problem of robotic grasping and manipulation in the presence of uncertainty. This uncertainty is due to noisy sensing, inaccurate models and hard-to-predict environment dynamics. Our approach emphasizes the importance of continuous, real-time perception and its tight integration with reactive motion generation methods. We present a fully integrated system where real-time object and robot tracking as well as ambient world modeling provides the necessary input to feedback controllers and continuous motion optimizers. Specifically, they provide attractive and repulsive potentials based on which the controllers and motion optimizer can online compute movement policies at different time intervals. We extensively evaluate the proposed system on a real robotic platform in four scenarios that exhibit either challenging workspace geometry or a dynamic environment. We compare the proposed integrated system with a more traditional sense-plan-act approach that is still widely used. In 333 experiments, we show the robustness and accuracy of the proposed system.

am

arxiv video video link (url) DOI Project Page [BibTex]


Online Learning of a Memory for Learning Rates
Online Learning of a Memory for Learning Rates

(nominated for best paper award)

Meier, F., Kappler, D., Schaal, S.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) 2018, IEEE, International Conference on Robotics and Automation, May 2018, accepted (inproceedings)

Abstract
The promise of learning to learn for robotics rests on the hope that by extracting some information about the learning process itself we can speed up subsequent similar learning tasks. Here, we introduce a computationally efficient online meta-learning algorithm that builds and optimizes a memory model of the optimal learning rate landscape from previously observed gradient behaviors. While performing task specific optimization, this memory of learning rates predicts how to scale currently observed gradients. After applying the gradient scaling our meta-learner updates its internal memory based on the observed effect its prediction had. Our meta-learner can be combined with any gradient-based optimizer, learns on the fly and can be transferred to new optimization tasks. In our evaluations we show that our meta-learning algorithm speeds up learning of MNIST classification and a variety of learning control tasks, either in batch or online learning settings.

am

pdf video code [BibTex]

pdf video code [BibTex]


Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks
Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks

Sutanto, G., Su, Z., Schaal, S., Meier, F.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) 2018, IEEE, International Conference on Robotics and Automation, May 2018 (inproceedings)

am

pdf video [BibTex]

pdf video [BibTex]


no image
Distributed Event-Based State Estimation for Networked Systems: An LMI Approach

Muehlebach, M., Trimpe, S.

IEEE Transactions on Automatic Control, 63(1):269-276, January 2018 (article)

am ics

arXiv (extended version) DOI Project Page [BibTex]

arXiv (extended version) DOI Project Page [BibTex]


no image
Memristor-enhanced humanoid robot control system–Part I: theory behind the novel memcomputing paradigm

Ascoli, A., Baumann, D., Tetzlaff, R., Chua, L. O., Hild, M.

International Journal of Circuit Theory and Applications, 46(1):155-183, 2018 (article)

am

DOI [BibTex]

DOI [BibTex]


no image
On Time Optimization of Centroidal Momentum Dynamics

Ponton, B., Herzog, A., Del Prete, A., Schaal, S., Righetti, L.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 5776-5782, IEEE, Brisbane, Australia, 2018 (inproceedings)

Abstract
Recently, the centroidal momentum dynamics has received substantial attention to plan dynamically consistent motions for robots with arms and legs in multi-contact scenarios. However, it is also non convex which renders any optimization approach difficult and timing is usually kept fixed in most trajectory optimization techniques to not introduce additional non convexities to the problem. But this can limit the versatility of the algorithms. In our previous work, we proposed a convex relaxation of the problem that allowed to efficiently compute momentum trajectories and contact forces. However, our approach could not minimize a desired angular momentum objective which seriously limited its applicability. Noticing that the non-convexity introduced by the time variables is of similar nature as the centroidal dynamics one, we propose two convex relaxations to the problem based on trust regions and soft constraints. The resulting approaches can compute time-optimized dynamically consistent trajectories sufficiently fast to make the approach realtime capable. The performance of the algorithm is demonstrated in several multi-contact scenarios for a humanoid robot. In particular, we show that the proposed convex relaxation of the original problem finds solutions that are consistent with the original non-convex problem and illustrate how timing optimization allows to find motion plans that would be difficult to plan with fixed timing † †Implementation details and demos can be found in the source code available at https://git-amd.tuebingen.mpg.de/bponton/timeoptimization.

am mg

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Active microrheology in corrugated channels

Puertas, A. M., Malgaretti, P., Pagonabarraga, I.

The Journal of Chemical Physics, 149(17), American Institute of Physics, Woodbury, N.Y., 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
First-passage dynamics of linear stochastic interface models: weak-noise theory and influence of boundary conditions

Gross, M.

Journal of Statistical Mechanics: Theory and Experiment, 2018, Institute of Physics Publishing, Bristol, England, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Cu@TiO2 Janus microswimmers with a versatile motion mechanism

Wang, L. L., Popescu, M. N., Stavale, F., Ali, A., Gemming, T., Simmchen, J.

Soft Matter, 14(34):6969-6973, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Probing interface localization-delocalization transitions by colloids

Kondrat, S., Vasilyev, O., Dietrich, S.

Journal of Physics: Condensed Matter, 30(41), IOP Publishing, Bristol, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Medical imaging for the tracking of micromotors

Vilela, D., Coss\’\io, U., Parmar, J., Mart\’\inez-Villacorta, A. M., Gómez-Vallejo, V., Llop, J., Sánchez, S.

ACS Nano, 12(2):1220-1227, American Chemical Society, Washington, DC, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Noncontinuous Super-Diffusive Dynamics of a Light-Activated Nanobottle Motor

Xuan, M., Mestre, R., Gao, C., Zhou, C., He, Q., Sánchez, S.

Angewandte Chemie International Edition, 57(23):6838-6842, Wiley-VCH, Weinheim, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
On the origin of forward-backward multiplicity correlations in pp collisions at ultrarelativistic energies

Bravina, L., Bleibel, J., Zabrodin, E.

Physics Letters B, 787, pages: 146-152, North-Holland, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Autophoretic motion in three dimensions

Lisicki, M., Reigh, S., Lauga, E.

Soft Matter, 14(17):3304-3314, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Order-disorder transitions in lattice gases with annealed reactive constraints

Dudka, M., Bénichou, O., Oshanin, G.

Journal of Statistical Mechanics: Theory and Experiment, 2018, Institute of Physics Publishing, Bristol, England, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Bacterial Biohybrid Microswimmers

Bastos-Arrieta, J., Revilla-Guarinos, A., Uspal, W., Simmchen, J.

Frontiers in Robotics and AI, 5, Frontiers Media, Lausanne, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Fluctuational electrodynamics for nonlinear materials in and out of thermal equilibrium

Soo, H., Krüger, M.

Physical Review B, 97(4), American Physical Society, Woodbury, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Local pressure for confined systems

Malgaretti, P., Bier, M.

Physical Review E, 97(2), American Physical Society, Melville, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


Combining learned and analytical models for predicting action effects
Combining learned and analytical models for predicting action effects

Kloss, A., Schaal, S., Bohg, J.

arXiv, 2018 (article) Submitted

Abstract
One of the most basic skills a robot should possess is predicting the effect of physical interactions with objects in the environment. This enables optimal action selection to reach a certain goal state. Traditionally, dynamics are approximated by physics-based analytical models. These models rely on specific state representations that may be hard to obtain from raw sensory data, especially if no knowledge of the object shape is assumed. More recently, we have seen learning approaches that can predict the effect of complex physical interactions directly from sensory input. It is however an open question how far these models generalize beyond their training data. In this work, we investigate the advantages and limitations of neural network based learning approaches for predicting the effects of actions based on sensory input and show how analytical and learned models can be combined to leverage the best of both worlds. As physical interaction task, we use planar pushing, for which there exists a well-known analytical model and a large real-world dataset. We propose to use a convolutional neural network to convert raw depth images or organized point clouds into a suitable representation for the analytical model and compare this approach to using neural networks for both, perception and prediction. A systematic evaluation of the proposed approach on a very large real-world dataset shows two main advantages of the hybrid architecture. Compared to a pure neural network, it significantly (i) reduces required training data and (ii) improves generalization to novel physical interaction.

am

arXiv pdf link (url) [BibTex]


no image
Charge polarization, local electroneutrality breakdown and eddy formation due to electroosmosis in varying-section channels

Chinappi, M., Malgaretti, P.

Soft Matter, 14(45):9083-9087, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Critical Casimir interactions and percolation: The quantitative description of critical fluctuations

Vasilyev, O.

Physical Review E, 98(6), American Physical Society, Melville, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Shear-density coupling for a compressible single-component yield-stress fluid

Gross, M., Varnik, F.

Soft Matter, 14(22):4577-4590, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Shape-dependent guidance of active Janus particles by chemically patterned surfaces

Uspal, W. E., Popescu, M. N., Tasinkevych, M., Dietrich, S.

New Journal of Physics, 20, IOP Publishing, Bristol, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Extrapolation to nonequilibrium from coarse-grained response theory

Basu, U., Helden, L., Krüger, M.

Physical Review Letters, 120(18), American Physical Society, Woodbury, N.Y., 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Directed Flow of Micromotors through Alignment Interactions with Micropatterned Ratchets

Katuri, J., Caballero, D., Voituriez, R., Samitier, J., Sánchez, S.

ACS Nano, 12(7):7282-7291, American Chemical Society, Washington, DC, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Spontaneous symmetry breaking of charge-regulated surfaces

Majee, A., Bier, M., Podgornik, R.

Soft Matter, 14(6):985-991, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Electrostatic interaction between dissimilar colloids at fluid interfaces

Majee, A., Schmetzer, T., Bier, M.

Physical Review E, 97(4), American Physical Society, Melville, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Wetting transition of a cylindrical cavity engraved on a hydrophobic surface

Kim, H., Ha, M. Y., Jang, J.

The Journal of Physical Chemistry C, 122(4):2122-2126, American Chemical Society, Washington, D.C., 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Curvature corrections to the nonlocal interfacial model for short-ranged forces

Romero-Enrique, J.M., Squarcini, Alessio, Parry, A. O., Goldbart, P. M.

Physical Review E, 97(6), American Physical Society, Melville, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Effective squirmer models for self-phoretic chemically active spherical colloids

Popescu, M. N., Uspal, W. E., Eskandari, Z., Tasinkevych, M., Dietrich, S.

The European Physical Journal E, 41(12), EDP Sciences; Società Italiana di Fisica; Springer, Les Ulis; Bologna; Heidelberg, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Two time scales for self and collective diffusion near the critical point in a simple patchy model for proteins with floating bonds

Bleibel, J., Habiger, M., Lütje, M., Hirschmann, F., Roosen-Runge, F., Seydel, T., Zhang, F., Schreiber, F., Oettel, M.

Soft Matter, 14(39):8006-8016, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Globulelike Conformation and Enhanced Diffusion of Active Polymers

Bianco, V., Locatelli, E., Malgaretti, P.

Physical Review Letters, 121(21), American Physical Society, Woodbury, N.Y., 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Rheological behavior of colloidal suspension with long-range interactions

Arietaleaniz, S., Malgaretti, P., Pagonabarraga, I., Hidalgo, R. C.

Physical Review E, 98(4), American Physical Society, Melville, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Solvent coarsening around colloids driven by temperature gradients

Roy, S., Dietrich, S., Maciolek, A.

Physical Review E, 97(4), American Physical Society, Melville, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Cross-stream migration of active particles

Katuri, J., Uspal, W., Simmchen, J., López, A. M., Sanchez, S.

Science Advances, 4(1), AAAS, Washington, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Transient dynamics of electric double-layer capacitors: Exact expressions within the Debye-Falkenhagen approximation

Janssen, M., Bier, M.

Physical Review E, 97(5), American Physical Society, Melville, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Coalescence preference and droplet size inequality during fluid phase segregation

Roy, S.

EPL, 121(3), EDP Science, Les-Ulis, 2018 (article)

icm

DOI [BibTex]


no image
Structure of interfaces at phase coexistence. Theory and numerics

Delfino, G., Selke, W., Squarcini, A.

Journal of Statistical Mechanics: Theory and Experiment, 2018, Institute of Physics Publishing, Bristol, England, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Power spectral density of a single Brownian trajectory: what one can and cannot learn from it

Krapf, D., Marinari, E., Metzler, Ralf, Oshanin, Gleb, Xu, Xinran, Squarcini, A.

New Journal of Physics, 20, IOP Publishing, Bristol, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Diffusiophoretically induced interactions between chemically active and inert particles

Reigh, Shang-Yik, Chuphal, P., Thakur, S., Kapral, R.

Soft Matter, 14(29):6043-6057, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Collective behavior of colloids due to critical Casimir interactions

Maciolek, A., Dietrich, S.

Reviews of Modern Physics, 90(4), American Physical Society, Minneapolis, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Heat radiation and transfer in confinement

Asheichyk, K., Krüger, M.

Physical Review B, 98(19), American Physical Society, Woodbury, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
A ferronematic slab in external magnetic fields

Zarubin, G., Bier, M., Dietrich, S.

Soft Matter, 14(48):9806-9818, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Surface-induced nonequilibrium dynamics and critical Casimir forces for model B in film geometry

Gross, M., Gambassi, A., Dietrich, S.

Physical Review E, 98(3), American Physical Society, Melville, NY, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]


no image
Phase separation around a heated colloid in bulk and under confinement

Roy, S., Maciolek, A.

Soft Matter, 14(46):9326-9335, Royal Society of Chemistry, Cambridge, UK, 2018 (article)

icm

DOI [BibTex]

DOI [BibTex]