Header logo is


2019


Soft-magnetic coatings as possible sensors for magnetic imaging of superconductors
Soft-magnetic coatings as possible sensors for magnetic imaging of superconductors

Ionescu, A., Simmendinger, J., Bihler, M., Miksch, C., Fischer, P., Soltan, S., Schütz, G., Albrecht, J.

Supercond. Sci. and Tech., 33, pages: 015002, IOP, December 2019 (article)

Abstract
Magnetic imaging of superconductors typically requires a soft-magnetic material placed on top of the superconductor to probe local magnetic fields. For reasonable results the influence of the magnet onto the superconductor has to be small. Thin YBCO films with soft-magnetic coatings are investigated using SQUID magnetometry. Detailed measurements of the magnetic moment as a function of temperature, magnetic field and time have been performed for different heterostructures. It is found that the modification of the superconducting transport in these heterostructures strongly depends on the magnetic and structural properties of the soft-magnetic material. This effect is especially pronounced for an inhomogeneous coating consisting of ferromagnetic nanoparticles.

pf mms

link (url) DOI [BibTex]

2019


link (url) DOI [BibTex]


Acoustic hologram enhanced phased arrays for ultrasonic particle manipulation
Acoustic hologram enhanced phased arrays for ultrasonic particle manipulation

Cox, L., Melde, K., Croxford, A., Fischer, P., Drinkwater, B.

Phys. Rev. Applied, 12, pages: 064055, November 2019 (article)

Abstract
The ability to shape ultrasound fields is important for particle manipulation, medical therapeutics and imaging applications. If the amplitude and/or phase is spatially varied across the wavefront then it is possible to project ‘acoustic images’. When attempting to form an arbitrary desired static sound field, acoustic holograms are superior to phased arrays due to their significantly higher phase fidelity. However, they lack the dynamic flexibility of phased arrays. Here, we demonstrate how to combine the high-fidelity advantages of acoustic holograms with the dynamic control of phased arrays in the ultrasonic frequency range. Holograms are used with a 64-element phased array, driven with continuous excitation. Moving the position of the projected hologram via phase delays which steer the output beam is demonstrated experimentally. This allows the creation of a much more tightly focused point than with the phased array alone, whilst still being reconfigurable. It also allows the complex movement at a water-air interface of a “phase surfer” along a phase track or the manipulation of a more arbitrarily shaped particle via amplitude traps. Furthermore, a particle manipulation device with two emitters and a single split hologram is demonstrated that allows the positioning of a “phase surfer” along a 1D axis. This paper opens the door for new applications with complex manipulation of ultrasound whilst minimising the complexity and cost of the apparatus.

pf

link (url) DOI [BibTex]

link (url) DOI [BibTex]


Decoding subcategories of human bodies from both body- and face-responsive cortical regions
Decoding subcategories of human bodies from both body- and face-responsive cortical regions

Foster, C., Zhao, M., Romero, J., Black, M. J., Mohler, B. J., Bartels, A., Bülthoff, I.

NeuroImage, 202(15):116085, November 2019 (article)

Abstract
Our visual system can easily categorize objects (e.g. faces vs. bodies) and further differentiate them into subcategories (e.g. male vs. female). This ability is particularly important for objects of social significance, such as human faces and bodies. While many studies have demonstrated category selectivity to faces and bodies in the brain, how subcategories of faces and bodies are represented remains unclear. Here, we investigated how the brain encodes two prominent subcategories shared by both faces and bodies, sex and weight, and whether neural responses to these subcategories rely on low-level visual, high-level visual or semantic similarity. We recorded brain activity with fMRI while participants viewed faces and bodies that varied in sex, weight, and image size. The results showed that the sex of bodies can be decoded from both body- and face-responsive brain areas, with the former exhibiting more consistent size-invariant decoding than the latter. Body weight could also be decoded in face-responsive areas and in distributed body-responsive areas, and this decoding was also invariant to image size. The weight of faces could be decoded from the fusiform body area (FBA), and weight could be decoded across face and body stimuli in the extrastriate body area (EBA) and a distributed body-responsive area. The sex of well-controlled faces (e.g. excluding hairstyles) could not be decoded from face- or body-responsive regions. These results demonstrate that both face- and body-responsive brain regions encode information that can distinguish the sex and weight of bodies. Moreover, the neural patterns corresponding to sex and weight were invariant to image size and could sometimes generalize across face and body stimuli, suggesting that such subcategorical information is encoded with a high-level visual or semantic code.

ps

paper pdf DOI [BibTex]

paper pdf DOI [BibTex]


AirCap -- Aerial Outdoor Motion Capture
AirCap – Aerial Outdoor Motion Capture

Ahmad, A., Price, E., Tallamraju, R., Saini, N., Lawless, G., Ludwig, R., Martinovic, I., Bülthoff, H. H., Black, M. J.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019), Workshop on Aerial Swarms, November 2019 (misc)

Abstract
This paper presents an overview of the Grassroots project Aerial Outdoor Motion Capture (AirCap) running at the Max Planck Institute for Intelligent Systems. AirCap's goal is to achieve markerless, unconstrained, human motion capture (mocap) in unknown and unstructured outdoor environments. To that end, we have developed an autonomous flying motion capture system using a team of aerial vehicles (MAVs) with only on-board, monocular RGB cameras. We have conducted several real robot experiments involving up to 3 aerial vehicles autonomously tracking and following a person in several challenging scenarios using our approach of active cooperative perception developed in AirCap. Using the images captured by these robots during the experiments, we have demonstrated a successful offline body pose and shape estimation with sufficiently high accuracy. Overall, we have demonstrated the first fully autonomous flying motion capture system involving multiple robots for outdoor scenarios.

ps

Talk slides Project Page Project Page [BibTex]

Talk slides Project Page Project Page [BibTex]


A Helical Microrobot with an Optimized Propeller-Shape for Propulsion in Viscoelastic Biological Media
A Helical Microrobot with an Optimized Propeller-Shape for Propulsion in Viscoelastic Biological Media

Li., D., Jeong, M., Oren, E., Yu, T., Qiu, T.

Robotics, 8, pages: 87, MDPI, October 2019 (article)

Abstract
One major challenge for microrobots is to penetrate and effectively move through viscoelastic biological tissues. Most existing microrobots can only propel in viscous liquids. Recent advances demonstrate that sub-micron robots can actively penetrate nanoporous biological tissue, such as the vitreous of the eye. However, it is still difficult to propel a micron-sized device through dense biological tissue. Here, we report that a special twisted helical shape together with a high aspect ratio in cross-section permit a microrobot with a diameter of hundreds-of-micrometers to move through mouse liver tissue. The helical microrobot is driven by a rotating magnetic field and localized by ultrasound imaging inside the tissue. The twisted ribbon is made of molybdenum and a sharp tip is chemically etched to generate a higher pressure at the edge of the propeller to break the biopolymeric network of the dense tissue.

pf

link (url) DOI [BibTex]


Acoustic Holographic Cell Patterning in a Biocompatible Hydrogel
Acoustic Holographic Cell Patterning in a Biocompatible Hydrogel

Ma, Z., Holle, A., Melde, K., Qiu, T., Poeppel, K., Kadiri, V., Fischer, P.

Adv. Mat., October 2019 (article)

Abstract
Acoustophoresis is promising as a rapid, biocompatible, non-contact cell manipulation method, where cells are arranged along the nodes or antinodes of the acoustic field. Typically, the acoustic field is formed in a resonator, which results in highly symmetric regular patterns. However, arbitrary, non-symmetrically shaped cell assemblies are necessary to obtain the irregular cellular arrangements found in biological tissues. We show that arbitrarily shaped cell patterns can be obtained from the complex acoustic field distribution defined by an acoustic hologram. Attenuation of the sound field induces localized acoustic streaming and the resultant convection flow gently delivers the suspended cells to the image plane where they form the designed pattern. We show that the process can be implemented in a biocompatible collagen solution, which can then undergo gelation to immobilize the cell pattern inside the viscoelastic matrix. The patterned cells exhibit F-actin-based protrusions, which indicates that the cells grow and thrive within the matrix. Cell viability assays and brightfield imaging after one week confirm cell survival and that the patterns persist. Acoustophoretic cell manipulation by holographic fields thus holds promise for non-contact, long-range, long-term cellular pattern formation, with a wide variety of potential applications in tissue engineering and mechanobiology.

pf

link (url) DOI [BibTex]


A High-Fidelity Phantom for the Simulation and Quantitative Evaluation of Transurethral Resection of the Prostate
A High-Fidelity Phantom for the Simulation and Quantitative Evaluation of Transurethral Resection of the Prostate

Choi, E., Adams, F., Gengenbacher, A., Schlager, D., Palagi, S., Müller, P., Wetterauer, U., Miernik, A., Fischer, P., Qiu, T.

Annals of Biomed. Eng., October 2019 (article)

Abstract
Transurethral resection of the prostate (TURP) is a minimally invasive endoscopic procedure that requires experience and skill of the surgeon. To permit surgical training under realistic conditions we report a novel phantom of the human prostate that can be resected with TURP. The phantom mirrors the anatomy and haptic properties of the gland and permits quantitative evaluation of important surgical performance indicators. Mixtures of soft materials are engineered to mimic the physical properties of the human tissue, including the mechanical strength, the electrical and thermal conductivity, and the appearance under an endoscope. Electrocautery resection of the phantom closely resembles the procedure on human tissue. Ultrasound contrast agent was applied to the central zone, which was not detectable by the surgeon during the surgery but showed high contrast when imaged after the surgery, to serve as a label for the quantitative evaluation of the surgery. Quantitative criteria for performance assessment are established and evaluated by automated image analysis. We present the workflow of a surgical simulation on a prostate phantom followed by quantitative evaluation of the surgical performance. Surgery on the phantom is useful for medical training, and enables the development and testing of endoscopic and minimally invasive surgical instruments.

pf

link (url) DOI [BibTex]

link (url) DOI [BibTex]


Interactive Materials – Drivers of Future Robotic Systems
Interactive Materials – Drivers of Future Robotic Systems

Fischer, P.

Adv. Mat., October 2019 (article)

Abstract
A robot senses its environment, processes the sensory information, acts in response to these inputs, and possibly communicates with the outside world. Robots generally achieve these tasks with electronics-based hardware or by receiving inputs from some external hardware. In contrast, simple microorganisms can autonomously perceive, act, and communicate via purely physicochemical processes in soft material systems. A key property of biological systems is that they are built from energy-consuming ‘active’ units. Exciting developments in material science show that even very simple artificial active building blocks can show surprisingly rich emergent behaviors. Active non-equilibrium systems are therefore predicted to play an essential role to realize interactive materials. A major challenge is to find robust ways to couple and integrate the energy-consuming building blocks to the mechanical structure of the material. However, success in this endeavor will lead to a new generation of sophisticated micro- and soft-robotic systems that can operate autonomously.

pf

[BibTex]


Active Perception based Formation Control for Multiple Aerial Vehicles
Active Perception based Formation Control for Multiple Aerial Vehicles

Tallamraju, R., Price, E., Ludwig, R., Karlapalem, K., Bülthoff, H. H., Black, M. J., Ahmad, A.

IEEE Robotics and Automation Letters, Robotics and Automation Letters, 4(4):4491-4498, IEEE, October 2019 (article)

Abstract
We present a novel robotic front-end for autonomous aerial motion-capture (mocap) in outdoor environments. In previous work, we presented an approach for cooperative detection and tracking (CDT) of a subject using multiple micro-aerial vehicles (MAVs). However, it did not ensure optimal view-point configurations of the MAVs to minimize the uncertainty in the person's cooperatively tracked 3D position estimate. In this article, we introduce an active approach for CDT. In contrast to cooperatively tracking only the 3D positions of the person, the MAVs can actively compute optimal local motion plans, resulting in optimal view-point configurations, which minimize the uncertainty in the tracked estimate. We achieve this by decoupling the goal of active tracking into a quadratic objective and non-convex constraints corresponding to angular configurations of the MAVs w.r.t. the person. We derive this decoupling using Gaussian observation model assumptions within the CDT algorithm. We preserve convexity in optimization by embedding all the non-convex constraints, including those for dynamic obstacle avoidance, as external control inputs in the MPC dynamics. Multiple real robot experiments and comparisons involving 3 MAVs in several challenging scenarios are presented.

ps

pdf DOI Project Page [BibTex]

pdf DOI Project Page [BibTex]


Arrays of plasmonic nanoparticle dimers with defined nanogap spacers
Arrays of plasmonic nanoparticle dimers with defined nanogap spacers

Jeong, H., Adams, M. C., Guenther, J., Alarcon-Correa, M., Kim, I., Choi, E., Miksch, C., Mark, A. F. M., Mark, A. G., Fischer, P.

ACS Nano, September 2019 (article)

Abstract
Plasmonic molecules are building blocks of metallic nanostructures that give rise to intriguing optical phenomena with similarities to those seen in molecular systems. The ability to design plasmonic hybrid structures and molecules with nanometric resolution would enable applications in optical metamaterials and sensing that presently cannot be demonstrated, because of a lack of suitable fabrication methods allowing the structural control of the plasmonic atoms on a large scale. Here we demonstrate a wafer-scale “lithography-free” parallel fabrication scheme to realize nanogap plasmonic meta-molecules with precise control over their size, shape, material, and orientation. We demonstrate how we can tune the corresponding coupled resonances through the entire visible spectrum. Our fabrication method, based on glancing angle physical vapor deposition with gradient shadowing, permits critical parameters to be varied across the wafer and thus is ideally suited to screen potential structures. We obtain billions of aligned dimer structures with controlled variation of the spectral properties across the wafer. We spectroscopically map the plasmonic resonances of gold dimer structures and show that they not only are in good agreement with numerically modeled spectra, but also remain functional, at least for a year, in ambient conditions.

pf

link (url) DOI [BibTex]


Method for providing a three dimensional body model
Method for providing a three dimensional body model

Loper, M., Mahmood, N., Black, M.

September 2019, U.S.~Patent 10,417,818 (misc)

Abstract
A method for providing a three-dimensional body model which may be applied for an animation, based on a moving body, wherein the method comprises providing a parametric three-dimensional body model, which allows shape and pose variations; applying a standard set of body markers; optimizing the set of body markers by generating an additional set of body markers and applying the same for providing 3D coordinate marker signals for capturing shape and pose of the body and dynamics of soft tissue; and automatically providing an animation by processing the 3D coordinate marker signals in order to provide a personalized three-dimensional body model, based on estimated shape and an estimated pose of the body by means of predicted marker locations.

ps

MoSh Project pdf [BibTex]


3D Morphable Face Models - Past, Present and Future
3D Morphable Face Models - Past, Present and Future

Egger, B., Smith, W. A. P., Tewari, A., Wuhrer, S., Zollhoefer, M., Beeler, T., Bernard, F., Bolkart, T., Kortylewski, A., Romdhani, S., Theobalt, C., Blanz, V., Vetter, T.

arxiv preprint arXiv:1909.01815, September 2019 (article)

Abstract
In this paper, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed. The challenges in building and applying these models, namely capture, modeling, image formation,and image analysis, are still active research topics, and we review the state-of-the-art in each of these areas. We also look ahead, identifying unsolved challenges, proposing directions for future research and highlighting the broad range of current and future applications.

ps

paper project page [BibTex]

paper project page [BibTex]


Genetically modified M13 bacteriophage nanonets for enzyme catalysis and recovery
Genetically modified M13 bacteriophage nanonets for enzyme catalysis and recovery

Kadiri, V. M., Alarcon-Correa, M., Guenther, J. P., Ruppert, J., Bill, J., Rothenstein, D., Fischer, P.

Catalysts, 9, pages: 723, August 2019 (article)

Abstract
Enzyme-based biocatalysis exhibits multiple advantages over inorganic catalysts, including the biocompatibility and the unchallenged specificity of enzymes towards their substrate. The recovery and repeated use of enzymes is essential for any realistic application in biotechnology, but is not easily achieved with current strategies. For this purpose, enzymes are often immobilized on inorganic scaffolds, which could entail a reduction of the enzymes’ activity. Here, we show that immobilization to a nano-scaled biological scaffold, a nanonetwork of end-to-end cross-linked M13 bacteriophages, ensures high enzymatic activity and at the same time allows for the simple recovery of the enzymes. The bacteriophages have been genetically engineered to express AviTags at their ends, which permit biotinylation and their specific end-to-end self-assembly while allowing space on the major coat protein for enzyme coupling. We demonstrate that the phages form nanonetwork structures and that these so-called nanonets remain highly active even after re-using the nanonets multiple times in a flow-through reactor.

pf

link (url) DOI [BibTex]


Light-controlled micromotors and soft microrobots
Light-controlled micromotors and soft microrobots

Palagi, S., Singh, D. P., Fischer, P.

Adv. Opt. Mat., 7, pages: 1900370, August 2019 (article)

Abstract
Mobile microscale devices and microrobots can be powered by catalytic reactions (chemical micromotors) or by external fields. This report is focused on the role of light as a versatile means for wirelessly powering and controlling such microdevices. Recent advances in the development of autonomous micromotors are discussed, where light permits their actuation with unprecedented control and thereby enables advances in the field of active matter. In addition, structuring the light field is a new means to drive soft microrobots that are based on (photo‐) responsive polymers. The behavior of the two main classes of thermo‐ and photoresponsive polymers adopted in microrobotics (poly(N‐isopropylacrylamide) and liquid‐crystal elastomers) is analyzed, and recent applications are reported. The advantages and limitations of controlling micromotors and microrobots by light are reviewed, and some of the remaining challenges in the development of novel photo‐active materials for micromotors and microrobots are discussed.

pf

link (url) DOI [BibTex]


Learning and Tracking the {3D} Body Shape of Freely Moving Infants from {RGB-D} sequences
Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences

Hesse, N., Pujades, S., Black, M., Arens, M., Hofmann, U., Schroeder, S.

Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019 (article)

Abstract
Statistical models of the human body surface are generally learned from thousands of high-quality 3D scans in predefined poses to cover the wide variety of human body shapes and articulations. Acquisition of such data requires expensive equipment, calibration procedures, and is limited to cooperative subjects who can understand and follow instructions, such as adults. We present a method for learning a statistical 3D Skinned Multi-Infant Linear body model (SMIL) from incomplete, low-quality RGB-D sequences of freely moving infants. Quantitative experiments show that SMIL faithfully represents the RGB-D data and properly factorizes the shape and pose of the infants. To demonstrate the applicability of SMIL, we fit the model to RGB-D sequences of freely moving infants and show, with a case study, that our method captures enough motion detail for General Movements Assessment (GMA), a method used in clinical practice for early detection of neurodevelopmental disorders in infants. SMIL provides a new tool for analyzing infant shape and movement and is a step towards an automated system for GMA.

ps

pdf Journal DOI [BibTex]

pdf Journal DOI [BibTex]


Self-Assembled Phage-Based Colloids for High Localized Enzymatic Activity
Self-Assembled Phage-Based Colloids for High Localized Enzymatic Activity

Alarcon-Correa, M., Guenther, J., Troll, J., Kadiri, V. M., Bill, J., Fischer, P., Rothenstein, D.

ACS Nano, March 2019 (article)

Abstract
Catalytically active colloids are model systems for chemical motors and active matter. It is desirable to replace the inorganic catalysts and the toxic fuels that are often used, with biocompatible enzymatic reactions. However, compared to inorganic catalysts, enzyme-coated colloids tend to exhibit less activity. Here, we show that the self-assembly of genetically engineered M13 bacteriophages that bind enzymes to magnetic beads ensures high and localized enzymatic activity. These phage-decorated colloids provide a proteinaceous environment for directed enzyme immobilization. The magnetic properties of the colloidal carrier particle permit repeated enzyme recovery from a reaction solution, while the enzymatic activity is retained. Moreover, localizing the phage-based construct with a magnetic field in a microcontainer allows the enzyme-phage-colloids to function as an enzymatic micropump, where the enzymatic reaction generates a fluid flow. This system shows the fastest fluid flow reported to date by a biocompatible enzymatic micropump. In addition, it is functional in complex media including blood where the enzyme driven micropump can be powered at the physiological blood-urea concentration.

pf

link (url) DOI [BibTex]


Absolute diffusion measurements of active enzyme solutions by NMR
Absolute diffusion measurements of active enzyme solutions by NMR

Guenther, J., Majer, G., Fischer, P.

J. Chem. Phys., 150(124201), March 2019 (article)

Abstract
The diffusion of enzymes is of fundamental importance for many biochemical processes. Enhanced or directed enzyme diffusion can alter the accessibility of substrates and the organization of enzymes within cells. Several studies based on fluorescence correlation spectroscopy (FCS) report enhanced diffusion of enzymes upon interaction with their substrate or inhibitor. In this context, major importance is given to the enzyme fructose-bisphosphate aldolase, for which enhanced diffusion has been reported even though the catalysed reaction is endothermic. Additionally, enhanced diffusion of tracer particles surrounding the active aldolase enzymes has been reported. These studies suggest that active enzymes can act as chemical motors that self-propel and give rise to enhanced diffusion. However, fluorescence studies of enzymes can, despite several advantages, suffer from artefacts. Here we show that the absolute diffusion coefficients of active enzyme solutions can be determined with Pulsed Field Gradient Nuclear Magnetic Resonance (PFG-NMR). The advantage of PFG-NMR is that the motion of the molecule of interest is directly observed in its native state without the need for any labelling. Further, PFG-NMR is model-free and thus yields absolute diffusion constants. Our PFG-NMR experiments of solutions containing active fructose-bisphosphate aldolase from rabbit muscle do not show any diffusion enhancement for the active enzymes nor the surrounding molecules. Additionally, we do not observe any diffusion enhancement of aldolase in the presence of its inhibitor pyrophosphate.

pf

link (url) DOI [BibTex]


Chemical Nanomotors at the Gram Scale Form a Dense Active Optorheological Medium
Chemical Nanomotors at the Gram Scale Form a Dense Active Optorheological Medium

Choudhury, U., Singh, D. P., Qiu, T., Fischer, P.

Adv. Mat., (1807382), Febuary 2019 (article)

Abstract
The rheological properties of a colloidal suspension are a function of the concentration of the colloids and their interactions. While suspensions of passive colloids are well studied and have been shown to form crystals, gels, and glasses, examples of energy‐consuming “active” colloidal suspensions are still largely unexplored. Active suspensions of biological matter, such as motile bacteria or dense mixtures of active actin–motor–protein mixtures have, respectively, reveals superfluid‐like and gel‐like states. Attractive inanimate systems for active matter are chemically self‐propelled particles. It has so far been challenging to use these swimming particles at high enough densities to affect the bulk material properties of the suspension. Here, it is shown that light‐triggered asymmetric titanium dioxide that self‐propel, can be obtained in large quantities, and self‐organize to make a gram‐scale active medium. The suspension shows an activity‐dependent tenfold reversible change in its bulk viscosity.

pf

link (url) DOI [BibTex]


 Perceptual Effects of Inconsistency in Human Animations
Perceptual Effects of Inconsistency in Human Animations

Kenny, S., Mahmood, N., Honda, C., Black, M. J., Troje, N. F.

ACM Trans. Appl. Percept., 16(1):2:1-2:18, Febuary 2019 (article)

Abstract
The individual shape of the human body, including the geometry of its articulated structure and the distribution of weight over that structure, influences the kinematics of a person’s movements. How sensitive is the visual system to inconsistencies between shape and motion introduced by retargeting motion from one person onto the shape of another? We used optical motion capture to record five pairs of male performers with large differences in body weight, while they pushed, lifted, and threw objects. From these data, we estimated both the kinematics of the actions as well as the performer’s individual body shape. To obtain consistent and inconsistent stimuli, we created animated avatars by combining the shape and motion estimates from either a single performer or from different performers. Using these stimuli we conducted three experiments in an immersive virtual reality environment. First, a group of participants detected which of two stimuli was inconsistent. Performance was very low, and results were only marginally significant. Next, a second group of participants rated perceived attractiveness, eeriness, and humanness of consistent and inconsistent stimuli, but these judgements of animation characteristics were not affected by consistency of the stimuli. Finally, a third group of participants rated properties of the objects rather than of the performers. Here, we found strong influences of shape-motion inconsistency on perceived weight and thrown distance of objects. This suggests that the visual system relies on its knowledge of shape and motion and that these components are assimilated into an altered perception of the action outcome. We propose that the visual system attempts to resist inconsistent interpretations of human animations. Actions involving object manipulations present an opportunity for the visual system to reinterpret the introduced inconsistencies as a change in the dynamics of an object rather than as an unexpected combination of body shape and body motion.

ps

publisher pdf DOI [BibTex]

publisher pdf DOI [BibTex]


First Observation of Optical Activity in Hyper-Rayleigh Scattering
First Observation of Optical Activity in Hyper-Rayleigh Scattering

Collins, J., Rusimova, K., Hooper, D., Jeong, H. H., Ohnoutek, L., Pradaux-Caggiano, F., Verbiest, T., Carbery, D., Fischer, P., Valev, V.

Phys. Rev. X, 9(011024), January 2019 (article)

Abstract
Chiral nano- or metamaterials and surfaces enable striking photonic properties, such as negative refractive index and superchiral light, driving promising applications in novel optical components, nanorobotics, and enhanced chiral molecular interactions with light. In characterizing chirality, although nonlinear chiroptical techniques are typically much more sensitive than their linear optical counterparts, separating true chirality from anisotropy is a major challenge. Here, we report the first observation of optical activity in second-harmonic hyper-Rayleigh scattering (HRS). We demonstrate the effect in a 3D isotropic suspension of Ag nanohelices in water. The effect is 5 orders of magnitude stronger than linear optical activity and is well pronounced above the multiphoton luminescence background. Because of its sensitivity, isotropic environment, and straightforward experimental geometry, HRS optical activity constitutes a fundamental experimental breakthrough in chiral photonics for media including nanomaterials, metamaterials, and chemical molecules.

pf

link (url) DOI [BibTex]

link (url) DOI [BibTex]


Perceiving Systems (2016-2018)
Perceiving Systems (2016-2018)
Scientific Advisory Board Report, 2019 (misc)

ps

pdf [BibTex]

pdf [BibTex]


The Virtual Caliper: Rapid Creation of Metrically Accurate Avatars from {3D} Measurements
The Virtual Caliper: Rapid Creation of Metrically Accurate Avatars from 3D Measurements

Pujades, S., Mohler, B., Thaler, A., Tesch, J., Mahmood, N., Hesse, N., Bülthoff, H. H., Black, M. J.

IEEE Transactions on Visualization and Computer Graphics, 25, pages: 1887,1897, IEEE, 2019 (article)

Abstract
Creating metrically accurate avatars is important for many applications such as virtual clothing try-on, ergonomics, medicine, immersive social media, telepresence, and gaming. Creating avatars that precisely represent a particular individual is challenging however, due to the need for expensive 3D scanners, privacy issues with photographs or videos, and difficulty in making accurate tailoring measurements. We overcome these challenges by creating “The Virtual Caliper”, which uses VR game controllers to make simple measurements. First, we establish what body measurements users can reliably make on their own body. We find several distance measurements to be good candidates and then verify that these are linearly related to 3D body shape as represented by the SMPL body model. The Virtual Caliper enables novice users to accurately measure themselves and create an avatar with their own body shape. We evaluate the metric accuracy relative to ground truth 3D body scan data, compare the method quantitatively to other avatar creation tools, and perform extensive perceptual studies. We also provide a software application to the community that enables novices to rapidly create avatars in fewer than five minutes. Not only is our approach more rapid than existing methods, it exports a metrically accurate 3D avatar model that is rigged and skinned.

ps

Project Page IEEE Open Access IEEE Open Access PDF DOI [BibTex]

Project Page IEEE Open Access IEEE Open Access PDF DOI [BibTex]

2017


Active colloidal propulsion over a crystalline surface
Active colloidal propulsion over a crystalline surface

Choudhury, U., Straube, A., Fischer, P., Gibbs, J., Höfling, F.

New Journal of Physics, 19, pages: 125010, December 2017 (article)

Abstract
We study both experimentally and theoretically the dynamics of chemically self-propelled Janus colloids moving atop a two-dimensional crystalline surface. The surface is a hexagonally close-packed monolayer of colloidal particles of the same size as the mobile one. The dynamics of the self-propelled colloid reflects the competition between hindered diffusion due to the periodic surface and enhanced diffusion due to active motion. Which contribution dominates depends on the propulsion strength, which can be systematically tuned by changing the concentration of a chemical fuel. The mean-square displacements obtained from the experiment exhibit enhanced diffusion at long lag times. Our experimental data are consistent with a Langevin model for the effectively two-dimensional translational motion of an active Brownian particle in a periodic potential, combining the confining effects of gravity and the crystalline surface with the free rotational diffusion of the colloid. Approximate analytical predictions are made for the mean-square displacement describing the crossover from free Brownian motion at short times to active diffusion at long times. The results are in semi-quantitative agreement with numerical results of a refined Langevin model that treats translational and rotational degrees of freedom on the same footing.

pf

link (url) DOI [BibTex]

2017


link (url) DOI [BibTex]


Wireless Acoustic-Surface Actuators for Miniaturized Endoscopes
Wireless Acoustic-Surface Actuators for Miniaturized Endoscopes

Qiu, T., Adams, F., Palagi, S., Melde, K., Mark, A. G., Wetterauer, U., Miernik, A., Fischer, P.

ACS Applied Materials & Interfaces, 9(49):42536 - 42543, November 2017 (article)

Abstract
Endoscopy enables minimally invasive procedures in many medical fields, such as urology. However, current endoscopes are normally cable-driven, which limits their dexterity and makes them hard to miniaturize. Indeed current urological endoscopes have an outer diameter of about 3 mm and still only possess one bending degree of freedom. In this paper, we report a novel wireless actuation mechanism that increases the dexterity and that permits the miniaturization of a urological endoscope. The novel actuator consists of thin active surfaces that can be readily attached to any device and are wirelessly powered by ultrasound. The surfaces consist of two-dimensional arrays of micro-bubbles, which oscillate under ultrasound excitation and thereby generate an acoustic streaming force. Bubbles of different sizes are addressed by their unique resonance frequency, thus multiple degrees of freedom can readily be incorporated. Two active miniaturized devices (with a side length of around 1 mm) are demonstrated: a miniaturized mechanical arm that realizes two degrees of freedom, and a flexible endoscope prototype equipped with a camera at the tip. With the flexible endoscope, an active endoscopic examination is successfully performed in a rabbit bladder. This results show the potential medical applicability of surface actuators wirelessly powered by ultrasound penetrating through biological tissues.

pf

link (url) DOI Project Page [BibTex]

link (url) DOI Project Page [BibTex]


Learning a model of facial shape and expression from {4D} scans
Learning a model of facial shape and expression from 4D scans

Li, T., Bolkart, T., Black, M. J., Li, H., Romero, J.

ACM Transactions on Graphics, 36(6):194:1-194:17, November 2017, Two first authors contributed equally (article)

Abstract
The field of 3D face modeling has a large gap between high-end and low-end methods. At the high end, the best facial animation is indistinguishable from real humans, but this comes at the cost of extensive manual labor. At the low end, face capture from consumer depth sensors relies on 3D face models that are not expressive enough to capture the variability in natural facial shape and expression. We seek a middle ground by learning a facial model from thousands of accurately aligned 3D scans. Our FLAME model (Faces Learned with an Articulated Model and Expressions) is designed to work with existing graphics software and be easy to fit to data. FLAME uses a linear shape space trained from 3800 scans of human heads. FLAME combines this linear shape space with an articulated jaw, neck, and eyeballs, pose-dependent corrective blendshapes, and additional global expression from 4D face sequences in the D3DFACS dataset along with additional 4D sequences.We accurately register a template mesh to the scan sequences and make the D3DFACS registrations available for research purposes. In total the model is trained from over 33, 000 scans. FLAME is low-dimensional but more expressive than the FaceWarehouse model and the Basel Face Model. We compare FLAME to these models by fitting them to static 3D scans and 4D sequences using the same optimization method. FLAME is significantly more accurate and is available for research purposes (http://flame.is.tue.mpg.de).

ps

data/model video code chumpy code tensorflow paper supplemental Project Page [BibTex]

data/model video code chumpy code tensorflow paper supplemental Project Page [BibTex]


Investigating Body Image Disturbance in Anorexia Nervosa Using Novel Biometric Figure Rating Scales: A Pilot Study
Investigating Body Image Disturbance in Anorexia Nervosa Using Novel Biometric Figure Rating Scales: A Pilot Study

Mölbert, S. C., Thaler, A., Streuber, S., Black, M. J., Karnath, H., Zipfel, S., Mohler, B., Giel, K. E.

European Eating Disorders Review, 25(6):607-612, November 2017 (article)

Abstract
This study uses novel biometric figure rating scales (FRS) spanning body mass index (BMI) 13.8 to 32.2 kg/m2 and BMI 18 to 42 kg/m2. The aims of the study were (i) to compare FRS body weight dissatisfaction and perceptual distortion of women with anorexia nervosa (AN) to a community sample; (ii) how FRS parameters are associated with questionnaire body dissatisfaction, eating disorder symptoms and appearance comparison habits; and (iii) whether the weight spectrum of the FRS matters. Women with AN (n = 24) and a community sample of women (n = 104) selected their current and ideal body on the FRS and completed additional questionnaires. Women with AN accurately picked the body that aligned best with their actual weight in both FRS. Controls underestimated their BMI in the FRS 14–32 and were accurate in the FRS 18–42. In both FRS, women with AN desired a body close to their actual BMI and controls desired a thinner body. Our observations suggest that body image disturbance in AN is unlikely to be characterized by a visual perceptual disturbance, but rather by an idealization of underweight in conjunction with high body dissatisfaction. The weight spectrum of FRS can influence the accuracy of BMI estimation.

ps

publisher DOI Project Page [BibTex]


Embodied Hands: Modeling and Capturing Hands and Bodies Together
Embodied Hands: Modeling and Capturing Hands and Bodies Together

Romero, J., Tzionas, D., Black, M. J.

ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):245:1-245:17, 245:1–245:17, ACM, November 2017 (article)

Abstract
Humans move their hands and bodies together to communicate and solve tasks. Capturing and replicating such coordinated activity is critical for virtual characters that behave realistically. Surprisingly, most methods treat the 3D modeling and tracking of bodies and hands separately. Here we formulate a model of hands and bodies interacting together and fit it to full-body 4D sequences. When scanning or capturing the full body in 3D, hands are small and often partially occluded, making their shape and pose hard to recover. To cope with low-resolution, occlusion, and noise, we develop a new model called MANO (hand Model with Articulated and Non-rigid defOrmations). MANO is learned from around 1000 high-resolution 3D scans of hands of 31 subjects in a wide variety of hand poses. The model is realistic, low-dimensional, captures non-rigid shape changes with pose, is compatible with standard graphics packages, and can fit any human hand. MANO provides a compact mapping from hand poses to pose blend shape corrections and a linear manifold of pose synergies. We attach MANO to a standard parameterized 3D body shape model (SMPL), resulting in a fully articulated body and hand model (SMPL+H). We illustrate SMPL+H by fitting complex, natural, activities of subjects captured with a 4D scanner. The fitting is fully automatic and results in full body models that move naturally with detailed hand motions and a realism not seen before in full body performance capture. The models and data are freely available for research purposes at http://mano.is.tue.mpg.de.

ps

website youtube paper suppl video link (url) DOI Project Page [BibTex]

website youtube paper suppl video link (url) DOI Project Page [BibTex]


An Online Scalable Approach to Unified Multirobot Cooperative Localization and Object Tracking
An Online Scalable Approach to Unified Multirobot Cooperative Localization and Object Tracking

Ahmad, A., Lawless, G., Lima, P.

IEEE Transactions on Robotics (T-RO), 33, pages: 1184 - 1199, October 2017 (article)

Abstract
In this article we present a unified approach for multi-robot cooperative simultaneous localization and object tracking based on particle filters. Our approach is scalable with respect to the number of robots in the team. We introduce a method that reduces, from an exponential to a linear growth, the space and computation time requirements with respect to the number of robots in order to maintain a given level of accuracy in the full state estimation. Our method requires no increase in the number of particles with respect to the number of robots. However, in our method each particle represents a full state hypothesis, leading to the linear dependency on the number of robots of both space and time complexity. The derivation of the algorithm implementing our approach from a standard particle filter algorithm and its complexity analysis are presented. Through an extensive set of simulation experiments on a large number of randomized datasets, we demonstrate the correctness and efficacy of our approach. Through real robot experiments on a standardized open dataset of a team of four soccer playing robots tracking a ball, we evaluate our method's estimation accuracy with respect to the ground truth values. Through comparisons with other methods based on i) nonlinear least squares minimization and ii) joint extended Kalman filter, we further highlight our method's advantages. Finally, we also present a robustness test for our approach by evaluating it under scenarios of communication and vision failure in teammate robots.

ps

Published Version link (url) DOI [BibTex]

Published Version link (url) DOI [BibTex]


Active Acoustic Surfaces Enable the Propulsion of a Wireless Robot
Active Acoustic Surfaces Enable the Propulsion of a Wireless Robot

Qiu, T., Palagi, S., Mark, A. G., Melde, K., Adams, F., Fischer, P.

Advanced Materials Interfaces, 4(21):1700933, September 2017 (article)

Abstract
A major challenge that prevents the miniaturization of mechanically actuated systems is the lack of suitable methods that permit the efficient transfer of power to small scales. Acoustic energy holds great potential, as it is wireless, penetrates deep into biological tissues, and the mechanical vibrations can be directly converted into directional forces. Recently, active acoustic surfaces are developed that consist of 2D arrays of microcavities holding microbubbles that can be excited with an external acoustic field. At resonance, the surfaces give rise to acoustic streaming and thus provide a highly directional propulsive force. Here, this study advances these wireless surface actuators by studying their force output as the size of the bubble-array is increased. In particular, a general method is reported to dramatically improve the propulsive force, demonstrating that the surface actuators are actually able to propel centimeter-scale devices. To prove the flexibility of the functional surfaces as wireless ready-to-attach actuator, a mobile mini-robot capable of propulsion in water along multiple directions is presented. This work paves the way toward effectively exploiting acoustic surfaces as a novel wireless actuation scheme at small scales.

pf

link (url) DOI Project Page [BibTex]


Corrosion-Protected Hybrid Nanoparticles
Corrosion-Protected Hybrid Nanoparticles

Jeong, H. H., Alarcon-Correa, M., Mark, A. G., Son, K., Lee, T., Fischer, P.

Advanced Science, 4(12):1700234, September 2017 (article)

Abstract
Nanoparticles composed of functional materials hold great promise for applications due to their unique electronic, optical, magnetic, and catalytic properties. However, a number of functional materials are not only difficult to fabricate at the nanoscale, but are also chemically unstable in solution. Hence, protecting nanoparticles from corrosion is a major challenge for those applications that require stability in aqueous solutions and biological fluids. Here, this study presents a generic scheme to grow hybrid 3D nanoparticles that are completely encapsulated by a nm thick protective shell. The method consists of vacuum-based growth and protection, and combines oblique physical vapor deposition with atomic layer deposition. It provides wide flexibility in the shape and composition of the nanoparticles, and the environments against which particles are protected. The work demonstrates the approach with multifunctional nanoparticles possessing ferromagnetic, plasmonic, and chiral properties. The present scheme allows nanocolloids, which immediately corrode without protection, to remain functional, at least for a week, in acidic solutions.

pf

link (url) DOI [BibTex]

link (url) DOI [BibTex]


Parameterized Model of {2D} Articulated Human Shape
Parameterized Model of 2D Articulated Human Shape

Black, M. J., Freifeld, O., Weiss, A., Loper, M., Guan, P.

September 2017, U.S.~Patent 9,761,060 (misc)

Abstract
Disclosed are computer-readable devices, systems and methods for generating a model of a clothed body. The method includes generating a model of an unclothed human body, the model capturing a shape or a pose of the unclothed human body, determining two-dimensional contours associated with the model, and computing deformations by aligning a contour of a clothed human body with a contour of the unclothed human body. Based on the two-dimensional contours and the deformations, the method includes generating a first two-dimensional model of the unclothed human body, the first two-dimensional model factoring the deformations of the unclothed human body into one or more of a shape variation component, a viewpoint change, and a pose variation and learning an eigen-clothing model using principal component analysis applied to the deformations, wherein the eigen-clothing model classifies different types of clothing, to yield a second two-dimensional model of a clothed human body.

ps

Google Patents [BibTex]


Crowdshaping Realistic {3D} Avatars with Words
Crowdshaping Realistic 3D Avatars with Words

Streuber, S., Ramirez, M. Q., Black, M., Zuffi, S., O’Toole, A., Hill, M. Q., Hahn, C. A.

August 2017, Application PCT/EP2017/051954 (misc)

Abstract
A method for generating a body shape, comprising the steps: - receiving one or more linguistic descriptors related to the body shape; - retrieving an association between the one or more linguistic descriptors and a body shape; and - generating the body shape, based on the association.

ps

Google Patents [BibTex]

Google Patents [BibTex]


Non-Equilibrium Assembly of Light-Activated Colloidal Mixtures
Non-Equilibrium Assembly of Light-Activated Colloidal Mixtures

Singh, D. P., Choudhury, U., Fischer, P., Mark, A. G.

Advanced Materials, 29, pages: 1701328, June 2017, 32 (article)

Abstract
The collective phenomena exhibited by artificial active matter systems present novel routes to fabricating out-of-equilibrium microscale assemblies. Here, the crystallization of passive silica colloids into well-controlled 2D assemblies is shown, which is directed by a small number of self-propelled active colloids. The active colloids are titania–silica Janus particles that are propelled when illuminated by UV light. The strength of the attractive interaction and thus the extent of the assembled clusters can be regulated by the light intensity. A remarkably small number of the active colloids is sufficient to induce the assembly of the dynamic crystals. The approach produces rationally designed colloidal clusters and crystals with controllable sizes, shapes, and symmetries. This multicomponent active matter system offers the possibility of obtaining structures and assemblies that cannot be found in equilibrium systems.

pf

link (url) DOI [BibTex]


Nanodiamonds That Swim
Nanodiamonds That Swim

Kim, J. T., Choudhury, U., Hyeon-Ho, J., Fischer, P.

Advanced Materials, 29(30):1701024, June 2017, Back Cover (article)

Abstract
Nanodiamonds are emerging as nanoscale quantum probes for bio-sensing and imaging. This necessitates the development of new methods to accurately manipulate their position and orientation in aqueous solutions. The realization of an “active” nanodiamond (ND) swimmer in fluids, composed of a ND crystal containing nitrogen vacancy centers and a light-driven self-thermophoretic micromotor, is reported. The swimmer is propelled by a local temperature gradient created by laser illumination on its metal-coated side. Its locomotion—from translational to rotational motion—is successfully controlled by shape-dependent hydrodynamic interactions. The precise engineering of the swimmer's geometry is achieved by self-assembly combined with physical vapor shadow growth. The optical addressability of the suspended ND swimmers is demonstrated by observing the electron spin resonance in the presence of magnetic fields. Active motion at the nanoscale enables new sensing capabilities combined with active transport including, potentially, in living organisms.

pf

link (url) DOI [BibTex]

link (url) DOI [BibTex]


System and method for simulating realistic clothing
System and method for simulating realistic clothing

Black, M. J., Guan, P.

June 2017, U.S.~Patent 9,679,409 B2 (misc)

Abstract
Systems, methods, and computer-readable storage media for simulating realistic clothing. The system generates a clothing deformation model for a clothing type, wherein the clothing deformation model factors a change of clothing shape due to rigid limb rotation, pose-independent body shape, and pose-dependent deformations. Next, the system generates a custom-shaped garment for a given body by mapping, via the clothing deformation model, body shape parameters to clothing shape parameters. The system then automatically dresses the given body with the custom- shaped garment.

ps

Google Patents pdf [BibTex]


Soft 3D-Printed Phantom of the Human Kidney with Collecting System
Soft 3D-Printed Phantom of the Human Kidney with Collecting System

Adams, F., Qiu, T., Mark, A., Fritz, B., Kramer, L., Schlager, D., Wetterauer, U., Miernik, A., Fischer, P.

Ann. of Biomed. Eng., 45(4):963-972, April 2017 (article)

Abstract
Organ models are used for planning and simulation of operations, developing new surgical instruments, and training purposes. There is a substantial demand for in vitro organ phantoms, especially in urological surgery. Animal models and existing simulator systems poorly mimic the detailed morphology and the physical properties of human organs. In this paper, we report a novel fabrication process to make a human kidney phantom with realistic anatomical structures and physical properties. The detailed anatomical structure was directly acquired from high resolution CT data sets of human cadaveric kidneys. The soft phantoms were constructed using a novel technique that combines 3D wax printing and polymer molding. Anatomical details and material properties of the phantoms were validated in detail by CT scan, ultrasound, and endoscopy. CT reconstruction, ultrasound examination, and endoscopy showed that the designed phantom mimics a real kidney's detailed anatomy and correctly corresponds to the targeted human cadaver's upper urinary tract. Soft materials with a tensile modulus of 0.8-1.5 MPa as well as biocompatible hydrogels were used to mimic human kidney tissues. We developed a method of constructing 3D organ models from medical imaging data using a 3D wax printing and molding process. This method is cost-effective means for obtaining a reproducible and robust model suitable for surgical simulation and training purposes.

pf

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Pattern formation and collective effects in populations of magnetic microswimmers
Pattern formation and collective effects in populations of magnetic microswimmers

Vach, P. J., (Walker) Schamel, D., Fischer, P., Fratzl, P., Faivre, D.

J. of Phys. D: Appl. Phys., 50(11):11LT03, Febuary 2017 (article)

Abstract
Self-propelled particles are one prototype of synthetic active matter used to understand complex biological processes, such as the coordination of movement in bacterial colonies or schools of fishes. Collective patterns such as clusters were observed for such systems, reproducing features of biological organization. However, one limitation of this model is that the synthetic assemblies are made of identical individuals. Here we introduce an active system based on magnetic particles at colloidal scales. We use identical but also randomly-shaped magnetic micropropellers and show that they exhibit dynamic and reversible pattern formation.

pf

DOI [BibTex]

DOI [BibTex]


On-chip enzymatic microbiofuel cell-powered integrated circuits
On-chip enzymatic microbiofuel cell-powered integrated circuits

Mark, A. G., Suraniti, E., Roche, J., Richter, H., Kuhn, A., Mano, N., Fischer, P.

Lab on a Chip, 17(10):1761-1768, Febuary 2017, Recent HOT Article (article)

Abstract
A variety of diagnostic and therapeutic medical technologies rely on long term implantation of an electronic device to monitor or regulate a patient's condition. One proposed approach to powering these devices is to use a biofuel cell to convert the chemical energy from blood nutrients into electrical current to supply the electronics. We present here an enzymatic microbiofuel cell whose electrodes are directly integrated into a digital electronic circuit. Glucose oxidizing and oxygen reducing enzymes are immobilized on microelectrodes of an application specific integrated circuit (ASIC) using redox hydrogels to produce an enzymatic biofuel cell, capable of harvesting electrical power from just a single droplet of 5 mM glucose solution. Optimisation of the fuel cell voltage and power to match the requirements of the electronics allow self-powered operation of the on-board digital circuitry. This study represents a step towards implantable self-powered electronic devices that gather their energy from physiological fluids.

Recent HOT Article.

pf

DOI [BibTex]

DOI [BibTex]


Strong Rotational Anisotropies Affect Nonlinear Chiral Metamaterials
Strong Rotational Anisotropies Affect Nonlinear Chiral Metamaterials

Hooper, D. C., Mark, A. G., Kuppe, C., Collins, J. T., Fischer, P., Valev, V. K.

Advanced Materials, 29(13):1605110, January 2017 (article)

Abstract
Masked by rotational anisotropies, the nonlinear chiroptical response of a metamaterial is initially completely inaccessible. Upon rotating the sample the chiral information emerges. These results highlight the need for a general method to extract the true chiral contributions to the nonlinear optical signal, which would be hugely valuable in the present context of increasingly complex chiral meta/nanomaterials.

pf

DOI [BibTex]

DOI [BibTex]


Early Stopping Without a Validation Set
Early Stopping Without a Validation Set

Mahsereci, M., Balles, L., Lassner, C., Hennig, P.

arXiv preprint arXiv:1703.09580, 2017 (article)

Abstract
Early stopping is a widely used technique to prevent poor generalization performance when training an over-expressive model by means of gradient-based optimization. To find a good point to halt the optimizer, a common practice is to split the dataset into a training and a smaller validation set to obtain an ongoing estimate of the generalization performance. In this paper we propose a novel early stopping criterion which is based on fast-to-compute, local statistics of the computed gradients and entirely removes the need for a held-out validation set. Our experiments show that this is a viable approach in the setting of least-squares and logistic regression as well as neural networks.

ps pn

link (url) Project Page Project Page [BibTex]


Data-Driven Physics for Human Soft Tissue Animation
Data-Driven Physics for Human Soft Tissue Animation

Kim, M., Pons-Moll, G., Pujades, S., Bang, S., Kim, J., Black, M. J., Lee, S.

ACM Transactions on Graphics, (Proc. SIGGRAPH), 36(4):54:1-54:12, 2017 (article)

Abstract
Data driven models of human poses and soft-tissue deformations can produce very realistic results, but they only model the visible surface of the human body and cannot create skin deformation due to interactions with the environment. Physical simulations can generalize to external forces, but their parameters are difficult to control. In this paper, we present a layered volumetric human body model learned from data. Our model is composed of a data-driven inner layer and a physics-based external layer. The inner layer is driven with a volumetric statistical body model (VSMPL). The soft tissue layer consists of a tetrahedral mesh that is driven using the finite element method (FEM). Model parameters, namely the segmentation of the body into layers and the soft tissue elasticity, are learned directly from 4D registrations of humans exhibiting soft tissue deformations. The learned two layer model is a realistic full-body avatar that generalizes to novel motions and external forces. Experiments show that the resulting avatars produce realistic results on held out sequences and react to external forces. Moreover, the model supports the retargeting of physical properties from one avatar when they share the same topology.

ps

video paper link (url) Project Page [BibTex]

video paper link (url) Project Page [BibTex]


Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs
Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

(Best Paper, Eurographics 2017)

Marcard, T. V., Rosenhahn, B., Black, M., Pons-Moll, G.

Computer Graphics Forum 36(2), Proceedings of the 38th Annual Conference of the European Association for Computer Graphics (Eurographics), pages: 349-360 , 2017 (article)

Abstract
We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body. Since the problem is heavily under-constrained, previous methods either use a large number of sensors, which is intrusive, or they require additional video input. We take a different approach and constrain the problem by: (i) making use of a realistic statistical body model that includes anthropometric constraints and (ii) using a joint optimization framework to fit the model to orientation and acceleration measurements over multiple frames. The resulting tracker Sparse Inertial Poser (SIP) enables motion capture using only 6 sensors (attached to the wrists, lower legs, back and head) and works for arbitrary human motions. Experiments on the recently released TNT15 dataset show that, using the same number of sensors, SIP achieves higher accuracy than the dataset baseline without using any video data. We further demonstrate the effectiveness of SIP on newly recorded challenging motions in outdoor scenarios such as climbing or jumping over a wall

ps

video pdf Project Page [BibTex]

video pdf Project Page [BibTex]


Efficient 2D and 3D Facade Segmentation using Auto-Context
Efficient 2D and 3D Facade Segmentation using Auto-Context

Gadde, R., Jampani, V., Marlet, R., Gehler, P.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017 (article)

Abstract
This paper introduces a fast and efficient segmentation technique for 2D images and 3D point clouds of building facades. Facades of buildings are highly structured and consequently most methods that have been proposed for this problem aim to make use of this strong prior information. Contrary to most prior work, we are describing a system that is almost domain independent and consists of standard segmentation methods. We train a sequence of boosted decision trees using auto-context features. This is learned using stacked generalization. We find that this technique performs better, or comparable with all previous published methods and present empirical results on all available 2D and 3D facade benchmark datasets. The proposed method is simple to implement, easy to extend, and very efficient at test-time inference.

ps

arXiv Project Page [BibTex]

arXiv Project Page [BibTex]


{ClothCap}: Seamless {4D} Clothing Capture and Retargeting
ClothCap: Seamless 4D Clothing Capture and Retargeting

Pons-Moll, G., Pujades, S., Hu, S., Black, M.

ACM Transactions on Graphics, (Proc. SIGGRAPH), 36(4):73:1-73:15, ACM, New York, NY, USA, 2017, Two first authors contributed equally (article)

Abstract
Designing and simulating realistic clothing is challenging and, while several methods have addressed the capture of clothing from 3D scans, previous methods have been limited to single garments and simple motions, lack detail, or require specialized texture patterns. Here we address the problem of capturing regular clothing on fully dressed people in motion. People typically wear multiple pieces of clothing at a time. To estimate the shape of such clothing, track it over time, and render it believably, each garment must be segmented from the others and the body. Our ClothCap approach uses a new multi-part 3D model of clothed bodies, automatically segments each piece of clothing, estimates the naked body shape and pose under the clothing, and tracks the 3D deformations of the clothing over time. We estimate the garments and their motion from 4D scans; that is, high-resolution 3D scans of the subject in motion at 60 fps. The model allows us to capture a clothed person in motion, extract their clothing, and retarget the clothing to new body shapes. ClothCap provides a step towards virtual try-on with a technology for capturing, modeling, and analyzing clothing in motion.

ps

video project_page paper link (url) DOI Project Page Project Page [BibTex]

video project_page paper link (url) DOI Project Page Project Page [BibTex]

2007


no image
Learning static Gestalt laws through dynamic experience

Ostrovsky, Y., Wulff, J., Sinha, P.

Journal of Vision, 7(9):315-315, ARVO, June 2007 (article)

Abstract
The Gestalt laws (Wertheimer 1923) are widely regarded as the rules that help us parse the world into objects. However, it is unclear as to how these laws are acquired by an infant's visual system. Classically, these “laws” have been presumed to be innate (Kellman and Spelke 1983). But, more recent work in infant development, showing the protracted time-course over which these grouping principles emerge (e.g., Johnson and Aslin 1995; Craton 1996), suggests that visual experience might play a role in their genesis. Specifically, our studies of patients with late-onset vision (Project Prakash; VSS 2006) and evidence from infant development both point to an early role of common motion cues for object grouping. Here we explore the possibility that the privileged status of motion in the developmental timeline is not happenstance, but rather serves to bootstrap the learning of static Gestalt cues. Our approach involves computational analyses of real-world motion sequences to investigate whether primitive optic flow information is correlated with static figural cues that could eventually come to serve as proxies for grouping in the form of Gestalt principles. We calculated local optic flow maps and then examined how similarity of motion across image patches co-varied with similarity of certain figural properties in static frames. Results indicate that patches with similar motion are much more likely to have similar luminance, color, and orientation as compared to patches with dissimilar motion vectors. This regularity suggests that, in principle, common motion extracted from dynamic visual experience can provide enough information to bootstrap region grouping based on luminance and color and contour continuation mechanisms in static scenes. These observations, coupled with the cited experimental studies, lend credence to the hypothesis that static Gestalt laws might be learned through a bootstrapping process based on early dynamic experience.

ps

link (url) DOI [BibTex]

2007


link (url) DOI [BibTex]


Neuromotor prosthesis development
Neuromotor prosthesis development

Donoghue, J., Hochberg, L., Nurmikko, A., Black, M., Simeral, J., Friehs, G.

Medicine & Health Rhode Island, 90(1):12-15, January 2007 (article)

Abstract
Article describes a neuromotor prosthesis (NMP), in development at Brown University, that records human brain signals, decodes them, and transforms them into movement commands. An NMP is described as a system consisting of a neural interface, a decoding system, and a user interface, also called an effector; a closed-loop system would be completed by a feedback signal from the effector to the brain. The interface is based on neural spiking, a source of information-rich, rapid, complex control signals from the nervous system. The NMP described, named BrainGate, consists of a match-head sized platform with 100 thread-thin electrodes implanted just into the surface of the motor cortex where commands to move the hand emanate. Neural signals are decoded by a rack of computers that displays the resultant output as the motion of a cursor on a computer monitor. While computer cursor motion represents a form of virtual device control, this same command signal could be routed to a device to command motion of paralyzed muscles or the actions of prosthetic limbs. The researchers’ overall goal is the development of a fully implantable, wireless multi-neuron sensor for broad research, neural prosthetic, and human neurodiagnostic applications.

ps

pdf [BibTex]

pdf [BibTex]


Frequency-domain displacement sensing with a fiber ring-resonator containing a variable gap
Frequency-domain displacement sensing with a fiber ring-resonator containing a variable gap

Vollmer, F., Fischer, P.

SENSORS AND ACTUATORS A-PHYSICAL, 134(2):410-413, 2007 (article)

Abstract
Ring-resonators are in general not amenable to strain-free (non-contact) displacement measurements. We show that this limitation may be overcome if the ring-resonator, here a fiber-loop, is designed to contain a gap, such that the light traverses a free-space part between two aligned waveguide ends. Displacements are determined with nanometer sensitivity by measuring the associated changes in the resonance frequencies. Miniaturization should increase the sensitivity of the ring-resonator interferometer. Ring geometries that contain an optical circulator can be used to profile reflective samples. (c) 2006 Elsevier B.V. All rights reserved.

pf

DOI [BibTex]

DOI [BibTex]


Observation of the Faraday effect via beam deflection in a longitudinal magnetic field
Observation of the Faraday effect via beam deflection in a longitudinal magnetic field

Ghosh, A., Hill, W., Fischer, P.

PHYSICAL REVIEW A, 76(5), 2007 (article)

Abstract
We show that magnetic-field-induced circular differential deflection of light can be observed in reflection or refraction at a single interface. The difference in the reflection or refraction angles between the two circular polarization components is a function of the magnetic-field strength and the Verdet constant, and permits the observation of the Faraday effect not via polarization rotation in transmission, but via changes in the propagation direction. Deflection measurements do not suffer from n-pi ambiguities and are shown to be another means to map magnetic fields with high axial resolution, or to determine the sign and magnitude of magnetic-field pulses in a single measurement.

pf

DOI [BibTex]


Circular differential double diffraction in chiral media
Circular differential double diffraction in chiral media

Ghosh, A., Fazal, F. M., Fischer, P.

OPTICS LETTERS, 32(13):1836-1838, 2007 (article)

Abstract
In an optically active liquid the diffraction angle depends on the circular polarization state of the incident light beam. We report the observation of circular differential diffraction in an isotropic chiral medium, and we demonstrate that double diffraction is an alternate means to determine the handedness (enantiomeric excess) of a solution. (c) 2007 Optical Society of America.

pf

DOI [BibTex]

DOI [BibTex]