Header logo is de


2018


Model-based Optical Flow: Layers, Learning, and Geometry
Model-based Optical Flow: Layers, Learning, and Geometry

Wulff, J.

Tuebingen University, April 2018 (phdthesis)

Abstract
The estimation of motion in video sequences establishes temporal correspondences between pixels and surfaces and allows reasoning about a scene using multiple frames. Despite being a focus of research for over three decades, computing motion, or optical flow, remains challenging due to a number of difficulties, including the treatment of motion discontinuities and occluded regions, and the integration of information from more than two frames. One reason for these issues is that most optical flow algorithms only reason about the motion of pixels on the image plane, while not taking the image formation pipeline or the 3D structure of the world into account. One approach to address this uses layered models, which represent the occlusion structure of a scene and provide an approximation to the geometry. The goal of this dissertation is to show ways to inject additional knowledge about the scene into layered methods, making them more robust, faster, and more accurate. First, this thesis demonstrates the modeling power of layers using the example of motion blur in videos, which is caused by fast motion relative to the exposure time of the camera. Layers segment the scene into regions that move coherently while preserving their occlusion relationships. The motion of each layer therefore directly determines its motion blur. At the same time, the layered model captures complex blur overlap effects at motion discontinuities. Using layers, we can thus formulate a generative model for blurred video sequences, and use this model to simultaneously deblur a video and compute accurate optical flow for highly dynamic scenes containing motion blur. Next, we consider the representation of the motion within layers. Since, in a layered model, important motion discontinuities are captured by the segmentation into layers, the flow within each layer varies smoothly and can be approximated using a low dimensional subspace. We show how this subspace can be learned from training data using principal component analysis (PCA), and that flow estimation using this subspace is computationally efficient. The combination of the layered model and the low-dimensional subspace gives the best of both worlds, sharp motion discontinuities from the layers and computational efficiency from the subspace. Lastly, we show how layered methods can be dramatically improved using simple semantics. Instead of treating all layers equally, a semantic segmentation divides the scene into its static parts and moving objects. Static parts of the scene constitute a large majority of what is shown in typical video sequences; yet, in such regions optical flow is fully constrained by the depth structure of the scene and the camera motion. After segmenting out moving objects, we consider only static regions, and explicitly reason about the structure of the scene and the camera motion, yielding much better optical flow estimates. Furthermore, computing the structure of the scene allows to better combine information from multiple frames, resulting in high accuracies even in occluded regions. For moving regions, we compute the flow using a generic optical flow method, and combine it with the flow computed for the static regions to obtain a full optical flow field. By combining layered models of the scene with reasoning about the dynamic behavior of the real, three-dimensional world, the methods presented herein push the envelope of optical flow computation in terms of robustness, speed, and accuracy, giving state-of-the-art results on benchmarks and pointing to important future research directions for the estimation of motion in natural scenes.

ps

Official link DOI Project Page [BibTex]


no image
Pattern forming systems under confinement

Maihöfer, Michael

Universität Stuttgart, Stuttgart, 2018 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Effective interactions between colloidal particles in critical solvents

Labbe-Laurent, M.

Universität Stuttgart, Stuttgart, 2018 (phdthesis)

icm

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Non-equilibrium dynamics of a binary solvent around heated colloidal particles

Wilke, Moritz

Universität Stuttgart, Stuttgart, 2018 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Monte Carlo study of colloidal structure formation at fluid interfaces

Meiler, Tim

Universität Stuttgart, Stuttgart, 2018 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Electrolyte solutions and simple fluids at curved walls

Reindl, A.

Universität Stuttgart, Stuttgart, 2018 (phdthesis)

icm

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Surface structure of liquid crystals

Sattler, Alexander

Universität Stuttgart, Stuttgart, 2018 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Dynamics of an active particle in confined viscous flows

Pöhnl, Ruben

Universität Stuttgart, Stuttgart, 2018 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Electrostatic interaction between colloids with constant surface potentials at fluid interfaces

Bebon, Rick

Universität Stuttgart, Stuttgart, 2018 (mastersthesis)

icm

[BibTex]

[BibTex]

2014


Advanced Structured Prediction
Advanced Structured Prediction

Nowozin, S., Gehler, P. V., Jancsary, J., Lampert, C. H.

Advanced Structured Prediction, pages: 432, Neural Information Processing Series, MIT Press, November 2014 (book)

Abstract
The goal of structured prediction is to build machine learning models that predict relational information that itself has structure, such as being composed of multiple interrelated parts. These models, which reflect prior knowledge, task-specific relations, and constraints, are used in fields including computer vision, speech recognition, natural language processing, and computational biology. They can carry out such tasks as predicting a natural language sentence, or segmenting an image into meaningful components. These models are expressive and powerful, but exact computation is often intractable. A broad research effort in recent years has aimed at designing structured prediction models and approximate inference and learning procedures that are computationally efficient. This volume offers an overview of this recent research in order to make the work accessible to a broader research community. The chapters, by leading researchers in the field, cover a range of topics, including research trends, the linear programming relaxation approach, innovations in probabilistic modeling, recent theoretical progress, and resource-aware learning.

ps

publisher link (url) [BibTex]

2014


publisher link (url) [BibTex]


Modeling the Human Body in 3D: Data Registration and Human Shape Representation
Modeling the Human Body in 3D: Data Registration and Human Shape Representation

Tsoli, A.

Brown University, Department of Computer Science, May 2014 (phdthesis)

ps

pdf [BibTex]

pdf [BibTex]


no image
Wetting phenomena in electrolyte solutions

Ibagon, I.

Universität Stuttgart, Stuttgart, 2014 (phdthesis)

icm

link (url) [BibTex]

link (url) [BibTex]


no image
Trajectory to trajectory fluctuations in first-passage phenomena in bounded domains

Mattos, Thiago G., Mejia-Monasterio, Carlos, Metzler, Ralf, Oshanin, Gleb, Schehr, G.

In First-passage phenomena and their applications, pages: 203-225, World Scientific Publishing, Singapore, 2014 (incollection)

icm

DOI [BibTex]

DOI [BibTex]


no image
Dynamik einer fast-kritischen binären Flüssigkeit in zwei Dimensionen

Maisch, J.

Universität Stuttgart, Stuttgart, 2014 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Brownsche Bewegung schraubenförmiger Nanoteilchen

Gehrmann, C.

Universität Stuttgart, Stuttgart, 2014 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Electromagnetic self-propulsion for small objects in the near field

Müller, Boris

Universität Stuttgart, Stuttgart, 2014 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
Thermal and structural properties of dense ionic liquids

Bartsch, Hendrik

Universität Stuttgart, Stuttgart, 2014 (mastersthesis)

icm

[BibTex]

[BibTex]


Simulated Annealing
Simulated Annealing

Gall, J.

In Encyclopedia of Computer Vision, pages: 737-741, 0, (Editors: Ikeuchi, K. ), Springer Verlag, 2014, to appear (inbook)

ps

[BibTex]

[BibTex]

2005


no image
Interplay between geometry and fluid properties

König, P.-M.

Universität Stuttgart, Stuttgart, 2005 (phdthesis)

icm

link (url) [BibTex]

2005


link (url) [BibTex]


no image
Molecular dynamics of wet granular media

Goll, C.

Universität Stuttgart, Stuttgart, 2005 (mastersthesis)

icm

[BibTex]

[BibTex]


no image
The Boolean Model: from Matheron till today

Stoyan, D., Mecke, K.

In Space, Structure and Randomness: contributions in honor of Georges Matheron in the fields of geostatistics, random sets, and mathematical morphology, 183, pages: 151-182, Lecture Notes in Statistics, Springer, New York, 2005 (incollection)

icm

[BibTex]

[BibTex]


no image
Grenzflächenfluktuationen binärer Flüssigkeiten

Hiester, T.

Universität Stuttgart, Stuttgart, 2005 (phdthesis)

icm

link (url) [BibTex]

link (url) [BibTex]

2004


no image
Morphometry of convection patterns in the earth\textquotesingles mantle

Kaminke, Ralf

Universität Stuttgart, Stuttgart, 2004 (mastersthesis)

icm

[BibTex]

2004


[BibTex]