Visual Computing

@IST Austria

Publications

Water Wave Packets

Water Wave Packets

ACM Transactions on Graphics 36(4) (SIGGRAPH 2017)

  • Jeschke
  • Wojtan

This paper presents a method for simulating water surface waves as a displacement field on a 2D domain. Our method relies on Lagrangian particles that carry packets of water wave energy; each packet carries information about an entire group of wave trains, as opposed to only a single wave crest. Our approach is unconditionally stable and can simulate high resolution geometric details. This approach also presents a straightforward interface for artistic control, because it is essentially a particle system with intuitive parameters like wavelength and amplitude. Our implementation parallelizes well and runs in real time for moderately challenging scenarios.

@article{Jeschke2017,
author = {Stefan Jeschke and Chris Wojtan},
title = {Water Wave Packets},
journal = {ACM Transactions on Graphics (SIGGRAPH 2017)},
year = {2017},
volume = {36},
number = {4}
}
Functionality-aware Retargeting of Mechanisms to 3D Shapes

Functionality-aware Retargeting of Mechanisms to 3D Shapes

ACM Transactions on Graphics 36(4) (SIGGRAPH 2017)

  • Zhang
  • Auzinger
  • Ceylan
  • Li
  • Bickel

We present an interactive design system to create functional mechanical objects. Our computational approach allows novice users to retarget an existing mechanical template to a user-specified input shape. Our proposed representation for a mechanical template encodes a parameterized mechanism, mechanical constraints that ensure a physically valid configuration, spatial relationships of mechanical parts to the user-provided shape, and functional constraints that specify an intended functionality. We provide an intuitive interface and optimization-in-the-loop approach for finding a valid configuration of the mechanism and the shape to ensure that higher-level functional goals are met. Our algorithm interactively optimizes the mechanism while the user manipulates the placement of mechanical components and the shape. Our system allows users to efficiently explore various design choices and to synthesize customized mechanical objects that can be fabricated with rapid prototyping technologies. We demonstrate the efficacy of our approach by retargeting various mechanical templates to different shapes and fabricating the resulting functional mechanical objects.

@article{Zhang2017,
author = {Zhang, Ran and Auzinger, Thomas and Ceylan, Duygu and Li, Wilmot and Bickel, Bernd},
title = {Functionality-aware Retargeting of Mechanisms to 3D Shapes},
journal = {ACM Transactions on Graphics (SIGGRAPH 2017)},
year = {2017},
volume = {36},
number = {4}
}
CurveUps: Shaping Objects from Flat Plates with Tension-Actuated Curvature

CurveUps: Shaping Objects from Flat Plates with Tension-Actuated Curvature

ACM Transactions on Graphics 36(4) (SIGGRAPH 2017)

  • Guseinov
  • Miguel
  • Bickel

We present a computational approach for designing CurveUps, curvy shells that form from an initially flat state. They consist of small rigid tiles that are tightly held together by two pre-stretched elastic sheets attached to them. Our method allows the realization of smooth, doubly curved surfaces that can be fabricated as a flat piece. Once released, the restoring forces of the pre-stretched sheets support the object to take shape in 3D. CurveUps are structurally stable in their target configuration. The design process starts with a target surface. Our method generates a tile layout in 2D and optimizes the distribution, shape, and attachment areas of the tiles to obtain a configuration that is fabricable and in which the curved up state closely matches the target. Our approach is based on an efficient approximate model and a local optimization strategy for an otherwise intractable nonlinear optimization problem. We demonstrate the effectiveness of our approach for a wide range of shapes, all realized as physical prototypes.

@article{Guseinov2017,
author = {Guseinov, Ruslan and Miguel, Eder and Bickel, Bernd},
title = {CurveUps: Shaping Objects from Flat Plates with Tension-Actuated Curvature},
journal = {ACM Transactions on Graphics (SIGGRAPH 2017)},
year = {2017},
volume = {36},
number = {4}
}
Computational Multicopter Design

Computational Multicopter Design

ACM Transactions on Graphics (SIGGRAPH Asia 2016)

  • Du
  • Schulz
  • Zhu
  • Bickel
  • Matusik

We present an interactive system for computational design, optimization, and fabrication of multicopters. Our computational approach allows non-experts to design, explore, and evaluate a wide range of different multicopters. We provide users with an intuitive interface for assembling a multicopter from a collection of components (e.g., propellers, motors, and carbon fiber rods). Our algorithm interactively optimizes shape and controller parameters of the current design to ensure its proper operation. In addition, we allow incorporating a variety of other metrics (such as payload, battery usage, size, and cost) into the design process and exploring tradeoffs between them. We show the efficacy of our method and system by designing, optimizing, fabricating, and operating multicopters with complex geometries and propeller configurations. We also demonstrate the ability of our optimization algorithm to improve the multicopter performance under different metrics.

FlexMolds: Automatic Design of Flexible Shells for Molding

FlexMolds: Automatic Design of Flexible Shells for Molding

ACM Transactions on Graphics (SIGGRAPH Asia 2016)

  • Malomo
  • Pietroni
  • Bickel
  • Cignoni

We present FlexMolds, a novel computational approach to automatically design flexible, reusable molds that, once 3D printed, allow us to physically fabricate, by means of liquid casting, multiple copies of complex shapes with rich surface details and complex topology. The approach to design such flexible molds is based on a greedy bottom-up search of possible cuts over an object, evaluating for each possible cut the feasibility of the resulting mold. We use a dynamic simulation approach to evaluate candidate molds, providing a heuristic to generate forces that are able to open, detach, and remove a complex mold from the object it surrounds. We have tested the approach with a number of objects with nontrivial shapes and topologies.

Seed, Expand and Constrain: Three Principles for Weakly-supervised Image Segmentation

Seed, Expand and Constrain: Three Principles for Weakly-supervised Image Segmentation

European Conference on Computer Vision (ECCV 2016)

  • Kolesnikov
  • Lampert

We introduce a new loss function for the weakly-supervised training of semantic image segmentation models based on three guiding principles: to seed with weak location cues, to expand objects based on the information about which classes can occur, and to constrain the segmentations to coincide with image boundaries. We show experimentally that training a deep convolutional neural network using the proposed loss function leads to substantially better segmentations than previous state-of-the-art methods on the challenging PASCAL VOC 2012 dataset. We furthermore give insight into the working mechanism of our method by a detailed experimental study that illustrates how the segmentation quality is affected by each term of the proposed loss function as well as their combinations.

@article{kolesnikov2014seed,
  title={Seed, Expand and Constrain: Three Principles for Weakly-supervised Image Segmentation},
  author={Kolesnikov, Alexander and Lampert, Christoph H},
  journal={European Conference on Computer Vision (ECCV)},
  year={2016}
}
Tracking, Correcting and Absorbing Water Surface Waves

Tracking, Correcting and Absorbing Water Surface Waves

IST Austria (PhD thesis)

  • Bojsen-Hansen

Computer graphics is an extremely exciting field for two reasons. On the one hand, there is a healthy injection of pragmatism coming from the visual effects industry that want robust algorithms that work so they can produce results at an increasingly frantic pace. On the other hand, they must always try to push the envelope and achieve the impossible to wow their audiences in the next blockbuster, which means that the industry has not succumb to conservatism, and there is *plenty* of room to try out new and *crazy* ideas if there is a chance that it will pan into something useful. Water simulation has been in visual effects for decades, however it still remains extremely challenging because of its high computational cost and difficult art-directability. The work in this thesis tries to address some of these difficulties. Specifically, we make the following three novel contributions to the state-of-the-art in water simulation for visual effects.

@phdthesis{TCaAWSW2016,
author = {Morten Bojsen-Hansen},
title = {Tracking, Correcting and Absorbing Water Surface Waves},
school = {IST Austria},
year = {2016},
month = {9}
}
Surface-Only Liquids

Surface-Only Liquids

ACM Transactions on Graphics 35(4) (SIGGRAPH 2016)

  • Da
  • Hahn
  • Batty
  • Wojtan
  • Grinspun
Generalized Non-Reflecting Boundaries for Fluid Re-Simulation

Generalized Non-Reflecting Boundaries for Fluid Re-Simulation

ACM Transactions on Graphics 35(4) (SIGGRAPH 2016)

  • Bojsen-Hansen
  • Wojtan

When aiming to seamlessly integrate a fluid simulation into a larger scenario (like an open ocean), careful attention must be paid to boundary conditions. In particular, one must implement special "non-reflecting" boundary conditions, which dissipate out-going waves as they exit the simulation. Unfortunately, the state of the art in non-reflecting boundary conditions (perfectly-matched layers, or PMLs) only permits trivially simple inflow/outflow conditions, so there is no reliable way to integrate a fluid simulation into a more complicated environment like a stormy ocean or a turbulent river. This paper introduces the first method for combining non-reflecting boundary conditions based on PMLs with inflow/outflow boundary conditions that vary arbitrarily throughout space and time. Our algorithm is a generalization of state-of-the-art mean-flow boundary conditions in the computational fluid dynamics literature, and it allows for seamless integration of a fluid simulation into much more complicated environments. Our method also opens the door for previously-unseen post-process effects like retroactively changing the location of solid obstacles, and locally increasing the visual detail of a pre-existing simulation.

@article{GNRBfFRS2016,
author = {Morten Bojsen-Hansen and Chris Wojtan},
title = {Generalized Non-Reflecting Boundaries for Fluid Re-Simulation},
journal = {ACM Transactions on Graphics (SIGGRAPH 2016)},
year = {2016},
volume = {35},
number = {4},
}
Fast approximations for boundary element based brittle fracture simulation

Fast approximations for boundary element based brittle fracture simulation

ACM Transactions on Graphics 35(4) (SIGGRAPH 2016)

  • Hahn
  • Wojtan
Computational Design of Stable Planar-Rod Structures

Computational Design of Stable Planar-Rod Structures

ACM Transactions on Graphics 35(4) (SIGGRAPH 2016)

  • Miguel
  • Lepoutre
  • Bickel

We present a computational method for designing wire sculptures consisting of interlocking wires. Our method allows the computation of aesthetically pleasing structures that are structurally stable, efficiently fabricatable with a 2D wire bending machine, and assemblable without the need of additional connectors. Starting from a set of planar contours provided by the user, our method automatically tests for the feasibility of a design, determines a discrete ordering of wires at intersection points, and optimizes for the rest shape of the individual wires to maximize structural stability under frictional contact. In addition to their application to art, wire sculptures present an extremely efficient and fast alternative for low-fidelity rapid prototyping because manufacturing time and required material linearly scales with the physical size of objects. We demonstrate the effectiveness of our approach on a varied set of examples, all of which we fabricated.

@article{CDoSPRS2016,
author = {Eder Miguel and Mathias Lepoutre and Bernd Bickel},
title = {Computational Design of Stable Planar-Rod Structures},
journal = {ACM Transactions on Graphics (SIGGRAPH 2016)},
year = {2016},
volume = {35},
number = {4}
}
Modeling and Estimation of Energy-Based Hyperelastic Objects

Modeling and Estimation of Energy-Based Hyperelastic Objects

Computer Graphics Forum 35(2) (EUROGRAPHICS 2016)

  • Miguel
  • Miraut
  • Otaduy
Total Variation on a Tree

Total Variation on a Tree

SIAM Journal on Imaging Sciences (SIIMS), 9(2):605-636, 2016

  • Kolmogorov
  • Pock
  • Rolinek

We consider the problem of minimizing the continuous valued total variation subject to different unary terms on trees and propose fast direct algorithms based on dynamic programming to solve these problems. We treat both the convex and the non-convex case and derive worst case complexities that are equal or better then existing methods. We show applications to total variation based 2D image processing and computer vision problems based on a Lagrangian decomposition approach. The resulting algorithms are very efficient, offer a high degree of parallelism and come along with memory requirements which are only in the order of the number of image pixels.

@article{kolmogorov2016total,
title={Total variation on a tree},
author={Kolmogorov, Vladimir and Pock, Thomas and Rolinek, Michal},
journal={SIAM Journal on Imaging Sciences},
volume={9},
number={2},
pages={605--636},
year={2016},
publisher={SIAM}
}
Narrow Band FLIP for Liquid Simulations

Narrow Band FLIP for Liquid Simulations

Computer Graphics Forum 35(2)

  • Ferstl
  • Ando
  • Wojtan
  • Westermann
  • Thuerey
A Practical Method for High-Resolution Embedded Liquid Surfaces

A Practical Method for High-Resolution Embedded Liquid Surfaces

Computer Graphics Forum 35(2)

  • Goldade
  • Batty
  • Wojtan
Generalized Diffusion Curves: An Improved Vector Representation for Smooth-Shaded Images

Generalized Diffusion Curves: An Improved Vector Representation for Smooth-Shaded Images

Computer Graphics Forum 35(2)

  • Jeschke

This paper generalizes the well-known Diffusion Curves Images (DCI), which are composed of a set of Bezier curves with colors specified on either side. These colors are diffused as Laplace functions over the image domain, which results in smooth color gradients interrupted by the Bezier curves. Our new formulation allows for more color control away from the boundary, providing a similar expressive power as recent Bilaplace image models without introducing associated issues and computational costs. The new model is based on a special Laplace function blending and a new edge blur formulation. We demonstrate that given some user-defined boundary curves over an input raster image, fitting colors and edge blur from the image to the new model and subsequent editing and animation is equally convenient as with DCIs. Numerous examples and comparisons to DCIs are presented.

@article{GDCI2016,
author = {Stefan Jeschke},
title = {Generalized Diffusion Curves: An Improved Vector Representation for Smooth-Shaded Images},
journal = {Computer Graphics Forum},
year = {2016},
volume = {35},
number = {2},
pages = {1--9}
}
DefSense: Computational Design of Customized Deformable Input Devices

DefSense: Computational Design of Customized Deformable Input Devices

ACM SIGCHI, May 2016

  • Bächer
  • Hepp
  • Pece
  • Kry
  • Bickel
  • Thomaszewski
  • Hilliges

We present a novel optimization-based algorithm for the design and fabrication of customized, deformable input devices, capable of continuously sensing their deformation. We propose to embed piezoresistive sensing elements into flexible 3D printed objects. These sensing elements are then utilized to recover rich and natural user interactions at runtime. Designing such objects manually is a challenging and hard problem for all but the simplest geometries and deformations. Our method simultaneously optimizes the internal routing of the sensing elements and computes a mapping from low-level sensor readings to user-specified outputs in order to minimize reconstruction error. We demonstrate the power and flexibility of the approach by designing and fabricating a set of flexible input devices. Our results indicate that the optimization-based design greatly outperforms manual routings in terms of reconstruction accuracy and thus interaction fidelity.

Computational Design of Walking Automata

Computational Design of Walking Automata

ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2015)

  • Bharaj
  • Coros
  • Thomaszewski
  • Tompkin
  • Bickel
  • Pfister

Creating mechanical automata that can walk in stable and pleasing manners is a challenging task that requires both skill and expertise. We propose to use computational design to offset the technical difficulties of this process. A simple drag-and-drop interface allows casual users to create personalized walking toys from a library of pre-defined template mechanisms. Provided with this input, our method leverages physical simulation and evolutionary optimization to refine the mechanical designs such that the resulting toys are able to walk. The optimization process is guided by an intuitive set of objectives that measure the quality of the walking motions. We demonstrate our approach on a set of simulated mechanical toys with different numbers of legs and various distinct gaits. Two fabricated prototypes showcase the feasibility of our designs.

A Stream Function Solver for Liquid Simulations

A Stream Function Solver for Liquid Simulations

ACM Trans. Graph. 34, 4 (SIGGRAPH 2015 Papers)

  • Ando
  • Thuerey
  • Wojtan
Detailed Spatio-Temporal Reconstruction of Eyelids

Detailed Spatio-Temporal Reconstruction of Eyelids

ACM Trans. Graph. 34, 4 (SIGGRAPH 2015 Papers)

  • Bermano
  • Beeler
  • Yeara
  • Bradley
  • Bickel
  • Gross

In recent years we have seen numerous improvements on 3D scanning and tracking of human faces, greatly advancing the creation of digital doubles for film and video games. However, despite the high-resolution quality of the reconstruction approaches available, current methods are unable to capture one of the most important regions of the face – the eye region. In this work we present the first method for detailed spatio-temporal reconstruction of eyelids. Tracking and reconstructing eyelids is extremely challenging, as this region exhibits very complex and unique skin deformation where skin is folded under while opening the eye. Furthermore, eyelids are often only partially visible and obstructed due to selfocclusion and eyelashes. Our approach is to combine a geometric deformation model with image data, leveraging multi-view stereo, optical flow, contour tracking and wrinkle detection from local skin appearance. Our deformation model serves as a prior that enables reconstruction of eyelids even under strong self-occlusions caused by rolling and folding skin as the eye opens and closes. The output is a person-specific, time-varying eyelid reconstruction with anatomically plausible deformations. Our high-resolution detailed eyelids couple naturally with current facial performance capture approaches. As a result, our method can largely increase the fidelity of facial capture and the creation of digital doubles.

OmniAD Data-driven Omni-directional Aerodynamics

OmniAD Data-driven Omni-directional Aerodynamics

ACM Trans. Graph. 34, 4 (SIGGRAPH 2015 Papers)

  • Martin
  • Umetani
  • Bickel

This paper introduces OmniAD, a novel data-driven pipeline to model and acquire the aerodynamics of three-dimensional rigid objects. Traditionally, aerodynamics are examined through elaborate wind tunnel experiments or expensive fluid dynamics computations, and are only measured for a small number of discrete wind directions. OmniAD allows the evaluation of aerodynamic forces, such as drag and lift, for any incoming wind direction using a novel representation based on spherical harmonics. Our data-driven technique acquires the aerodynamic properties of an object simply by capturing its falling motion using a single camera. Once model parameters are estimated, OmniAD enables realistic real-time simulation of rigid bodies, such as the tumbling and gliding of leaves, without simulating the surrounding air. In addition, we propose an intuitive user interface based on OmniAD to interactively design three-dimensional kites that actually fly. Various non-traditional kites were designed to demonstrate the physical validity of our model.

Microstructures to Control Elasticity in 3D Printing

Microstructures to Control Elasticity in 3D Printing

ACM Trans. Graph. 34, 4 (SIGGRAPH 2015 Papers)

  • Schumacher
  • Bickel
  • Marschner
  • Rys
  • Daraio
  • Gross

We propose a method for fabricating deformable objects with spatially varying elasticity using 3D printing. Using a single, relatively stiff printer material, our method designs an assembly of small-scale microstructures that have the effect of a softer material at the object scale, with properties depending on the microstructure used in each part of the object. We build on work in the area of metamaterials, using numerical optimization to design tiled microstructures with desired properties, but with the key difference that our method designs families of related structures that can be interpolated to smoothly vary the material properties over a wide range. To create an object with spatially varying elastic properties, we tile the object's interior with microstructures drawn from these families, generating a different microstructure for each cell using an efficient algorithm to select compatible structures for neighboring cells. We show results computed for both 2D and 3D objects, validating several 2D and 3D printed structures using standard material tests as well as demonstrating various example applications.

Learning Shape Placements by Example

Learning Shape Placements by Example

ACM Trans. Graph. 34, 4 (SIGGRAPH 2015 Papers)

  • Guerrero
  • Jeschke
  • Wimmer
  • Wonka

We present a method to learn and propagate shape placements in 2D polygonal scenes from a few examples provided by a user. The placement of a shape is modeled as an oriented bounding box. Simple geometric relationships between this bounding box and nearby scene polygons define a feature set for the placement. The feature sets of all example placements are then used to learn a probabilistic model over all possible placements and scenes. With this model we can generate a new set of placements with similar geometric relationships in any given scene. We introduce extensions that enable propagation and generation of shapes in 3D scenes, as well as the application of a learned modeling session to large scenes without additional user interaction. These concepts allow us to generate complex scenes with thousands of objects with relatively little user interaction.

@article{guerrero-2015-lsp,
title =      "Learning Shape Placements by Example",
author =     "Paul Guerrero and Stefan Jeschke and Michael Wimmer and Peter Wonka",
year =       "2015",
pages =      "1--13",
month =      aug,
event =      "ACM SIGGRAPH 2015",
journal =    "ACM Transactions on Graphics",
location =   "Los Angeles, CA",
keywords =   "complex model generation, modeling by example",
}
High-Resolution Brittle Fracture Simulation with Boundary Elements

High-Resolution Brittle Fracture Simulation with Boundary Elements

ACM Trans. Graph. 34, 4 (SIGGRAPH 2015 Papers)

  • Hahn
  • Wojtan
Design and Fabrication of Flexible Rod Meshes

Design and Fabrication of Flexible Rod Meshes

ACM Trans. Graph. 34, 4 (SIGGRAPH 2015 Papers)

  • Perez
  • Thomaszewski
  • Coros
  • Bickel
  • Canabal
  • Sumner
  • Otaduy

We present a computational tool for fabrication-oriented design of flexible rod meshes. Given a deformable surface and a set of deformed poses as input, our method automatically computes a printable rod mesh that, once manufactured, closely matches the input poses under the same boundary conditions. The core of our method is formed by an optimization scheme that adjusts the cross-sectional profiles of the rods and their rest centerline in order to best approximate the target deformations. This approach allows us to locally control the bending and stretching resistance of the surface with a single material, yielding high design flexibility and low fabrication cost.

Double Bubbles Sans Toil and Trouble: Discrete Circulation-Preserving Vortex Sheets for Soap Films and Foams

Double Bubbles Sans Toil and Trouble: Discrete Circulation-Preserving Vortex Sheets for Soap Films and Foams

ACM Trans. Graph. 34, 4 (SIGGRAPH 2015 Papers)

  • Da
  • Batty
  • Wojtan
  • Grinspun
Predicting the Future Behavior of a Time-Varying Probability Distribution

Predicting the Future Behavior of a Time-Varying Probability Distribution

IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015)

  • Lampert

We study the problem of predicting the future, though only in the probabilistic sense of estimating a future state of a time-varying probability distribution. This is not only an interesting academic problem, but solving this extrapolation problem also has many practical application, e.g. for training classifiers that have to operate under time-varying conditions. Our main contribution is a method for predicting the next step of the time-varying distribution from a given sequence of sample sets from earlier time steps. For this we rely on two recent machine learning techniques: embedding probability distributions into a reproducing kernel Hilbert space, and learning operators by vector-valued regression. We illustrate the working principles and the practical usefulness of our method by experiments on synthetic and real data. We also highlight an exemplary application: training a classifier in a domain adaptation setting without having access to examples from the test time distribution at training time.

Curriculum Learning of Multiple Tasks

Curriculum Learning of Multiple Tasks

IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015)

  • Pentina
  • Sharmanska
  • Lampert

Sharing information between multiple tasks enables algorithms to achieve good generalization performance even from small amounts of training data. However, in a realistic scenario of multi-task learning not all tasks are equally related to each other, hence it could be advantageous to transfer information only between the most related tasks. In this work we propose an approach that processes multiple tasks in a sequence with sharing between subsequent tasks instead of solving all tasks jointly. Subsequently, we address the question of curriculum learning of tasks, i.e. finding the best order of tasks to be learned. Our approach is based on a generalization bound criterion for choosing the task order that optimizes the average expected classi- fication performance over all tasks. Our experimental results show that learning multiple related tasks sequentially can be more effective than learning them jointly, the order in which tasks are being solved affects the overall performance, and that our model is able to automatically discover a favourable order of tasks.

Classifier Adaptation at Prediction Time

Classifier Adaptation at Prediction Time

IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015)

  • Royer
  • Lampert

Classifiers for object categorization are usually evaluated by their accuracy on a set of i.i.d. test examples. This provides us with an estimate of the expected error when applying the classifiers to a single new image. In real application, however, classifiers are rarely only used for a single image and then discarded. Instead, they are applied sequentially to many images, and these are typically not i.i.d. samples from a fixed data distribution, but they carry dependencies and their class distribution varies over time. In this work, we argue that the phenomenon of correlated data at prediction time is not a nuisance, but a blessing in disguise. We describe a probabilistic method for adapting classifiers at prediction time without having to retraining them. We also introduce a framework for creating realistically distributed image sequences, which offers a way to benchmark classifier adaptation methods, such as the one we propose. Experiments on the ILSVRC2010 and ILSVRC2012 datasets show that adapting object classification systems at prediction time can significantly reduce their error rate, even with additional human feedback.

A Multi-Plane Block-Coordinate Frank-Wolfe Algorithm for Training Structural SVMs with a Costly max-Oracle

A Multi-Plane Block-Coordinate Frank-Wolfe Algorithm for Training Structural SVMs with a Costly max-Oracle

IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015)

  • Shah
  • Kolmogorov
  • Lampert

Structural support vector machines (SSVMs) are amongst the best performing methods for structured computer vision tasks, such as semantic image segmentation or human pose estimation. Training SSVMs, however, is computationally costly, because it requires repeated calls to a structured prediction subroutine (called max-oracle), which has to solve an optimization problem itself, e.g. a graph cut. In this work, we introduce a new algorithm for SSVM training that is more efficient than earlier techniques when the max-oracle is computationally expensive, as it is frequently the case in computer vision tasks. The main idea is to (i) combine the recent stochastic Block-Coordinate Frank-Wolfe algorithm with efficient hyperplane caching, and (ii) use an automatic selection rule for deciding whether to call the exact max-oracle or to rely on an approximate one based on the cached hyperplanes. We show experimentally that this strategy leads to faster convergence towards the optimum with respect to the number of required oracle calls, and that this also translates into faster convergence with respect to the total runtime when the max-oracle is slow compared to the other steps of the algorithm. A C++ implementation is provided at http://www.ist.ac.at/˜vnk.

Water Wave Animation via Wavefront Parameter Interpolation

Water Wave Animation via Wavefront Parameter Interpolation

ACM Transactions on Graphics 34(3)

  • Jeschke
  • Wojtan

We present an efficient wavefront tracking algorithm for animating bodies of water that interact with their environment. Our contributions include: a novel wavefront tracking technique that enables dispersion, refraction, reflection, and diffraction in the same simulation; a unique multivalued function interpolation method that enables our simulations to elegantly sidestep the Nyquist limit; a dispersion approximation for efficiently amplifying the number of simulated waves by several orders of magnitude; and additional extensions that allow for time-dependent effects and interactive artistic editing of the resulting animation. Our contributions combine to give us multitudes more wave details than similar algorithms, while maintaining high frame rates and allowing close camera zooms.

@article{WWAvWPI2015,
author = {Stefan Jeschke and Chris Wojtan},
title = {Water Wave Animation via Wavefront Parameter Interpolation},
journal = {ACM Transactions on Graphics},
year = {2015},
volume = {34},
number = {3},
pages = {1--14}
}
Recent Advances in Facial Appearance Capture

Recent Advances in Facial Appearance Capture

Computer Graphics Forum 34(2) (Eurographics 2015)

  • Klehm
  • Rousselle
  • Papas
  • Bradley
  • Hery
  • Bickel
  • Jarosz
  • Beeler

Facial appearance capture is now firmly established within academic research and used extensively across various application domains, perhaps most prominently in the entertainment industry through the design of virtual characters in video games and films. While significant progress has occurred over the last two decades, no single survey currently exists that discusses the similarities, differences, and practical considerations of the available appearance capture techniques as applied to human faces. A central difficulty of facial appearance capture is the way light interacts with skin—which has a complex multi-layered structure—and the interactions that occur below the skin surface can, by definition, only be observed indirectly. In this report, we distinguish between two broad strategies for dealing with this complexity. “Image-based methods” try to exhaustively capture the exact face appearance under different lighting and viewing conditions, and then render the face through weighted image combinations. “Parametric methods” instead fit the captured reflectance data to some parametric appearance model used during rendering, allowing for a more lightweight and flexible representation but at the cost of potentially increased rendering complexity or inexact reproduction. The goal of this report is to provide an overview that can guide practitioners and researchers in assessing the tradeoffs between current approaches and identifying directions for future advances in facial appearance capture.

A Dimension-reduced Pressure Solver for Liquid Simulations

A Dimension-reduced Pressure Solver for Liquid Simulations

ACM Transactions on Graphics 34(2) (SIGGRAPH 2015)

  • Ando
  • Thürey
  • Wojtan
Partial Shape Matching using Transformation Parameter Similarity

Partial Shape Matching using Transformation Parameter Similarity

Computer Graphics Forum, 33(8)

  • Paul
  • Auzinger
  • Wimmer
  • Jeschke

In this paper, we present a method for non-rigid, partial shape matching in vector graphics. Given a user-specified query region in a 2D shape, similar regions are found, even if they are non-linearly distorted. Furthermore, a non-linear mapping is established between the query regions and these matches, which allows the automatic transfer of editing operations such as texturing. This is achieved by a two-step approach. First, point-wise correspondences between the query region and the whole shape are established. The transformation parameters of these correspondences are registered in an appropriate transformation space. For transformations between similar regions, these parameters form surfaces in transformation space, which are extracted in the second step of our method. The extracted regions may be related to the query region by a non-rigid transform, enabling non-rigid shape matching.

@article{Guerrero-2014-TPS,
author = {Paul Guerrero and Thomas Auzinger and Michael Wimmer and Stefan Jeschke},
title = {Partial Shape Matching using Transformation Parameter Similarity},
journal = {Computer Graphics Forum},
year = {2014},
volume = {33},
number = {8},
pages = {1--14}
issn =  {1467-8659}
}
Closed-Form Approximate CRF Training for Scalable Image Segmentation

Closed-Form Approximate CRF Training for Scalable Image Segmentation

European Conference on Computer Vision (ECCV 2014)

  • Kolesnikov
  • Gauillaumin
  • Ferrari
  • Lampert

We present LS-CRF, a new method for training cyclic Conditional Random Fields (CRFs) from large datasets that is inspired by classical closed-form expressions for the maximum likelihood parameters of a generative graphical model with tree topology. Training a CRF with LS-CRF requires only solving a set of independent regression problems, each of which can be solved efficiently in closed form or by an iterative solver. This makes LS-CRF orders of magnitude faster than classical CRF training based on probabilistic inference, and at the same time more flexible and easier to implement than other approximate techniques, such as pseudolikelihood or piecewise training. We apply LS-CRF to the task of semantic image segmentation, showing that it achieves on par accuracy to other training techniques at higher speed, thereby allowing efficient CRF training from very large training sets. For example, training a linearly parameterized pairwise CRF on 150,000 images requires less than one hour on a modern workstation.

@article{kolesnikov2014closed,
  title={Closed-Form Approximate CRF Training for Scalable Image Segmentation},
  author={Kolesnikov, Alexander and Guillaumin, Matthieu and Ferrari, Vittorio and Lampert, Christoph H},
  journal={European Conference on Computer Vision (ECCV)},
  year={2014}
}
Spin-It: Optimizing Moment of Inertia for Spinnable Objects

Spin-It: Optimizing Moment of Inertia for Spinnable Objects

ACM Trans. Graph. 33, 4 (SIGGRAPH 2014 Papers)

  • Baecher
  • Whiting
  • Bickel
  • Sorkine-Hornung

Spinning tops and yo-yos have long fascinated cultures around the world with their unexpected, graceful motions that seemingly elude gravity. We present an algorithm to generate designs for spinning objects by optimizing rotational dynamics properties. As input, the user provides a solid 3D model and a desired axis of rotation. Our approach then modifies the mass distribution such that the principal directions of the moment of inertia align with the target rotation frame. We augment the model by creating voids inside its volume, with interior fill represented by an adaptive multi-resolution voxelization. The discrete voxel fill values are optimized using a continuous, nonlinear formulation. Further, we optimize for rotational stability by maximizing the dominant principal moment. We extend our technique to incorporate deformation and multiple materials for cases where internal voids alone are insufficient. Our method is well-suited for a variety of 3D printed models, ranging from characters to abstract shapes. We demonstrate tops and yo-yos that spin surprisingly stably despite their asymmetric appearance.

Deep Fisher Kernels - End to End Learning of the Fisher Kernel GMM Parameters

Deep Fisher Kernels - End to End Learning of the Fisher Kernel GMM Parameters

IEEE Computer Vision and Pattern Recognition (CVPR)

  • Sydorov
  • Sakurada
  • Lampert

Fisher Kernels and Deep Learning were two developments with significant impact on large-scale object categorization in the last years. Both approaches were shown to achieve state-of-the-art results on large-scale object categorization datasets, such as ImageNet. Conceptually, however, they are perceived as very different and it is not uncommon for heated debates to spring up when advocates of both paradigms meet at conferences or workshops. In this work, we emphasize the similarities between both architectures rather than their differences and we argue that such a unified view allows us to transfer ideas from one domain to the other. As a concrete example we introduce a method for learning a support vector machine classifier with Fisher kernel at the same time as a task-specific data representation. We reinterpret the setting as a multi-layer feed forward network. Its final layer is the classifier, parameterized by a weight vector, and the two previous layers compute Fisher vectors, parameterized by the coefficients of a Gaussian mixture model. We introduce a gradient descent based learning algorithm that, in contrast to other feature learning techniques, is not just derived from intuition or biological analogy, but has a theoretical justification in the framework of statistical learning theory. Our experiments show that the new training procedure leads to significant improvements in classification accuracy while preserving the modularity and geometric interpretability of a support vector machine setup.

@inproceedings{ sydorov-cvpr2014,
author = {Vladyslav Sydorov and Mayu Sakurada and Christoph H. Lampert},
title = {Deep Fisher Kernels: Jointly Learning a Fisher Kernel SVM and its GMM Parameters},
booktitle = "IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)",
year = 2014,
}
Blending Liquids

Blending Liquids

ACM Transactions on Graphics 33(4)

  • Raveendran
  • Wojtan
  • Thuerey
  • Turk
A General Framework for Bilateral and Mean Shift Filtering

A General Framework for Bilateral and Mean Shift Filtering

ArXiv: 1405.4734

  • Solomon
  • Crane
  • Butscher
  • Wojtan

We present a generalization of the bilateral filter that can be applied to feature-preserving smoothing of signals on images, meshes, and other domains within a single unified framework. Our discretization is competitive with state-of-the-art smoothing techniques in terms of both accuracy and speed, is easy to implement, and has parameters that are straightforward to understand. Unlike previous bilateral filters developed for meshes and other irregular domains, our construction reduces exactly to the image bilateral on rectangular domains and comes with a rigorous foundation in both the smooth and discrete settings. These guarantees allow us to construct unconditionally convergent mean-shift schemes that handle a variety of extremely noisy signals. We also apply our framework to geometric edge-preserving effects like feature enhancement and show how it is related to local histogram techniques.

Edit Propagation using Geometric Relationship Functions

Edit Propagation using Geometric Relationship Functions

ACM Transactions on Graphics, 33(2)

  • Paul
  • Jeschke
  • Wimmer
  • Wonka

We propose a method for propagating edit operations in 2D vector graphics, based on geometric relationship functions. These functions quantify the geometric relationship of a point to a polygon, such as the distance to the boundary or the direction to the closest corner vertex. The level sets of the relationship functions describe points with the same relationship to a polygon. For a given query point we ?rst determine a set of relationships to local features, construct all level sets for these relationships and accumulate them. The maxima of the resulting distribution are points with similar geometric relationships. We show extensions to handle mirror symmetries, and discuss the use of relationship functions as local coordinate systems. Our method can be applied for example to interactive ?oor-plan editing, and is especially useful for large layouts, where individual edits would be cumbersome. We demonstrate populating 2D layouts with tens to hundreds of objects by propagating relatively few edit operations.

@article{Guerrero-2014-GRF,
author = {Paul Guerrero and Stefan Jeschke and Michael Wimmer and Peter Wonka},
title = {Edit Propagation using Geometric Relationship Functions},
journal = {ACM Transactions on Graphics},
year = {2014},
volume = {33},
number = {2},
pages = {15:1--15:15}
}
Putting Holes in Holey Geometry: Topology Change for Arbitrary Surfaces

Putting Holes in Holey Geometry: Topology Change for Arbitrary Surfaces

ACM Transactions on Graphics 32(4) (SIGGRAPH 2013)

  • Bernstein
  • Wojtan
Liquid Surface Tracking with Error Compensation

Liquid Surface Tracking with Error Compensation

ACM Transactions on Graphics 32(4) (SIGGRAPH 2013)

  • Bojsen-Hansen
  • Wojtan

Our work concerns the combination of an Eulerian liquid simulation with a high-resolution surface tracker (e.g. the level set method or a Lagrangian triangle mesh). The naive application of a high-resolution surface tracker to a low-resolution velocity field can produce many visually disturbing physical and topological artifacts that limit their use in practice. We address these problems by defining an error function which compares the current state of the surface tracker to the set of physically valid surface states. By reducing this error with a gradient descent technique, we introduce a novel physics-based surface fairing method. Similarly, by treating this error function as a potential energy, we derive a new surface correction force that mimics the vortex sheet equations. We demonstrate our results with both level set and mesh-based surface trackers.

@article{LSTwEC2013,
author = {Morten Bojsen-Hansen and Chris Wojtan},
title = {Liquid Surface Tracking with Error Compensation},
journal = {ACM Transactions on Graphics (SIGGRAPH 2013)},
year = {2013},
volume = {32},
number = {4},
pages = {79:1--79:10}
}
Controlling Liquids Using Meshes

Controlling Liquids Using Meshes

ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA 2012)

  • Raveendran
  • Thuerey
  • Wojtan
  • Turk
Highly Adaptive Liquid Simulations on Tetrahedral Meshes

Highly Adaptive Liquid Simulations on Tetrahedral Meshes

ACM Transactions on Graphics 32(4) (SIGGRAPH 2013)

  • Ando
  • Thürey
  • Wojtan
Computational Design of Mechanical Characters

Computational Design of Mechanical Characters

ACM Trans. Graph. 32, 4 (SIGGRAPH 2013 Papers)

  • Coros
  • Thomaszewski
  • Noris
  • Sueda
  • Forberg
  • Sumner
  • Matusik
  • Bickel

We present an interactive design system that allows non-expert users to create animated mechanical characters. Given an articulated character as input, the user iteratively creates an animation by sketching motion curves indicating how different parts of the character should move. For each motion curve, our framework creates an optimized mechanism that reproduces it as closely as possible. The resulting mechanisms are attached to the character and then connected to each other using gear trains, which are created in a semi-automated fashion. The mechanical assemblies generated with our system can be driven with a single input driver, such as a hand-operated crank or an electric motor, and they can be fabricated using rapid prototyping devices. We demonstrate the versatility of our approach by designing a wide range of mechanical characters, several of which we manufactured using 3D printing. While our pipeline is designed for characters driven by planar mechanisms, significant parts of it extend directly to non-planar mechanisms, allowing us to create characters with compelling 3D motions.

Tracking Surfaces with Evolving Topology

Tracking Surfaces with Evolving Topology

ACM Transactions on Graphics 31(4) (SIGGRAPH 2012)

  • Bojsen-Hansen
  • Li
  • Wojtan

We present a method for recovering a temporally coherent, deforming triangle mesh with arbitrarily changing topology from an incoherent sequence of static closed surfaces. We solve this problem using the surface geometry alone, without any prior information like surface templates or velocity fields. Our system combines a proven strategy for triangle mesh improvement, a robust multi-resolution non-rigid registration routine, and a reliable technique for changing surface mesh topology. We also introduce a novel topological constraint enforcement algorithm to ensure that the output and input always have similar topology. We apply our technique to a series of diverse input data from video reconstructions, physics simulations, and artistic morphs. The structured output of our algorithm allows us to efficiently track information like colors and displacement maps, recover velocity information, and solve PDEs on the mesh as a post process.

@article{TSwET2012,
author = {Morten Bojsen-Hansen and Hao Li and Chris Wojtan},
title = {Tracking Surfaces with Evolving Topology},
journal = {ACM Transactions on Graphics (SIGGRAPH 2012)},
year = {2012},
volume = {31},
number = {4},
pages = {53:1--53:10}
}
Explicit Mesh Surfaces for Particle Based Fluids

Explicit Mesh Surfaces for Particle Based Fluids

Computer Graphics Forum 31 (Eurographics 2012)

  • Yu
  • Wojtan
  • Turk
  • Yap
Liquid Simulation with mesh-based Surface Tracking

Liquid Simulation with mesh-based Surface Tracking

ACM SIGGRAPH 2011 Courses

  • Wojtan
  • Müller-Fischer
  • Brochu
Analysis of Human Faces using a Measurement-Based Skin Reflectance Model

Analysis of Human Faces using a Measurement-Based Skin Reflectance Model

ACM Transactions on Graphics 25(3) (SIGGRAPH 2006)

  • Weyrich
  • Matusik
  • Pfister
  • Bickel
  • Donner
  • Tu
  • McAndless
  • Lee
  • Ngan
  • Jensen
  • Gross

We have measured 3D face geometry, skin reflectance, and subsurface scattering using custom-built devices for 149 subjects of varying age, gender, and race. We developed a novel skin reflectance model whose parameters can be estimated from measurements. The model decomposes the large amount of measured skin data into a spatially-varying analytic BRDF, a diffuse albedo map, and diffuse subsurface scattering. Our model is intuitive, physically plausible, and – since we do not use the original measured data – easy to edit as well. High-quality renderings come close to reproducing real photographs. The analysis of the model parameters for our sample population reveals variations according to subject age, gender, skin type, and external factors (e.g., sweat, cold, or makeup). Using our statistics, a user can edit the overall appearance of a face (e.g., changing skin type and age) or change small-scale features using texture synthesis (e.g., adding moles and freckles). We are making the collected statistics publicly available to the research community for applications in face synthesis and analysis.