Continuous and Discrete Optimization Methods in Computer Vision

Continuous and Discrete 

Optimization Methods 

in Computer Vision 

Daniel Cremers 

Department of Computer Science 

University of Bonn 

Oxford, August 16 2007

Segmentation by Energy Minimization 

Given an image , minimize the energy 

Blake, Zisserman ’87, Mumford, Shah ’89, 

Ising ’27, Potts ’59, Geman, Geman ‘84 

boundary length 

Daniel Cremers Continuous and Discrete Optimization in Computer Vision 

2

Continuous Versus Discrete Optimization 

Osher, Sethian, J. of Comp. Phys. ’88 

Min. cut Max. flow 

Ford, Fulkerson ’59 

Greig et al. ’89 

Boykov, Kolmogorov ’02 


3

Level Set Formulation of Mumford-Shah 

Chan & Vese ’99 


4

Level Set Segmentation in Feature Space 

efficient coarse-to-fine scheme 

Brox, Weickert ’04, ’06 


5

Motion segmentation 

Obstacle segmentation 

Prior shape knowledge 

Shape priors in global optimization 

Convex 3D reconstruction 

Overview 

Markerless human motion capture 


6

Overview 





3D reconstruction: a convex formulation 


Cremers CVPR ’03, Cremers & Soatto IJCV ’05 


7

Motion 


8

Problem underconstrained (aperture problem). 

2. assumption: Velocity in each region is a 

Gaussian-distributed random variable 

Minimize 

Motion-based Image Segmentation 

1. assumption: Intensity of moving points is preserved. 

Cremers CVPR ’03, Cremers & Soatto IJCV ’05 

Segmentation 

& 

Motion 


9

Motion Estimation 

Motion Competition 

Segmentation 


10

Motion Competition via Level Sets 

Cremers, Yuille, ’04 


11


What is moving? 


12


Cremers, Yuille, ’04 


13


Cremers, Soatto, IJCV ’05: Piecewise Parametric Motion 


14

Motion Competition via Graph Cuts 

piecewise constant piecewise affine 

Schoenemann & Cremers, DAGM ’06 

up to 30 fps 


15

Overview 







Wedel, Schoenemann, Brox, Cremers DAGM ’07 


16

WarpCut: Obstacle Segmentation 

Segmentation of obstacles in monocular video 


17


Local scale changes depend on distance from the camera. 


18


Competing hypotheses: Obstacle or road? 


19


Wedel, Schoenemann, Brox, Cremers DAGM ’07 


20

Overview 








21

Statistical Priors for Image Segmentation 

initial contour no prior with prior 

with prior with prior scale invariance 

Cremers, Kohlberger, Schnörr, ECCV 2002 


22

Statistical Learning of Familiar Shapes 

Cremers, Osher, Soatto, IJCV ’06 

Rosenblatt ’56, 

Parzen ’62 


23

Statistical Learning of Implicit Shapes 

Without statistical prior 


With statistical prior 

Sequence data courtesy of Alessandro Bissacco. 


24

Statistical Priors for Image Segmentation 

Purely geometric prior 

Nonparametric prior 



25

Dynamical Models for Implicit Shapes 

Training sequence 


26


1. Dimensionality reduction by PCA (Leventon et al. ’00, Tsai et al. ’01): 

2. Autoregressive model for the shape vectors: 

mean eigen modes 

3. Synthesized sequence of shape vectors and embedding functions: 

mean transition matrices Gaussian noise 

Cremers, IEEE Trans. on PAMI ’06 


27

Statistically Sampled Embedding Functions 



28


1. Low-dim. representation via PCA (Leventon et al. ’00, Tsai et al. ’01): 

2. Autoregressive model for the shape coefficients: 

3. Synthesize shape vectors and embedding surfaces: 


29


1. Low-dim. representation via PCA (Leventon et al. ’00, Tsai et al. ’01): 

2. Autoregressive model for the shape coefficients: 

3. Synthesize shape vectors and embedding surfaces: 


30

Dynamical Priors for Level Set Segmentation 

Bayesian Aposteriori Maximization: 

Image formation model Dynamical shape model 

(color, texture, motion,…) 

Optimization by gradient descent: 


31


Input sequence with 90% uniform noise 


32




33


Pure deformation model 



34


Model of joint deformation and transformation 



35

Overview 







Schoenemann & Cremers ICCV ‘07 


36

Limitations of Previous Approaches 

• Local optimization 

• Simple geometric distances 

• Little generalization 

• Requires large training sets 

• Point correspondence, 

• Missing parts, 

• Stretching/shrinking,... 

• Combinatorial problem 

Goal: Combinatorial solution for joint segmentation and alignment 


37

The Product Space of Image and Template 

Each node of the graph represents an image pixel and a point on the template. 

All template points S are allowed to stretch over at most K image pixels. 


38

The Product Space of Image and Template 

Each node of the graph represents an image pixel and a point on the template. 

All template points S are allowed to stretch over at most K image pixels. 

A cycle in this graph defines an assignment of template points to image pixels. 


39

Segmentation & Alignment by Energy Min. 

Data term Alignment with template Stretching cost 

Favors strong gradients. Favors minimal stretching / shrinking. 


40

Segmentation with a Single Template 

Template 

Segmentation results 

Optimization by Minimum Ratio Cycles on GPU: ~15 sec / frame 


41

No rotational invariance 

Rotational Invariance 

Rotational invariance 


42

Input sequence 

Tracking 

Globally optimal segmentations 


43

Overview 







Kolev, Klodt, Brox, Esedoglu & Cremers EMMCVPR ‘07 


44

3D Reconstruction 


45

Probabilistic Formulation 


46

Probabilistic Formulation 

Kolev, Brox, Cremers DAGM ’06 


47

Continuous Convex Formulation 

Introduce binary variable: 

Relax binary constraint: 

weighted TV norm 

Convex functional optimized over a convex set. 

Solution of binary-valued problem by thresholding. 

Global minima by gradient descent (or SOR). 



48

Multiview 3D Reconstruction 



49

Level sets 

Continuous 

Local optima 

Comparison 

Convex formulation 

Continuous 

Global optima* 

* for certain cost functionals only 

Graph cuts 

Discrete 

Global optima* 

Memory limitations 

Metrication errors 


50

Metrication Errors in Discrete Energies 

Synthetic data: 

Discrete graph cut optimization 

blank out data term 

in upper octant 

Continuous convex optimization 



51

Overview 







Brox et al. DAGM 2005 

Brox, Rosenhahn, Cremers & Seidel, ECCV ’06 


52

Human Motion Tracking 


53





54

Coupled Segmentation and 3D Tracking 




55

3D Tracking with Statistical Priors 

T. Brox, B. Rosenhahn, U. Kersting, D. Cremers, DAGM ’06 


56


T. Brox, B. Rosenhahn, U. Kersting, D. Cremers, DAGM ’06 


57

Motion competition 

3D reconstruction 

Summary 


Human motion tracking 

Dynamical shape priors 


58

Announcement: IPAM Workshop 

Boykov, Cremers, Darbon, Ishikawa, Kolmogorov, Osher: 

IPAM Workshop on “Graph Cuts and Related Discrete or 

Continuous Optimization Problems”, Feb. 25-29 2008 

https://www.ipam.ucla.edu/programs/gc2008/ 


59

Continuous and Discrete Optimization Methods in Computer Vision

Create successful ePaper yourself

Delete template?

Save as template?