GrabCut Interactive Foreground Extraction using Iterated Graph Cuts ...
GrabCut Interactive Foreground Extraction using Iterated Graph Cuts ... GrabCut Interactive Foreground Extraction using Iterated Graph Cuts ...
GrabCut Interactive Foreground Extraction using Iterated Graph Cuts Carsten Rother Vladimir Kolmogorov Andrew Blake Microsoft Research Cambridge-UK
- Page 2 and 3: Photomontage GrabCut - Interactive
- Page 4 and 5: Graph Cuts Boykov and Jolly (2001)
- Page 6 and 7: Iterated Graph Cuts 1 2 3 4 Result
- Page 8 and 9: Coherence Model An object is a cohe
- Page 10 and 11: Border Matting Foreground Mix 1 Noi
- Page 12 and 13: Moderately straightforward examples
- Page 14 and 15: Difficult Examples
- Page 16 and 17: Lazy Snapping Overview • Main Ste
- Page 18 and 19: Lazy Snaping - Speedup 18
- Page 20 and 21: User Study • Results • Easy of
<strong>GrabCut</strong><br />
<strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong><br />
<strong>using</strong> <strong>Iterated</strong> <strong>Graph</strong> <strong>Cuts</strong><br />
Carsten Rother<br />
Vladimir Kolmogorov<br />
Andrew Blake<br />
Microsoft Research Cambridge-UK
Photomontage<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 1
Problem<br />
Fast &<br />
Accurate ?<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 2
<strong>Graph</strong> <strong>Cuts</strong><br />
Boykov and Jolly (2001)<br />
Image<br />
<strong>Foreground</strong><br />
(source)<br />
Min Cut<br />
Background<br />
(sink)<br />
Cut: separating source and sink; Energy: collection of edges<br />
Min Cut: Global minimal enegry in polynomial time<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 5
<strong>Iterated</strong> <strong>Graph</strong> Cut<br />
?<br />
User Initialisation<br />
K-means for learning<br />
colour distributions<br />
<strong>Graph</strong> cuts to<br />
infer the<br />
segmentation<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 6
<strong>Iterated</strong> <strong>Graph</strong> <strong>Cuts</strong><br />
1 2 3 4<br />
Result<br />
Energy after each Iteration<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 7
Colour Model<br />
R<br />
<strong>Foreground</strong> &<br />
Background<br />
<strong>Iterated</strong><br />
graph cut<br />
R<br />
<strong>Foreground</strong><br />
Background<br />
G<br />
Background<br />
G<br />
Gaussian Mixture Model (typically 5-8 components)<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 8
Coherence Model<br />
An object is a coherent set of pixels:<br />
Blake et al. (2004): Learn<br />
jointly<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 9
Border Matting<br />
Hard Segmentation Automatic Trimap Soft Segmentation<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 16
Border Matting<br />
<strong>Foreground</strong><br />
Mix<br />
1<br />
Noisy alpha-profile<br />
Background<br />
0<br />
<strong>Foreground</strong> Mix Background<br />
Fit a smooth alpha-profile with parameters<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 19
Comparison<br />
Boykov and Jolly (2001)<br />
<strong>GrabCut</strong><br />
User<br />
Input<br />
Result<br />
Error Rate: 1.87% 1.81% 1.32% 1.25% 0.72%<br />
Error Rate: 0.72%<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 13
Moderately straightforward<br />
examples<br />
… <strong>GrabCut</strong> completes automatically<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 10
Difficult Examples<br />
Initial<br />
Rectangle<br />
Initial<br />
Result<br />
<strong>GrabCut</strong> – <strong>Interactive</strong> <strong>Foreground</strong> <strong>Extraction</strong> 11
Difficult Examples
Lazy Snapping<br />
15
Lazy Snapping Overview<br />
• Main Steps:<br />
1. Mark strokes as foreground & background (object<br />
marking)<br />
2. Perform Boykov & Jolly style graphcut<br />
3. Boundary Editing<br />
• Familiar Formulation<br />
• Minimize Gibbs Energy:<br />
• E 1 (X) – Cluster (K-means), encodes color similarity of<br />
node (likelihood energy)<br />
• E 2 (X) – encodes energy due to gradient along object<br />
boundary (color gradient between two nodes)<br />
16
Lazy Snapping—Pre-processing
Lazy Snaping – Speedup<br />
18
Boundary Editing<br />
• <strong>Graph</strong> Cut is only step 1<br />
• Allow user to edit boundary directly<br />
• Formulate as <strong>Graph</strong> Cut<br />
• Allow user to guide boundary regions w/ brush<br />
19
User Study<br />
• Results<br />
• Easy of Use – Lazy Snapping 20% less mistakes<br />
• Speed – Lazy Snapping 60% less time<br />
• Accuracy – Lazy Snapping 60% less pixels wrong<br />
• Feedback<br />
• “Almost Magic”<br />
• “Much Easier”<br />
• Suggestions<br />
• Get rid of 2 step process<br />
• Make it easy to switch between graph cut strokes and<br />
boundary editing<br />
20
More Results<br />
21
Change language