Second generation benchmarking and application oriented ... - CVML

Second generation benchmarking
and application oriented evaluation
S.Pereira, S. Voloshynovskiy, M. Madueno,
S. Marchand-Maillet and Thierry Pun
Computer Vision Group
University of Geneva
Computer Vision Group
University of Geneva

Outline
1. Benchmarking state-of-art
2. New attacks
3. New quality metrics
4. Checkmark: Application oriented
evaluation
5. Conclusions
Computer Vision Group
University of Geneva

1. Benchmarking state-of-art
• Robustness cannot be based on secrecy of the algorithms
(Kerchoff’s principle)
WM embedder
Robustness=f(secrecy)
• Necessity of theoretical and practical proofs
WM embedder
Test Tool
• The development of efficient tools for testing WM schemes
is as important as the development of the WM algorithms
themselves.
Computer Vision Group
University of Geneva

1. Benchmarking state-of-art
Deficiencies of Stirmark 3.1
• Not application driven
• Inadequate perceptual quality evaluation (PSNR)
• Not enough sophisticated geometrical transformations
• Does not take into account prior information on the
image and on the watermark
• Ignores issue of speed at embeding/detection
Computer Vision Group
University of Geneva

1. Benchmarking state-of-art
Classification of attacks
Removal and
Interference
Attacks
Geometrical
Attacks
Cryptographic
Attacks
Protocol
Attacks
Denoising
Lossy Compression
Quantization
Remodulation
Collusion
Averaging
…
Global warping
Local warping
Global transforms
Local transforms
Projective transforms
…
Brute force
Oracle
…
Inversion
Copy attack
…
Computer Vision Group
University of Geneva

1. Benchmarking state-of-art
Stirmark implemented attacks
• Convolutions filters
• Median filters
• JPEG
• Color Quantisation
• Scaling
• Shearing
• Change of aspect-ratio
• Linear transformations
• Rotation & cropping
• Rotation, scale and cropping
• Cropping
• Flip
• Row and column removal
• Random Bending
}
HIGH
GEOMETRICAL
WEIGHT
Computer Vision Group
University of Geneva

2. New attacks
Estimation-based
Assumption: WM technology is linear additive
y = x + w
x: Original data
w: Watermark
• estimation of cover data (denoising, perceptual remodulation,…)
• estimation of watermark (copy attack, synchronization removal,…)
Different stochastic methods:
• ML-estimate (local mean, local median, …)
• MAP-estimate ( Wiener Lee,…)
Computer Vision Group
University of Geneva

2. New attacks
Denoising and remodulation attacks
(estimation-based)
Marked data
WM
estimation
-
+
+
Attacked data
Mask
estimation
Additional
noise
Scaling factor
Aim at modification of WM using modulation opposite to that used at embedding
Computer Vision Group
University of Geneva

2. New attacks
The copy attack (estimation-based)
Marked
data
WM
estimation
Scaling factor
Target
data
Perceptual
mask
Attacked
data
Computer Vision Group
University of Geneva

2. New attacks
Synchronization removal (estimation-based)
TEMPLATE
Marked
data
Watermark
estimation
F
| · |
Peak
detector
| · |
Interpolator
F
arg
F -1
Affine
Transform
Attacked
data
The basic idea is:
•detect synchronization patterns
•remove
•apply desynchronization techniques.
Computer Vision Group
University of Geneva

2. New attacks
• Non-uniform line removal (random interval)
• Collage attacks (ex: magazine applications)
• Projective Transforms
Geometrical attacks
Attacked data
Simulation of
Record TV screen
x
Record cinema screen
y
Marked
data
2D
z=f (x,y)
Sphere
Cylinder
Wave
…
3D
Geometrical
Transforms
Rotations
Shearing
Scale changes
…
3D
Projection
back
Perspective projection
Parallel projection
2D
Random Bending
Computer Vision Group
University of Geneva

3. New quality metrics
Objective: Take into consideration the HVS!
• Weighted PSNR (global)
• Watson Model
• Local (no. of visible & possible visible blocks,TPE)
• Weight the errors for each DCT coefficient by its
corresponding sensivity threshold
• Sensivity threshold is function of luminance, texture
and frequency
Computer Vision Group
University of Geneva

3. New quality metrics
Comparison
• PSNR is greater than 40dB
• Watson metric detects 2 visible blocks
Computer Vision Group
University of Geneva

4. Checkmark
• Partially in the scope of the CERTIMARK European
Project:
http://www.certimark.org
• Contains a set of attacks not yet considered
• Performs better perceptual evaluation
• Introduces the concept of application-based evaluation
or benchmarking
Computer Vision Group
University of Geneva

4. Checkmark
Application oriented evaluation
• XML description
• Consider each attack as a function of application
• Each attack is weighted differently
• Easy to integrate new attacks and applications
• Automatic generation of results as an HTML file
Computer Vision Group
University of Geneva

4. Checkmark
Illustration of applications
• MediaBridge
• Passports
• Banknotes
• Copyright protection of images
• Analog
• BW & color newspapers
• Books
• Professional photography
• Digital
• Website
• digital camera
• Maps
• BW & color medical images
• Offline commerce
Computer Vision Group
Defocusing
Denoising …
Slightly geometrical attacks
Copy attack
Medium geometrical attacks
fl Large geometrical attacks
Medium compression
High compression
fl Low compression
…
In general all attacks
University of Geneva

5. Conclusions
• Current Stirmark 3.1 benchmark is inadequate for
evaluating WM schemes
• New benchmarking tool Checkmark
• Includes much more sophisticated attacks
• Application based evaluation
• Performs more appropriate perceptual evaluation
CHECKMARK TOOL: http://watermarking.unige.ch
• Open source, Matlab/XML implementation
• Results will be posted and software will be available
for downloading in a few weeks
Computer Vision Group
University of Geneva

5. Conclusions
We welcome contributions for:
• New useful attacks and
• New applications
• Submision of XML results and related publications
Advantages:
• Common place for results in HTML
• Easy to compare
• Easy to determine strength of algoritms
Computer Vision Group
University of Geneva