Audiveris: Optical Music Recognition
Audiveris: Optical Music Recognition
Audiveris: Optical Music Recognition
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
MUSIC HACK DAY, Vienna 2013<br />
<strong>Audiveris</strong><br />
<strong>Optical</strong> <strong>Music</strong> <strong>Recognition</strong><br />
Presented by Hervé Bitteur<br />
herve.bitteur@audiveris.org February 2, 2013<br />
V1
Agenda<br />
● User demo<br />
● The big picture<br />
● <strong>Audiveris</strong> technology<br />
● Current 4.2 release<br />
● Integration<br />
● Future<br />
VIENNA 2013<br />
www.audiveris.org<br />
2
User Demo<br />
● User oriented<br />
<br />
<br />
<br />
● Case<br />
Input<br />
Corrections<br />
Outputs<br />
Input : Dichterliebe01.pdf<br />
Language : DEU<br />
Tempo : 80<br />
Output : Finale Notepad<br />
VIENNA 2013<br />
www.audiveris.org<br />
3
Score Transcription « big picture »<br />
<strong>Optical</strong> <strong>Music</strong> <strong>Recognition</strong><br />
Printed<br />
Scores<br />
Scan<br />
Scores<br />
Images<br />
OMR<br />
<strong>Audiveris</strong><br />
Edition<br />
Edit<br />
View<br />
Play<br />
Transpose<br />
MuseScore<br />
Pivot Format<br />
Symbolic<br />
Notation<br />
<strong>Music</strong>XML<br />
<strong>Music</strong><br />
Info<br />
Retrieval<br />
PeachNote<br />
Index<br />
Search<br />
VIENNA 2013<br />
www.audiveris.org<br />
4
<strong>Music</strong> notation<br />
● <strong>Music</strong>XML<br />
Symbol level<br />
High level<br />
Meant for recorded music<br />
Meant for score interchange<br />
Note level<br />
Meant for digital instruments<br />
Sound level<br />
Low level<br />
● MIDI<br />
● MP3<br />
VIENNA 2013<br />
www.audiveris.org<br />
5
OCR vs OMR<br />
Complexity<br />
OCR<br />
(text)<br />
Horizontal lines of<br />
characters<br />
OMR<br />
(music)<br />
Combined horizontal &<br />
vertical directions<br />
Technology Rather mature Several years<br />
behind OCR<br />
Products Many Just a few.<br />
One FOSS: <strong>Audiveris</strong><br />
VIENNA 2013<br />
www.audiveris.org<br />
6
Basic definitions: Runs & Sections<br />
Black run of 3 pixels<br />
Section of 6 adjacent runs<br />
White run of 18 pixels<br />
VIENNA 2013<br />
www.audiveris.org<br />
7
Tech Demo<br />
● Technology oriented<br />
<br />
<br />
Runs<br />
Grid<br />
● Case<br />
<br />
<br />
<br />
<br />
Input : Chula.png<br />
Language : FRA<br />
Params : showRuns, showCombs<br />
Output : MuseScore<br />
VIENNA 2013<br />
www.audiveris.org<br />
8
Pixels: Staff Lines & other objects<br />
VIENNA 2013<br />
www.audiveris.org<br />
9
Black runs histo. → line thickness<br />
VIENNA 2013<br />
www.audiveris.org<br />
10
White runs histogram → interline<br />
VIENNA 2013<br />
www.audiveris.org<br />
11
Vertical runs > line thickness<br />
VIENNA 2013<br />
www.audiveris.org<br />
12
Vertical runs
Short horizontal runs put aside<br />
VIENNA 2013<br />
www.audiveris.org<br />
14
Long hori. runs → lines skeleton<br />
VIENNA 2013<br />
www.audiveris.org<br />
15
Clusters of equidistant lines<br />
VIENNA 2013<br />
www.audiveris.org<br />
16
Staff lines detected<br />
VIENNA 2013<br />
www.audiveris.org<br />
17
Staff lines removed<br />
VIENNA 2013<br />
www.audiveris.org<br />
18
Wavy staff lines: Splines<br />
VIENNA 2013<br />
www.audiveris.org<br />
19
Wavy staff lines: Dewarped image<br />
VIENNA 2013<br />
www.audiveris.org<br />
20
Stems detected<br />
VIENNA 2013<br />
www.audiveris.org<br />
21
Sections → Glyphs → Shapes<br />
VIENNA 2013<br />
www.audiveris.org<br />
22
Generated score<br />
VIENNA 2013<br />
www.audiveris.org<br />
23
<strong>Audiveris</strong> version 4.2<br />
● Released in December 2012<br />
● Features:<br />
<br />
<br />
<br />
<br />
<br />
Ghostscript integration<br />
– for PDF input<br />
Adaptive Binarization<br />
– for non-uniform images<br />
Tesseract OCR V3<br />
– for text handling<br />
ART-based moments<br />
– for shape recognition<br />
Plugins<br />
– for <strong>Music</strong>XML consumers<br />
VIENNA 2013<br />
www.audiveris.org<br />
24
Local OMR Integration<br />
client<br />
scan<br />
display<br />
plugin<br />
<strong>Audiveris</strong><br />
<strong>Music</strong>XML<br />
plugin<br />
MuseScore<br />
VIENNA 2013<br />
www.audiveris.org<br />
25
OMR as a Service (hack of the day)<br />
client<br />
server<br />
display<br />
scan<br />
internet<br />
Front<br />
End<br />
<strong>Music</strong>XML<br />
MuseScore<br />
<strong>Audiveris</strong><br />
VIENNA 2013<br />
www.audiveris.org<br />
26
<strong>Audiveris</strong> evolution<br />
● Technology<br />
<br />
<br />
<br />
Good building blocks available<br />
4.2+: iterative approach OK for good-quality scans<br />
5.x: fuzzy logic needed for poor-quality scans<br />
● Architecture<br />
<br />
Focus on OMR service<br />
● « Freemium » model<br />
<br />
<br />
Free for basic features or low volume<br />
Premium for more serious usage<br />
VIENNA 2013<br />
www.audiveris.org<br />
27
Thank you<br />
www.audiveris.org<br />
<strong>Audiveris</strong> [latin] := « you will have heard »<br />
herve.bitteur@audiveris.org
Change language