Tools for mass digitization and long-term perservation in cultural ...

Tools for mass digitisation and long-term
preservation in cultural heritage institutions
Cezary Mazurek, Tomasz Parkoła, Marcin Werla
SEEDI conference 17-18 of May 2012

Agenda
• Introduction
• Mass digitisation
• Long-term preservation
• Tools supporting digitisation activities
• dLibra and dMuseion – digital library/repository/museum
• dArceo – long-term preservation services
• dLab – digitisation workflow management tool
• Summary
2

Introduction
Mass digitisation is a large-scale automated process
of capturing the analog signal into digital form,
including enhancements such as OCR and
transcription.
Long-term preservation assures that the digital
information is accesible today, tomorrow, in a year, 10
years, etc.
3

Supporting tools
Developed by Poznań Supercomputing and
Networking Center (Polish Academy of Sciences)
• dLibra – digital library framework
• dMuseion – digital museum framwork
• dArceo – long-term preservation services
• dLab – digitisation workflow management
4

dLibra
Developed by Poznań Supercomputing and
Networking Center (Polish Academy of Sciences)
• Has been developed by PSNC since 1999
• The first Polish software for building digital libraries
• Key element in stimulating the growth of digital
libraries in Poland
5

dLibra – deployments
60
50
Number of deployments
40
30
20
10
0
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Institutional deployments 0 0 1 1 5 7 13 22 26 28
Regional deployments 1 1 1 5 10 11 17 25 28 32

dLibra
Polish Digital Libraries
Approx. no of digital objects:
1 000 000
Number of digital libraries:
Over 80
Participating institutions:
Several hundreds
All resources available via the Polish
national aggregator – Digital Libraries
Federation:
http://fbc.pionier.net.pl
Regional digital libraries
Institutional digital libraries
7

dLibra – regional digital library
8

dLibra – institutional digital library
9

dMuseion
• Dedicated software tool for digital
museums
– Has been developed in cooperation with National Museum in
Warsaw
– The aim is to make the museum resources available in the Internet
and prepare an easy to use software package for building digital
museums
• Why a dedicated solution?
– Different type of resources than in libraries/respositories and
archives (paintings, sculptures, etc.)
– Themed collections of museum resources
– Terminology
10

dMuseion – main page
11

dMuseion – digital object metadata
12

dMuseion – 3D object
13

dArceo
• Has been developed by PSNC since 2011
• It is based on the prototype services developed in
frame of the SYNAT project, financed by the Polish
National Center for Research and Development
• Dedicated to preserve master, optimized and even
presentation files with primary focus on:
– Textual data (e.g. PDF/A)
– Images (e.g. TIFF, JPEG2000)
– Audiovisual (e.g. MPEG-4)
14

dArceo
Basic functions and characteristics (1)
• Can utilise various types of data storage
– Local hard drive (RAID is suggested), disk array, etc.
– (S)FTP, e.g. archiving services of PLATON U4 (R&D project)
• Internal representation uses well-known standards
and formats (e.g. METS, PREMIS)
• Dedicated build-in mechanism for data monitoring
– loss risk calculation based on PRONOM/UDFR
15

dArceo
Basic functions and characteristics (2)
• The most important functionality
– Data migration for the needs of long-term preservation (OAIS
transformation approach)
– Data conversion, e.g. for the needs of digital libraries and
online availability of resources
– Advanced data delivery, e.g. dedicated tool for large-size data
visualisation, transcription tool
• It is possible to define migration/conversion plans
– Migration or conversion can have several steps (pipelining of
the services)
– Semantic technologies applied for the orchestration of the
data manipulation services
16

dArceo
Basic functions and characteristics (3)
• Capability of sharing migration, conversion and
advanced delivery services
– By means of synchronisation and P2P-like communication
• Data safety and security
– AAA of the users and external services
– Data safety should be assured by the data storage (e.g.
redundancy, distant locations)
17

dLab
Basic functions and characteristics
• General
– Support digitisation activities
– Management of the digitisation workflow
– Monitoring capability
• Terminology
– Digitisation task
• Basic element in the system, related to particular element, e.g. book, issue,
etc.
• Covers all the activities necessary to finish digitisation of particular element
– Activity in frame of the digitisation task
• Represents certain work to be done, e.g. prepare optimized files
• The work is done by human (user) or a machine (tool)
• Flexibility
– A set of activities in scope of a task
– Order constraints
– Pluggable architecture
18

dLab – digitisation task
Editor
Scanner
operator
Tool
QA
Task A
Select object
to digitise
Prepare
master files
Create PDF
Archive
master files
Verify
Submit PDF to
digital library
19

dLab – external tools
dLab
Plugins for external tools
dArceo
FineReader
Document
Express
External tools
dLibra
plugin
dArceo
plugin
FR plugin
DE plugin
dLab
dLibra
Activity
A
Activity
B
Activity
C
Activity
…
dLab UI
Working space
(e.g. disk array)

Summary (1)
• Digitisation is an important mission of all cultural
heritage institutions
• Long-term preservation is a key element of each digital
library or museum
• Mass digitisation needs a strong support in terms of
management
• Poznań Supercomputing and Networking Center acts
as a supporting institution
– R&D center (PAS) in the IT area, including digital libraries
– Cooperates with various national and international bodies
to stimulate growth and advancements of information
society
21

Summary (2)
Complex software solution for cultural
heritage institutions
dLab
dArceo
master files
long-term preservation
digitisation process
dLibra or
dMuseion
presentation files
online availability of resources
22

Questions?
Tomasz Parkoła
tparkola@man.poznan.pl
http://dl.psnc.pl
23

Thank you!