What publishers need to know about digitization

What publishers need to
know about digitization
Liza Daly
Consultant, Threepress Consulting Inc.
http://threepress.org/

Thursday, November 13, 2008

Introduction
Liza Daly liza@threepress.org

Software engineer and consultant specializing in
web-based publishing applications
Digitization projects for Ford Foundation, Arnold
Arboretum, Rosen Publishing and SAGE Publications
Online reference products for Oxford University Press
and Columbia University Press
Current: ebook applications and consulting


Introduction
What I’ll cover

1. Digitization 101: from scanning to OCR to XML
2. Smart vendor selection
3. A gentle introduction to XML
4. I’ve got digital content: now what?

?

What we talk about
when we talk about digitization

Turning printed content... text

...or microﬁlm archives
...or documents in legacy systems
...into modern digital forms.
(sometimes starting from print is easier)
<text>

Digitization 101

Assume that we’re starting from a print archive.
(If you’re starting from a digital ﬁle, congratulations,
your costs just went down -- but not to zero!)


Scan

From paper to digital images...


OCR

...to digital text...


XML

...to reusable markup.


Digitization 101
Scanning

http://www.ﬂickr.com/photos/heather-dietz/448629362/


Digitization 101
Scanning

Scan

http://www.ﬂickr.com/photos/heather-dietz/448629362/


Digitization 101
Scanning methods

Destructive scanning
Pages are cut out of the binding and
machine-fed into the scanner in batch.
(Imagine a huge ofﬁce copier.)
Scanned copies are normally destroyed.


Digitization 101
Scanning methods

Non-destructive scanning
Pages kept in their original binding
Manual page-turning
Originals are returned to the source
Primarily for rare or historical works


Digitization 101
Scanning methods

High-volume,
non-destructive
automated
scanning also
exists.


Digitization 101
OCR
Optical Character Recognition
OCR software “guesses” the letters that appear in an
image. A dictionary is used to help correct errors.

Common errors include wordsruntogether or
speling mistakes.


Digitization 101
OCR

OCR quality is sensitive to a number of factors.
Is the document in good condition with clear type?
Is the layout simple or complex?
Is a custom dictionary required for proper names or
obscure terms?


This is easy.


This is hard.


http://timesmachine.nytimes.com/


Digitization 101
OCR

Better OCR Worse OCR

Multicolumn,
Layout Simple text
sidebars

Vocabulary Common Specialized

Damaged, dirty or
Source quality Clean and legible
partial


Digitization 101
OCR

Limitations and cautions:
Documents with specialized jargon, such as medical
journals or archaic texts, will require custom
dictionaries.
Tables and equations aren’t suitable for OCR.
A human check is always advisable.


If the goal of digitization is to
make content ﬁndable on
the web, the text needs to
be correct.


SCAN the documents to
convert to digital ﬁles

Apply OCR to the scans to get
computer-ready text

Convert the text into XML X


Digitization 101
XML

Not all digitization projects end with XML.

Why?


Characters-per-page versus digitization cost/time

1,000 1,500 2,000 3,000+
XML
Human-checked OCR
Machine OCR


Vendor selection
and costs


Consider: But also:
Quantity of material Project management
Quality of the originals Shipping
Layout complexity Heterogeneous content
Vocabulary Front/back matter &
indexes


Vendor tips
Send samples before considering any estimate
...and have the output evaluated.
Compare not just cost-per-page but estimated time.
Feel comfortable with their project management.
Check references!


Should you partner?


?

?

It’s too early to say whether
Google Books is right for all
publishers.

But you’re certainly giving up:
1. Control
2. Revenue share
3. Ownership


Creative partnerships
Consider whether some of
your backlist is public
domain or can be released
under a Creative
Commons license.


XML 101


XML 101
What’s XML?

XML is just plain text, with markers to
tell a computer what the text means
and how it should be laid out.


XML 101
What’s XML?

Text with “markup” is an old idea.

This is a paragraph.¶
This is another paragraph.


XML 101
What’s XML?

XML just changes the symbols around.

This is a paragraph.
This is another paragraph.

XML 101
What’s XML good for?

1. Everybody speaks it.

2. Once you have one kind of XML,
it’s easy to turn it into another kind.


When you decide to digitize to XML,
you’ll need to pick what kind of XML you want.


Kinds of XML


Kinds of XML

DTD


Kinds of XML

Language
DTD


Kinds of XML

Language
DTD

Format


Kinds of XML

Language
DTD

Schema
Format


Kinds of XML

Language
DTD

Schema
Format
XSD


XML 101
Schema vocabulary

The schema deﬁnes the list of <tags> that appear in a
document, and what they mean.
A paragraph ¶ in one schema might be , but in
another it might be <para>.

METS/
DocBook
ALTO

ePub PRISM

DAISY TEI


METS/
DocBook
ALTO

ePub XML PRISM

DAISY TEI


XML 101
Choosing a schema

Books DocBook, DAISY, ePub, TEI

Magazines/
Newspapers METS/ALTO, PRISM

Scholarly TEI, MathML


XML 101
DIY schemas

Creating your own schema
should be a last resort.

Expensive to build and maintain.
High training and hiring costs.
Reduced opportunities for interoperability.
Regulatory compliance.


Complex schemas cost more...

$$$

$
Low High

...but also provide more opportunity
for product development.

Now what?


Monetizing
XML conversion

XML


Monetizing
XML conversion

XML web


XML web


UGC web


Remixing content

XML allows content
to be distributed, altered,
and recontextualized
in unexpected ways.

http://ﬂickr.com/photos/thomashawk/2492298772/

Small Beer Press


Questions?

Liza Daly
Threepress Consulting Inc.
+01 617 301 0552
liza@threepress.org


What publishers need to know about digitization

Recomendados

Recomendados

Más contenido relacionado

Similar a What publishers need to know about digitization

Similar a What publishers need to know about digitization (11)

Más de Liza Daly

Más de Liza Daly (8)

Último

Último (20)

What publishers need to know about digitization