This document discusses using artificial intelligence to automatically classify historical photographs. It notes that classifying a large collection of 140,000 photos would require an impractical amount of manual labor. The document then outlines challenges in classifying old photos due to uncertainty about details like location, time of day, or people present. It proposes using computer vision and deep learning techniques like OpenCV, edge detection, and TensorFlow to conduct initial classifications of photos into categories like portraits, landscapes, and architecture. The document concludes by discussing expanding the classification capabilities and identifying new applications and data sources for the techniques.

1. www.tc-freyung.th-deg.de
Artificial intelligence and the
automatic classification of
historical photographs
Florian Eiler, Simon Graf, Wolfgang Dorner

2. www.tc-freyung.th-deg.de

3. www.tc-freyung.th-deg.de

4. www.tc-freyung.th-deg.de
4
Motivation

5. www.tc-freyung.th-deg.de
Documentation of cultural heritage and
conservation (Arias et al. 2006, Yilmaz et al. 2007)
Geomorphology – history and archeology
(Estes et. al. 1977)
Landscape planning
(Turner 1990, Chandler and Brundsten 1995)
Using historical photographs
Urban planning, ethnology, environmental planning,
environmental modelling, history of art, …

6. www.tc-freyung.th-deg.de
The quantitative problem
140.000 photos x 15 min
Scanning
Cropping
Metadata
= 2.1 Mmin
= 35 kH
= 4.375 labor days
= 20 labor years

7. www.tc-freyung.th-deg.de
The qualitative problem and uncertainty
Same or different
persons
different days
during the year
different
daytimes
classify
one phote
Epistemic
uncertainty
Aleatoric
uncertainty
Village
buildings
church
tower
Kaplice
Sv Marie
1944
Wolfgang Klara

8. www.tc-freyung.th-deg.de

9. www.tc-freyung.th-deg.de
History and now

10. www.tc-freyung.th-deg.de
Pre-classification
Portrait Group Scenes Landscape Architecture

11. www.tc-freyung.th-deg.de
Face detection with OpenCV

12. www.tc-freyung.th-deg.de
Canny algorithm for edge detection Line detection
Edge and line detection

13. www.tc-freyung.th-deg.de
Canny algorithm for edge detection Line detection
Edge and line detection

14. www.tc-freyung.th-deg.de
Image classification with Tensorflow
Category: Building

15. www.tc-freyung.th-deg.de
Basic training data
1.2 m
ImageNet
80% of data for training
10% for validation
10% for test

16. www.tc-freyung.th-deg.de
Preperatory steps
299px
299px
greyscale

17. www.tc-freyung.th-deg.de
Transfer Learning
Historical
photographs
(629)
portraits
(120)
landscape
(254)
group photographs
(119)
buildings/architecture
(136)
training
(50)
validation
(64)
test
(59)

18. www.tc-freyung.th-deg.de
Training steps and accuracy of classification with the test dataset

19. www.tc-freyung.th-deg.de
Face detection
Image
identification
SexAge
PortraitGroupScenes
Proportion of areasStructures
Landscape Architecture
Object recognition
Church Animals Car
What? Who? Where?
Classical
Computer
Vision
Artifical
Intelligence
HumanCrowd Sourcing / Citizen Science

20. www.tc-freyung.th-deg.de
20
§ Extend the classification
§ Identify pitfalls for algorithms
§ Identify details (symbols, signs, letters)
§ Partner for new data sets and applications
Next Steps

21. www.tc-freyung.th-deg.de
21
Funding & Partners
30.01.19
Photographs: Museum Fotoatelier Seidel in Cesky Krumlov

22. www.tc-freyung.th-deg.de
22
Wolfgang Dorner, Prof. Dr.
Florian Eiler, B. Eng.
Simon Graf, M. Sc.
INSTITUTE OF APPLIED INFORMATICS
Technische Hochschule Deggendorf
Technologie Campus Freyung
Grafenauerstr. 22
D - 94078 Freyung
Email: wolfgang.dorner@th-deg.de