Open Data Trentino presented at the European Commission (JRC)

Open Government Data*

http://dati.trentino.it/
Lorenzino Vaccari
Autonomous Province of Trento, Trento, Italy
lorenzino.vaccari@provincia.tn.it
*Part of this presentation is taken from the “Open Government Data Tutorial” gave at CLEI2013 Conference
by Lorenzino Vaccari and Juan Pane (Universidad Nacional de Asuncion, Paraguay)
1

Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - lorenzino.vaccari@provincia.tn.it 20/12/13

In this presentation…
• Introduce Open Government Data
•
•

Intro (Part 1)
Issues (Part 2)

• If you need it, how can you organize it?
•

Real experience (Part 3)

• Reusing open data
•
•

2

Applications (Part 4)
Semantic layer (Part 5)


3

Lorenzino Vaccari - Autonomous Province
Italy - lorenzino.vaccari@provincia.tn.it 20/12/13
Juan Pane, Lorenzino Vaccari of Trento, Trento,http://www.point-fort.com/index.php?2012/01/25/805-why-how-what
15/10/2013

http://www.point-fort.com/index.php?2012/01/25/805-why-how-what

What?

“is data that can be freely used, reused and redistributed by anyone –
subject only, at most, to the requirement to attribute and sharealike.” *

*(Source:

4

)

http://www.opendefinition.org

use
reuse
“open” =

redistribution
commercial reuse
derivative works

BUT, may require:
- attribution
- share alike
http://myfbcovers.com/uploads/covers/2012/06/09/16628a1094aa012f7c6e0025902480d2/watermarked_cover.jpg

5


J. Gray (OKF): http://www.slideshare.net/jwyg/open-government-data-what-why-how

The value is in its use

6


7

Maurizio Napolitano: http://www.youtube.com/watch?v=YlkjrVAW43Q

New visualizations

http://wheredoesmymoneygo.org/
8



New visualizations

http://openspending.org

9

13/11/2013


Why
 The Open data are the knowledge base to:
 Improve the economic grow and the

entrepreneurship based on the development of digital
services reusing Public Sector Information
 Answer to social needs through the publication of
innovative services and applications
 Aims at reducing the cost of the public
administrative activities within Public – Private
Partnerships (PPP)
 Improve the transparency of the activities of the public
institutions and the participation of the citizens to
these activities

10



How - Principles
Tim Berners-Lee (5-Stars of Linked Open Data)
http://5stardata.info/
vs.
Tim Davis (5-Stars of Open Data Engagement)
http://www.timdavies.org.uk/2012/01/21/5-stars-of-open-data-engagement/
vs.
OGD: Ten principles for opening up government
information http://sunlightfoundation.com/policy/documents/ten-open-data-principles/
…

11


5 Stars Linked Open Data

12

Tim Berners-Lee

http://5stardata.info

5-Stars of Open Data Engagement

Tim Davis

Create Community
13

http://msnbcmedia.msn.com/j/MSNBC/Components/Photo/_new/pb-121007-spain-tarragona-pyramid-nj-02.photoblog900.jpg

Open Government Data: Ten principles
for opening up government information
1.

Completeness

2.

Primacy (primary source)

3.

Timeliness

4.

Ease of Physical and Electronic Access

5.

Machine readability

6.

Non-discrimination

7.

Use of Commonly Owned Standards

8.

Licensing

9.

Permanence

10. Usage Costs

14


State of the Art
What is happening around us?
• Globally
• Europe
• Italy

15


Open Data Charter - G8
The principles are:
 Open Data by Default
 Quality and Quantity
 Useable by All
 Releasing Data for Improved Governance
 Releasing Data for Innovation

https://www.gov.uk/government/publications/open-data-charter/g8-open-data-charter-and-technical-annex
16

http://opensource.com/government

Open Data Census (OKF)

http://census.okfn.org/
17


http://census.okfn.org/country/
18


Open Data Barometer (ODI)

http://census.okfn.org/
19

http://opensource.com/government/13/7/open-data-charter-

OGD in Europe
screenshots

20

http://epsiplatform.eu/content/european-psi-scoreboard

OGD in Europe
Insert table

http://epsiplatform.eu/content/european-psi-scoreboard
http://epsiplatform.eu/content/psi-scoreboard-indicator-list

21


22

http://open-data.europa.eu/

OGD in Italy

23

http://www.dati.gov.it/content/infografica

OGD: Part 2 - Issues

24


25

Juan Pane, Lorenzino Vaccari of Trento, Trento, Italy - lorenzino.vaccari@provincia.tn.it 20/12/13
08/10/2013
http://evian-thesource.com/kids-having-fun/
http://evian-thesource.com/kids-having-fun/

Open Data. Oh ohh

Organizational

26

Legal

Adoption
Barriers

Technical

Contextual

Juan Pane, Lorenzino Vaccari of Trento, Trento, Italy - lorenzino.vaccari@provincia.tn.it 20/12/13
08/10/2013
http://www.wallpapermania.eu/wallpaper/trick-or-treat-cute-pumpkins-lanterns-halloween-wallpaper

27

http://de.straba.us/wp-content/uploads/2012/08/barrieres_for_implementation_of_ogd.png

Organizational Barriers

• Not ready
• Lack of resources
• IT
• Human

• Don’t want to be ready
28

http://montcomediation.org/images/MCMC_MyWayYourWay.jpg

Legal barriers
Open the Data
All the data that was produced using public money

has to be made publicly available (with exceptions)

vs Privacy
You cannot open data that could allow correlation

of private personal data

Or the complete lack of
legislation!
29


Adoption barriers
 Data is not contextualized
 People are not informed
 Opening data is a complex task, opening cleaned data is

even more complex.
 Unclear licenses

30


Technical Barriers
 Access to data:
 Organizational
 Technical, Downtimes, logins,
 Payment fees
 Fragmentation, incomplete data,

scattered
 Format
 Cataloging, indexing, search
 Lack of explicit semantics, metadata
 Data is not reliable
 Conflicting standards, models,
ontologies
31


Barriers
Zuiderwijk et al 2010
http://www.ejeg.com/issue/download.html?idArticle=255

Listed 118 socio-technical impediments for

opening data in the literature.
 Findability
 Usability
 Understandablity
 Quality
 Linking
 Comparability and compatibility
 Metadata
 ….

32


Context Barriers
 Privileged access to data
 Other companies what to avoid legislation of privacy.
 Transparency is bad for fraudulent business

33

http://img.gawkerassets.com/img/182n8vzdlg1iojpg/original.jpg

34

http://netdna.webdesignerdepot.com/uploads/photo_manipulation/manipulation-9.jpg

Part 3 - Real Experience

35


Our story started with GeoData…

36

http://www.territorio.provincia.tn.it

5 Stars Linked Geo Data Catalog
DBpedia

37

TrentinoGeoData

Freebase


The “Open Data in Trentino” project
• The “Open Data in Trentino” project is a 3 years initiative finalized to
develop an open data infrastructure to enhance Service
Innovation for Trentino following the PAT strategy for services
innovation enabled by ICT. The project will be developed within a
partnership between Trento RISE and the Autonomous Province of
Trento (PAT) according to the innovation PAT model
• Goals
• Improved quality of life for citizens
• Open Data and local businesses
• Transparency
• Improved efficiency and productivity

38


Workplan – Best practices
Not only a Project, but also a “Change
management process”
Best Practices:
-

39

Guidelines (metadata, formats, licences)
Point of contact (domain, operator)
ONE dataset each provider
Community Building
Distributed catalog
Clear Licences
Enterprises
Courses
Contest


Guidelines
Nome (Acronimo)
Descrizione

Tipo di Dato

Estensione
del file

Comma Separated Value (CSV)
Dato tabellare
Formato testuale per l'interscambio testuale di tabelle, le cui righe corrispondono a
linee e i cui valori delle singole colonne sono separati da una virgola (o punto e
virgola)
Geographic Markup Language (GML)
Dato geografico
Formato XML utile allo scambio di dati territoriali di tipo vettoriale
vettoriale
Keyhole Markup Language (KML)
Formato basato su XML creato per gestire dati territoriali in tre dimensioni nei
programmi Google Earth, Google Maps
Open Document Format (ODF)
Formato per l'archiviazione e lo scambio di documenti di testo, fogli di calcolo,
diagrammi e presentazioni
Resource Description Framework (RDF)
Basato su XML, e' lo strumento base proposto da World Wide Web
Consortium (W3C) per la codifica, lo scambio e il riutilizzo di metadati
strutturati e consente l'interoperabilità tra applicazioni che si scambiano
informazioni sul Web
ESRI Shapefile (SHP)
Lo Shapefile ESRI è un popolare formato vettoriale per sistemi informativi
geografici. Il dato geografico viene distribuito normalmente attraverso tre o quattro
files (se indicato il sistema di riferimento delle coordinate). Il formato è stato
rilasciato da ESRI come formato (quasi) aperto
Extensible Markup Language (XML)
E' un formato di markup, ovvero basato su un meccanismo che consente di
definire e controllare il significato degli elementi contenuti in un documento o in un
testo attraverso delle etichette (markup)

40

.csv

Dato geografico
vettoriale

.kml

Dato tabellare

.odc

.gml

Dato strutturato .rdf

Dato geografico
vettoriale

.shp, .shx, .dbf,
.prj

Dato strutturato .xml


Tecnological platform

…

Comune
Meteo GeoDati Statistica Comune Trasporti
Meteo GeoDati Statistica Trento
Trasporti
Trento

41

…

Etc…
Etc…


Data Sources

42


Data Sources Plan
Dati SA T #28
Catasto #
18

S GC CSW
#9

Bilanci o
PAT #37,
38

A ff F inanziari
Consuienze
#36

Gestione
S trade
#16

Immi grazi one
#52

P ersonal
e
P AT #41

Imp Funivie
#50

Idrom etri
ci
#26 Comune
T rento
update
20

Novembre

43

30

Turism o S TU
#53

A genzie F orestali
#64
Incendi #65

A ttività Culturalii
Dati Progettone#59
#63
S ovr. Beni A rch
#60
Dati
Motorizzazione Dati Scuola #61
#72

Dati Energia
#30
Trentino Cultura
#32
S ervzio Europa #56

Dati Traffico Stra
#70

20

Dicembre

Osservatorio
Lavori Pubb #17
T rasporti 2.0 #6

P ub. Eserci zi

#49

Gesti oni Patri monialii
#71

M in. Linguisti che
#48
Ufficio Ri fiuti
#34

10

Dati Cons. P rov #3

Attività
Form #58

Imp Depurazione
#68
Opere Civili #69

El ettorali #35

10

S ervizio
Istruzione #57

Ci nform i Stranieri
#66

30

10

20

Gennaio


30

Catalog
http://okfn.org
(2004)
The Open Knowledge Foundation (OKF) is a
non-profit organisation founded in 2004 and
dedicated to promoting open data and open
content in all their forms – including government
data, publicly funded research and public domain
cultural content.

44


http://dati.trentino.it*

* Available for all the data providers of Trentino
45

Analysis: http://dati.trentino.it/stats
Admin: http://dati.trentino.it/admin
Harvesting: http://dati.trentino.it/harvest


Services

46


Legal Issues

Permissions: share, create, adapt
Actual interoperability!
Constraints: nothing!

Permissions: share, create, adapt
Actual interoperability!
Constraints: nothing!

http://www.hoax-slayer.com/images/privacy.jpg
http://www.destateparks.com/images/general_info/privacy_policy.jpg

47


Organizational Issues - Macro

48


Organizational Issues - Micro

49


Community building
Municipalities
Municipalities
“Consorzio dei
“Consorzio dei
Comuni”
Comuni”

Educational Institutes
Educational Institutes

“Comunità di Valle”
“Comunità di Valle”
of Trentino
of Trentino

Research Institutes
Research Institutes

Private Companies
Private Companies

Citizens
Citizens

50


International Community

51


Also Trentino is going to launch a challenge to build software applications and creative
products (multimedia, audiovisual products, posters, illustrations) based on the datasets
published on the http://dati.trentino.it open data catalog.
#ODTChallenge will be the official hashtag for our first open data challenge in Trentino!

52


53


580 datasets
provided by 10 departments of PAT…
8 months until now
68.555 visits
7.988 unique visits
2.516 downloads
62,64% new
visitors
37,36% returning
visitors

20
15
10
6
54

reporting errors
asking for new data
new suggestions
OD Applications

Agriculture
Culture
Geographical Data
Welfare
Weather Forecast
Social policies
Statistics
Transports
…MUNICIPALITY OF TRENTO, and
INFORMATICA TRENTINA

NOW
- ALL the departmnets demand to be involved
- Plus other local actors
100% ENTHUSIASTIC
REACTIONS


Want to Know more? A couple of links

55


56

http://www.theodi.org/

57


http://schoolofdata.org/

58


http://schoolofdata.org/online-resources/

OGD: Part 4 - Applications

59


Apps4Italy

60


Best Application: http://parlamento17.openpolis.it/

61


Open
Bilancio

Best Idea: http://opendata.comune.fi.it/open_bilancio/

62


Open Source, Open Data, Open Hardware

http://limaio.innovacion.pe/
http://www.limaio.com/demo

63


64

http://www.mysociety.org/2007/more-travel-maps/morehousing

Johann MITTHEISZ (CIO der Stadt Wien)

Total hours to develop 38 applications:
around 2.600
City of Wien saved around
208.000 Euro

65

http://www.slideshare.net/BrigitteLutz/keynote-mittheisz-cio-stadt-wien/16

Beyond Data
(The OpenStreetMap Case)

66


OpenStreetMap
OpenStreetMap project creates and provides
geographical data, such as road maps, freely
available to anyone. Behind the establishment and
growth of the project have been restrictions on
use or availability of map information across
much of the world and the advent of inexpensive
portable satellite navigation devices.

OpenStreetMap is a free map of the
world, created by someone like you
67

~


68


http://tools.geofabrik.de/mc/?mt0=mapnik&mt1=googlemap&lon=11.12042&lat=46.07224&zoom=18

Watercolor maps

69

http://content.stamen.com/files/cartography/index_watercolor.html#18.00/46.07204/11.12097

From maps to blankets…

70

http://softcities.net

Sharing Data Globally
(the eHabitat example)

71


The Group of Earth Observation

84 GEO members and 61 Participating organizations

72

Source: http://www.slideshare.net/angeled/geoss © GEO secretariat


GEOSS Data Sharing
Principles

• Full and Open
Exchange of Data,
recognizing Relevant
International
Instruments and
National Policies
• Data and Products at
Minimum Time delay
and Minimum Cost

http://www.geoportal.org/web/guest/geo_home

73

• Free of Charge or
minimal Cost for
Research and
Education


GEOSS for biodiversity

74

http://www.eurogeoss-broker.eu/

The eHabitat Model

75

http://ehabitat-wps.jrc.ec.europa.eu/ehabitat/

OGD: Part 5 – Semantic Layer

76


Available

Structured

Linked Open
Data

Open formats
Redefenceable
Linked

The best data is
an open data
Vs.
All data must be
perfect

77


Lack of explicit semantics
The real meaning of the data was kept in the
developers mind when creating the data

78


http://goo.gl/npEHKr

Lack of explicit semantics
Can lead to things like:

79


Semantic heterogeneity
Difference in the meaning of local data

80


Available

Data Catalog
Data Catalog

Structured
Open formats
Redefenceable
Linked

81

Entity centric
Entity centric
Importing tool
Importing tool


Entity centric: Added value
Aggregated data
Accurate data, manually curated
Unique identifiers, distributed perspectives
Re-think identifiers

Semantified values
E1
name

name

Ignacio P. F.

nationality italian

born in

Paraguay

lives in

Trento

date of birth

1980

affiliation

82

Juan Pane

E2

Univ. Trento

affiliation

PF-UNA


Entities
Real world: is something that has a distinct,

separate existence, although it need not be a
material (physical) existence. Has a set of
properties, which evolve over time. Example:

Mental: personal (local) model created and

maintained by a person that references and
describes a real world entity.

Digital: capture the semantics of real world

entities, provided by people.

83


Entity based Semantic Layer:
• Address the integration problems due to

semantic heterogeneity:
• Different formats
• Different identifiers
• Implicit semantics
• Homonyms, synonyms, aliases
• Partial knowledge
• Knowledge evolution

http://www.webfoundation.org/2011/11/5-staropen-data-initiatives/

84


The semantic Layer: why?
Applications use
entities instead of raw
data
REST/HTTP
REST/HTTP

Importing
Importing
Tool
Tool

Importing
Importing
Tool
Tool

v0

85

Importing
Importing
Tool
Tool

i


i+1

Importing
steps

1.

Selection

Schema Matching

3.

Data Validation

4.

Semantic Enrichment

5.

Reconciliation

6.

Exporting

7.
86

2.

Publishing

8.

Take raw data from dati.trentino.it

Map to an EntityType

Cleanse data

Link data to entities/concepts

Update/insert entities

Export to Entitypedia

Publish to dati.trentino.it

Get insights about entities
Visualization

1. Source Selection
Import one data file at a time

87


2. Schema Matching
Select a target type of entity -> correspondences between the input
columns and the output attributes
nome

provincia

descrizione

Andalo (1047)

Provincia di
Trento

Canazei (1450)

Trento Prov.

88

funivie

lat

long

Sorge su un'ampia sella 3
prativa al centro...

654463

712857

Situato all'estremità
settentrionale della...

511504

147444

2


3. Data Validation
Applies format and structure validation and possible automatic
transformations needed to have the input data in the expected
format.

89


4. Semantic Enrichment (1/2)
Entity disambiguation : Transform text references into links to
existing entities.

90


4. Semantic Enrichment (2/2)
Natural Language Processing : Extract concepts and entity
references from free-text.

91


5. Reconciliation
Run Identity Management Algorithms to identify each row as a new
or existing entity.
Result
•No Match
•Match
•Multiple
Matches

Action:
•Use ID
•New ID
•Ignore Row

92


6. Exporting
At this point:
We know what to export.
All values for target attributes conform to the expected format.
All text has been semantified (NLP).
All textual references to entities are converted to links
Each row has an identifier

v0
93

i


i+1

7. Publishing
Put back the semantified entities into CKAN so that the
entities can be Open Data and can be found in the same
catalog as the original data.
 Developers and find the data files of the cleaned,
aggregated entities
 But can also interact with the entities via the Entitypedia
APIs

8. Visualization
Search and Navigation
94


Our Goal
TN

UK

BE

95

ES


BEYOND

96

http://www.youtube.com/watch?v=Bq_ZWl1ZXA0

Thanks to all the Open Data in Trentino Team and in particular to:
Juan Pane, Maurizio Napolitano, Marco Combetto, Moaz Reyad and Luca Paolazzi

97


Open Data Trentino presented at the European Commission (JRC)

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (13)

Similar a Open Data Trentino presented at the European Commission (JRC)

Similar a Open Data Trentino presented at the European Commission (JRC) (20)

Más de European Commission, Joint Research Centre

Más de European Commission, Joint Research Centre (12)

Último

Último (20)

Open Data Trentino presented at the European Commission (JRC)

Notas del editor