My PhD thesis presentation

1. Organization is Sharing:
From eScience to
Personal Information Management
Rodrigo Dias Arruda Senra
Advisor: Profa Dra. Claudia Bauzer Medeiros
Defesa de Tese de Doutorado em Ciência da Computação
Universidade Estadual de Campinas
Instituto de Computação
Campinas 2012-12-10

2. Outline
• Motivation
• Objectives
• Contributions
• Results
2
• SciFrame
• Database Descriptors
• Organographs
{

3. Motivation

4. 4
Study the relation
Heterogeneity ↔ Organization ↔ Sharing

5. 5
NDVI Profile Generation
PostGIS
Filesystem
Postgres
WebMAPS

6. 5
NDVI Profile Generation
Geometries (IBGE)
Spectral Images(NASA)
Crops(Min.Agr)
PostGIS
Filesystem
Postgres
HTTP
FTP
WebMAPS

7. 5
NDVI Profile Generation
Geometries (IBGE)
Spectral Images(NASA)
Crops(Min.Agr)
PostGIS
Filesystem
Postgres
HTTP
FTP
WebMAPS

8. 5
NDVI Profile Generation
Geometries (IBGE)
Spectral Images(NASA)
Crops(Min.Agr)
PostGIS
Filesystem
Postgres
HTTP
FTP
WebMAPS

9. 5
NDVI Profile Generation
Geometries (IBGE)
Spectral Images(NASA)
Crops(Min.Agr)
PostGIS
Filesystem
Postgres
HTTP
FTP
WebMAPS

10. 5
NDVI Profile Generation
Geometries (IBGE)
Spectral Images(NASA)
Crops(Min.Agr)
PostGIS
Filesystem
Postgres
HTTP
FTP
WebMAPS

11. 5
NDVI Profile Generation
Geometries (IBGE)
Spectral Images(NASA)
Crops(Min.Agr)
PostGIS
Filesystem
Postgres
HTML, Microformats, 2D Plots
HTTP
FTP
HTTP
WebMAPS

13. Objectives

14. 8

15. • describe and compare eScience systems
• match Applications needs with DBMS capabilities
• manage digital content hierarchies
8

16. Motivation
Objectives
• Contributions
• Results
9
• SciFrame
• Database Descriptors
• Organographs
{

17. SciFrame

18. 11
SciFrame
The Scientific Digital Data Processing Framework is a
conceptual framework that describes systems or
processes involving digital data manipulation.

19. Interfacing
Acquisition
Publication
(discovery - extraction - transference )
Information Management Data Management

20. SciFrame
Interfacing
Acquisition
Publication
(discovery - extraction - transference )
Information Management Data Management

21. SciFrame
Interfacing
Acquisition
Publication
(discovery - extraction - transference )
Information Management Data Management

22. SciFrame
Interfacing
Acquisition
Publication
(discovery - extraction - transference )
Information Management Data Management

23. SciFrame
Interfacing
Acquisition
Publication
(discovery - extraction - transference )
Information Management Data Management
Data Management
Manipulation
Create
Retrieve
Update
Delete
Index
Storage

24. SciFrame
Interfacing
Acquisition
Publication
(discovery - extraction - transference )
Information Management Data Management
Data Management
Manipulation
Create
Retrieve
Update
Delete
Index
Storage
Information Management

25. SciFrame
Interfacing
Acquisition
Publication
(discovery - extraction - transference )
Information Management Data Management
Information Management

26. SciFrame
Interfacing
Acquisition
Discovery
Extraction
Transference
Publication
Data Management
Storage
Manipulation
Information Management
Description
Transformation
Fusing
Filtering

27. WebMaps
Interfacing
Acquisition
Discovery Geometries (IBGE), Raster(NASA), Crops(Min.Agr)
Extraction ad hoc extractor scripts (paparazzi)
Transference FTP and HTTP
Publication HTML, Microformats, 2D Plots
Data Management
Storage Geometries(PostGIS), Raster(Files), Crops(Postgres)
Manipulation Geometries(CRDI), Raster(CRD), Crops(CRUDI)
Information Management
Description Geometries(SHP,WKT), Raster(HDF,GeoTIFF)
Transformation
Fusing NDVI Time Series
Filtering Cloud and noise removal (HANTS)

28. Research Problems
Interfacing
Acquisition
Discovery data scattered, many providers, search engines ?
Extraction feasibility, preserve provenance, lack of semantics
Transference availability, voluminous data, bandwidth, protocol
Publication lack of intention, access control, traceability
Data Management
Storage scalability, distribution, consistency, preservation
Manipulation multimedia, impedance mismatch
Information Management
Description implicit x explicit, semantic web, social, trust, privacy
Transformation
information lost: conceptual > logical > physical
multi-modality
handle uncertain and incomplete data

29. Technologies
Interfacing
Acquisition
Discovery DAS Registry, BIOCatalogue, SciScope
Extraction Scrappers,Wrappers, PiggyBank, Operator
Transference Streaming, P2P, OpenDAP
Publication SOA x ROA, Microformats x RDFa
Data Management
Storage Scientific Datasets, XML, Cloud Computing
Manipulation SQL extensions, ORMs, LINQ
Information Management
Description In Loco Semantics
Transformation
Array Algebra (RASDAMAN)
Topological Operators (GIS)
Proximity Search and Report Language (ISIS)

30. Interfacing
Acquisition
Publication
(discovery - extraction - transference )
Information ManagementData Management

31. Data Management

32. Data Management

33. Data Management
✓enforce loose coupling between Apps and DBMS
✓DBMS product/vendor independence
✓seamless cross-database migration
✓capability verification, validation and negotiation
✓support Apps and DBMS in the cloud!

34. Database
Descriptors

35. DBMS
Descriptors
Feature descriptor
Desiderata descriptor
specifies what a client application needs
12
App

36. DBMS
Descriptors
Feature descriptor
Desiderata descriptor
specifies what a client application needs
specifies what a DBMS provides
12
App

37. Architecture
15
Web
DMS X
DMS Y
DMS Z

38. Architecture
15
Web
DMS X
DMS Y
DMS Z
Descriptor
Registry
descriptor X
descriptorY

39. Architecture
15
Web
DMS X
DMS Y
DMS Z
Descriptor
Registry
Descriptor
RegistryDescriptor
RegistryDescriptor
Registry
descriptor X
descriptorY

40. Architecture
15
Web
DMS X
DMS Y
DMS Z
Descriptor
Registry
Descriptor
RegistryDescriptor
RegistryDescriptor
Registry
App
descriptor X
descriptorY

41. Architecture
15
Web
DMS X
DMS Y
DMS Z
Descriptor
Registry
Negotiator
Descriptor
RegistryDescriptor
RegistryDescriptor
Registry
App
descriptor X
descriptorY

42. Architecture
15
Web
DMS X
DMS Y
DMS Z
Descriptor
Registry
Negotiator
Descriptor
RegistryDescriptor
RegistryDescriptor
Registry
App
descriptor X
descriptorY

43. Architecture
15
Web
DMS X
DMS Y
DMS Z
Descriptor
Registry
Negotiator
Descriptor
RegistryDescriptor
RegistryDescriptor
Registry
App
descriptor X
descriptorY
binding

44. DBD Structure
13 * http://dublincore.org/documents/dces/
App
DBMS

45. @prefix : <http://www.lis.ic.unicamp.br/purl/DBD/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
:Cmbm a foaf:Person ;
foaf:name “Claudia Bauzer Medeiros” .
:DBD1 dc:identifier “DBD1” ;
dc:type “Feature DBD” ;
dc:format “text/turtle” ;
dc:title “Sample Feature Descriptor” ;
dc:description “Hypothetical Feature DBD in RDF/Turtle” ;
dc:creator :Cmbm ;
dc:date “2009-12-18” ;
dc:language “EN” ;
:isolation :READ_COMMITED ;
:versioning “unsupported” ;
:storage “RDF Triples” ;
:DML [ a rdf:Bag ;
rdf:_1 RDQL ;
rdf:_2 SPARQL ;
] .
Feature Descriptor

46. @prefix : <http://www.lis.ic.unicamp.br/purl/DBD/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
:Rodsenra a foaf:Person ;
foaf:name “Rodrigo Dias Arruda Senra” .
:DBD2 dc:identifier “DBD2” ;
dc:type “Desiderata DBD” ;
dc:format “text/turtle” ;
dc:title “Sample Desiderata Descriptor” ;
dc:description “Desiderata DBD for hypothetical App” ;
dc:creator :Rodsenra;
dc:date “2010-01-05” ;
dc:language “EN” ;
:isolation :READ_COMMITED ;
:concurrency “Two phase lock” ;
:storage “RDF Triples” ;
:DML SPARQL .
Desiderata Descriptor

47. Understanding Hierarchies...
SciFrame DBDs

48. Organographs

49. 27

50. 28
Which of the following sets better
accommodate the object above ?

51. 29
Red ? Triangles ? Metric Related ?

52. Problems
30
1. Single Category versus Multi-faceted Content
2. Manually-defined categories
3.Criteria is not explicit
4.Static Membership Relation
5. Organization is not reusable

53. 31

54. 31
Organograph
... artifact to make explicit how to organize
information in the context of a particular task.

55. Organograph
32
Hout = forg(Hin)
vcnt
eagg
ecnt
H(V,E)
vagg
vagg

56. Organograph
32
Hout = forg(Hin)
forg:
• navigation (crawler/iterador)
• feature extraction
• FHil(vagg,vagg): hierarchical structuring
• FCat(vagg,vcnt): categorization
URL
HoutHin
URL
vcnt
eagg
ecnt
H(V,E)
vagg
vagg

57. NLP
Author
ML
Content
Domain
Expert Roles
Ontologies
ClassifiersInformation
Extraction
Algorithms
Similarity
forg
Vizualization
Strategies
33
Iterators
Data
Container UX
Organograph Composition
Task !

58. NLP
Author
ML
Content
Domain
Expert Roles
Ontologies
ClassifiersInformation
Extraction
Algorithms
Similarity
forg
Vizualization
Strategies
33
Iterators
Data
Container UX
Organograph Composition
Task !
• patterns
• dictionaries
• rules
• probabilities
• templates/wrappers

59. NLP
Author
ML
Content
Domain
Expert Roles
Ontologies
ClassifiersInformation
Extraction
Algorithms
Similarity
forg
Vizualization
Strategies
33
Iterators
Data
Container UX
Organograph Composition
Task !
• matching
• dice
• jaccard
• overlap
• cosine

60. NLP
Author
ML
Content
Domain
Expert Roles
Ontologies
ClassifiersInformation
Extraction
Algorithms
Similarity
forg
Vizualization
Strategies
33
Iterators
Data
Container UX
Organograph Composition
Task !
• FOAF
• Dbpedia
• Schema.org
• Freebase
• MusicBrainz
• Geonames

61. NLP
Author
ML
Content
Domain
Expert Roles
Ontologies
ClassifiersInformation
Extraction
Algorithms
Similarity
forg
Vizualization
Strategies
33
Iterators
Data
Container UX
Organograph Composition
Task !
• Naive Bayes
• SVM
• Nearest Neighbors
• LDA
• LSI

62. NLP
Author
ML
Content
Domain
Expert Roles
Ontologies
ClassifiersInformation
Extraction
Algorithms
Similarity
forg
Vizualization
Strategies
33
Iterators
Data
Container UX
Organograph Composition
Task !
• Filesystem
• Gmail
• Evernote
• Delicious
• Dropbox
DBDs!

63. NLP
Author
ML
Content
Domain
Expert Roles
Ontologies
ClassifiersInformation
Extraction
Algorithms
Similarity
forg
Vizualization
Strategies
33
Iterators
Data
Container UX
Organograph Composition
Task !
• Fuse, Dokan
• Infoviz
• D3

64. Metodology
34
collection

65. Metodology
34
collection
organize

66. Metodology
34
collection
organize
evaluate

67. Metodology
34
collection
organize
evaluate
reorganize

68. Metodology
34
collection
organize
evaluate
reorganize
share

69. Evaluating Hierarchies
35

70. Evaluating Hierarchies
35
too much content

71. Evaluating Hierarchies
35
too much content
duplicated or misplaced

72. Evaluating Hierarchies
35
too much content
too many
aggregators
duplicated or misplaced

73. Evaluating Hierarchies
35
too much content
too many
aggregators
duplicated or misplaced
too deep

74. Reorganizing Hierarchies
36
Alice
Bob
2011
2008
2011
Author
Publication Date
paper 1
paper 2
paper 3

75. Reorganizing Hierarchies
36
Alice
Bob
2011
2008
2011
Author
Publication Date Author
Publication Date
paper 1
paper 2
paper 3

76. Reorganizing Hierarchies
36
Alice
Bob
2011
2008
2011 Alice
Bob
2008
2011
Alice
Author
Publication Date Author
Publication Date
Task is important!
paper 1
paper 2
paper 3

77. Reuse Organization
37

78. Reuse Organization
37

79. Reuse Organization
37
Hacm Vcnt
mine

80. Hin
Hout
Internal
Indexes
Pre-processing
Feature
Extraction
Transformation Workflow
Organograph Execution
FCat()
FHil()
Visualization

81. Hin
Hout
Internal
Indexes
Pre-processing
Feature
Extraction
Transformation Workflow
Organograph Execution
FCat()
FHil()
Visualization

82. Hin
Hout
Internal
Indexes
Pre-processing
Feature
Extraction
Transformation Workflow
Organograph Execution
FCat()
FHil()
Visualization

83. Hin
Hout
Internal
Indexes
Pre-processing
Feature
Extraction
Transformation Workflow
Organograph Execution
FCat()
FHil()
Visualization

85. @organograph
def forg_ccs98(self, input):
self.id = new_uuid() #‘ff7d8e21-4226-11e2-b2f1-109add6b426c’
self.description = ‘docs by ACM CCS98’
ccs98 = acm_extract(‘http://www.acm.org/about/class/1998/ccs98.xml’)
trainset = []
for category,words in nlp_clean_titles(ccs98.Vcnt.paths):
for w in words:
trainset.append((make_feature(w), category))
classifier = NaiveBayes(trainset)
self.Ecnt = classifier.classify(input) # FCat
self.Eagg = ccs98.Eagg.Level[:1] # FHil

86. @organograph
def forg_ccs98(self, input):
self.id = new_uuid() #‘ff7d8e21-4226-11e2-b2f1-109add6b426c’
self.description = ‘docs by ACM CCS98’
ccs98 = acm_extract(‘http://www.acm.org/about/class/1998/ccs98.xml’)
trainset = []
for category,words in nlp_clean_titles(ccs98.Vcnt.paths):
for w in words:
trainset.append((make_feature(w), category))
classifier = NaiveBayes(trainset)
self.Ecnt = classifier.classify(input) # FCat
self.Eagg = ccs98.Eagg.Level[:1] # FHil
input = collection(‘file:///some/local/dir/docs’)
output = forg_ccs98(input)
publish(output, ‘rodsenra@dropbox:/output’)
organicer.render(output, organicer.views.HYPERBOLIC_TREE)

87. forg_ccs_98
Interfacing
Acquisition
Discovery ACM CCS98, Hin
Extraction pdf2txt,pdfbox, pypdf; NLTK (tokenizer)
Transference HTTP, WebDAV, NFS, SMB
Publication Hout :HTML+CSS, JS(Infoviz,D3); Dropbox
Data Management
Storage NoSQL DB (Mongo, Neo4J)
Manipulation Indexes (CRDI)
Information Management
Description SKOS, GraphML, JSON
Transformation
Mining NaiveBayes
Filtering Vcnt(unconverted pdfs); Vagg (empty or ambiguous)

88. Related Work

89. Related Work (SciFrame)
• CLRC scientific metadata model
B. Matthews and S. Sufi
The CLRC Scientific Metadata Model, version 1, DL TR 02001, CLRC
2001
• myGrid Information Model
Sharman, Nick, et al.
"The myGrid information model." UK e-Science programme All Hands Conference.
2004.

90. Related Work (DBDs)
Madnick and Wang.
EvolutionTowards Strategic Applications Of DatabasesThrough
Composite Information Systems.
Journal of Management Information Systems 5(2):5-22 1988
“In order to: separate data from the application processing, it is necessary to employ a
process descriptor and a database descriptor.
The process descriptor describes the name, the input/output data requirement, and other
resource requirements of the processing components.
The database descriptor contains information about the data (e.g., data model, schema,
access rights) in the database, similar to data dictionaries.
These two descriptors can be used by the execution environment to coordinate the
interaction between the processing component and the database.”

91. Related Work (Organographs)
•Topic Modeling
LSA, LDA, Hierarchical Bayesian
Blei 201; Blei, Ng, & Jordan, 2003; Griffiths & Steyvers, 2002; 2003; 2004; Hofmann, 1999;
2001
• Personal Information Management
CALO, UMEA, X-COSIM, Haystack, UpLib, Iris
Zimmermann 2005; Arndt 2007; Lansdale 1988; Kaptelinin 2003; Janssen & Popat 2003;
Karger et al 2003
• Semantic Desktop
Nepomuk, SEMSOC
Giannakidou et al 2008; Groza et al 2007
• Personal Digital Libraries
Zotero, Mendeley, Papers

92. Results

93. Contributions
• SciFrame
• Database Descriptors (DBDs)
• Organographs
• Software tools & algorithms:
WebMAPS, Paparazzi & Organicer
46

94. Publications
submitted to
JODS
Evaluating, Reorganizing and Sharing Digital Information Hierarchies.
Rodrigo D. A. Senra, Claudia B. Medeiros.
Journal on Data Semantics (submetido em 2012-10-25)
2011
Organographs - Multi-faceted Hierarchical Categorization of Web Documents.
Rodrigo D. A. Senra, Claudia B. Medeiros.
Proceeding of the 7th International Conference on Web Information Systems and Technologies - WEBIST: 583-588
2010
Database Descriptors: Laying the Path to Commodity Web Data Services.
Rodrigo D. A. Senra, Claudia B. Medeiros.
Proceedings of Engineering of Computer-Based Systems (ECBS): 386-392
2009
SciFrame: a conceptual framework to describe data sharing in eScience.
Rodrigo D. A. Senra, Claudia B. Medeiros.
Proceedings of the III Brazilian eScience workshop (XXIV SBBD)
2009
A standards-based framework to foster geospatial data and process interoperability.
Gilberto Z. Pastorello Jr., Rodrigo D. A. Senra, Claudia B. Medeiros.
Journal of the Brazilian Computer Society 15(1): 13-25
2008
Bridging the gap between geospatial resource providers and model developers.
Gilberto Z. Pastorello Jr., Rodrigo D. A. Senra, Claudia B. Medeiros.
Proceedings of the 16th International Conference on Advances in Geographic Information Systems - ACM SIGSPATIAL
2007
O projeto WebMAPS: desafios e resultados.
Carla G. N. Macário, Claudia B. Medeiros, Rodrigo D. A. Senra.
Proceedings of 9th Brazilian Symposium on Geoinformatics - GeoInfo: 239-250
47

95. Publications
submitted to
JODS
Evaluating, Reorganizing and Sharing Digital Information Hierarchies.
Rodrigo D. A. Senra, Claudia B. Medeiros.
Journal on Data Semantics (submetido em 2012-10-25)
2011
Organographs - Multi-faceted Hierarchical Categorization of Web Documents.
Rodrigo D. A. Senra, Claudia B. Medeiros.
Proceeding of the 7th International Conference on Web Information Systems and Technologies - WEBIST: 583-588
2010
Database Descriptors: Laying the Path to Commodity Web Data Services.
Rodrigo D. A. Senra, Claudia B. Medeiros.
Proceedings of Engineering of Computer-Based Systems (ECBS): 386-392
2009
SciFrame: a conceptual framework to describe data sharing in eScience.
Rodrigo D. A. Senra, Claudia B. Medeiros.
Proceedings of the III Brazilian eScience workshop (XXIV SBBD)
2009
A standards-based framework to foster geospatial data and process interoperability.
Gilberto Z. Pastorello Jr., Rodrigo D. A. Senra, Claudia B. Medeiros.
Journal of the Brazilian Computer Society 15(1): 13-25
2008
Bridging the gap between geospatial resource providers and model developers.
Gilberto Z. Pastorello Jr., Rodrigo D. A. Senra, Claudia B. Medeiros.
Proceedings of the 16th International Conference on Advances in Geographic Information Systems - ACM SIGSPATIAL
2007
O projeto WebMAPS: desafios e resultados.
Carla G. N. Macário, Claudia B. Medeiros, Rodrigo D. A. Senra.
Proceedings of 9th Brazilian Symposium on Geoinformatics - GeoInfo: 239-250
47
SciFrame
WebMaps
DBDs
Organographs

96. Extensions
Theoretical Practical
SciFrame • formalize design pattern
• enhance the operations vocabulary
• online catalog of eScience systems
• describe as ontology (RDF)
Database
Descriptors
• analyse negotiation frameworks
• expand DBDs expressivity
• explore ranking algorithms
• catalog of concrete DBDs
• adapt Organicer to use DBDs
• experiment with dynamic negotiation
Organographs • model with CategoryTheory
• explore DSLs to describe forg
• support non-textual media (eg.:img)
• expand component palette
48

97. Agradecimentos
• Laboratório de Sistemas de Informação (IC-Unicamp)
http://www.lis.ic.unicamp.br
• Brazilian Institute for Web Science Research
http://webscience.org.br
• Fapesp - CNPQ - CAPES
49

98. Rodrigo Dias Arruda Senra
http://rodrigo.senra.nom.br
rsenra@acm.org

99. Rodrigo Dias Arruda Senra
http://rodrigo.senra.nom.br
rsenra@acm.org
Thank you.
Agradeço sua atenção.

100. Support Material

101. Hierarquia
de Origem

102. Hierarquia
de Origem
Pre-processamento
BeautifulSoup
pyPdf

103. Hierarquia
de Origem
Extração
NLTK
Pre-processamento
BeautifulSoup
pyPdf

104. Hierarquia
de Origem
Extração
NLTK
Pre-processamento
BeautifulSoup
pyPdf
Índice de
Facetas
pymongo

105. Hierarquia
de Origem
Workflow de Transformação
Extração
NLTK
Pre-processamento
BeautifulSoup
pyPdf
Índice de
Facetas
pymongo

106. Hierarquia
de Origem
Workflow de Transformação
Extração
NLTK
Pre-processamento
BeautifulSoup
pyPdf
Índice de
Facetas
pymongo
networkx gensim
numpy scikit-learn

107. Hierarquia
de Origem
Workflow de Transformação
Hierarquia
Resultante
Visualização
Extração
NLTK
Pre-processamento
BeautifulSoup
pyPdf
Índice de
Facetas
pymongo
networkx gensim
numpy scikit-learn

108. Hierarquia
de Origem
Workflow de Transformação
Hierarquia
Resultante
Visualização
Extração
NLTK
Pre-processamento
BeautifulSoup
pyPdf
Índice de
Facetas
pymongo
networkx gensim
numpy scikit-learn
matplotlib
ObsPy
InfoViz.js
D3.js

109. Hierarquia
de Origem
Workflow de Transformação
Hierarquia
Resultante
Visualização
Navegação da
Hierarquia
Iterador
Extração
NLTK
Pre-processamento
BeautifulSoup
pyPdf
Índice de
Facetas
pymongo
networkx gensim
numpy scikit-learn
matplotlib
ObsPy
InfoViz.js
D3.js

110. Hierarquia
de Origem
Workflow de Transformação
Hierarquia
Resultante
Visualização
Navegação da
Hierarquia
Iterador
Extração
NLTK
Pre-processamento
BeautifulSoup
pyPdf
Índice de
Facetas
pymongo
networkx gensim
numpy scikit-learn
matplotlib
ObsPy
InfoViz.js
D3.js
os.walk
pydelicious
evernote

111. Hin Hout
Internal
Indexes
Pre-processing
Feature
Extraction
Transformation Workflow
FCat()
FHil()
Visualization

112. NLP
Author
ML
Content
Domain
Expert Roles
Ontologies
ClassifiersInformation
Extraction
Algorithms
Similarity
forg
Vizualization
Strategies
54
Iterators
Data
Container UX
Task !

113. 55
forg:
• navigation (crawler/iterador)
• feature extraction
• FHil(vagg,vagg): hierarchical structuring
• FCat(vagg,vcnt): categorization
Hin:
URL
Hout:
URL

114. <?xml version="1.0"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:dbd="http://www.lis.ic.unicamp.br/purl/DBD">
<rdf:Description rdf:about="http://www.lis.ic.unicamp.br/purl/DBD/DBD1">
<!-- metadata -->
<dc:creator>Claudia Bauzer Medeiros</dc:creator>
<dc:description>Hypothetical DBD for an RDF DBMS</dc:description>
<dc:identifier>DBD1</dc:identifier>
<dc:format>application/rdf+xml</dc:format>
<dc:type><rdf:Description> <dbd:Type>Feature DBD</dbd:Type></rdf:Description> </dc:type>
<dc:title>Descriptor of an RDF DBMS</dc:title>
<dc:date>2009-12-18</dc:date>
<dc:language>EN</dc:language>
<!-- dimensions and values -->
<dbd:concurrency>Two phase lock</dbd:concurrency>
<dbd:versioning>unsupported</dbd:versioning>
<dbd:storage>RDF triples</dbd:storage>
<dbd:DML> <rdf:Bag><rdf:li>RDQL</rdf:li><rdf:li>SPARQL</rdf:li> </rdf:Bag>
</dbd:DML>
</rdf:Description>
</rdf:RDF>

115. <?xml version="1.0"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:dbd="http://www.lis.ic.unicamp.br/purl/DBD">
<rdf:Description rdf:about="http://www.lis.ic.unicamp.br/purl/DBD/DBD1">
<!-- metadata -->
<dc:creator>Rodrigo Dias Arruda Senra</dc:creator>
<dc:description>Desiderata DBD for an hypothetical application</dc:description>
<dc:identifier>DBD2</dc:identifier>
<dc:format>application/rdf+xml</dc:format>
<dc:type><rdf:Description> <dbd:Type>Desiderata DBD</dbd:Type>
</rdf:Description> </dc:type>
<dc:title>Desiderata descriptor of an hypothetical application</dc:title>
<dc:date>2010-01-05</dc:date> <dc:language>EN</dc:language>
<!-- dimensions and values -->
<dbd:concurrency>Two phase lock</dbd:concurrency>
<dbd:storage>RDF triple store</dbd:storage>
<dbd:DML>RDQL</dbd:DML>
</rdf:Description>
</rdf:RDF>

116. 58
NDVI Profiles

117. Data Management
Manipulation
Create
Retrieve
Update
Delete
Index
Storage

118. Information Management
Transformations
‣Browsing
‣Iterating
‣Searching
‣ Augmenting
‣Mining
‣Description
‣Annotation
‣ Schematization ‣Summarizing
‣Structuring
‣Sorting
‣Merging
‣ Decreasing
‣ Filtering
‣ Fusing

119. Example
61

120. Example
62
Input Collection
Task: info extraction
Task: transformation
Task: visualization

121. 63
WebMAPS: DataFlow
Correio
FTP
MODIS Reprojection Tool
Imagens
Recorte
da região
Geometria
(IBGE)‫‏‬

122. 64
NDVI

123. Related Work
9
• embedded
• n-tier client/server (including web services)
• mediators
Approaches to App-to-DMS binding
Information Integration [1]
Process
• Understanding
• Standardization
• Specification
• Execution [1] Beauty and the Beast: The Theory and Practice of
Information Integration
Laura Haas
Mechanism
• Materialization
• Federation
• Indexing

124. Related Work
9
• embedded
• n-tier client/server (including web services)
• mediators
Descriptors are orthogonal to all of these!
Approaches to App-to-DMS binding
Information Integration [1]
Process
• Understanding
• Standardization
• Specification
• Execution [1] Beauty and the Beast: The Theory and Practice of
Information Integration
Laura Haas
Mechanism
• Materialization
• Federation
• Indexing

125. 66
Extração dos Dados Sensorias
dataset = gdal.Open(raster_file, GA_ReadOnly )‫‏‬
# Obtenção dos coeficientes para funções afins de mapeamento de coordenadas
gt = dataset.GetGeoTransform()‫‏‬
# Obtenção da banda de dados de interesse
band = dataset.GetRasterBand(1)‫‏‬
# Identificação do padrão de codificação dos dados.
# No caso do arquivo TIF os dados são bytes sem sinal ('Byte')‫‏‬
data_type = gdal.GetDataTypeName(band.DataType)
# Obtenção das dimensões da imagem
width, height = band.XSize, band.YSize
# Conversão do MBR do sistema de coordenadas lat/long para linha/coluna
# Xgeo = GT(0) + Xpixel*GT(1) + Yline*GT(2)‫‏‬
# Ygeo = GT(3) + Xpixel*GT(4) + Yline*GT(5)
ul_pixel, lr_pixel = g2p(gt,*ul_geo), g2p(gt,*lr_geo)‫‏‬

126. 67
WebMAPS

127. Case Study:WebMaps

128. Case Study:WebMaps

129. 69
Extração dos Dados
def raster2array(ul_pixel, lr_pixel, dtype='B'):
"""Using ul_pixel and lr_pixel it generates a numpy array
with the extracted interest region from the raster file
"""
col_size = lr_pixel[1]-ul_pixel[1]+1
row_size = lr_pixel[0]-ul_pixel[0]+1
scanline = band.ReadRaster(ul_pixel[1], ul_pixel[0],
col_size, row_size)‫‏‬
num_pixels = col_size*row_size
roi = numpy.array(struct.unpack(dtype*num_pixels, scanline))‫‏‬
roi.shape = (row_size, col_size)‫‏‬
return roi
# Read data from raster file into a numpy array
# defining a region of interest matrix
roi = raster2array(ul_pixel, lr_pixel)‫‏‬

130. 70
Extração da Geometria
shp = ogr.Open(filepath)‫‏‬
# Layer correspondente ao Estado de São paulo
layer = vf.shp.GetLayerByName('35mu500gc')
# Feature correspondente ao município de Campinas
feature = layer.GetFeature(501)
# Extração dos pontos de controle do perímetro
geometry = feature.GetGeometryRef()‫‏‬
poly = geometry.GetGeometryRef(0)‫‏‬
centroid = geometry.Centroid()‫‏‬
centroid_geo = centroid.GetX(), centroid.GetY()‫‏‬
# Definição do Retângulo Envoltório Mínimo (MBR)‫‏‬
lg_left, lg_right, lt_bot, lt_up = poly.GetEnvelope()‫‏‬
ul_geo, lr_geo = (lg_left, lt_up), (lg_right, lt_bot)‫‏‬

131. 71
Operações Espaciais

132. Organicer
72

133. Organicer
72

134. Organicer
72

135. Organicer
72

136. Organicer
72