Ontopia tutorial

Ontopia Tutorial TMRA 2010-09-29 Lars Marius Garshol & Geir Ove Grønmo

Agenda About you who are you? About Ontopia The product The future Participating in the project

Brief history 1999-2000 private hobby project for Geir Ove 2000-2009 commercial software sold by Ontopia AS lots of international customers in diverse fields 2009- open source project

The project Open source hosted at Google Code Contributors Lars Marius Garshol, Bouvet Geir Ove Grønmo, Bouvet Thomas Neidhart, SpaceApps Lars Heuer, Semagia Hannes Niederhausen, TMLab Stig Lau, Bouvet Baard H. Rehn-Johansen, Bouvet Peter-Paul Kruijssen, Morpheus Quintin Siebers, Morpheus Matthias Fischer, HTW Berlin

Recent work Ontopia/Liferay integration Matthias Fischer & LMG Various fixes and optimizations everyone Tropics (RESTful web service interface) SpaceApps Porting build system to Maven2 Morpheus

Architecture and modules Product overview

The big picture Auto-class. A.N.other A.N.other Other CMSs A.N.other A.N.other DB2TM Portlet support OKP TMSync Engine CMSintegration Data integration Escenic Taxon.import Ontopoly Web service

The engine Core API TMAPI 2.0 support Import/export RDF conversion TMSync Fulltext search Event API tolog query language tolog update language Engine

The backends In-memory no persistent storage thread-safe no setup RDBMS transactions persistent thread-safe uses caching clustering Remote uses web service read-only unofficial Engine Memory RDBMS Remote

DB2TM Upconversion to TMs from RDBMS via JDBC or from CSV Uses XML mapping can call out to Java Supports sync either full rescan or change table TMRAP Nav DB2TM Classify Engine Memory RDBMS Remote

TMRAP Web service interface via SOAP via plain HTTP Requests get-topic get-topic-page get-tolog delete-topic ... TMRAP Nav DB2TM Classify Engine Memory RDBMS Remote

Navigator framework Servlet-based API manage topic maps load/scan/delete/create JSP tag library XSLT-like based on tolog JSTL integration TMRAP Nav DB2TM Classify Engine Memory RDBMS Remote

Automated classification Undocumented experimental Extracts text autodetects format Word, PDF, XML, HTML Processes text detects language stemming, stop-words Extracts keywords ranked by importance uses existing topics supports compound terms TMRAP Nav DB2TM Classify Engine Memory RDBMS Remote

Vizigator Viz Ontopoly Graphical visualization VizDesktop Swing app to configure filter/style/... Vizlet Java applet for web uses configuration loads via TMRAP uses “Remote” backend TMRAP Nav DB2TM Classify Engine Memory RDBMS Remote

Ontopoly Viz Ontopoly Generic editor web-based, AJAX meta-ontology in TM Ontology designer create types and fields control user interface build views incremental dev Instance editor guided by ontology TMRAP Nav DB2TM Classify Engine Memory RDBMS Remote

Typical deployment Viewing application Engine Users DB Backend Ontopoly Frameworks Editors DB TMRAP DB2TM HTTP DB External application Application server

Core APIs net.ontopia.topicmaps.core.* Fairly direct mapping from TMDM TopicIF AssociationIF TopicMapIF ... Set/get methods reflect TMDM properties

TopicIF Interface, not a class getTopicNames() addTopicName(TopicNameIF) removeTopicName(TopicNameIF) getOccurrences() + add + remove getSubjectIdentifiers() + add + remove getItemIdentifiers() + add + remove getSubjectLocators() + add + remove getRoles() getRolesByType(TopicIF)

Core interfaces TopicMapStoreIF TopicMapIF TopicIF AssociationIF TopicNameIF OccurrenceIF AssociationRoleIF VariantNameIF

How to get a TopicMapIF Create one directly new net...impl.basic.InMemoryTopicMapStore() Load one from file using an importer (next slide) Connect to an RDBMS covered later Use a topic map repository covered later

TopicMapReaderIF import net.ontopia.topicmaps.core.TopicMapIF; import net.ontopia.topicmaps.core.TopicMapReaderIF; import net.ontopia.topicmaps.utils.ImportExportUtils; public class TopicCounter { public static void main(String[] argv) throws Exception { TopicMapReaderIF reader = ImportExportUtils.getReader(argv[0]); TopicMapIF tm = reader.read(); System.out.println("TM contains " + tm.getTopics().size() + " topics"); } } [larsga@c716c5ac1 tmp]$ java TopicCounter ~/data/bilder/privat/metadata.xtm TM contains 17035 topics [larsga@c716c5ac1 tmp]$

The utility classes A set of classes outside the core interfaces that perform common tasks a number of these utilities are obsolete now that tolog is here They are all built on top of the core interfaces Some important utilities ImportExportUtils creates readers and writers MergeUtils merges topics and topic maps PSI contains important PSIs DeletionUtils cascading delete of topics DuplicateSuppressionUtils removes duplicates TopicStringifiers find names for topics

Topic Maps repository Uses a set of topic maps sources to build a set of topics maps topic maps can be looked up by ID Many kinds of sources scan directory for files matching pattern download from URL connect to RDBMS ... Configurable using an XML file tm-sources.xml Used by Navigator Framework

Event API Allows clients to receive notification of changes Must implement TopicMapListenerIF static class TestListener extends AbstractTopicMapListener { public void objectAdded(TMObjectIF snapshot) { System.out.println("Topic added: " + snapshot.getObjectId()); } public void objectModified(TMObjectIF snapshot) { System.out.println("Topic modified: " + snapshot.getObjectId()); } public void objectRemoved(TMObjectIF snapshot) { System.out.println("Topic removed: " + snapshot.getObjectId()); } }

Using the API // register to listen for events TestListener listener = new TestListener(); TopicMapEvents.addTopicListener(ref, listener); // get the store through the reference so the listener is registered ref.createStore(false); // let's add a topic System.out.println("Off we go"); TopicMapBuilderIF builder = tm.getBuilder(); TopicIF newbie = builder.makeTopic(tm); System.out.println("Let's name this topic"); builder.makeTopicName(newbie, "Newbie topic"); // then let's remove it System.out.println("And now, the exit"); DeletionUtils.remove(newbie); System.out.println("Goodbye, short-lived topic"); [larsga@dhcp-98 tmp]$ java EventTest bkclean.xtm Off we go Topic added: 3409 Let's name this topic Topic modified: 3409 Topic modified: 3409 And now, the exit Topic removed: 3409 Goodbye, short-lived topic

For more information See the Engine Developer's Guide http://www.ontopia.net/doc/current/doc/engine/devguide.html

Persistence & transactions RDBMS backend

RDBMS backend Stores Topic Maps in an RDBMS generic schema access via JDBC Provides full ACID transactions concurrency ... Supports several databases Oracle, MySQL, PostgreSQL, MS SQL Server, hsql Clustering support

Core API implementation Implements same API as in-memory impl theoretically, a switch requires only config change Lazy loading of objects objects loaded from DB as needed Considerable internal caching for performance reasons Separate objects for separate transactions in order to provide isolation Shared cache between transactions

Configuration A Java property file Specifies database type JDBC URL username + password cache settings clustering settings ...

jdbcspy A built-in SQL profiler Useful for identifying cause of performance issues

tolog A logic-based query language a mix of Prolog and SQL effectively equivalent to Datalog Two parts queries (data retrieval) updates (data modification) Developed by Ontopia not an ISO standard eventually to be replaced by TMQL

tolog The recommended way to interact with the data API programming is slow and cumbersome tolog queries perform better Available via Java API Web service API Forms interface in Omnigator tolog queries return API objects

Finding all operas by a composer Collection operas = new ArrayList(); TopicIF composer = getTopicById("puccini"); TopicIF composed_by = getTopicById("composed-by"); TopicIF work = getTopicById("work"); TopicIF creator = getTopicById("composer"); for (AssociationRoleIF role1 : composer.getRolesByType(creator)) { AssociationIF assoc = role1.getAssociation(); if (assoc.getType() != composed_by) continue; for (AssociationRoleIF role2 : assoc.getRoles()) { if (role2.getType() != work) continue; operas.add(role2.getPlayer()); } }

Finding all operas by a composer composed-by(puccini : composer, $O : work)? composed-by($C : composer, tosca : work)? composed-by($C : composer, $O : work)? composed-by(puccini : composer, tosca : work)?

Features Access all aspects of a topic map Generic queries independent of ontology AND, OR, NOT, OPTIONAL Count Sort LIMIT/OFFSET Reusable inference rules

Chaining predicates (AND) Predicates can be chained born-in($PERSON : person, $PLACE : place),located-in($PLACE : containee, italy : container)? The comma between the predicates means AND This query finds all the people born in Italy It first builds a two-column table of all born-in associations Then, those rows where the place is not located-in Italy are removed (Note that when the PLACE variable is reused above that means that the birthplace and the location must be the same topic in each match) Any number of predicates can be chained Their order is insignificant Actually, the optimizer reorders the predicates It will start with located-in because it has a topic constant

Thinking in predicates ,[object Object]

function(arg1, arg2, arg3) -> result

Predicates, however, are in a sense bidirectional, because of the way the pattern matching works

predicate(topic : role1, $VAR : role2)

predicate($VAR : role1, topic : role2)

The order of the roles are, on the other hand, insignificant

predicate($VAR : role2, topic : role1),[object Object]

The instance-of predicate ,[object Object]

NOTE: the order of the arguments is significant

Like players, instance and class may be specified in two ways:

instance-of makes use of the superclass-subclass associations in the topic map

this means that composers will be considered musicians, and musicians will be considered persons,[object Object]

All non-hidden photos select $PHOTO from instance-of($PHOTO, op:Photo) not(ph:hide($PHOTO : ph:hidden)), not(ph:taken-at($PHOTO : op:Image, $PLACE : op:Place), ph:hide($PLACE : ph:hidden)), not(ph:taken-during($PHOTO : op:Image, $EVENT : op:Event), ph:hide($EVENT : ph:hidden)), not(ph:depicted-in($PHOTO : ph:depiction, $PERSON : ph:depicted), ph:hide($PERSON : ph:hidden))?

Demo Show running queries in Omnigator Also show query tracing Shakespeare /* #OPTION: optimizer.reorder = false */

tologspy tolog query profiler shares code with jdbcspy

Using the query engine API ,[object Object],get a QueryProcessorIF object run a query in the QueryProcessorIF and get a QueryResultIF loop over the results and use them close the result object go back to step 2, or do something else ,[object Object]

the API lets you write code without worrying about that, however

the two implementations behave identically,[object Object]

Advanced options ,[object Object]

the processor returns a ParsedQueryIF object, which can be executed

parameters can be passed to the query on each execution

It is possible to make declarations and use them across executions,[object Object]

QueryWrapper Designed to make all of this easier QueryWrapper qw = new QueryWrapper(tm); TopicIF topic = qw.queryForTopic(...); List topics = qw.queryForList(...); List<Person> people = qw.queryForList(..., mapper);

tolog updates Greatly simplifies TM modification Also means you can do modification without API programming useful with RDBMS topic maps useful with TMs in running web servers By performing a sequence of updates, just about any change can be made Potentially allows much more powerful architecture

DELETE Static form delete lmg Dynamic form delete $person from instance-of($person, person) Delete a value delete subject-identifier(topic, “http://ex.org/tst”)

MERGE Static form MERGE topic1, topic2 Dynamic form MERGE $p1, $p2 FROM instance-of($p1, person), instance-of($p2, person), email($p1, $email), email($p2, $email)

INSERT Static form INSERT lmg isa person; - “Lars Marius Garshol” . Dynamic form INSERT tmcl:belongs-to-schema(tmcl:container : theschema, tmcl:containee: $c) FROM instance-of($c, tmcl:constraint)

INSERT again INSERT ?y $psi . event-in-year(event: $e, year: ?y) FROM start-date($e, $date), str:substring($y, $date, 4), str:concat($psi, "http://psi.semagia.com/iso8601", $y)

UPDATE Static form UPDATE value(@3421, “New name”) Dynamic form UPDATE value($TN, “Ontopia”) FROM topic-name(oks, $TN)

More information Look at sample queries in Omnigator tolog tutorial http://www.ontopia.net/doc/current/doc/query/tutorial.html tolog built-in predicate reference http://www.ontopia.net/doc/current/doc/query/predicate-reference.html

Conversion from RDBMS data DB2TM

DB2TM Upconversion of relational data either from CSV files or over JDBC Based on an XML file describing the mapping very highly configurable Support for all of Topic Maps (except variants) value transformations synchronization

Standard use case Pull in data from external source turn it into Topic Maps following some ontology Enrich it usually manually, but not necessarily Resync from source at intervals

DB2TM example Ontopia + = United Nations Bouvet <relation name="organizations.csv" columns="id name url"> <topic type="ex:organization">
<item-identifier>#org${id}</item-identifier>
<topic-name>${name}</topic-name>
<occurrence type="ex:homepage">${url}</occurrence>
</topic> </relation>

Creating associations <relation name="people.csv" columns="id given family employer phone"> <topic id="employer"> <item-identifier>#org${employer}</item-identifier> </topic> <topic type="ex:person"> <item-identifier>#person${id}</item-identifier> <topic-name>${given} ${family}</topic-name> <occurrence type="ex:phone">${phone}</occurrence> <player atype="ex:employed-by" rtype="ex:employee"> <other rtype="ex:employer" player="#employer"/> </player> </topic> </relation>

Value transformations <relation name="SCHEMATA" columns="SCHEMA_NAME"> <function-column name='SCHEMA_ID' method='net.ontopia.topicmaps.db2tm.Functions.makePSI'> <param>${SCHEMA_NAME}</param> </function-column> <topic type="mysql:schema"> <item-identifier>#${SCHEMA_ID}</item-identifier> <topic-name>${SCHEMA_NAME}</topic-name> </topic> </relation>

Running DB2TM java net.ontopia.topicmaps.db2tm.Execute command-line tool also works with RDBMS topic maps net.ontopia.topicmaps.db2tm.DB2TM API class to run transformations methods "add" and "sync"

More information DB2TM User's Guide http://www.ontopia.net/doc/current/doc/db2tm/user-guide.html

Synchronizing with other sources TMSync

TMSync Configurable module for synchronizing one TM against another define subset of source TM to sync (using tolog) define subset of target TM to sync (using tolog) the module handles the rest Can also be used with non-TM sources create a non-updating conversion from the source to some TM format then use TMSync to sync against the converted TM instead of directly against the source

How TMSync works Define which part of the target topic map you want, Define which part of the source topic map it is the master for, and The algorithm does the rest

If the source is not a topic map TMSync convert.xslt Simply do a normal one-time conversion let TMSync do the update for you In other words, TMSync reduces the update problem to a conversion problem source.xml

The City of Bergen usecase Norge.no Service Unit Person LOS City of Bergen LOS

TMRAP basics Abstract interface that is, independent of any particular technology coarse-grained operations, to reduce network traffic Protocol bindings exist plain HTTP binding SOAP binding Supports many syntaxes XTM 1.0 LTM TM/XML custom tolog result-set syntax

get-topic Retrieves a single topic from the remote server topic map may optionally be specified syntax likewise Main use to build client-side fragments into a bigger topic map to present information about a topic on a different server

get-topic Parameters identifier: a set of URIs (subject identifiers of wanted topic) subject: a set of URIs (subject locators of wanted topic) item: a set of URIs (item identifiers of wanted topic) topicmap: identifier for topic map being queried syntax: string identifying desired Topic Maps syntax in response view: string identifying TM-Views view used to define fragment Response topic map fragment representing topic in requested syntax default is XTM fragment with all URI identifiers, names, occurrences, and associations in default view types and scopes on these constructs are only identified by one <*Ref xlink:href=“...”/> XTM element the same goes for associated topics

get-topic-page Returns link information about a topic that is, where does the server present this topic mainly useful for realizing the portal integration scenario result information contains metadata about server setup

get-topic-page Parameters identifier: a set of URIs (subject identifiers of wanted topic) subject: a set of URIs (subject locators of wanted topic) item: a set of URIs (item identifiers of wanted topic) topicmap: identifier for topic map being queried syntax: string identifying desired Topic Maps syntax in response Response is a topic map fragment [oks : tmrap:server = "OKS Samplers local installation"] [opera : tmrap:topicmap = "The Italian Opera Topic Map"] {opera, tmrap:handle, [[opera.xtm]]} tmrap:contained-in(oks : tmrap:container, opera : tmrap:containee) tmrap:contained-in(opera : tmrap:container, view : tmrap:containee) tmrap:contained-in(opera : tmrap:container, edit : tmrap:containee) [view : tmrap:view-page %"http://localhost:8080/omnigator/models/..."] [edit : tmrap:edit-page %"http://localhost:8080/ontopoly/enter.ted?..."] [russia = "Russia” @"http://www.topicmaps.org/xtm/1.0/country.xtm#RU"]

get-tolog Returns query results main use is to extract larger chunks of the topic map to the client for presentation more flexible than get-topic can achieve more with less network traffic

get-tolog Parameters tolog: tolog query topicmap: identifier for topic map being queried syntax: string identifying desired syntax of response view: string identifying TM-Views view used to define fragment Response if syntax is“tolog” an XML representation of the query result useful if order of results matter otherwise, a topic map fragment containing multiple topics is returned as for get-topic

add-fragment Adds information to topic map on the server does this by merging in a fragment Parameters fragment: topic map fragment topicmap: identifier for topic map being added to syntax: string identifying syntax of request fragment Result fragment imported into named topic map

update-topic Can be used to update a topic add-fragment only adds information update sets the topic to exactly the uploaded information Parameters topicmap: the topic map to update fragment: fragment containing the new topic syntax: syntax of the uploaded fragment identifier: a set of URIs (subject identifiers of wanted topic) subject: a set of URIs (subject locators of wanted topic) item: a set of URIs (item identifiers of wanted topic) Update happens using TMSync

delete-topic Removes a topic from the server Parameters identifier: a set of URIs (subject identifiers of wanted topic) subject: a set of URIs (subject locators of wanted topic) item: a set of URIs (item identifiers of wanted topic) topicmap: identifier for topic map being queried Result deletes the identified topic includes all names, occurrences, and associations

tolog-update Runs a tolog update statement Parameters topicmap: topic map to update statement: tolog statement to run Runs the statement & commits the change

HTTP binding basics The mapping requires a base URL e.g http://localhost:8080/tmrap/ This is used to send requests http://localhost:8080/tmrap/method?param1=value1&... GET is used for requests that do not cause state changes POST for requests that do Responses returned in response body

Exercise #1: Retrieve a topic Use the get-topic request to retrieve a topic from the server base URL is http://localhost:8080/tmrap/ find the identifying URI in Omnigator just print the retrieved fragment to get a look at it Note: you must escape the “#” character in URIs otherwise it is interpreted as the anchor and not transmitted at all escape sequence: %23 Note: you must specify the topic map ID otherwise results will only be returned from loaded topic maps in other words: if the topic map isn’t loaded, you get no results

Solution #1 (in Python) import urllib BASE = "http://localhost:8080/tmrap/tmrap/" psi = "http://www.topicmaps.org/xtm/1.0/country.xtm%23RU" inf = urllib.urlopen(BASE + "get-topic?identifier=" + psi) print inf.read() inf.close()

Solution #1 (response)
<topicMap xmlns="http://www.topicmaps.org/xtm/1.0/"
xmlns:xlink="http://www.w3.org/1999/xlink">
<topic id="id458">
<instanceOf>
<subjectIndicatorRef xlink:href="http://psi.ontopia.net/geography/#country"/>
</instanceOf>
<subjectIdentity>
<subjectIndicatorRef xlink:href="http://www.topicmaps.org/xtm/1.0/country.xtm#RU"/>
<topicRef xlink:href="file:/.../WEB-INF/topicmaps/geography.xtmm#russia"/>
</subjectIdentity>
<baseName>
<baseNameString>Russia</baseNameString>
</baseName>
</topic>

Processing XTM with XSLT This is possible, but unpleasant the main problem is that the XML is phrased in terms of Topic Maps, not in domain terms this means that all the XPath will talk about “topic”, “association”, ... and not “person”, “works-for” etc The structure is also complicated this makes queries complicated for example, the XPath to traverse an association looks like this: //xtm:association [xtm:member[xtm:roleSpec / xtm:topicRef / @xlink:href = '#employer'] [xtm:topicRef / @xlink:href = concat('#', $company)]] [xtm:instanceOf / xtm:topicRef / @xlink:href = '#employed-by']

TM/XML Non-standard XML syntax for Topic Maps defined by Ontopia (presented at TMRA’05) implemented in the OKS XSLT-friendly much easier to process with XSLT than XTM can be understood by developers who do not understand Topic Maps dynamic domain-specific syntaxes instead of generic syntax predictable (can generate XML Schema from TM ontology)

TM/XML example <topicmap ... reifier="tmtopic"> <topicmap id="tmtopic"> <iso:topic-name><tm:value>TM/XML example</tm:value> </iso:topic-name> <dc:description>An example of the use of TM/XML.</dc:description> </topicmap> <person id="lmg"> <iso:topic-name><tm:value>Lars Marius Garshol</tm:value> <tm:variant scope="core:sort">garshol, lars marius</tm:variant> </iso:topic-name> <homepage datatype="http://www.w3.org/2001/XMLSchema#anyURI" >http://www.garshol.priv.no</homepage> <created-by role="creator" topicref="tmtopic" otherrole="work"/> <presentation role="presenter"> <presented topicref="tmxml"/> <event topicref="tmra05"/> </presentation> </person> </topicmap>

tmphoto Category Person Photo Event Location http://www.garshol.priv.no/tmphoto/ A topic map to organize my personal photos contains ~15,000 photos A web gallery runs on Ontopia on www.garshol.priv.no

tmtools http://www.garshol.priv.no/tmtools/ Organization An index of Topic Maps tools organized as shown on the right Again, web application for browsing screenshots below Person Software product Platform Category Technology

The person page Boring! No content.

get-illustration A web service in tmphoto receives the PSI of a person then automatically picks a suitable photo of that person Based on vote score for photos, categories (portrait), other people in photo ... The service returns a topic map fragment with links to the person page and a few different sizes of the selected photo http://www.garshol.priv.no/blog/183.html

get-illustration Hmmm. Scores, categories, people in photo, ... Do you have a photo of http://psi.ontopedia.net/Benjamin_Bock ? http://www.garshol.priv.no/tmphoto/get-illustration?identifier=http://psi.on.... tmphoto tmtools Topic map fragment

Points to note No hard-wiring of links just add identifiers when creating people topics photos appear automatically if a better photo is added later, it’s replaced automatically No copying of data no duplication, no extra maintenance Very loose binding nothing application-specific Highly extensible once the identifiers are in place we can easily pull in more content from other sources

My blog Has more content about people (tmphoto & tmtools), events (tmphoto), tools (tmtools), technologies (tmtools) Should be available in those applications

Solution My blog posts are tagged but the tags are topics, which can have PSIs these PSIs are used in tmphoto and tmtools, too The get-topic-page request lets tmphoto & tmtools ask the blog for links to relevant posts given identifiers for a topic, returns links to pages about that topic http://www.garshol.priv.no/blog/145.html

get-topic-page Do you have pages about http://psi.ontopedia.net/TMRA_2008 ? http://www.garshol.priv.no/blog/get-topic-page?identifier=http://psi.on.... Blog tmphoto Topic map fragment Topics linking to individual blog posts

Making web applications Navigator Framework

Ontopia Navigator Framework Java API for interacting with TM repository JSP tag library based on tolog kind of like XSLT in JSP with tolog instead of XPath has JSTL integration Undocumented parts web presentation components some wrapped as JSP tags want to build proper portlets from them

How it works Web server with JSP containere.g. Apache Tomcat JSP page Browser Topic MapEngine TagLibraries JSP page Browser JSP page Browser JSP page Browser Topic Map

The two tag libraries ,[object Object]

makes up nearly the entire framework

used to extract information from topic maps

lets you execute tolog queries to extract information from the topic map

looping and control flow structures

separates layout and structure from content

optional, but recommended,[object Object]

collected from the tm-sources.xml configuration file

each topic map has its own id (usually the file name)

Each page also holds a set of variable bindings

each variable holds a collection of objects

objects can be topics, base names, locators, strings, ...

some tags set the values of variables, while others use them,[object Object]

Tells the page which tag library to include and binds it to a prefix

Prefixes are used to qualify the tags (and avoid name collisions)

Use the <tolog:context> tag around the entire page

The "topicmap" attribute specifies the ID of the current topic map

The first time you access the page in your browser the page gets compiled

If you modify the page then it will be recompiled the next time it is accessed,[object Object]

Navigator tag library example <%-- assume variable 'composer' is already set --%> Operas: <tolog:foreach query=”composed-by(%composer% : composer, $OPERA : opera), { premiere-date($OPERA, $DATE) }?”> <li> <a href="opera.jsp?id=<tolog:id var="OPERA"/>” ><tolog:out var="OPERA"/></a> <tolog:if var="DATE"> <tolog:out var="DATE"/> </tolog:if> </li></tolog:foreach>

Possible configuration Application directories webapps myApp/ *.jsp omnigator/ WEB-INF/ config/ *.xml i18n/ topicmaps/ *.xtm, *.ltm web.xml

The navigator configuration files ,[object Object]

where to find the other files, plus plug-ins

tells the navigator where to find topic maps

configuration of the log4j logging

More details in the "Configuration Guide" document,[object Object]

What is automated classification? Create parts of a topic map automatically using the text in existing content as the source not necessarily 100% automatic; user may help out A hard task natural language processing is very complex result is never perfect However, it’s possible to achieve some results

Why automate classification? Creating a topic map requires intellectual effort that is, it requires work by humans Human effort = cost added value must be sufficient to justify the cost in some cases either the cost is too high, or the value added is too limited The purpose of automation is to lower the cost this increases the number of cases where the use of Topic Maps is justified

Automatable tasks Project Person Department Worked on Worked on Jane Doe worked on employed in XYZ Project IT group Ontology hard depends on requirements one time only Instance data hard usually exists in other sources Document keywords easier frequent operation usually no other sources

Two kinds of categorization Broad: Environment, Crisis management Narrow: Water, Norway, drought, Drought Act, Cloud seeding, Morecambe Bay Broad categorization categories are broadly defined include many different subjects Narrow categorization uses very specific keywords each keyword is a single subject

What it does Extract keywords from content goal is to use these for classification Not entity recognition we only care about identifying what the content is about Uses statistical approach no attempt at full formal parsing of the text

Steps of operation Identify format then, extract the text Identify language then, remove stop words stem remaining words Classify can use terms from preexisting Topic Maps exploits knowledge of the language Return proposed keywords

Example of keyword extraction topic maps 1.0 metadata 0.57 subject-based class. 0.42 Core metadata 0.42 faceted classification 0.34 taxonomy 0.22 monolingual thesauri 0.19 controlled vocabulary 0.19 Dublin Core 0.16 thesauri 0.16 Dublin 0.15 keywords 0.15

Example #2 Automated classification 1.0 5 Topic Maps 0.51 14 XSLT 0.38 11 compound keywords 0.29 2 keywords 0.26 20 Lars 0.23 1 Marius 0.23 1 Garshol 0.22 1 ...

So how could this be used? To help users classify new documents in a CMS interface suggest appropriate keywords, screened by user before approval Automate classification of incoming documents this means lower quality, but also lower cost Get an overview of interesting terms in a document corpus classify all documents, extract the most interesting terms this can be used as the starting point for building an ontology (keyword extraction only)

Example user interface The user creates an article this screen then used to add keywords user adjusts the proposals from the classifier

Interfaces java net.ontopia.topicmaps.classify.Chew <topicmapuri> <inputfile> produces textual output only net.ontopia.topicmaps.classify.SimpleClassifier classify(uri, topicmap) -> TermDatabase classify(uri) -> TermDatabase

Supported formats and languages XML (any schema) HTML (non-XML) PDF Word (.doc, .docx) PowerPoint (.ppt, .pptx) Plain text English Norwegian

Visualization of Topic Maps Vizigator

The Vizigator Graphical visualization of Topic Maps Two parts VizDesktop: Swing desktop app for configuration Vizlet: Java applet for web deployment Configuration stored in XTM file

The uses of visualization Not really suitable for navigation doesn't work for all kinds of data Great for seeing the big picture

The Vizigator The Vizigator uses TMRAP the Vizlet runs in the browser (on the client) a fragment of the topic map is downloaded from the server the fragment is grown as needed Server TMRAP

Embedding the Vizlet Set up TMRAP service Add ontopia-vizlet.jar Add necessary HTML <applet code="net.ontopia.topicmaps.viz.Vizlet.class" archive="ontopia-vizlet.jar"> <param name="tmrap" value="/omnigator/plugins/viz/"> <param name="config" value="/omnigator/plugins/viz/config.jsp?tm=<%= tmid %>"> <param name="tmid" value="<%= tmid %>"> <param name="idtype" value="<%= idtype %>"> <param name="idvalue" value="<%= idvalue %>"> <param name="propTarget" value="VizletProp"> <param name="controlsVisible" value="true"> <param name="locality" value="1"> <param name="max-locality" value="5"> </applet>

Omnigator Generic Topic Maps browser very useful for seeing what's in a topic map the second-oldest part of Ontopia Contains other features beyond simple browsing statistics management console merging tolog querying/updates export

Ontology designer and editor Ontopoly

Ontopoly A generic Topic Maps editor, in two parts ontology editor: used to create the ontology and schema instance editor: used to enter instances based on ontology Features works with both XTM files and topic maps stored in RDBMS backend supports access control to administrative functions, ontology, and instance editors existing topic maps can be imported parts of the ontology can be marked as read-only, or hidden

Ontology designer Create ontology based on topic, association, name, occurrence, and role types Supports iterative ontology development modify and prototype the ontology until it's right Supports ontology annotation add fields to topic types, for example Supports views define restricted views of certain topic types

Instance editor Configured by the ontology editor shows topics as defined by the ontology Has several ways to pick associations drop-down list by search from hierarchy Avoids conflicts pages viewed by one user are locked to others

Ontopoly is embeddable The Ontopoly instance editor can be embedded basically, the main panel can be inserted into another web application uses an iframe Requires only ID of topic being edited can also be restricted to a specific view Makes it possible to build easier-to-use editors so users don't have to learn all of Ontopoly

Adding content features CMS integrations

CMS integration The best way to add content functionality to Ontopia the world doesn’t need another CMS better to reuse those which already exist So far two integrations exist Escenic OfficeNet Knowledge Portal more are being worked on

Implementation A CMS event listener the listener creates topics for new CMS articles, folders, etc the mapping is basically the design of the ontology used by this listener Presentation integration it must be possible to list all topics attached to an article conversely, it must be possible to list all articles attached to a topic how close the integration needs to be here will vary, as will the difficulty of the integration User interface integration it needs to be possible to attach topics to an article from within the normal CMS user interface this can be quite tricky Search integration the Topic Maps search needs to also search content in the CMS can be achieved by writing a tolog plug-in

Articles as topics is about Elections New city council appointed Goal: associate articles with topics mainly to say what they are about typically also want to include other metadata Need to create topics for the articles to do this in fact, a general CMS-to-TM mapping is needed must decide what metadata and structures to include

Mapping issues Article topics what topic type to use? title becomes name? (do you know the title?) include author? include last modified? include workflow state? should all articles be mapped? Folders/directories/sections/... should these be mapped, too? one topic type for all folders/.../.../...? if so, use associations to connect articles to folders use associations to reproduce hierarchical folder structure Multimedia objects should these be included? what topic type? what name? ...

Two styles of mappings Articles as articles Topic represents only the article Topic type is some subclass of “article” “Is about” association connects article into topic map Fields are presentational title, abstract, body Articles as concepts Topic represents some real-world subject (like a person) article is just the default content about that subject Type is the type of the subject (person) Semantic associations to the rest of the topic map works in department, has competence, ... Fields can be semantic name, phone no, email, ...

Article as article Article about building of a new school Is about association to “Primary schools” Topic type is “article”

Article as concept Article about a sports hall Article really represents the hall Topic type is “Location” Associations to ,[object Object]

category “Sports”,[object Object]

Ontopia/Liferay An integration with the Liferay CMS and portal is in progress presented Friday 1130-1150 in Schiller 2

The project A new citizen’s portal for the city administration strategic decision to make portal main interface for interaction with citizens as many services as possible are to be moved online Big project started in late 2004, to continue at least into 2008 ~5 million Euro spent by launch date 1.7 million Euro budgeted for 2007 Topic Maps development is a fraction of this (less than 25%) Many companies involved Bouvet/Ontopia Avenir KPMG Karabin Escenic

Simplified original ontology Service catalog Escenic (CMS) LOS Form Article nearly everything Category Service Subject Department Borough External resource Employee Payroll++

Ontopia tutorial

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Ontopia tutorial

Similar a Ontopia tutorial (20)

Más de Lars Marius Garshol

Más de Lars Marius Garshol (20)

Último

Último (20)

Ontopia tutorial