Devoxx 2010 | Tools In Action : Kauri and Lily

1IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org
Scalable and RestFul Web Apps
At the xroads of Kauri and Lily
&

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org
Speaker Context
Marc Portier, ±14 years of Java, XML based web
development. Co-founder of Outerthought.
• Outerthought?
– Innovative Content Management system that follows
ReST principles: “DaisyCMS”
– New ReST supporting development framework:
“Kauri project”
– Scalable Search and Store: “Lily Project”

Looking for You!
• User, Contributor of
our Open Source
Projects
• Teacher, University,
School
• Student
• Talented worker
– Job openings!

The Wonderful
Web Machine

Amazing Web
Supercalifragilisticexpialidocious !
FRANTIC
COMPLEX
UNMANAGEABLE
CHAOTIC
Heterogenous
Layered
Decentralized
Elegant Just Works
Extensible
Simple
Usable
Functional
Entertaining
Scalable

How does it keep up?
• Diversity / Compatibility
• Amount of Servers
• Amount of Users
• Amount of resources
• Response Times / Latency
Wouldn't you want to do that too?

In woodworking it's important
to work with the grain
of the wood.
The Web, too, has a grain, and
a RESTful web service is
one that works with it.
－ Leonard Richardson
and Sam Ruby
RestFull Web Services

Meeting high expectations
• Current web-experiences are spoiling us
– Google Maps
– Mash-ups
– Atom
– Amazon S3
• It all looks so simple (and mostly works)
• General customer feeling:
“How hard could it be?””

Riding the
Web Project Wave

kauriproject.org
• Resource Oriented WebDevelopment PlatForm
• Spans WebServices (headless)
• and WebApplications (human consumer)
• Leverages modern and popular techniques and
systems
(+Jax-RS )
• Currently 0.4-RC2 – ( 0.4 upcoming )

Heritage
Kauri
• Maven
• jQuery
• Spring Framework
• Restlet
Outerthought
• Apache Cocoon
• XReporter
• DaisyCms
➢ Build & Deploy
➢ Front-end Lib
➢ Dependency Injection
➢ REST API
➢ XML, URI Maps
➢ Statemanagement
➢ Runtime, RIA, ...

Kauri Core Concepts
• modules,
– java/rest services,
– service containers,
– service wiring,
• prototyping,
• RIA,
• techniques
• routing, pages,
• templates

Reuse Paradox

Perfection is achieved,
not when there is
nothing more to add,
but when there is
nothing more to remove.
－ Antoine de Saint-Exupery

Plugable Systems
• Interfaces
• Implementations
• Wiring

Challenges
• Jar Dependency Management
– Maven POM >> Deployment Descriptoor
– Runtime Shielding
• Java interface aspect (obvious)
• REST interface aspect
– RestService Notion (instantly web-distributed)
– URI-Templating:
/service/{rsrc_id}/aspect/{subresource}.json
– Wiring + internal protocol:
service:/

Kauri Component Model: Modules
• Unit of reuse
• Maven managed JAR
• Service Interfaces
– Java aspect
– REST aspect
• Maps to One Spring
Container
• Has own classloading
scope

Kauri Runtime System

Kauri Attitude
• No Servlet container
– (Although Kauri Apps can be deployed to...)
– Servlets miss out on REST and IO threading
• Own Application Container
– Spring Based
• Enhanced with own injection directives
• Wrapped in managed classloading hierarchy
• Allows wiring between modules
– Maven Repository Based
– Required support for the REST aspect
– Advanced configuration

Module vs. war
• war != jar
– Different layout
– =deployment unit != reuse/component unit
– Too high level for dependency shielding
• Deployment Strategy:
– conf/kauri/wiring.xml: deployment descriptor
– Reuse maven repository layout
– Different targets:
• Standalone, rpm, war,...

Kauri Routing and Representations
• MVC-Split (Inspiration: Spring webmvc)
• Routing
– Groovy syntax support (builder @ init time)
– Request time: Java performance
– JaxRS support
• Separate Representation Builder
– Similar groovy syntax (again builder)
– Produces Representation
– Handle (presentation of) error conditions centrally

Routing Samples
• Groovy router
• JaxRS
builder.router {
directory(uri: "/resources", root: "module:/resources/")
resource(uri: "/persons/{id}.html", ofClass: "com.foobar.Person")
}
@Path("/persons/{id}.html")
public class PersonResource {
@GET
@Produces("text/html")
public KauriRepresentation getPerson(
@PathParam("id") String idParam) {
....
}
}

Kauri Pages & Prototyping
• Combination of
– Instant reload: module-source from FS vs. jar
– Intuitive FS based router
• pages/person/{id}.html.xml
– Advanced templates
– Simple mock-dataservice
• Entity modelling through sample json files
• Loaded in the templates via service:/
• Easily re-wired to JPA based implementation.

Web Development Process

Kauri Prototyping

Continuous Prototyping
• -p prototype mode
– Possibility to blend and match production stuff with
new try-out stuff
• Pages router
– Invite in design-aware HTML/CSS/JS client hacker
with feeling for programming & FS based toolchain
• -s module-source.properties
– fast roundtrip mode directly mounted in the source

Kauri Templating
• XML (tag) Based
– Conditionals, repeat, ...
– Security – protect,
• ${placeholder} replacement
– through el or groovy
– + Specific functions (i18n, config, publicUri, ...)
• + Add your own
• Noteworthy
– Page inheritance
– Local service loading

Kauri Template Inheritance
<?xml version="1.0" encoding="UTF-8"?>
<html xmlns:t="http://kauriproject.org/template"
t:inherit="module:/templates/layout/layout.xml">
<t:block name="title">My page</t:block>
<t:block name="main">My interesting content</t:block>
</html>
<?xml version="1.0"?>
<html xmlns:t="http://kauriproject.org/template">
<head>
<title>
<t:block name="title">The title</t:block>
</title>
</head>
<body>
<t:block name="main">Main content</t:block>
</body>
</html>
< base
“layout.xml”
usage >
“mypage.xml”

Kauri Template Data-Loading
{
“name”: “Marc Portier”,
“city”: “Bredene”,
...
}
<?xml version="1.0"?>
<html xmlns:t="http://kauriproject.org/template">
<t:variable name="person"
src="service:/data/persons/${request.attributes.id}"
overwrite="false"
accept="application/json"/>
<body>
<h1>Person: ${person.name}</h1>
<p>City: ${person.city}</p>
</body>
</html>
Template : “persons/{id}.html.xml”
URI : “/persons/mpo.html”
Data : “mock/persons/mpo.json”

Kauri Forms
• Browser-based Data Value Interaction System
• RIA approach
• jQuery Based
– UI effects & Controls
• Adds
– Standard rest-full ajax logic
– Validation (local and remote)
– Formatting

Kauri Forms Sample
var fconf = {
createURI: "${publicUri('service:/data/contact/')}",
type: {
members: {
name: "string",
email: {
base: "string",
"+validators": { isEmail: {} },
label: "e-mail"
},
birthday: {
base: "date",
yearRange: "-100:+0",
label: "What is your date of birth?"
},
}
}};
Json Form-config:
HTML:
<form id=”myform” />

Kauri Form Controls
• Simple
– Basic data-types
– Classic validations
– HTML basics
• Aggregates
– Collection
– Composite
– Choice
• + Add your own
• Specific
– GoogleMaps-Location
– Ajax based file-upload
– ...

More Kauri
• I18n support
• XML & HTML loading (and cleaning)
• Unified Configuration
• Unified Security
– Spring Security (ACEGI)
– Support for various mehanisms
• Basic, digest, https, form, ...
• Belgian e-ID
• Database resources (JPA)
• ...

Future Kauri
• State management through temporary
resources
• More Advanced controls
• Functional Modules

Kauri Goals
How
• All team members
• All project stages
• Efficiency
– Prototype as a design
mode
– Enhance, not throw
away
What
• ROA & RIA
• Modules
• Templates
• Data Interaction

Observing the Obvious
• REST-full stateless
approach in web-
access layer pretends
“Ready for scale out”
• Shifting the problem
towards “Storage”
• ie. Indexed Storage

Scalable Store and
Search

lilyproject.org
• Scalable Store AND Search
– Massively distributed searchable data store
• Blended HBASE with SOLR (shard), adding
– Schema with support for
• versioning, varianting and Blob handling
– 2ndary indices
– Rowlog/Queue for sync & async updates
– M/R batch re-indexing
• 0.2 since October 2010,
1.0 expected March 2011

Scale on all ends

Lily Core Concepts
● Storage
● Hbase
● repository model
● versioning, varianting, mixins
● Indexing
● Mapping
● Search
● SOLR

Repository Model

Mixins

Sample Schema (json)
namespaces: {
/* Declaration of namespace prefixes. */
"org.lilyproject.bookssample": "b",
"org.lilyproject.vtag": "vtag"
},
fieldTypes: [
{
name: "b$title",
valueType: { primitive: "STRING" },
scope: "versioned"
},
{
name: "b$pages",
valueType: { primitive: "INTEGER" },
scope: "versioned"
},
{
name: "b$language",
scope: "versioned"
},
{
name: "b$authors",
valueType: { primitive: "LINK", multiValue: true },
scope: "versioned"
},
{
name: "b$name",
scope: "versioned"
},
{
name: "b$bio",
scope: "versioned"
},
{
name: "vtag$last",
valueType: { primitive: "LONG" },
scope: "non_versioned"
}
],
recordTypes: [
{
name: "b$Book",
fields: [
{name: "b$title", mandatory: true },
{name: "b$pages", mandatory: false },
{name: "b$language", mandatory: false },
{name: "b$authors", mandatory: false },
{name: "vtag$last", mandatory: false }
]
},
...

Flexible content model
• generic enough to accomodate many popular
content schemas
– HTML5, CMIS, RDF, NewsML, Dublin Core, …
– academically verified (UGent MMLab / IBBT)
• developer convenience
– higher level constructs
– schema reuse
– versioning, linking, ...

Lily Versioning
`

Indexer
1. denormalize
2. multi-version
3. incremental
4. batch
5. extraction
6. sharded

Lily and HBase
• adds high-level content model
– data types
– versioning
– blob storage on HDFS
• focus on sparse (efficient) storage
• RowLog for synchronous cross-table updates
and async message queues

Lily and SOLR
• provides flexible mapping between HBase
content model and SOLR index fields
• interactive and batch (M/R) index maintenance
• sharding
• use(s) SOLR as-is: loose, flexible, extensible
coupling
• search access via SOLR (HTTP) API

Lily Roadmap

Time to take a ReST
Questions?

Devoxx 2010 | Tools In Action : Kauri and Lily

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (8)

Similar a Devoxx 2010 | Tools In Action : Kauri and Lily

Similar a Devoxx 2010 | Tools In Action : Kauri and Lily (20)

Más de NGDATA

Más de NGDATA (10)

Último

Último (20)

Devoxx 2010 | Tools In Action : Kauri and Lily