Talk given in the CERN School of Computing 2012, Uppsala, Sweden Total time : 8 minutes for presentation, 2 for questions

1. NoSQL
“Easy” web monitoring

2. HEP Computing environment
• Big datacenters
• Existent monitoring
o Nagios/Cacti/similars monitoring
o Site Availability Monitoring/Hammercloud
o High level tests (Grid services)
o General Monitoring of grid services
• Missing/incomplete monitoring
o Specific/detailed information about specific Grid services
o Transfer Management tool (PhEDEx for CMS)
o Batch system (Condor,PBS,LSF,Torque, etc)
o Quota report based on Experiment's characteristic
organization (Physics groups, T2 federation, etc)

3. Options
• SGDB's & SQL - Might be too much
o More costs for deployment
o Less flexibility
o Increase application's complexity/time spent
o Black sheep - SQLite - however, scaling problems
• Scripting - Might be too bad
o Database is too much, query the info from CLI,
format and print (text file, HTML)
o Parse logs, format and print (text file, HTML)
• Bottom line is evaluate – Effort x Result x Quality you
need

4. NoSQL
It doesn't mean SQL is a bad idea =)
CouchDB,MongoDB, etc...

5. JSON

My apologies to the ones that know it...

Format that represents associative arrays (ke
value pairs)
{
"Name": "Smith",
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021"
},
}

6. For the curious... MapReduce

Trivial parallelization

First Map (filter, get documents subset)

Then reduce (count, sum, etc)

7. Overview

Server

Databases

Documents

Unique ID, revision

All the rest is free

Key/Value – datatype for many languages

Views

Static sets of what you want to show

View code scans all documents in the database

Caching

For Your Information – “CouchApps”

8. {
Documents
"_id": "osgce@osgce.hepgrid.uerj.br#1146503.0#1335286041",
"_rev": "10-12a26ecf9bc0fecf5ed9073c374a620c",
"data": {
"NumJobStarts": {
"i": "1"
},
"EnteredCurrentStatus": {
"i": "1335415262"
},
"CurrentHosts": { http://<MYSERVER>/<MYDATABASE>/<DOCUMENT_ID>
"i": "0"
}
}
}

9. Views

Extract from the documents, only what you want to see

Only documents you want to see
function(doc) {
if (doc.Type == "customer") {
emit(doc._id, {LastName: doc.LastName, FirstName: doc.FirstName,
Address: doc.Address});
}
}
http://<MYSERVER>/<MYDATABASE>/_design/<MYDESIGN>/
_view/<MYVIEW>

10. CouchApps

Webserver

CouchDB hosts your HTML(5...)+Javascript
application

Upload to the given “DB”, has access to it's
views, and docs.

DB is replicated, everything goes together
http://<MYSERVER>/<MYDATABASE>/_design/<MYCOUCHAPPAME>/index.html

11. GlobalMonitor