Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Kristoffer Dyrkorn – Beating the traffic jam using NoSQL - NoSQL matters Barcelona 2014

1.033 visualizaciones

Publicado el

Kristoffer Dyrkorn – Beating the traffic jam using NoSQL

Most people have experienced the boredom of being stuck in traffic. Up-to-date and credible information about congestions and detours could save us time and frustration in our everyday lives.The Norwegian Public Roads Administration is now building a new infrastructure for road traffic measurements, and the system will provide high-quality, near-realtime information as publicly available open data. The project relies heavily on NoSQL technology (Elasticsearch) for high-performance data gathering and statistical analysis.This talk will give a walkthrough of the project and the solution and show how NoSQL has helped in building an application that meets demanding requirements. Several use cases that illustrate the value of the system, both for the general public and for private companies and public institutions, will be given.

Publicado en: Datos y análisis
  • Sé el primero en comentar

Kristoffer Dyrkorn – Beating the traffic jam using NoSQL - NoSQL matters Barcelona 2014

  1. 1. BEATING THE TRAFFIC JAM USING NOSQL NoSQL Matters Kristoffer Dyrkorn, BEKK 22/11/14
  2. 2. BEKK Public Roads Administration Norwegian consulting firm Private and public sector enterprises Strategy, technology, digital services 370 employees Responsible for state and county roads Planning, construction, operation 7500 employees Spending: € 7 Billion (2013) BACKGROUND
  3. 3. Population (mill) Area (km2) Roads (1000 km) Germany 81 357 644 Spain 46 500 683 Norway 5 385 95 CONTEXT
  4. 4. E18 Vestfold – Undrumsdal. Photo: Hans A. Rosbach
  5. 5. Atlanterhavsveien, www.nasjonaleturistveger.no. Photo: Harald Mowinckel
  6. 6. Trollstigen, www.nasjonaleturistveger.no. Photo: Steinar Skaar
  7. 7. Aurlandsfjellet, www.nasjonaleturistveger.no. Photo: Bjørn Andresen
  8. 8. ROADS ARE INFRASTRUCTURE BUILDING AND MAINTAINING ROADS IS EXPENSIVE PROPER PLANNING DEPENDS ON TRAFFIC ANALYSIS VEHICLE COUNTS & WEIGHTS DECIDE PRECISE REPORTS ARE NEEDED
  9. 9. Butunellen. Photo: Knut Opeide, Statens Vegvesen
  10. 10. Photo: Statens Vegvesen
  11. 11. Photo: Statens Vegvesen
  12. 12. TIME IS ESSENTIAL ROAD TRAFFIC IS DYNAMIC UPDATED TRAFFIC INFORMATION HELPS: • SAFETY • FLOW CONTROL • ROUTE PLANNING REAL TIME DATA IS NEEDED
  13. 13. Intelligent Transport Systems: Measure & adjust road traffic
  14. 14. SYSTEM GOALS & REQUIREMENTS EASE OF INSTALLATION AND VERIFICATION OF ROADSIDE EQUIPMENT INCREASED DATA QUALITY INCREASED DATA AVAILABILITY ALL EVENTS MUST BE KEPT (NO PRE-AGGREGATION) MINIMAL LATENCY AD-HOC REPORTING SCALABILITY ROBUSTNESS
  15. 15. A TRAFFIC EVENT Voltage signature OPC-UA event Bulked data Sensor Data logger Application Storage
  16. 16. Sensors System status SenSsenosrosr s System Other backends GUI Vehicle info Reports DATA FLOW
  17. 17. SOLUTION ARCHITECTURE HTML5 GUI (HTTP, JSON) Application logic Support libraries Java VM OS Traffic events Reports (CSV, SOAP) Elasticsearch Java VM OS Request/ response Data logger N data loggers M application servers K storage servers
  18. 18. "measure_point_number": 1601436, "county_id": 16, "region_id": 2, "server_local_timestamp": "2014-­‐10-­‐01T01:58:44.330+02:00", "server_utc_timestamp": "2014-­‐09-­‐30T23:58:44.330Z", "client_utc_timestamp": "2014-­‐09-­‐30T23:58:45.229Z", "event_number": 2319762, "vehicle_type": 3, "vehicle_type_raw": "9", "vehicle_type_quality": 22228, "vehicle_number": 2319762, "speed": 80.9, "length": 16.46, "lane": 1, "gap": 10
  19. 19. HOW WE USE ELASTICSEARCH BULK INDEXING, JAVA API DATA IS INDEXED, STORED, NOT ANALYZED TEMPORAL SHARDING SPATIAL SHARDING DATA CENTER-AWARE REPLICATION NO SPECIAL OPTIMIZATIONS! RAM, CPU, DISK
  20. 20. REPORTING FOR A GIVEN TIME INTERVAL, PROVIDE: • TOTAL VEHICLE COUNT AND AVERAGE SPEED, • THE 85 AND 95 PERCENTILE SPEEDS, • IN EACH OF 5 LENGTH CATEGORIES: THE VEHICLE COUNT AND AVERAGE SPEED, • IN EACH OF 12 SPEED CATEGORIES: THE VEHICLE COUNT, ...AND ALL OF THIS FOR • EACH TRAFFIC LANE AT A MEASURE POINT, • EACH MEASURE POINT IN A REGION
  21. 21. VEHICLE FLOW (10 MIN), NOV 4TH 2014
  22. 22. VEHICLE SPEED (1 MIN), NOV 4TH 2014
  23. 23. SYSTEM VALUE REPORTING: • MORE COST-EFFICIENT ROAD MAINTENANCE REAL TIME: • ROUTING OF EMERGENCY VEHICLES • GENERAL TRAFFIC INFORMATION TO THE PUBLIC • ROUTE PLANNING ON HOLIDAYS • ROUTE PLANNING FOR PARCEL SERVICES
  24. 24. EXPERIENCES USING ELASTICSEARCH ENSURE FITNESS-TO-PURPOSE UPGRADE CONTINUOUSLY REVISE THE RUN-TIME ENVIRONMENT CONTINUOUSLY THE AGGREGATIONS MODULE IS FANTASTIC USE TOOLING (WE LIKE KOPF) MONITOR THE RESOURCE UTILIZATION THE JAVA API IS SOMETIMES COMPLEX WE ARE HAPPY!
  25. 25. ?
  26. 26. THANK YOU! Kristoffer Dyrkorn BEKK kristoffer.dyrkorn@bekk.no

×