Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Towards a frictionless data future

558 visualizaciones

Publicado el

Presented at the Research Data Network workshop, St Andrews, 30 Nov 2016

Publicado en: Educación
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Towards a frictionless data future

  1. 1. Toward a Frictionless Data FuturePRESENTED BY Jo Barratt (@jobarratt/@okfnlabs) AT 3rd Research Data Network - St Andrews University - 30 November 2016 Licensed under cc-by v3.0 (any jurisdiction)
  2. 2. International non-profit founded in 2004 Who we are ● Vision o A world where open knowledge is ubiquitous, enabling citizens and organizations to create insights that drive change on global and local challenges, combat injustice and inequality and hold governments and corporations to account. ● Mission o Open up all essential, public interest information and see it used to create insight that drives positive change o Build communities, tools and skills to empower individuals and organizations to use open information to create insights that drive change.
  3. 3. Widely adopted - over 20 national governments and 60+ local governments & cities
  4. 4. £4m in 15mins
  5. 5. Frictionless Data is… ● Lightweight specifications for “packaging” datasets ● Integrations for loading datasets into tools and platforms relevant to researchers The Goals... ● Introduce a significant, measurable improvement in how research data is shared, consumed, and analyzed. ● Make it easier to maintain and improve data quality.
  6. 6. The Problem
  7. 7. Treemap of issues … Legal barriers (open data, sharing agreements etc) Data Quality Hard to find Interoperability No tool integration
  8. 8. Cargo loading ~1955 Manual, Slow, Costly (and Dangerous)
  9. 9. Data is Shipping Pre- Containerization
  10. 10. Containerization
  11. 11. Standards - Standards (a few, simple ones) - Tools (primarily for integration) - Documentation - Datasets Data Containerization
  12. 12. Key Principles 1. Simplicity 2. Web Oriented 3. Existing Tools 4. Open 5 Distributed
  13. 13. Tabular Data Package Data Package JSON Table Schema CSV Tabular Data Package
  14. 14. Tooling … Data Package Tabular Data Package JSON Table Schema CSV Tool e.g. import to R, SQL etc Tool e.g. data checking ala GoodTables
  15. 15. Validation
  16. 16. Validation
  17. 17. Continuous Validation ● If you’re working in a group, you need continuous validation… for data! ● In < 1 hour, we integrated elements (datapackage.json + Python libraries + GoodTables API) to support continuous data validation
  18. 18.
  19. 19. Platform Integrations
  20. 20. Partners View more: Dataship YOUR RESEARCH ORG
  21. 21. ● Project website: ● Specifications: ● GitHub: ● User Stories:
  22. 22. ● Newsletter: involved/#newsletter ● Follow @okfnlabs on Twitter (#frictionlessdata)