Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

NoSQL and Data Modeling for Data Modelers

2.357 visualizaciones

Publicado el

Karen Lopez's presentation for data modelers and data architects. Why data modeling is still relevant for big data and NoSQL projects.

Plus 10 tips for data modelers for working on NoSQL projects.

Publicado en: Datos y análisis
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website!
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

NoSQL and Data Modeling for Data Modelers

  1. 1. Big Data, NoSQL & Data Modeling 10 Tips for Data Modeling Success on Modern Data Projects Karen Lopez, InfoAdvisors
  2. 2. Data Models – Traditional Process Conceptual (Data) Model Logical Data Model Physical Data Model(s) OLTP OLTP OLTP OLTP OLTP MARTMART OLTP OLTP OLTP Aug 2014©InfoAdvisors -
  3. 3. Relational Aug 2014©InfoAdvisors - Data Models started with relational modeling, so they look like relational database structures.
  4. 4. But…. That doesn’t mean they can’t be used to model data that goes into a non- relational format. All that formatting happens at build OR consumption time, not requirements time. Aug 2014©InfoAdvisors -
  5. 5. The Big Data Story Lots of data Coming at us fast Lots of variety in format & quality We want all the data Highly available “It’s web scale” Aug 2014©InfoAdvisors -
  6. 6. What do we really mean by scale? Bringing computing to the data Massively parallel processing Cheap, commodity hardware, but lots of it Optimized for Query/Reads/Questions/Telling stories Aug 2014©InfoAdvisors -
  7. 7. We’ve been down this road before… Traditional transactional applications Reporting- optimized tables/structures Data Warehouse / Dimensional Modeling Aug 2014©InfoAdvisors - Highly normalized Highly Denormalized
  8. 8. ETL EDW Data Mart Data Mart
  9. 9. Hadoop ETL EDW Analytics Mart Data Mart
  10. 10. NoSQL, Not Only SQL Relational Graph Columnar/Column Family Key Value Document Databases Others Aug 2014©InfoAdvisors -
  11. 11. Sample Hive Statement CREATE EXTERNAL TABLE TaxRebateUsage ( state string, zipcode string, agi_class int, n1 int, mars2 int, prep int, n2 int, ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE Aug 2014©InfoAdvisors -
  12. 12. Sample JSON/MongoDB Notation Aug 2014©InfoAdvisors -
  13. 13. Sample FoundationDB Statement Aug 2014©InfoAdvisors -
  14. 14. Sample Cassandra Statement Aug 2014©InfoAdvisors -
  15. 15. Sample Vertica Statement Aug 2014©InfoAdvisors -
  16. 16. Sample Neo4j Statement Aug 2014©InfoAdvisors -
  17. 17. Those weren’t SCHEMALESS…. They had data facts, which had meanings. And sometimes expected formats, precisions, and types. In the NoSQL world, we don’t apply those necessarily at write time, but at read time. SCHEMALESS really is MULTIPLE SCHEMAs (Polyschematic) or VARYING SCHEMAs. Aug 2014©InfoAdvisors -
  18. 18. The Big Data Big Lies Schemaless • Schema on Read, not Schema on Write • Polyschematic Big • New data stories • New technologies • Not just volume Aug 2014©InfoAdvisors -
  19. 19. 10 Tips For Modeling in a Hybrid World 1. Models require a modeler 2. Data modeling tools are essential 3. There are many types of data models: know which ones you need 4. Modeling does not have to happen at the same time in every project. It should happen at the right time 5. Modeling is not just schema design. Think outside the boxes and lines Aug 2014©InfoAdvisors -
  20. 20. 10 Tips for Modeling in a Hybrid World 6. A data model is much more than a diagram 7. You will need training. 8. Team members may not understand modeling. They will need training 9. NoSQL is not one thing. Learn many patterns 10.Modern data architectures are likely hybrid solutions. You can’t just support one part. Aug 2014©InfoAdvisors -
  21. 21. What does this mean for data modelers? There will be jobs for traditional, ERD, relational modelers…. ….just like there are still jobs of RPG and COBOL programmers All data has a data story. Many data stories. A good modeler is a an architect at heart – finding the right solution for the data story. Aug 2014©InfoAdvisors -
  22. 22. Business Intelligence Journal Look for September 2014 Issue Article on Modern Data Architectures Aug 2014©InfoAdvisors -
  23. 23. Thank You! #TEAMDATA Aug 2014©InfoAdvisors -