Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Using BigQuery as a main Big Data solution

1.370 visualizaciones

Publicado el

My recent slide deck on BigQuery and how we use it in Wego.com

Publicado en: Datos y análisis
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/ROpLZ ◀ ◀ ◀ ◀
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/ROpLZ ◀ ◀ ◀ ◀
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • Sé el primero en recomendar esto

Using BigQuery as a main Big Data solution

  1. 1. Nikolay Novozhilov Wego.com Using BigQuery as a main Big Data solution
  2. 2. About Wego Wego.com is Asia Pacific and the Middle East’s leading flight/hotel metasearch engine used by millions of travelers. Wego was founded in 2005 in Singapore
  3. 3. Introducing BigQuery Service for interactive analysis of massive datasets (TBs) Query billions of rows: seconds to write, seconds to return Uses a SQL-style query syntax It's a service, accessed by a RESTful API Pay only for what you use Based on internal Google tool - Dremel Column oriented, append only…
  4. 4. Data architecture in Wego ...
  5. 5. Why did we do it? MySQL “Zoo” BigQuery
  6. 6. Why Hadoop is more popular?
  7. 7. My collection of concerns Your data goes to cloud Not open-source, Google can stop the service “Strange” pricing model Hadoop is trending, has bigger community Append only database ???
  8. 8. Costs: storage + cost per query Same fallacy again:  “I want to launch a mom@pop – let’s buy a building”  “I want to build a site – let’s by servers”  “I want big data – let’s build a data-warehouse” Usual concerns:  No realistic estimate upfront  “Fear of running a query”
  9. 9. StackOverflow support 53 minutes!
  10. 10. Append only… Slowly changing dimensions:  daily re-load from MySQL  daily upload from MySQL, keeping history Absolutely necessary updates:  do you really need it?  BigQuery allows to save query to initial table: Your table Query
  11. 11. Actually useful - “Discovery mode”
  12. 12. Actually useful Huge joins REGEXT_MATCH(), … Rich SQL - window functions Nested data
  13. 13. My answer
  14. 14. What is Big Data revolution? There is no difference between big data and small data anymore
  15. 15. Contacts Blog: www.novozhilov.co Email: nik@wego.com
  16. 16. “Yes, Sir, I tired to build an ROI case for our BI project - but I couldn’t access any reliable data!” TimoElliott.com

×