SlideShare una empresa de Scribd logo
1 de 32
Descargar para leer sin conexión
Eric.kavanagh@bloorgroup.com




Twitter Tag: #briefr
!   Reveal the essential characteristics of enterprise
       software, good and bad

    !   Provide a forum for detailed analysis of today s
       innovative technologies

    !   Give vendors a chance to explain their product to
       savvy analysts

    !   Allow audience members to pose serious questions...
       and get answers!



Twitter Tag: #briefr
!   May: Analytics
     !   June: Intelligence
     !   July: Disruption
     !   August: Analytics
     !   September: Integration
     !   October: Database

Twitter Tag: #briefr
!   Analytics is, and always has been, about discovering insights
        that lead to better business decisions. The range of
        technologies and use cases that inhabit this area is wide:
        statistical analysis, data and process mining, predictive
        analytics and modeling, and complex event processing.

     !   What is now referred to as Big Data has pushed analytics
        beyond the capabilities of traditional solutions. “Big
        Analytics” has organizations diving into large heaps of data
        that previously was not available or usable.

     !   The growing volume, variety, velocity and complexity of
        data has proven to be a major challenge to organizations
        who leverage analytics to maintain a competitive edge.

Twitter Tag: #briefr
Dr. Richard Hackathorn is a well-known
      industry analyst, technology innovator
      and international educator. He has
      pioneered innovations in database
      management, decision support and data
      warehousing. Richard has published
      numerous articles, presented at leading
      industry conferences, and conducted
      professional seminars in eighteen
      countries. He has written three books,
      entitled Enterprise Database
      Connectivity, Using the Data Warehouse
      (with William H. Inmon), and Web
      Farming for the Data Warehouse.
      Richard taught at the Wharton School
      and at the University of Colorado.



Twitter Tag: #briefr
!   Teradata is known for its analytic data solutions with
        a focus on integrated data warehousing, big data
        analytics and business applications.

    !   It offers a broad suite of technology platforms and
        solutions, and a wide range of data management
        applications and data mining capabilities.

    !   Teradata features Teradata Aster is its MapReduce
        platform to handle big data and big analytics on
        multi-structured data.


Twitter Tag: #briefr
Chris Twogood is Vice President of
        Product and Services Marketing for
        Teradata Corporation. He is
        responsible for marketing products
        (database, utilities, and platform),
        and services (professional and
        customer services), plus technical
        field sales support. Chris has twenty-
        five years of experience in the
        computer industry specializing in
        Data Warehousing, Decision Support,
        Customer Management and Appliance
        platforms. Chris has held roles that
        span Strategy, Application Definition,
        Marketing, Product Requirements/
        Management, Platform Solutions and
        Product Marketing.



Twitter Tag: #briefr
Unified Big Data Architecture
Big Data: From Transactions to Interactions

                                                                              BIG DATA
                            User Generated
                                                                                                      Social Network
                               Content
                                                      Mobile Web
                                                                                                        External
                         User Click Stream                                         Sentiment
                                                                                                      Demographics


                                  Web logs
                                                          WEB                   A/B testing        Business Data Feeds

                       Offer history                                          Dynamic Pricing
                                                                                                        HD Video
                                                                             Affiliate Networks
                                  CRM                                                                 Speech to Text
                                                      Segmentation
                                                                             Search marketing
                                                         Offer details
                                                                                                   Product/Service Logs
                          ERP                                               Behavioral Targeting
                                                Customer Touches
                   Purchase detail
                   Purchase record               Support Contacts            Dynamic Funnels            SMS/MMS
                   Payment record




                                               Increasing data variety and complexity



10   Confidential and proprietary. Copyright © 2012 Teradata Corporation.
Unified Big Data Architecture
Bridging Classic & Big Data Worlds

                                                                Classic BI
                                                   Structured & Repeatable Analysis




 Business determines what                                                                 IT structures the data to
      questions to ask                                                                     answer those questions
                                               SQL performance and structure
                                                                                         “Capture only what’s
                                                                                              needed”


                                            MapReduce processing flexibility




  IT delivers a platform for                           Big Data Analytics
    storing, refining, and                                                               Business explores data for
                                                 Multi-structured & Iterative Analysis
 analyzing all data sources                                                              questions worth answering

     “Capture in case it’s
          needed”
11      Confidential and proprietary. Copyright © 2012 Teradata Corporation.
Need for a Unified Big Data Architecture for New Insights
Enabling All Users for Any Data Type from Data Capture to Analysis




           Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualization, etc.


                                                                              Reporting and Execution
               Discover and Explore
                                                                                 in the Enterprise


                                            Capture, Store and Refine


     Audio/                                                          Web &       Machine
                   Images             Docs            Text                                 CRM   SCM   ERP
     Video                                                           Social       Logs



12    Confidential and proprietary. Copyright © 2012 Teradata Corporation.
Unified Big Data Architecture for the Enterprise



              Engineers                      Data Scientists                  Quants         Business Analysts

           Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualization, etc.

                                                             ANALYTICS



           Discovery Platform                                                     Active Data Warehouse




                                                      Capture, Store, Refine


        Audio/                                              Web &            Machine
                        Images             Text                                        CRM       SCM       ERP
        Video                                               Social            Logs




 13   Confidential and proprietary. Copyright © 2012 Teradata Corporation.
Analyst’s Goal: Get Insights from Data in Hadoop


      Engineers                         Data Scientists                      Quants       Business Analysts




                                                      Aster MapReduce Portfolio       Teradata Analytics Portfolio
        Custom Code and
          Development

                                                             SQL & MapReduce                    SQL

           MR, Pig, Hive
                                                          Teradata Aster                      Teradata
            IT is the optimizer                         MapReduce Platform                      IDW



                                                          HDFS




 14   Confidential and proprietary. Copyright © 2012 Teradata Corporation.
Analytics on Hadoop Data


      Engineers                         Data Scientists                      Quants        Business Analysts




                              Aster MapReduce Portfolio
                                           Aster MapReduce Portfolio                  Teradata Analytics Portfolio




                                    SQL & MapReduce& MapReduce
                                               SQL                                                SQL
                                                                                                  SQL



                                                          Teradata Aster                       Teradata
                                                        MapReduce Platform                       IDW


                                                          HDFS




 15   Confidential and proprietary. Copyright © 2012 Teradata Corporation.
What’s Technically Different in Big Data Analytics
Variety of data types requires different schemas
•  Data that uses a stable schema (structured)
     -  Data from packaged business processes with well-defined & known attributes
        (e.g., ERP data, Inventory Records, Supply Chain records, …)


•  Data that has an evolving schema (semi-structured)
     -  Data generated by machine processes; known but changing set of attributes
        (e.g., Web logs, CDRs, Sensor logs, JSON, Social profiles, Twitter feeds, …)


•  Data that has a format, but no schema (unstructured)
     -  Data captured by machines with well-defined format, but no semantics
        (e.g., images, videos, web pages, PDF documents, …)
     -  Semantics can be extracted from raw data by interpreting the format and
        extracting semantics
        (e.g., shapes from video, face recognition in images, logo detection, …)
     -  Sometimes format data is accompanied by meta-data that can have (Stable
        Schema or Evolving Schema) – that needs to be classified and treated
        separately

16      Confidential and proprietary. Copyright © 2012 Teradata Corporation.
When to Use Which?
 The best approach by workload and data type
 Processing as a Function of Schema Requirements by Data Type

                                                        Loading and Refining
                    Low Cost                                                                                       Analytics
                    Storage &            Data Pre-Processing,                                      Reporting     (User-driven,
                    Retention              Prep, Cleansing                      Transformations                   interactive)




Stable             Teradata /                                                                                     Teradata
                                                   Teradata                        Teradata        Teradata
Schema              Hadoop                                                                                      (SQL analytics)




                                                                                      Aster                          Aster
Evolving                                            Aster /
                      Hadoop                                                      (joining with      Aster     (SQL + MapReduce
Schema                                              Hadoop
                                                                                structured data)                   Analytics)




                                                                                                                   Aster
Format,
                      Hadoop                        Hadoop                         Hadoop                        (MapReduce
No Schema
                                                                                                                  Analytics)



  17     Confidential and proprietary. Copyright © 2012 Teradata Corporation.
Architecture Flexibility – Stable Schema


      Solid State
        Drives

       300-600
       GB drives

     High Capacity
         Drives


                            Extreme Data                  Data Warehouse         Active Enterprise Data
                              Appliance                      Appliance                Warehouse

     Low Cost Storage
       & Retention
     Load, Data Prep &
         Refining

      Transformation

                                                                                Low Latency, Minimize Data
                           High volume data                  CPU Intense
                                                                                  Movement/Complexity,
         Benefits             storage, light          transformations, medium
                                                                                 transformation aligned to
                            transformations              volume data storage
                                                                                      reference data

                                                       Automatic compression
       Compression       Software compression                                         Compress on cold
                                                             engines

18     List Price/TB
          5/29/12                $4K                                $11K
                                               Teradata Copyright ©2012                      $30K*
                                                                                * price/TB on cold storage only
Unified Big Data Architecture and Data Flow
Enabling a Data-Driven Business

     Transaction Architecture

        Traditional
       Data Sources
                                                                                                                                       Business
                                      ETL
                                                                               SQL Analytics                                          Applications




                                                                                  Dimensional Data

                                                                                                     Analytic Results
     Interaction Architecture

       Multi-Structured                                                                                                                Analytic
          Raw Data                                                                                                                      Tools
                                                                                                                                       & Users
       Sensors, Scientific                                           Unified Analytic Access
      and Geospatial Data

                                                                                                                         Iterative
                                                                Store &                                                 Discovery
           Social Media                                         Refine                                                  & Analytics

                                                                Unified Big Data Architecture



19      Confidential and proprietary. Copyright © 2012 Teradata Corporation.
Twitter Tag: #briefr
Thinking Beyond the
     Enterprise Data Warehouse

                                 Richard Hackathorn
                                   richardh@bolder.com




© Bolder Technology, Inc. 2012
A New Ballgame!

•  Big Data is forcing us to rethink the goals
      and architecture for data warehousing

•  Traditional EDW is no longer sufficient
        §    Exclusive collection of corporate information
        §    Striving toward a single version of truth
        §    Only structured data has business value
        §    Predefined questions are the norm

•  We are now facing a new set of issues!

© Bolder Technology, Inc. 2012                      Slide 23
Issue: Exclusive to Inclusive

•  All data can not be managed within the
      boundaries of the EDW
        §    Too much and too fast
        §    Too complex and changing
        §    Controlled by others
        §    New data sources are critical
        §    Short-lived data sources are also critical
•  Need to be more agile, flexible, responsive
•  Requiring ‘smart’ curating of new sources
        §  What should be captured, stored, and retained?
•  Requiring ‘smart’ data exploration

© Bolder Technology, Inc. 2012                       Slide 24
Issue: Ever-Changing Multiple Truths

•  “More things in heaven and earth than
      are dreamt of in your philosophy”
        §  IOW we do not know what we do not know!

•  Example: multiple personalities for the
      same customer

•  Business semantic analysis is critical and
      continuous activity



© Bolder Technology, Inc. 2012               Slide 25
Issue: Discovering Structure

•  Need for a constant refining of all data
        §  Constantly maturing data by enhancing,
            compressing, and structuring

•  Business value comes from leveraging
      structured data into process variations
        §  What do you do differently with what you know?
        §  Analytics and data mining add structure




© Bolder Technology, Inc. 2012                 Slide 26
•    An interesting (and seldom discussed) facet of Big Data is the
           emerging applications that are NOT social networking analytics on
           web logs and website behaviors. What are the ‘killer’ apps in this
           area? Do they involve the “Internet of Things”?

      •    Big Data is big in volume and in variety. It is also big in velocity.
           There is a lot per second…per minute…per day. How should a
           unifying architecture handle the velocity of Big Data?

      •    Many are trying to “Capture in case it is needed” as their approach
           to Big Data. But, can you capture all the data? At what point does
           cost of data capture/storage exceed the business benefits? How do
           you decide what to capture, store, and retain?

      •    Data exploration is an increasingly popular term. How does it differ
           from data analysis? Can you really find useful information through
           data exploration when you do not know what you are looking for?
           Examples?

Twitter Tag: #briefr
•    When you unify the architecture for Big Data (as contrasted with
           isolated islands of Big Data applications), the data needs to move
           through several physical stores. Given the volume and velocity of
           data flows, can/should Big Data be duplicated in multiple stores?

      •    What is the difference between the Hadoop (Hive, etc) system and
           the Teradata Aster system? Could you use both for analytics? Do you
           need both in your unifying architecture?

      •    Are the ‘traditional’ BI tools (like BusinessObjects, Cognos) relevant
           to Big Data analytics? Are they needed in companies that are heavily
           Big Data? Are they evolving and expanding to incorporate the new
           approaches and techniques required for Big Data?

      •    A key requirement in any unifying Big Data architecture is managing
           the complexity of schemas. It seems that we need a new generation
           of semantic analysis tools to assist with schema management. What
           tools are emerging to support this requirement?

Twitter Tag: #briefr
•    Gregory Piatetsky-Shapiro of KDnuggets ran a recent poll on the
           largest dataset that his audience of data miners has so far analyzed.
           The median size for 2012 was in the range 10-100 GB. If most of the
           data for half of the analytics projects can fit into main memory on a
           server platform, why is there such a need for expensive
           architectures supporting MPP, MapReduce, and the like?

        •    http://www.kdnuggets.com/polls/2012/largest-dataset-analyzed-
             data-mined.html




Twitter Tag: #briefr
!   June: Intelligence
     !   July: Disruption
     !   August: Analytics
     !   September: Integration
     !   October: Database
     !   November: Cloud

Twitter Tag: #briefr
The Comprehensive Approach: A Unified Information Architecture

Más contenido relacionado

La actualidad más candente

Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831
Cana Ko
 
Rubik Solutions - Open Integration Portal
Rubik Solutions - Open Integration PortalRubik Solutions - Open Integration Portal
Rubik Solutions - Open Integration Portal
viviankap
 
01 im overview high level
01 im overview high level01 im overview high level
01 im overview high level
James Findlay
 
E12 Sox And Identity Management
E12 Sox And Identity ManagementE12 Sox And Identity Management
E12 Sox And Identity Management
Alexandre Luna
 
Gartner Session Final Prez October 2012
Gartner Session Final Prez October 2012Gartner Session Final Prez October 2012
Gartner Session Final Prez October 2012
Rebecca Croucher
 
Rubik Open Integration Portal
Rubik Open Integration PortalRubik Open Integration Portal
Rubik Open Integration Portal
bob_ark
 
BI Forum 2009 - BI Mega Trends
BI Forum 2009 - BI Mega TrendsBI Forum 2009 - BI Mega Trends
BI Forum 2009 - BI Mega Trends
OKsystem
 

La actualidad más candente (19)

B13 Driving Business Intelligence
B13 Driving Business IntelligenceB13 Driving Business Intelligence
B13 Driving Business Intelligence
 
Empowering the Business with Agile Analytics
Empowering the Business with Agile AnalyticsEmpowering the Business with Agile Analytics
Empowering the Business with Agile Analytics
 
Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831
 
When Worlds Collide: Intelligence, Analytics and Operations
When Worlds Collide: Intelligence, Analytics and OperationsWhen Worlds Collide: Intelligence, Analytics and Operations
When Worlds Collide: Intelligence, Analytics and Operations
 
Introduccion a SQL Server Master Data Services
Introduccion a SQL Server Master Data ServicesIntroduccion a SQL Server Master Data Services
Introduccion a SQL Server Master Data Services
 
Knowledgelevers expanded
Knowledgelevers expandedKnowledgelevers expanded
Knowledgelevers expanded
 
SmartData - Monetizing Data Assets
SmartData - Monetizing Data AssetsSmartData - Monetizing Data Assets
SmartData - Monetizing Data Assets
 
Rubik Solutions - Open Integration Portal
Rubik Solutions - Open Integration PortalRubik Solutions - Open Integration Portal
Rubik Solutions - Open Integration Portal
 
01 im overview high level
01 im overview high level01 im overview high level
01 im overview high level
 
From Big Legacy Data to Insight: Lessons Learned Creating New Value from a Bi...
From Big Legacy Data to Insight: Lessons Learned Creating New Value from a Bi...From Big Legacy Data to Insight: Lessons Learned Creating New Value from a Bi...
From Big Legacy Data to Insight: Lessons Learned Creating New Value from a Bi...
 
Datamine corporate profile
Datamine corporate profileDatamine corporate profile
Datamine corporate profile
 
E12 Sox And Identity Management
E12 Sox And Identity ManagementE12 Sox And Identity Management
E12 Sox And Identity Management
 
Gartner Session Final Prez October 2012
Gartner Session Final Prez October 2012Gartner Session Final Prez October 2012
Gartner Session Final Prez October 2012
 
Rubik Open Integration Portal
Rubik Open Integration PortalRubik Open Integration Portal
Rubik Open Integration Portal
 
Rubik Open Integration Portal
Rubik Open Integration PortalRubik Open Integration Portal
Rubik Open Integration Portal
 
The Digital Intelligence Imperative — Driving Digital Customer Experiences W...
 The Digital Intelligence Imperative — Driving Digital Customer Experiences W... The Digital Intelligence Imperative — Driving Digital Customer Experiences W...
The Digital Intelligence Imperative — Driving Digital Customer Experiences W...
 
BI Forum 2009 - BI Mega Trends
BI Forum 2009 - BI Mega TrendsBI Forum 2009 - BI Mega Trends
BI Forum 2009 - BI Mega Trends
 
Mobile Analytics
Mobile AnalyticsMobile Analytics
Mobile Analytics
 
4.4.2013 Software, System, & IT Architecture - Good Design is Good Business:...
4.4.2013  Software, System, & IT Architecture - Good Design is Good Business:...4.4.2013  Software, System, & IT Architecture - Good Design is Good Business:...
4.4.2013 Software, System, & IT Architecture - Good Design is Good Business:...
 

Similar a The Comprehensive Approach: A Unified Information Architecture

Manthan biim services and solutions
Manthan   biim services  and solutionsManthan   biim services  and solutions
Manthan biim services and solutions
Jaikumar Karuppannan
 
Intel Cloud Summit: Big Data
Intel Cloud Summit: Big DataIntel Cloud Summit: Big Data
Intel Cloud Summit: Big Data
IntelAPAC
 
Hortonworks roadshow
Hortonworks roadshowHortonworks roadshow
Hortonworks roadshow
Accenture
 
Tackling big data with hadoop and open source integration
Tackling big data with hadoop and open source integrationTackling big data with hadoop and open source integration
Tackling big data with hadoop and open source integration
DataWorks Summit
 

Similar a The Comprehensive Approach: A Unified Information Architecture (20)

Unified big data architecture
Unified big data architectureUnified big data architecture
Unified big data architecture
 
Enterprise Services Solutions
Enterprise Services SolutionsEnterprise Services Solutions
Enterprise Services Solutions
 
The Next Generation of Big Data Analytics
The Next Generation of Big Data AnalyticsThe Next Generation of Big Data Analytics
The Next Generation of Big Data Analytics
 
Teradata Big Data London Seminar
Teradata Big Data London SeminarTeradata Big Data London Seminar
Teradata Big Data London Seminar
 
Left Brain, Right Brain: How to Unify Enterprise Analytics
Left Brain, Right Brain: How to Unify Enterprise AnalyticsLeft Brain, Right Brain: How to Unify Enterprise Analytics
Left Brain, Right Brain: How to Unify Enterprise Analytics
 
2012 06 hortonworks paris hug
2012 06 hortonworks paris hug2012 06 hortonworks paris hug
2012 06 hortonworks paris hug
 
vBACD July 2012 - Apache Hadoop, Now and Beyond
vBACD July 2012 - Apache Hadoop, Now and BeyondvBACD July 2012 - Apache Hadoop, Now and Beyond
vBACD July 2012 - Apache Hadoop, Now and Beyond
 
EDF2013: Selected Talk: Bryan Drexler: The 80/20 Rule and Big Data
EDF2013: Selected Talk: Bryan Drexler: The 80/20 Rule and Big Data EDF2013: Selected Talk: Bryan Drexler: The 80/20 Rule and Big Data
EDF2013: Selected Talk: Bryan Drexler: The 80/20 Rule and Big Data
 
Talend Open Studio and Hortonworks Data Platform
Talend Open Studio and Hortonworks Data PlatformTalend Open Studio and Hortonworks Data Platform
Talend Open Studio and Hortonworks Data Platform
 
Manthan biim services and solutions
Manthan   biim services  and solutionsManthan   biim services  and solutions
Manthan biim services and solutions
 
Barak regev
Barak regevBarak regev
Barak regev
 
Microsoft SQL Server 2012 Master Data Services
Microsoft SQL Server 2012 Master Data ServicesMicrosoft SQL Server 2012 Master Data Services
Microsoft SQL Server 2012 Master Data Services
 
Intel Cloud Summit: Big Data
Intel Cloud Summit: Big DataIntel Cloud Summit: Big Data
Intel Cloud Summit: Big Data
 
[Webinar] Drawing insights from social media
[Webinar] Drawing insights from social media[Webinar] Drawing insights from social media
[Webinar] Drawing insights from social media
 
Hortonworks roadshow
Hortonworks roadshowHortonworks roadshow
Hortonworks roadshow
 
Big Data LDN 2018: THE THIRD REVOLUTION IN ANALYTICS
Big Data LDN 2018: THE THIRD REVOLUTION IN ANALYTICSBig Data LDN 2018: THE THIRD REVOLUTION IN ANALYTICS
Big Data LDN 2018: THE THIRD REVOLUTION IN ANALYTICS
 
Tackling big data with hadoop and open source integration
Tackling big data with hadoop and open source integrationTackling big data with hadoop and open source integration
Tackling big data with hadoop and open source integration
 
Scaling MySQL: Catch 22 of Read Write Splitting
Scaling MySQL: Catch 22 of Read Write SplittingScaling MySQL: Catch 22 of Read Write Splitting
Scaling MySQL: Catch 22 of Read Write Splitting
 
B13 Driving Business Intelligence John Robson
B13 Driving Business Intelligence John RobsonB13 Driving Business Intelligence John Robson
B13 Driving Business Intelligence John Robson
 
The New Enterprise Data Platform
The New Enterprise Data PlatformThe New Enterprise Data Platform
The New Enterprise Data Platform
 

Más de Inside Analysis

Rethinking Data Availability and Governance in a Mobile World
Rethinking Data Availability and Governance in a Mobile WorldRethinking Data Availability and Governance in a Mobile World
Rethinking Data Availability and Governance in a Mobile World
Inside Analysis
 

Más de Inside Analysis (20)

An Ounce of Prevention: Forging Healthy BI
An Ounce of Prevention: Forging Healthy BIAn Ounce of Prevention: Forging Healthy BI
An Ounce of Prevention: Forging Healthy BI
 
Agile, Automated, Aware: How to Model for Success
Agile, Automated, Aware: How to Model for SuccessAgile, Automated, Aware: How to Model for Success
Agile, Automated, Aware: How to Model for Success
 
First in Class: Optimizing the Data Lake for Tighter Integration
First in Class: Optimizing the Data Lake for Tighter IntegrationFirst in Class: Optimizing the Data Lake for Tighter Integration
First in Class: Optimizing the Data Lake for Tighter Integration
 
Fit For Purpose: Preventing a Big Data Letdown
Fit For Purpose: Preventing a Big Data LetdownFit For Purpose: Preventing a Big Data Letdown
Fit For Purpose: Preventing a Big Data Letdown
 
To Serve and Protect: Making Sense of Hadoop Security
To Serve and Protect: Making Sense of Hadoop Security To Serve and Protect: Making Sense of Hadoop Security
To Serve and Protect: Making Sense of Hadoop Security
 
The Hadoop Guarantee: Keeping Analytics Running On Time
The Hadoop Guarantee: Keeping Analytics Running On TimeThe Hadoop Guarantee: Keeping Analytics Running On Time
The Hadoop Guarantee: Keeping Analytics Running On Time
 
Introducing: A Complete Algebra of Data
Introducing: A Complete Algebra of DataIntroducing: A Complete Algebra of Data
Introducing: A Complete Algebra of Data
 
The Role of Data Wrangling in Driving Hadoop Adoption
The Role of Data Wrangling in Driving Hadoop AdoptionThe Role of Data Wrangling in Driving Hadoop Adoption
The Role of Data Wrangling in Driving Hadoop Adoption
 
Ahead of the Stream: How to Future-Proof Real-Time Analytics
Ahead of the Stream: How to Future-Proof Real-Time AnalyticsAhead of the Stream: How to Future-Proof Real-Time Analytics
Ahead of the Stream: How to Future-Proof Real-Time Analytics
 
All Together Now: Connected Analytics for the Internet of Everything
All Together Now: Connected Analytics for the Internet of EverythingAll Together Now: Connected Analytics for the Internet of Everything
All Together Now: Connected Analytics for the Internet of Everything
 
Goodbye, Bottlenecks: How Scale-Out and In-Memory Solve ETL
Goodbye, Bottlenecks: How Scale-Out and In-Memory Solve ETLGoodbye, Bottlenecks: How Scale-Out and In-Memory Solve ETL
Goodbye, Bottlenecks: How Scale-Out and In-Memory Solve ETL
 
The Biggest Picture: Situational Awareness on a Global Level
The Biggest Picture: Situational Awareness on a Global LevelThe Biggest Picture: Situational Awareness on a Global Level
The Biggest Picture: Situational Awareness on a Global Level
 
Structurally Sound: How to Tame Your Architecture
Structurally Sound: How to Tame Your ArchitectureStructurally Sound: How to Tame Your Architecture
Structurally Sound: How to Tame Your Architecture
 
SQL In Hadoop: Big Data Innovation Without the Risk
SQL In Hadoop: Big Data Innovation Without the RiskSQL In Hadoop: Big Data Innovation Without the Risk
SQL In Hadoop: Big Data Innovation Without the Risk
 
The Perfect Fit: Scalable Graph for Big Data
The Perfect Fit: Scalable Graph for Big DataThe Perfect Fit: Scalable Graph for Big Data
The Perfect Fit: Scalable Graph for Big Data
 
A Revolutionary Approach to Modernizing the Data Warehouse
A Revolutionary Approach to Modernizing the Data WarehouseA Revolutionary Approach to Modernizing the Data Warehouse
A Revolutionary Approach to Modernizing the Data Warehouse
 
The Maturity Model: Taking the Growing Pains Out of Hadoop
The Maturity Model: Taking the Growing Pains Out of HadoopThe Maturity Model: Taking the Growing Pains Out of Hadoop
The Maturity Model: Taking the Growing Pains Out of Hadoop
 
Rethinking Data Availability and Governance in a Mobile World
Rethinking Data Availability and Governance in a Mobile WorldRethinking Data Availability and Governance in a Mobile World
Rethinking Data Availability and Governance in a Mobile World
 
DisrupTech - Dave Duggal
DisrupTech - Dave DuggalDisrupTech - Dave Duggal
DisrupTech - Dave Duggal
 
Modus Operandi
Modus OperandiModus Operandi
Modus Operandi
 

Último

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Último (20)

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 

The Comprehensive Approach: A Unified Information Architecture

  • 1.
  • 3. !   Reveal the essential characteristics of enterprise software, good and bad !   Provide a forum for detailed analysis of today s innovative technologies !   Give vendors a chance to explain their product to savvy analysts !   Allow audience members to pose serious questions... and get answers! Twitter Tag: #briefr
  • 4. !   May: Analytics !   June: Intelligence !   July: Disruption !   August: Analytics !   September: Integration !   October: Database Twitter Tag: #briefr
  • 5. !   Analytics is, and always has been, about discovering insights that lead to better business decisions. The range of technologies and use cases that inhabit this area is wide: statistical analysis, data and process mining, predictive analytics and modeling, and complex event processing. !   What is now referred to as Big Data has pushed analytics beyond the capabilities of traditional solutions. “Big Analytics” has organizations diving into large heaps of data that previously was not available or usable. !   The growing volume, variety, velocity and complexity of data has proven to be a major challenge to organizations who leverage analytics to maintain a competitive edge. Twitter Tag: #briefr
  • 6. Dr. Richard Hackathorn is a well-known industry analyst, technology innovator and international educator. He has pioneered innovations in database management, decision support and data warehousing. Richard has published numerous articles, presented at leading industry conferences, and conducted professional seminars in eighteen countries. He has written three books, entitled Enterprise Database Connectivity, Using the Data Warehouse (with William H. Inmon), and Web Farming for the Data Warehouse. Richard taught at the Wharton School and at the University of Colorado. Twitter Tag: #briefr
  • 7. !   Teradata is known for its analytic data solutions with a focus on integrated data warehousing, big data analytics and business applications. !   It offers a broad suite of technology platforms and solutions, and a wide range of data management applications and data mining capabilities. !   Teradata features Teradata Aster is its MapReduce platform to handle big data and big analytics on multi-structured data. Twitter Tag: #briefr
  • 8. Chris Twogood is Vice President of Product and Services Marketing for Teradata Corporation. He is responsible for marketing products (database, utilities, and platform), and services (professional and customer services), plus technical field sales support. Chris has twenty- five years of experience in the computer industry specializing in Data Warehousing, Decision Support, Customer Management and Appliance platforms. Chris has held roles that span Strategy, Application Definition, Marketing, Product Requirements/ Management, Platform Solutions and Product Marketing. Twitter Tag: #briefr
  • 9. Unified Big Data Architecture
  • 10. Big Data: From Transactions to Interactions BIG DATA User Generated Social Network Content Mobile Web External User Click Stream Sentiment Demographics Web logs WEB A/B testing Business Data Feeds Offer history Dynamic Pricing HD Video Affiliate Networks CRM Speech to Text Segmentation Search marketing Offer details Product/Service Logs ERP Behavioral Targeting Customer Touches Purchase detail Purchase record Support Contacts Dynamic Funnels SMS/MMS Payment record Increasing data variety and complexity 10 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 11. Unified Big Data Architecture Bridging Classic & Big Data Worlds Classic BI Structured & Repeatable Analysis Business determines what IT structures the data to questions to ask answer those questions SQL performance and structure “Capture only what’s needed” MapReduce processing flexibility IT delivers a platform for Big Data Analytics storing, refining, and Business explores data for Multi-structured & Iterative Analysis analyzing all data sources questions worth answering “Capture in case it’s needed” 11 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 12. Need for a Unified Big Data Architecture for New Insights Enabling All Users for Any Data Type from Data Capture to Analysis Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualization, etc. Reporting and Execution Discover and Explore in the Enterprise Capture, Store and Refine Audio/ Web & Machine Images Docs Text CRM SCM ERP Video Social Logs 12 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 13. Unified Big Data Architecture for the Enterprise Engineers Data Scientists Quants Business Analysts Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualization, etc. ANALYTICS Discovery Platform Active Data Warehouse Capture, Store, Refine Audio/ Web & Machine Images Text CRM SCM ERP Video Social Logs 13 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 14. Analyst’s Goal: Get Insights from Data in Hadoop Engineers Data Scientists Quants Business Analysts Aster MapReduce Portfolio Teradata Analytics Portfolio Custom Code and Development SQL & MapReduce SQL MR, Pig, Hive Teradata Aster Teradata IT is the optimizer MapReduce Platform IDW HDFS 14 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 15. Analytics on Hadoop Data Engineers Data Scientists Quants Business Analysts Aster MapReduce Portfolio Aster MapReduce Portfolio Teradata Analytics Portfolio SQL & MapReduce& MapReduce SQL SQL SQL Teradata Aster Teradata MapReduce Platform IDW HDFS 15 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 16. What’s Technically Different in Big Data Analytics Variety of data types requires different schemas •  Data that uses a stable schema (structured) -  Data from packaged business processes with well-defined & known attributes (e.g., ERP data, Inventory Records, Supply Chain records, …) •  Data that has an evolving schema (semi-structured) -  Data generated by machine processes; known but changing set of attributes (e.g., Web logs, CDRs, Sensor logs, JSON, Social profiles, Twitter feeds, …) •  Data that has a format, but no schema (unstructured) -  Data captured by machines with well-defined format, but no semantics (e.g., images, videos, web pages, PDF documents, …) -  Semantics can be extracted from raw data by interpreting the format and extracting semantics (e.g., shapes from video, face recognition in images, logo detection, …) -  Sometimes format data is accompanied by meta-data that can have (Stable Schema or Evolving Schema) – that needs to be classified and treated separately 16 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 17. When to Use Which? The best approach by workload and data type Processing as a Function of Schema Requirements by Data Type Loading and Refining Low Cost Analytics Storage & Data Pre-Processing, Reporting (User-driven, Retention Prep, Cleansing Transformations interactive) Stable Teradata / Teradata Teradata Teradata Teradata Schema Hadoop (SQL analytics) Aster Aster Evolving Aster / Hadoop (joining with Aster (SQL + MapReduce Schema Hadoop structured data) Analytics) Aster Format, Hadoop Hadoop Hadoop (MapReduce No Schema Analytics) 17 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 18. Architecture Flexibility – Stable Schema Solid State Drives 300-600 GB drives High Capacity Drives Extreme Data Data Warehouse Active Enterprise Data Appliance Appliance Warehouse Low Cost Storage & Retention Load, Data Prep & Refining Transformation Low Latency, Minimize Data High volume data CPU Intense Movement/Complexity, Benefits storage, light transformations, medium transformation aligned to transformations volume data storage reference data Automatic compression Compression Software compression Compress on cold engines 18 List Price/TB 5/29/12 $4K $11K Teradata Copyright ©2012 $30K* * price/TB on cold storage only
  • 19. Unified Big Data Architecture and Data Flow Enabling a Data-Driven Business Transaction Architecture Traditional Data Sources Business ETL SQL Analytics Applications Dimensional Data Analytic Results Interaction Architecture Multi-Structured Analytic Raw Data Tools & Users Sensors, Scientific Unified Analytic Access and Geospatial Data Iterative Store & Discovery Social Media Refine & Analytics Unified Big Data Architecture 19 Confidential and proprietary. Copyright © 2012 Teradata Corporation.
  • 20.
  • 22. Thinking Beyond the Enterprise Data Warehouse Richard Hackathorn richardh@bolder.com © Bolder Technology, Inc. 2012
  • 23. A New Ballgame! •  Big Data is forcing us to rethink the goals and architecture for data warehousing •  Traditional EDW is no longer sufficient §  Exclusive collection of corporate information §  Striving toward a single version of truth §  Only structured data has business value §  Predefined questions are the norm •  We are now facing a new set of issues! © Bolder Technology, Inc. 2012 Slide 23
  • 24. Issue: Exclusive to Inclusive •  All data can not be managed within the boundaries of the EDW §  Too much and too fast §  Too complex and changing §  Controlled by others §  New data sources are critical §  Short-lived data sources are also critical •  Need to be more agile, flexible, responsive •  Requiring ‘smart’ curating of new sources §  What should be captured, stored, and retained? •  Requiring ‘smart’ data exploration © Bolder Technology, Inc. 2012 Slide 24
  • 25. Issue: Ever-Changing Multiple Truths •  “More things in heaven and earth than are dreamt of in your philosophy” §  IOW we do not know what we do not know! •  Example: multiple personalities for the same customer •  Business semantic analysis is critical and continuous activity © Bolder Technology, Inc. 2012 Slide 25
  • 26. Issue: Discovering Structure •  Need for a constant refining of all data §  Constantly maturing data by enhancing, compressing, and structuring •  Business value comes from leveraging structured data into process variations §  What do you do differently with what you know? §  Analytics and data mining add structure © Bolder Technology, Inc. 2012 Slide 26
  • 27. •  An interesting (and seldom discussed) facet of Big Data is the emerging applications that are NOT social networking analytics on web logs and website behaviors. What are the ‘killer’ apps in this area? Do they involve the “Internet of Things”? •  Big Data is big in volume and in variety. It is also big in velocity. There is a lot per second…per minute…per day. How should a unifying architecture handle the velocity of Big Data? •  Many are trying to “Capture in case it is needed” as their approach to Big Data. But, can you capture all the data? At what point does cost of data capture/storage exceed the business benefits? How do you decide what to capture, store, and retain? •  Data exploration is an increasingly popular term. How does it differ from data analysis? Can you really find useful information through data exploration when you do not know what you are looking for? Examples? Twitter Tag: #briefr
  • 28. •  When you unify the architecture for Big Data (as contrasted with isolated islands of Big Data applications), the data needs to move through several physical stores. Given the volume and velocity of data flows, can/should Big Data be duplicated in multiple stores? •  What is the difference between the Hadoop (Hive, etc) system and the Teradata Aster system? Could you use both for analytics? Do you need both in your unifying architecture? •  Are the ‘traditional’ BI tools (like BusinessObjects, Cognos) relevant to Big Data analytics? Are they needed in companies that are heavily Big Data? Are they evolving and expanding to incorporate the new approaches and techniques required for Big Data? •  A key requirement in any unifying Big Data architecture is managing the complexity of schemas. It seems that we need a new generation of semantic analysis tools to assist with schema management. What tools are emerging to support this requirement? Twitter Tag: #briefr
  • 29. •  Gregory Piatetsky-Shapiro of KDnuggets ran a recent poll on the largest dataset that his audience of data miners has so far analyzed. The median size for 2012 was in the range 10-100 GB. If most of the data for half of the analytics projects can fit into main memory on a server platform, why is there such a need for expensive architectures supporting MPP, MapReduce, and the like? •  http://www.kdnuggets.com/polls/2012/largest-dataset-analyzed- data-mined.html Twitter Tag: #briefr
  • 30.
  • 31. !   June: Intelligence !   July: Disruption !   August: Analytics !   September: Integration !   October: Database !   November: Cloud Twitter Tag: #briefr