IBM big data platfrom - Arild Kristensen

1. © 2014 IBM Corporation
Big Data Platform
Arild Kristensen
Nordic Sales Manager, Big Data Analytics
Tlf.: +47 90532591
Email: arild.kristensen@no.ibm.com

2. © 2014 IBM Corporation3

3. © 2014 IBM Corporation4
Welcome to the Big Data Opportunity
“The list of life's certainties has gotten longer.
Along with death and taxes we can now include
information overload.”

4. © 2014 IBM Corporation5
We have for the first time an economy based on a key resource
[Information] that is not only renewable, but self-generating.
Running out of it is not a problem, but drowning in it is.
– John Naisbitt
Source, Megatrends, Naisbitt, John, Grand Central Publishing 1988
We are not suffering from Information Overload. We
are suffering from Filter Failure.
– Clay Shirky
Sourcehttp://www.ted.com/talks/view/lang/en//id/575

5. © 2014 IBM Corporation6
Welcome to the Big Data Opportunity
Research firm IDC expects Big Data to grow from
$3.2 billion in 2010 to $16.9 billion in 2015
by 2015 we'll see 4.4 million jobs devoted to the
global support of Big Data
each IT job created by Big Data will generate
three more positions outside of IT.

6. © 2014 IBM Corporation11
Big Data Analytics And Natural Language
Cognitive: The Next Wave of Disruptive Technology

7. © 2014 IBM Corporation14
Understands
natural language and
human style
communication
Adapts and learns from
training, interaction,
and outcomes
Generates and
evaluates evidence-
based hypothesis
1 2
3
• Understands me
• Engages me
• Learns and improves over time
• Helps me discover
• Establishes trust
• Has endless capacity for insight
• Operates in a timely fashion
Watson combines transformational capabilities to deliver a
new world experience using cognitive computing
Watson:

8. © 2014 IBM Corporation15
IBM Watson
family
IBM Watson
Solutions
IBM Watson
Transformation
IBM Watson
Foundations
IBM Watson
Innovations
Provides the big data and analytics
capabilities that fuel Watson
Products based on
Watson’s core
attributes and
capabilities
APIs, tools, methodologies,
SDKs, and infrastructure that
fuels Watson
Bespoke solutions designed to
meet some of industries most
demanding needs leveraging
cognitive capabilities
IBM Watson
Ecosystems
The Watson Developer Cloud,
Watson Content Store and
Watson Talent Hub driving
innovation from partners
Introducing the IBM Watson family

9. © 2014 IBM Corporation16
How is Big Data transforming the way
organizations analyze information and
generate actionable insights?

10. © 2014 IBM Corporation17
Paradigm shifts enabled by big data
Leverage more of the data being captured
TRADITIONAL APPROACH BIG DATA APPROACH
Analyze small subsets
of information
Analyze
all information
Analyzed
information
All available
information
All available
information
analyzed

11. © 2014 IBM Corporation18
Paradigm shifts enabled by big data
Reduce effort required to leverage data
TRADITIONAL APPROACH BIG DATA APPROACH
Carefully cleanse information
before any analysis
Analyze information as is,
cleanse as needed
Small
amount of
carefully
organized
information
Large
amount of
messy
information

12. © 2014 IBM Corporation19
Paradigm shifts enabled by big data
Data leads the way—and sometimes correlations are good enough
TRADITIONAL APPROACH BIG DATA APPROACH
Start with hypothesis and
test against selected data
Explore all data and
identify correlations
Hypothesis Question
DataAnswer
Data Exploration
CorrelationInsight

13. © 2014 IBM Corporation20
Paradigm shifts enabled by big data
Leverage data as it is captured
TRADITIONAL APPROACH BIG DATA APPROACH
Analyze data after it’s been
processed and landed in a warehouse
or mart
Analyze data in motion as it’s
generated, in real-time
Repository InsightAnalysisData
Data
Insight
Analysis

14. © 2014 IBM Corporation21
Hadoop &
Streaming
Data
New
Sources
Unstructured
Exploratory
Iterative
Structured
Repeatable
Linear
Data
Warehouse
Traditional
Sources
Traditional Approach
Structured, analytical, logical
New Approach
Creative, holistic thought, intuition
Enterprise
Integration
Customer Data
Transaction Data
3rd Party Data
Core System Data
Web Logs, URLs
Social Data
Text Data: emails, chats
Log data
Analytics is expanding from enterprise data to big data,
creating new opportunities for competitive advantage
Contact Center notes
Geolocation data

15. © 2014 IBM Corporation22
Addressing Client Challenges through Big
Data Platform

16. © 2014 IBM Corporation23
A New Architectural Approach is Required
Information Integration & Governance
Systems Security
On premise, Cloud, As a service
Storage
New/Enhanced
Applications
All Data
What action
should I
take?
Decision
management
Landing,
Exploration
and Archive
data zone
EDW and
data mart
zone
Operational
data zone
Real-time Data Processing & Analytics What is
happening?
Discovery and
exploration
Why did it
happen?
Reporting and
analysis
What could
happen?
Predictive
analytics and
modeling
Deep
Analytics
data zone What did
I learn,
what’s best?
Cognitive

17. © 2014 IBM Corporation24
Information Integration & Governance
Actionable insight
Exploration,
landing and
archive
Trusted data
Reporting &
interactive
analysis
Deep
analytics &
modeling
Data types Real-time processing & analytics
Transaction and
application data
Machine and
sensor data
Enterprise
content
Social data
Image and video
Third-party data
Decision
management
Predictive analytics
and modeling
Reporting,
analysis, content
analytics
Discovery and
exploration
Operational
systems
Information
Integration
Data Matching
& MDM
Security &
Privacy
Lifecycle
Management
Metadata &
Lineage
IBM Big Data Analytics (Watson Foundations) - One architecture
that fits together
BigInsights
Streams
PureData
for
Analytics
DB2 Blu
Watson
Explorer
Cognos
Cognos
SPSSPureData
for
Analytics
PureData
Operational
Analytics

18. © 2014 IBM Corporation25
InfoSphere
DataStage
Automatically push transformational processing close to where the
data resides, both SQL for DBMS and MapReduce for Hadoop,
leveraging the same simple data flow design process and coordinate
workflow across all platforms
“Big Data Expert”

19. © 2014 IBM Corporation
IBM InfoSphere Streams:
Get real-time insights from data in-motion

20. © 2014 IBM Corporation27
27
Current fact finding
Analyze data in motion – before it is stored
Low latency paradigm, push model
Data driven – bring data to the analytics
Historical fact finding
Find and analyze information stored on disk
Batch paradigm, pull model
Query-driven: submits queries to static data
Traditional Computing Stream Computing
Stream Computing Represents a Paradigm Shift
Real-time
Analytics

21. © 2014 IBM Corporation28
28
Modify
Filter / Sample
Classify
Fuse
Annotate
Big Data in Real Time with InfoSphere Streams
Score
Windowed
Aggregates
Analyze

22. © 2014 IBM Corporation29
29
Streams Analyzes All Variety of Data
Mining in Microseconds
(included with Streams)
Image & Video
(Open Source)
Simple & Advanced Text
(included with Streams)
Text
(listen, verb),
(radio, noun)
Acoustic
(IBM Research)
(Open Source)
Geospatial
(Included with
Streams)
Predictive
(Included with
Streams)
Advanced
Mathematical
Models
(Included with
Streams)
Statistics
(included with
Streams)
∑population
tt asR ),(
Blue = included with the product
Red = built for Streams and used in
projects but not yet part of the product

23. © 2014 IBM Corporation30
30
How is Streams Being Used?
Stock market
Impact of weather on
securities prices
Analyze market data at
ultra-low latencies
Momentum Calculator
Fraud prevention
Detecting multi-party fraud
Real time fraud prevention
e-Science
Space weather prediction
Detection of transient events
Synchrotron atomic research
Genomic Research
Transportation
Intelligent traffic
management
Automotive Telematics
Energy & Utilities
Transactive control
Phasor Monitoring Unit
Down hole sensor monitoring
Natural Systems
Wildfire management
Water management
Other
Manufacturing
Text Analysis
ERP for Commodities
Real-time multimodal surveillance
Situational awareness
Cyber security detection
Law Enforcement,
Defense & Cyber Security
Health & Life
SciencesICU monitoring
Epidemic early
warning system
Remote healthcare
monitoring
Telephony
CDR processing
Social analysis
Churn prediction
Geomapping

24. © 2014 IBM Corporation
Watson (Data) Explorer
IBM Software Group
Information Management
Big Data

25. © 2014 IBM Corporation32
Watson Explorer solves #1 challenge customers face in Big Data:
Unlocking the value of information through a single interface
Create unified view of
ALL information for
real-time monitoring
Identify areas of information
risk & ensure data
compliance
Analyze customer analytics
& data to unlock true
customer value
Increase productivity &
leverage past work
increasing speed to market
Improve customer
service & reduce
call times
InfoSphere
Data Explorer
• Analyzes structured &
unstructured data—in place
• Unique positional indexing
• Unlimited scalability
• Advanced data asset navigation
• Pattern clustering
• Virtual documents
Contextual intelligence
• Text analytics
• Secure data integration
• Query transformation
• Easy-to-deploy big data applications
• User-friendly customisable interface
Providing unified, real-time
access and fusion of big
data unlocks greater
insight and ROI
Zoom in
Zoom out
12/05/201432

26. © 2014 IBM Corporation33
Watson Explorer Application Architecture
User Profiles
360O View
Applications
Information
Discovery
Applications
Big Data
Applications
Discovery &
navigation
applications
Web
Results
FeedsSubscriptions
Federated Query Routing
Application Framework
Authentication/Authorization
Query transformation
Personalization
Display
Meta-Data
User Profiles
Application layer
managing user
interactions, apps,
creating context,
routing queries
Thesauri
Clustering
Ontology Support
Semantic Processing
Entity Extraction
Relevancy
Text Analytics
Search Engine Metadata Extraction
Faceting
BI
Tagging
Taxonomy
Collaboration
Processing layer
for indexing,
analysis &
conversion
CM, RM, DM RDBMS Feeds Web 2.0 Email Web CRM, ERP File
Systems
Connector
Framework
Framework for
accessing data
sources
12/05/201433

27. © 2014 IBM Corporation34
Highly relevant, secure &
personalized results
Access all sources
or individual source
Refinements based
on metadata
Dynamic
categorization
Narrow down results set
Information Navigation, Discovery & Insight Through One Interface
Live link here
Setup alert to
notify change
Identify topical experts
Tag results
Rate results
Comment results
Store &
share results

28. © 2014 IBM Corporation35
Big Data Use cases

29. © 2014 IBM Corporation36
Top sources of information used as part of initial big data efforts –
typically start with data already being captured
Source: The real world use of Big Data, IBM
& University of Oxford
Big data sources
Respondents with active big data efforts were asked which data sources are
currently being collected and analyzed as part of active big data efforts within
their organization.
88%
73%
59%
57%
43%
42%
42%
41%
41%
40%
38%
34%
92%
81%
70%
65%
27%
19%
36%
47%
32%
0%
21%
22%
Transactions
LogData
Events
Emails
Social Media
Sensors
External Feeds
RFID Scans or POS Data
Free-formText
Geospatial
Audio
Still Images / Videos
Banking & Fin Mgmt
respondents
Global respondents
3
6

30. © 2014 IBM Corporation37
Big Data Exploration
Find, visualize, and understand
all big data for improved decision
making
Enhanced 360o View
of the Customer
View all internal and external
information sources to know
everything about your customers
Operations Analysis
Analyze a variety of machine data
for improved business results
Data Warehouse
Modernization
Modernize the data warehouse with
new technology: in-memory, stream
computing, Hadoop, appliances,
while building confidence in all data
Security Intelligence
Extension
Lower risk, detect fraud and
monitor cyber security in real-time
Big Data Use Cases

31. © 2014 IBM Corporation38
Arild Kristensen IBM Norway
Nordic Sales Manager Forusbeen 10
Big Data Analytics 4033 Stavanger
IBM Software Group Mobile: +47 90 53 25 91
Information Management arild.kristensen@no.ibm.com
linkedin.com/pub/arild-
kristensen/34/96b/184
twitter.com/ArildWK
www.ibmbigdatahub.com
www.analyticszone.com