Future of big data speaker compendium march 2013

1. Future of Big Data - Compendium Nitin Kabra Page 1 of 3
Challenges and future of big data: March 2013
1) The Rise of the Big Data Discovery Platform will continue: The largest wave of Big Data value creation
is still to come and it will focus on exploiting the infrastructure to create new applications that analytically
optimize business processes
2) Explosive Big Data Application Growth: Internet of things, meet big data
We are fast approaching an era where every device from a car to a fridge to a Kindle is connected to the
Internet - and with the rollout of IPv6, big data is only going to get bigger. Integration between physical and
digital world. All devices will be connected. "Having the data isn't the hard thing. The hard thing is exploring
the data.”
3) Imagining anything as a service: PaaS, DBaas, IaaS (Heroku, Microsoft Azure, Cloudfoundry) trends will
continue to rise and grow exponentially with the adoption of Cloud architecture.
4) Data Lake: Ingesting, distilling, processing and decision-making impact of big data will have maximum
influence moving business intelligence and analytics solution development from periphery of operations to
the center of how business gets done
5) Collaboration at scale= Sales, marketing, finance, IT, Operations, HR will collate. Move business
intelligence and solutions development from the periphery of operations to the center of how business gets
done.
6) Industry will need people who put business, technology and data analysis and decision making together-
build a congregation. People with business knowledge, mathematical modeling or stats understanding, quants
who understand analytical models or project management of analytical solution, Visualization and business
discovery tools.
7) Big Data in Motion: Or Interactive Queries in Hadoop :
Distributed
File System
Batch
processing
Interactive analysis NoSQL
GFS MapReduce Dremel BigTable
HDFS
Hadoop
MapReduce
??? ==APACHE DRILL. open
source solution for interactive
analysis of Big Data. Bottom
Line: Apache Drill enables
NoSQL and SQL Work Side-
by-Side to Tackle Real-time
Big Data Needs.
HBase

2. Future of Big Data - Compendium Nitin Kabra Page 2 of 3
BDAS HDFS Spark
Shark and for Realtime
Analytics –Spark streaming,
MLLib, GraphX
HBase
8) Continuous Expansion and Unification of SQL on Hadoop. A number of technology companies are
working hard to build a layer of technology on non-SQL enabled big data solutions like Hadoop. The depth
and breadth of support for the SQL language varies, but SQL smart professionals will be able to take
advantage of these advances to enable highly interactive SQL on big data. Examples include Hadapt, Impala,
Teradata Aster and Pivotal HAWQ.
9) Explosion of big data real time analytics: the ability to make better decisions and take meaningful actions
at the right time. It’s about detecting fraud while someone is swiping a credit card, or triggering an offer
while a shopper is standing on a checkout line, or placing an ad on a website while someone is reading a
specific article. It’s about combining and analyzing data so you can take the right action, at the right time,
and at the right place.”
10) Technological Convergence: Business intelligence and visualization, SQL, Hadoop-data discovery and
storage, inmemory databases will converge and grow exponentially. Correlation DB may be the order of the
day.
11) Details on Demand: Enterprises cannot afford to wait around for big data to be processed at its own time -
they will need near-real-time results that match the speed of traditional business intelligence.
12) From Fragmentation to a Unified Integrated DATA Architecture: All things will get connected right
from sales to inventory or manufacturing in the next 3 years.
13) Blending: of big data and data warehousing to provide hidden truths of data. They will co-exist.
14) Storage(HDFS, Greenplum, Hbase, MongoDB, Accumulo) is Not Enough-Provide innovative intelligence
and advanced analytics for the data… eg: Why are sales down for a particular region, not just a dashboard
15) Connecting the dots in the data analytics for forecasting and decision making…Mining of data will
reach never seen levels=When you goto Shoprite, you swipe your card. When you goto TRU, you swipe
your loyalty card. All that information is ingested and analyzed.
16) Governance and security will be a concern.
17) FROM TECHNOLOGY SIDE= Added Data Mining and Analytic Functions. Industry leaders in the big
data space understand the requirements to expand the underlying analytics and statistical capabilities in their
platform. This goes beyond typical analytic functions into the world of very sophisticated data mining
functionality. Teradata Aster Data includes a wide variety of analytic capabilities including support for
statistical, text analytics, graph, sentiment analysis and in-database PMML execution through the support of
Zementis. Other companies including IBM Netezza have embedded support for the popular R statistical
language as well as Matrix engine, a parallelized linear algebra package. Over time, we will see a significant
expansion of these capabilities across a broad range of big data solutions.
18) Gains in Popularity of the R Language: There is no doubt that R is becoming more and more popular as
an open statistical language. Revolution Analytics has made significant progress in developing a "production-
grade" version of R with performance enhancements and other enterprise features. Furthermore, they have
developed solutions including R for Hadoop, parallel R, R for IBM PureData as well as R for Big Data.
19) Analytics growth= Text analytics, semantics, NER, narrative science will be the common mode of things.
20) It is getting easier to build BIG DATA applications= Hadoop’s low-cost, scale-out architecture has made
it a new platform for data storage. With a storage system in place, the Hadoop community is slowly building
a collection of open source, analytic engines. Beginning with batch processing (MapReduce, Pig, Hive),
Cloudera has added interactive SQL (Impala), analytics (Cloudera ML + a partnership with SAS), and as of
early this week, real-time search. The economics that led to Hadoop dominating batch processing is
permeating other types of analytics.
Another collection of open source, Hadoop-compatible analytic engines, the Berkeley Data Analytics Stack
(BDAS), is being built just across the San Francisco Bay. Starting with a batch-processing framework that’s

3. Future of Big Data - Compendium Nitin Kabra Page 3 of 3
faster than MapReduce (Spark), it now includes interactive SQL (Shark), and real-time analytics (Spark
Streaming). Sometime this summer, frameworks for machine-learning (MLbase) and graph analytics
(GraphX) will be released. A cluster manager (Mesos) and an in-memory file system (Tachyon) allow users
of other analytic frameworks to leverage the BDAS platform. (The Python data community is looking at
Tachyon closely.)
Talk on Storm, Spark, Druid, CEP combined with Big Data and Cloud.
TO SUM IT UP: Big data – of the people, by the people, for the people: Cloud-based and open source
tools will help democratize big data to take it out of the realm of expensive resources and high-computing
infrastructure - giving even smaller companies the ability to leverage big data for business insights. Making
the network the organization.