4. ■ There are some things that are so big that
they have implications for everyone,
whether we want it or not.
■ Big Data is one of those things, and is
completely transforming the way we do
business and is impacting most other parts
of our lives.
5. ■ The basic idea behind the phrase 'Big
Data' is that everything we do is
increasingly leaving a digital trace (or
data), which we (and others) can use
and analyse.
■ Big Data therefore refers to our ability to
make use of the ever-increasing
volumes of data.
6. ■ Big Data may well be the Next BigThing in the IT world.
■ Big data burst upon the scene in the first decade of the 21st century.
■ The first organizations to embrace it were online and startup firms.
Firms like Google, eBay, LinkedIn, and Facebook were built around big
data from the beginning.
■ Like many new information technologies, big data can bring about
dramatic cost reductions, substantial improvements in the time
required to perform a computing task, or new product and service
offerings.
7. From the dawn of civilization until
2003, humankind generated five
exabytes of data. Now we produce
five exabytes every two days…and
the pace is accelerating.
Eric Schmidt,
Executive Chairman, Google
8. How much data?
• Google processes 20 PB a day
• Wayback Machine has 3 PB + 100TB/month
• Facebook has 2.5 PB of user data + 15TB/day
• eBay has 6.5 PB of user data + 50TB/day
• CERN’s Large Hydron Collider (LHC) generates 15 PB a
year
640K ought to be
enough for anybody.
9. Walmart handles more than 1 million customer
transactions every hour.
Facebook handles 40 billion photos from its user base.
Decoding the human genome originally took 10years to
process; now it can be achieved in one week.
12. Volume:
■ The amount of data. While volume indicates more data, it is the granular
nature of the data that is unique. Big data requires processing high
volumes of low-density, unstructured Hadoop data—that is, data of
unknown value, such asTwitter data feeds, click streams on a web page
and a mobile app, network traffic, sensor-enabled equipment capturing
data at the speed of light, and many more.
■ It is the task of big data to convert such Hadoop data into valuable
information. For some organizations, this might be tens of terabytes,
for others it may be hundreds of petabytes.
13. 1st Character of Big Data
Volume
•A typical PC might have had 10 gigabytes of storage in 2000.
•Today, Facebook ingests 500 terabytes of new data every day.
•Boeing 737 will generate 240 terabytes of flight data during a single
flight across the US.
•The smart phones, the data they create and consume; sensors
embedded into everyday objects will soon result in billions of new,
constantly-updated data feeds containing environmental, location,
and other information, including video.
14. Velocity
■ The fast rate at which data is received and perhaps acted upon.The
highest velocity data normally streams directly into memory versus
being written to disk.
■ Some Internet ofThings (IoT) applications have health and safety
ramifications that require real-time evaluation and action. Other
internet-enabled smart products operate in real time or near real time.
For example, consumer eCommerce applications seek to combine
mobile device location and personal preferences to make time-sensitive
marketing offers.
■ Operationally, mobile application experiences have large user
populations, increased network traffic, and the expectation for
immediate response
15. 2nd Character of Big Data
Velocity
■ Click streams and ad impressions capture user behavior at millions
of events per second
■ high-frequency stock trading algorithms reflect market changes
within microseconds
■ machine to machine processes exchange data between billions of
devices
■ infrastructure and sensors generate massive log data in real-time
■ on-line gaming systems support millions of concurrent users, each
producing multiple inputs per second.
16. Variety
■ New unstructured data types. Unstructured and semi-structured data
types, such as text, audio, and video require additional processing to
both derive meaning and the supporting metadata.
■ Once understood, unstructured data has many of the same
requirements as structured data, such as summarization, lineage,
auditability, and privacy. Further complexity arises when data from a
known source changes without notice.
■ Frequent or real-time schema changes are an enormous burden for both
transaction and analytical environments.
17. 3rd Character of Big Data
Variety
■ Big Data isn't just numbers, dates, and strings. Big
Data is also geospatial data, 3D data, audio and
video, and unstructured text, including log files and
social media.
■ Traditional database systems were designed to
address smaller volumes of structured data, fewer
updates or a predictable, consistent data structure.
■ Big Data analysis includes different types of data
18. Value
■ Data has intrinsic value—but it must be discovered.There are a range of quantitative
and investigative techniques to derive value from data—from discovering a consumer
preference or sentiment, to making a relevant offer by location, or for identifying a
piece of equipment that is about to fail.
■ The technological breakthrough is that the cost of data storage and compute has
exponentially decreased, thus providing an abundance of data from which statistical
analysis on the entire data set versus previously only sample.The technological
breakthrough makes much more accurate and precise decisions possible.
■ However, finding value also requires new discovery processes involving clever and
insightful analysts, business users, and executives.The real big data challenge is a
human one, which is learning to ask the right questions, recognizing patterns, making
informed assumptions, and predicting behavior.
19. Benefits of Big Data
•Real-time big data isn’t just a process for storing
petabytes or Exabyte of data in a data warehouse, It’s
about the ability to make better decisions and take
meaningful actions at the right time.
•Fast forward to the present and technologies like
Hadoop give you the scale and flexibility to store data
before you know how you are going to process it.
•Technologies such as MapReduce,Hive and Impala
enable you to run queries without changing the data
structures underneath.
20. Benefits of Big Data
■ Our newest research finds that organizations are using big data to target
customer-centric outcomes, tap into internal data and build a better
information ecosystem.
■ Big Data is already an important part of the $64 billion database and data
analytics market
■ It offers commercial opportunities of a comparable
scale to enterprise software in the late 1980s
■ And the Internet boom of the 1990s, and the social media explosion of today.
26. ■ Better understand and target customers:
■ To better understand and target customers,
companies expand their traditional data sets with
social media data, browser, text analytics or
sensor data to get a more complete picture of
their customers. The big objective, in many cases,
is to create predictive models. Using big data,
Telecom companies can now better predict
customer churn; retailers can predict what
products will sell, and car insurance companies
understand how well their customers actually
drive.
27. Data collection & analysis
■ Modern marketers can use big data to collect information
on their customer’s purchasing history, personal likes
and dislikes, the time they spend on different websites,
their personal habits, and an enormous range of other
data collection points.
■ Once this data is passed through customer relationship
management systems. (CRM)
■ With careful selection of the right data for the company,
the CRM analysis can bring the business the three
improvements listed above, and provide them with a
greater return on investment.
28.
29. How It Can Help
competitive advantage
reduce costs,
improve performance/ productivity
find potential customers
existing data - more valuable asset
insights about customers/ customer sentiment
clues about market shifts
big picture
transform customers‘ experience
recommendations for better services, products, employees
smarter, quicker, more informed decisions
improve innovation, service creation
identify trends, new business opportunities