Internet of things. Like we didn't have enough "things", yet everything is getting connected lately. IDC projects that the digital universe will reach 40 zettabytes. Even if only a fraction of all of the huge data will need to be processed, that’s a lot of processing power that will need to be available for organizations. How do DBAs prepare for this challenge?
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Internet of Things (IOT) - impact on databases and DBAs
1. The Impact of
The Internet of Things (IOT)
On Databases
The Stuff CIO’s and DBA’s should prepare for
Mordechai Danielov
bitwiseMnm.com
2. “The Internet of Things will augment your
brain”
Eric Schmidt
"The Internet of Things is reaching a
tipping point that will make it a
sustainable paradigm for practical
applications."
Massimiliano Claps, research director, IDC
EMEA Government Insights
3.
4. In 2008, the number of devices on the
Internet already exceeded the number of
people. By 2020, it will reach 50
billion devices.
5. Today, IT is dependent on data created
by people.
With IoT, computers will gather data
independently of humans and track and
count everything.
The next generation of Internet
applications using (IPv6) will
communicate with devices attached to
virtually all human-made objects enabled
by the extremely large address space of
the IPv6 protocol.
6. Big Data DB
Cloud Servers
Devices & “Things”
Technologies that make IoT
Possible
Sensors
(like RFID,
pressure etc)
Send out
information
Applications
analyze and send
instructions back
to devices
7. With IoT,
“Big Data”
will turn to
“Huge Data”
IDC projects that the digital
universe will reach 40 zettabytes
40 ZB is equivalent to 57 times the amount of
all the grains of sand on all the beaches on
earth.
http://www.digitalnewsasia.com/digital-economy/massive-amounts-of-data-but-only-05percent-being-analyzed#sthash.1W4wJcp9.dpuf
8. Processing Power
Even if only a fraction of all of
the huge data will need to be
processed, that’s a lot of
processing power that will need to
be available for organizations.
Decoding the data generated by the
Human Genome took 10 years. Today, it
would take less than 1 week.
http://wikibon.org/blog/big-data-statistics/
9. Peaks and Valleys
Transaction Rate Fluctuations can
create inadvertent DoS situations
Scenarios:
An alarm company
hooked up to many home
devices may get
“sensory overload”
during an earthquake
from multiple
endpoints.
Servers and Databases
will get a high volume
surge of data that
requires speedy
processing.
10. When Scaling up is just not Good Enough
Scale Out Your Resources
-Smart distribution of read/write activity
-Design smart: writing hubs and reading spokes
-Spread out your databases and replicate
Read Only DB
11. Know Thy Peak
30%-70% rule
Utilize no more than
30% of your resources
in “off-peak” time and
reserve 70% for peaks. 30%
12. Distribute, Cache and
Share Nothing
Sharing is caring?
Not with data
Data must be available everywhere
in the world and fast -- highly
distributed databases with local
application caching.
The most popular approach is to
build loosely connected "shared
nothing" instances of databases
that can be brought online in no
time.
Shared Nothing
13. Master the Cloud
It’s all about
the Money
Use Cloud computing to get
a handle on your cost of
computing. Instead of
saving 70% of capacity for
peak traffic, procure it
on demand with Cloud based
databases and smart load
balancing middle tier.
Master working with cloud
vendors and remote
utilities.
Fully
distributed
files?
14. Parallel Processing
Adaptive Architecture
Data Compression
Learn from the Big Guys
Facebook DB Architecture
http://www.flickr.com/photos/ikhnaton2/533233247/
Google BigTable
"It is not a relational
database and can be
better defined as a
sparse, distributed
multi-dimensional sorted
map"
http://en.wikipedia.org/wiki/BigTable
15. Data Must Be Available Under
Adverse Conditions
•Redundant data pathways
•Smart middle layer
•Fully meshed topology
16. Distributed data is more exposed
yet
Data Must Be Secured
•Encrypt anything you can
•Automate Certificate
management
•Know how to secure data in
and out of the cloud and in
between
•Distribute “meaningless” data
and assemble it as needed
17. Data Must Not be Lost
• Smart transaction control built into
the data access tier
•Make sure you can get data out of
local cache if something goes wrong
•Watch data replication latency. Make
sure everyone is comfortable with it
and that it doesn’t deteriorate.
•Set up alerts so that you detect a
problem before it’s too late.