SlideShare una empresa de Scribd logo
1 de 14
DATA NETWORK AND SECURITY
Trends, Applications & Security
Shahbaz Ahmad Anjam
BSITF15MM020
BSIT (5th
) UOS (Mianwali
2
Big Data
Big Data word mean a massive volume of both structured and
unstructured data that is so large it is difficult to process using
traditional database and software techniques. In most enterprise
scenarios the volume of data is too big or it moves too fast or it
exceeds current processing capacity.
Big Data in a another term
Volume, variety, velocity, and veracity
 Volume
The quantity of generated and stored data. The quantity of data
flowing to the system within certain time
 Variety
The type and nature of the data. Incoming quantity of data and
types.
3
 Velocity
The speed at which the data is generated and processed to meet
the demands and challenges that lie in the path of growth and
development.
 Veracity
The data quality of captured data can vary greatly, affecting the
accurate analysis.
Big Sources of Big Data
Some data sources that generate huge data volumes in just 60 seconds.
 More than 60 blog and 1500 blog posts are done.
 More than 70 domains are registered.
 More than 600 new videos are uploaded on you tube.
 More than 40 questions asked on yahoo are answered and 100 + questions
answered on answers.com
 More than 1300 iPhone applications are downloaded every minute.
 694445 searches are made on Google every minute.
 About 320 accounts are created on Twitter and more than 98000 tweets are
made every minute.
Trends
Big Data trends shift rapidly, but experts expect machine learning, predictive
analytics, IoT and edge computing to have a big impact on big data projects in the
years ahead.
Researchers at Forrester have "found that, in 2016, almost 40 percent of firms
are implementing and expanding big data technology adoption. Another 30 percent
are planning to adopt big data in the next 12 months." Similarly, the Big Data
Executive Survey 2016 from New Vantage Partners found that 62.5 percent of
4
firms now have at least one big data project in production, and only 5.4 percent of
organizations have no big data initiatives planned or underway.
2015 Trends
 More Sensor Driven Data
This would be sensor-to-sensor data being collected,
collated and analyzed through purely sensor based collection.
This can be done in multiple ways from the way that objects
are interacting with another object, to the settings that
people are using on particular devices.
 Non-Data Scientists
Automated platforms that can allow employees who may not
have as much skill with data as others, to collect, analyze
and make decisions based on this data. This could be
anything from simple to use interfaces with more complex
backends or simpler tasks that could create business
results.
 In-Memory Databases
Databases allow companies the freedom to access, analyze
and take actions based on data much quicker than regular
databases. This in turn means that either decisions can be
made quicker as data can be analyzed faster or more
informed as more data can be analyzed in the same amount
of time.
5
2016 Trends
 The NoSQL
NoSQL technologies, commonly associated with unstructured
data, have seen significant adoption over the last 12 months.Going
forward, the shift to NoSQL databases as a leading piece of the
enterprise IT landscape becomes clear as the benefits of
schema-less database concepts become more pronounced.
 Hadoop
Hadoop is an open source software framwork used for
distributed storage and processing of dataset of big data
using Map reduce programing model.
The Hadoop includes four modules: Hadoop Common
(utilities), Hadoop Distributed File System (HDFS), Hadoop
YARN (scheduler) and Hadoop MapReduce (parallel
processing).
 IoT
IoT has been touted for smart homes, wearables, smart
cities, smart grids, industrial internet, connected vehicles,
connected health, smart retail, agriculture, and a host of
other scenarios. The Internet of Things (IoT) is a system of
interrelated computing devices, mechanical and digital
machines, objects, animals or people that are provided with
unique identifiers and the ability to transfer data over a
network without requiring human-to-human or human-to-
computer interaction.
6
 Cloud Storage
Cloud storage enables applications to upload data to a network of
remote, connected servers. Applications can then maintain that data
and access it from anywhere. Applications access data using a web-
based API that works with client applications.
Storage is available in four main types:
 Personal storage: Services that enable individuals to store data and
sync it across multiple devices.
 Public storage: A cloud storage provider that fully manages data for an
enterprise offsite.
 Private storage: The cloud storage provider works on premises at an
organization’s data center.
 Hybrid storage: A mix of public and private cloud storage.
.
2017 Big Data Trends
Artificial Intelligence is most common trend of 2017
 Neural networks
A neural network is a system of hardware and/or software
patterned after the operation of neurons in the human brain. Neural
networks -- also called artificial neural networks --Neural networks
can analyze the flow of incoming data and highlight the patterns or
abnormalities according to preconfigured parameters. This helps
greatly automate the analysis and provides powerful tools for finding
valuable information in the flow of unstructured data.
 Deep learning custom tools
Is a machine learning technique, which relies on learning data
representations, unlike task-specific machine learning algorithms. it
7
definitely excels in analyzing unstructured data according to pre-
configured parameters and statistics, providing a great degree of
precision in evaluating the incoming data.
 Machine Learning
Machine learning is a branch of artificial intelligence that focuses on
allowing computers to learn new things without being explicitly
programmed. In other words, it analyzes existing big data stores to
come to conclusions which change how the application behaves.
 Data monetization
Predictive and prescriptive analytics are the approaches that help
organizations minimize expenses and earn money using their Big Data
tools. Selling the results of analytics as a service, disseminating and
structuring the data for customers can also become a profitable
market.
Is the act of generating measurable economic benefits from available
data sources. Typically these benefits accrue as revenue or expense
savings, but may also include market share or corporate market value
gains.
 R language and Jupiter notebooks
Data scientists have a number of option to analyze data using
statistical methods. One of the most convenient and powerful
methods is to use the free R programming language. R is one of the
best ways to create reproducible, high-quality analysis, since unlike a
spreadsheet, R scripts can be audited and re-run easily. The R
language and its package repositories provide a wide range of
statistical techniques, data manipulation and plotting, to the point
that if a technique exists, it is probably implemented in an R
package. R is almost as strong in its support for machine learning,
although it may not be the first choice for deep neural networks,
8
which require higher-performance computing than R currently
delivers.
 Self-Service
As the cost of hiring big experts rises, many organizations are likely
to be looking for tools that allow regular business professionals to
meet their own big data analytics needs. IDC has previously
predicted "Visual data discovery tools will be growing 2.5 times
faster than rest of the business intelligence (BI) market.Several
vendors have already launched big data analytics tools with "self-
service" capabilities, and experts expect that trend to continue into
2018 and beyond.
9
Applications of Big Data
1. Banking and Securities
In this industry include: securities fraud early warning, tick
analytics, card fraud detection, archival of audit trails, enterprise
credit risk reporting, trade visibility, customer data
transformation, social analytics for trading, IT operations
analytics, and IT policy compliance analytics, among others.
10
2. Media and Entertainment
Big data is changing the media and entertainment industry, giving
users and viewers a much more personalized and enriched
experience. Big data is used for increasing revenues, understanding
real-time customer sentiment, increasing marketing effectiveness
and ratings and viewership.
3. Healthcare
Big data is used for analyzing data in the electronic medical record
(EMR) system with the goal of reducing costs and improving patient
care. This Data includes the unstructured data from physician notes,
pathology reports etc. Big Data and healthcare analytics have the
power to predict, prevent & cure diseases..
4. Energy and Utilities
In utility companies the use of big data also allows for better
asset and workforce management which is useful for
recognizing errors and correcting them as soon as possible
before complete failure is experienced.
5. Manufacturing
Increasing demand for natural resources including oil,
agricultural products, minerals, gas, metals, and so on has led to
an increase in the volume, complexity, and velocity of data that
is a challenge to handle.
11
6. Education
In a different use case of the use of big data in education, it is
also used to measure teacher’s effectiveness to ensure a good
experience for both students and teachers. Teacher’s
performance can be fine-tuned and measured against student
numbers, subject matter, student demographics, student
aspirations, behavioral classification and several other variables
7. Transportation
 Governments use of big data: traffic control, route planning,
intelligent transport systems, congestion management (by
predicting traffic conditions)
 Private sector use of big data in transport: revenue
management, technological enhancements, logistics and for
competitive advantage (by consolidating shipments and
optimizing freight movement)
 Individual use of big data includes: route planning to save on
fuel and time, for travel arrangements in tourism etc.
8. Insurance
Big data has been used in the industry to provide customer
insights for transparent and simpler products, by analyzing and
predicting customer behavior through data derived from social
media, GPS-enabled devices and CCTV footage. The big data also
allows for better customer retention from insurance companies
12
Big Data Security
Security Issues Comes due to:-
 There are limited levels of protection in majority of distributed
systems’ computations.
 Security solutions are not being able to tackle the demand with
several non-relational databases constantly evolving.
 There is lack of appropriate security processes for the transfer of
automated data.
 System updates, audits, patches are not always carried out.
 Information coming in should be constantly validated, to ensure its
credibility and accuracy, but that is not the case for most systems
 Attack on systems that contain sensitive and personal information
of the customers can put the customers at risk.
 Certain organizations do not deploy any kind of access controls to
differentiate between the confidentiality levels of data within the
organization.
 Monitoring and tracking of systems is difficult with the current
scale of Big Data application.
 Cyber Security Threats are Rising.
13
How to secure the Big data
 Detection of frauds and replacement of SIEM (Security
Information and Event Management) systems
 Optimization of sales and marketing campaigns through business
intelligence as there is an abundance of data and analytics
available.
 Real-Time Security Monitoring
 Endpoint Input Validation and Filtering
 Secure Storage and Transaction Logging.
 Granular Access Control.
 Encrypted Data-Centric Security.
 Best Security Practices for Non-Relational Data Stores.
 Application Software Security
 Account Monitoring and Control
14
End

Más contenido relacionado

La actualidad más candente

Big Data Storage Challenges and Solutions
Big Data Storage Challenges and SolutionsBig Data Storage Challenges and Solutions
Big Data Storage Challenges and Solutions
WSO2
 

La actualidad más candente (20)

What is Big Data?
What is Big Data?What is Big Data?
What is Big Data?
 
Big Data Analytics for Healthcare
Big Data Analytics for HealthcareBig Data Analytics for Healthcare
Big Data Analytics for Healthcare
 
Big data by Mithlesh sadh
Big data by Mithlesh sadhBig data by Mithlesh sadh
Big data by Mithlesh sadh
 
Big Data
Big DataBig Data
Big Data
 
Big Data
Big DataBig Data
Big Data
 
Big Data
Big DataBig Data
Big Data
 
Big Data: Its Characteristics And Architecture Capabilities
Big Data: Its Characteristics And Architecture CapabilitiesBig Data: Its Characteristics And Architecture Capabilities
Big Data: Its Characteristics And Architecture Capabilities
 
Big Data Evolution
Big Data EvolutionBig Data Evolution
Big Data Evolution
 
Big Data & Analytics (Conceptual and Practical Introduction)
Big Data & Analytics (Conceptual and Practical Introduction)Big Data & Analytics (Conceptual and Practical Introduction)
Big Data & Analytics (Conceptual and Practical Introduction)
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
 
NPTEL BIG DATA FULL PPT BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...
NPTEL BIG DATA FULL PPT  BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...NPTEL BIG DATA FULL PPT  BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...
NPTEL BIG DATA FULL PPT BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...
 
Big data
Big dataBig data
Big data
 
Our big data
Our big dataOur big data
Our big data
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
 
Big data ppt
Big data pptBig data ppt
Big data ppt
 
Big Data Storage Challenges and Solutions
Big Data Storage Challenges and SolutionsBig Data Storage Challenges and Solutions
Big Data Storage Challenges and Solutions
 
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)Big data PPT prepared by Hritika Raj (Shivalik college of engg.)
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)
 
Social media with big data analytics
Social media with big data analyticsSocial media with big data analytics
Social media with big data analytics
 
Big data ppt
Big data pptBig data ppt
Big data ppt
 
Big Data Fundamentals
Big Data FundamentalsBig Data Fundamentals
Big Data Fundamentals
 

Similar a Big data (word file)

Big Data Analytics: Recent Achievements and New Challenges
Big Data Analytics: Recent Achievements and New ChallengesBig Data Analytics: Recent Achievements and New Challenges
Big Data Analytics: Recent Achievements and New Challenges
Editor IJCATR
 
UNIT 1 -BIG DATA ANALYTICS Full.pdf
UNIT 1 -BIG DATA ANALYTICS Full.pdfUNIT 1 -BIG DATA ANALYTICS Full.pdf
UNIT 1 -BIG DATA ANALYTICS Full.pdf
vvpadhu
 
Introduction to big data – convergences.
Introduction to big data – convergences.Introduction to big data – convergences.
Introduction to big data – convergences.
saranya270513
 
15 DATA SCIENCE TRENDS TO RULE IN 2023.pdf
15 DATA SCIENCE TRENDS TO RULE IN 2023.pdf15 DATA SCIENCE TRENDS TO RULE IN 2023.pdf
15 DATA SCIENCE TRENDS TO RULE IN 2023.pdf
USDSI
 
CS309A Final Paper_KM_DD
CS309A Final Paper_KM_DDCS309A Final Paper_KM_DD
CS309A Final Paper_KM_DD
David Darrough
 
Fundamentals of data mining and its applications
Fundamentals of data mining and its applicationsFundamentals of data mining and its applications
Fundamentals of data mining and its applications
Subrat Swain
 

Similar a Big data (word file) (20)

Complete-SRS.doc
Complete-SRS.docComplete-SRS.doc
Complete-SRS.doc
 
Big Data Testing Using Hadoop Platform
Big Data Testing Using Hadoop PlatformBig Data Testing Using Hadoop Platform
Big Data Testing Using Hadoop Platform
 
Big data – A Review
Big data – A ReviewBig data – A Review
Big data – A Review
 
Smart Data Module 6 d drive the future
Smart Data Module 6 d drive the futureSmart Data Module 6 d drive the future
Smart Data Module 6 d drive the future
 
An Encyclopedic Overview Of Big Data Analytics
An Encyclopedic Overview Of Big Data AnalyticsAn Encyclopedic Overview Of Big Data Analytics
An Encyclopedic Overview Of Big Data Analytics
 
Big data
Big dataBig data
Big data
 
Big data
Big dataBig data
Big data
 
Big Data Analytics: Recent Achievements and New Challenges
Big Data Analytics: Recent Achievements and New ChallengesBig Data Analytics: Recent Achievements and New Challenges
Big Data Analytics: Recent Achievements and New Challenges
 
IRJET- A Scrutiny on Research Analysis of Big Data Analytical Method and Clou...
IRJET- A Scrutiny on Research Analysis of Big Data Analytical Method and Clou...IRJET- A Scrutiny on Research Analysis of Big Data Analytical Method and Clou...
IRJET- A Scrutiny on Research Analysis of Big Data Analytical Method and Clou...
 
UNIT 1 -BIG DATA ANALYTICS Full.pdf
UNIT 1 -BIG DATA ANALYTICS Full.pdfUNIT 1 -BIG DATA ANALYTICS Full.pdf
UNIT 1 -BIG DATA ANALYTICS Full.pdf
 
Module 6 The Future of Big and Smart Data- Online
Module 6 The Future of Big and Smart Data- Online Module 6 The Future of Big and Smart Data- Online
Module 6 The Future of Big and Smart Data- Online
 
Introduction to big data – convergences.
Introduction to big data – convergences.Introduction to big data – convergences.
Introduction to big data – convergences.
 
15 DATA SCIENCE TRENDS TO RULE IN 2023.pdf
15 DATA SCIENCE TRENDS TO RULE IN 2023.pdf15 DATA SCIENCE TRENDS TO RULE IN 2023.pdf
15 DATA SCIENCE TRENDS TO RULE IN 2023.pdf
 
Big data
Big dataBig data
Big data
 
The book of elephant tattoo
The book of elephant tattooThe book of elephant tattoo
The book of elephant tattoo
 
A SURVEY OF BIG DATA ANALYTICS
A SURVEY OF BIG DATA ANALYTICSA SURVEY OF BIG DATA ANALYTICS
A SURVEY OF BIG DATA ANALYTICS
 
Snowball Group Whitepaper - Spotlight on Big Data
Snowball Group Whitepaper - Spotlight on Big DataSnowball Group Whitepaper - Spotlight on Big Data
Snowball Group Whitepaper - Spotlight on Big Data
 
Big data analytics and large-scale computers
Big data analytics and large-scale computersBig data analytics and large-scale computers
Big data analytics and large-scale computers
 
CS309A Final Paper_KM_DD
CS309A Final Paper_KM_DDCS309A Final Paper_KM_DD
CS309A Final Paper_KM_DD
 
Fundamentals of data mining and its applications
Fundamentals of data mining and its applicationsFundamentals of data mining and its applications
Fundamentals of data mining and its applications
 

Más de Shahbaz Anjam (10)

Cryptanalysis and Attacks
Cryptanalysis and AttacksCryptanalysis and Attacks
Cryptanalysis and Attacks
 
Usecase diagram
Usecase diagramUsecase diagram
Usecase diagram
 
Sequence diagram
Sequence diagramSequence diagram
Sequence diagram
 
Project overview
Project overviewProject overview
Project overview
 
Dataflow diagram
Dataflow diagramDataflow diagram
Dataflow diagram
 
Class diagram
Class diagramClass diagram
Class diagram
 
Activity Diagram
Activity Diagram Activity Diagram
Activity Diagram
 
Mms resolution
Mms resolutionMms resolution
Mms resolution
 
Resolution
 Resolution Resolution
Resolution
 
RAID its types
RAID  its typesRAID  its types
RAID its types
 

Último

Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
amilabibi1
 
Chiulli_Aurora_Oman_Raffaele_Beowulf.pptx
Chiulli_Aurora_Oman_Raffaele_Beowulf.pptxChiulli_Aurora_Oman_Raffaele_Beowulf.pptx
Chiulli_Aurora_Oman_Raffaele_Beowulf.pptx
raffaeleoman
 
If this Giant Must Walk: A Manifesto for a New Nigeria
If this Giant Must Walk: A Manifesto for a New NigeriaIf this Giant Must Walk: A Manifesto for a New Nigeria
If this Giant Must Walk: A Manifesto for a New Nigeria
Kayode Fayemi
 
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
David Celestin
 
Uncommon Grace The Autobiography of Isaac Folorunso
Uncommon Grace The Autobiography of Isaac FolorunsoUncommon Grace The Autobiography of Isaac Folorunso
Uncommon Grace The Autobiography of Isaac Folorunso
Kayode Fayemi
 

Último (15)

Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
 
Chiulli_Aurora_Oman_Raffaele_Beowulf.pptx
Chiulli_Aurora_Oman_Raffaele_Beowulf.pptxChiulli_Aurora_Oman_Raffaele_Beowulf.pptx
Chiulli_Aurora_Oman_Raffaele_Beowulf.pptx
 
AWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdf
AWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdfAWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdf
AWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdf
 
Digital collaboration with Microsoft 365 as extension of Drupal
Digital collaboration with Microsoft 365 as extension of DrupalDigital collaboration with Microsoft 365 as extension of Drupal
Digital collaboration with Microsoft 365 as extension of Drupal
 
The workplace ecosystem of the future 24.4.2024 Fabritius_share ii.pdf
The workplace ecosystem of the future 24.4.2024 Fabritius_share ii.pdfThe workplace ecosystem of the future 24.4.2024 Fabritius_share ii.pdf
The workplace ecosystem of the future 24.4.2024 Fabritius_share ii.pdf
 
lONG QUESTION ANSWER PAKISTAN STUDIES10.
lONG QUESTION ANSWER PAKISTAN STUDIES10.lONG QUESTION ANSWER PAKISTAN STUDIES10.
lONG QUESTION ANSWER PAKISTAN STUDIES10.
 
My Presentation "In Your Hands" by Halle Bailey
My Presentation "In Your Hands" by Halle BaileyMy Presentation "In Your Hands" by Halle Bailey
My Presentation "In Your Hands" by Halle Bailey
 
Report Writing Webinar Training
Report Writing Webinar TrainingReport Writing Webinar Training
Report Writing Webinar Training
 
If this Giant Must Walk: A Manifesto for a New Nigeria
If this Giant Must Walk: A Manifesto for a New NigeriaIf this Giant Must Walk: A Manifesto for a New Nigeria
If this Giant Must Walk: A Manifesto for a New Nigeria
 
ICT role in 21st century education and it's challenges.pdf
ICT role in 21st century education and it's challenges.pdfICT role in 21st century education and it's challenges.pdf
ICT role in 21st century education and it's challenges.pdf
 
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
 
Dreaming Marissa Sánchez Music Video Treatment
Dreaming Marissa Sánchez Music Video TreatmentDreaming Marissa Sánchez Music Video Treatment
Dreaming Marissa Sánchez Music Video Treatment
 
Uncommon Grace The Autobiography of Isaac Folorunso
Uncommon Grace The Autobiography of Isaac FolorunsoUncommon Grace The Autobiography of Isaac Folorunso
Uncommon Grace The Autobiography of Isaac Folorunso
 
SOLID WASTE MANAGEMENT SYSTEM OF FENI PAURASHAVA, BANGLADESH.pdf
SOLID WASTE MANAGEMENT SYSTEM OF FENI PAURASHAVA, BANGLADESH.pdfSOLID WASTE MANAGEMENT SYSTEM OF FENI PAURASHAVA, BANGLADESH.pdf
SOLID WASTE MANAGEMENT SYSTEM OF FENI PAURASHAVA, BANGLADESH.pdf
 
Dreaming Music Video Treatment _ Project & Portfolio III
Dreaming Music Video Treatment _ Project & Portfolio IIIDreaming Music Video Treatment _ Project & Portfolio III
Dreaming Music Video Treatment _ Project & Portfolio III
 

Big data (word file)

  • 1. DATA NETWORK AND SECURITY Trends, Applications & Security Shahbaz Ahmad Anjam BSITF15MM020 BSIT (5th ) UOS (Mianwali
  • 2. 2 Big Data Big Data word mean a massive volume of both structured and unstructured data that is so large it is difficult to process using traditional database and software techniques. In most enterprise scenarios the volume of data is too big or it moves too fast or it exceeds current processing capacity. Big Data in a another term Volume, variety, velocity, and veracity  Volume The quantity of generated and stored data. The quantity of data flowing to the system within certain time  Variety The type and nature of the data. Incoming quantity of data and types.
  • 3. 3  Velocity The speed at which the data is generated and processed to meet the demands and challenges that lie in the path of growth and development.  Veracity The data quality of captured data can vary greatly, affecting the accurate analysis. Big Sources of Big Data Some data sources that generate huge data volumes in just 60 seconds.  More than 60 blog and 1500 blog posts are done.  More than 70 domains are registered.  More than 600 new videos are uploaded on you tube.  More than 40 questions asked on yahoo are answered and 100 + questions answered on answers.com  More than 1300 iPhone applications are downloaded every minute.  694445 searches are made on Google every minute.  About 320 accounts are created on Twitter and more than 98000 tweets are made every minute. Trends Big Data trends shift rapidly, but experts expect machine learning, predictive analytics, IoT and edge computing to have a big impact on big data projects in the years ahead. Researchers at Forrester have "found that, in 2016, almost 40 percent of firms are implementing and expanding big data technology adoption. Another 30 percent are planning to adopt big data in the next 12 months." Similarly, the Big Data Executive Survey 2016 from New Vantage Partners found that 62.5 percent of
  • 4. 4 firms now have at least one big data project in production, and only 5.4 percent of organizations have no big data initiatives planned or underway. 2015 Trends  More Sensor Driven Data This would be sensor-to-sensor data being collected, collated and analyzed through purely sensor based collection. This can be done in multiple ways from the way that objects are interacting with another object, to the settings that people are using on particular devices.  Non-Data Scientists Automated platforms that can allow employees who may not have as much skill with data as others, to collect, analyze and make decisions based on this data. This could be anything from simple to use interfaces with more complex backends or simpler tasks that could create business results.  In-Memory Databases Databases allow companies the freedom to access, analyze and take actions based on data much quicker than regular databases. This in turn means that either decisions can be made quicker as data can be analyzed faster or more informed as more data can be analyzed in the same amount of time.
  • 5. 5 2016 Trends  The NoSQL NoSQL technologies, commonly associated with unstructured data, have seen significant adoption over the last 12 months.Going forward, the shift to NoSQL databases as a leading piece of the enterprise IT landscape becomes clear as the benefits of schema-less database concepts become more pronounced.  Hadoop Hadoop is an open source software framwork used for distributed storage and processing of dataset of big data using Map reduce programing model. The Hadoop includes four modules: Hadoop Common (utilities), Hadoop Distributed File System (HDFS), Hadoop YARN (scheduler) and Hadoop MapReduce (parallel processing).  IoT IoT has been touted for smart homes, wearables, smart cities, smart grids, industrial internet, connected vehicles, connected health, smart retail, agriculture, and a host of other scenarios. The Internet of Things (IoT) is a system of interrelated computing devices, mechanical and digital machines, objects, animals or people that are provided with unique identifiers and the ability to transfer data over a network without requiring human-to-human or human-to- computer interaction.
  • 6. 6  Cloud Storage Cloud storage enables applications to upload data to a network of remote, connected servers. Applications can then maintain that data and access it from anywhere. Applications access data using a web- based API that works with client applications. Storage is available in four main types:  Personal storage: Services that enable individuals to store data and sync it across multiple devices.  Public storage: A cloud storage provider that fully manages data for an enterprise offsite.  Private storage: The cloud storage provider works on premises at an organization’s data center.  Hybrid storage: A mix of public and private cloud storage. . 2017 Big Data Trends Artificial Intelligence is most common trend of 2017  Neural networks A neural network is a system of hardware and/or software patterned after the operation of neurons in the human brain. Neural networks -- also called artificial neural networks --Neural networks can analyze the flow of incoming data and highlight the patterns or abnormalities according to preconfigured parameters. This helps greatly automate the analysis and provides powerful tools for finding valuable information in the flow of unstructured data.  Deep learning custom tools Is a machine learning technique, which relies on learning data representations, unlike task-specific machine learning algorithms. it
  • 7. 7 definitely excels in analyzing unstructured data according to pre- configured parameters and statistics, providing a great degree of precision in evaluating the incoming data.  Machine Learning Machine learning is a branch of artificial intelligence that focuses on allowing computers to learn new things without being explicitly programmed. In other words, it analyzes existing big data stores to come to conclusions which change how the application behaves.  Data monetization Predictive and prescriptive analytics are the approaches that help organizations minimize expenses and earn money using their Big Data tools. Selling the results of analytics as a service, disseminating and structuring the data for customers can also become a profitable market. Is the act of generating measurable economic benefits from available data sources. Typically these benefits accrue as revenue or expense savings, but may also include market share or corporate market value gains.  R language and Jupiter notebooks Data scientists have a number of option to analyze data using statistical methods. One of the most convenient and powerful methods is to use the free R programming language. R is one of the best ways to create reproducible, high-quality analysis, since unlike a spreadsheet, R scripts can be audited and re-run easily. The R language and its package repositories provide a wide range of statistical techniques, data manipulation and plotting, to the point that if a technique exists, it is probably implemented in an R package. R is almost as strong in its support for machine learning, although it may not be the first choice for deep neural networks,
  • 8. 8 which require higher-performance computing than R currently delivers.  Self-Service As the cost of hiring big experts rises, many organizations are likely to be looking for tools that allow regular business professionals to meet their own big data analytics needs. IDC has previously predicted "Visual data discovery tools will be growing 2.5 times faster than rest of the business intelligence (BI) market.Several vendors have already launched big data analytics tools with "self- service" capabilities, and experts expect that trend to continue into 2018 and beyond.
  • 9. 9 Applications of Big Data 1. Banking and Securities In this industry include: securities fraud early warning, tick analytics, card fraud detection, archival of audit trails, enterprise credit risk reporting, trade visibility, customer data transformation, social analytics for trading, IT operations analytics, and IT policy compliance analytics, among others.
  • 10. 10 2. Media and Entertainment Big data is changing the media and entertainment industry, giving users and viewers a much more personalized and enriched experience. Big data is used for increasing revenues, understanding real-time customer sentiment, increasing marketing effectiveness and ratings and viewership. 3. Healthcare Big data is used for analyzing data in the electronic medical record (EMR) system with the goal of reducing costs and improving patient care. This Data includes the unstructured data from physician notes, pathology reports etc. Big Data and healthcare analytics have the power to predict, prevent & cure diseases.. 4. Energy and Utilities In utility companies the use of big data also allows for better asset and workforce management which is useful for recognizing errors and correcting them as soon as possible before complete failure is experienced. 5. Manufacturing Increasing demand for natural resources including oil, agricultural products, minerals, gas, metals, and so on has led to an increase in the volume, complexity, and velocity of data that is a challenge to handle.
  • 11. 11 6. Education In a different use case of the use of big data in education, it is also used to measure teacher’s effectiveness to ensure a good experience for both students and teachers. Teacher’s performance can be fine-tuned and measured against student numbers, subject matter, student demographics, student aspirations, behavioral classification and several other variables 7. Transportation  Governments use of big data: traffic control, route planning, intelligent transport systems, congestion management (by predicting traffic conditions)  Private sector use of big data in transport: revenue management, technological enhancements, logistics and for competitive advantage (by consolidating shipments and optimizing freight movement)  Individual use of big data includes: route planning to save on fuel and time, for travel arrangements in tourism etc. 8. Insurance Big data has been used in the industry to provide customer insights for transparent and simpler products, by analyzing and predicting customer behavior through data derived from social media, GPS-enabled devices and CCTV footage. The big data also allows for better customer retention from insurance companies
  • 12. 12 Big Data Security Security Issues Comes due to:-  There are limited levels of protection in majority of distributed systems’ computations.  Security solutions are not being able to tackle the demand with several non-relational databases constantly evolving.  There is lack of appropriate security processes for the transfer of automated data.  System updates, audits, patches are not always carried out.  Information coming in should be constantly validated, to ensure its credibility and accuracy, but that is not the case for most systems  Attack on systems that contain sensitive and personal information of the customers can put the customers at risk.  Certain organizations do not deploy any kind of access controls to differentiate between the confidentiality levels of data within the organization.  Monitoring and tracking of systems is difficult with the current scale of Big Data application.  Cyber Security Threats are Rising.
  • 13. 13 How to secure the Big data  Detection of frauds and replacement of SIEM (Security Information and Event Management) systems  Optimization of sales and marketing campaigns through business intelligence as there is an abundance of data and analytics available.  Real-Time Security Monitoring  Endpoint Input Validation and Filtering  Secure Storage and Transaction Logging.  Granular Access Control.  Encrypted Data-Centric Security.  Best Security Practices for Non-Relational Data Stores.  Application Software Security  Account Monitoring and Control