Big Data Shouldn't Be Big
•Download as PPTX, PDF•
2 likes•520 views
A presentation from Dr Jamie Graves at The Cyber Academy
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Big Data Shouldn't Be Big
Similar to Big Data Shouldn't Be Big (20)
More from Napier University
More from Napier University (20)
Recently uploaded
Recently uploaded (20)
Big Data Shouldn't Be Big
- 1. Big Data Shouldn’t be Big
- 2. ZoneFox – Who Are We ? • Spin-out from Edinburgh Napier University • Endpoint Detection and Response
- 3. ZoneFox as Big Data • ZoneFox streams and collects endpoint data • On average we write more than 300 million events per day, per client • 1 Trillion events on average for a 500-endpoint customer
- 4. How do our Customers use ZoneFox?
- 5. What we learned • Theoretical • Practical
- 6. In Theory…
- 7. “Big data is a term for data sets that are so large or complex that traditional data processing applications are inadequate.”
- 9. Vs.
- 10. • 332 million active users • 300-500 million tweets per- day • 1.65 billion active users • Every 60 Seconds: – 510 Comments – 293,000 statuses – 136,000 photos
- 11. Source: Pew Research Center Source: Cowen & Company
- 13. What does Facebook get right? • Meaning • Focus
- 14. Meaning • Data is not Information – Relevant vs Irrelevant – Signal to Noise
- 15. Focus • Volume • Velocity • Variety = Value
- 16. In Practice…
- 17. ZoneFox – Architecture Endpoint Agents [Windows, Linux & SQL Server] Collector & Presentation Server(s) [Microsoft Windows] Database, Reporting & Search Server(s) [Ubuntu] Horizontally Scalable. Proven to handle 10,000+ endpoint agents Secure, Highly Compressed Messages Lightweight, Zero-Configuration Agent
- 18. ZoneFox Data • Simple Event Model – Machine – User – Process – Object – Behaviour – Time
- 19. What Does this Mean? • We’re able to tell when – Bob in HR starts to act like Alice in Engineering – Alice is thinking of leaving the organisation – Eve uses shadow IT out of frustration
- 20. Rolling your own • Technology is often immature, resulting in frequent release cycles with breaking changes. • Usually, 1 big data solution solves 1 niche problem • Expect to trade some speed for consistency: getting results faster means you have to compromise on data "freshness"
- 21. Rolling your own • Very small talent pool, as most devs won't have much experience with the latest technology X • Security in big data is an actively developing area • Any additional features are usually not available out of the box.
- 22. Thanks – Any Questions? Jamie Graves, CEO j.graves@zonefox.com
Editor's Notes
- We follow the data, which allows us to answer various questions relating to all of these aspects.
- What is the definition of Big Data?
- Because of this set of traditional processing applications, organisations can do anything from figuring out how we’re doing our shopping
- Twitter is down from a high of $40 Billion in 2014.
- Twitter is down from a high of $40 Billion in 2014.
- Volume: big data doesn't sample; it just observes and tracks what happens Velocity: big data is often available in real-time Variety: big data draws from text, images, audio, video; plus it completes missing pieces through data fusion
- Expect a release with breaking change every 2-4 weeks. But your business may need several problems solved, potentially forcing you to use multiple solutions Technology approach is different to what you were used to seeing. For example,
- you need to manually configure even basic security features (authentication, authorisation, SSL, encryption, ACL, etc)
- The problem with big data is… well, that it’s big! When you have what is supposed to be a very powerful tool, you’re likely to try and apply it to every problem that you have