This talk was held at the 10th meeting on February 3rd 2014 by Daniel Fasel.
Many traditional Swiss companies, such as banks, insurance companies and government agencies, are highly interested in Big Data and Data Science but don’t know exactly what the business value of Big Data is for them. Often Big Data is misinterpreted as large amounts of data and companies are unaware of the innovation behind the new technologies of Big Data and how these technologies can be profitable to them. In this presentation, I discuss sample cases that demonstrate a set of these new technologies and how they can be applied not only for large web scale data but also for data sets of traditional companies. First, I demonstrate how multi-structured data can be indexed and searched using Autonomy. I show how fast new analytical application can be built based on a real-time streaming example using STORM, Redis and Node.js. And the last demonstration shows how machine learning algorithms and visualization can be applied for improving analytics using AsterData.
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Big Data and Data Science for traditional Swiss companies
1. Scigility!
Scire propter agere!
Big Data and Data Science for
traditional Swiss companies!
Dr. Daniel Fasel!
Scigility AG !
www.scigility.com!
2. Who are we?!
– Dr. Daniel Fasel has been the first data scientist on the
business intelligence team at Swisscom and was key in
implementing NoSQL technologies for explorative analytics
during his time at Swisscom!
!
– Prof. Dr. Philippe Cudré-Mauroux is the director of the
eXascale Infolab and Professor at the University of Fribourg
in Switzerland. Previously, he was a postdoctoral associate
working in the Database Systems group at MIT. He also
worked on distributed information and media management for
HP, IBM Watson Research (NY), and Microsoft Research
Asia.!
www.scigility.com
2!
3. Our services!
• We provide consulting, software
development, operations and training in the
areas of large-scale information systems,
NoSQL technologies and real time streaming
solutions. Technologies commonly known as
Big Data.!
• We provide the optimal solution to your
specific IT problems, based on the latest and
most appropriate technologies available!!
www.scigility.com
3!
4. Content!
• Short overview of Big Data!
• Techniques & Technologies!
• 3 Demonstrations how to use these new
Techniques and Technologies!
www.scigility.com
4!
5. Big Data!
• Big – Volume!
– Big is relative!
• Google > 400PB !
• Traditional Swiss companies < 400 PB ;-)!
– But Big Data (Volume) is a fact!
• Data constantly grow!
• More and more systems produce data!
• Classical relational schemas already do a pre-selection
of data!
• There is more interesting data in your company than
you potentially assume (dark data on Intranet or File
Shares, application silos, etc.)!
www.scigility.com
5!
7. Big Data!
• Variety!
– Structures change over time!
• Think about your legacy system!
– The structure of data is determined at the time
on analysis and not at the time of storing!
• Schema on purpose / schema on read!
– Combination of classical and new data
sources!
7!
8. Big Data!
• Velocity!
– Data is produced faster!
– Data becomes more and more ephemeral!
– Analytics gets real time!
– Data flows are as interesting as data itself!
www.scigility.com
8!
9. Big Data!
• What is the innovation of Big Data?!
• The real new innovations of Big Data are!
– optimized techniques and technologies!
– that address the specific characteristics which
are commonly summarized as Big Data!
• Big Data is not a new kind of data!!
– You all have Big Data! !
www.scigility.com
9!
10. Techniques!
• New techniques!
– MapReduce for analysis of highly distributed
data!
– Combination of linear algebra, statistics,
computer science, visualization for broader
users groups!
– Improved agility and rapid prototyping!
– Explorative analytics!
www.scigility.com
10!
11. Technologies!
• New technologies!
– Massive horizontal scalable / elastic!
– Optimized for specific types of problems!
– Not necessarily ACID compliant!
– Follow BASE & CAP concepts!
– Integration and combination of classical and
new technologies (like SQL-MR)!
www.scigility.com
11!