Designing, implementing, and testing an Apache Spark Streaming application is necessary to deploy to production but is not sufficient for long term management and monitoring. Simply learning the Spark Streaming APIs only gets you part of the way there. In this talk, I’ll be focusing on everything that happens after you’ve implemented your application in the context of a real-time alerting system for IT operational data.
YMMV Not necessarily true for you
Enterprise software – shipping stuff to people
Fine grained events – logs, user behavior, etc.
For everything – solving the problem of “enterprise wide” ops, so it’s everything from everywhere from everyone for all time (until they run out of money for nodes).
This isn’t condemnation of general purpose search engines as much as what we had to do for our domain