This document provides an overview of Apache NiFi and data flow fundamentals. It begins with an introduction to Apache NiFi and outlines the agenda. It then discusses data flow and streaming fundamentals, including challenges in moving data effectively. The document introduces Apache NiFi's architecture and capabilities for addressing these challenges. It also previews a live demo of NiFi and discusses the NiFi community.
In reality, dataflows move all over. Data is moved and stored in multiple places – sometimes interim, sometimes longterm. Data is procesed in different places, and then moved again. Complicated, convoluted, messy.
Kafka
Reads events in memory and write to distributed log
NiFi: simple event processing
Spark: complex event processing
Build predictive model from Historical insights.
Deploy predictive model for real-time insights.
Can put NiFi on a Gateway server but probably don’t want to mess with a UI on ever single one
Maybe not best fit
Let me get the key parts of NiFi close to where data begins and provide bidrectional communication
NiFi lives in the data center. Give it an enterprise server or a cluster of them.
MiNiFi lives close to where data is born and may be a guest on that device or system
Framework – put a new wrapper on the framework, or in maven terms, we kept the underlying modules and wrote minifi-framework-core replacing nifi-framework-core
Talking about MiNiFi-Java, Cpp version also exists
Initiates with ./bin/nifi.sh start
user, only need bootstrap and config.yml
nifi.properties and flow.xml are implementation details
Smart Cities
Monitor:
Public transportation vehicles
Pedestrian levels
Optimize public transit duration and walking routes
Source:
http://www.libelium.com/resources/top_50_iot_sensor_applications_ranking/