An important underlying concept behind location-based applications is called geofencing. Geofencing is a process that allows acting on users and/or devices who enter/exit a specific geographical area, known as a geo-fence. A geo-fence can be dynamically generated—as in a radius around a point location, or a geo-fence can be a predefined set of boundaries (such as secured areas, buildings, boarders of counties, states or countries). Geofencing lays the foundation for realizing use cases around fleet monitoring, asset tracking, phone tracking across cell sites, connected manufacturing, ride-sharing solutions and many others. GPS tracking tells constantly and in real time where a device is located and forms the stream of events which needs to be analyzed against the much more static set of geo-fences. Many of the use cases mentioned above require low-latency actions taken place, if either a device enters or leaves a geo-fence or when it is approaching such a geo-fence. That’s where streaming data ingestion and streaming analytics and therefore the Kafka ecosystem comes into play. This session will present how location analytics applications can be implemented using Kafka and KSQL & Kafka Streams. It highlights the exiting features available out-of-the-box and then shows how easy it is to extend it by custom defined functions (UDFs). The design of such solution so that it can scale with both an increasing amount of position events as well as geo-fences will be discussed as well.

1. BASEL | BERN | BRUGG | BUCHAREST | DÜSSELDORF | FRANKFURT A.M. | FREIBURG I.BR. | GENEVA
HAMBURG | COPENHAGEN | LAUSANNE | MANNHEIM | MUNICH | STUTTGART | VIENNA | ZURICH
http://guidoschmutz.wordpress.com@gschmutz
Location Analytics
Real-Time Geofencing using Kafka
Guido Schmutz
UKOUG Techfest 2019
updated
for ksqlDB

2. Agenda
1. Introduction & Motivation
2. Implementing Geo Fencing with Kafka
• Using Oracle Stream Analytics
• Using ksqlDB
• Using Kafka Streams
• Using Tile38
3. Visualization using ArcadiaData
4. Summary

3. BASEL | BERN | BRUGG | BUKAREST | DÜSSELDORF | FRANKFURT A.M. | FREIBURG I.BR. | GENF
HAMBURG | KOPENHAGEN | LAUSANNE | MANNHEIM | MÜNCHEN | STUTTGART | WIEN | ZÜRICH
Guido
Working at Trivadis for more than 22 years
Consultant, Trainer, Platform Architect for Java,
Oracle, SOA and Big Data / Fast Data
Oracle Groundbreaker Ambassador & Oracle ACE
Director
@gschmutz guidoschmutz.wordpress.com
176th
edition

4. Introduction

5. Geofencing – What is it?
Use of GPS or RFID technology to create a
virtual geographic boundary, enabling
software to trigger a response when an
object/device enters or leaves a particular
area
Possible Events
• ENTER
• EXIT
• OUTSIDE
• lNSIDE
Source: https://tile38.com

6. Apache Kafka – A Streaming Platform
Source
Connector
Sink
Connector
trucking_
driver
ksqlDB Engine
Kafka Streams
Kafka Broker
ksqlDB

7. Apache Kafka
Kafka Cluster
Consumer 1 Consume 2r
Broker 1 Broker 2 Broker 3
Zookeeper
Ensemble
ZK 1 ZK 2ZK 3
Schema
Registry
Service 1
Management
Control Center
Kafka Manager
KAdmin
Producer 1 Producer 2
kafkacat
Data Retention:
• Never
• Time (TTL) or Size-based
• Log-Compacted based
Producer3Producer3
ConsumerConsumer 3

8. Apache Kafka – How does it scale?
• horizontally scalable, guaranteed order

9. Streaming Analytics: ksqlDB or Kafka Streams
Stream
• unbounded sequence of structured data
("facts")
• Facts in a stream are immutable
Table
• collected state of a stream
• Latest value for each key in a stream
• Facts in a table are mutable
ksqlDB: Stream Processing with zero
coding using SQL-like language (now
supporting push and pull queries)
Kafka Streams: Java library providing
stream analytics capabilities
trucking_
driver
Kafka Broker
ksqlDB Engine
Kafka Streams
ksqlDB REST
Commands
ksqlDB CLI
push pull

10. Stream vs. Table
Vehicle Position Stream
id latitude longitude
1 52.3924 13.0514
id latitude longitude
1 52.3924 13.0514
4 38.4847 -90.23345
id latitude longitude
1 52.3924 13.0514
4 38.4847 -90.23345
1 52.39052 13.06455
id name geometry_wkt
10 St. Louis POLYGON ((13.297920227050781
52.56195151687443, …))
11 Berlin POLYGON ((-90.23345947265625
38.484769753492536,…))
id name geometry_wkt
10 St. Louis POLYGON ((13.297920227050781
52.56195151687443, …))
id name geometry_wkt
10 St. Louis
(US)
POLYGON ((13.297920227050781
52.56195151687443, …))
11 Berlin
(GE)
POLYGON ((-90.23345947265625
38.484769753492536,…))
GeoFence Table

11. First "naïve " idea when answering the CFP
• geofence being an UDF (User Defined Function)
• would need access to the geofences defined somewhere
• what about updates?
• a function should not cause any side-effects
SELECT geofence(latitude, longitude) geo_event
FROM geo_position_stream

12. Dash
board
High Level Overview of Test Case
geofence
Join Position
& Geofences
Vehicle
Position
vehicle
position
pos &
geofences
Geo
fencing
geofence
status
key=10
{ "id" : "10", "latitude" : 38.35821, "longitude" : -90.15311}
key=3
{"id":3,"name":"Berlin, Germany","geometry_wkt":"POLYGON
((13.297920227050781 52.56195151687443,
…))","last_update":1560607149015}
Geofence
Mgmt
Vehicle
Position
Weather
Service

13. Geo-Processing
Well-known text (WKT) is a text markup language for representing vector geometry
objects on a map
GeoTools is a free software GIS toolkit for developing
standards compliant solutions

14. Implementing Geo Fencing -
using Oracle Stream Analytics

15. Oracle Stream Analytics (OSA)
• Expressive Patterns Library,
including Geo Processing
• Abstracted visual façade to
interrogate live real time
streaming data
• Integrate with Apache Kafka
• Runs on top of Spark Cluster
as a Spark Streaming Job

16. Implementing Geo Fencing -
using ksqlDB

17. ksqlDB – Streams and Tables
geofence
Table
vehicle
position
Stream
KSQL
Geofencing
id latitude longitude
1 52.3924 13.0514
4 38.4847 -90.23345
3 .. ..
id name geometry_wkt
10 St. Louis POLYGON ((13.297920227050781
52.56195151687443, …))
11 Berlin POLYGON ((-90.23345947265625
38.484769753492536,…))
12 xxxxx POLYGON ((…))

18. ksqlDB – Streams and Tables
geofence
Table
vehicle
position
Stream
CREATE STREAM vehicle_position_s
(id VARCHAR,
latitude DOUBLE,
longitude DOUBLE)
WITH (KAFKA_TOPIC='vehicle_position',
VALUE_FORMAT='DELIMITED');
CREATE TABLE geo_fence_t
(id BIGINT,
name VARCHAR,
geometry_wkt VARCHAR)
WITH (KAFKA_TOPIC='geo_fence',
VALUE_FORMAT='JSON',
KEY = 'id');
KSQL
Geofencing

19. How to determine "inside" or "outside" geofence?
Only one standard UDF for geo processing in KSQL: GEO_DISTANCE
Implement custom UDF using functionality from GeoTools Java library
public String geo_fence(final double latitude, final double longitude,
final String geometryWKT){ .. }
public List<String> geo_fence_bulk(final double latitude
, final double longitude, List<String> idGeometryListWKT) { .. }
ksql> SELECT geo_fence(latitude, longitude, 'POLYGON ((13.297920227050781
52.56195151687443, 13.2440185546875 52.530216577830124, ...))')
FROM test_geo_udf_s;
52.4497 | 13.3096 | OUTSIDE
52.4556 | 13.3178 | INSIDE

20. Custom UDF to determine if Point is inside a geometry
@Udf(description = "determines if a lat/long is inside or outside the
geometry passed as the 3rd parameter as WKT encoded ...")
public String geo_fence(@UdfParameter(value="latitude") final double latitude,
@UdfParameter(value="longitude") ) final double longitude,
@UdfParameter(value="geometryWKT") ) final String geometryWKT) {
String status = "";
GeometryFactory geometryFactory = JTSFactoryFinder.getGeometryFactory();
WKTReader reader = new WKTReader(geometryFactory);
Polygon polygon = (Polygon) reader.read(geometryWKT);
Coordinate coord = new Coordinate(longitude, latitude);
Point point = geometryFactory.createPoint(coord);
if (point.within(polygon)) {
status = "INSIDE";
} else {
status = "OUTSIDE";
}
return status;
}

21. 1) Using Cross Join
geofence
Table
Join Position
& Geofences
vehicle
position
Stream
Stream
pos &
geofences
CREATE STREAM vp_join_gf_s
AS
SELECT vp.id,
geo_fence(vp.latitude,
vp.longitude,
gf.geometry_wkt) status
FROM vehicle_position_s AS vp
CROSS JOIN geo_fence_t AS gf
There is no Cross Join in ksqlDB!
id latitude longitude
1 52.3924 13.0514
4 38.4847 -90.23345
3 .. ..
id name geometry_wkt
10 St. Louis POLYGON ((13.297920227050781
52.56195151687443, …))
11 Berlin POLYGON ((-90.23345947265625
38.484769753492536,…))
12 xxxxx POLYGON ((…))
X

22. 2) INNER Join
geofence
Stream
Join Position
& Geofences
vehicle
position
Stream
Stream
pos &
geofences
{ "group":1", "name":"Berlin", "geometry_wkt":"POLYGON ((-
90.23345947265625 38.484769753492536,…))",
"last_update":1560607149015}
Enrich Group
Table
geofences
by group 1
Enrich Group
Stream
postion by
group 1
Cannot insert into Table
from Stream
>INSERT INTO geo_fence_t
>SELECT '1' AS group_id, geof.id, …
>FROM geo_fence_s geof;
INSERT INTO can only be used to insert into
a stream. A02_GEO_FENCE_T is a table.
{ "group":"1", "id" : "10", "latitude" : 52.3924, "longitude" : 13.0514}
gid id latitude longitude
1 1 52.3924 13.0514
1 4 38.4847 -90.23345
1 3 .. ..
gid id name geometry_wkt
1 10 St. Louis POLYGON ((13.297920227050781
52.56195151687443, …))
1 11 Berlin POLYGON ((-90.23345947265625
38.484769753492536,…))
{ "group":1", "name":"St. Louis", "geometry_wkt":"POLYGON
((13.297920227050781 52.56195151687443, …))",
"last_update":1560607149015}

23. 3) Geofences aggregated in one group
Join Position
& Geofences
Stream
geofence
event
Geofences
aggby group
Table
{ "group":"1", [ "1:POLYGON ((13.297920227050781
52.56195151687443, …))","2:POLYGON ((-
90.23345947265625 38.484769753492536,…))" ] }
geo_fence_bulk
geofence
Stream
vehicle
position
Stream
{ "group":1", "name":"St. Louis",
"geometry_wkt":"POLYGON ((13.297920227050781
52.56195151687443, …))",
"last_update":1560607149015}
Enrich With
Group-1
Stream
geofences
by group 1
Enrich With
Group-1
Stream
postion by
group 1
geofences
by group 1
high low
low high
low high
Scalable
Latency
"Code Smell"
medium
medium
medium
{ "group":"1", "id" : "10", "latitude" : 52.3924, "longitude" : 13.0514}
{ "group":1", "name":"Berlin", "geometry_wkt":"POLYGON
((-90.23345947265625 38.484769753492536,…))",
"last_update":1560607149015}
gid id latitude longitude
1 1 52.3924 13.0514
1 4 38.4847 -90.23345
1 3 .. ..
gid geometry_wkt_list
1 1:POLYGON ((13.297920227050781
52.56195151687443, …))
2:POLYGON ((-90.23345947265625
38.484769753492536,…))
X:POLGON ((…))
always only one row

24. 3) Geofences aggregated in one group
CREATE TABLE a03_geo_fence_aggby_group_t
AS
SELECT group_id
, collect_set(id + ':' + geometry_wkt) AS id_geometry_wkt_list
FROM a03_geo_fence_by_group_s geof
GROUP BY group_id;
CREATE STREAM a03_vehicle_position_by_group_s
AS
SELECT '1' group_id, vehp.id, vehp.latitude, vehp.longitude
FROM vehicle_position_s vehp
PARTITION BY group_id;

25. 3) Geofences aggregated in one group
CREATE STREAM a03_geo_fence_status_s
AS
SELECT vehp.id, vehp.latitude, vehp.longitude,
geo_fence_bulk(vehp.latitude, vehp.longitude,
geofaggid_geometry_wkt_list) AS geofence_status
FROM a03_vehicle_position_by_group_s veh
LEFT JOIN a03_geo_fence_aggby_group_t geofagg
ON vehp.group_id = geofagg.group_id;
ksql> SELECT * FROM a03_geo_fence_status_s;
46 | 52.47546 | 13.34851 | [1:OUTSIDE, 3:INSIDE]
46 | 52.47521 | 13.34881 | [1:OUTSIDE, 3:INSIDE]
...
As many as there are geo-fences

26. Geo Hash for a better distribution
Geohash is a geocoding which
encodes a geographic location
into a short string of letters and
digits
Length Area width x height
1 5,009.4km x 4,992.6km
2 1,252.3km x 624.1km
3 156.5km x 156km
4 39.1km x 19.5km
12 3.7cm x 1.9cm
http://geohash.gofreerange.com/

27. Geo Hash Custom UDF
ksql> SELECT latitude, longitude, geo_hash(latitude, longitude, 3)
>FROM vehicle_position_s;
38.484769753492536 | -90.23345947265625 | 9yz
public String geohash(final double latitude,
final double longitude, int length)
public List<String> neighbours(String geohash)
public String adjacentHash(String geohash, String directionString)
public List<String> coverBoundingBox(String geometryWKT, int length)
ksql> SELECT name, wkt, geo_hash(geometry_wkt, 3) FROM a04_geo_fence_s;
St. Louis, POLYGON ((-90.25749206542969 38.71551876930462, -90.31723022460938
38.69301319283493, ...)) | [9yz]
ksql> SELECT name, wkt, geo_hash(geometry_wkt, 4) FROM a04_geo_fence_s;
St. Louis, POLYGON ((-90.25749206542969 38.71551876930462, -90.31723022460938
38.69301319283493, ...)) | [9yzg, 9yzu]

28. 4) Geofences aggregated by GeoHash
Join Position
& Geofences
Stream
geofence
event
Geofences
gpby geohash
Table
{ "geohash":"u33", ["2:POLYGON ((-
90.23345947265625 38.484769753492536,…))"], …}
geo_fence()
geofence
Table
vehicle
position
Stream
{ "geohash":"9yz", "name":"St. Louis", "geometry_wkt":"POLYGON
((13.297920227050781 52.56195151687443, …))",
"last_update":1560607149015}}
Enrich with
GeoHash
Stream
geofences
& geohash
Enrich with
GeoHash
Stream
position &
geohash
geofences
by geohash
geo_hash()
geo_hash()
{ "geohash":"u33", "id" : "10", "latitude" : 52.3924, "longitude" : 13.0514}
{ "group":"u33", "name":"Berlin", "geometry_wkt":"POLYGON ((-
90.23345947265625 38.484769753492536,…))",
"last_update":1560607149015}
{ "geohash":"dnb", "name":"St. Louis", "geometry_wkt":"POLYGON
((13.297920227050781 52.56195151687443, …))",
"last_update":1560607149015}}
{ "geohash":"9yz", [ "1:POLYGON
((13.297920227050781 52.56195151687443, …))", ..]}
{ "geohash":"dnb", [ "1:POLYGON
((13.297920227050781 52.56195151687443, …))",..]}
geohash id latitude longitude
u33 1 52.3924 13.0514
9yz 4 38.4847 -90.23345
u34 3 .. ..
geohash geometry_wkt_list
9yz 1:POLYGON ((13.297920227050781 52.56195151687443, …))
N:POLYGON (( …. ))
u33 2:POLYGON ((-90.23345947265625 38.484769753492536,…))
dnb 1:POLYGON ((13.297920227050781 52.56195151687443, …))
high low
low high
low high
Scalable
Latency
"Code Smell"
medium
medium
medium

29. 4) Geofences aggregated by GeoHash
CREATE STREAM a04_geo_fence_by_geohash_s
AS SELECT EXPLODE(geo_hash(geometry_wkt, 3)) geo_hash,
id,
name,
geometry_wkt
FROM a04_geo_fence_s
PARTITION by geo_hash;
There was no explode() / unpivot() functionality in KSQL but now
ksqlDB provides it!
geofence
Table
Enrich with
GeoHash
Stream
geofence &
geohash
ksql> SELECT name, geo_hash FROM a04_geo_fence_and_geohash_s EMIT CHANGES;
| Colombia, Missouri | 9yz
| Berlin, Germany | u33
| St. Louis, Missouri | 9yz

30. 4) Geofences aggregated by GeoHash
CREATE TABLE a04_geo_fence_by_geohash_t
AS
SELECT geohash,
COLLECT_SET(geometry_wkt) AS id_geometry_wkt_list,
COLLECT_SET(id) AS id_list
FROM a04_geo_fence_and_geohash_s
GROUP BY geo_hash;
Geofences
gpby geohash
Table
Stream
geofences
& geohash
geofences
by geohash
ksql> SELECT * FROM a04_geo_fence_by_geohash_t EMIT CHANGES;
| 9yz | [POLYGON ((13.297920227050781 52.56195151687443, …)), POLYGON((…))] |[1,N]
| u33 | [POLYGON ((-90.23345947265625 38.484769753492536] |[2]
| dnb | [POLYGON ((13.297920227050781 52.56195151687443, …)), POLYGON((…))] |[1]
...

31. 4) Geofences aggregated by GeoHash
CREATE STREAM a04_vehicle_position_by_geohash_s
AS
SELECT vp.id, vp.latitude, vp.longitude,
geo_hash(vp.latitude, vp.longitude, 3) geo_hash
FROM vehicle_position_s vp
PARTITION BY geo_hash;
vehicle
position
Stream
Enrich with
GeoHash
Stream
position &
geohash
ksql> SELECT * FROM a04_vehicle_position_by_geohash_s EMIT CHANGES;
| 10 | 52.4497 | 13.3096 | u33
| 11 | 38.521846880854966 | -90.19912719726561 | 9yz
...

32. 4) Geofences aggregated by GeoHash
geohash id latitude longitude
u33 1 52.3924 13.0514
9yz 4 38.4847 -90.23345
nnn 3 .. ..
geohash geometry_wkt_list
9yz 1:POLYGON ((13.297920227050781 52.56195151687443, …))
N:POLYGON (( …. ))
u33 2:POLYGON ((-90.23345947265625 38.484769753492536,…))
dnb 1:POLYGON ((13.297920227050781 52.56195151687443, …))
geoh id latitude longitude geometry_wkt_list
u33 1 52.3924 13.0514 1:POLYGON ((13.297920227050781 52.56195151687443, …))
9yz 4 38.4847 -90.23345 1:POLYGON ((13.297920227050781 52.56195151687443, …))
N:POLYGON (( …. ))
nnn 3 .. ..

33. 4) Geofences aggregated by GeoHash
CREATE STREAM a04_geo_fence_status_s
AS
SELECT vp.id, vp.latitude, vp.longitude, vp.geo_hash,
explode (gf.id_list) AS geofenceId,
geo_fence (vp.latitude,
vp.longitude,
explode (gf.wkt_list)) AS geo_event
FROM a04_vehicle_position_by_geohash_s vp
LEFT JOIN a04_geo_fence_by_geohash_t gf
ON (vp.geo_hash = gf.geo_hash);
ksql> SELECT * FROM a04_geo_fence_status_s;
| 10 | 52.4497 | 13.3096 | u33 | 3 | OUTSIDE
| 11 | 38.521846880854966 | -90.19912719726561 | 9yz | 2 | OUTSIDE
| 11 | 38.521846880854966 | -90.19912719726561 | 9yz | 1 | OUTSIDE
...
Join Position
& Geofences
Stream
geofence
event

34. Berne
Fribourg
It works …. but ….
• Becaue of re-partitioning by
geohash we lose the guaranteed
order for a given vehicle
• Can be problematic, if there is a
backlog in one of the
topics/partitions
u0m5
u0m4
u0m7
u0m6
Consumer 1 Consumer 2

35. Implementing Geo Fencing -
using Kafka Streams

36. Geo-Fencing with Kafka Streams and Global KTable
Enrich Position with
GeoHash & Join
with Geofences
Global
KTable
geofence
KTable
vehicle
position
{ "geohash":u33", "name":"Potsdam",
"geometry_wkt":"POLYGON ((5.668945 51.416016, …))",
"last_update":1560607149015}
Enrich and Group
by GeoHash
matched
geofences
Detect Geo
Event
geofence_
status
high low
low high
low high
Scalable
Latency
"Code Smell"
medium
medium
medium
geofence
by geohash
{"id":"10", "latitude" : 52.3924,
"longitude" : 13.0514, [
{"name":"Berlin"} ] }
{ "geohash":"u33", "id" : "10", "latitude" : 52.3924, "longitude" : 13.0514}
{"id":"10", "status" : "ENTER", "geofenceName":"Berlin"} }
position &
geohash
{ "geohash":u33", "name":"Berlin",
"geometry_wkt":"POLYGON ((-90.23345947265625
38.484769753492536,…))",
"last_update":1560607149015}
{ "geohash":"u33", [ {"name":"Berlin",
"geometry_wkt":"POLYGON ((-90.23345947265625
38.484769753492536,…))"},{"name":"Potsdam",
"geometry_wkt":"POLYGON ((5.668945 51.416016,
…))"} ] }
{ "geohash":"9yz", [ {"name":"St. Louis",
"geometry_wkt":"POLYGON ((13.297920227050781
52.56195151687443, …))"} ] }{ "geohash":"9yz", "name":"St. Louis",
"geometry_wkt":"POLYGON ((13.297920227050781
52.56195151687443, …))",
"last_update":1560607149015}}

37. Geo-Fencing with Kafka Streams and Global KTable
KStream<String, GeoFence> geoFence = builder.stream(GEO_FENCE);
KStream<String, GeoFence> geoFenceByGeoHash =
geoFence.map((k,v) -> KeyValue.<GeoFence, List<String>> pair(v,
GeoHashUtil.coverBoundingBox(v.getWkt().toString(), 5)))
.flatMapValues(v -> v)
.map((k,v) -> KeyValue.<String,GeoFence>pair(v, createFrom(k, v)));
KTable<String, GeoFenceList> geofencesByGeohash =
geoFenceByGeoHash.groupByKey().aggregate(
() -> new GeoFenceList(new ArrayList<GeoFenceItem>()),
(aggKey, newValue, aggValue) -> {
GeoFenceItem geoFenceItem = new
GeoFenceItem(newValue.getId(), newValue.getName(),
newValue.getWkt(), "");
if (!aggValue.getGeoFences().contains(geoFenceItem))
aggValue.getGeoFences().add(geoFenceItem);
return aggValue;
},
Materialized.<String, GeoFenceList,
KeyValueStore<Bytes,byte[]>>as("geofences-by-geohash-store"));
geofencesByGeohash.toStream().to(GEO_FENCES_KEYEDBY_GEOHASH,
Produced.<String, GeoFenceList> keySerde(stringSerde));

38. Geo-Fencing with Kafka Streams and Global KTable
final GlobalKTable<String, GeoFenceList> geofences =
builder.globalTable(GEO_FENCES_KEYEDBY_GEOHASH);
KStream<String, VehiclePositionWithMatchedGeoFences> positionWithMatchedGeoFences =
vehiclePositionsWithGeoHash.leftJoin(geofences,
(k, pos) -> pos.getGeohash().toString(),
(pos, geofenceList) -> {
List<MatchedGeoFence> matchedGeofences = new ArrayList<MatchedGeoFence>();
if(geofenceList != null) {
for (GeoFenceItem geoFenceItem : geofenceList.getGeoFences()) {
boolean geofenceStatus =
GeoFenceUtil.geofence(pos.getLatitude(), pos.getLongitude(),
geoFenceItem.getWkt().toString());
if(geofenceStatus)
matchedGeofences.add(new MatchedGeoFence(geoFenceItem.getId(),
geoFenceItem.getName(), null));
}
}
return new VehiclePositionWithMatchedGeoFences(pos.getVehicleId(), 0L,
pos.getLatitude(), pos.getLongitude(),
pos.getEventTime(), matchedGeofences);
});

39. Implementing Geo Fencing -
using Tile38

40. Tile38
• https://tile38.com
• Open Source Geospatial Database & Geofencing Server
• Real Time Geofencing
• Roaming Geofencing
• Fast Spatial Indices
• Pluggable Event Notifications

41. Tile38 – How does it work?
> SETCHAN berlin WITHIN vehicle FENCE OBJECT
{"type":"Polygon","coordinates":[[[13.297920227050781,52.56195151687443],[1
3.2440185546875,52.530216577830124],[13.267364501953125,52.45998421679598],
[13.35113525390625,52.44826791583386],[13.405036926269531,52.44952338289473
],[13.501167297363281,52.47148826410652], ...]]}
> SUBSCRIBE berlin
{"ok":true,"command":"subscribe","channel":"berlin","num":1,"elapsed":"5.85
µs"}
.
.
.
{"command":"set","group":"5d07581689807d000193ac33","detect":"outside","hoo
k":"berlin","key":"vehicle","time":"2019-06-
17T09:06:30.624923584Z","id":"10","object":{"type":"Point","coordinates":[1
3.3096,52.4497]}}
SET vehicle 10 POINT 52.4497 13.3096

42. Tile38 – How does it work?
> SETHOOK berlin_hook kafka://broker-1:9092/tile38_geofence_status WITHIN
vehicle FENCE OBJECT
{"type":"Polygon","coordinates":[[[13.297920227050781,52.56195151687443],[1
3.2440185546875,52.530216577830124],[13.267364501953125,52.45998421679598],
[13.35113525390625,52.44826791583386],[13.405036926269531,52.44952338289473
],[13.501167297363281,52.47148826410652], ...]]}
bigdata@bigdata:~$ kafkacat -b localhost -t tile38_geofence_status
% Auto-selecting Consumer mode (use -P or -C to override)
{"command":"set","group":"5d07581689807d000193ac34","detect":"outside","hoo
k":"berlin_hook","key":"vehicle","time":"2019-06-
17T09:12:00.488599119Z","id":"10","object":{"type":"Point","coordinates":[1
3.3096,52.4497]}}
SET vehicle 10 POINT 52.4497 13.3096

43. 1) Enrich with GeoFences – aggregated by geohash
geofence
Stream
vehicle
position
Stream
Invoke UDF
{ "id" : "10", "latitude" : 38.35821, "longitude" : -90.15311}
Invoke UDF
Geofence
Service
geofence
status
set_pos()
set_fence()
Stream
null
high low
low high
low high
Scalable
Latency
"Code Smell"
medium
medium
medium
{ "id":11", "name":"Berlin", "geometry_wkt":"POLYGON ((-
90.23345947265625 38.484769753492536,…))",
"last_update":1560607149015}
{ "id":10", "name":"St. Louis", "geometry_wkt":"POLYGON
((13.297920227050781 52.56195151687443, …))",
"last_update":1560607149015}

44. 2) Using Custom Kafka Connector for Tile38
geofence
vehicle
position
Geofence
Service
kafka-to-
tile38
kafka-to-
tile38
geofence
status
high low
low high
low high
Scalable
Latency
"Code Smell"
medium
medium
medium
{ "id":11", "name":"Berlin", "geometry_wkt":"POLYGON ((-
90.23345947265625 38.484769753492536,…))",
"last_update":1560607149015}
{ "id":10", "name":"St. Louis", "geometry_wkt":"POLYGON
((13.297920227050781 52.56195151687443, …))",
"last_update":1560607149015}
{ "id" : "10", "latitude" : 38.35821, "longitude" : -90.15311}

45. 2) Using Custom Kafka Connector for Tile38
curl -X PUT
/api/kafka-connect-1/connectors/Tile38SinkConnector/config
-H 'Content-Type: application/json'
-H 'Accept: application/json'
-d '{
"connector.class":
"com.trivadis.geofence.kafka.connect.Tile38SinkConnector",
"topics": "vehicle_position",
"tasks.max": "1",
"tile38.key": "vehicle",
"tile38.operation": "SET",
"tile38.hosts": "tile38:9851"
}'
Currently only supports SET command

46. Visualization using Arcadia
Data

47. Arcadia Data https://www.arcadiadata.com/