Presentation I gave to Developer Week Europe 2022 on the use of Fluentd in Hybrid and distributed use cases. This builds on previous Fluentd presentations

1. Fluentd – Making Logging Easy & Effective
in a Multi-cloud & Hybrid Environment
Name
Phil Wilkins
OCI
April 2022

2. Speaker
PhilWilkins
Cloud Developer Evangelist
Copyright © 2022, Oracle and/or its affiliates
Philip.Wilkins@Oracle.com
cloud-native.info / mp3monster.org / oracle-
integration.cloud
www.linkedin.com/in/philwilkins/
@mp3monster

3. 3 Copyright © 2022, Oracle and/or its affiliates
https://www.manning.com/books/unified
-logging-with-fluentd
https://bit.ly/FluentdBook
Philip.Wilkins@Oracle.com
cloud-native.info / mp3monster.org / oracle-
integration.cloud
www.linkedin.com/in/philwilkins/
@mp3monster

4. The following is intended to present ideas & application of open
technologies. It is intended for information purposes only, and may
not be incorporated into any contract. It is not a commitment to
deliver any material, code, or functionality, and should not be relied
upon in making purchasing decisions. The development, release,
timing, and pricing of any features or functionality described for
Oracle’s products may change and remains at the sole discretion of
Oracle Corporation.
This presentation contains the thoughts and ideas of
this presenter & does not necessarily represent the
views of Oracle Corporation.
4 Copyright © 2022, Oracle and/or its affiliates

5. Putting Monitoring & Log Analytics into Perspective
Our view of monitoring is heavily influenced by our role.
• Devs think app logs –(structured) text
• Infra think numeric – server performance/resource
consumption etc.
• Business / Low Coders use both numeric & text –
transaction counts & status
DevOps is challenging that and ideas like 3 Pillars of
Observability have matured:
Copyright © 2022, Oracle and/or its affiliates
Metrics – Typically
Numerical (sample based
data e.g. CPU use)
Logs Textual (event based
data e.g. app logging,
SNMP traps)
Observability Pillars / Key
Traces (execution flow &
timing – transaction
based, sampled)
Host / Infrastructure
Monitoring
Virtualization /
Container Monitoring
Application Monitoring
Business Application
Monitoring
Security
/
SIEM
Capacity
Monitoring
5

6. What is Fluentd ?
6 Copyright © 2022, Oracle and/or its affiliates

7. Highly Pluggable Framework
Input
• TCP/UDP
• Unix Sockets
• HTTP
• Many file formats
• SNMP traps
• OS (Linux/UNIX)
• Log4J, SLF4J and other
related frameworks for .Net,
JavaScript
Output
• ALM solutions e.g. Splunk, cloud native
solutions, loggly, logzio etc.
• HTTP
• Prometheus
• Grafana
• Many file formats
• DB (SQL/NoSQL)
• Event Streams e.g. Kafka, Kenesis, MQTT
• Social notifications e.g. Jabber, Slack, emai,
twilio l etc
• Support mgmt tools like Pager Duty
Buffer /
Cache
• Custom in memory cache
• Redis & Coherence
Storage
• S3 buckets
• DB (No) SQL
• File
Formatter
• XML
• JSON
• CSV/TSV etc.
• Compressed formats
Parser
• Multline text to single event
• Event info extraction e.g. date & time Filters
• Value based conditions
• REGEX expressions
Custom components
Amusing Ruby Gems it is possible to build any
custom components using the framework provided

8. Fluentd Provided Language Libraries
Copyright © 2022, Oracle and/or its affiliates
8

9. Language Logging Frameworks with Fluentd direct support
Copyright © 2022, Oracle and/or its affiliates
9
Java, .Net
Rust - logrs
Perl
NodeJS - Pino
Multiple Open Source
Appenders/Adaptors
Multiple Open Source
Appenders/Adaptors
Serilog – VB.net R – lgr

10. Data to Actionable Information
10 Copyright © 2022, Oracle and/or its affiliates
Information
Source Capture
• Infra structure such as
CPU, memory use
• JVM use
• App Log Files
• SNMP Traps
Structure
& Route
• Get the raw data
to the
appropriate
tooling in a
format that can
be processed
Aggregate &
Analyze
•Data from
multiple sources
•Merge in time
series
Visualize
Data
• Search for log
events
• Present trends
e.g. memory
consumption,
• Rate of storage
consumption
Notify &
Alert
•Push events
into JIRA Svc
Desk, Slack, etc
•Rules on
severity dictate
behaviours
Fluentd – Optimal
Fluentd – Leverage Other Tools

11. How Does this relate to Oracle and other hyperscalers?
11 Copyright © 2022, Oracle and/or its affiliates
• For Oracle Cloud Log management…
• leverages Fluentd – offers Fluentd
compliant endpoints.
• can share logging content back to
Fluentd endpoints
• Other Hyperscalers (AWS, GCP) also
leverage Fluentd to varying degrees (GCP
lead the way)
• Fluentd has plugins for Cloud Vendor
monitoring integration
• Verrazano (open-source pure CNCF
platform) as a pure multi-cloud solution
uses Fluentd
• Other Container platforms like OpenShift
also leverage Fluentd

12. Data Egress costs $€£¥
12 Copyright © 2022, Oracle and/or its affiliates
OCI
$0.02 - $0.16 per GB
$0.08 per GB
$0.09 per GB
$0.01 - $0.02 per GB
$0.0085 - $0.05
per GB
$0.01 - $0.15 per GB
$0.085 per GB
$0.043 per GB
Traffic costs vary by
Geographic region –
South America & Far
East most expensive
Europe & North
America cheapest

13. Balancing $€£¥ with visibility
13 Copyright © 2022, Oracle and/or its affiliates
OCI
- Consider
adapting a centre
of mass for
critical E2E
visibility
- Centre of mass =
where most log
traffic is
generated
- Do need to trade
off technology
need, capacity &
bandwidth cost
- But have regional
richer logs in the
event of needing
more detail

14. Addressing Real-world Challenges
In the realworld, we have more significant challenges …
• Highly distributed solutions that need to have logging and monitoring consolidated
• Tracing for reactive and solutions – context switching rather than threads in the execution
• Often different teams want to use different tools – security want Splunk, DBAs want OEM,
infrastructure teams want Nagios – making the setup of environments more complex than needs be
• Some operational events are more critical than others – need to filter those out
• Make legacy solutions easier to operate, isolate log events and tag them with operational code
references – so process & care are embedded without impacting the app
14 Copyright © 2022, Oracle and/or its affiliates

15. Possible scaling & deployment approaches
15 Copyright © 2022, Oracle and/or its affiliates
Single instance
- Often how people tend to think
of log unification tools (multiples
when app bundles solution
- Presents lots of issues for multi-
cloud / hybrid
Resilient Pairing
- Monolith approach of hot
standby approach (Presents
lots of issues for multi-cloud /
hybrid)
- OR 1 Node per cloud/DC
Distributed Instances with Central Focus
- More resilient,
- Ability to control data flow, local control
of connection failure
- Supports highly distributed
Fluentd has a very small footprint
- Lots of deployments – becomes an
issue of patching/config change
- Even smaller footprint possible with
Fluent Bit
- From deployment on IoT to containers,
servers and Mainframes

16. Possible scaling & deployment approaches
16 Copyright © 2022, Oracle and/or its affiliates
Single instance
- Often how people tend to think
of log unification tools (multiples
when app bundles solution
- Presents lots of issues for multi-
cloud / hybrid
Resilient Pairing
- Monolith approach of hot
standby approach
- Presents lots of issues for multi-
cloud / hybrid
Distributed Instances with Central Focus
- More resilient,
- Ability to control data flow, local control
of connection failure
- Supports highly distributed
Fluentd has a very small footprint
- Lots of deployments – becomes an
issue of patching/config change
- Even smaller footprint possible with
Fluent Bit
- From deployment on IoT to containers,
servers and Mainframes

17. Fluentd – Scaling & Aggregation
17 Copyright © 2022, Oracle and/or its affiliates
App A
(Front End)
Svr
App B
VM
Svc
Shared Persistence /
Analytics Platform
Ops
Alerting
Service
Pod
App A
(Mid Tier)
Svr
App A
(Mid Tier)
VM
App C
VM
Svc
Pod

18. Fluentd – Scaling & Aggregation – Cloud / DC
18 Copyright © 2022, Oracle and/or its affiliates
App A
(Front End)
Svr
App B
VM
Svc
Shared Persistence /
Analytics Platform
Ops
Alerting
Service
Pod
App A
(Mid Tier)
Svr
App A
(Mid Tier)
VM
App C
VM
Svc
Pod
Cloud Region
Own DC

19. Fluentd – Scaling & Aggregation - Kubernetes / Container
19 Copyright © 2022, Oracle and/or its affiliates
App A
(Front End)
Pod
App B
Pod
Svc
Shared Persistence /
Analytics Platform
Ops
Alerting
Service
Pod
App A
(Mid Tier)
Pod
App A
(Mid Tier)
Pod
App C
Pod
Svc
Pod
Worker Node
Worker Node

20. Fluentd – Scaling & Aggregation - Kubernetes / Container – using Side
Cars
20 Copyright © 2021, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted
App A
(Front End)
Pod
App B
Pod
Pod
Shared Persistence /
Analytics Platform
Ops
Alerting
Service
Pod
App A
(Mid Tier)
Pod
App A
(Mid Tier)
Pod
App C
Pod
Pod Pod
Worker Node
Worker Node
Side
Car
Side
Car
Side
Car

21. Demo…
Copyright © 2022, Oracle and/or its affiliates

22. 22 Copyright © 2022, Oracle and/or its affiliates
Central Node (Node 2)
(Single Instance)
Node 1
(Instance n)
common
• filter
Slack
Stdout /
Monitoring
Warboard
Op Analytics &
AI Ops
filters
• Transsform (event 
message)
• Match + copy:
• Out file
• relabel
labelPipeline
basic-file.txt
basic-file2.txt
label-pipeline-file-output.*
forwarder
We need to control this
flow to avoid a
notification storm!

23. Questions / Thank you
Copyright © 2022, Oracle and/or its affiliates
Phil Wilkins
Cloud Developer Evangelist
Philip.Wilkins@Oracle.com
cloud-native.info / mp3monster.org / oracle-
integration.cloud
www.linkedin.com/in/philwilkins/
@mp3monster