The difference between successfully defending an attack or failing to compromise is your ability to understand what’s happening in your network better than your adversary. Choosing the right network security monitoring (NSM) toolset is crucial to effectively monitor, detect, and respond to any potential threats in an organisation’s network. In this webinar, we’ll uncover the best practices, trends, and challenges in network security monitoring (NSM) and how Elastic is being used as a core component to network security monitoring. Highlights: - What is network security monitoring (NSM)? - Types of network data - Common toolset - Overcoming challenges with network security monitoring - Using Machine Learning for network security monitoring - Demo

1. Elastic for Network Security Monitoring
Mouaz Alnouri
Skilledfield
16 June 2021
1

2. About me
I’m Mouaz Alnouri, the Founder/Managing Director of Skilledfield.
We help clients unleash the power of big data technology to detect cyber
security events and utilise automation to efficiently alert, escalate and
respond to security threats.
Technologist with years of experience in solving complex business
problems through creative client-centric strategies and value-driven
solutions. A change agent, capable of orchestrating a transformative
business strategy through data-driven decisions.
2
Mouaz leads the Skilledfield
team with an unrivalled passion
for data and a zest for problem
solving. With over a decade in
the IT services industry, he’s
provided intelligent solutions
for complex problems
throughout his career. He’s
worked with major technology
and telecommunications firms
including Telstra and NBN Co.
Limited, where he’s delivered
data focused solutions that
have significantly improved
operational efficiency. He’s a
customer-focused problem
solver that leads the Skilledfield
team towards their vision to
become Australia’s leading Big
Data solutions provider.
https://www.linkedin.com/in/malnouri/

3. About Skilledfield: A Field of Skilled professionals!
3
Uplift Security Detection and
Response Capability
Uplift Observability Capability BI to AI Analytics Services
● Centralised Security Event
logging and auditing
● Endpoint Protection
● SOAR
● Machine Learning Security
Analytics
● Co-Managed Services
● Centralised Operational event
monitoring and alerting
● AIOps, (Artificial Intelligence for
IT operations)
● Co-Managed Services
● Big Data Analytics using Elastic
● Big Data Analytics using
Databricks
● Big Data Analytics using
Microsoft Azure Services
Solving Complex Problems with Simplified Solutions

4. Elastic Premier Partner
Skilledfield has been working closely with
our partner Elastic for more than 2 years on
significant deployments. We are thrilled to
be a "Premier" partner for Elastic in the
APAC region.
Our capabilities span all aspects of
Elasticsearch, Elastic SIEM, Elastic
Observability and Elastic Stack (ELK), with
certified consultants supporting our clients’
deployments and diverse use cases.
4

5. Areas of Expertise around ELK
➔ Elastic Cluster design and build
➔ Elastic Cluster tuning, upgrade and migration
➔ Beats deployment and configuration
➔ Dashboards build
➔ Parsers development
➔ Watchers Development
➔ ML Jobs Development
➔ Kibana Plugin Development
5
Experience with Security and
Observability use cases.

6. What we are covering today
➔ What is NSM
➔ Network Data
➔ NSM Common Toolset
➔ Overcoming Challenges with NSM
➔ Using ML for NSM
➔ Demo
6

7. What is Network Security Monitoring
Network Security Monitoring is simply the collection and analysis of
network traffic data to identify intrusions.
Network traffic data is one of the most important sources to monitor.
This is because all attacks have to go through the network. And while
endpoints can be compromised and the the reliability of what they
are telling you can be compromised as well; network traffic doesn’t lie.
Certainly bad guys try to hide inside of a network traffic. However,
they’ll still leave a footprint.
7

8. Network Data
Full Packet Capture (PCAP): Full network traffic packet capture where
you collecting every bit and byte crossing the wire. It’s 100% of what’s
happening on the network traffic. High volume data and very hard to
analyse.
Netflows: Provides high level contextual information about the
communication going across the network.
PCAP OR And Netflows
8

9. Product
Description An open source
network intrusion
intrusion prevention
system (IPS) uses a
series of rules that
help define malicious
network activity.
A rule and
signature-based
threat detection
engine.
A passive,
open-source network
traffic analyzer. It
monitors network
metadata and
produces logs.
Large scale packet
capture and search
to store and index
network traffic in
standard PCAP
format.
A data shipper installed
on your servers to sniff
the traffic, parse the
application-level
protocols and correlate
the messages into
transactions.
Features ● A simple, scriptable
configuration
● Plugin framework,
make key
components
pluggable (and 200+
plugins)
● Multi-threading
● Logging not only
packets but also
HTTP requests, DNS
requests,etc.
● Alert filtering
● Can detect patterns
of activity
● Supports a large
number of
protocols
● Supports a
large-scale
deployment
● PCAP browsing
search interface
● Timeline search
● Scalable
● Export PCAP
capability
● Integrated in to ELK
Stack
● Provides real-time
monitoring metrics
Limitations Snort has no real GUI
or easy-to-use
administrative
console.
Suricata has a
smaller community. It
also needs a web
front end tool for
analysis.
There’s no native GUI.
Also it is complicated
to set up
Not easy to pivot
since there is no
unique identifier for
the flows.
Protocols supported
are limited. Also it can
only support up to 40
Gbps, after that volume
there is packet drop.
NSM Common Toolset Analysis
9

10. Tools working together
PCAP
Endpoint Telemetry
Netflow
Alerts
10

11. Overcoming Correlation Challenges
With multiple tools, one needs to locate the required parts of the flow tuple (typically the IP
address and port of each endpoint, plus the transport protocol) in each log’s rendering,
combine them, and match them up. This “join” is tedious in the best case, and in corner
cases (specific ICMP message types, for example) can become fairly tricky.
Community ID flow hashing standard
The Community ID aims to simplify the correlation of flow-level logs produced by multiple
network monitoring applications. The ID standardizes the rendering of flow tuples into
hash-like strings, reducing the correlation to a simple string comparison.
Supported by Zeek, Suricata, Beats, Arkime, MISP, VAST for network Forensic, HELK hunting
Platform.
11

12. Centralised Architecture
12

13. Using ML for NSM
13
13

14. Indicators Of Behavior (IOB)
The New Telemetry To Find Advanced Cyber Attackers
Focus on what are people, applications and systems doing
Provides the new data source to single out the hackers from the
admins and legitimate users and applications
Requires new standards, new tools, new language to develop
There is no one size fits all. It depends on your business
14

15. Type of Behavioural Analysis
Unsupervised: Find patterns in your data then use time series
modeling to detect anomalies in your current data and forecast
trends based on historical data.
Supervised: Apply classification, regression, and outlier detection to
your data for an end-to-end workflow experience across a wide range
of use cases.
15

16. DNS Tunneling - Technique: T1071.004
Description Adversaries may communicate using the Domain Name System (DNS)
application layer protocol to avoid detection/network filtering by
blending in with existing traffic. Commands to the remote system, and
often the results of those commands, will be embedded within the
protocol traffic between the client and server.
Filter Network
Traffic
Consider filtering DNS requests to unknown, untrusted, or known bad
domains and resources. Resolving DNS requests with on-premise/proxy
servers may also disrupt adversary attempts to conceal data within
DNS packets.
IDS Network intrusion detection and prevention systems that use network
signatures to identify traffic for specific adversary malware can be used
to mitigate activity at the network level.
ML A machine learning job detected unusually large numbers of DNS
queries for a single top-level DNS domain. 16

17. Unusual Network Destination Domain Name
Description Command and Control consists of techniques that adversaries may use
to communicate with systems under their control within a victim
network. There are many ways an adversary can establish command
and control with various levels of stealth depending on the victim’s
network structure and defenses.
For example: when a user clicks on a link in a phishing email or opens a
malicious document, a request may be sent to download and run a
payload from an uncommon web server name. When malware is already
running, it may send requests to an uncommon DNS domain the
malware uses for command-and-control communication.
ML A machine learning job detected an unusual network destination
domain name. This can be due to initial access, persistence,
command-and-control, or exfiltration activity.
17

18. Unusual Web Request
Description Command and Control consists of techniques that adversaries may use to
communicate with systems under their control within a victim network. There are
many ways an adversary can establish command and control with various levels of
stealth depending on the victim’s network structure and defenses.
For example: in a strategic web compromise or watering hole attack, when a trusted
website is compromised to target a particular sector or organization, targeted
users may receive emails with uncommon URLs for trusted websites. These URLs
can be used to download and run a payload. When malware is already running, it
may send requests to uncommon URLs on trusted websites the malware uses for
command-and-control communication. When rare URLs are observed being
requested for a local web server by a remote source, these can be due to web
scanning, enumeration or attack traffic, or they can be due to bots and web
scrapers which are part of common Internet background traffic.
ML A machine learning job detected a rare and unusual URL that indicates unusual
web browsing activity. This can be due to initial access, persistence,
command-and-control, or exfiltration activity.
18

19. Demo
19
19

20. Elastic Enterprise Search Elastic Security
Elastic Observability
Kibana
Elasticsearch
Beats Logstash
Elastic
Cloud
Elastic Cloud
on Kubernetes
Elastic Cloud
Enterprise
Self-Managed
Elastic Stack
Endpoint
SaaS Orchestration
Powered by the
Elastic Stack
3 solutions
Deployed
anywhere
The Elastic Stack
20

21. Elastic Common Schema (ECS)
● Defines a common set of fields
and objects to ingest data into
Elasticsearch
● Enables cross-source analysis of
diverse data
● Designed to be extensible
● ECS is in GA and is being
adopted throughout the Elastic
Stack
● Contributions & feedback
welcome at
https://github.com/elastic/ecs
21

22. Architecture
22

23. Take a quick spin
23
Demo

24. Thank you!
info@skilledfield.com.au
24
https://skilledfield.com.au/
https://www.linkedin.com/company/skilled-field/