1. SAST tools do not find all vulnerabilities on their own due to lack of code context and unknown user code. The human touch is needed to filter findings and identify the vulnerabilities that matter.
2. Not all applications have the same security needs, and "secure applications do not exist." How much effort is put into security depends on acceptable risk levels and the number of applications.
3. SAST is best for finding issues in your own code where you have context, while SCA is better for analyzing third party libraries with known vulnerabilities.
4. 1. SAST doesn’t find vulnerabilities
• Lack of context
• Unknown user code
• Mitigating factors
Finding Issue
The GREAT filter
• internal policy
• threat modelling
• industry standards
Vulnerability
everything the tool finds stuff you care about stuff that will hurt you
The human touch
WHY?
5. 1. SAST doesn’t find vulnerabilities
• Enterprise customer
• Too many issues
• Mostly dealing with Java web
Finding Issue
The GREAT filter
• Internal policy
Vulnerability
A few Thousand A few tens/hundreds A handfull
Security Champion
Story time
6. 2. Not all applications are born equal
“Secure applications do not exist” – me & others who tried
How much you should care
No. of apps
Understand acceptable risk
€£$
7. 2. Not all applications are born equal
“Secure applications do not exist” – me & others who tried
How much you should care
No. of apps
Understand acceptable risk
€£$
8. 2. Not all applications are born equal
“Secure applications do not exist” – me & others who tried
2.a – Shifting goes both ways
Code Build Test Security Release
“Shift left” – most marketing materials
9. 2. Not all applications are born equal
“Secure applications do not exist” – me & others who tried
2.a – Shifting goes both ways
Story time
”ALL code must be
scanned”
If every developer
scans their code
All the code is
scanned
Deploy a SAST tool to all developer
IDEs:
- report metrics
- cofirm scans happened
- generate reports
10. 3. Your code vs code you use
S A S T S C A
Static Application Security Testing Software Composition Aanalysis
Dangerous patterns in your code Dangerous libraries used by your application
11. Why not SCA:
3. Your code vs code you use
Your code 3rd Party Code
Infrastructure Code
Application Code
Why SAST: • Can explain context
• Know the code
• Have access to the code
Why not SAST: • Extremely high amounts of code
• No in depth code knowledge
Containers & stuff
App Dependencies
• Doesn’t really exist for your
own code
Why SCA: • Faster than SAST
• Information is specific to known
vulnerabilities in libraries
• Licensing information
• Constantly updated by researchers
Story time
12. 4. Some effort up front goes a long way
From this To this
Vulnerabilities
• Define the things you care about
• Remove the ones you doubt about
• Find your friends (security champions)
• Educate & set expectations
• Find out about policies & security frameworks
13. 4. Some effort up front goes a long way
Story time
”We want to
automate our SAST”
The tool can
integrate with CI
We’ll add a SAST
step
Tool was successfully deployed, but
developers complained:
- Too many findings
- False possitives
- Not enough time to review
- No knowledge on the topics
14. 5. Try it on your stuff
1. When should I test a SATS tool on a real app?
2. When should I make a decission based on a benchmark?
15. Some other things
Benchmarks:
- Highest score: “If you ask us to scan this 1 application we're like super really good at it“
- Popular apps: It's never "out of the box”
“Most accurate on the market”: only finds a few things
There's always a cost
16. Some other things
CWE count != coverage or quality (Vulns can be represented by multiple CWEs. Just because a
CWE is ticket it doesn't mean it will have the right rule for your scenario)
The vendors are mean: but we do know the tool and we
can help
But…
Avoid technology bias (eg. some vendors will overplay the importance of SCA vs the
importance of SAST, etc)
JavaScript sucks It really does!
17. Quick recap
1. SAST doesn’t find vulnerabilities
2. Not all applications are born equal
3. Your code vs code you use
4. Some effort up front goes a long way
5. Try it on your stuff