HouSecCon 2019 Offensive Security - Starting from Scratch. Learn from Spencer Koch and Altaz Valani about how to build an offensive security program from scratch, incorporating application security, infrastructure vulnerability management, hardening, devsecops, security champions, and red teaming. Be able to organize these capabilities to tell a story and build maturity to help your organization be more secure. Includes gotchas and lessons learned from industry experience.

1. Offensive Security Transformation:
Starting from Scratch
Spencer Koch
Offensive Security Leader, Securimancy
Altaz Valani
Cyber Security Researcher, Security Compass

2. Who We Are
Spencer Koch
Offensive Security Leader
@sp3nx0r
securimancy.com
Altaz Valani
Cyber Security Researcher, Security
Compass
securitycompass.com

3. This talk is:
• Aimed at security leaders early on
in the maturity curve
• A cohesive storyboard for
disparate capabilities that may
already exist within SecOps teams
• Pointers on what worked well and
resources to start down this path
• Both a practitioner and research
perspective
This talk is not:
• Going to go deep in implementation
or tooling details
• Going to be news to anyone doing
red teaming or having mature
appsec capabilities
About This Talk

4. How I structured my OffSec department:
Application
Security DevSecOps Security
Champions
Hardening Infra Vuln
Management Red Team

5. You have inventories galore...?
• Application listing
• Infrastructure
• Crown jewels / risk rankings
/ BIAs
You have a Dev Team...?
You have a Blue Team...?
You have permission to fire...?
• Buy-in from leadership
• Know where the fragile
network bits are
You aren't a jerk...?
Huge Assumptions

6. Application Security (Appsec) Overview
Design
Requirements
Dependency
Checking
Source Code
Analysis
Dynamic App
Testing
Gray Matter
Testing
http://safecode.org/wp-content/uploads/2018/01/SAFECode_Dev_Practices1108.pdf
Require-
ments Design Develop Build Test Deploy

7. • Doesn't work: Providing a long list of policies (be
specific)
• Begin with driving the right security requirements
• Goal is traceable and reusable requirements into the
DevOps pipeline
• Don't start from scratch, fork and make your own
• OWASP ASVS or NIST, pepper in regulatory reqs
• Tie into your DevOps training
• Establish de facto minimum standard against range of
app architectures
Design Requirements
Design
Requirements
Dependency
Checking
Source Code
Analysis
Dynamic App
Testing
Gray Matter
Testing
https://github.com/OWASP/Top10
https://github.com/OWASP/ASVS/tree/master/4.0

8. • One small step into the business world...
• Eventually need to convert 'policy speak' to 'DevOps
speak'
• Agile stories? Microsoft SDL? Custom?
• Create a "policy to execution" pipeline for scaling
• For large orgs, eventually extend policy to
execution into services layer
Design Requirements
Business
Risk
Policies
mitigate
d by Procedures
enforced
through
https://www.owasp.org/index.php/OWASP_Security_Knowledge_Framework
https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.04162018.pdf

9. • Dependency management
• Start here! Easy, cheap, gateway drug
• Built in Github, OSS like OWASP Dependency Check,
freemium Snyk
• Frameworks need patching just OSes do, don't
forget about them
• Linters / Code Quality
• Easy mode checks: dangerous/deprecated functions,
simple regexes, easy analytics
• Lots of devs think this is enough
• Examples: SonarQube, TSLint
Source Code Analysis
https://github.com/OWASP/Top10
https://github.com/OWASP/ASVS/tree/master/4.0
Design
Requirements
Dependency
Checking
Source Code
Analysis
Dynamic App
Testing
Gray Matter
Testing

10. • Dedicated security scanners,
able to follow source to sink
• Pay an arm/leg (money and
resources) to get false
positives
• No one SAST to rule them all –
and that's ok!
• Give/take with UI, CI/CD
integration, SaaS, languages,
speed
• Great SAST listing:
https://github.com/mre/awesome-
static-analysis
• Hardest problem? Sizing for
licenses (how's that inventory)
Source Code Analysis -
SAST

11. • How are you pushing security
defects to backlog? ALM
Integrations
• DO NOT PRINT TO PDF & CHUCK
• Customize your own ruleset /
criticality ranking
• You will spend a lot of time here
trying to get this right...
Source Code Analysis Gotchas
• Use those IDE integrations – devs don't
want another tool

12. • Coding standards for the team: it should exist and
include security topics!
• Appsec team sets the structure (e.g. how to solve for
input sanitization, output encoding, database
interaction, authX, logging)
• Dev teams fill in their language/framework specific
ways of achieving these guidelines
• Build secure libraries if devs' framework doesn't
handle it!
• Infrastructure as Code (IaC) also needs standards too
• Leverage others' work! CERT, OWASP, something...
Secure Coding Standards
https://wiki.sei.cmu.edu/confluence/display/seccode
https://www.owasp.org/index.php/OWASP_Secure_Coding_Practices_Checklist

13. • Build on what is already done today
(peer review? self review? no
review?)
• Define what needs to be reviewed:
security features, APIs, code
dealing with money/PII, first-of
code using new
framework/design/etc.) at a minimum
• Keep it short! Don't kill them with
checklists...
• Code review notes become training
vehicle
• Architectural design decisions vs.
syntax
Code Reviews including Security!

14. • Requires compiled code, comes later in SDLC cycle
• Can be automated / integrated into CI/CD
• Works via HTTP request/response - great for web/mobile
apps & APIs
• Catches a different & overlapping set of vulns from
SAST
• Fewer false positives, and triage effort generally
less than SAST
• What about WAF / virtual patching? Yep, do this...
• What about IAST/RASP? Don't worry about it yet...
Dynamic Application Testing (DAST)
https://github.com/OWASP/Top10
https://github.com/OWASP/ASVS/tree/master/4.0
Design
Requirements
Dependency
Checking
Source Code
Analysis
Dynamic App
Testing
Gray Matter
Testing

15. • Every PMs' favorite step, or AKA "where the greenlight
happens”
• Pentesting != Secure App <- culture change
• Shifting security left is hard, so spin as security ==
quality
• Don't waste resources on simple, repeatable tasks - save
these folks for the "hard stuff": business logic, authX,
session management, crypto
• Go broad: systemic testing beyond the app (adversaries
won't respect app boundaries), work top-down to provide
contextual priority of vulns
• Expensive to execute, expensive to fix defects, always
timeboxed
• Avoid RCP: pick a partner(s), pick a framework, get
involved
Gray Matter Testing
Design
Requirements
Dependency
Checking
Source Code
Analysis
Dynamic App
Testing
Gray Matter
Testing

16. • That's a lot of testing, so what's your plan?
• Utilize a framework (OWASP SAMM 2.0?) and benchmark your
progress
• Don't lose focus while chasing maturity – learn from CMMI
• This never ends – continual improvement, continual
reassessment
• Don't be afraid to add your own capabilities – remain relevant
and look for what's critical for assurance to your business
Appsec Framework

17. • Start small:
• # of apps onboarded /
total apps
• Vulnerability type by
time (SQLi, XSS, etc.)
• Average time to
remediation
• Even those 3 can be hard
• Focus on telling a story
and grow
• Stay away from # of vulns
metrics, stick to behavior
around those vulns
Metrics

18. • if (!infra_vuln && !appsec): devsecops = doomed
• DevSecOps Manifesto by Larry Macherone: Build
security in more than bolt it on
• All about about automating manual security testing
processes
• Make friends with DevOps, understand pipelines
• Build at lab/home, follow along with Securing DevOps
by Julien Vehent
• With this understanding, where to start putting Sec
in DevOps?
DevSecOps – Getting Started
https://medium.com/continuous-agile/the-devsecops-manifesto-94579e0eb716

19. • SANS Secure DevOps Toolchain -
epically good at framing your
capability discussion!
• Build a mindmap, add your priorities
and start filling in blanks within
your org
• Be Agile just like the dev teams –
iterate!
• Keep it automated, reduce friction
SANS Secure DevOps Toolchain
https://www.sans.org/security-resources/posters/secure-devops-toolchain-swat-checklist/60/download

20. • Be choosey about what "breaks the build",
start without any breaks
• Plan out how security defects make it into
the backlog: manual is painful
• Get involved in backlog prioritization,
don't be unreasonable
• Come up with a plan and prioritize: ease to
implement, bang for buck
• Examples: dependency checking, to secrets
scanning, to CIS benchmarks, to OWASP Top
10 DAST scanning, to …
• Infra is always easier than app
• Expect to have to code some stuff, get
your hands dirty
DevSecOps Gotchas

21. Don't be afraid of "bimodal DevSecOps": quick & dirty vs.
deep-dive
DevSecOps Gotchas
CI/CD Integrated Testing
•Focus on speed and keeping
signal-to-noise ratio low
•Completes in minutes, not
days
•Catch "heavy hitting" vulns
(OWASP Top 10)
Out-of-Band Testing
•Scheduled on a frequency
•Full code base, full URI
path, full everything
•Appsec engineer prunes
false positives

22. • Devs will resist; too much on their plate already
• Start with a respected developer interested in
security
• Designate as security champion and commit (not a part
time role; needs budget & training)
• Keep the momentum – it's change management (Kotter,
PDCA, ..)
• Get some wins (measurable ROI on pre/post prod vuln
metrics) across different champions and build toward
a SC program
• Build word of mouth, sell career development for
devs, success spreads
• Not worth pursuing nay-sayers, let them get FOMO
• Scale to a community of practice that offers advice,
tools, onboarding, etc.
Security Champions
https://safecode.org/?s=security+champions
https://www.managementstudyguide.com/kotters-8-step-model-of-change.htm

23. • Work with your sysadmins, be reasonable, keep them
happy
• Where are you today? Start with CIS/NIST for core
OSes, make the standards your own, document your
deviations
• You will have hundreds of findings at the beginning –
don't panic, it's normal
• Work the list - prioritize in phases, based on
criticality and testing timing
• After a few rounds, expand to less familiar platforms
(middleware, cloud platforms, network gear)
• Patterns will emerge, don't worry if there's no CIS
benchmark for $STUFF
Hardening
https://www.cisecurity.org/cis-benchmarks/
https://nvd.nist.gov/ncp/repository
https://nvd.nist.gov/ncp/checklist/811 - RHEL
https://nvd.nist.gov/ncp/checklist/560 - Win2012

24. • Scan all the things – this is the easiest part of
this talk
• Don't trust your asset data, and add your results to
IT asset data
• Automate infrastructure scanning/validation and tie
to risk tolerance for key applications (don't do this
alone, get the business involved – they own risk
register)
• Infra vuln management is collaborative - understand
impact to sysadmins and get their help to create the
right processes and data
• Infrastructure as Code gets devs involved too, codify
your hardening/scanning
Infra Vuln Management
https://www.inspec.io/
https://archerysec.info/

25. • Frame your scan objectives:
discovery, subnet/asset, agent
• DON'T PRINT TO CSV – utilize
best fix paths and pick
reasonable goals
• If you can't explain what the
vuln is / how the vuln was
detected, go research it
• Sell to sysadmins as
validation of patching/config,
you're in it together!
• Spot the patterns – don't
forget about feedback loops
Infra Vuln Management Gotchas

26. • Don't do this if you aren't
doing basic infra vuln
management, go focus on that
• Focus on blue team
effectiveness at the
beginning
• Visualize gaps and help Blue
Team be better
• Your reports have power –
write each scenario up as a
budget request
Red Team

27. • Phishing activities
• OSS tools galore like Gophish, just add AWS
infrastructure
• Social engineering
• Call center password reset
• Signature Emulation – is your $TOOL detecting what
you think it does?
• (Network) BT3 – Blue Team Training Toolkit
/ (Host) Red Canary
• Active Directory health (Bloodhound / PingCastle)
• Awesome Redteaming lists – pick your poison
Red Team Introductory Activities
https://www.redcanary.com/blog/atomic-red-team-testing/
https://github.com/BloodHoundAD/BloodHound
https://www.pingcastle.com/
https://github.com/yeyintminthuhtut/Awesome-Red-Teaming
https://github.com/gophish/gophish
https://www.bt3.no/

28. • Continue to build, automate where possible
• See a technique on Twitter, try it out and see if
Blue Team sees it
Red Team Final Notes
• Feedback loop forevah
• Always focus on ROI and value
proposition, or face budget cuts
• Don't expect or try to be the next
APT

29. Problem: Everything's going to the cloud! Oh nooooes…
Approaches that work:
• Fight for testing as it moves – you may never see
that app again!
• Shift security testing focus from depth to breadth
• Change methodologies – don't be the blocker
• Change the duration – 3 apps per week
• Stress importance of risk registers and good
documentation
• Pick the hill(s) to die on – care about the most
critical vulns
Bonus: Cloud Migrations

30. Ditch FUD, embrace the new
• Use Infrastructure as Code to your benefit
• Use Cloud Service Provider tools to your benefit
• Push for network segmentation
• Encourage use of cloud devops tools – they're likely
more secure than self-managed
Hardest part of migration: getting funding to fix
those uncovered vulns
Bonus: Cloud Migrations
https://github.com/dev-sec - IaC hardening
https://github.com/aws-quickstart/quickstart-compliance-cis-benchmark - CSP tooling
for hardening/alerting

31. • Still a fledgling discipline, room to
explore
• Get involved with DevSecOps
communities, local chapters
• Partner with researchers
• Industry collaboration - reach out to
us!
Next Steps / Where To Go From Here