Presented at the CSS in Prague in 2016 by Thomas Malmberg Risk & IT-Security Consultant, Mint Security Ltd.

1. Thomas Malmberg
Risk & IT-Security Consultant, Mint Security Ltd
Software Development Life Cycle –
Managing Risk and Measuring Security

2. Thomas Malmberg - @tsmalmbe  tweet!
oWorks as an independent consultant @ Mint
Security
oThis presentation contains anonymized and
generalized cases based on real life
experience during the past few years.
2010 2015

3. Mint Security Ltd
o Founded in 2015 – specialized in data & cyber
security and IT-risk & security management
Software and Software
Development Security
Purchasing, Project & Management
Consulting
Architectures and Security Cyber & IT Risk Management

4. An Introduction
to this Presentation
(SECURE) Software Development Life Cycle

5. What is an SDLC?
oSoftware Development Life Cycle
o Defines a process – does not imply any specific
model (agile, waterfall, whatever rocks your
ship)
o Relevant for companies who are doing serious
software development
o Is not the same as DevOps
o Works as a base framework for all development
o Can be audited, reviewed and assessed
o Can be aligned to security standards
o Can be communicated and understood

6. What’s your relation to the SDLC?
1. You develop software for yourself
o You need quality and measurability
2. You develop products
o You need reliability and continuity
3. You develop software for your
customers
o You face contracts and warranty-demands
4. You buy custom developed software
o You demand transparency and warranty

7. Why should my SDLC be secure?
“Organizations with a proper SDLC will
experience an 80 percent decrease in
critical vulnerabilities”
“Organizations that acquire products and
services with just a 50 percent reduction in
vulnerabilities will reduce configuration
management and incident response costs
by 75 percent each”

8. Is this presentation purely theoretical?
”SOMETHING
GOES WRONG”
Plant bad code
(add vulnerability)
CUSTOMER
Deliver (vulnerable) devices
CUSTOMER CUSTOMER
EXPLOIT !

9. Is this presentation purely theoretical?
Breached!?!
CUSTOMER
OTHER
CUSTOMER
CUSTOMER
OTHER
CUSTOMER
< CONFUSION > < CONFUSION^2 >
< CONFUSION^3 >

10. ”SOMETHING GOES WRONG”
DEVELOPMENT ENVIRONMENT
BREACH
Juniper said that someone managed to get
into its systems and write “unauthorized
code” that “could allow a knowledgeable
attacker to gain administrative access.“
(CNN 18.12.2015)

11. How do I know my SDLC is secure?
o In a nutshell
o You develop securely
o You develop secure software
o How do I achieve this?
o Harden your processes
o Harden your development
environment
o Harden your people

12. What we will cover today
1. Extending Your Security Policy to Cover the
SDLC
2. Aligning with Development Standards and
Best Practices
3. Assessing SDLC Risk – Different Viewpoints
4. Metrics and Correlations to the Rescue
5. Real Life Examples and Tools

13. Extending Your Security Policy
to Cover the SDLC
ISO27002 – PCI-DSS

14. Everyone has a security policy..
o...but policies differ a lot – some are not
formalized – or even written down!
oA proper SDLC may not demand a security
policy – but a proper security policy
demands an SDLC
oISO27002 provides ”good common
security ground” for this presentation
oThere are demands on the development
process, the development environment
as well as the deliverables

15. ISO27002 implicit requirements
oYou must protect your source code (9.4.5)
oDon’t mix development and production
(12.1.4)
oYour development process must be
secure (14.2.1)
oAssess your SDLC risk (14.2.6)
oPerform security testing (14.2.8)
NOTE! In addition to this, there are 1) OPERATIONAL
requirements for running software and 2) other (but not as
implicit) requirements that software development should adhere
to.

16. Dissecting the ISO27002 for SDLC requirements
27002 Description SDLC viewpoints & examples
6.1.1 Information security roles and
responsibilities
Who is responsible for the environment
security
6.1.2 Segregation of duties Coders, administrators, network operators
7.2.2 Information security, awareness,
education and training
Let the people know there are policies and
requirements and how to work
8.1.3 Acceptable Use of Assets Guides for working in the development env
8.2.1 Classification of information Development code, production code, docs
9.1.1 Access Control Policy VCS system is not an island
9.2.1 Access to networks and network
services
Dev systems must be part of the
enterprise IAM

17. Dissecting the ISO27002 for SDLC requirements
27002 Description SDLC viewpoints & examples
9.2.3 Management of privileged access
rights
Limit roots and admins on the dev servers
9.4.1 Information access restriction Not all code is available to everyone
9.4.5 Access to program source code Prevent introduction of unauthorized
functionality
12.1.4 Separation of development,
testing and operational
environments
Self explanatory
12.4.1 Event logging Logging is not only for production systems
12.4.3 Administrator and operator logs What are your admin’s credentials doing
13.1.1 Network controls Segragate and protect

18. Dissecting the ISO27002 for SDLC requirements
27002 Description SDLC viewpoints & examples
13.1.3 Segregation in Networks Really, segregate
13.2.1 Information transfer policies and
procedures
Can your code travel
14.2.1 Secure development policy ”Code of conduct” for coding
14.2.2 System change control
procedures
Make the dev environment part of change
management
14.2.5 Secure system engineering
principles
Documented instructions for creating
systems
14.2.6 Secure development environment Assess your risks
14.2.8 System security testing Test the software you create

19. Suggestion for alignment
oAdd at least some concrete notes on the
SDLC directly to your policy
o If you have a well-documented SDLC that
includes security – put a reference to it into
14.2.1
oIf you start fresh or have a big gap, write
a policy for your SDLC based on hand-
picked sections of the 27002
o This will help and guide your software team

20. What about PCI-DSS?
oThe only implicit demand is domain 6 –
”Develop and Maintain Secure Systems
and Applications” – but
o PCI-DSS and ISO27002 can be mapped to each
other (but it is far from 1:1 though so caveat
emptor)
o Don’t force more than one set of requirements
on your teams – so enforce and expose them to
your security policy – and then
o Map your policy to other requirements and
standards as you need to and see fit – behind
the scenes

21. Dissecting the PCI-DSS for SDLC requirements
27002 PCI-DSS 27002 PCI-DSS 27002 PCI-DSS
6.1.1 7 9.2.3 7,2 13.1.3 1,6
6.1.2 7 9.4.1 3 13.2.1 4
7.2.2 12 9.4.5 6 14.2.1 6
8.1.3 12 12.1.4 3 14.2.2 6
8.2.1 - 12.4.1 10 14.2.5 2,6,11
9.1.1 7 12.4.3 10 14.2.6 6
9.2.1 8 13.1.1 1 14.2.8 6

22. Prepare to measure
oSpend some time thinking about KPI’s and
KRI’s
oConsider adding some measuring
guidelines to your policy as concrete
demands
1. You get what you measure
2. You are what you measure
3. You can’t manage what you don’t measure

23. Aligning with Development Standards
and Best Practices
SAMM - DevOps

24. Ideas for a secure SDLC?
oDefining security domains and
classifications – defining your ”security
quality”
o Managing development phase code access
o Software design
o Writing and auditing code
o Release management
o QA & pentesting
o Managing production environments

25. Example security domains
External
Services
Privacy
Intensive
Development process
Security requirements
Policies
PROCESSES
Internal
Services
SECURITY
DOMAINS

26. When writing detailed policies and
setting demands,
choose and use an existing SDLC
framework to ensure coverage.

27. Say hello to SAMM
OWASP SAMM - Software Assurance Maturity Model
 Covers mostly development, not the dev environment

28. SAMM is a maturity model
1. Evaluate where you are – and more
importantly – where you need to be
2. Evaluate your top-10 areas of risk – the
areas where failure is not an option, or
in any case very expensive – and create
a roadmap
3. Evaluate areas where added security
and quality can be capitalized upon
4. Start measuring as soon as possible to
create a good baseline

29. Other frameworks than SAMM?
oNIST SP800-64
oBSIMM
oGAISP/GASSP
oCLASP
oMicrosoft SDL
oTouchpoints
oTSP-Secure

30. SAMM and ISO27002?
6.1.1
6.1.2
9.1.1
7.2.2
8.1.3
8.2.1
9.2.1
9.2.3
9.4.1
9.2.3
9.4.1
9.4.5
9.4.5
13.1.3
14.2.1
9.4.5
12.1.4
14.2.6
14.2.5 14.2.8
12.4.1
12.4.3
13.1.1
14.2.2
13.2.1
NOTE! Operations
security has
intentionally been
left out!

31. SAMM and DevOps?
SECOPSDEV
Underlying governance
and management

32. A quote on SAMM and DevOps
Adam Muntner, Security engineer at Mozilla
http://devops.com/2015/07/16/the-myth-of-devops-as-a-catalyst-to-improve-security/
“The thing to keep in mind is that the security
of an application environment is inherited –
it’s the aggregate result of all its component
pieces.”
”There is nothing inherent to DevOps that solves these [security]
problems and DevOps comes with its own potential pitfalls. The
problem is external to what DevOps can deliver. For organizations
that want to better understand the maturity of security in their
Software Development Lifecycle, OWASP OpenSAMM is a good place
to start.“

33. My take on SAMM and DevOps
oTo be truly secure, you will (anyway) need
to spend endless amounts of time and
money
oSAMM is about developing secure
software, not developing in a secure
environment
oSAMM does NOT fix everything
oDevOps is not a quick fix for your
development environment security

34. Your secure (?) environment
VPN
QA
VCS & CI
DEV
Juniper said that someone
managed to get into its
systems and write
“unauthorized code”

35. I’m confused - how do I spend my time money then?
oMoney is spent based on risk
assessments
o Spend money where you can loose
money  impact X probability
o Challenge – finding the risks and
assessing them realistically is always hard
and requires experience
o Saving on mapping out what your threats
and weaknesses are is NOT a good way to
spend money

36. Manage risk with tools already in your SDLC!

37. Assessing SDLC Risk –
Different Viewpoints
Risk Management for Software Development

38. The biggest risk is not assessing
SDLC risk at all.
The second biggest risk is failing
your risk assessment.

39. A key risk indicator (KRI) is a metric for
measuring the likelihood that the combined
probability of an event and its consequence will
exceed the organization's risk appetite and have
a profoundly negative impact on an
organization's ability to be successful.

40. Recap – Your role equals your risk-map
1. You develop software for yourself
o You need quality and measurability
2. You develop products
o You need reliability and continuity
3. You develop software for your
customers
o You face contracts and warranty-demands
4. You buy custom developed software
o You demand transparency and warranty

41. Risk – You develop software for yourself
1. Lack of (security) quality assurance
2. Unplanned and/or badly planned
production installations
3. Neglicence towards architecture
4. Trivializing of threats
5. Nonexistent privilege management in
the development environment

42. KRI – You develop software for yourself
1. The amount of security incidents
related to your development
2. Missing/uninstalled security updates or
patches
3. Security reviews (static code analysis)
vs. production installations (releases)
4. Threat monitoring for installed software
5. Access management system vs. access
logs

43. Risk – You develop products
1. Responsibility to your customer base –
including legal liabilities
2. Difficulty to patch on time
3. Planting of rogue code in your
environment
4. Outsourcing and IPR theft

44. KRI – You develop products
1. Quality Assurance results
2. Security testing results
3. Test pass rate
4. Test case comprehensiveness
5. External security reviews vs. internal
security reviews
6. Access management system vs. access
logs

45. Risk – You develop software for your customers
1. Responsibility to your customer base –
including legal liabilities
2. Customer inability to define and verify
security requirements
3. Nonexistent privilege management in
the development environment – with
the addition of customer access

46. KRI – You develop software for your customers
1. The contents of your agreements vs.
change request logs
2. Access management system vs. access
logs
3. External security reviews vs. internal
security reviews

47. Risk – You buy custom developed software
1. Responsibilites are not fulfilled
2. Requirements and contracts do not
reflect reality
3. Vulnerabilities due to bad quality code
4. Management of test-data (PII and EU
requirements etc.)

48. KRI – You buy custom developed software
1. The contents of your agreements vs.
change request logs
2. Supplier defect count vs. buyer
acceptance tests vs. warranty fixes
3. Defect amounts in code audits and
security tests
4. Process effectivity – deliverables vs.
Vulnerabilities

49. Notes about measurements and KRI
Measuring is part of
your maturity
Quantitative
measurements are
key – they provide
much more than
qualitative
guessworkKRI’s enable
proactivity instead of
reactivity

50. Metrics and Correlations
to the Rescue
Closer to practice with risk assessments and KRI’s

51. Where do we want to put our probes?
oThe deliverables – code
o Code quality
o Vulnerabilities
o Changes
oThe development environment
o Audit logs
o Access management

52. Do we have the necessary information?
oYes! Information is available but spread
around different systems – examples:
o Source code commits and fixes – in Git
o Builds, failures, QA tests – in the CI
o Static code analysis results – in the cloud
o Tickets and bugs from testing and production
– in Jira
o Code audit and review results – somewhere
o QA and test environments – here and there
o Access – VPN-gateways, ssh-logs, firewalls, ...
o Bug Bounty programs

53. So how should this information be used?
o Extract information from the development process
and measure and improve weak points and find and
emphasize strong points
o Make agile more reportable without taking away
the agility – enable both devs and managers
o C3 – Collect, Combine & Correlate
o Results for release metadata
o Automated test results
o Static code analysis
o Tickets and bugs – from customers and manual test cycles
o Find problems in your releases, processes, code
– find anomalies and “whoops” before the customers do
o Insight is everything

54. ...you get some valuable KPI’s..
o Knowledge will let you optimize important business
decisions
o Amounts of features per release
o The need for external audits & code reviews
o Development team sizes & outsourcing
o Knowledge will let your team leaders & scrum
masters better understand
o How the teams work
o Where there is lack of knowledge
o Workload effects on quality
o Knowledge will enable your business and your
development to work more closely together making
decisions based on hard facts

55. ..and insight into your level of security
o Audit trails – who did what and why it was approved
– and by whom.
o Security defects and issues reported to
issue-trackers are correlated to releases and projects
o Outsourced and “far-away-shored” teams can be
monitored and assisted
o Source-code leaks and tampering can be detected
o Security scan results can be leveraged
o Automatic (static) code-analysis can be leveraged
o Correlate with threat intel
o Transparency and visibility in the process adds natural
social pressure for correct behavior - You can even gamify
quality and security!

56. Real Life
Examples and Tools
Quantitive Measurements

57. Important metrics
o Commits per release
o Commits per team
o Static code analysis results per release
o Issues & bugs per release
o New issues & bugs per week
o Average release time
o Open issues per release
o Code coverage
o Executed unit tests executed per relases
o Executed unit test per week
o Test coverage
o Team overall activity
o Failed test cycles per commit
o Failed unit tests per release
o Ad-hoc commits (”no ticket”)
o Access to version management system
o Access to test environments
o Dormant privileged users
o Build / version management system
configuration changes
o Failed security tests
o Audit-logs for version management and CI
o New third-party libraries
o Deployers
o Deployment status

58. Correlations – KRI’s/KPI’s
o Commits vs. features
o Commits vs. reported issues
o Commits vs. test cycles
o Issues in static code analysis vs.
manual code review issue amounts
o Open issues vs. closed issues
o Closed issues vs. added lines of code
o Test cycles vs. reported productions
issues
o Unit test vs. static code analysis
results
o Defect rate vs. test cycles
o Static code issues vs. commits
o Performance figures vs. commits
o Release frequency vs. code issues vs.
defects
o Remote vs. local access to systems vs.
threat intel
o Malicious patterns in remote access vs.
normal access

59. Source: Secure Development Lifecycle – Eoin Keary & Jim Manico (OWASP)

60. The tools and the datasources
o System and access logs collected using Splunk
forwarders (SSH and RDP) or syslog (VPN)
o Static code analysis using the Veracode platform API
with Jenkins and integrated to Splunk using REST
o Security scan results can be integrated through logfiles
or existing Splunk Apps (example: Nessus & Beyond
Security)
o Performance data is integrated through logfiles
o Issue trackers (example: Jira) are integrated through
database connectors or API’s
o CI, version management, release automation tools are
integrated through logfiles
o Threat intel is added with Splunk Apps (example:
Verizon)

61. Connect the bits and pieces
o A vast amount of sources can
be hooked up and correlated
o Databases
o Text-based logs
o API’s
o Information can be visualized
and displayed differently to
various stakeholders
o NOTE! The schematic displays example
sources and is used to serve an example and
an overview – there is no limit to the
imagination of the stakeholders – anything can
be hooked up – but start simple

62. Generic case – Remote Development Teams
oReasoning and business case
o Mitigate risk related to remote teams and/or
outsourcing.
oIssues
o Sharing of VPN-tokens
o Tracking remote access IP’s against threat intel
o Alert on abnormal VPN-2-Code -behaviour

63. Generic case – Release Notes
oReasoning and business case
o Add relevant and valuable security information
to your release notes.
oIssues
o Commits not related to tickets
o Commits by people not part of the core team
o Changed vs. added vs. deleted code
o Residual static code scan findings

64. Generic case – Superuser access
oReasoning and business case
o Development teams are managing their own
access, including superuser access.
o Privileges are handled ”under the radar”.
oIssues
o Track all superuser accounts
o Monitor superuser access
o Alert on anomalous activities

65. Generic case – Policy violations
oReasoning and business case
o We want to enforce and monitor our security
policies even in the farthest of IT-trenches
oIssues
o Monitor password policies
o Address continuity requirements

66. This is the End
Almost

67. Studies and background material
o Coverity report
o http://www.coverity.com/library/pdf/open_source_quality_report.pdf
o Department of Homeland Security, Software Assurance Working Group - Software
Quality Metrics to Identify Risk
o http://www.mccabe.com/ppt/SoftwareQualityMetricsToIdentifyRisk.ppt
o Software Quality Metrics for your Continuous Delivery Pipeline – Part I
o http://apmblog.dynatrace.com/2014/03/13/software-quality-metrics-for-your-continuous-
delivery-pipeline-part-i/
o Secure Development LifeCycles (SDLC) – Bart De Win SecAppDev 2013
o https://handouts.secappdev.org/handouts/2013/Bart%20De%20Win/SecAppDev2013%20-
%20SDLC%20Session%20Bart%20De%20Win%20v1.0.pdf
o Secure Development Lifecycle – Eoin Keary & Jim Manico
o https://www.owasp.org/images/7/76/Jim_Manico_%28Hamburg%29_-
_Securiing_the_SDLC.pdf
o Motivation for Security Testing – manu Khari & Chetna Bajaj
o http://www.rroij.com/open-access/motivation-for-security-testing-26-32.php?aid=37877

68. Cologne, Germany Helsinki, Finland
Copenhagen, Denmark Helsinki, FinlandCologne, Germany
Prague, Czech Republic
@tsmalmbe
fi.linkedin.com/in/thomasmalmberg
thomas@mintsecurity.fi