Many companies develop strong software development practices that include ongoing testing throughout the development lifecycle but fail to account for the testing of security-related use cases. This leads to security controls being tacked on to an application just before it goes to production. With security controls implemented in this manner, more security vulnerabilities will be found with less time to correct them. As more applications move to cloud-based architectures, this will become an even greater problem as some of the protection enjoyed by applications hosted on-premise no longer exists. Jeremy Faircloth discusses a better approach—ensuring that testing throughout the development lifecycle includes the appropriate focus on security controls. Jeremy illustrates this through the establishment of security-related use cases, static code analysis, dynamic analysis, fuzzing, availability testing, and other techniques. Save yourself from last minute security issues by proactively testing the security of your application!

1. T23
Continuous Testing
5/8/2014 3:00:00 PM
Ensuring Security through
Continuous Testing
Presented by:
Jeremy Faircloth
UnitedHealth Group
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Jeremy Faircloth
UnitedHealth Group
An IT practitioner with a background in a wide variety of technologies, Jeremy Faircloth has
experience managing technical teams at multiple Fortune 50 companies. Described as a
"Renaissance man of IT" with more than twenty years of real-world IT experience, Jeremy has
become an expert in many areas including web development, database administration,
enterprise security, network design, large enterprise applications, and project management. He
is an author who has contributed to more than a dozen technical books on a variety of topics
and teaches courses on many of those topics.

3. 4/26/2014
1
Ensuring Security through
Continuous Testing
Jeremy Faircloth
Who am I?
• Security Guy
• Enterprise Applications Admin/Architect
• Development/Operations Manager
• Author – 16 Published Titles

4. 4/26/2014
2
What’s this about?
• Uh Ensuring security through
continuous testing?
• Let’s break that down
Background
• What are “vulnerabilities”?
– Vulnerabilities are software bugs that allow the
software to behave in an unexpected manner.
• So what?
– Vulnerabilities lead to “exploits”, the malicious
usage of a vulnerability to disrupt or misuse the
software.
• One of the functions of “IT Security” is the
prevention of this malicious activity.
– This can be through inclusion in the SDLC or
through controls that exist external to the SDLC.

5. 4/26/2014
3
You’re losing me
• What does a vulnerability look like and
how is it exploited?

6. 4/26/2014
4
Gettin’ Funky
I’m lazy.

7. 4/26/2014
5
What’s the impact?
Security = Important Got it!
Whose job is it?
The security guys!!! The developers!!!

8. 4/26/2014
6
What are developers doing
today?
• The developers are trained on security (a
little) and are told to use “secure
development patterns”.
• Does this work?
– Security is NOT a developer priority – features
are.
– Developer skill and knowledge varies – and
security is very specialized.
– Development means “figuring out how to make it
work”. When it’s working then security elements
are typically added.
What are security guys doing
today?
• While the development is happening,
sometimes “static code analysis” is done.
– But few developers understand the results.
• When development and testing are complete,
the application is scanned with “dynamic
analysis” and “penetration testing”.
• Security-related bugs are found and quickly
patched by development just before the
application goes to Production.
– Usually by developers who have already moved
on to building the next feature.

9. 4/26/2014
7
This works!
• Well defined secure coding patterns help
ensure secure code.
• Scanning and remediating issues helps
harden the application.
• Having individuals trained in security
performing development and interpreting
scan results means more secure code.
until it doesn’t.
• Good developers with strong security
knowledge have a high turnover rate.
• Future application updates tend to
introduce new vulnerabilities and aren’t
always subject to full security scanning.
– SDLCs move FAST now!
• Security bugs are usually classified as less
important than bugs in functionality.

10. 4/26/2014
8
What about all that “security stuff”?
• Security is changing
– The “perimeter” is gone.
– The world is moving faster.
– The attackers are getting better.
• “With more attacks, more applications,
less control, and less time to work, I can’t
keep up.” – A Security Guy
What’s the solution?
TESTERS!!!

11. 4/26/2014
9
Why just the security guys?
• Many tools used by the security guys can
be used by anyone as long as they can
interpret the results.
• This is no different from other forms of
application testing; it’s just different use
cases.
• Tools + knowledge = Better testing results!
Three steps to paradise!
•• Teach security to testersTeach security to testers
and provide tools.and provide tools.
•• Teach security to testersTeach security to testers
and provide tools.and provide tools.1
•• Develop securityDevelop security--relatedrelated
use cases and test.use cases and test.
•• Develop securityDevelop security--relatedrelated
use cases and test.use cases and test.2
•• Record the defectsRecord the defects
correctly.correctly.
•• Record the defectsRecord the defects
correctly.correctly.3

12. 4/26/2014
10
Teach testers security and provide
tools.
• Tools are important, but interpreting
results is even more important.
– Many false positives
and assumed false positives
– Code vs. infrastructure
– False negatives ?!?
Develop security-related use cases
and test.
• Negative/misuse cases:
– A misuse case is the negative form of a use
case.
– A use case is a countermeasure to a misuse
case.
• Examples!!!
– Single quote
– Field size
– Unexpected input

13. 4/26/2014
11
A better example
Requirement: The field shall accept numbers between 1 and 600
Positive Use Cases:
– Enter 1
– Enter 600
– Enter 10 numbers between 1 and 600
Negative Use Cases:
– Enter -1
– Enter 0
– Enter 601
– Enter A
– Enter 1.1
– Enter @#$
– Enter ¿
– Enter nothing (blank)
– Enter ‘
– Enter <script>alert();</script>1
Record the defects correctly
• Security bugs should be HIGH priority (Sev 1)
• Developers need bugs presented in a way they
understand
– Bad Bug: Field XYZ is vulnerable to SQL injection
– Good Bug: Field XYZ is accepting input outside of the
required format/length causing a critical error
• Sending bugs like this back to developers forces
them to create more secure code

14. 4/26/2014
12
Why testers are our heroes
• Security guys know security, but not the
application.
– Testers can test for vulnerabilities in process flows.
– Testers know the application better than anyone
and know how to break it.
– Tools only get you so far
• Testers have “early access” to the application
– The earlier a vulnerability is caught, the better the fix
will be.
• If testers can test for common vulnerabilities,
security guys can focus on the less common or
more complex.
What’s this about?
• Ensuring security through continuous testing.
– Ensuring security:
• Identifying vulnerabilities
• Recording security defects correctly
• Forcing development of secure code
– through continuous testing.
• Creating negative use cases
• Testing for common vulnerabilities
• Including security testing earlier in the SDLC
• Testing security use cases constantly

15. 4/26/2014
13
How do we do this?
• If you’re a tester
– Learn more about security!
• Attend security courses
• Learn how to hack!
• Try to break things
– Encourage your security guys to work with you.
• Some security guys don’t like to share sorry.
• Most do! Learn from them!
– Work with your management to change how
testing is done.
• Include negative use cases
• Include use cases that focus on security
How do we do this?
• If you lead a testing team
– Learn more about security!
• Attend security courses
• Learn how to hack!
• Try to break things
– Encourage your security guys to work with your team
• Forge partnerships between security and testing
• Include security guys in conversations about testing
processes
– Change how testing is done
• Train your testers on security
• Help them to understand negative use cases from the
security perspective
• Support them in including security testing

16. 4/26/2014
14
Summary
• Vulnerabilities = Bad
• Testing = Good
• Testing for Vulnerabilities = Very Good
• Secure Applications = GREAT!
Questions?

17. 4/26/2014
15
THANK YOU!