Introduction to Application Security Testing

Security Verified

Introducing Security Testing

Mohamed Ridha Chebbi, CISSP
ridha.chebbi@icodesecurity.com
Mobile : +216 26 88 10 11

© 2012 iCode information security All rights reserved

Analyzing Applications with a Security Mindset Security Verified

Testing the security of software requires a certain mindset, as a security
test engineer needs to look at software from the perspective of someone
who wants to attack it.
Adopting this mindset involves understanding the main information
sources for security testing and how input can affect software.
Testing the security of a given piece of software often entails searching a
very large set of inputs in order to find issues that can compromise that
security.
Automation plays an important role in this search.


The Value of Security Testing Security Verified

In the past, Microsoft products used to be the primary targets for attackers. As a result
of their prominent position in the industry, Microsoft operating systems received high
scrutiny from the security community. This coupled with the fact that Microsoft
products did contain a fair number of vulnerabilities, hackers and vulnerability finders
had little need and interest in searching other platforms.

The ubiquity of Microsoft products exacerbated this situation, as viruses and worms
were able to cause epidemics by taking advantage of vulnerabilities in commonly used
server and client applications.

The Security Development Life Cycle (SDL) introduced at Microsoft in 2002 has raised
the bar. Vulnerability finders have to work harder to penetrate Microsoft applications,
and the focus of the security research community is now shifting to those vendors and
platforms once considered to be secure.

Well-engineered security testing plays a key role in the effectiveness of the SDL.
Vulnerabilities found during the SDL have less chance of damaging customer confidence
in Microsoft, and removing critical flaws before shipping leaves attackers with fewer
reasons to focus on Microsoft.


Security Testing and the SDL Security Verified

Anecdotal evidence indicates that engineers spend less time testing for security bugs
in products covered by the SDL.
As the design and coding for these products is more secure in the first place, security
testing in the SDL is more concerned with verifying that the product is secure, rather
than with finding and eliminating security bugs.
The purpose of security testing in the SDL is to allow the product team to perform a
thorough assessment of the security and privacy of their application.

Performing security testing during the verification phase of the SDL allows the product
team to:
Perform adequate penetration testing activities on new and legacy code.
Verify that the application is adequately reviewed for security vulnerabilities.
Ensure that threats against the application are properly mitigated.
Document the evidence indicating that users can trust the application.
Perform a security push for those legacy components that were not covered by the
SDL.


Functional Testing vs Security Testing Security Verified

Security testing and functional testing differ in four main areas:
Identifying helpful sources of information for test planning : Functional testing involves
testing an application under realistic user scenarios, while in security testing the test
engineer frequently deals with scenarios that may not be realistic for the common user.
Choosing an appropriate set of inputs : During functional testing the test engineer will
most commonly use expected inputs derived from use cases, whereas during security
testing the test engineer is required to use unexpected and malicious inputs that may
expose a security problem.
Automating security tests. Interfaces that are not visible to the human eye are most
interesting for security testing. In order to be viewed, such interfaces often require the
use of specialized tools. Security test engineers must be familiar with these tools and
their limitations. During functional testing it is simpler to automate a testing process, as
results for a specific test are easily verified. Establishing whether a security defect has
been triggered during security testing is much harder.
Deciding when testing is complete : When carrying out functional testing, it is
straightforward to determine when testing is complete. When carrying out security
testing, however, deciding when to stop is more complicated because the test engineer
needs to help to quantify the risk and decide whether enough testing has been done to
declare the product "safe".


Gathering Information for Security Testing Security Verified

The User Interface as a Source of Information :
The user interface is the first place to investigate when planning security testing. The
information on the screen is primarily directed at legitimate users, but it is important to
remember that hackers are users as well, and they can learn quite a bit from this
information.
Identify the Field :
When testing a Web application, testers should be aware of how their interactions with
the client will affect the behavior of the server application
Error Messages as a Source of Information :
Many applications output error messages to their users to inform them on what the
expected format of input is. This can provide very useful information to the tester on the
possible assumptions that developers made on the type of data that are expected
through a given input vector.
Abuse Cases as a Source of Information :
Whereas functional test engineers consider use cases, security test engineers must
consider abuse cases.
Abuse cases can often be gleaned from sources of historical information such as:
CERT and Bugtraq, which contain security advisories and discussions about existing
and new vulnerabilities.
Black-hat literature.


Considering Inputs with a Security Testing Mindset Security Verified

Identifying Entry Points of the Application :

Not all inputs come from the keyboard or mouse. From a security perspective, the
Graphical User Interface (GUI) is often the least of concerns, and testers need to
worry more about input sources that are not quite so visible. In fact, a security
tester has to learn not to think like a GUI tester.

There are four sources of input that need to be considered when planning security
testing :
OS/RTE : Input from the OS and the Run-time Environment (RTE) includes input from the
OS APIs and any run-time environment that is supporting the application, such as the
Common Language Run-time (CLR).
User Interface (UI) : Examples of user interfaces are a local GUI and a Web front-end.
External Resource : A Web server and a database are examples of resources that are
external to the application.
File System : Input from the file system includes temporary files, registry values, and
configuration files.


Categorizing Inputs Security Verified

Categorizing Dangerous Inputs
Lists of dangerous inputs are a key asset for security test engineers.
Within an organization, such lists are often kept secret for fear that they may be used
against a system by attackers.
A group should be diligent about doing the necessary research to ensure it has a
complete list, as well as ensuring that everyone in the organization is aware of such a list
and utilizes it.
Dangerous input can be categorized into the following groups:
Long strings
Format strings
Numeric boundaries
Scripts
Code
OS commands
Control characters
Error codes
Return values

Note that such lists are never static

Automating Security Testing Security Verified

The Value of Automated Security Testing
Larger coverage : overage refers to the percentage of security-critical
functions that are exercised by different inputs during a security test.
Regression testing : During regression testing an application is tested
for newly introduced bugs, usually by re-running previous tests and
checking whether previously fixed faults have re-emerged
Efficiency : Automated security tests are more efficient than manual
testing. Automated tests are repeatable with no human intervention,
as opposed to manual testing, which is far more time-consuming.
Automation allows for more targeted manual testing as they disclose
areas of potential weakness in various execution, trust, and I/O
boundaries, which means the test engineer need not try to find all
weaknesses manually, but can focus on key areas and provide
feedback to threat modeling and other phases of SDL.


Generating the Input Security Verified

In order to generate inputs that are dangerous to your application, a list
detailing all possible inputs must be created. This list might be generated
in real time as the software is being tested, manually or by using a fuzz
test automation suite, or it may already be available to you from other
security tests applied to similar products. If the latter, you must keep in
mind that often a list is not cross-applicable, and you may need to modify
it or create your own list.

Some interfaces, such as the network and the file system, lend
themselves well to automation. For other interfaces it might be more
difficult to automate testing because the instrumentation of them is more
difficult and/or because their outputs are harder to observe.


Software you Actually Build Security Verified

The actual behavior of an application, as coded by developers, rarely matches perfectly
with the intended software behavior, as specified in the design documents.

Most security bugs are found in the area where behavior that was not part of the
application’s intended behavior has been inadvertently implemented by developers.


Quantifying the Risk Security Verified

Quantifying the Risk : To help quantify the risk, you have to ask the following:
Was the process which was established to prevent and detect security flaws fully
implemented?
How many unmitigated vulnerabilities are currently extant?
Is additional testing necessary?
What can we do to make this product safer?
Assigning Severity: To help test engineers assign severity to defects, the SDL has established
four different levels of severity:
Critical :
Remote, anonymous user escalation of privilege
Arbitrary code execution
Important
DoS (low bandwidth attack, blue-screen, or long duration)
Local elevation of privilege
Information disclosure with privacy implications
Tampering with user data
Spoofing a user or computer
Moderate
Temporary DoS
Information disclosure, general
Low
Non-persistent or hard-to-replicate issues


Review the Process Security Verified

Collecting the Proper Metrics:
To make a well-informed decision on
whether to stop testing requires
relevant metrics.
questions need to be asked :

How well did the team:
- Create required documentation?
- Adhere to best practices?
- Use all appropriate tools?
- Respond to problems that arose?
How well did the testing team test:
- Untrusted interfaces? resources? protocols? files?
How well did the testing team cover the:
-Threat model?
-Set of possible attacks that apply to the application?


Security Verified

Security Assurance



Security Assurance Maturity Security Verified

High

Typical
Progression
Curve
Tools & Technology

Low Panic Scramble Pit of Despair Security as Core Business Process
Low High

People & Process


Stage 1 : Reactionary Security Verified


Stage 2 : Apply Band Aids Security Verified


Stage 3 : Beyond Badness-Ometer Security Verified


Stage 4 : Teach a Man to Fish Security Verified


Stage 5 : Homo Securous Security Verified


Typical Level 5 SAMM Assessments in SDLC Security Verified

Software Development Life Cycle

Design Build Deploy Operate

Code Review

Build & Test Automation

White Box Testing

Black Box Testing

Pen Test
Decision Support & Process Optimization


Security Verified

Security Testing
Simplified



Automated Testing During SDLC Security Verified

Design Build Test Deploy Operate

Internal Review Annually
Static Testing …
Dynamic Testing
…
PEN Testing

New Versions/Releases


Application Security Assurance Security Verified

Design Build Test Deploy Operate

Internal Review Annually
Static Assessment …
Dynamic Assessment

…
PEN Testing

New Versions/Releases


Security Verified

Static Testing Vectors


Static Assessment Vectors Security Verified

CWE ID CWE Name
Directory Traversal
73 External Control of File Name or Path
Untrusted Search Path
114 Process Control
426 Untrusted Search Path
427 Uncontrolled Search Path Element
Buffer Overflow
121 Stack-based Buffer Overflow
122 Heap-based Buffer Overflow
Buffer Management Errors
129 Unchecked Array Indexing
135 Incorrect Calculation of Multi-Byte String Length
170 Improper Null Termination
249 Often Misused: Path Manipulation
125 Out-of-bounds Read
Format String
134 Uncontrolled Format String
Numeric Errors
190 Integer Overflow or Wraparound
191 Integer Underflow (Wrap or Wraparound)
192 Integer Coercion Error
195 Signed to Unsigned Conversion Error
196 Unsigned to Signed Conversion Error
197 Numeric Truncation Error
API Abuse
234 Failure to Handle Missing Parameter
243 Failure to Change Working Directory in chroot Jail
193 Off-by-one Error
245 J2EE Bad Practices: Direct Management of Connections
560 Use of umask() with chmod-style Argument



CWE ID CWE Name
Information Leakage
201 Information Leak Through Sent Data ✔
209 Error Message Information Leak ✔
215 Information Leak Through Debug Information ✔
497 Information Leak of System Data ✔
498 Information Leak through Class Cloning ✔
Dangerous Functions
242 Use of Inherently Dangerous Function ✔
Credentials Management
256 Plaintext Storage of a Password ✔
259 Hard-Coded Password ✔
Authentication Issues
247 Reliance on DNS Lookups in a Security Decision ✔
Cryptographic Issues
311 Failure to Encrypt Sensitive Data ✔
316 Plaintext Storage in Memory ✔
321 Use of Hard-coded Cryptographic Key ✔
326 Weak Encryption ✔
331 Insufficient Entropy ✔
347 Improper Verification of Cryptographic Signature ✔
354 Improper Validation of Integrity Check Value ✔
Sensitive Cookie in HTTPS Session Without
614 'Secure' ✔
310 Cryptographic Issues ✔



CWE ID CWE Name
Race Conditions
364 Signal Handler Race Condition ✔
367 Time-of-check Time-of-use (TOCTOU) Race Condition ✔
Time and State
377 Insecure Temporary File ✔
378 Creation of Temporary File With Insecure Permissions ✔
382 J2EE Bad Practices: Use of System.exit() ✔
557 Concurrency Issues ✔
708 Incorrect Ownership Assignment ✔
Session Fixation
384 Session Fixation ✔
Error Handling
391 Unchecked Error Condition ✔
Code Quality
111 Direct Use of Unsafe JNI ✔
Failure to Release Memory Before Removing Last
401 Reference ('Memory Leak') ✔
404 Improper Resource Shutdown or Release ✔
415 Double Free ✔
416 Use After Free ✔
Use of Externally-Controlled Input to Select Classes or
470 Code ('Unsafe Reflection') ✔
479 Unsafe Function Call from a Signal Handler ✔
489 Leftover Debug Code ✔
597 Use of Wrong Operator in String Comparison ✔



CWE ID CWE Name
OS Command Injection
Failure to Preserve OS Command Structure ('OS Command
78 Injection') ✔
SQL Injection
89 Failure to Preserve SQL Query Structure ('SQL Injection') ✔
564 SQL Injection: Hibernate ✔
Cross-Site Scripting (XSS)
Improper Sanitization of Script-Related HTML Tags in a
80 Web Page (Basic XSS) ✔
CRLF Injection
93 Failure to Sanitize CRLF Sequences ('CRLF Injection') ✔
Failure to Sanitize CRLF Sequences in HTTP Headers ('HTTP
113 Response Splitting') ✔
117 Improper Output Sanitization for Logs ✔
Encapsulation
494 Download of Code Without Integrity Check ✔
501 Trust Boundary Violation Malicious ✔
Malicious Code
506 Embedded Malicious Code ✔
511 Logic/time bomb ✔
Reliance on Security through Obscurity Untrusted
656 Initialization ✔
Untrusted Initialization
15 External Control of System or Configuration Setting ✔
Insufficient Input Validation
103 Struts: Incomplete validate() Method Definition ✔
104 Struts: Form Bean Does Not Extend Validation Class ✔
112 Missing XML Validation ✔
601 URL Redirection to Untrusted Site ('Open Redirect') ✔

OWASP Top 10 Compliance Security Verified

Automated Scanning
•Cross Site Scripting (XSS)
•Injection Flaws
•Malicious File execution
•Insecure Direct Object Reference
•Information Leakage and Improper Error Handling
•Insecure Cryptographic Storage

Dynamic/Manual Testing Required to identify
•Cross Site Request Forgery (CSRF)
•Broken Authentication and Session Management
•Insecure Communications
•Failure to restrict URL access


SANS Top 25 Compliance Security Verified

Automated Scanning
CWE-20: Improper Input Validation
CWE-89: Failure to Preserve SQL Query Structure ('SQL Injection')
CWE-78: Injection Flaws Failure to Preserve OS Command Structure (‘OS Command Injection’)
CWE-209: Error Message Information Leak
CWE-426: Untrusted Search Path
CWE-494: Download of Code Without Integrity Check
CWE-259: Hard-Coded Password
CWE-330: Use of Insufficiently Random Values
CWE-116: Improper Encoding or Escaping of Output
CWE-79: Failure to Preserve Web Page Structure ('Cross-site Scripting')
CWE-362: Race Condition
CWE-119: Failure to Constrain Operations within the Bounds of a Memory Buffer
CWE-404: Improper Resource Shutdown or Release
CWE-682: Incorrect Calculation
CWE-327: Use of a Broken or Risky Cryptographic Algorithm
CWE-73: External Control of File Name or Path


SANS Top 25 Compliance Security Verified

Dynamic/Manual Testing Required to identify
CWE-319: Cleartext Transmission of Sensitive Information
CWE-665: Improper Initialization
CWE-285: Improper Access Control (Authorization)
CWE-642: External Control of Critical State Data
CWE-352: Cross Site Request Forgery (CSRF)
CWE-94: Failure to Control Generation of Code (aka 'Code Injection‘)
CWE-732: Insecure Permission Assignment for Critical Resource
CWE-602: Client-Side Enforcement of Server-Side Security
CWE-250: Execution with Unnecessary Privileges


Security Verified

Dynamic Testing Vectors


Internal Security Vector Security Verified

06-01 Application Path Disclosure
06-02 Browse HTTP from HTTPS
06-03 Brute Force Login
06-04 Auth Over HTTP
Internal Security 06-05 Credit Card Disclosure
06-06 Document Caching
06-07 Find Pages
06-08 Flush Permissions
06-09 Form Caching
06-10 HTTP Parameter Pollution
06-11 Login Redirect
06-12 Non-masked Password
06-13 Plateform Path Disclosure
06-14 Privacy Notification
06-15 Private IP Disclosure
06-16 Remote File Inclusion
06-17 Social Insurance Disclosure
06-18 Social Security Disclosure
06-19 Username & Password in HTTP request


Entry Validation Vector Security Verified

07-01 Application Exception
07-02 Application Path Disclosure
07-03 Blind SQL Injection
07-04 Buffer Overflow
Entry Validation 07-05 Cross-Site Scripting
07-06 Format String
07-07 Frame Injection
07-08 GET for POST
07-09 HTTP Parameter Pollution
07-10 HTTP Response Splitting
07-11 Integer Overflow
07-12 LDAP Exception
07-13 LDAP Injection
07-14 Open Redirect
07-15 Parameter Addition
07-16 PHP & Perl Code Injection
07-17 Remote File Inclusion
07-18 SQL Disclosure
07-19 SQL Error Message
07-20 SSI Injection
07-21 Unix Command Injection
07-22 Unix Relative Path
07-23 Windows Command Injection
07-24 Windows Relative Path


Session Management Vector Security Verified

09-01 Acquire Session ID
09-02 Authentication Bypass
09-03 Authorization Boundary
Session Mgmt 09-04 Clickjacking
09-05 Cross Site Request Forgery
09-06 Ineffective Session Termination
09-07 Javascript Hijacking
09-08 Lockout
09-09 Password Change
09-10 Privilege Escalation
09-11 Register Password
09-12 Run Traversal Only
09-13 Session Fixation
09-14 Session Hijacking
09-15 Session ID Identification
09-16 Session ID in URL
09-17 Session ID Length
09-18 Session ID Randomness
09-19 Weak Password


OWASP TOP 10 Compliance Security Verified

10-A01-01 Blind SQL Injection
10-A01-02 LDAP Exception
10-A01-03 LDAP Injection
10-A01-04 SQL Error Message
10-A01-05 SQL Disclosure
10-A01-06 Unix Command Injection
10-A01-07 Windows Command Injection
10-A02-01 Cross-Site Scripting
10-A03-01 Authentication Bypass
10-A03-02 Cookie Vulnerabilities
10-A03-03 Login Redirect

OWASP Top 10
10-A03-04 Non-masked Password
10-A03-05 Non-SSL Password
10-A03-06 Password Autocomplete
10-A03-07 Session ID Randomness
10-A04-01 Frame Injection
10-A04-02 Remote File Inclusion
10-A06-01 Cross Site Request Forgery
10-A06-01 Application Exception
10-A06-02 Application Path Disclosure
10-A06-03 HTTP Methods
10-A06-04 Directory Browsing
10-A06-05 Plateform Path Disclosure
10-A06-06 Web Server Vulnerabilities
10-A08-01 File & Directory Discovery
10-A08-02 Unix Relative Path
10-A08-03 Windows Relative Path
10-A09-01 Browse HTTP from HTTPS
10-A09-02 Non-SSL Form
10-A09-03 SSL Checks
10-A10-01 Cross-Frame Scripting
10-A10-02 HTTP Response Splitting
10-A10-03 Open Redirect


Security Verified

Thanks

ridha.chebbi@icodesecurity.com
Mobile : +216 26 88 10 11


Introduction to Application Security Testing

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a Introduction to Application Security Testing

Similar a Introduction to Application Security Testing (20)

Último

Último (20)

Introduction to Application Security Testing