AppSensor is an OWASP project that defines a conceptual framework, methodology, guidance and reference implementation to design and deploy malicious behavior detection and automated responses directly within software applications. There are many security protections available to applications today. AppSensor builds on these by providing a mechanism that allows architects and developers to build into their applications a way to detect events and attacks, then automatically respond to them. Not only can this stop and/or reduce the impact of an attack, it gives you incredibly valuable visibility and security intelligence about the operational state of your applications.

1. AppSensor
~real-time event detection and response

2. me
• appsensor dev lead (OWASP)
• dev / security
!
• twitter: @_jtmelton
• email: jtmelton at gmail.com
• github: jtmelton

3. agenda
• history (recent)
• motivations / problems
• solution / tech
• future / wrap-up

4. thesis:
!
modern secure
applications protect
themselves against
attackers

6. history

7. ~5 yrs ago dev
• mostly web apps
[RoR, PHP, .NET, Java)
• ajax (jquery) use
growing
• mobile just getting
started
• deployment to VMs
• hadoop picking up
• BI tools
• AWS starting
• cloud hype cycle
(NIST defines)

8. ~now dev
• JS everywhere
• functional / rx programming
• cloud everything
• ci/cd
• nosql / CAP light
• containers
• big data
• stream processing
• config management
• iot
• beacons [usage, ads,
errors, performance]
• actors/csp
• microservices
• cqrs / event sourcing
• mobile

9. ~now dev
• JS everywhere
• functional / rx programming
• cloud everything
• ci/cd
• nosql / CAP light
• containers
• big data
• stream processing
• config management
• iot
• beacons [usage, ads,
errors, performance]
• actors/csp
• microservices
• cqrs / event sourcing
• mobile
1 .. * of [scale, speed, cloud, lack of environmental access]

10. (brief) history

12. - LinkedIn, March 2015
“the Kafka ecosystem at LinkedIn is sent over
800 billion* messages per day..
At the busiest times of day, we are receiving
over 13 million messages per second.”
* Update (Sept 2015) : 1.1 Trillion messages per day

13. last ~5 yrs security
• 3rd party libs (dep-check)
• bug bounties
• sast / dast evolve (ZAP)
• iast / rasp
• http security headers
• automatic encoding (JXT)
• *-monkey -NetflixOSS
• bdd-security/gauntlt
• ci/cd plugins
• 2fa
• osquery
1 .. * of [scale, speed, cloud, lack of environmental access]

14. dev vs. security
• dev is exploiting fundamental
architectural and deployment changes to
add business value
!
• security is iterating on existing solutions -
and - trying to close gaps (known
problems)

15. security ~5 yrs ago
vs today
?

16. motivations

17. “security”
• confidentiality and
integrity important
• availability often
ignored by security
(informs the whole
industry- eg. tooling)
• if availability important,
runtime important

18. http://i.imgur.com/TdtgvtW.png

19. your environment
• how many concurrent users do you have right
now?
• what are your users doing in the app?

20. https://github.com/aphyr/jepsen-talks/blob/master/2015/goto/goto.pdf

21. Security defects are a
subset of all defects

24. catching defects
• what do dev/qa do for functionality?
• test [unit, integration, system, manual, tools]
• what do attackers do for security?
• test [automated tools, manual]

26. observations
• attackers do bad things
• bad things often easily recognizable (to you)
• attacker success often* requires > 1 attempt
* If not, you lose

27. robbing a bank
Physical Controls
Electronic
Monitoring
Human Monitoring
Instant Detection
and Response
Controlled Access
Multi Factor Auth
Transaction
Verification

28. would you bank here?
Alternate
Admin
Access
Partial
External
Controls
Ineffective
Monitoring
No Real Time
Analysis
Unnecessary
Partner Trust No Response
Capability
Single
Factor Auth
Limited Security
Training

29. http://worth1000.s3.amazonaws.com/submissions/414000/414200_9830_1024x2000.jpg

30. defender’s dilemma
• attacker needs ONE successful attack
• defender must defend ALL attacks
!
• NOTE: you are defenders

31. on people
• 18.2 million devs
• 200K security (all, not appsec only)
• ~ 1.1 sec : 100 dev
!
• 1.75 sec : 100 dev (bsimm)

32. security modern dev
• polyglot static and
dynamic languages
• microservices / soa
• json, thrift, protobuf,
etc. endpoints
• WebAssembly ???
• a single mature,
static language
• monolith
• http (really html)
endpoints

33. in summary …
• “traditional” security, dev, ops doesn’t know what’s going
on in the app at runtime
• security defects exist
• attackers don’t magically know what’s vulnerable
• existing “monitoring” usually terrible
• there will never be enough “security” people
• “traditional” security tooling doesn’t fit modern dev
• actual defense is _really_ hard

34. the pitch
!
(a humble proposal)

35. having to deal with [scale,
speed, cloud, lack of
environmental access]..
!
..this as of now incomplete
transition..
!
..represents an enormous
opportunity for security

36. the pitch
• in addition to a secure SDLC … (ie. > 1 request/attack)
!
• figure out what’s happening at runtime
• make intrusion detection primitives available in app
• exploit automated response > manual response
• stop attacker before success
• get self-protecting applications and valuable intel
X success
AppSensor

37. X

38. prior work
• Network IDS (Denning, * others, NIST
SP800-94)
• Intrusion prevention
• Fraud detection
• Rules engines, Risk analysis/reduction, HIDS

39. terminology
• event - suspicious
• attack - malicious (1 .. * events)
• response - take action (1 .. 1 attack)
• detection point - activity category (e.g. cookie
modification)

40. the tech

41. Architecture

42. • picture 1

44. Correlation

48. Adding
Detection Points

49. adding detection points
• manually
• reverse proxy
• owasp aside

50. manual
POST /account/transfer HTTP/1.1
!
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (Win…)
Accept: text/html,application/xhtml+xml
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://127.0.0.1/account.php
Cookie: PHPSESSID=l9…lgt5
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 30
from_acct=xxx1234&to_acct=xxx9876&amt=20.00

51. manual
POST /account/transfer HTTP/1.1
!
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (Win…)
Accept: text/html,application/xhtml+xml
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://127.0.0.1/account.php
Cookie: PHPSESSID=l9…lgt5
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 30
from_acct=xxx1234&to_acct=xxx9876&amt=20.00

52. manual
POST /account/transfer HTTP/1.1
!
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (Win…)
Accept: text/html,application/xhtml+xml
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://127.0.0.1/account.php
Cookie: PHPSESSID=l9…lgt5
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 30
from_acct=xxx1234&to_acct=xxx9876&amt=20.00

53. manual
@POST
public Response transfer(
String from,
String to,
String amount) {
!
transfer(from, to, amount);
!
return Response.ok();
}

54. manual
@POST
public Response transfer(
String from,
String to,
String amount) {
!
if ( currentUser.owns(from) ) {
transfer(from, to, amount);
}
!
return Response.ok();
}

55. manual
@POST
public Response transfer(
String from,
String to,
String amount) {
!
if ( currentUser.owns(from) ) {
transfer(from, to, amount);
} else {!
showErrorPage();! // normal error handling!
}!
!
return Response.ok();
}

56. manual
@POST
public Response transfer(
String from,
String to,
String amount) {
!
if ( currentUser.owns(from) ) {
transfer(from, to, amount);
} else {!
! appsensor.addEvent( new Event(currentUser, "ACE2") );!
showErrorPage(); // normal error handling!
}!
!
return Response.ok();
}

57. appsensor-reverse-proxy

58. appsensor-reverse-proxy
• written in go
• blocks requests
• canned detection points (toggle-able)
• easily extendable
• https://github.com/jtmelton/appsensor-
reverse-proxy

59. OWASP ASIDE
• UNCC SIS project
• secure programming IDE plugin
• educational component
• https://www.owasp.org/index.php/
OWASP_ASIDE_Project

60. OWASP ASIDE
• eclipse IDE
• reminder icon or highlight
• drop down list of applicable sensors
• auto-insertion of ASIDE sensor APIs and code
refactoring

61. OWASP ASIDE

62. OWASP ASIDE
Based
on
ESAPI
code
(length
checked),
ASIDE
infers
that
this
may
be
a
point
to
insert
an
app
sensor;
whether
a
sensor
is
placed
relies
on
developer’s
decision.

63. OWASP ASIDE
It
not
only
captures
the
context
informaFon
(e.g.
the
sensor
event
is
from
username
field),
but
also
records
that
the
sensor
event
is
due
to
an
exceedingly
lengthy
input.

64. Detec%on(Point(Type( Detec%on(Points(Covered(
Authen'ca'onExcep'on. AE4:.Unexpected.Quan'ty.of.Characters.in.Username.
AE5:.Unexpected.Quan'ty.of.Characters.in.Password.
AE6:.Unexpected.Type.of.Character.in.Username.
AE7:.Unexpected.Type.of.Character.in.Password.
InputExcep'on. IE1:.Cross.Site.Scrip'ng.AEempt.
EncodingExcep'on. EE1:.Double.Encoded.Character.
EE2:.Unexpected.Encoding.Used.
CommandInjec'onExcep
'on.
CIE1:.Blacklist.Inspec'on.for.Common.SQL.Injec'on.Values.
Detec%on(Points(Picked( Corresponding(ASIDE(APIs(
AE4:%Unexpected%Quan1ty%
of%Characters%in%Username%
AE5:%Unexpected%Quan1ty%
of%Characters%in%Password%
Java.lang.String%
ASIDE.Quan%tyExcep%onSensor(Java.lang.String%parameter)%
AE6:%Unexpected%Type%of%
Character%in%Username%
AE7:%Unexpected%Type%of%
Character%in%Password%
Java.lang.String%
ASIDE.TypeExcep%onSensor(Java.lang.String%parameter)%
IE1:%Cross%Site%Scrip1ng%
AKempt%
Java.lang.String%
ASIDE.XSSSensor(Java.lang.String%parameter)%
EE1:%Double%Encoded%
Character%
EE2:%Unexpected%Encoding%
Used%
Java.lang.String%
ASIDE.EncodingExcep%onSensor(Java.lang.String%parameter)%
CIE1:%Blacklist%Inspec1on%for%
Common%SQL%Injec1on%
Values%
Java.lang.String%
ASIDE.SQLInjec%onSensor(Java.lang.String%parameter)%

65. Viewing Data

66. owasp SoC sprint
• Sumanth Damarla
• appsensor -> ELK stack
• appsensor -> influxdb -> grafana

67. owasp SoC sprint

68. owasp SoC sprint

69. machine learning
• very simple analysis
• generated demo dataset for 1-week
• build base model
• look for “anomalies”

71. appsensor-ui
• spring boot + reactjs
• simple dashboards / trends

77. future plans

78. future
• reverse proxy
• appsensor-ui
• framework integration for detection points
(spring security exists, add others)
• analysis engines - 2 grad students (David
Scrobonia, Kelvin Brown Bomah) doing
projects here (rules and ML)
• ???

79. you
• help wanted!
• plenty of places to contribute and improve
• friendly, helpful community
• https://github.com/jtmelton/appsensor/issues
• https://www.owasp.org/index.php/
OWASP_AppSensor_Project#tab=Road_Map_
and_Getting_Involved

80. wrap-up

81. related projects
• repsheet
• ensnare
• fido
• riemann
• elastalert

82. pick a tool (or 2) …
!
but use the idea

83. contributors
• https://www.owasp.org/index.php/
OWASP_AppSensor_Project#tab=Acknowledgements

84. links
• https://www.owasp.org/index.php/
OWASP_AppSensor_Project
• http://appsensor.org/
• https://github.com/jtmelton/appsensor

85. ?