Running DDoS simulations on your own. Why would you want to do such a thing? Preparing for destruction Running the tests – best practices What happens after the fact Moving forward towards more robust RTC

1. TADSummit 2022 -
How to bring your own
RTC platform down
Running DDoS simulations on your own
Sandro Gauci
2022-11-08

2. Introduction

3. What is this about?
provide a quick walk-through of performing DDoS attacks
speci c to RTC services
not complete but should help you get started
Distributed Denial of Service: distributed attacks that block
legitimate usage of your service

4. What makes me quali ed to
talk about this?
author of SIPVicious OSS - security testing toolset for SIP
in Infosec / Cyber security since > 20 years
last 14 years doing offensive security and leading Enable Security

5. What we do
we focus on RTC security testing
various cyber-security services
we do DDoS simulation for our clients

6. Why would you do such
a thing?

7. Pinky and the Brain, pondering why

8. obviously, to have some dangerous fun!

9. Serious answers only
to nd out how your critical services are going to react to a DDoS
attack
What if we got attacked?
the purpose is to create a system that can reasonably withstand
most DDoS attacks

10. What if we have some form
of protection mechanism?
most organisations operating in RTC have security solutions
or use technology that is known to be robust
often a hybrid approach, some things are better protected than
others

11. security protection is often implemented but rarely properly tested
until you test, you should not make any assumptions
because …

12. what often happens is that service providers nd out that they
don’t work during an actual attack
this is not ideal!

13. What if we’re trying to
evaluate an anti-DDoS
solution?
how well does it work?
comparing two or more solutions
which one works better given your situation?

14. Our experience
almost all DoS protection mechanisms fail at some point
maybe network tra c bursts are not handled well
maybe slow attacks lead to bypass but still trigger DoS
maybe the security solution looks for particular patterns which can
be bypassed

15. every organisation has limitations
bandwidth
system resources
application bugs / e ciency
the point is to understand if your current protection-level is
su cient against real life attacks

16. Part of Threat modelling
what are your most critical services?
how are they exposed to DDoS attacks?
what do we need to do to protect against these attacks?

17. once you have understood Why, you should move
on to What and How

18. Preparing for
destruction!

19. Decide on what your want to
attack and how
bandwidth saturation
generic protocol attacks
speci c app attacks

20. Bandwidth or generic
protocol attacks
often require generic or simple tools
e.g. targeting APIs or SIP servers ( ooding with GET or REGISTER
messages)

21. Attacking speci c
application functionality
needs some homework
Initial tests to determine which parts should be attacked
Look for:
errors, especially if generated during fuzzing
slow responses
increase in memory consumption (even a slight increase)
potential exhaustion of resources

22. Attack tools
for bandwidth saturation and generic protocol attacks, standard or
simple tools are enough
attack tools need to be more e cient than the target application
normally, they generate tra c - replaying messages

23. func flood() {
payload := "OPTIONS sip:demo.sipvicious.pro SIP/2.0rn" +
"Content-Length: 0rnrn"
b := make([]byte, 1024)
for {
c, err := net.Dial("udp", "demo.sipvicious.pro:5060")
if err != nil { log.Fatal(err) }
go func() { // Read loop
for { c.Read(b) }
}()
go func() { // Write loop
for { c.Write([]byte(payload)) }
}()
}
}
func main() {
flood()
}

24. func flood() {
payload := "OPTIONS sip:demo.sipvicious.pro SIP/2.0rn" +
"Content-Length: 0rnrn"
b := make([]byte, 1024)
for {
c, err := net.Dial("udp", "demo.sipvicious.pro:5060")
if err != nil { log.Fatal(err) }
go func() { // Read loop
for { c.Read(b) }
}()
go func() { // Write loop
for { c.Write([]byte(payload)) }
}()
}
}
func main() {
for i:=0;i<100;i++ { go flood() }
select{}
}

25. Useful features for such
tools
rate limiting
concurrency (e.g. connection count)

26. You need control tools
distribute the attack tools (e.g. use Terraform)
ability to start the attack
and stop the attack

27. #!/bin/bash
IPS=$(linode-cli linodes list --format ipv4 --json | jq -r '[.[][][]] | join(" ")')
for ip in $IPS;
do
ssh root@${ip} sipvicious sip dos flood udp://demo.sipvicious.pro:5060 &
done

28. #!/bin/bash
IPS=$(linode-cli linodes list --format ipv4 --json | jq -r '[.[][][]] | join(" ")')
for ip in $IPS;
do
ssh root@${ip} killall sipvicious &
done

29. The killswitch
emphasis on stopping the attack
if attacker machine is no longer reachable but still attacking
may lead to a real security incident!

30. #!/bin/bash
IDS=$(linode-cli linodes list --format id --json | jq -r '[.[][]] | join(" ")')
for id in $IDS;
do
linode-cli linodes shutdown ${id}
done

31. You need attack nodes
these are systems from where to launch your distributed attacks

32. Options
VPS: easy to get started and distributing attacks at low price
VMs: very useful for internal tests in lab environment
Bare metal servers: great for attacks that consume
bandwidth/require decent resources but more expensive

33. Caution when using third-
parties
you will want to make sure that the activity is allowed
that you are not affecting other customers
especially watch out for bandwidth saturation

34. Monitor your bandwidth
usage
even if not testing for bandwidth saturation
might give a false positives (incorrect results)
see our blog post: Why volumetric DDoS cripples VoIP providers
and what we see during pentesting

35. Monitor your bandwidth
usage
at the target
at the attack nodes

36. Monitor your application
and system resources
having the ability to switch pro ling on is very useful
ability to debug

37. Don’t forget the people!
the engineers who need to be involved/monitor systems: they
need to be booked
testing on live systems usually means doing tests during off-peak
hours

38. Finally
the test environment: you need a place to test
make sure it is as close to production as possible
sometimes, it is production

39. One more thing

40. i love it when a plan comes together

41. gure out what tests to do
start with the simpler tests rst
you will encounter problems that should be solved before moving
to more complex tests

42. Fun time!

43. a beverage

44. Start monitoring
bandwidth
application
system resources
dependencies (e.g. databases)

45. External monitoring
use a pinger
simulates legitimate tra c periodically (e.g. every 1s)
will indicate major problems but will miss more subtle issues

46. Demo time
we show our Attack Platform that we use to do such tests
Target server - a Kamailio/Asterisk server
Have 3 parts in this demo
Target server
Monitoring system sending SIP ping
Attack platform client (controlling the attacker nodes)

47. 0:00 / 0:53

48. Next
when things break, you should get noti ed through monitoring
stop and assess: real work starts here
some of it might be done during the exercise, some later
understanding root cause

49. Best practices
1. Automate as much as possible (capturing of monitoring results,
bandwidth stats)
2. Have a real-time communication channel with the engineers
(e.g. Google Meet)
3. Work with the engineers not against them - collaborate don’t
compete
4. Test your tests ahead of time! don’t do your own QA during the test
5. Document every step done and the results (can be semi-
automated); otherwise you might forget what actually happened
. Set a xed scope + timeframe; know your limits (we stick to 2hrs,
very tiring)

50. What happens after the
fact

51. accept the risk?

52. really, it depends on your ndings
generally: root cause analysis
might have started already during the actual exercise
might need further exploration, dedicated time and effort

53. Once the root cause for
each nding is determined
solutions or mitigation techniques need to be discussed
jumping to solutions without proper assessment undermines the
whole effort
solutions need to be:
practical and make economic sense
not introduce new problems that prevent legitimate usage
actually address the problems that were identi ed
implemented

54. Examples of solutions
outdated logging library caused locks - updating that library (but
dependency hell)
changes in application con guration
rate limiting solutions
application code changes (result of pro ling and debugging)

55. Caveat
real solutions rarely consist of buying more
resources

56. Finally - documentation &
retest
update your documentation to include details about the solutions
test the solution again
does it work? where does it fail?
feedback loop

57. Moving forward
towards a more robust
RTC

58. by doing your own DDoS simulations, you learn about your system
no longer should have to guess if you will handle a real attack
as an RTC provider, you have a duty to keep your RTC <real(-time)=

59. One more thing
Security is often a cat and mouse game
No security solution is perfect, incidence response is critical
How will we handle the next time we get DDoSed?

60. This is the end

61. Thanks!
Alan Quayle
My colleague, Alfred Farrugia for the demos
Our clients for allowing us to cause trouble on their applications
and networks

62. Get in touch & References
Email:
Enable Security:
sandro@enablesecurity.com
https://www.enablesecurity.com
Communication Breakdown blog @ rtcsec.com
Subscribe to the RTCSec Newsletter