TeamStation AI System Report LATAM IT Salaries 2024
Techniques for scaling application with security and visibility in cloud
1. Techniques for Scaling Application
with Security and Visibility
in Cloud
Akshay Mathur
@akshaymathu of @appcito
2. Let’s Know Each Other
• Do you Manage applications?
• Hosting providers?
• Priorities?
• Tools?
• Why are you attending?
• What are your Goal?
• Happy Users, Happy DevOps, Happy Servers
2@akshaymathu
3. Akshay Mathur
• 15+ years in IT industry
• Currently Product Manager at Appcito
• Mostly worked with Startups
• From Conceptualization to Stabilization
• At different functions i.e. development, testing, release, marketing, devops
• With multiple technologies
• Founding Team Member of
• ShopSocially (Enabling “social” for retailers)
• AirTight Neworks (Global leader of WIPS)
@akshaymathu 3
4. Ground Rules
• Tweet now: #TechNext @akshaymathu @appcito
• Disturb Everyone later
• Not by phone rings
• Not by local talks
• By more information and questions
@akshaymathu 4
6. Traditional Application
• Monolithic components
• All application layers in a box
• Complex objects
• Box specific sessions
• Designed for vertical scale
• Self managed deployment
@akshaymathu 6
10. Modern Application
• Light weight services
• Application layers designed for
network communication
• Cloud deployment
• Designed for horizontal scale
@akshaymathu 10
13. Growth Phase 1: Load Balancing
• Replicate the box
• Have a load balancer
@akshaymathu 13
14. Questions before Growing Further
• About Insights:
• Are all server instances healthy?
• When should I add more servers?
• What is the traffic volume and its
pattern?
• What areas of application are used
most?
• What are problematic areas?
• Who access my application?
• What devices, browsers, apps are in
use?
• About Optimization:
• How can I serve more traffic using
existing servers?
• Does all the serves must be of same
type, running same code?
• Can the content be compressed,
cached?
• What to do for optimizing content for
various devices?
• Do I really need to redirect traffic to a
different URLs for specific servings?
• Does managing so many URLs for same
functionality makes sense?
• Can someone take care of SSL
termination?
@akshaymathu 14
15. Growth Phase 2: Insights
• Google Analytics, Statcounter etc. only provide
information after page load
• Information about programmatic access is missing
• Access logs provide true information about traffic
• Logs are typically in each box rather than a central place
• Difficult to read; log parsers also provide minimal
information
• Need to push logs to some analytics engine and
configure analytics engine for getting meaningful
information out
@akshaymathu 15
16. Growth Phase 3: Content Optimization
• Compressing the response
• Optimizing images
• In-lining the external resources
• JS
• CSS
• Images as base64
• Caching (as needed)
• Prefetching (if possible)
• Google’s PageSpeed does it well for HTML pages
@akshaymathu 16
17. Growth Phase 4: Offloading
• SSL Handshake
• Encryption and Decryption
• Connection handling
• Content optimization
• Anything that can be done asynchronously
e.g. sending email, tweets etc.
• Point solutions are available for each of
these
@akshaymathu 17
App Servers
Apache + Pylons
Message Queue
RabbitMQ
Background
Worker Nodes
Celery
SSL Terminator
Content Optimizer
18. Growth Phase 5: Content Switching
• Serve different content from different servers (reverse proxy)
• Static files (JS, CSS, Images) may be served from a web server; App server is not
needed
• High frequency requests may be served from different server
• Different app servers may be used for the use case they are optimized for
• Have different set of servers for different geographies
• Dedicate a few servers for specific customer
• Dedicate servers for specific functions e.g. authentication, API serving etc.
• HA Proxy is most popular tool here
• NginX is also used as reverse proxy
@akshaymathu 18
19. Web Servers
NginX
App Servers
Mongral + Brubeck
App Servers
Apache + Pylons
Web Servers
Apache + Wordpress
NoSql Datastore
Redis
NoSql Datastore
MongoDB
Sql Datastore
MySql
Corporate
website
Main dynamic
content
High frequency
requests
High speed storage Main Storage
Content Switching
Reverse Proxy
20. Growth Phase 6: Denying BOT Traffic
• Traffic from bad BOTs is about 30%
• Amounts to 30% wastage of server
resources
• Various fingerprinting techniques
are there for identifying the BOTs
• IP reputation
• UA analysis
• Pattern analysis
• JS insertion
• Advance algorithms
@akshaymathu 20
21. Growth Phase 7: Preventing Data Theft
• Typical ways are:
• SQL/object injection
• Cross Site Scripting (XSS)
• File include
• Malware inclusion
• Exploiting vulnerabilities of coding, framework,
language, platform
• Scan the deployment regularly
• Fix any vulnerability by applying patches
• Use Web Application Firewall (WAF)
@akshaymathu 21
22. Growth Phase 8: Preventing from DDoS Attack
• Volumetric attack
• Many clients make connections with
server
• Clients send huge traffic to the server
• Traffic is typically bogus
• Prevention
• Rapidly increase scale to consume
connections/traffic
• Rate limit connections/requests
• Delay/Deny bogus traffic
• Blacklist BAD clients
• Protocol exploits
• Attacker crafts traffic knowing the
timeouts and limits of protocol
• Slow moving bogus traffic hogs
resources of server
• Prevention
• Setup policy to apply aggressive limits
and timeouts in case of heavy load
• Terminate connection when unusual
behavior is observed
• Blacklist BAD client
@akshaymathu 22
23. Growth Phase 7: Continuous Delivery
• Upgrade the system without disturbing availability
• Why Continuous Delivery?
@akshaymathu 23
24. Continuous Delivery
• Considerations:
• Zero down time
• Even a little downtime means a lot for
high volume applications
• Seamless re-orientation of live traffic
from old to new deployment
• User experience has to be smooth
• Easy roll back
• Minimize the impact in case something
goes wrong
• Technique: Blue Green deployments
• Deploy old and new version in parallel
and switch the traffic
• Switch using DNS
• Switch using fixed NATed IP addresses
• Switch using external tools like load
balancer or reverse proxy
26. App & Traffic
Metrics
What is Needed Overall?
26
Availability Performance Security DevOps
Advanced Load
Balancing
Content Switching
Application Fluency
Elastic & Self-Scaling
Continuous
Deployment
Request Mirroring
Request Replay
Programmable
Policies
Per Application
Control
Front-End
Optimization
Mobile and Web
Client App
optimization
Caching &
compression
Predictive API
caching
Application & Server
offloading
Application Firewall
Elastic SSL
Anomaly Detection
DDoS Prevention
BOT Protection
Trends &
Correlations Anomalies
Policy
Recommendations
Analytics & Insights
27. CDN
Custom Scripts, Rules, Alert Management Aggregation across instances
Application Front-End Architecture
• Spaghetti of point solutions
• Multiple points of failure, redundancy difficult to setup
• Not elastic and cloud native
@akshaymathu 27
28. CDN
Application Front-End Architecture with CAFE
• All services for application under one consolidated product
• Easy Activation of capabilities closer to application
• Application policy is coordinated across services and policy enforced
@akshaymathu 28
Availability Security Performance Continuous
Deployment
Appcito Cloud Application Front-End (CAFE)
32. Deployment with CAFE
Customer’s Cloud
Customer’s
End Users
app
server
app
server
Load
Balancer
app
server
Appcito Cloud
CAFE Barista
Management, Control, Analytics
DNS
CAFE
PEP
Network Subnet
Availability Zone
33. CAFE Configuration Model
• Think Out of the box (literally)
• Think in terms of
• Applications
• Traffic flow
• Request patterns
• Forget about
• Box provisioning
• Box configuration
• Networking flow
• L2/L3 access control
@akshaymathu 33
34. Production A (Blue)
Production B (Green)
Launch
Upgrade
Traffic Splitting
80% 20%
Appcito CAFE
80%
20%
CAFE Blue/Green Technique
• Steer traffic NOT switch
• Test with production traffic
• Move with confidence
• Compare performance and take informed
decisions