2. Performance Engineering Methodology
Performance Characterization
Production System Analysis
Detailed Architecture Review
Detailed code review analysis
Performance Tuning
Database architecture, Query Analysis and entity relationships
3. Performance Characterization
Performance goals and objectives Testing for performance
Metrics and performance criteria Identify the most common use cases and
load levels
Expected peak load
Development of performance test suites
Throughput
Test Tool identification
Response time
Test data generation
No of users
Define and capture work load Runs tests and analyze data
Workload objectives
Base Line characterization
Analyze statistics
Repeat baseline with optimized application
Test Tool identification
Grinder, webload, loadrunner, funkload,
rational tester etc
4. Production System Analysis
Understand Deployment requirements Technology stack
Fault tolerance and Redundancy Key components along the response path
Understand transactional requirements Web Server, Middleware, Database
Caching and Clustering Server diagnostics
Load Balancing and state management Memory, gc analysis
Deployment configuration Using Java profilers
Caching and clustering issues? Run time statistics
Configurable parameters like no. of Latencies
thread, socket buffer sizes Concurrency and threading issues
Statistics collection (Memory analysis ,
garbage collection etc) Database Service times
Estimate the Capacity SQL queries
Size the applications Connection pools
Database cache
5. Architecture Review
Architecture review Requirements
Major modules, interfaces No of users, response time, throughput
Inter and intra product message exchanges Capacity and sizing requirements
Transactional analysis Understand transactional requirements
Data path and its size Load characterization
Architectural design patterns and their Run tests and analyze data
applicability
Top down approach
Failure (severance) scenarios
• OS, VM, AppServer, Application, Network, DB
Recovery scenarios(Recoverability)
Choose profiling tools
Data integrity and fault tolerance
Evaluation of deployment parameters
Object instances and scalability
Clustering
Memory requirements
Load balancing etc
Optimum performance through load
characterization
6. Detailed Code Review
Analysis Understand Code
Standard coding practices Different modules
Design patterns Layers and Interfaces
Class design and Code reuse Dependency and boundary tests
Thread safety and deadlocks UI interaction
Unit test suites Caching and Clustering
Continuous integration Session management and state
Peer code review process Clustering and session state
Modularity
Intra and Inter product message exchanges
7. Performance Tuning
Understand the problem Performance improvement
Generic tuning Repeatable test load, with baseline and
target
Specific operations slow
Generating load
Slow on high volume of load
Run tests and analyze data, interpret
Tunable components data
Processors, memory, I/O, Network BW Repeat performance tests
OS parameters, VM Parameters
Tune OS and Network
Schema Design, Table indexes
Buffer sizes, paging, no of threads, disk
No of connections, DB cache reads and writes
Performance Criteria and Test Tune JVM
environment GC times, Memory, I/O
Repeatable test load, with baseline and
target Tune Database
Tools for generating load Queries, indexes, partitioning
Run tests and analyze data Tune Application
Sessions, application cache, data paths
8. Database Analysis
Understand the Entity model Scaling the Db Reads
Understand domain model, relationships, Replication with slave servers
attributes
Slave
Replication requirements
BI requirements Scaling the DB writes
Sharading: Partition data multiple
Analysis servers
Optimum normalization of the schema Managing queries across shrads
SQL query analysis, joins (cost analysis)
Data redundancy and backups
Caching
Slave DB for backup
Index sizes and its effect on writes and reads
Sharding, Partitioning and application design
Statistics collection and analysis
10. SocialTwist: 59 mn queries & 6 bn buttons served
Worlds leading social media sharing widget developed on the Java platform and deployed on EC2
for extremely high scalability and 24x7x365 availability. Here are some key characteristics
2.79 billion impressions served
16
Auto scale up or down
Configures load balancer automatically
15
New instance to share load after configuration
Instant deployment 12
11
Monitor health, average load, requests etc 13 S3
Balance
Uptime for more than a year Loader
24% reads and 76% writes on master 14
EC2
3 slaves and 100% reads
Remote Shell
Cacti
System
Admin
11. MyPicks Beijing: Facebook Gaming Application
Sun Microsystems sought Imaginea’s expertise to develop and test the Mypicks application
that had to be designed to scale for more than 3mn users
User DB is sharded (with hot standby)
Look up table for sharding S3
Shard Lookup DB
10MB cache size for mil users S
M
Sharding logic in Node chooser Result Processor
Friends DB replicated instead of MapReduce
Attaching Web Servers to shard reduces no of User Shard 1
connections to DB
M S
Base AMI types (LB, Web, Database) S
LB WEB
Join Cluster, publish stats, join monitor system M
Instances on demand using AMI’s
User Shard 2 S
Instance join the cluster
M S
Friends DB
WEB
12. Performance Improvement for a contract service
management platform
Short introduction: Imaginea executed a performance engg optimization project for a
contract mgmt platform (Tech stack). The engagement involved
Performance Improvement of 30% Core Services Containers Applications
Redundant DB calls Resource
App1
Web
Optimize and batch SQL queries DB
Tuning
Caching the Metadata Naming
App2
Database EJB
Performance Test Suite
App3
Tested up to 1500 users Transaction
JMS
Web Load Test tool
LB
Designed framework for script reuse
and schedule load JVM
Monitoring hit ratio, response times, Threading
Apache/
JVM Memory GC IIS
heap, gc times, cpu, disk, network, Models I/O
plugin
database queries
Operating Systems/ Hardware
Cluster
CPU MEMORY RAM NETWORK
13. Performance Engineering for a SaaS application
Imaginea delivered a 3X improvement in response time and throughput for a leading
enterprise SaaS application vendor.
The vendor expected load on the application to increase significantly. App is based on
Java/J2EE middleware technology stack.
Imaginea executed an engagement that involved a systematic process of analysis,
quantification, recommendations & remedy as described below
Understand current implementation - Functionality, architecture, modules & frameworks
Identify representative test workloads and setup test environments
Characterization & analysis exercise
Analyze scalability – User scaling & hardware scaling
Scalability metrics (concurrent users, throughput, response)
Improving response times & throughput
Identify code & design bottlenecks suggest fix(s)
Scalability reports for the Application (with varying user loads)