Do you consider the performance impact when adding a new JavaScript file, a single AJAX call, or a new database query to your app? Negligible, you say? I disagree―and so should you. Andreas Grabner demonstrates the severe impact small changes can have on performance and scalability. Many small changes will have an even bigger impact so it is important to catch them early. If you are working with a delivery pipeline, make sure to look into performance, scalability, and architectural metrics such as the number of resources on your page, size of resources, number of requests hitting your web servers, database statements executed, and log messages created. Monitoring these allows you to add a new quality gate to your delivery pipeline and prevents major problems. Andi shares a handful of metrics to teach to your developers, testers, and operations folks, and explains why they are important to performance.

1. T11
Continuous Delivery
10/16/2014 11:15:00 AM
Checking Performance along
Your Build Pipeline
Presented by:
Andreas Grabner
Compuware
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Andreas Grabner
Compuware
Andreas Grabner has more than fifteen years of experience as an architect and
developer in Java, .NET, and Web 2.0 with a strong focus on application
performance. At Compuware APM (formerly dynaTrace) Andi is a technology
strategist, helping companies improve their applications’ performance across the
development lifecycle by embracing ideas of continuous delivery and DevOps.
He is a frequent speaker on software performance, testing, and architecture
topics at technology conferences including Velocity, STAREAST, STARWEST,
and JavaOne. Andi regularly publishes articles and blogs on
apmblog.compuware.com. Before dynaTrace, Andi was an engineer and product
manager for Segue Software and Borland on their Silk Product Line.

3. 1"
Checking)
Performance)Along)
your)Build)Pipeline)
)
@grabnerandi))
h9p://apmblog.compuware.com)
2"
h%p://starwest.techwell.com/sessions/
starwest62014/checking6performance6along6your6
build6pipeline"
Do"you"consider"the"performance"impact"when"adding"a"new"JavaScript"file,"a"single"AJAX"
call,"or"a"new"database"query"to"your"app?"Negligible,"you"say?"I"disagree―and"so"
should"you."Andreas"Grabner"demonstrates"the"severe"impact"small"changes"can"have"
on"performance"and"scalability."Many"small"changes"will"have"an"even"bigger"impact"so"
it"is"important"to"catch"them"early."If"you"are"working"with"a"delivery"pipeline,"make"
sure"to"look"into"performance,"scalability,"and"architectural"metrics"such"as"the"
number"of"resources"on"your"page,"size"of"resources,"number"of"requests"hiPng"your"
web"servers,"database"statements"executed,"and"log"messages"created."Monitoring"
these"allows"you"to"add"a"new"quality"gate"to"your"delivery"pipeline"and"prevents"
major"problems."Andi"shares"a"handful"of"metrics"to"teach"to"your"developers,"testers,"
and"operaSons"folks,"and"explains"why"they"are"important"to"performance."

4. 3"
4"

5. 5"
6"

6. 7"
And"just"very"recently"@"Euro"Song"Contest"
8"

7. 9"
What"you"should"check"out"…"
10"
Commit"Stage"
• Compile(
• Execute(Unit(Test(
• Code(Analysis(
• Build(installers(
Automated""
Acceptance""
TesSng"
Automated""
Capacity""
TesSng"
Manual"tesSng"
• Key(showcases(
• Exploratory(tes;ng(
"
"
Release"

8. 11"
12"
Who"is"doing"it?"How"many"successful"deployments"
can"they"do?"
300"Deployments"/"Year"
50660"Deployments"/"Day"
10+"Deployments"/"Day"
Every"11.6$seconds$

9. 13"
More"on"Amazons"Story"
75%$fewer"outages"since"2006"
90%$fewer"outage"minutes"
~0.001%$of"deployments"cause"a"problem"
Instantaneous$automaSc"rollback"
Deploying"every"11.6s"
14"

10. 15"
16"

11. 17"
TesSng"is"Important"–"and"gives"Confidence"
18"
But"are"we"ready"for"“The"Real”"world?"

12. 19"
Measure"Performance"during"the"game"
Ball"Possession:" "40":"60)
Fouls:""""""""""""""""""""""""0":"0"
Score:""""""""""""""""""""""0":"0"
Minute"1"6"5"
20"
Measure"Performance"during"the"game"
Minute"6"6"35"
Ball"Possession:" "80":"20)
Fouls:""""""""""""""""""""""""2":"12)
Score:""""""""""""""""""""""0":"0"

13. 21"
Deep"Dive"Analysis"
22"
OpSons"“To"Fix”"the"situaSon"

14. 23"
Not"always"a"happy"ending"!"
Minute"90"
Ball"Possession:" "80":"20)
Fouls:""""""""""""""""""""""""4":"25)
Score:""""""""""""""""""""""3":"0"
24"
FRUSTRATED"FANS!!"
24"

15. 25"
How)does"that"
relate)to)
SoGware?$
26"
From"Deploy"to"…"
Deploy Promotion/Event Problems Ops Playbook War Room
Timeline"

16. 27"
The"“War"Room”"–"back"then"
"
'Houston,"we"have"a"problem‘"
NASA"Mission"Control"Center,"Apollo"13,"1970"
28"
The"“War"Room”"–"NOW"
"
Facebook"–"December"2012"

17. 29"
4)SituaJons)on)
WHY)this)happens,)
HOW)to)avoid)it$
30"

18. 31"
“Blindly”)(Re)use)
ExisJng)
Components)
32"
Requirement:"We"need"a"report"

19. 33"
Using"Hibernate"results"in"4k+"SQL"Statements"to"
display"3"items!"
Hibernate"
Executes"4k+"
Statements"
Individual"
ExecuSon"VERY"
FAST"
But"Total"SUM"
takes"6s$
34"
Requirement:"We"need"a"fancy"UI"

20. 35"
Using"Telerik"Controls"Results"in"9s"for"Data6Binding"
of"UI"Controls"
#1:$Slow$Stored$Procedure$
Depending"on"Request"
execuSon"Sme"of"this"SP"
varies"between"1"and"7.5s"
#2:$240!$Similar$SQL$Statements$
Most"of"these"240!"Statements"
are"not"prepared"and"just"differ"in"
things"like"Column"Names"
36"
Metrics:"#"Total"SQLs"
#"SQLs"/"Web"Request"
#"Same"SQLs"/"Request"
Transferred"Rows"
Test:"With"realisSc"Data"
Dev:"“Learn”"Frameworks"

21. 37"
38"
ImplementaJon)
Flaws)

22. 39"
Business"Impact"requires"AcSon!"
40"
SoluSon:"Cache"to"the"RESCUE!!"

23. 41"
ImplementaSon"and"Rollout"
Implemented"
InMemory"Cache"
Worked"well"in"
Load"TesSng"
42"
Result:"Out"of"Memory"Crashes!!"
SSll"crashes"
Problem)fixed!)Fixed)Version)Deployed)

24. 43"
Metrics:"Heap"Size,""
#"Objects"Allocated,"
#"Objects"in"Cache"
Cache"Hit"RaSo"
Test:"With"realisSc"Data"
44"

25. 45"
#No)“Agile”)
Deployment)
46"
Ad)on)air)
Load"Spike"resulted"in"Unavailability"

26. 47"
AlternaSve:"“GoDaddy"goes"DevOps”"
1h"before""
SuperBowl"KickOff"
1h"auer""
Game"ended"
48"
Behind"the"Scenes"

27. 49"
Metrics:"Availability"
Page"Size,"#"Objects"
#"Hosts,"#"ConnecSons"
DevOps:"“Feature”"
Switches"
50"

28. 51"
#Push))
without)a)Plan)
52"
Mobile"Landing"Page"of"Super"Bowl"Ad"
434$Resources$in"total"on"that"page:"
230"JPEGs,"75"PNGs,"50"GIFs,"…"
Total"size"of"~"
20MB$

29. 53"
m.store.com"redirects"to"www.store.com"
ALL"CSS"and"JS"files"are"
redirected"to"the"www"domain"
This"is"a"lot"of"Sme"“wasted”"
especially"on"high"latency"mobile"
connecSons"
54"
Metrics:"Load"Time,""
#"Resources"(Images,"…),""
#"HTTP"3xx,"4xx,"5xx"
Dev:"Build"for"Mobile"
Test:"Test"on"Mobile"
Ops:"Monitor"Mobile"

30. 55"
56"
#"of"Requests"/"User"
#"of"Log"Messages"
#"of"ExcepSons"
#"Objects"Allocated"
#"Objects"In"Cache"
Cache"Hit"RaSo"
#"of"Images"
#"of"SQLs"
#"SQLs"per"Request"Availability"
#"HTTP"3xx,"4xx"
Page"Size"

31. 57"
Commit"Stage"
• Compile(
• Execute(Unit(Test(
• Code(Analysis(
• Build(installers(
Automated""
Acceptance""
TesSng"
Automated""
Capacity""
TesSng"
Manual"tesSng"
• Key(showcases(
• Exploratory(tes;ng(
"
"
Release"
Unit)&)IntegraJon)Tests)
FuncJonal)Tests)
Performance)Tests)
ProducJon)
Monitoring)
FuncJonal)Tests)
58"58"

32. 59"
60"
If"we"do"all"that"

33. 61"
Which"gives"you"more"Sme"for"the"real"important"
things"in"life"…"
62"
Want"MORE"of"these"and"more"details?"
h%p://apmblog.compuware.com""

34. 63"
FREE"Products"&"More"Info"
• dynaTrace$Enterprise$
– Full"End6to6End"Visibility"in"your"Java,".NET,"PHP"Apps"
– Sign"up"for"a"15"Days"Free"Trial"on"h%p://goo.gl/1ZFQZ0""
• dynaTrace$AJAX$EdiQon$
– Browser"DiagnosScs"for"IE"+"FF"
– Download"@"h%p://ajax.dynatrace.com""
• Our$Blog:"h%p://apmblog.compuware.com"