Build And Test In The Cloud JavaGruppen 2011 Denmark http://www.carlossanchez.eu Building and testing software can be a time- and resource-consuming task. Cloud computing/on-demand services such as Amazon EC2 provide a cost-effective way to scale applications and, for building and testing software, can reduce the time needed to find and correct problems, meaning a cost reduction as well. Properly configuring your build tools (Maven, Ant,...), continuous integration servers (Continuum, Hudson, Cruise Control,...), and testing tools (TestNG, Selenium,...) can enable you to run all the building/testing process in a cloud environment, simulating high-load environments, distributing long-running tests to reduce their execution time, using different environments for client or server applications, and so on—and in the case of on-demand services such as Amazon EC2, pay only for the time you use it. In this presentation we will introduce a development process and architecture using popular open source tools for the build and test process such as Apache Maven or Ant for building, Apache Continuum as continuous integration server with distributed capabilities, TestNG and Selenium for parallel, distributed and load testing, and how to configure them to achieve the best results and performance in several typical use cases (long running testing processes, different environments, different client platforms,...) by using he Amazon Elastic Computing Cloud EC2, and therefore reducing time and costs compared to other solutions.

1. Build and Test
in the Cloud
Carlos Sanchez
MaestroDev

2. Hej!
Solutions Architect at
MaestroDev
Member of Apache
Software Foundation
Maven PMC, Continuum,
Archiva
Eclipse IAM co-lead
etc...

3. Index
The tools
why the Cloud?
Distributed building
Maven integration tests
Maven & Selenium
Testing in different environments
And now in the cloud

4. The tools

5. Build: Apache Maven
Build tool and more
Dependency management
Execution of unit/integration tests
Start/stop application server
Automatic application deployment

6. Test Cases: TestNG
Unit/integration tests
parameterized tests
parallel testing

7. Integration Tests: Selenium
UI and Integration testing
Tests run in the browser
support for multiple browsers
Tests can be recorded
no need for developers writing tests
Selenium IDE Firefox plugin

8. Continuous Integration: Apache Continuum
Tight integration with Apache Maven
dependency handling
Trigger builds
when integration tests change
when the webapp changes
when the webapp dependencies change
Distributed building

9. Cloud Computing: Amazon Web Services
Amazon S3
storage
Amazon SQS
queue service
Amazon Mechanical Turk
manpower
Amazon EC2
computation

10. why the cloud?

11. $$$
Machines cost money
Bandwidth costs money
Electricity costs money
Server administration costs money

12. Resource Optimization
build servers can be repurposed
test servers working 10-20% of the time
ability to do expensive builds/tests
for a short amount of time

13. Predictions cost money
Infrastructure
Cost $
time
Source: Amazon

14. Predictions cost money
Infrastructure
Cost $
Predicted
Demand
time
Source: Amazon

15. Predictions cost money
Infrastructure
Cost $
Predicted
Demand
Traditional
Hardware
time
Source: Amazon

16. Predictions cost money
Infrastructure
Cost $
Large
Capital
Expenditure
Predicted
Demand
Traditional
Hardware
time
Source: Amazon

17. Predictions cost money
Infrastructure
Cost $
Predicted
Demand
Traditional
Hardware
time
Source: Amazon

18. Predictions cost money
Infrastructure
Cost $
Predicted
Demand
Traditional
Hardware
Actual
Demand
time
Source: Amazon

19. Predictions cost money
Infrastructure
Cost $
Predicted
Demand
Opportunity Traditional
Cost Hardware
Actual
Demand
time
Source: Amazon

20. Predictions cost money
Infrastructure
Cost $
You just lost
customers
Predicted
Demand
Opportunity Traditional
Cost Hardware
Actual
Demand
time
Source: Amazon

21. Predictions cost money
Infrastructure
Cost $
Predicted
Demand
Traditional
Hardware
Actual
Demand
Automated
Virtualization
time
Source: Amazon

22. Availability
Can you get 3000 machines in 3 days?

23. Amazon EC2
Elastic Compute Cloud
Machines on demand

24. Amazon EC2
pay per hour
same price for
10 machines / 1 hour
1 machine / 10 hours
unlimited number of
machines
start in couple minutes

25. AMIs

26. AMIs
Amazon Machine Images

27. AMIs
Amazon Machine Images
Bundle your own

28. Distributed building

29. Credits: XKCD

30. Continuum
Master Continuum server
Centralized management
Builds, users, output,...
Can build too
Distributed build agents
distribute load
different environments
different networks/clouds

31. Distributed builds
Coupling
Maven dependencies determine parallelism
Environments
OS, JDK, installed libraries...
VM size

32. Configuration
Use properties
Allow customizing environments
-Djetty.port=8081
-Dweb.host=184.73.250.39

33. Configuration
Use settings.xml <settings>
For user / agents <profiles>
configuration <profile>
<id>acme</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<jetty.port>8081</jetty.port>
<web.host>184.73.250.39</web.host>
</properties>
</profile>
</profiles>
</settings>

34. Build agents in the cloud
Startup
Automatically start with instance
API
Auto-connect to master when instance starts
ie. using user-data on EC2

35. Starting more agents
Not worth at build time
Instance startup time
Pay for full hours
Use a pool
Keep a minimum number of instances in pool
Start/Stop more as needed

36. Maven EC2 plugin
http://mojo.codehaus.org/maven-ec2-plugin
start Amazon AMIs
stop at the end of the build
pass on user data

37. Multicloud APIs
Java - JClouds
http://code.google.com/p/jclouds/
Python (+ Java) - libcloud
http://incubator.apache.org/libcloud/

38. Provisioning
DevOps!
Puppet / Chef
Manage instances automagically

39. class buildagent {
$path="/usr/bin:/usr/sbin:/bin:/sbin"
# Java 6
package { jdk:
ensure => "1.6.0_23-fcs",
require => Class[yum],
}
# Builder
user{ "builder":
name => builder,
ensure => present,
home => "/home/builder",
managehome => true,
shell => "/bin/bash",
}
# Builder Agent packages
$builder_agent_packages = ["maven", "continuum-buildagent", "subversion",
"xorg-x11-server-Xvfb", "xorg-x11-server-utils"]
package {
$builder_agent_packages:
ensure => latest,
require => [Package[jdk]]
}

40. package {
firefox:
ensure => latest,#["3.6.9", "3.0.19"],
require => Class[yum]
}
$settings_xml = "/home/builder/.m2/settings.xml"
file { "/home/builder/.m2":
owner => "builder",
group => "builder",
mode => "0600",
ensure => directory,
require => User["builder"],
}
file { $settings_xml :
owner => "builder",
group => "builder",
mode => "0600",
content => template("builder/settings.xml.erb"),
require => File["/home/builder/.m2"],
}
service { continuum-buildagent :
enable => true,
ensure => running,
hasstatus => true,
hasrestart => true,
require => [Package[continuum-buildagent], File[$settings_xml]]
}

41. Fear of change
Affects behavior
ability of doing things that could not be done before
but requires new knowledge too
Embrace change!

42. Maven integration tests

43. The webapp to test
Apache Maven 2
Effective Implementation
Centrepoint sample webapp

44. A new Maven project
Centrepoint
selenium-tests
<groupId>com.effectivemaven.centrepoint</groupId>
<artifactId>selenium-tests</artifactId>
<version>1.0-SNAPSHOT</version>
<name>Centrepoint Selenium Test Suite</name>

45. that depends on
Centrepoint webapp
<dependency>
<groupId>com.effectivemaven.centrepoint</groupId>
<artifactId>webapp</artifactId>
<version>${project.version}</version>
<type>war</type>
</dependency>

46. using Jetty plugin to run it
automatically
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>

47. Record Selenium tests
SeleniumIDE

48. as Java

49. test cases
using Selenium API public void setUp() throws Exception {
setUp("http://localhost:18880/", "*chrome");
}
public void testNew() throws Exception {
selenium.open("/mainMenu.html");
selenium.click("link=Administration");
selenium.waitForPageToLoad("30000");
selenium.type("j_password", "admin");
selenium.click("rememberMe");
selenium.click("//input[@name='login']");
selenium.waitForPageToLoad("30000");
selenium.click("link=Edit Profile");
selenium.waitForPageToLoad("30000");
selenium.click("link=Main Menu");
selenium.waitForPageToLoad("30000");
selenium.click("link=Edit Profile");
selenium.waitForPageToLoad("30000");
selenium.click("document.forms[0].elements[24]");
selenium.waitForPageToLoad("30000");
}

50. Maven & Selenium

51. Selenium Architecture
Selenium
server

52. Maven & Selenium
Start Selenium server locally
Run integration tests
Stop selenium server

53. Selenium Maven plugin
http://mojo.codehaus.org/selenium-maven-plugin
selenium:start-server
selenium:stop-server

54. Selenium Maven plugin
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>selenium-maven-plugin</artifactId>
<version>1.0-rc-1</version>
<executions>
<execution>
<id>start-selenium</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start-server</goal>
</goals>
</execution>
</executions>
<configuration>
<background>${selenium.background}</background>
</configuration>
</plugin>

55. Testing in different
environments

56. Multi environment
Multiple Browsers
Multiple Operating Systems

57. Multi environment
Multiple Browsers
Multiple Operating Systems

58. Multi environment
Multiple Browsers
Multiple Operating Systems

59. Parallel
Run tests in parallel
in each browser
several browsers per test
Load testing

60. Selenium Grid

63. TestNG properties
Allows to run the same tests in different environments
and in parallel

64. TestNG properties
<suite name="Example" parallel="tests" thread-count="3">
<test name="iexplore">
<parameter name="browser" value="*iexplore"/>
<packages>
<package name="com.effectivemaven.centrepoint.selenium" />
</packages>
</test>
<test name="firefox2">
<parameter name="browser" value="*firefox2"/>
<packages>
<package name="com.effectivemaven.centrepoint.selenium" />
</packages>
</test>
<test name="opera">
<parameter name="browser" value="*opera"/>
<packages>
<package name="com.effectivemaven.centrepoint.selenium" />
</packages>
</test>
</suite>

65. TestNG properties
<suite name="Example" parallel="tests" thread-count="3">
<test name="iexplore">
<parameter name="browser" value="*iexplore"/>
<packages>
<package name="com.effectivemaven.centrepoint.selenium" />
</packages>
</test>
<test name="firefox2">
<parameter name="browser" value="*firefox2"/>
<packages>
<package name="com.effectivemaven.centrepoint.selenium" />
</packages>
</test>
<test name="opera">
<parameter name="browser" value="*opera"/>
<packages>
<package name="com.effectivemaven.centrepoint.selenium" />
</packages>
</test>
</suite>

66. and now in the Cloud

67. Countless possibilities
Internet Explorer
Firefox 2
Opera
Safari
Firefox 3
Firefox 2
etc...
no OS X :(

68. Amazon AMIs
Create AMIs for different environments to test
Read user data to customize the images at runtime
where is the Selenium Hub

69. Linux
Selenium headless
Use *nix X Virtual Frame Buffer
In Maven
selenium:xvfb

70. Windows
Run Selenium Agent in user account
not local service
Selenium needs to have access to Desktop
Disable password expiration
Disable/Open windows firewall
netsh firewall add portopening TCP 8181 “Continuum BuildAgent”
netsh firewall add portopening TCP 9000 “Selenium RC 9000”

71. Browser config
Uninstall Internet Explorer Enhanced Security
Configuration
Windows 2003
http://www.visualwin.com/IE-enhanced-security/ in Add/Remove programs
Windows 2008
http://windowsitpro.com/article/articleid/96159/how-do-i-turn-off-internet-explorer-
enhanced-security-configuration-in-windows-server-2008.html
Safari/Firefox
Disable popup blocker

72. Questions?

73. Tak!
http://maestrodev.com
http://carlossanchez.eu
csanchez@maestrodev.com
carlos@apache.org
csanchez

74. Credits
Son of Man Lego
http://www.flickr.com/photos/hoyvinmayvin/4702772452/
XKCD - Compiling
http://xkcd.com/303/