Today there are a lot of cloud providers, with a wide range of offers. Web projects usually have continuously changing needs: what worked well yesterday may not be enough today. These two facts became quite obvious for us while migrating a large PHP application from Rackspace to Amazon. In this session I’d like to share this experience highlighting infrastructure and code evolution, migration steps, cost analisys, issues.

1. Hopping
a tale of migration from
one cloud provider to another
in Clouds

2. Michele Orselli
CTO@Ideato
_orso_
micheleorselli / ideatosrl
mo@ideato.it

9. Let’s start from the beginning…

11. What do we Italians like?

16. “Italians lose wars as if they were
football matches, and football
matches as if they were wars”
Winston Churchill

18. Peaks on gen - jun - aug up to 70 M pg/mth
Peaks during big matches

20. PaaS
IaaS
SaaS

21. PaaS
IaaS
SaaS

22. PaaS Platform as a Service
Zero configuration (almost)
Push the code “on the cloud” and
you’re done

24. Hard limits on resource (e.g 50 db con)
Deploy via ftp on shared nfs dir (sf cache mess)
Blackbox: No realtime log, no access
PHP 5.3

25. Web: all articles
categories, team
details, ...
symfony1

26. Mobile: all articles
categories, team
details, ...
Sf2 components

27. Community Site
with user
generated content
Symfony2

28. μServices

30. Macro services

31. Talk: api for
comments, votes,
ratings
Symfony2

32. Adv: api for
serving ads
Symfony2

33. Media: api for
asset mgmt
Symfony2

35. The problems started with talk…

37. Quick wins
Tuning HTTP response headers
Caching more endpoints
Optimize queries

39. Tuning HTTP Headers

40. 1 $date = new DateTime();
2 $date->modify("+$lifetime seconds");
3
4 $response->setExpires($date);
5 $response->setMaxAge($lifetime);
6 $response->setSharedMaxAge($lifetime);

43. PaaS
IaaS
SaaS

44. PaaS
IaaS
SaaS

45. First candidate for migration was…
the talk app

47. PHP from 5.3 to 5.6
Mysql from 5.0 to 5.6
Apache to nginx (+ php fpm)

48. Web servers ip are dynamic
Can connect only through bastion
Share user sessions between servers

49. Web servers ip are dynamic
Can connect only through bastion
Share user sessions between servers

50. 1 'hosts' => function () {
2 $c = Ec2Client::factory([
3 'profile' => 'calciomercato',
4 'region' => 'eu-central-1',
5 ]);
6
7 $ips = new GetInstancesIps($c);
8
9 return $ips->execute();
10 }

51. 1 public function execute()
2 {
3 $instances = $this->ec2Client
4 ->describeInstances(
5 [
6 'DryRun' => false,
7 'Filters' => [
8 [
9 'Name' => 'instance.group-name',
10 'Values' => ['Web Public Auto-assign SG'],
11 ],
12 ],
13 ]);
14
15 return $instancesDescription->getPath(
16 'Reservations/*/Instances/*/NetworkInterfaces/*/
PrivateIpAddresses/*/PrivateIpAddress'
18 );
19 }

52. Web servers ip are dynamic
Can connect only through bastion
Share user sessions between servers

53. 1 Host cmbastion
2 HostName xx.xx.xx.xx
3 User ec2-user
4 Port 9760
5 StrictHostKeyChecking no
6 UserKnownHostsFile /dev/null
7 IdentityFile ~/.ssh/cm_bastion.pem
8 LogLevel quiet

54. 10 Host 10.0.14.*
11 User centos
12 StrictHostKeyChecking no
13 UserKnownHostsFile /dev/null
14 IdentityFile ~/.ssh/cm_production.pem
15 ProxyCommand ssh -W %h:%p cmbastion
16 LogLevel quiet
17
18 Host 10.0.24.*
19 User centos
20 StrictHostKeyChecking no
21 UserKnownHostsFile /dev/null
22 IdentityFile ~/.ssh/cm_production.pem
23 ProxyCommand ssh -W %h:%p cmbastion
24 LogLevel quiet

55. Web servers ip are dynamic
Can connect only through bastion
Share user sessions between servers

56. Nginx static cache

57. 1 fastcgi_cache_key
“$scheme$request_method$host$request_uri";
2 fastcgi_cache_lock on;
3 fastcgi_cache_revalidate on;
4 fastcgi_cache_valid 3m;

58. 1 if ($request_method ~ ^(POST|PUT|DELETE)$ ) {
2 set $no_cache 1;
3 }
4
5 if ($request_uri ~* "/api/queue") {
6 set $no_cache 1;
7 }
8
9 location ~ ^/(app|dev).php(/|$) {
[..]
17
18 # Enable fastcgi_cache
19 add_header X-Cache $upstream_cache_status;
20 fastcgi_cache CALCIOMERCATO_TALK;
21 fastcgi_cache_bypass $no_cache;
22 fastcgi_no_cache $no_cache;
23 }

59. 8
9 location ~ ^/(app|dev).php(/|$) {
10 fastcgi_split_path_info ^(.+.php)(/.*)$;
11 include fastcgi_params;
12 fastcgi_param SCRIPT_FILENAME
$realpath_root$fastcgi_script_name;
13 fastcgi_param DOCUMENT_ROOT $realpath_root;
14 fastcgi_param HTTPS off;
15 fastcgi_index app.php;
16 fastcgi_intercept_errors on;
17
18 # Enable fastcgi_cache
[..]
23 }

60. Load test using old logs
Create AMI Images
Deploy latest version of the code
Switch DNS

64. The platform now runs on two clouds: RCS
and AWS

65. Backup

67. DB and Machine snapshotted every night
Copied to another region

68. snapshot_id=`
aws --profile snapshotdr rds
describe-db-snapshots
--db-instance-identifier $instance_identifier
--region eu-central-1
| tail -n 1
| awk -F '{print $5}’
`

69. aws rds copy-db-snapshot
--source-db-snapshot-identifier arn:aws:rds:eu-
central-1:$id:snapshot:$snapshot_id
--region eu-west-1
--target-db-snapshot-identifier mysqlrds-snap-copy-$NOW
--copy-tags

70. snapshot_id=`aws --profile snapshotdr rds
describe-db-snapshots
--region eu-west-1
--db-instance-identifier $instance_identifier
| head -n 1
| awk -F '{print $4}’`
aws rds delete-db-snapshot
--region eu-west-1
--db-snapshot-identifier $snapshot_id

71. Adv

72. Only stateless apis
Small database small traffic
Infrastructure was already set
Easy!

73. Created a Cloudfront distribution for
dynamic content (adv.calciomercato.com)

75. Mobile

76. No database, it consumes data from other
services
High impact, 40% of the total traffic

77. How to deal with static assets?

78. s3://com-calciomercato-cdn-mobile/

79. Created 2 Cloudfront distribution
dynamic content (m.calciomercato.com)
static content (cdnmobile.calciomercato.com)

80. Sync asset to s3 via s3cmd
s3cmd
-m text/javascript
--no-preserve sync
/var/www/mobile/content/js
s3://com-calciomercato-cdn-mobile/

81. Deploy on a sample machine
Performance test based on log
Deploy
Rebuilding AMI
Switch DNS

82. Tweaking assets caching
s3cmd
--recursive modify
--add-header='Cache-Control:max-age=3600'
s3://com-calciomercato-cdn-mobile/assets/

83. Tweaking assets caching
s3cmd
--recursive modify
--add-header='Cache-Control:max-age=3600'
s3://com-calciomercato-cdn-mobile/assets/
Less CF -> S3 requests
Less $$$

84. Community

85. Allows uses to create their own personal blog
First app that can be considered “complete”
Exposes api for user related stuff
Works as SSO

86. Web servers ip are dynamic
Can connect only through bastion
Share user sessions between servers

88. 1 services:
2 memcache:
3 class: Memcache
4 calls:
5 - [ addServer, [%memc_host%, %memc_port% ]]
6 session.handler.memcache:
7 class:
8
SymfonyComponentHttpFoundationSessionStorageHa
ndlerMemcacheSessionHandler
10 arguments: [
11 @memcache,
12 { prefix: %session_memcache_prefix%,
13 expiretime: %session_memcache_expire% }
14 ]

89. 1 framework:
2 session:
3 handler_id: %session_handler_id%

90. Deal with User Generated Content

91. Created 2 Cloudfront distributions
dynamic content (vxl.calciomercato.com)
static content (cdnvxl.calciomercato.com)

92. Gaufrette
Filesystem abstraction layer
http://knplabs.github.io/Gaufrette/

93. 1 knp_gaufrette:
2 adapters:
3 photo_storage:
4 aws_s3:
5 service_id: cdn.amazon_s3
6 bucket_name: %amazon_s3_bucket_name%
7 options:
8 directory: data
9 filesystems:
10 photo_storage:
11 adapter: photo_storage
12 alias: photo_storage_filesystem
13

94. 8 cdn.amazon_s3:
9 class: AwsS3S3Client
10 factory_class: AwsS3S3Client
11 factory_method: 'factory'
12 arguments:
13 -
14 key: %cdn.amazon_s3.aws_key%
15 secret: %cdn.amazon_s3.aws_secret_key%
16 region: eu-central-1

95. 7 class PhotoUploader
8 {
[..]
27 public function upload(File $file, $dir)
28 {
29 $fullPath = $dir.'/'.$file->getFilename();
30
31 if (!in_array($file->getMimeType(), self::$allowedTypes)) {
32 throw new InvalidArgumentException($file->getMimeType());
35 }
36
40 return $this->filesystem->write(
41 $fullPath,
42 file_get_contents($file->getPathname())
43 );
44 }

96. 7 class PhotoUploader
8 {
[..]
27 public function upload(File $file, $dir)
28 {
29 $fullPath = $dir.'/'.$file->getFilename();
30
31 if (!in_array($file->getMimeType(), self::$allowedTypes)) {
32 throw new InvalidArgumentException($file->getMimeType());
35 }
36
40 return $this->filesystem->write(
41 $fullPath,
42 file_get_contents($file->getPathname())
43 );
44 }
return $this->filesystem->write(
41 $fullPath,
42 file_get_contents($file->getPathname())
43 );

97. Deploy on a sample machine
Performance test based on log
Deploy
Rebuilding AMI
Copy User Assets on S3
Switch DNS

99. Web

100. Oldest and biggest codebase
Proxy for mobile calls
High traffic 60%

101. PHP 5.6 not supported by symfony 1

102. Plan A: try to upgrade sf1 to support php 5.6
Plan B: deploy web on different machines

103. https://github.com/LExpress/symfony1

104. 1 protected function camelize($text)
2 {
3 return preg_replace(array('#/(.?)#e', '/(^|_|-)+(.)/e'), array("'::'.
4 strtoupper('1')", "strtoupper('2')"), $text);
5 }
6
7 public static function camelize($text)
8 {
9 return strtr(ucwords(strtr($text, array('/' =>
10 ':: ', '_' => ' ', '-' => ' '))), array(' ' => ''));
11 }

105. Created 1 Cloudfront distribution for static
content (cdnweb.calciomercato.com)

106. https://blog.cloudflare.com/zone-apex-naked-domain-root-domain-cname-supp/
calciomercato.com => cmelb-463612445.eu-central-1.elb.amazonaws.com

107. Deploy on a sample machine
Performance test based on log
Deploy
Rebuilding AMI
Switch DNS

109. Media

110. Only stateful api
Handles image thumbailing
Pretty big archive (70GB)

111. 1 public function generateThumbAndUploadToCdn(File $file, $width, $height)
2 {
3 $downloadedFile = $this->downloadFromFileManager($file);
4
5 $cdnKey = $this->generateThumbCdnKey($file, $width, $height);
6 $resizedFile = $this->resizeFilesystemImage($downloadedFile, $width,
$height);
8
9 $optimizedFile = $this->optimizeImage($resizedFile);
10
11 $this->uploadFileToCdn($optimizedFile, $cdnKey)
12
13 $this->updateFileInfoTumbs($file, $width, $height, $cdnKey);
14
15 $this->deleteTemporaryFile($downloadedFile);
16 $this->deleteTemporaryFile($optimizedFile);
17
18 return true;
19 }

112. Transfer from Rackspace CDN to S3

113. #!/bin/bash
login="USERNAME_FTP"
pass="FTP_PASSWORD"
host="HOST_FTP_RACKSPACE"
remote_dir='/web/content/data'
local_dir=“/var/www/vhosts/media.calciomercato.pro/data"
base_name="$(basename "$0")"
lftp -u $login,$pass $host << EOF
set ftp:ssl-allow no
set mirror:use-pget-n 5
mirror -c -P5 --log="/var/log/$base_name.log" "$remote_dir"
"$local_dir"
quit

115. 1 public function slugifyFilename($text)
2 {
3 $text = preg_replace('~[^pLd]+~u', '.', $text);
4 $text = trim($text, '-');
5
6 if (function_exists('iconv')) {
7 $text = iconv('utf-8', 'us - ascii//TRANSLIT', $text);
8 }
9
10 $text = preg_replace('~[^-w.]+~', '', $text);
11
12 return $text;
13 }

117. Monitoring

118. cloudwatch -> nagios -> catci
cw retention max 2 sett

120. 2 weeks retention:
integration with
cacti and nagios

121. ...and Slack

122. Autoscaling

123. 2 autoscaling groups
3 metrics
- CPU % (> 70%)
- Response Time (> 100 ms)
- # of req/sec (> 10000)

125. Full migration took 1 year

126. 50% cost reduction

127. This talk is not about blaming RCS…

128. …simply it wasn’t suitable anymore for our
needs :-)

130. Macro services FTW!
HTTP cache helped us a lot!
Measure measure measure

131. Rationalising api: less call, less $$$
Reorganise frontend stuff
Get rid of sf1

132. Upgrading to php7
Upgrading to ALB (HTTP/2)

134. Verona

135. JsDay: May 10th-11th
PHPDay: May 12th-13th
Verona

136. Thank you!
https://joind.in/talk/cd6af
Michele Orselli
CTO@Ideato
_orso_
micheleorselli / ideatosrl
mo@ideato.it