"The Cloud" is a buzzword these days, but in this talk Nate Aune and Sally Kleinfeldt describe what you need to know about public and private clouds, why it's important, and why you might want to consider it for your company/organization. We'll discuss the pros/cons of public cloud providers such as Amazon Web Services and Rackspace Cloud, and how you can build your own private cloud using OpenStack or Eucalpytus, or even a hybrid that leverages both public and private clouds. Lastly, we'll look at how Plone can be deployed onto the cloud, and best practices for setting up your Plone site in a flexible and scalable way to take full advantage of all that these cloud providers have to offer. In particular, we'll examine the Connexions/Rhaptos project as a case study, and how they benefited from an Amazon-backed infrastructure.
Video of the presentation is available here: https://streaming.psu.edu/media/?movieId=13392
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Moving to the Cloud
1. Moving
to the
Cloud
Nate Aune
(Presented by Sally Kleinfeldt)
Jazkarta.com
Plone Symposium East 2011
Penn State University
2. Agenda
• Benefits of cloud computing
• What services does Amazon provide?
• 5 minute launch of Plone on EC2
• Case study: Rhaptos hosting on EC2
• Questions?
11. Characteristics of Truly
Scalable Service
• Increasing resources results in a
proportional increase in performance
• capable of handling heterogeneity, loosely
coupled
• operationally efficient
• resilient
• becomes more cost effective when it grows
17. Compared to Rackspace
• Rackspace is still catching up to AWS (just
announced load balancing a few weeks ago)
• Rackspace is a bit easier to get started with
• sends you the root password
• machines already have a static IP
• the disk storage is persisted
• Prices are comparable to AWS
• Cheapest Rackspace VM is $14/mo
18. Private clouds
• Provision virtual machines on your own
hardware
• Pros: you control your own machines.
• Cons: you have to manage your own
machines
• Examples: Eucalyptus, OpenStack
http://open.eucalyptus.com
http://openstack.org/
24. Set your access keys
File: setkeys.sh
#!/bin/bash
export AWS_ACCESS_KEY_ID="<your_access_key_id>"
export AWS_SECRET_ACCESS_KEY="<your_secret_access_key>"
Set these environment variables.
$ source setkeys.sh
$ env
25. Start the demo server
$ ./bin/aws start demo-server
INFO: Instance 'demo-server' unavailable
INFO: Creating instance 'demo-server'
INFO: Instance created, waiting until it's available
....
INFO: Instance 'demo-server' available
INFO: Instance running.
INFO: Instances DNS name ec2-50-16-25-92.compute-1.amazonaws.com
INFO: Instances public DNS name ec2-50-16-25-92.compute-1.amazonaws.com
26. Benefits of EC2
• Seamless disaster recovery with repeatable
deployments using buildout and mr.awsome
• Easy launching of testing, staging and QA
servers that are isolated from your
production environment (stop when not
used to avoid paying for them)
• Pay-for-what-you-use metered pricing to
launch servers on-demand during peak
periods
• Auto-scaling to launch new Zeo client
machines and shut them down when no
longer needed
28. Log In Contact Us Report a Bug
Search
Home Content Lenses About Us Help MyCNX
You are here: Home
MY ACCOUNT
Connexions is: FIND CONTENT
a place to view and share educational material made of Username
16547 reusable modules woven into 1007 collections.
small knowledge chunks called modules that can be Password
organized as courses, books, reports, etc. Anyone may view Search Content Go
Log in
or contribute: or browse by ...
Arts Get an account
authors create and collaborate Subject
Business Forgot your password?
s
instructors rapidly build and share custom collections Language Humanities
Mathematics and Statistics
learners find and explore content Popularity
Science and Technology
More about us ... New features Title, author, etc. Social Sciences
FEATURED CONTENT
CREATE CONTENT
Advanced Algebra II
Creating content in Connexions is as easy as 1, 2, 3:
The world of second-year SPOTLIGHT
algebra comes alive in Kenny
Felder's Advanced Algebra II. User feedback
Felder, a high school math
"I enjoy using
teacher in North Carolina,
the
designed his course using a Connexions
nontraditional approach to a very site and the
traditional subject. Centered
around a Homework and 1 Get an account
and log in to 2 Make a module
from scratch or 3 Publish your
works, sharing
philosophy
behind it
Activities book, along with your workspace. convert it from a them with the world. encourages
accompanying Conceptual Word doc. one to learn
Explanations and Teacher's Guide collections, new fields and
Advanced Algebra II tackles topics by focusing on to network with others. Your
Jump right in Guides and tutorials
comprehension rather than straight lecture delivery. site is my new homeschool on
Get an account New author guide the internet. It is helping me
Advanced Algebra II was selected by California's Free to grow as a writer of
Digital Textbook Initiative to be included in a list of How to create a module in Connexions Tutorial and academic materials as I read
free textbooks available to California schools starting minutes Reference what others have written so as
in Fall 2009. How to create a collection to shape my own. I am also
amazed at the range of
with existing modules institutions where your writers
Collaborative Statistics come from. Thanks for the use
of these great facilities."
Collaborative Statistics was
written by two faculty members
29. Search
Login Preferences Help/Guide About Trac
Wiki Timeline Roadmap Browse Source View Tickets New Ticket Search
Start Page Index History Last Change
Welcome to Connexions and Rhaptos Software Development
Connexions ( cnx.org) is legos for education. It is a world-wide repository and publishing system to build textbooks, courses, lessons, and
journals. Individual authors, major content publishers, and educational organizations all contribute. Lenses provide a way for high-quality material
to be endorsed, recommended, categorized, and aligned to standards. Rhaptos is the open source software that runs the Connexions site and
Enterprise Rhaptos is the software for running private content installations. Read more about Connexions and Rhaptos mission and architecture
here.
Enterprise Rhaptos Connexions and Rhaptos Development Table of Contents
Welcome to Connexions and Rhaptos Software Development
Enterprise Rhaptos
API's for Accessing Connexions Data
Accessing content, feeds, and statistics
API's for Accessing Data Quality Assurance and Testing Downloading, transforming and branding content
Software Architecture and Application Programming Interfaces
Connexions Rhaptos Development
Developer Resources (Getting involved)
Roadmap, Designs, Desires
Starting Points for Connexions and Rhaptos Software Development
Quality Assurance and Testing
Test, Release, and Configuration Docs
Using the Connexions Rhaptos Trac Bug System
Release and Configuration Details for Connexions and Rhaptos
Enterprise Rhaptos More about TRAC
Enterprise Rhaptos enables the creation, sharing, modification, and vetting of educational materials accessible to anyone, anywhere, anytime via
the World Wide Web. Rhaptos' unique features of permanent versions of all materials, a simple, semantic document language, and a powerful
lensing system for post-publication quality control, customized tagging, and community-based search and discovery are the engine behind a truly
reusable repository of knowledge and learning. Enterprise Rhaptos allows companies and individual organizations to host their own private
authoring platform and publishing repository.
Installation Instructions for Enterprise Rhaptos
API's for Accessing Connexions Data
30. Virtualization of Rhaptos
Two primary reasons to move to the cloud:
1) Hurricane territory (disaster recovery)
2) Increase adoption by other universities
Cost was also a consideration but not a
primary reason for carrying out the project.
31. Deploy to Amazon EC2
with Python scripts
• Define server profiles with OS dependencies
and turn into an AMI (Amazon Machine Image)
• Install & configure application on EBS volume
• Take snapshot of volume
• Launch new AMIs and attach EBS volume
created from saved snapshot.
37. #!/bin/bash
set -e -x
export DEBIAN_FRONTEND=noninteractive
apt-get update && apt-get upgrade -y
echo "----- Install system packages for Plone"
apt-get -y install python2.6-dev subversion build-essential
echo "----- Create an unprivileged user zope with password disabled"
adduser zope --disabled-password --gecos ""
echo "----- Check out the buildout"
cd /home/zope
echo 'pn' | svn co http://svn.jazkarta.com/.../.../trunk buildout
echo "----- Make the zope user the owner of this directory"
chown -R zope:zope buildout
echo "----- Run the buildout as user zope"
cd buildout
sudo -u zope python2.6 bootstrap.py -c backend.cfg
sudo -u zope ./bin/buildout -v -c backend.cfg
echo "----- Start Zeo processes"
sudo -u zope ./bin/supervisord
sudo -u zope ./bin/supervisorctl status
echo "----- Make sure that Supervisor starts on reboot"
ln -s /home/zope/buildout/etc/supervisor /etc/init.d/supervisor
chmod +x /etc/init.d/supervisor
update-rc.d supervisor defaults
38. Must define the FQDN
of the Zeo clients and
Zeo servers
[buildout]
backend-dns-name = ec2-174-129-192-63.compute-1.amazonaws.com
frontend1-dns-name = ec2-174-129-116-33.compute-1.amazonaws.com
frontend2-dns-name = ec2-184-73-240-104.compute-1.amazonaws.com
When these names are resolved *on* the server, the DNS
server returns the private IP, which means you avoid paying
bandwidth charges for outgoing/incoming traffic.
39. Start up the servers
with one command
$ ./bin/aws start backend
$ ./bin/aws start frontend1
$ ./bin/aws start frontend2
$ ./bin/aws start proxy
42. Heartbeat
monitors Zeo
server and moves to
backup in case of
failure
DRDB
Distributed storage
system similar in
principle in RAID,
but which works
over the network.
43. Improvements to the
deployment process
Before After
• time to launch a new site
was days
• brittle non-repeatable
installation process
• delay in procuring
hardware and installing
OS dependencies
44. Improvements to the
deployment process
Before After
• time to launch a new site • time to launch a new site
was days was minutes
• brittle non-repeatable • streamlined and
installation process automated installation
process
• delay in procuring
hardware and installing • virtual machines are
OS dependencies launched instantly and
OS dependencies are
already installed on the disk
image.
47. Deploy
$ fab deploy
What does this one command do?
• Provides a local packaging of the source into
a tarball
• Removes the old remotely deployed code
• Puts the source on the remote host
• Restarts the site
48. Benefits of Fabric
• Chain local-command, remote-command,
and remote-copy commands
• Substitute variables, and the splitting of
distinct operations (like in the case of the
'restart' command) so you run them
separately, with a fab restart, for example.
49. Example Fabric file
• See Alex Clark’s Fabric file:
• http://github.com/aclark4life/Debian-Deploy-
Plone/blob/master/fabfile.py
50. Boto
• Python library to script AWS services
• Programmatically:
• Launch new EC2 instances
• Attach EBS volumes
• Create S3 buckets
http://code.google.com/p/boto/
54. Chef & Puppet
• Configuration management tools to prepare
machines to behave according to their role.
• Unlike Fabric which “pushes” config to the
server, with Chef/Puppet, the server “pulls” it’s
configuration from a master node.
• Usually overkill for a simple deployment, but
with a multi-server deployment can be useful,
especially to keep machines up-to-date.
http://opscode.com/chef/
http://www.puppetlabs.com/