This document discusses configuring HTCondor to flock jobs between two HTCondor pools running on different cloud platforms. It describes installing HTCondor on a Windows Server 2008 machine on Rackspace as pool 1 and another on Amazon EC2 as pool 2. It then explains modifying the condor_config file on both pools to specify the other pool's Central Manager IP address in the FLOCK_FROM and FLOCK_TO statements, allowing jobs to flock between the two pools. Testing shows the flocking works as each pool can see and submit jobs to the other.
1. Flocking HTCondor
Between
Two Clouds
Guy Tel-Zur
tel-zur@computer.org
March 2013
2. Goal
• Submit HTCondor jobs from Amazon EC2
HTCondor pool and flock to another HTCondor
pool on Rackspace.
• Both pools use Windows Server 2008
3.
4. Outline
• Install Windows Server 2008 with HTCondor
on Rackspace (Pool 1)
• Install Windows Serve 2008 with HTCondor on
EC2 (Pool 2)
• Modify the condor_config file
FLOCK_FROM and FLOCK_TO specifying the IP
addresses of the Central Managers.
• Verify the Flocking works
31. Add Flocking
• Edit condor_config
• At both ends:
– Set the IP addresses of the condor master nodes
in the flock_from and flock_to statements
– Save and restart condor
• That’s it !!!
32. Check Flocking at
both ends
Each pool “sees”
the other pool
NOTE! In this demo for simplicity firewalls were disabled. This is not recommended in
real production runs!