Learn how to configure Rundeck Enterprise to automatically failover to another cluster member if an instance goes down during a job execution or is not available when a job is scheduled to run. In this webinar you will learn how to configure: Automatic Job Schedule takeovers Remote execution policies Automatic job resume for executions running during an outage Manually move job schedule ownership to another Rundeck instance See the full video here: https://rundeck.wistia.com/medias/tuznbnw8eg Links: List Running Executions: https://docs.rundeck.com/docs/api/rundeck-api.html#listing-running-executions Cluster mode, including takeover and listing schedules: https://docs.rundeck.com/docs/api/rundeck-api.html#cluster-mode List incomplete executions: https://docs.rundeck.com/docs/api/rundeck-api.html#list-executions-with-incomplete-log-storage Metrics Data: https://docs.rundeck.com/docs/api/rundeck-api.html#metrics-data Cluster Overview: https://docs.rundeck.com/docs/enterprise/#overview Heartbeat setting: https://docs.rundeck.com/docs/administration/cluster/autotakeover/#configure-autotakeover Remote Execution: https://docs.rundeck.com/docs/administration/configuration/remote-job-execution.html#cluster-remote-execution Autotaker of job schedule: https://docs.rundeck.com/docs/administration/cluster/autotakeover/#configure-autotakeover

1. High Availability - Jobs
Automatic schedule and execution retry

2. Agenda
Cluster overview
Job execution
Heartbeat
Recovery/retry
Schedule auto takeover

3. Rundeck cluster

4. Remote execution options
# Policy type None, Random, RoundRobin, Load, Preset
rundeck.clusterMode.remoteExecution.policy = RoundRobin
# Identifies rundeck servers
rundeck.clusterMode.remoteExecution.config.allowed = other,self
# Send to a member marked inactive
rundeck.clusterMode.remoteExecution.profile.profile1.config.activeOnly = true

5. Heartbeat settings
# Frequency
rundeck.clusterMode.heartbeat.interval=30
# Startup delay
rundeck.clusterMode.heartbeat.delay=10
# Mark another member inactive
rundeck.clusterMode.heartbeat.considerInactive=60
# Mark another member dead
rundeck.clusterMode.heartbeat.considerDead=120
# remote execute/abort message processing
rundeck.clusterMode.remoteExec.process.interval=10

6. Job execution recover/auto retry
# enable auto cleanup of stale jobs on member death
rundeck.clusterMode.recoverExecutions.enabled=true
# policy for members to accept as targets of auto cleanup. Can be 'Any' or 'Static'
rundeck.clusterMode.recoverExecutions.policy=Any
# delay in seconds before proceeding with autoretry proposal
rundeck.clusterMode.recoverExecutions.delay=30
# delay in seconds before doing another autoretry of the same member
rundeck.clusterMode.recoverExecutions.sleep=60

7. Scheduled job autotakeover
# Enable for members detected as "dead"
rundeck.clusterMode.autotakeover.enabled=true
# "Any": all dead nodes. "Static": only allowed uuids
rundeck.clusterMode.autotakeover.policy=any
# Wait after sending autotakeover proposal
rundeck.clusterMode.autotakeover.delay = 60
# Sleep min between autotakeover attempts
rundeck.clusterMode.autotakeover.sleep = 300

8. Combining policies for HA

9. Questions
pwonson@rundeck.com

10. Resource/References
● API
○ List running executions
○ Cluster mode, including takeover and listing schedules
○ List incomplete executions
○ Metrics data
● Cluster Overview
● Heartbeat setting
● Remote Execution
● Autotaker of job schedule