Product SiteDocumentation Site

Pacemaker Administration

Managing Pacemaker Clusters

Edition 1

Andrew Beekhof

Red Hat

Ken Gaillot

Red Hat

Legal Notice

Copyright © 2009-2019 Andrew Beekhof.
The text of and illustrations in this document are licensed under version 4.0 or later of the Creative Commons Attribution-ShareAlike International Public License ("CC-BY-SA")[1].
In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
In addition to the requirements of this license, the following activities are looked upon favorably:
  1. If you are distributing Open Publication works on hardcopy or CD-ROM, you provide email notification to the authors of your intent to redistribute at least thirty days before your manuscript or media freeze, to give the authors time to provide updated documents. This notification should describe modifications, if any, made to the document.
  2. All substantive modifications (including deletions) be either clearly marked up in the document or else described in an attachment to the document.
  3. Finally, while it is not mandatory under this license, it is considered good form to offer a free copy of any hardcopy or CD-ROM expression of the author(s) work.


This document has instructions and tips for system administrators who need to manage high-availability clusters using Pacemaker.

Table of Contents

1. Document Conventions
1.1. Typographic Conventions
1.2. Pull-quote Conventions
1.3. Notes and Warnings
2. We Need Feedback!
1. Read-Me-First
1.1. The Scope of this Document
1.2. What Is Pacemaker?
1.3. Cluster Architecture
1.4. Pacemaker Architecture
1.5. Node Redundancy Designs
2. Installing Cluster Software
2.1. Installing the Software
2.2. Enabling Pacemaker
2.2.1. Enabling Pacemaker For Corosync version 2 and greater
3. The Cluster Layer
3.1. Pacemaker and the Cluster Layer
3.2. Managing Nodes in a Corosync-Based Cluster
3.2.1. Adding a New Corosync Node
3.2.2. Removing a Corosync Node
3.2.3. Replacing a Corosync Node
4. Configuring Pacemaker
4.1. How Should the Configuration be Updated?
4.1.1. Editing the CIB Using XML
4.1.2. Quickly Deleting Part of the Configuration
4.1.3. Updating the Configuration Without Using XML
4.2. Making Configuration Changes in a Sandbox
4.3. Testing Your Configuration Changes
4.3.1. Small Cluster Transition
4.3.2. Complex Cluster Transition
4.4. Do I Need to Update the Configuration on All Cluster Nodes?
4.5. Working with CIB Properties
4.6. Querying and Setting Cluster Options
4.6.1. When Options are Listed More Than Once
4.7. Connecting from a Remote Machine
5. Monitoring a Pacemaker Cluster
5.1. Using crm_mon
6. Troubleshooting Cluster Problems
6.1. Logging
6.2. Transitions
6.3. Further Information About Troubleshooting
7. Upgrading a Pacemaker Cluster
7.1. Pacemaker Versioning
7.2. Upgrading Cluster Software
7.2.1. Complete Cluster Shutdown
7.2.2. Rolling (node by node)
7.2.3. Detach and Reattach
7.3. Upgrading the Configuration
7.4. What Changed in 2.0
7.5. What Changed in 1.0
7.5.1. New
7.5.2. Changed
7.5.3. Removed
8. Resource Agents
8.1. Resource Agent Actions
8.2. OCF Resource Agents
8.2.1. Location of Custom Scripts
8.2.2. Actions
8.2.3. How are OCF Return Codes Interpreted?
8.2.4. OCF Return Codes
8.3. LSB Resource Agents (Init Scripts)
8.3.1. LSB Compliance
A. Revision History

List of Figures

1.1. Example Cluster Stack
1.2. Internal Components
1.3. Active/Passive Redundancy
1.4. Shared Failover
1.5. N to N Redundancy

List of Tables

4.1. Environment Variables Used to Connect to Remote Instances of the CIB
4.2. Extra top-level CIB properties for remote access
7.1. Upgrade Methods
7.2. Version Compatibility Table
8.1. Required Actions for OCF Agents
8.2. Optional Actions for OCF Resource Agents
8.3. Types of recovery performed by the cluster
8.4. OCF Return Codes and their Recovery Types

List of Examples

2.1. Corosync configuration file for two nodes myhost1 and myhost2
2.2. Corosync configuration file for three nodes myhost1, myhost2 and myhost3
4.1. Safely using an editor to modify the cluster configuration
4.2. Safely using an editor to modify only the resources section
4.3. Searching for STONITH-related configuration items
4.4. Creating and displaying the active sandbox
4.5. Use sandbox to make multiple changes all at once, discard them, and verify real configuration is untouched
4.6. Attributes set for a cib object
4.7. Deleting an option that is listed twice
5.1. Sample output from crm_mon -1
5.2. Sample output from crm_mon -n -1