3.2. Cluster Options

Cluster options, as you might expect, control how the cluster behaves when confronted with certain situations.

They are grouped into sets within the crm_config section, and, in advanced configurations, there may be more than one set. (This will be described later in the section on Chapter 8, Rules where we will show how to have the cluster use different sets of options during working hours than during weekends.) For now, we will describe the simple case where each option is present at most once.

You can obtain an up-to-date list of cluster options, including their default values, by running the man pengine and man crmd commands.

Table 3.2. Cluster Options

Option	Default	Description
`dc-version`		Version of Pacemaker on the cluster’s DC. Determined automatically by the cluster. Often includes the hash which identifies the exact Git changeset it was built from. Used for diagnostic purposes.
`cluster-infrastructure`		The messaging stack on which Pacemaker is currently running. Determined automatically by the cluster. Used for informational and diagnostic purposes.
`expected-quorum-votes`		The number of nodes expected to be in the cluster. Determined automatically by the cluster. Used to calculate quorum in clusters that use Corosync 1.x without CMAN as the messaging layer.
`no-quorum-policy`	stop	What to do when the cluster does not have quorum. Allowed values: `ignore:` continue all resource management `freeze:` continue resource management, but don’t recover resources from nodes not in the affected partition `stop:` stop all resources in the affected cluster partition `suicide:` fence all nodes in the affected cluster partition
`batch-limit`	30	The number of jobs that the Transition Engine (TE) is allowed to execute in parallel. The TE is the logic in pacemaker’s CRMd that executes the actions determined by the Policy Engine (PE). The "correct" value will depend on the speed and load of your network and cluster nodes.
`migration-limit`	-1	The number of migration jobs that the TE is allowed to execute in parallel on a node. A value of -1 means unlimited.
`symmetric-cluster`	TRUE	Can all resources run on any node by default?
`stop-all-resources`	FALSE	Should the cluster stop all resources?
`stop-orphan-resources`	TRUE	Should deleted resources be stopped?
`stop-orphan-actions`	TRUE	Should deleted actions be cancelled?
`start-failure-is-fatal`	TRUE	Should a failure to start a resource on a particular node prevent further start attempts on that node? If FALSE, the cluster will decide whether to try starting on the same node again based on the resource’s current failure count and `migration-threshold` (see Section 9.3.2, “Moving Resources Due to Failure”).
`enable-startup-probes`	TRUE	Should the cluster check for active resources during startup?
`maintenance-mode`	FALSE	Should the cluster refrain from monitoring, starting and stopping resources?
`stonith-enabled`	TRUE	Should failed nodes and nodes with resources that can’t be stopped be shot? If you value your data, set up a STONITH device and enable this. If true, or unset, the cluster will refuse to start resources unless one or more STONITH resources have been configured.
`stonith-action`	reboot	Action to send to STONITH device. Allowed values are `reboot` and `off`. The value `poweroff` is also allowed, but is only used for legacy devices.
`stonith-timeout`	60s	How long to wait for STONITH actions (reboot, on, off) to complete
`cluster-delay`	60s	Estimated maximum round-trip delay over the network (excluding action execution). If the TE requires an action to be executed on another node, it will consider the action failed if it does not get a response from the other node in this time (after considering the action’s own timeout). The "correct" value will depend on the speed and load of your network and cluster nodes.
`dc-deadtime`	20s	How long to wait for a response from other nodes during startup. The "correct" value will depend on the speed/load of your network and the type of switches used.
`cluster-recheck-interval`	15min	Polling interval for time-based changes to options, resource parameters and constraints. The Cluster is primarily event-driven, but your configuration can have elements that take effect based on the time of day. To ensure these changes take effect, we can optionally poll the cluster’s status for changes. A value of 0 disables polling. Positive values are an interval (in seconds unless other SI units are specified, e.g. 5min).
`pe-error-series-max`	-1	The number of PE inputs resulting in ERRORs to save. Used when reporting problems. A value of -1 means unlimited (report all).
`pe-warn-series-max`	-1	The number of PE inputs resulting in WARNINGs to save. Used when reporting problems. A value of -1 means unlimited (report all).
`pe-input-series-max`	-1	The number of "normal" PE inputs to save. Used when reporting problems. A value of -1 means unlimited (report all).
`remove-after-stop`	FALSE	Advanced Use Only: Should the cluster remove resources from the LRM after they are stopped? Values other than the default are, at best, poorly tested and potentially dangerous.
`startup-fencing`	TRUE	Advanced Use Only: Should the cluster shoot unseen nodes? Not using the default is very unsafe!
`election-timeout`	2min	Advanced Use Only: If you need to adjust this value, it probably indicates the presence of a bug.
`shutdown-escalation`	20min	Advanced Use Only: If you need to adjust this value, it probably indicates the presence of a bug.
`crmd-integration-timeout`	3min	Advanced Use Only: If you need to adjust this value, it probably indicates the presence of a bug.
`crmd-finalization-timeout`	30min	Advanced Use Only: If you need to adjust this value, it probably indicates the presence of a bug.
`crmd-transition-delay`	0s	Advanced Use Only: Delay cluster recovery for the configured interval to allow for additional/related events to occur. Useful if your configuration is sensitive to the order in which ping updates arrive. Enabling this option will slow down cluster recovery under all conditions.
`default-resource-stickiness`	0	Deprecated: See Section 5.4.2, “Setting Global Defaults for Resource Meta-Attributes” instead
`is-managed-default`	TRUE	Deprecated: See Section 5.4.2, “Setting Global Defaults for Resource Meta-Attributes” instead
`default-action-timeout`	20s	Deprecated: See Section 5.5.3, “Setting Global Defaults for Operations” instead

3.2.1. Querying and Setting Cluster Options

Cluster options can be queried and modified using the crm_attribute tool. To get the current value of cluster-delay, you can run:

# crm_attribute --query --name cluster-delay

which is more simply written as

# crm_attribute -G -n cluster-delay

If a value is found, you’ll see a result like this:

# crm_attribute -G -n cluster-delay
scope=crm_config name=cluster-delay value=60s

If no value is found, the tool will display an error:

# crm_attribute -G -n clusta-deway
scope=crm_config name=clusta-deway value=(null)
Error performing operation: No such device or address

To use a different value (for example, 30 seconds), simply run:

# crm_attribute --name cluster-delay --update 30s

To go back to the cluster’s default value, you can delete the value, for example:

# crm_attribute --name cluster-delay --delete
Deleted crm_config option: id=cib-bootstrap-options-cluster-delay name=cluster-delay