patroni.postgresql.sync module

class patroni.postgresql.sync.SyncHandler(postgresql: Postgresql)

Bases: object

Class responsible for working with the synchronous_standby_names.

Sync standbys are chosen based on their state in pg_stat_replication. When synchronous_standby_names is changed we memorize the _primary_flush_lsn and the current_state() method will count newly added names as “sync” only when they reached memorized LSN and also reported as “sync” by pg_stat_replication

__init__(postgresql: Postgresql) → None

_handle_synchronous_standby_names_change() → None

Handles changes of “synchronous_standby_names” GUC.

If “synchronous_standby_names” was changed, we need to check that newly added replicas have reached self._primary_flush_lsn. Only after that they could be counted as synchronous.

_process_replica_readiness(cluster: Cluster, replica_list: _ReplicaList) → None

Flags replicas as truly “synchronous” when they have caught up with _primary_flush_lsn.

Parameters:

cluster – current cluster topology from DCS
replica_list – collection of replicas that we want to evaluate.

current_state(cluster: Cluster) → _SyncState

Find the best candidates to be the synchronous standbys.

Current synchronous standby is always preferred, unless it has disconnected or does not want to be a synchronous standby any longer.

Standbys are selected based on values from the global configuration:

maximum_lag_on_syncnode: would help swapping unhealthy sync replica in case it stops responding (or hung). Please set the value high enough, so it won’t unnecessarily swap sync standbys during high loads. Any value less or equal to 0 keeps the behavior backwards compatible. Please note that it will also not swap sync standbys when all replicas are hung.

synchronous_node_count: controls how many nodes should be set as synchronous.

Parameters:: cluster – current cluster topology from DCS
Returns:: current synchronous replication state as a _SyncState object

set_synchronous_standby_names(sync: Collection[str], num: int | None = None) → None

Constructs and sets synchronous_standby_names GUC value.

Note

standbys in synchronous_standby_names will be sorted by name.

Parameters:

sync – set of nodes to sync to
num – specifies number of nodes to sync to. The num is set only in case if quorum commit is enabled

class patroni.postgresql.sync._Replica(pid: int, application_name: str, sync_state: str, lsn: int, nofailover: bool, sync_priority: int)

Bases: NamedTuple

Class representing a single replica that is eligible to be synchronous.

Attributes are taken from pg_stat_replication view and respective Cluster.members.

Variables:

pid – PID of walsender process.
application_name – matches with the Member.name.
sync_state – possible values are: async, potential, quorum, and sync.
lsn – write_lsn, flush_lsn, or replay_lsn, depending on the value of synchronous_commit GUC.
nofailover – whether the corresponding member has nofailover tag set to True.

_asdict(): Return a new dict which maps field names to their values.

_field_defaults = {}

_fields = ('pid', 'application_name', 'sync_state', 'lsn', 'nofailover', 'sync_priority')

classmethod _make(iterable): Make a new _Replica object from a sequence or iterable

_replace(**kwds): Return a new _Replica object replacing specified fields with new values

application_name: str: Alias for field number 1

lsn: int: Alias for field number 3

nofailover: bool: Alias for field number 4

pid: int: Alias for field number 0

sync_priority: int: Alias for field number 5

sync_state: str: Alias for field number 2

class patroni.postgresql.sync._ReplicaList(postgresql: Postgresql, cluster: Cluster)

Bases: List[_Replica]

A collection of :class:_Replica objects.

Values are reverse ordered by _Replica.sync_state and _Replica.lsn. That is, first there will be replicas that have sync_state == sync, even if they are not the most up-to-date in term of write/flush/replay LSN. It helps to keep the result of choosing new synchronous nodes consistent in case if a synchronous standby member is slowed down OR async node is receiving changes faster than the sync member. Such cases would trigger sync standby member swapping, but only if lag on this member is exceeding a threshold (maximum_lag_on_syncnode).

Variables:: max_lsn – maximum value of _Replica.lsn among all values. In case if there is just one element in the list we take value of pg_current_wal_flush_lsn().

__init__(postgresql: Postgresql, cluster: Cluster) → None

Create :class:_ReplicaList object.

Parameters:

postgresql – reference to :class:Postgresql object.
cluster – currently known cluster state from DCS.

static _should_cascade(members: CaseInsensitiveDict, replication: CaseInsensitiveDict, member: Member) → bool

Check whether member is supposed to cascade from another standby node.

Parameters:

members – members that are eligible to stream (state=running and don’t have nosync tag)
replication – state of pg_stat_replication, already filtered by member names from members
member – member that we want to check

Returns:

True if provided member should stream from other standby node in the cluster (according to replicatefrom tag), because some standbys in a chain already streaming from the primary, otherwise False

class patroni.postgresql.sync._SSN(sync_type: str, has_star: bool, num: int, members: CaseInsensitiveSet)

Bases: NamedTuple

class representing “synchronous_standby_names” value after parsing.

Variables:

sync_type – possible values: ‘off’, ‘priority’, ‘quorum’
has_star – is set to True if “synchronous_standby_names” contains ‘*’
num – how many nodes are required to be synchronous
members – collection of standby names listed in “synchronous_standby_names”

_asdict(): Return a new dict which maps field names to their values.

_field_defaults = {}

_fields = ('sync_type', 'has_star', 'num', 'members')

classmethod _make(iterable): Make a new _SSN object from a sequence or iterable

_replace(**kwds): Return a new _SSN object replacing specified fields with new values

has_star: bool: Alias for field number 1

members: CaseInsensitiveSet: Alias for field number 3

num: int: Alias for field number 2

sync_type: str: Alias for field number 0

class patroni.postgresql.sync._SyncState(sync_type: str, numsync: int, sync: CaseInsensitiveSet, sync_confirmed: CaseInsensitiveSet, active: CaseInsensitiveSet)

Bases: NamedTuple

Class representing the current synchronous state.

Variables:

sync_type – possible values: off, priority, quorum
numsync – how many nodes are required to be synchronous (according to synchronous_standby_names). Is 0 if synchronous_standby_names value is invalid or contains *.
sync – collection of synchronous node names from synchronous_standby_names.
sync_confirmed – collection of synchronous node names from synchronous_standby_names that are confirmed to be synchronous according to the pg_stat_replication view. Only nodes that caught up with the SyncHandler._primary_flush_lsn are counted.
active – collection of node names that are streaming and have no restrictions to become synchronous.

_asdict(): Return a new dict which maps field names to their values.

_field_defaults = {}

_fields = ('sync_type', 'numsync', 'sync', 'sync_confirmed', 'active')

classmethod _make(iterable): Make a new _SyncState object from a sequence or iterable

_replace(**kwds): Return a new _SyncState object replacing specified fields with new values

active: CaseInsensitiveSet: Alias for field number 4

numsync: int: Alias for field number 1

sync: CaseInsensitiveSet: Alias for field number 2

sync_confirmed: CaseInsensitiveSet: Alias for field number 3

sync_type: str: Alias for field number 0

patroni.postgresql.sync.parse_sync_standby_names(value: str) → _SSN

Parse postgresql synchronous_standby_names to constituent parts.

Parameters:: value – the value of synchronous_standby_names
Returns:: _SSN object
Raises:: ValueError – if the configuration value can not be parsed

>>> parse_sync_standby_names('').sync_type
'off'

>>> parse_sync_standby_names('FiRsT').sync_type
'priority'

>>> 'first' in parse_sync_standby_names('FiRsT').members
True

>>> set(parse_sync_standby_names('"1"').members)
{'1'}

>>> parse_sync_standby_names(' a , b ').members == {'a', 'b'}
True

>>> parse_sync_standby_names(' a , b ').num
1

>>> parse_sync_standby_names('ANY 4("a",*,b)').has_star
True

>>> parse_sync_standby_names('ANY 4("a",*,b)').num
4

>>> parse_sync_standby_names('1')
Traceback (most recent call last):
    ...
ValueError: Unparsable synchronous_standby_names value

>>> parse_sync_standby_names('a,')
Traceback (most recent call last):
    ...
ValueError: Unparsable synchronous_standby_names value

>>> parse_sync_standby_names('ANY 4("a" b,"c c")')
Traceback (most recent call last):
    ...
ValueError: Unparsable synchronous_standby_names value

>>> parse_sync_standby_names('FIRST 4("a",)')
Traceback (most recent call last):
    ...
ValueError: Unparsable synchronous_standby_names value

>>> parse_sync_standby_names('2 (,)')
Traceback (most recent call last):
    ...
ValueError: Unparsable synchronous_standby_names value

patroni.postgresql.sync.quote_standby_name(value: str) → str

Quote provided value if it is nenecessary.

Parameters:: value – name of a synchronous standby.
Returns:: a quoted value if it is required or the original one.