Changelog

v1.9.0 - 2024-11-26

Features

  • Add support for null (with null string) value representation when defining settings with environment variable.

  • Emit DEBUG log messages when accessing site or distribution configuration files (e.g. postgresql/postgresql.conf template file when configuring PostgreSQL).

  • Add a valid_until field to Role model. This is meant to replace validity eventually.

Bug fixes

  • Really return default settings from site-settings --defaults and no longer read pglift environment variables.

  • Fix failures upon reconfigure of PostgreSQL when requested settings would affect an entry in a file included in the main postgresql.conf.

  • Fix installation through pipx.

  • For PostgreSQL 14 and below, use the database owner as “grantor” (the role used in FOR ROLE clause when defining default privileges) in set-profile command targeting the “public” schema.

Deprecations and Removals

  • The validity field of the role object is deprecated. It is replaced by valid_until.

Documentation

  • Document concurrent operation locking in the command-line interface in a new reference section.

Misc.

  • Declare support for Python versions 3.12 and 3.13.

v1.8.0 - 2024-10-29

Features

  • pg_dump and pg_restore commands are run with -vv when the command database create ... --clone-from is executed with DEBUG log level.

  • Delete PostgreSQL socket directory at site-configure uninstall.

  • Entries can be added/removed to/from pg_hba.conf when creating/updating/dropping roles. This is only available through the Ansible interface by setting the hba_records field.

  • Add allow_group_access setting in postgresql.initdb section.

  • Add support for converting a standalone instance into a member of a Patroni cluster, either through instance alter command (with --patroni-cluster, and possibly --patroni-node options specified) or through the Ansible interface by adding patroni options in the argument spec of a dalibo.pglift.instance task.

  • Add support for upgrading an instance managed by Patroni.

  • Log a warning when trying to set PostgreSQL parameters controlled by Patroni, such as max_connections, as this will have no effect.

  • Add ability to manipulate entries in pg_hba.conf from the command line interface.

  • Add support for creating / dropping replication slots.

  • Render validation errors as JSON in command-line operations when --output-format is set to json.

Bug fixes

  • Allow empty settings.yaml configuration file.

  • Check presence of pg_ctl before adding a PostgreSQL version to list of supported versions by pglift.

  • Fix clone of databases including “large” tables, previously hanging the pg_restore process.

  • Fix missing description in the JSON Schema of Instance and Role manifest models, now correctly returned by pglift <type> --schema command.

  • Properly deactivate systemd unit for Patroni upon instance drop.

  • Fix detection of PostgreSQL port change when altering a Patroni-managed instance.

  • Hide --patroni-etcd-{username,password}, --patroni-restapi-{connect-address,listen}, --patroni-postgresql-replication-ssl-{cert,key,password} and --patroni-postgresql-rewind-ssl-{cert,key,password} options from instance alter command as they actually had no effect since underlying settings are read-only and cannot be modified through pglift.

  • Emit a user-friendly error when trying to restart/reload a Patroni instance and the REST API server is unreachable (often because the instance is not running).

  • Consistently insert a create_replica_methods entry in Patroni YAML configuration for instances with pgBackRest enabled. (Previously, only instances with a valid backup had such an entry.)

  • Reload Patroni when altering an instance even if changes are not related to PostgreSQL parameters.

Deprecations and Removals

  • Mark usage of SETTINGS environment variable as deprecated.

    PGLIFT_CONFIG_DIR environment variable or variables based on settings name should be used instead.

  • Mark the v2 field of patroni etcd settings as deprecated and add version field to replace it.

Misc.

  • Handle creation/deletion of PostgreSQL socket directory during site configuration (pglift site-configure commands) instead of at instance creation.

    On sites where pglift was installed and configure (pglift site-configure install command already executed) but no instance exists yet, running pglift site-configure install again is needed.

  • Log systemctl operations (e.g. enable, disable, start, etc.) at INFO level.

v1.7.0 - 2024-09-10

Features

  • Add concept of profile to give some specific privileges on role for a given database and schema. Currently pglift only supports the read-only and read-write profiles.

  • Add patroni setting (patroni.enforce_config_validation) to enforce configuration validation.

  • Check that system directories (e.g. PostgreSQL log directory) are writable when needed.

  • Add a -x, --exclude-database option to database list to filter out listed databases.

  • Show extensions and schemas in database get table format output.

  • Add the ability to add/remove extensions or schemas to/from a database through the CLI (database alter command).

  • Creation of a primary instance from a pgBackRest backup is now possible. What is required is a valid backup and a stanza configuration for the instance to be created. It can be useful to restore an instance after a disaster, for example if the PGDATA directory is not available anymore.

  • Add a memberships field to Role model. This is meant to replace in_roles eventually.

  • It is now possible to pass extra options to the pg_upgrade underlying command when executing the pglift instance upgrade command.

Bug fixes

  • Allow include directives in postgresql.conf template (only directives referencing an absolute path are allowed).

  • Add check to ensure we do not try to run pg_upgrade to downgrade an instance.

  • Check that a value for --surole-password was provided if the local authentication policy requires a password upon instance creation.

  • Abort early the instance upgrade operation on a Patroni instance as this is not supported.

  • Avoid querying Patroni REST API for cluster members when the instance is stopped, thus resolving some command failures (e.g. pglift pgconf set) on stopped instances.

  • Only reload Patroni upon configuration changes, e.g. through pglift pgconf set command, when the instance is running, thus avoiding such commands to fail previously.

  • Fix read of PostgreSQL collected log files in case the target log destination (e.g. stderr) is not first in log_destination configuration option (thus properly handling situations such as log_destination=syslog,stderr).

  • Avoid prompting for super-user password when it got provided through the command-line at creation of an instance with “password” local authentication method.

  • Fix failing listing of instances when datadir is set with {name} and {version} on the same path segment (or other exotic forms).

  • Prevent errors when trying to delete a partially upgraded instance that has not been initialized.

Deprecations and Removals

  • The in_roles field of the role object is deprecated and will be removed in the next major release. It is replaced by memberships.

  • --jobs option for the instance upgrade command is deprecated. Extra options should be used instead.

Documentation

  • Improve the description of model fields and command-line options accepting multiple values (e.g. database.schemas).

  • Indicate, in the interactive command-line help (--help option), which options can be specified multiple times (e.g. --in-role option of role create command).

  • Add a section for upgrade in operations user guides.

v1.6.0 - 2024-06-21

Features

  • The privileges commands now display privileges independently of the search_path.

  • Forward stdout messages from pg_ctl to our logger when failing to start the instance. This helps understanding issues when starting PostgreSQL when log files are not preserved (e.g. at instance creation).

Bug fixes

  • Resolve misleading creating role 'backup' and updating role 'backup' log messages (INFO level) when creating an instance, only keep the first one.

  • Fix instance backups and instance restore commands on an old instance after upgrade, previously producing a traceback, and now exiting with a more meaningful error message.

  • Preserve PostgreSQL port set from site template along repeated usages of dalibo.pglift.instance Ansible module.

  • Validate the availability of PostgreSQL instance port if the value comes from postgresql.conf site template.

Deprecations and Removals

  • Disallow the port entry in the settings field of dalibo.pglift.instance Ansible module. Support for this entry, in combination with the main port field, was quite fragile and arguably confusing. Users should now consistently use the port field to set PostgreSQL port when using the Ansible module.

v1.5.0 - 2024-06-07

Features

  • Add an auditing capability to the command-line interface.

  • Make it possible to set postgresql.logpath setting to null in order to make the PostgreSQL log directory unmanaged by pglift. Note that when this is done, no logrotate and rsyslog configuration will be installed for PostgreSQL on site.

  • Restore database restore and database dumps commands and make database logical backup work with pg_back used in postgresql.dump_commands site settings.

  • Validate that path-like settings do not contain template variables when none is accepted. Especially, this avoids silently using values of, e.g., postgresql.logpath, if it contains {name} and/or {version}.

Bug fixes

  • Account for instance’s port when explicitly set to the default value (5432) despite the site postgresql.conf template may define another value.

  • Account for port setting possibly defined in site postgresql.conf template when creating Patroni-managed instances (and no port option is specified in the command-line or Ansible module).

  • Set schema owner to database owner if no owner for the schema got specified but one was for the database. Previously, schema owner was set to the current user in that case.

Deprecations and Removals

  • No longer forward log messages from the “filelock” library in the command-line interface (previously only visible in debug mode).

  • Deprecate cli.logpath setting as its purpose was confusing users. The setting value now defaults to null, meaning that a temporary file will be used to store debug information. As before, the file will only be kept if invoked command exits with an unexpected error.

    Site administrators are recommended to adjust their site settings to remove that cli.logpath setting.

Documentation

Misc.

  • Avoid setting a port entry in postgresql.parameters within Patroni configuration file as this parameter is controlled by Patroni (through postgresql.listen).

v1.4.0 - 2024-03-29

Features

  • Add support for “versioned” PostgreSQL template files in site configuration. This allows, for example, the site administrator to install a postgresql/16/postgresql.conf template file in site configuration directory which would be used when creating an instance with PostgreSQL version 16 to initialize the postgresql.conf file. Creating an instance with another version would, on the other hand, use postgresql/postgresql.conf if found in site configuration or fall back to the default template file distributed with pglift.

  • Validate TCP port availability when creating standalone Prometheus postgres_exporter services.

Bug fixes

  • Fix the executable path of pglift in systemd units when pglift is not installed from the binary; the problem occurs from version 1.3.0 as the command-line interface got moved to a dedicated Python package.

  • Properly revert the creation of a database when requested with an unknown extension instead of showing a full traceback and leaving the database over.

  • Avoid blocking Patroni instance deletion if the configuration file backup fails (e.g. in case of network error, while querying the REST API).

  • Only try to initialize instances from a pgbackrest backup if a replica. This fixes instance upgrade command when pgbackrest was configured on site and a backup was available as the command would try to initialize the new cluster from this backup, which was not in expected version.

  • Possibly inject PGLIFT_CONFIG_DIR environment variable in systemd unit templates invoking the pglift executable (this applies to pglift-postgresql@.service and pglift-backup@.service).

Documentation

  • Change the dev setup command to use requirements/dev.txt file.

  • Add a link to Patroni cluster removal documentation page in the WARNING message emitted when backing up the Patroni configuration file when dropping the last node of the cluster.

Misc.

  • Improve log message upon database errors in the command-line interface by only showing the traceback if the --debug is set and displaying the error message as well as the SQLSTATE value (PostgreSQL error code).

  • Improve the user experience when checking for the availability of TCP ports for PostgreSQL or satellite services.

  • Improve the help text of --standby-for option in instance create command.

v1.3.1 - 2024-03-11

Bug fixes

  • Restore the possibility to run pglift --version without site settings being loaded; in particular, this allows the command to be run as root user (despite pglift cannot be used with this user still).

v1.3.0 - 2024-03-05

Features

  • Add a -o,--output DIRECTORY option to database dump command to specify an alternative output directory to write dump file(s) to.

  • Forward PostgreSQL messages to pglift logger (DEBUG level) while performing setup operations on satellite components (configuration, reload, promote).

  • Load site settings from settings.yaml if found in the directory referenced by PGLIFT_CONFIG_DIR environment variable.

  • Possibly drop the new (upgraded) instance if the upgrade fails in instance upgrade command.

Bug fixes

  • Fix an oversight about pgBackRest configuration check before running commands.

  • Remove misleading “failed to read (postgres_exporter|temboard-agent) configuration …” log messages (DEBUG level) during instance creation.

  • Fix logging of pg_restore stderr (at DEBUG level) broken since pglift 1.2.0.

  • Enable (or disable) pglift-pgbackrest systemd service during site-configure install (or uninstall), thus making the service active at user login (or at boot, if the user is lingering).

  • Fix wrong invocation of systemctl status command; the command was mostly used as a check so the impact was not important, but an actual error was visible in DEBUG log messages.

  • Setting temboard.certificate.ca_cert is now optional as it is not actually required to run temboard-agent.

  • Fix a typo in allowed values of temboard.plugins setting.

  • Do not manipulate PostgreSQL objects (databases, roles) when upgrading an instance through instance upgrade. This resolves an issue when trying to upgrade an instance with PoWA enabled from PostgreSQL version 14 to higher which resulted in a failure to upgrade pg_stat_statements extension due to reverse-dependency from pg_stat_kcache.

  • Fix database alter command when the database has extensions installed. The command previously failed with an “invalid” validation error.

Deprecations and Removals

  • Deprecate (undocumented) PGLIFT_CONFIG_PATH environment variable, renamed as PGLIFT_CONFIG_DIR.

Documentation

  • Document PGLIFT_CONFIG_DIR environment variable.

Misc.

  • Log an INFO message when running pgbackrest check.

  • Log the output of systemctl is-enabled command (at DEBUG level).

  • Print both the CLI and the library versions in pglift --version command when packages version is not the same.

  • Extract the command-line interface code into a dedicated Python package. See the installation documentation for details.

  • No longer raise a validation error, but simply warn, upon empty postgresql.versions setting. With this change, a PostgreSQL installation is no longer required on site in order to operate pglift (although most operations will not be available). Beside making local testing easier, this change also enables the configuration of a Prometheus postgres_exporter service on a site where PostgreSQL is not installed, such as a side-car host to the PostgreSQL instance.

  • Merge the code of the Ansible collection in main pglift repository.

v1.2.0 - 2023-12-22

Features

  • Add an instance shell command to start a new shell with instance’s environment, as a shorthand for pglift instance exec or export $(pglift instance env).

  • Add a check option to site-configure command.

Bug fixes

  • Catch “database errors” (psycopg.errors.DatabaseError) globally in the CLI, thus preventing a traceback to be displayed.

  • Handle possible absence of “backup set size” in the output of pgbackrest info when listing instance’s backups as this field is not set for block incremental backup. Accordingly, key repo_size in instance backups command output will be null.

  • Fix log messages about Prometheus postgres_exporter service configuration (previously erroneously referring to PostgreSQL instead of “postgres_exporter”).

  • Improve error handling when parsing (possibly modified) Prometheus postgres_exporter configuration file by returning a user error instead of an internal error (traceback).

  • Start Prometheus postgres_exporter with a --web.listen-address argument instead of relying on the PG_EXPORTER_WEB_LISTEN_ADDRESS environment variable that is no longer used by postgres_exporter from version 0.12. The latter variable is no longer written to postgres_exporter configuration file for newly configured instances, but will still be read for existing instances.

    The compatibility is handled when starting postgres_exporter directly but not when using systemd as service manager. In the latter case, the configuration file for all existing postgres_exporter services should be adjusted (if using postgres_exporter version 0.12 or higher) by changing:

    PG_EXPORTER_WEB_LISTEN_ADDRESS=:9187
POSTGRES_EXPORTER_OPTS='--log.level=info ...'

    into

    POSTGRES_EXPORTER_OPTS='--log.level=info --web.listen-address=:9187 ...'

  • Make site-configure install (resp. site-configure uninstall) commands more idempotent by properly handling existing (resp. non-existing) files or directories. This implies that, upon “re-configuration”, existing files (possibly modified by the user) will not be overridden.

  • Only invoke systemctl daemon-reload if some unit files actually got installed in site-configure install.

  • Properly account for shared_preload_libraries option defined in postgresql.conf site template when building the configuration of an instance with PoWA enabled.

Documentation

  • Mention in “Instance environment” how-to that satellite components also get their environment variables exported.

Misc.

  • The binary built by PyOxidizer is no longer standalone and now needs shared libraries for extension modules in the lib/ directory alongside the pglift binary. The package published by GitLab CI now contains a .tar.gz archive containing all needed files.

  • Use pg_ctl start (instead of postgres previously) to temporarily start PostgreSQL at instance creation.

  • Upgrade Pydantic dependency to version 2.

  • Make the project conform to the REUSE recommendations about licensing and copyright.

v1.1.0 - 2023-11-13

Features

  • When dumping a database, through database dump command, we now forward the role password (that would possibly be prompted for upon database existence check) to dump commands (such as pg_dump or pg_back).

  • Limit pg_ctl status invocations in most operations for better performance.

  • Return the runtime status of Patroni API as part of pglift instance status result.

Bug fixes

  • Remove gss from local authentication methods, as it’s only available for TCP/IP connections.

  • Fix bug preventing instance creation when setting a custom surole name.

  • Retrieve super-user role’s password from environment or password_command setting when dumping a database.

  • Improve previously misleading errors and tracebacks when something got wrong at instance initialization by not showing unrelated errors (fixing a bad programming pattern).

  • Handle possibly absence of PostgreSQL log file during Patroni bootstrap “retry” logic, resolving a crash possibly due to a race condition.

  • Log an INFO message when the PostgreSQL instance has been successfully created by Patroni.

  • Improve log message about “Patroni log file” not being found during bootstrap to make it less misleading by indicating that this is transient (by nature of the bootstrap) and eventually logging a successful message.

  • Log a DEBUG message when checking for Patroni “readiness” during bootstrap.

Documentation

  • Remove “pgbackrest” and “Prometheus postgres_exporter” pages from the Python API section, as they fail to build with up-to-date dependencies.

  • Mention upfront in the installation documentation that Python 3.9 (or higher) is required.

Misc.

  • Build the pglift binary using latest PyOxidizer version (0.24 or higher).

  • Use Python 3.10 to build the binary.

v1.0.0 - 2023-10-17

Features

  • The owner of a schema can now be specified.

  • Log the target database name when creating an extension.

Bug fixes

  • Avoid reconfiguring pgBackRest upon PostgreSQL configuration changes when it’s not needed but only check if respective changes would need a reconfiguration of this service (e.g. the socket path).

  • Only invoke pgbackrest stanza-create upon instance creation, not when modifying it.

  • Avoid reconfiguring Prometheus postgres_exporter upon PostgreSQL configuration changes when it’s not needed but only check if respective changes would need a reconfiguration of this service (e.g. the socket path).

  • Avoid reconfiguring temBoard agent upon PostgreSQL configuration changes when it’s not needed but only check if respective changes would need a reconfiguration of this service (e.g. the port).

Deprecations and Removals

  • In the declarative API (Ansible), the clone_from field of database object, deprecated in previous release, got removed.

  • The passfile and use_pg_rewind settings under patroni section, deprecated in previous release, are removed; use eponymous fields under the patroni.postgresql section.

  • In the declarative API (Ansible), the patroni.postgresql_connect_host field of instance object, deprecated in previous release, got removed.

  • Set the default value of prometheus.queriespath site setting to null, following its deprecation in version 0.38.0. As a consequence, the “queries” file will no longer be installed at instance creation.

Documentation

  • Document how to configure the managed node with Ansible in the Ansible tutorial.

  • The documentation has been reviewed overall, fixing examples (previously invalid due to “recent” changes), adjusting incomplete instructions, clarifying things here and there.

Misc.

  • Set the development status to “Production/Stable”.

v0.40.0 - 2023-10-03

Features

  • Allow to clone a database by restoring only the schema (data definitions) through --clone-schema-only command-line flag or the equivalent declarative API.

  • Add support for database PUBLICATION and SUBSCRIPTION objects through the declarative API.

  • Allow to configure hostssl authentication method at instance creation.

  • Add support for ctl.{keyfile,certfile}} for Patroni in site settings. This is now required after a breaking change in version 3.1.0 of Patroni.

  • Add support for basic-authentication (username/password) to etcd from instances managed by Patroni.

  • Client connection options for the replication and rewind users of Patroni-managed instances can be specified through --patroni-postgresql-{replication|rewind}-ssl-{cert,key,password} options when creating an instance (or similar fields in the declarative API) along with patroni.postgresql.connection.ssl.{mode,crl,crldir,rootcert} site settings.

  • Stream PostgreSQL log messages to our logger (at DEBUG level) during Patroni bootstrap.

  • Honour postgresql.waldir setting when deploying Patroni instances; also, when pgBackRest is used, and a backup is available to create a replica from, pgbackrest restore is now invoked with --link-map pg_wal=....

  • Add support for passwordless SSH connection for pgbackrest remote repository mode.

  • Add support for PostgreSQL version 16.

Bug fixes

  • Disallow extra fields, previously ignored, in interface models such as patroni.restapi or postgresql.auth fields.

  • Remove the invalid cert value for --auth-local or --auth-host options at instance creation as it only applies to hostssl.

  • Run local pgbackrest server with the PGPASSFILE environment variable so that connections made by pgbackrest (through the libpq) can use the passfile when it’s not in the default location.

  • In Patroni REST API settings, fix the validator of verify_client to only require that certfile is set when the former is.

Deprecations and Removals

  • In the declarative API (Ansible), the clone_from field of database object is deprecated; instead the new clone field (especially its dsn option) should be used. E.g. in JSON, replace "clone_from": "<your connection string>" by "clone": {"dsn": "<your connection string>"}.

  • Remove the clone_from field in database get return value, as it was always null (not preserved from user input).

  • Commands database dumps and database restore, deprecated in version 0.38.0, are removed.

  • In the declarative API (the instance Ansible module), the patroni.postgresql_connect_host field is deprecated. Instead patroni.postgresql.connect_host can be used for the same purpose.

  • Add a new patroni.postgresql setting field, holding passfile and use_pg_rewind fields, previously under the top-level patroni key.

  • A mode option (with value in ['path', 'host-tls', 'host-ssh']) now needs to be explicitly provided for pgbackrest.repository in site settings. This is a BREAKING CHANGE for which installed site-settings will need an update.

  • Remove support for PostgreSQL version 11.

Documentation

  • Improve Patroni settings descriptions, especially concerning TLS certificates.

  • Add a “how to” perform major online upgrade of a database through Ansible.

  • Clarify and extend security notes about etcd and Patroni.

  • Add missing entry in 0.38.0 changelog about the deprecation of database dumps|restore commands.

Misc.

  • Run functional tests under Debian bookworm in CI.

  • Use pg_dump --format=custom and pg_restore (instead of plain psql) when cloning a database.

  • In tests, run etcd with HTTPS and let Patroni verify server certificates.

v0.39.0 - 2023-08-25

Bug fixes

  • Forbid extra (unknown) keys in site settings by issuing a validation error instead of silently ignoring them previously.

  • Use WantedBy=default.target in systemd units instead of multi-user.target, which is not generally available in user mode. This makes user services starts properly at boot.

Deprecations and Removals

  • The default value for pgbackrest.repository.path got removed; this setting needs an explicit value.

Documentation

  • Mention how to install pglift with pipx.

v0.38.0 - 2023-08-03

Features

  • Add a confirmation to pglift database run to warn about the databases that will be affected by the sql command.

  • Add ability to provide a .psqlrc template as file postgresql/psqlrc in site configuration.

  • Setting replrole: null (or not providing it) disables the creation of the replication role.

Bug fixes

  • No longer create pgbackrest’s lock-path directory during site-configure but let pgbackrest handle this itself. This makes the configure remain valid upon reboot, by not requiring this directory to be present whereas it might have been removed if set to a volatile system like /run.

Deprecations and Removals

  • Setting prometheus.queriespath is deprecated and will be removed in the next release.

    This follows from the deprecation of extend.query-path option in postgres_exporter 0.13. In the future, support for a dedicated sql_exporter will be added to provide equivalent features.

  • The default value for replrole is now null. Users relying on this role for replication now have to provide it explicitly in the settings. Also entries for replication are not part of the default pg_hba.conf file anymore. Administrators may have to provide a template for this.

  • Due to their fragile implementation, especially when custom commands are defined in site settings, database dumps and database restore commands are deprecated and will be removed in a future release.

Misc.

v0.37.0 - 2023-07-18

Features

  • Validate postgresql.bindir setting to make sure it contains the {version} template placeholder.

  • Validate that postgresql.default_version setting has a value within postgresql.versions.

  • If setting postgresql.default_version is undefined, guess the default version from the latest PostgreSQL version available as defined in postgresql.versions setting.

  • Check pgBackRest configuration upon instance promotion.

  • Skip the check of pgBackRest configuration for standby instances on sites using the repository.path mode for pgbackrest. A warning is emitted instead, but this should unblock the creation of standby instances in this mode.

  • Validate that postgresql.versions setting is a non-empty list, possibly after having inferred values from bindir template.

  • Validate that path-like settings only contain expected template variables: e.g, a validation error will be raised if a settings field contains {version} or {name} placeholders whereas none is allowed.

Bug fixes

  • Also add a password file entry for the backup role upon standby instances creation.

  • Consider only the first item of unix_socket_directories PostgreSQL setting to determine the host part of libpq connection string.

  • No longer return PGHOST=localhost in instance env command when no unix_socket_directories configuration entry is defined in order to let PostgreSQL use the default value.

  • Set the default answer to No in prompt asking for deletion of pgBackRest backups upon instance drop.

Removals

  • The “default version” is no longer guessed from pg_config executable available in $PATH; only site settings are used from now on.

Documentation

  • Fix first item of restore_commands example with pg_back missing the {conninfo}.

v0.36.1 - 2023-06-20

Misc.

  • Switch to hatch build system.

v0.36.0 - 2023-06-15

Features

  • Check installation before performing any operation: when the site-configure install has not been run or the installation is broken, any operational command will fail fast suggesting to perform a proper installation while installation issues are logged at ERROR level.

  • Improve the command-line interface with respect to the -i/--instance option. The option is no longer required to display the help of a subcommand (e.g. pglift database create --help). Also, a more accurate error message is displayed when no instance is found or when several ones are found.

  • Validate postgresql.{dump,restore}_commands settings to that the programs used by each command exist. This only applies to commands using non-PostgreSQL binaries (e.g. pg_dump) as these are typically defined relative to instance’s binary directory (e.g. {bindir}/pg_dump).

Bug fixes

  • Report failure to start a child process (e.g. the database dump command) with a user error instead of throwing a traceback.

  • Command pglift instance exec INSTANCE COMMAND... now exits with status 2 when no command got specified.

  • Fix instance exec command to make it clear that the INSTANCE argument is not optional.

Documentation

  • Document the need for -- in instance exec command.

v0.35.0 - 2023-05-17

Bug fixes

  • Implicitly convert None value to the default value for patroni.node and patroni.restapi fields when using Ansible modules.

v0.34.0 - 2023-04-21

Features

  • The instance status command has been extended to return the status of all satellite components. It will exit with code 3 if any service is not running. The prometheus and temBoard agent statuses have been implemented in addition to the existing PostgreSQL status.

  • When creating a standby instance, if a pgbackrest backup for target stanza exists, it will be used instead of pg_basebackup.

  • Upon deletion of an instance, do not prompt for possible deletion of its pgbackrest stanza when another instance is using it.

  • Upon deletion of an instance, delete its pgbackrest configuration even if stanza deletion was not confirmed.

  • Remove log, spool and lock paths for pgbackrest upon site-configure uninstall.

  • Remove, after confirmation, the backup directory for pgbackrest upon site-configure uninstall.

  • Add a default value for pgbackrest.repository.path setting with value $prefix/pgbackrest.

Bug fixes

  • Fix deletion of pgbackrest include directory upon site-configure uninstall: the command previously emitted a warning and the directory was left empty; now it is correctly removed.

Removals

  • The {version} template variable is once again required in postgresql.datadir and postgresql.waldir settings.

  • Option --pgbackrest-restore-stanza got removed as it is confusing now that --pgbackrest-stanza option is required.

Misc.

  • Set project’s development status to beta.

v0.33.0 - 2023-04-14

Features

  • Attributes CREATEROLE and CREATEDB can now be set when creating or altering roles.

  • The version of an extension can now be specified.

  • The temBoard logging can be configured via site settings with logpath, logmethod and loglevel. By default logmethod is set to stderr. If file is selected, a logfile for each instance will be created in the logpath folder named temboard_agent_{qualname}.log. loglevel can be set to DEBUG, INFO, WARNING, ERROR or CRITICAL.

  • The option --pgbackrest-restore-stanza is now taken into account when using Patroni. Using this option will try to provision new standby from pgbackrest backups using create_replica_methods

Bug fixes

  • Do not fail upon socket creation error while checking for port availability; emit a DEBUG log message instead in that case.

  • Let the user-defined port take precedence over what’s defined in postgresql.conf site template.

Removals

  • Change the completion command into a --completion=SHELL option to pglift. This is now implemented as an eager callback which does not load site settings or any user data and can thus be safely used by any user (e.g. root).

  • After being marked as required extension schema field is optional again.

v0.32.0 - 2023-03-29

Features

  • Add the logpath setting within PostgreSQL settings section.

    This new field allows to determine the directory containing log for our instances.

    The postgresql.conf template distributed with pglift now sets log_directory based on this field, along with a log_filename value that includes the instance qualified name (i.e. <version>-<name>) as a prefix.

  • Add cli.log_format and cli.date_format settings to control the format of log messages when writing to a file from the command-line interface.

  • Add a --defaults/--no-defaults options to site-settings command to control whether default settings values should be shown, possibly along with site configuration.

  • Add support for handling database schemas.

  • Allow postgresql.datadir and postgresql.waldir settings to not contain the {version} template variable; only the {name} is required by now.

  • Allow to set already encrypted password to a Role using --encrypted-password instead of --password.

  • A new rsyslog configuration option has been added to generate rsyslog config when running pglift site-configure install.

  • Logrotate configuration is now handled at site-configure step and no longer when creating/dropping an instance. The logorate configuration is now shared among the PostgreSQL instances and satellites components.

  • The required ca_cert field has been added to the temBoard settings, it’s part of the certificate field and must be defined as temboard.certificate.ca_cert. This makes the use of ssl more consistent. It’s used in the temBoard agent configuration file.

Bug fixes

  • Fix crash upon early pglift command invocation when the creation of (CLI) log directory fails.

  • Avoid starting a stopped instance when no role or database changes are needed.

  • Do not override environment from parent process in instance exec.

  • Fix logrotate configuration file for Patroni, which was missing templating.

  • Patroni postgresql.pgpass configuration item is now configurable with patroni.passfile site setting and defaults to etc/patroni/<instance qualname>.pgpass. This passfile is deleted when instance is deleted.

  • Fix a validation error when patroni watchdog device setting was not a file but a character device.

Removals

  • Remove possibility to template logpath setting for Patroni

    We remove the placeholder {name} from default value for patroni logpath setting. Using the {name} within the patroni logpath is no longer supported, we now always append the instance name at the end of the logpath.

  • Extension schema field is now required. As a consequence, it’s not possible to provide a list of extensions to install upon database creation in the CLI.

  • Extensions now have a “state” field. To drop an extension from a database users now have to explicitly use “state: absent”.

  • log_directory for PostgreSQL is no longer created (automatically) by pglift.

    We remove the portion of code parsing the postgresql.conf and creating the corresponding log_directory. User should make sure the log_directory is present when they change this setting on postgresql.conf.

  • In pgBackRest settings, ca_cert is now a part of certificate field. pgbackrest.repository.ca_cert should now be defined as pgbackrest.repository.certificate.ca_cert.

  • Pglift usage as root user is now prevented.

    According to PostgreSQL documentation, initdb or pg_ctl commands cannot be run as root.

Documentation

  • The documentation explaining the steps to configure the site when using systemd in system mode has been changed to avoid calling pglift commands with sudo.

Misc.

  • Move command-line specific settings (lock_file and logpath) to a new cli field.

v0.31.0 - 2023-02-28

Features

  • A new logrotate service has been added to generate logrotate configuration file for each instance.

  • The passfile site setting, under postgresql.auth section now accepts a null value in order to completely disable support for the password file. When disabled, --pgpass option to role commands are no longer available.

  • Validate existence of bindir fields set in postgresql.versions setting.

Bug fixes

  • Define the cluster_name in postgresql.conf template file, thus allowing to create instances without the value (which used to be hard-coded from instance name).

Removals

  • The pgpass field in roles items for an Instance is no longer supported (in Ansible or the declarative API). The field can still be specified on Role objects.

  • To enable SSL in PostgreSQL configuration file, in addition to setting ssl to true, providing ssl_cert_file and ssl_key_file is required. The previous self-signed certificate is no longer generated.

  • No longer output the pgpass field when listing roles.

Documentation

  • The commands for exporting the Ansible doc fragments have been simplified for the release workflow. Now it is only a copy of the data files already generated for the tests.

  • Add a note about the ability for devs to run systemd jobs on sourcehut.

  • Document sudo pre-requisites for systemd “system” mode with a sudoers entry example.

v0.30.0 - 2023-02-06

Features

  • Make it possible to specify the schema in which a database extension would be installed.

    Until now, when an extension was added to a database, the extension’s objects were installed by default on the current schema of the database (usually public schema).

    Now, the name of the schema in which to install the extension’s objects can be specified when adding or altering extensions, by specifying it in the manifest.

  • Add a -f/--follow option to instance logs command to follow log output and log file change.

  • Log create/alter/delete operations on database extensions.

  • Add support for TLS encryption settings for patroni REST API.

  • Log messages from pgBackRest commands: pgbackrest commands are now invoked with --log-level-stderr=info and respective messages are forwarded to pglift’s logger at DEBUG level (as are all stderr messages from subprocesses).

  • Configure pgBackRest on standby instances, even in repository.path mode, removing a previous limitation from the implementation.

    In addition, when calling instance backup <instance> with <instance> being a standby, pgbackrest is now invoked with --backup-standby option.

  • Setup pgbackrest on standby instances when using a remote repository.

  • Add support for TLS for Etcd for HA with Patroni via site-settings.

  • Make host_port and host_config item of pgbackrest’s repo-host settings optional.

  • Separate server from client pgbackrest configuration in remote repository mode.

  • temboard-agent V8 is now needed for pglift, older versions are no longer supported.

Bug fixes

  • Do not write the port value in postgresql.conf if it has the default value.

  • If any change in the configuration files is detected for prometheus or temboard, we now perform a restart of the services for the changes to take effect.

  • Fix possibly not working Exec command in postgresql systemd unit file.

Removals

  • If pgbackrest is enabled, the stanza name must now be provided upon instance creation.

  • Temboard-agent SSL files are no longer auto-generated, their path must be provided in site-settings. The certificate field containing cert and key is required in temboard section.

  • CLI option --extension of database alter command has been removed.

  • CLI option --in-role of role alter command has been removed.

  • Patroni etcd host setting has been replaced by hosts.

  • Configuration for etcd for HA with patroni is now managed in site settings.

    Etcd host can no longer be provided by user when creating an instance.

Documentation

  • Update the Ansible tutorial to refer to the collection and simplify installation steps.

  • Warn about the prerequisites for using systemctl --user.

  • Document patroni etcd hosts setting usage

  • Document Patroni security (TLS support)

  • Recommend to use systemd as a service manager when operating with pgBackRest in remote repository mode.

Misc.

  • Add --pg1-path option to pgbackrest archive-push command set in PostgreSQL archive_command.

v0.29.0 - 2022-12-30

Features

  • Improve warning message when failing to connect to primary instance in instance get.

  • Make replication role a member of pg_read_all_stats.

  • Add WAL sender state (from pg_stat_replication view) to standby information (as available in instance get command).

  • Export paths to PostgreSQL data and WAL directories when getting an Instance (e.g. through instance get -o json command).

  • Introduce $PGLIFT_CONFIG_PATH environment variable.

    This new variable allows users to provide a path to site configuration files to be taken into account prior to $XDG_CONFIG_HOME/pglift or /etc/pglift.

  • Preserve user edits of Patroni configuration file.

  • Add support for pgbackrest remote host repository.

Bug fixes

  • Catch JSON decode exception when parsing SETTINGS environment variable.

    This prevents showing a traceback when the json provided for SETTINGS environment variable is invalid.

  • Catch SettingsError when loading site settings in CLI.

    Prevents displaying a traceback if there’s an error when parsing the site settings YAML file.

  • Fix path to pglift in systemd service when using pre-built binary

    ExecPath in pglift-postgresql@ systemd service which was wrongly set to an inexistent path.

Removals

  • Require pgbackrest>=2.41

    pglift instance backups now runs pgbackrest info --set=<backup set> --output=json which only works since pgbackrest 2.41.

  • Hide standby.status field from instance get output: this field is not very useful since it will only appear on standby instances, which are by definition in demoted state.

  • Change priority order of site config files. Order is now xdg > etc > dist.

  • Improve instance privileges command help message

  • Drop archive-push section in global pgbackrest configuration.

  • Replace pgbackrest.repopath setting by pgbackrest.repository, now an object with keys path and retention (see pglift site-settings --schema for details). The path field is now required and has no default value, in contrast with repopath previously.

  • The site-settings command output format is now YAML by default.

Documentation

  • Add a section in docs for site configuration templates.

  • Explain how base pgBackRest configuration is installed, and how to override it.

  • Improve and clarify documentation about systemd in system mode.

Misc.

  • Use pgbackrest’s recovery-option when restoring a standby from a backup.

  • Only restart PostgreSQL upon configuration changes, not all satellite services.

v0.28.0 - 2022-12-02

Features

  • If pgbackrest is enabled, log install and uninstall operations at site-configure.

  • Configure systemd timer for instance backup with a randomized delay.

  • Add a --dry-run option to apply commands.

  • Add support for “force” option for database drop.

  • Improve logging when starting/stopping Prometheus postgres_exporter and temboard-agent.

  • Allow to pass any command to instance exec (not just Postgres commands or absolute ones as previously).

  • Make it possible to operate normal instances even when patroni is enabled in site settings.

  • Add support for PostgreSQL 15.

  • Make check for port availability more robust.

  • Improve systemd unit template for PostgreSQL. It is now defined as a Type=notify service and does not use a PIDFile anymore, following more closely what’s suggested in PostgreSQL documentation.

Bug fixes

  • pglift 0.27.0 is now the minimum required version for the Ansible collection.

  • Fixed error during enabling/disabling temboard service with systemd caused by a bad service name.

  • Fix error in instance env command for a standby instance with pgbackrest enabled.

  • Only start Patroni once at instance creation (avoid a stop and a start). This should make concurrent setups (e.g. from Ansible targeting different hosts in parallel) work without dead-locking Patroni.

  • Avoid starting / stopping PostgreSQL many times at instance creation.

Removals

  • The Ansible collection got moved to its own repository.

  • Avoid useless pgbackrest start invocation after stanza creation.

  • Separate management of shared_preload_libraries and database extensions.

    The extensions key in instance’s model has been dropped. To install extensions in an instance, you now need to provide the shared_preload_libraries in instance settings.

  • No longer error out, but simply warn, upon invalid Patroni configuration as reported by patroni --validate-config.

  • Only validate generated Patroni configuration for Patroni version higher than 2.1.5.

Documentation

  • Extend how to about standby management with Ansible to illustrate promote operation.

  • Add some details about site configuration in installation documentation.

Misc.

  • Add a hidden --debug command-line flag to set log level to DEBUG and eventually get tracebacks displayed.

  • Unconditionally call pgbackrest stanza-create upon instance. re-configuration whereas this was previously only done at instance creation. Conversely, the --no-online option is used to avoid superfluous instance startup. On the other hand the pgbackrest check command is still only emitted at instance creation.

  • Add --output=json option to postgres_exporter apply command.

  • Rework systemd installation through site-configure hook.

  • Use pglift CLI in systemd unit for PostgreSQL.

  • Use towncrier to manage news fragments.

v0.27.0 - 2022-11-02

Features

  • Support for RockyLinux 9

  • Ability to provide a name for pgbackrest stanza

  • Handling of REASSIGN OWNED and DROP OWNED when dropping a role

  • Better handling of model validation errors in the CLI

  • Ability to create a database as a clone of an existing one

  • JSON output to instance env command

  • JSON output to apply sub-commands

  • Prometheus password change upon instance alter

  • Prometheus password kept upon instance upgrade

  • Raise a specific error if role being dropped has dependent database objects

  • Raise a specific error when Postgres binary directory for requested version does not exist

Bug fixes

  • SETTINGS environment variable takes precedence over YAML setting file

  • Fix systemd service name for Patroni-managed instances

  • Fix service name inconsistency for temboard-agent

  • Entries of postgresql.conf, set by initdb, no longer commented

  • Fix a type error when retrieve instance environment from Ansible module

  • Replication password passed through environment when invoking pg_basebackup

Removals

  • Field pgbackrest_restore excluded from instance get command output

  • Database auto discover in default postgres_exporter configuration

  • CLI option --json, replaced by --output-format=json

  • Instance model’s configuration, renamed as settings, to be consistent with eponymous field on Database objects

  • Standby’s for field renamed as primary_conninfo in the declarative API

Documentation

  • Added an example playbook for a standby instance

  • Fix settings in Ansible tutorial (pgpass fields missing for surole and backuprole)

Misc.

  • Limit database connection openings in instance get

  • Installation of global pgbackrest configuration through site-configure command

  • Setting postgresql.versions now defined as a list

  • Use pglift CLI in Ansible modules, instead of the Python API

  • PyOxidizer configuration to build a binary version of pglift