ESS LTS 24.10 Change Logs and Upgrade Notes

Upgrade Notes for the 24.10 LTS

If you plan on upgrading to this LTS we always recommend upgrading to the latest patch version of your current LTS and then updating to the latest version of this LTS.

If you plan on updating, we recommend installing the latest patch version.

Whether upgrading or updating, you should be aware of all significant upgrade notes from each prior patch version. Any highlighted patch notes for this specific LTS have been collated for convenience below, you can find the full changelogs of each release thereafter.

24.10.01-gui

The required Python versions are now 3.10, 3.11, 3.12.

As a result, Ubuntu 24.04 is now supported but Ubuntu 20.04 support is dropped. Please consult the Ubuntu documentation for upgrading between Ubuntu LTS versions.

The installer will attempt to install the required packages in some scenarios.

Airgapped customers should ensure that Python 3.12 packages are available in their package mirrors.

Alternatively, Python 3.10, 3.11, or 3.12 can be preinstalled on the server in all situations.

24.10.01-gui

Release Summary

The required Python versions are now 3.10, 3.11, 3.12. As a result, Ubuntu 24.04 is now supported but Ubuntu 20.04 support is dropped. Please consult the Ubuntu documentation for upgrading between Ubuntu LTS versions. The installer will attempt to install the required packages in some scenarios. Airgapped customers should ensure that Python 3.12 packages are available in their package mirrors. Alternatively, Python 3.10, 3.11, or 3.12 can be preinstalled on the server in all situations.

New Features

Enterprise	XMPP Bridge and IRC Bridge both support Authenticated Medias. Their signing key is generated automatically by the installer UI.
Enterprise / Starter	Authenticated Media is now enforced by default. All components but Matrix Content Scanner are compatible with it. If you need to disable it, please add `enable_authenticated_media: false` to Synapse -> Additional YAML.
Enterprise / Starter	Add the possibility to allow/deny rooms and log events for Auditbot.
Enterprise / Starter	Support overriding just the server and path in the image digest ConfigMap.
Enterprise / Starter	Support Element Call in Element X.
Enterprise / Starter	Matrix Authentication Service and Synapse only use internal paths to communicate, removing the need for `hostAliases` setup between the two.
Enterprise	All ESS Images are now hosted behind `registry.element.io`.
Enterprise	Synapse workers supporting multiple replicas can now be configured for automatic horizontal scaling.
Enterprise / Starter	Expose `images_digests.yml` file in the Download screen for Airgapped customers who want to sync their registry directly with `registry.element.io`.

Upgrade Notes

Enterprise / Starter	Upgrade to cert-manager 1.15.3.
Enterprise / Starter	Operator - Upgrade Python to 3.12, Ansible to 2.17.
Enterprise / Starter	Upgrade Synapse to v1.116.0.
Enterprise / Starter	Upgrade Element Web to v1.11.82.
Enterprise	Update XMPP Bridge to 2.0.1.
Enterprise	Update Adminbot and Auditbot to 6.3.1.
Enterprise	Update IRC Bridge to 3.0.2.
Enterprise	Update Hydrogen to 0.5.0.
Enterprise / Starter	Update Admin Console to v16.105.4.
Enterprise / Starter	Upgrade microk8s to 1.31. As per 24.10 releases, the standalone installer only supports upgrading microk8s installed from 23.10 releases. As per 23.10.35/24.04.05/24.05.01, the standalone installer now upgrades microk8s automatically. The microk8s upgrade procedure does not involve an uninstall/reinstall of microk8s anymore. It now will automatically upgrade microk8s to the expected version, and as such, the `--upgrade-cluster` flag has been removed. Any customization to CNI Configuration in `/var/snap/microk8s/current/args/cni-network/cni.yaml` will have to be reconfigured. During the upgrade, microk8s & workloads will restart several times. Managed addons that require upgrading will be temporarily disabled to be upgraded. This all will induce a small downtime of a couple of minutes.
Enterprise / Starter	The installer now makes sure the upgrade comes from a supported version.

Security Issues

Enterprise / Starter

Upgrade to Ansible 9 for security fixes and Python compatibility.

Bug Fixes

Enterprise	Allow only one VoIP platform (Jitsi or Element Call) to be enabled.
Enterprise	Fix migration of authentication settings from <24.07.01 with Matrix Authentication Service installed.
Enterprise / Starter	Fix an issue where, after update, the installer UI would ask to save for changes on the Host screen when the user actually did not click anything.
Enterprise	Fix monitoring integration tab not rendering.
Enterprise	Fix Auditbot logs viewer when Matrix Authentication Service is setup.

Deprecations

Starter

Matrix Content Scanner is not available anymore in Starter Edition.

Non-LTS Monthly Release Changes

This section summarises all the changes between the previous LTS and this one during the monthly non-LTS releases. Duplicate entries where individual components received upgrades have been removed so only the latest version is mentioned.

You can then compare the below changelog against the above LTS releases for an accurate overall changelog if upgrading from a previous LTS.

Some changes added to non-LTS monthly releases are backported into older LTS releases if required. As such, some of the below features may already be present in a previous LTS. You can check the associated LTS books' respective changelog page to compare.

Release Summary

The required Python versions are now 3.9, 3.10, 3.11. These are available on all supported OS distributions. The installer will attempt to install the required packages in some scenarios. Airgapped customers should ensure that Python 3.9 packages are available in their package mirrors. Alternatively, Python 3.9, 3.10, or 3.11 can be preinstalled on the server in all situations.

Enterprise	This release adds the possibility to enable Matrix Authentication Service during initial setup. Enabling Matrix Authentication Service is experimental; a couple of features do not work yet with it (Auditbot, Adminbot, Element Call, GroupSync, Admin UI). Enabling MAS allows you to use Element X with OIDC or LDAP login.
Enterprise	This release now makes ESS ElementX ready by default. Any new installation will deploy Matrix Authentication Service. Existing setups will not profit from this change, migration paths are planned later in the future.

New Features

General	Support knocking with `generic_worker` federation.
Enterprise / Starter	Major Change: The standalone installer now upgrades microk8s gracefully and automatically. The microk8s upgrade procedure no longer involves an uninstall/reinstall of microk8s. It now automatically upgrades microk8s to the expected version, and the `--upgrade-cluster` flag has been removed. Any customization to CNI Configuration in `/var/snap/microk8s/current/args/cni-network/cni.yaml` will need to be reconfigured. During the upgrade, microk8s will restart, and addons will be disabled to force an upgrade. This process may induce a small downtime of a couple of minutes.
Enterprise	Status watchers are now golang containers, reducing resources used by the operator and updater.
Enterprise	Allow configuration of Synapse database connection pool sizes.
Enterprise	Add a ServiceMonitor to scrape metrics of microk8s ingress.
Enterprise	Expose Operator & Updater metrics.
Enterprise	Add support for Outbound webhooks in Hookshot.
Enterprise	Synapse OIDC support attribute requirements.
Enterprise	Add a new experimental feature to enable Matrix Authentication Service during ESS bootstrap.
Enterprise	Simplification of the OIDC provider configuration. After upgrading, please make sure that your OIDC settings were properly migrated to the new view.
Enterprise	It is now possible to enable the new Matrix Authentication Service when bootstrapping a new ESS setup. It is an experimental feature, incompatible with Groupsync, Element Call, Auditbot, and Adminbot at this time. It is required to try out Element X with OIDC login.
Enterprise	It is now possible to use LDAP with Matrix Authentication Service.
Enterprise / Starter	Properly enforce patterns check in UI inputs under cards that can be enabled/disabled.
Enterprise	Display deployment availability in the UI, in addition to the reconciliation status.
Enterprise	Element Call is now MAS-Compatible.
Enterprise	Add the possibility to configure a matrix stats endpoint.
Enterprise	Setup the onprem-admin user as a MAS admin.
Enterprise	Allow configuration of empty (no) disallowed IP ranges in Hookshot.
Enterprise	Validate Synapse Telemetry is consistently set.
Enterprise / Starter	Synapse improve worker configuration.
Enterprise / Starter	Allow blocking of non-scanned media.
Enterprise	Adminbot/Auditbot + MAS compatibility.
Enterprise / Starter	The UI now properly marks secrets as required when necessary.
Enterprise / Starter	The reconciliation process now ensures that all secrets are present and shows missing secrets if necessary.
Enterprise	Add Hookshot permissions configuration.
Enterprise	Add the possibility to manage Federation dynamically from the Admin Console when Secure Border Gateway is enabled.
Enterprise / Starter	Speed up initial Synapse deployment.
Enterprise	Add the possibility to configure user deprovisioning and room cleanup in GroupSync.
Enterprise	Synapse auto invite: use Synapse native feature, run on background worker if it exists.
Enterprise	Allow to override a container image without configuring a new digest.
Enterprise / Starter	Support MSC4186 / Simplified Sliding Sync natively in Synapse.
Enterprise / Starter	Support authenticated media APIs (MSC3916) in Synapse.
Enterprise / Starter	Scrape Synapse HAProxy metrics.
Enterprise	Scrape Adminbot and Auditbot HAProxy metrics.
Enterprise	Set default volume sizes for Matrix Content Scanner volumes.
Enterprise	Set default volume sizes for Adminbot, Auditbot & Sydent volumes.
Enterprise / Starter	The administration interface can now manage users on deployments using Matrix OIDC.
Enterprise	Administrators can now configure the SBG allowlist within the Admin UI.
Enterprise / Starter	The user management page now allows admins to toggle the locked status of users.
Enterprise / Starter	The user management page now displays the primary email address of users.
Enterprise / Starter	The user management page will now default to showing locked and deactivated users when searching by name.
Enterprise	Enabling MAS is not experimental anymore, and is now the default setup mode.
Enterprise	Allow to override a container image without configuring a new digest.
Enterprise / Starter	Allow configuration of the operator and updater with debug logs.
Enterprise / Starter	Check for supported Python versions when starting a deployment run. Recreate the virtual environment if it is using the wrong Python version.
Enterprise / Starter	The installer now makes sure that the microk8s version on the host is supported before starting the upgrade process.
Enterprise / Starter	Speed improvements in the operator/updater reconciliation process.

Upgrade Notes

Enterprise	Upgrade Telegram bridge to 0.15.1-mod-1.
Enterprise	Upgrade WhatsApp bridge to 0.10.7-mod-1.
Enterprise	Upgrade Sygnal to 0.14.3 to support the latest Firebase API.
Enterprise	Update Synapse Admin to v16.92.0.
Enterprise	Update Adminbot to Pipe 6.1.1.
Enterprise / Starter	Matrix Content Scanner upgrade to 1.0.8.
Enterprise / Starter	On RHEL and derived platforms, it now requires `python 3.11` installed.
Enterprise	Upgrade SecureBorderGateway to v1.2.0.
Enterprise	Upgrade Auditbot to 6.1.2 to improve overall request handling efficiency, especially at high-loads.
Enterprise / Starter	Upgrade to Synapse 1.114.0.
Enterprise	Upgrade to Element Call 0.6.3 with improved call layout.
Enterprise	Upgrade to Matrix Authentication Service 0.11.0 and support password auth.
Enterprise	Synapse registration and password policy settings are now moved to Authentication configuration, under Local Password Database mode.
Enterprise	Upgrade Hydrogen to v0.4.1-fix.
Enterprise / Starter	Upgrade microk8s to 1.30. As per 23.10.35/24.04.05/24.05.01, the standalone installer now upgrades microk8s automatically. The microk8s upgrade procedure does not involve an uninstall/reinstall of microk8s anymore. It now will automatically upgrade microk8s to the expected version, and as such, the `--upgrade-cluster` flag has been removed. Any customization to CNI Configuration in `/var/snap/microk8s/current/args/cni-network/cni.yaml` will have to be reconfigured. During the upgrade, microk8s & workloads will restart several times. Managed addons that require upgrading will be temporarily disabled to be upgraded. This all will induce a small downtime of a couple of minutes.
Enterprise / Starter	Upgrade to cert-manager 1.12.13.
Enterprise / Starter	Upgrade ElementWeb to v1.11.81.
Enterprise / Starter	Services got renamed, `-headless` suffixes are all removed. If you are using Network Policies, those will need to be upgraded to the new names.
Enterprise	Global upgrade of the monitoring stack. Victoria Metrics is now on version 1.101.
Enterprise	Now that Synapse brings native Sliding Sync protocol, the Sliding Sync proxy has been discontinued. Its PostgreSQL cluster instance is being cleaned-up.

Security Issues

Enterprise	Previous update might have enabled unexpectedly outbound webhooks in Hookshot. If you don't need this feature, make sure that it is disabled in Hookshot integration, under Generic Webhooks settings.
Enterprise	Better image signatures, enterprise is now published to sigstore.
Enterprise / Starter	Upgrade to Ansible 8 for security fixes.

Bug Fixes

Enterprise / Starter	Fix Remove button not working for some integrations.
Enterprise / Starter	Fix cert-manager upgrade failing to remove old resources.
Enterprise / Starter	Fix operator and updater having permissions issues under Openshift.
Enterprise / Starter	Fix Jitsi JVB failing to get ready when STUN servers list is empty and Coturn is not deployed.
Starter	Fix upgrade failing.
Enterprise	Fix missing storage class on some Monitoring PVCs.
Enterprise	Fix media screen on standalone setup.
Enterprise / Starter	Remove `--upgrade-cluster` parameter as microk8s is now upgraded gracefully.
Enterprise	Fix inconsistent behavior when switching between S3/Persistent volume option under the media tab.
Enterprise / Starter	Fix watchers to avoid triggering unneeded reconciliation loops.
Enterprise	GroupSync: Fix issue when LDAP identities contain commas in their names.
Enterprise	Configuring monitoring stack persistent volumes properly in microk8s requires recreating their statefulsets.
Starter / Enterprise	Fix haproxy failing on IPv4-only nodes.
Enterprise / Starter	The installer no longer flakes between bootstrap and installer view when the Kubernetes cluster is intermittently unreachable.
Enterprise	Fix an Ansible error when installing the telemetry script on the local host when user GID != UID.
Enterprise / Starter	Allow well-known delegation to omit configuration of the ingress entirely without triggering unknown variable errors.
Enterprise / Starter	Allow configuration of Matrix Content Scanner without a storage class name.
Enterprise / Starter	Mark Postgres configuration as required for all components that use a Postgres database.
Enterprise	Mark the source for GroupSync as required.
Enterprise	Remove workloads and dependent CRs from statuses when they're no longer deployed.
Enterprise	Fix provisioning of users that are not rate-limited.
Enterprise	Better identification for the Telegram and WhatsApp bridges in their respective apps.
Enterprise / Starter	Fix an issue where the cert-manager issuer would try to be created but the cert-manager webhook would not be ready.
Starter / Enterprise	Fix haproxy failing on IPv4-only nodes.
Enterprise	Fix monitoring of kube etcd and kube scheduler on microk8s.
Enterprise	Don't include cert-manager in the airgapped tarball. ESS doesn't install or manage cert-manager in airgapped deploys.
Enterprise	Avoid leaking Postgres connections when there are issues provisioning Synapse users.
Enterprise	SIPBridge - Disable Virtual rooms.
Enterprise	Attempt to detect OpenShift and configure operator & updater installation values appropriately.
Enterprise / Starter	Fix an issue preventing setup when a proxy is configured on the host.
Enterprise	Fix a critical issue which would prevent users from accessing Adminbot and Auditbot UI.
Enterprise	Fixes an issue where Auditbot UI would fail to open because tokens were unable to refresh.
Enterprise	Revert change of 24.04.07 which prevented Adminbot and Auditbot from doing an initial sync.
Enterprise	Create new devices for Adminbot and Auditbot to work with the new Rust SDK cryptographic libraries.
Enterprise	Reduce secrets leaks from operator & updater logs. If you need, for debugging purposes, to enable secrets logging, you must edit the operator & updater deployments and set the environment variable `DEBUG_MANIFESTS=1`.
Enterprise / Starter	Refactor Synapse config files to own the priority of each setting managed by ESS.
Enterprise	Sygnal upgrade to 0.15.0 for further Firebase API fixes.
Enterprise	Adminbot and Auditbot are currently incompatible with MAS.
Enterprise	Synapse - Override botocore CA bundle to allow pushing against non-AWS S3 providers.
Enterprise	Add support for Element Call configuration in Element Well Known file.
Enterprise	Matrix Authentication Service - Fix UI configuration of certificates for ingresses.
Enterprise	Minor speed up to initial setup of Synapse.
Starter	Fix MAU Limit, which was configured at 250 instead of 200.
Enterprise	Prevent users from manually editing the Auditbot/Adminbot passphrase.
Enterprise	Fix display of the status of the reconciliation.
Enterprise	Fix Coturn page causing a memory leak.
Enterprise / Starter	Ensure the `nf_conntrack` module is loaded in the kernel when deploying in standalone mode.
Enterprise / Starter	Fix microk8s services subnet parsing.
Enterprise / Starter	Fix some CVEs in the operator/updater/conversion webhook.
Enterprise / Starter	Fix Matrix Content Scanner not working as expected.
Enterprise	Configure max upload size in Secure Border Gateway request body size limit.
Enterprise	Prevent users from editing Auditbot and Adminbot passphrases in the UI.
Enterprise	Enforce pattern checks against inputs under options.
Enterprise / Starter	Increase Matrix Content Scanner ClamAV startup reliability.
Enterprise / Starter	Reduce false positives from Matrix Content Scanner.
Enterprise / Starter	On RHEL and derived platforms, the installer should not rely on `platform-python` for tasks other than Firewalld and SELinux tasks for microk8s setup.
Enterprise / Starter	Fix proxy variables configuration check preventing the installer to go through.
Enterprise / Starter	Fix an issue preventing setup when a proxy is configured on the host. On a proxy configuration errors, the installer will now continue the setup process after displaying the verification error message.
Enterprise / Starter	Enable MSC 3967 on Synapse to avoid some device verification issues.
Enterprise	Setup the onprem-admin user as a MAS admin.
Enterprise / Starter	Fix pulling operator & updater images from behind a proxy.
Enterprise / Starter	Expired sessions are now automatically logged out of the admin interface.
Enterprise / Starter	OIDC sessions are now refreshed correctly when the token expires.
Enterprise	An error is now displayed when the standalone admin UI cannot load the audit/admin interface configuration.
Enterprise	Ensure operator and updater metrics are correctly scraped.
Enterprise	Ensure Telemetry room permissions are consistent.
Enterprise	Ensure component settings for storageClassName override the global setting.
Enterprise / Starter	Removing an item from a list field will now only delete one item.
Enterprise	Setup the onprem-admin user as a MAS admin.
Enterprise / Starter	Fix Synapse being stuck with registration closed even if explicitly allowed.
Enterprise / Starter	Improve reliability of changing the Postgres password in cluster if the password seed changes.
Enterprise / Starter	Fix potential permissions issues during microk8s upgrades.
Enterprise	Construct storage for Matrix Content Scanner if deploying on ESS managed microk8s.
Enterprise	Correctly import airgapped registry settings when upgrading from before 24.04.
Enterprise / Starter	Remove unneeded reconciliations due to bad orphan detection.
Enterprise / Starter	Fix updater metrics scraping.
Enterprise / Starter	Improve reliability of setting up CoreDNS.
Enterprise / Starter	Validate that the node IP is excluded from an HTTP Proxy if one is configured.
Enterprise	Fix empty dashboards (NGinx, Kubernetes Workloads, etc) in Grafana.
Enterprise	Fix missing VMAlert component which is required to gather record metrics.
Enterprise / Starter	Fix microk8s stop command not stopping running containers.
Enterprise / Starter	Improve reliability of some microk8s interactions.

Deprecations

Enterprise	Element Call participants limits feature is deprecated. The option has been removed from the UI.
Enterprise	Jitsi and Element Call can not be deployed together.