ESS LTS 24.10 Change Logs and Upgrade Notes
LTS 24.10 Changelogs and important Update Notes, always check here before upgrading!
Upgrade Notes for the 24.10 LTS
If you plan on upgrading to this LTS we always recommend upgrading to the latest patch version of your current LTS and then updating to the latest version of this LTS.
If you plan on updating, we recommend installing the latest patch version.
Whether upgrading or updating, you should be aware of all significant upgrade notes from each prior patch version. Any highlighted patch notes for this specific LTS have been collated for convenience below, you can find the full changelogs of each release thereafter.
24.10.01-gui | The required Python versions are now 3.10, 3.11, 3.12. As a result, Ubuntu 24.04 is now supported but Ubuntu 20.04 support is dropped. Please consult the Ubuntu documentation for upgrading between Ubuntu LTS versions. The installer will attempt to install the required packages in some scenarios. Airgapped customers should ensure that Python 3.12 packages are available in their package mirrors. Alternatively, Python 3.10, 3.11, or 3.12 can be preinstalled on the server in all situations. |
24.10.01-gui
Release Summary
The required Python versions are now 3.10, 3.11, 3.12. As a result, Ubuntu 24.04 is now supported but Ubuntu 20.04 support is dropped. Please consult the Ubuntu documentation for upgrading between Ubuntu LTS versions. The installer will attempt to install the required packages in some scenarios. Airgapped customers should ensure that Python 3.12 packages are available in their package mirrors. Alternatively, Python 3.10, 3.11, or 3.12 can be preinstalled on the server in all situations.
New Features
Enterprise | XMPP Bridge and IRC Bridge both support Authenticated Medias. Their signing key is generated automatically by the installer UI. |
Enterprise / Starter | Authenticated Media is now enforced by default. All components but Matrix Content Scanner are compatible with it. If you need to disable it, please add enable_authenticated_media: false to Synapse -> Additional YAML. |
Enterprise / Starter | Add the possibility to allow/deny rooms and log events for Auditbot. |
Enterprise / Starter | Support overriding just the server and path in the image digest ConfigMap. |
Enterprise / Starter | Support Element Call in Element X. |
Enterprise / Starter | Matrix Authentication Service and Synapse only use internal paths to communicate, removing the need for hostAliases setup between the two. |
Enterprise | All ESS Images are now hosted behind registry.element.io. |
Enterprise | Synapse workers supporting multiple replicas can now be configured for automatic horizontal scaling. |
Enterprise / Starter | Expose images_digests.yml file in the Download screen for Airgapped customers who want to sync their registry directly with registry.element.io. |
Upgrade Notes
Enterprise / Starter | Upgrade to cert-manager 1.15.3. |
Enterprise / Starter | Operator - Upgrade Python to 3.12, Ansible to 2.17. |
Enterprise / Starter | Upgrade Synapse to v1.116.0. |
Enterprise / Starter | Upgrade Element Web to v1.11.82. |
Enterprise | Update XMPP Bridge to 2.0.1. |
Enterprise | Update Adminbot and Auditbot to 6.3.1. |
Enterprise | Update IRC Bridge to 3.0.2. |
Enterprise | Update Hydrogen to 0.5.0. |
Enterprise / Starter | Update Admin Console to v16.105.4. |
Enterprise / Starter | Upgrade microk8s to 1.31. As per 24.10 releases, the standalone installer only supports upgrading microk8s installed from 23.10 releases. As per 23.10.35/24.04.05/24.05.01, the standalone installer now upgrades microk8s automatically. The microk8s upgrade procedure does not involve an uninstall/reinstall of microk8s anymore. It now will automatically upgrade microk8s to the expected version, and as such, the --upgrade-cluster flag has been removed.Any customization to CNI Configuration in /var/snap/microk8s/current/args/cni-network/cni.yaml will have to be reconfigured.During the upgrade, microk8s & workloads will restart several times. Managed addons that require upgrading will be temporarily disabled to be upgraded. This all will induce a small downtime of a couple of minutes. |
Enterprise / Starter | The installer now makes sure the upgrade comes from a supported version. |
Security Issues
Enterprise / Starter | Upgrade to Ansible 9 for security fixes and Python compatibility. |
Bug Fixes
Enterprise | Allow only one VoIP platform (Jitsi or Element Call) to be enabled. |
Enterprise | Fix migration of authentication settings from <24.07.01 with Matrix Authentication Service installed. |
Enterprise / Starter | Fix an issue where, after update, the installer UI would ask to save for changes on the Host screen when the user actually did not click anything. |
Enterprise | Fix monitoring integration tab not rendering. |
Enterprise | Fix Auditbot logs viewer when Matrix Authentication Service is setup. |
Deprecations
Starter | Matrix Content Scanner is not available anymore in Starter Edition. |
Non-LTS Monthly Release Changes
This section summarises all the changes between the previous LTS and this one during the monthly non-LTS releases. Duplicate entries where individual components received upgrades have been removed so only the latest version is mentioned.
You can then compare the below changelog against the above LTS releases for an accurate overall changelog if upgrading from a previous LTS.
Some changes added to non-LTS monthly releases are backported into older LTS releases if required. As such, some of the below features may already be present in a previous LTS. You can check the associated LTS books' respective changelog page to compare.
Release Summary
The required Python versions are now 3.9, 3.10, 3.11. These are available on all supported OS distributions. The installer will attempt to install the required packages in some scenarios. Airgapped customers should ensure that Python 3.9 packages are available in their package mirrors. Alternatively, Python 3.9, 3.10, or 3.11 can be preinstalled on the server in all situations.
Enterprise
This release adds the possibility to enable Matrix Authentication Service during initial setup. Enabling Matrix Authentication Service is experimental; a couple of features do not work yet with it (Auditbot, Adminbot, Element Call, GroupSync, Admin UI). Enabling MAS allows you to use Element X with OIDC or LDAP login.Enterprise
This release now makes ESS ElementX ready by default. Any new installation will deploy Matrix Authentication Service. Existing setups will not profit from this change, migration paths are planned later in the future.New Features
General
Support knocking withgeneric_worker federation.
Enterprise / Starter
Major Change: The standalone installer now upgrades microk8s gracefully and automatically. The microk8s upgrade procedure no longer involves an uninstall/reinstall of microk8s. It now automatically upgrades microk8s to the expected version, and the--upgrade-cluster flag has been removed.Any customization to CNI Configuration in
/var/snap/microk8s/current/args/cni-network/cni.yaml will need to be reconfigured. During the upgrade, microk8s will restart, and addons will be disabled to force an upgrade. This process may induce a small downtime of a couple of minutes.
Enterprise
Status watchers are now golang containers, reducing resources used by the operator and updater.Enterprise
Allow configuration of Synapse database connection pool sizes.Enterprise
Add a ServiceMonitor to scrape metrics of microk8s ingress.Enterprise
Expose Operator & Updater metrics.Enterprise
Add support for Outbound webhooks in Hookshot.Enterprise
Synapse OIDC support attribute requirements.Enterprise
Add a new experimental feature to enable Matrix Authentication Service during ESS bootstrap.Enterprise
Simplification of the OIDC provider configuration. After upgrading, please make sure that your OIDC settings were properly migrated to the new view.Enterprise
It is now possible to enable the new Matrix Authentication Service when bootstrapping a new ESS setup. It is an experimental feature, incompatible with Groupsync, Element Call, Auditbot, and Adminbot at this time. It is required to try out Element X with OIDC login.Enterprise
It is now possible to use LDAP with Matrix Authentication Service.Enterprise / Starter
Properly enforce patterns check in UI inputs under cards that can be enabled/disabled.Enterprise
Display deployment availability in the UI, in addition to the reconciliation status.Enterprise
Element Call is now MAS-Compatible.Enterprise
Add the possibility to configure a matrix stats endpoint.Enterprise
Setup the onprem-admin user as a MAS admin.Enterprise
Allow configuration of empty (no) disallowed IP ranges in Hookshot.Enterprise
Validate Synapse Telemetry is consistently set.Enterprise / Starter
Synapse improve worker configuration.Enterprise / Starter
Allow blocking of non-scanned media.Enterprise
Adminbot/Auditbot + MAS compatibility.Enterprise / Starter
The UI now properly marks secrets as required when necessary.Enterprise / Starter
The reconciliation process now ensures that all secrets are present and shows missing secrets if necessary.Enterprise
Add Hookshot permissions configuration.Enterprise
Add the possibility to manage Federation dynamically from the Admin Console when Secure Border Gateway is enabled.Enterprise / Starter
Speed up initial Synapse deployment.Enterprise
Add the possibility to configure user deprovisioning and room cleanup in GroupSync.Enterprise
Synapse auto invite: use Synapse native feature, run on background worker if it exists.Enterprise
Allow to override a container image without configuring a new digest.Enterprise / Starter
Support MSC4186 / Simplified Sliding Sync natively in Synapse.Enterprise / Starter
Support authenticated media APIs (MSC3916) in Synapse.Enterprise / Starter
Scrape Synapse HAProxy metrics.Enterprise
Scrape Adminbot and Auditbot HAProxy metrics.Enterprise
Set default volume sizes for Matrix Content Scanner volumes.Enterprise
Set default volume sizes for Adminbot, Auditbot & Sydent volumes.Enterprise / Starter
The administration interface can now manage users on deployments using Matrix OIDC.Enterprise
Administrators can now configure the SBG allowlist within the Admin UI.Enterprise / Starter
The user management page now allows admins to toggle the locked status of users.Enterprise / Starter
The user management page now displays the primary email address of users.Enterprise / Starter
The user management page will now default to showing locked and deactivated users when searching by name.Enterprise
Enabling MAS is not experimental anymore, and is now the default setup mode.Enterprise
Allow to override a container image without configuring a new digest.Enterprise / Starter
Allow configuration of the operator and updater with debug logs.Enterprise / Starter
Check for supported Python versions when starting a deployment run. Recreate the virtual environment if it is using the wrong Python version.Enterprise / Starter
The installer now makes sure that the microk8s version on the host is supported before starting the upgrade process.Enterprise / Starter
Speed improvements in the operator/updater reconciliation process.Upgrade Notes
Enterprise
Upgrade Telegram bridge to 0.15.1-mod-1.Enterprise
Upgrade WhatsApp bridge to 0.10.7-mod-1.Enterprise
Upgrade Sygnal to 0.14.3 to support the latest Firebase API.Enterprise
Update Synapse Admin to v16.92.0.Enterprise
Update Adminbot to Pipe 6.1.1.Enterprise / Starter
Matrix Content Scanner upgrade to 1.0.8.Enterprise / Starter
On RHEL and derived platforms, it now requirespython 3.11 installed.
Enterprise
Upgrade SecureBorderGateway to v1.2.0.Enterprise
Upgrade Auditbot to 6.1.2 to improve overall request handling efficiency, especially at high-loads.Enterprise / Starter
Upgrade to Synapse 1.114.0.Enterprise
Upgrade to Element Call 0.6.3 with improved call layout.Enterprise
Upgrade to Matrix Authentication Service 0.11.0 and support password auth.Enterprise
Synapse registration and password policy settings are now moved to Authentication configuration, under Local Password Database mode.Enterprise
Upgrade Hydrogen to v0.4.1-fix.Enterprise / Starter
Upgrade microk8s to 1.30.As per 23.10.35/24.04.05/24.05.01, the standalone installer now upgrades microk8s automatically. The microk8s upgrade procedure does not involve an uninstall/reinstall of microk8s anymore. It now will automatically upgrade microk8s to the expected version, and as such, the
--upgrade-cluster flag has been removed.Any customization to CNI Configuration in
/var/snap/microk8s/current/args/cni-network/cni.yaml will have to be reconfigured.During the upgrade, microk8s & workloads will restart several times. Managed addons that require upgrading will be temporarily disabled to be upgraded.
This all will induce a small downtime of a couple of minutes.
Enterprise / Starter
Upgrade to cert-manager 1.12.13.Enterprise / Starter
Upgrade ElementWeb to v1.11.81.Enterprise / Starter
Services got renamed,-headless suffixes are all removed. If you are using Network Policies, those will need to be upgraded to the new names.
Enterprise
Global upgrade of the monitoring stack. Victoria Metrics is now on version 1.101.Enterprise
Now that Synapse brings native Sliding Sync protocol, the Sliding Sync proxy has been discontinued. Its PostgreSQL cluster instance is being cleaned-up.Security Issues
Enterprise
Previous update might have enabled unexpectedly outbound webhooks in Hookshot. If you don't need this feature, make sure that it is disabled in Hookshot integration, under Generic Webhooks settings.Enterprise
Better image signatures, enterprise is now published to sigstore.Enterprise / Starter
Upgrade to Ansible 8 for security fixes.Bug Fixes
Enterprise / Starter
Fix Remove button not working for some integrations.Enterprise / Starter
Fix cert-manager upgrade failing to remove old resources.Enterprise / Starter
Fix operator and updater having permissions issues under Openshift.Enterprise / Starter
Fix Jitsi JVB failing to get ready when STUN servers list is empty and Coturn is not deployed.Starter
Fix upgrade failing.Enterprise
Fix missing storage class on some Monitoring PVCs.Enterprise
Fix media screen on standalone setup.Enterprise / Starter
Remove--upgrade-cluster parameter as microk8s is now upgraded gracefully.
Enterprise
Fix inconsistent behavior when switching between S3/Persistent volume option under the media tab.Enterprise / Starter
Fix watchers to avoid triggering unneeded reconciliation loops.Enterprise
GroupSync: Fix issue when LDAP identities contain commas in their names.Enterprise
Configuring monitoring stack persistent volumes properly in microk8s requires recreating their statefulsets.Starter / Enterprise
Fix haproxy failing on IPv4-only nodes.Enterprise / Starter
The installer no longer flakes between bootstrap and installer view when the Kubernetes cluster is intermittently unreachable.Enterprise
Fix an Ansible error when installing the telemetry script on the local host when user GID != UID.Enterprise / Starter
Allow well-known delegation to omit configuration of the ingress entirely without triggering unknown variable errors.Enterprise / Starter
Allow configuration of Matrix Content Scanner without a storage class name.Enterprise / Starter
Mark Postgres configuration as required for all components that use a Postgres database.Enterprise
Mark the source for GroupSync as required.Enterprise
Remove workloads and dependent CRs from statuses when they're no longer deployed.Enterprise
Fix provisioning of users that are not rate-limited.Enterprise
Better identification for the Telegram and WhatsApp bridges in their respective apps.Enterprise / Starter
Fix an issue where the cert-manager issuer would try to be created but the cert-manager webhook would not be ready.Starter / Enterprise
Fix haproxy failing on IPv4-only nodes.Enterprise
Fix monitoring of kube etcd and kube scheduler on microk8s.Enterprise
Don't include cert-manager in the airgapped tarball. ESS doesn't install or manage cert-manager in airgapped deploys.Enterprise
Avoid leaking Postgres connections when there are issues provisioning Synapse users.Enterprise
SIPBridge - Disable Virtual rooms.Enterprise
Attempt to detect OpenShift and configure operator & updater installation values appropriately.Enterprise / Starter
Fix an issue preventing setup when a proxy is configured on the host.Enterprise
Fix a critical issue which would prevent users from accessing Adminbot and Auditbot UI.Enterprise
Fixes an issue where Auditbot UI would fail to open because tokens were unable to refresh.Enterprise
Revert change of 24.04.07 which prevented Adminbot and Auditbot from doing an initial sync.Enterprise
Create new devices for Adminbot and Auditbot to work with the new Rust SDK cryptographic libraries.Enterprise
Reduce secrets leaks from operator & updater logs. If you need, for debugging purposes, to enable secrets logging, you must edit the operator & updater deployments and set the environment variableDEBUG_MANIFESTS=1.
Enterprise / Starter
Refactor Synapse config files to own the priority of each setting managed by ESS.Enterprise
Sygnal upgrade to 0.15.0 for further Firebase API fixes.Enterprise
Adminbot and Auditbot are currently incompatible with MAS.Enterprise
Synapse - Override botocore CA bundle to allow pushing against non-AWS S3 providers.Enterprise
Add support for Element Call configuration in Element Well Known file.Enterprise
Matrix Authentication Service - Fix UI configuration of certificates for ingresses.Enterprise
Minor speed up to initial setup of Synapse.Starter
Fix MAU Limit, which was configured at 250 instead of 200.Enterprise
Prevent users from manually editing the Auditbot/Adminbot passphrase.Enterprise
Fix display of the status of the reconciliation.Enterprise
Fix Coturn page causing a memory leak.Enterprise / Starter
Ensure thenf_conntrack module is loaded in the kernel when deploying in standalone mode.
Enterprise / Starter
Fix microk8s services subnet parsing.Enterprise / Starter
Fix some CVEs in the operator/updater/conversion webhook.Enterprise / Starter
Fix Matrix Content Scanner not working as expected.Enterprise
Configure max upload size in Secure Border Gateway request body size limit.Enterprise
Prevent users from editing Auditbot and Adminbot passphrases in the UI.Enterprise
Enforce pattern checks against inputs under options.Enterprise / Starter
Increase Matrix Content Scanner ClamAV startup reliability.Enterprise / Starter
Reduce false positives from Matrix Content Scanner.Enterprise / Starter
On RHEL and derived platforms, the installer should not rely onplatform-python for tasks other than Firewalld and SELinux tasks for microk8s setup.
Enterprise / Starter
Fix proxy variables configuration check preventing the installer to go through.Enterprise / Starter
Fix an issue preventing setup when a proxy is configured on the host. On a proxy configuration errors, the installer will now continue the setup process after displaying the verification error message.Enterprise / Starter
Enable MSC 3967 on Synapse to avoid some device verification issues.Enterprise
Setup the onprem-admin user as a MAS admin.Enterprise / Starter
Fix pulling operator & updater images from behind a proxy.Enterprise / Starter
Expired sessions are now automatically logged out of the admin interface.Enterprise / Starter
OIDC sessions are now refreshed correctly when the token expires.Enterprise
An error is now displayed when the standalone admin UI cannot load the audit/admin interface configuration.Enterprise
Ensure operator and updater metrics are correctly scraped.Enterprise
Ensure Telemetry room permissions are consistent.Enterprise
Ensure component settings for storageClassName override the global setting.Enterprise / Starter
Removing an item from a list field will now only delete one item.Enterprise
Setup the onprem-admin user as a MAS admin.Enterprise / Starter
Fix Synapse being stuck with registration closed even if explicitly allowed.Enterprise / Starter
Improve reliability of changing the Postgres password in cluster if the password seed changes.Enterprise / Starter
Fix potential permissions issues during microk8s upgrades.Enterprise
Construct storage for Matrix Content Scanner if deploying on ESS managed microk8s.Enterprise
Correctly import airgapped registry settings when upgrading from before 24.04.Enterprise / Starter
Remove unneeded reconciliations due to bad orphan detection.Enterprise / Starter
Fix updater metrics scraping.Enterprise / Starter
Improve reliability of setting up CoreDNS.Enterprise / Starter
Validate that the node IP is excluded from an HTTP Proxy if one is configured.Enterprise
Fix empty dashboards (NGinx, Kubernetes Workloads, etc) in Grafana.Enterprise
Fix missing VMAlert component which is required to gather record metrics.Enterprise / Starter
Fix microk8s stop command not stopping running containers.Enterprise / Starter
Improve reliability of some microk8s interactions.Deprecations
Enterprise
Element Call participants limits feature is deprecated. The option has been removed from the UI.Enterprise
Jitsi and Element Call can not be deployed together.