Skip to main content

Kubernetes Installations

How to Use the Installer

This video covers the single node installer, but many of the concepts are also applicable to our multi-node installer.

Overview

Our Element Enterprise Kubernetes Installer can handle the installation of Element Enterprise into your production kubernetes (k8s) environment.

To get started with a kubernetes installation, there are several things that need to be considered and this guide will work through them:

  • k8s Environments
  • Postgresql Database
  • TURN Server
  • SSL Certificates
  • Extra configuation items

Once these areas have been covered, you'll be able to install a production environment!

Unpacking the Installer

Please make sure that you unpack element-enterprise-installer onto a system that has access to your k8s environment. The directory that it unpacks into will be referenced in this document as the installer directory.

You will also need to create a directory for holding the configurations for the installer. This will be referenced as the config directory going forward.

mkdir  ~/.element-onpremise-config

k8s Environments

Element Enterprise Installer allows you to either deploy directly into a kubernetes environment or to render a set of manifests for a future deployment in a kubernetes environment.

To configure your kubernetes environment for a direct deployment, you need to :

  • Configure a kubectl context able to connect to your kubernetes instance
  • Copy k8s.yml.sample to k8s.yml in your config directory. Edit k8s.yml with the following values :
  • provider_storage_class_name: The storage class to use when creating PVCs.
  • ingress_annotations: The annotations to add to the ingresses created by the operator.
  • tls_managed_externally: Should be true if you don't expect the operator to manage the certificates of your kubernetes deployment. In this case, you will be able to skip the *Certificates- chapter of the CONFIGURE.md file.
  • operator_namespace: The namespace to create to deploy the operator.
  • element_namespace: The namespace to create to deploy the element resources.
  • k8s_auth_context: The value of the context used in kubectl. If you want to use cert-manager for your tls certificates, it needs to be already installed in the targeted k8s cluster.

An example k8s.yml file would look like:

provider_storage_class_name: gp8-delete  # select an available storage class
ingress_annotations: ## below are expected annotations for an aws deployment
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/group.name: global
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/ip-address-type: ipv4
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80},{"HTTPS": 443}]'
synapse_ingress_annotations:  # below are required annotations if using the NGINX ingress controller
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
tls_managed_externally: true  # true if the certificates are managed externaly to k8s
security_context_force_uid_gid: true  # true to enable pod runAsUser and fsGroup in security context. false if it should not be used, in the case of openshift for example.
security_context_set_seccomp: true  # true to enable RuntimeDefault pod seccomp. false if it should not be used, in the case of openshift for example.
operator_namespace: <namespace to create to deploy the operator>
element_namespace: <namespace to create to deploy the element resources>
k8s_auth_context: <the k8s auth context>
out_dir: # Absolute path to the directory where to render manifests, if render mode is used
# operator_manager_limits:  # Can be used to defined upper limits if the default one are not large enough for your operator deployment
#   cpu: "2"
#   memory: 8Gi

If you do not want to deploy directly to kubernetes, but wish to render manifests instead, set all of the above mentioned variables except for k8s_auth_context and define a value for the parameter out_dir, which specifies where to write the kubernetes manifests. Further, when you go to run the installer, you need to invoke it as such:

bash install.sh  ~/.element-onpremise-config --target render

Using the above syntax, you will have a set of manifests written out to out_dir that you can then deploy into your kubernetes environment.

N.B. You will need to set your ingress controller's upload size to be at least 50 Mb to match synapse's default upload size if you wish to be able to have users upload files up to 50 Mb in size. Instructions for doing this with nginx are included in the parameters.yml section below.

Postgresql Database

The installation requires that you have a postgresql database with a locale of C and UTF8 encoding set up. See https://matrix-org.github.io/synapse/latest/postgres.html#set-up-database for further details.

Please make note of the database hostname, database name, user, and password as you will need these to begin the installation.

Migrating from our older installer (NOT DOABLE TODAY)

NOTE: If you are running an installation installed with version 2023-02.01 and earlier, you cannot use the GUI installer today. This message will be removed once the migration script is available. For older documentation on the installer, please see our archived documentation repository.

If you have not used our installer before, you may safely ignore this step.

TURNBeginning Serverthe Installation

ForHead installationsto https://ems.element.io/on-premise/download and download the latest installer. The installer will be called element-enterprise-graphical-installer-YYYY-MM.VERSION-gui.bin. You will take this file and copy it to the machine where you will be installing the Element Server Suite. Once you have this file on the machine in whicha directory accessible to your sudo-enabled user, you desirewill torun:

use

chmod video+x conferencing./element-enterprise-graphical-installer-YYYY-MM.VERSION-gui.bin

functionality,

replacing the YYYY-MM.VERSION with the appropriate tag for the installer you downloaded.

Once you have done this, you will run:

./element-enterprise-graphical-installer-YYYY-MM.VERSION-gui.bin

replacing the YYYY-MM.VERSION with the appropriate tag for the installer you downloaded, and this will start a web server with the installer loaded.

You will see a message similar to:

[user@element-demo ~]$ ./element-enterprise-graphical-installer-2023-02.02-gui.bin 
Testing network...

Using self-signed certificate with SHA-256 fingerprint:
         F3:76:B3:2E:1B:B3:D2:20:3C:CD:D0:72:A3:5E:EC:4F:BC:3E:F5:71:37:0B:D7:68:36:2E:2C:AA:7A:F2:83:94 

To start configuration open:
        https://192.168.122.47:8443 or https://10.1.185.64:8443 or https://127.0.0.1:8443

At this point, you will need to haveopen a TURNweb server installedbrowser and available for Elementbrowse to use.

If you do not have an existing TURN server, our installer can configure one forof youthese byIPs. followingYou themay extraneed stepsto inopen Settingport Up Jitsi and TURN With the Installer.

If you have an existing TURN server, please create a file called synapse/turn.yml8443 in your config directory and put the following in it:

turn_uris: [ "turn:turn.matrix.org?transport=udp", "turn:turn.matrix.org?transport=tcp" ]
turn_shared_secret: "n0t4ctuAllymatr1Xd0TorgSshar3d5ecret4obvIousreAsons"
turn_user_lifetime: 86400000
turn_allow_guests: True

based on your TURN server specifics. This will allow the installer to configure synapse to use your TURN server.

A few notes on TURN servers:

  • The TURN server has to be directly accessible by end-users. Normally this means a public IP, however if all the end-users are going to be on a VPN/private network then they just needfirewall to be able to access this address from a different machine.

    If you are unable to open port 8443 or you are having difficulty connecting from a different machine, you may want to try ssh port forwarding in which you would run:

    ssh <host> -L 8443:127.0.0.1:8443

    replacing host with the privateIP IPaddress or hostname of the TURNmachine server.

    • that
  • Theis onlyrunning reasonthe installer. At this point, with ssh connected in this manner, you should be able to haveuse TURNthe onhttps://127.0.0.1:8443 link as this will then forward that request to the installer box via ssh.

    Upon loading this address for the first time, you may be greeted with a privatemessage networkinforming isyou ifthat theyour private network disallows user <-> user traffic and only allows user <-> TURN server traffic. If user <-> user is allowed within the private network then a TURN serverconnection isn't needed.

    • private
such

SSLas Certificates

this:

For SSL Certificates, you have three options:not_private.png

  • Signed PEM encoded certificates from an internet recognized authority.
  • Signed PEM encoded certificates from an internal to your company authority.
  • LetsEncrypt

In this case, you'll need to click "Advanced" and then "Continue to  (unsafe)" in order to view the caseinstaller. ofAs LetsEncrypt,the exact button names and links can vary between browsers, it would be hard for us to document them all, so you may have slightly different wording depending on your hostnamesbrowser.

must

The beHosts accessibleScreen

on

The very first page that you come to is the internet.host screen.

k8s1.png

You will needwant to configuremake certificatessure that "Kubernetes Application" is selected. You can opt to skip the update setup or the operator setup, but unless you know why you are doing that, you should leave them alone.

The very next prompt that you come to is for thean followingEMS names:

Image
    Store
  • fqdn.tld
    • Username
  • element.fqdn.tld
    • and
  • synapse.fqdn.tld
    • Token.
  • dimension.fqdn.tld
    • These
  • hookshot.fqdn.tld
    • are
provided

Usingto you by element as access tokens for our exampleenterprise hosts,container registries. If you have lost your token, you can always generate a new token at https://ems.element.io/on-premise/subscriptions.

k8s2.png

Here, we find the ability to set the namespaces that the application will be deployed into.

The final options on the hostpage are related to connectivity. For this guide, we are assuming "Connected" and you can leave that be. If you are doing "Airgapped", you would pick airgapped at this point and then please see the section on airgapped installations.

You are presented with the option to provide docker hub credentials. These are optional, but if you do not provide them, you may be rate limited by Docker and this could cause issues pulling container images.

The Domains Screen

domains.png

On this page, we get to specify the domains for our installation. In this example, we have a domain name of airgap.local and this would mean our MXIDs would look like @kabbott:airgap.local.

Our domain page has checking to ensure that wethe needhost certificatesnames for:

resolve.
  • local
  • element.local
  • synapse.local
  • dimension.local
  • hookshot.local

Certificates without LetsEncrypt

IfOnce you haveget certificatesgreen forchecks all ofacross the aforementioned host names, thenboard, you can simplyclick placecontinue.

The Certificates Screen

On the .crt and .key files in the certs directory under the config directory. Certificates in the certs directory must take the form of fqdn.cert and fqdn.key.

Certificates with LetsEncrypt

Our installer also supports using LetsEncrypt to build certificates for your host names and automatically install them into your environment. If your hosts are internet accessible, this is the easiest method and only requires an admin email address to provide to LetsEncrypt.

parameters.yml

Now it is time to set parameters.yml. A sample has been provided and to get started, it is easiest to do:

cp config-sample/parameters.yml.sample ~/.element-onpremise-config/parameters.yml

Using the example hostnames of element.local and synapse.local, we would set the following parameters first in parameters.yml:

domain_name: local
element_fqdn: element.local
synapse_fqdn: synapse.local

Next, we need to set the variables related to Postgres. As you are installing into kubernetes,screen, you will needprovide toSSL setcertificate the followinginformation for yourwell-known Postgresdelegation, database:

Synapse, 
postgres_create_in_cluster:Element falseWeb, postgres_fqdn:Synapse `PostgresAdmin, Server`and postgres_user: `Postgres User`
postgres_db: `Postgres Database for Element`

The next line states:

media_size: "50Gi"

You wll want to adjust that to match the size of storage you've allocated for your media. It must be at least 50Gb.Integrator.

The next section pertains to certmanager. If you are using Let's Encrypt, then each of the sections should look like:

certificate_letsencyrpt.png

If you are using certificate files, then you will see a screen like:

certificate_file.png

which allows you to upload a .crt and .key file for each host. These files must be in PEM encoding. Our installer does accept wildcard certificates.

Once you have completed the certificate section for each host on the page, you may click continue.

The Database Screen

database.png

As you must use an external PostgreSQL database with our k8s install, on this page, we provide the option to specify the database name, the database host name, the port to connect to, the SSL mode to use, and finally, the username and password to connect with. Once you have completed this section, you may click continue.

The Media Screen

On this page, you can specify the size of your ownsynapse media volume. Please leave "Create New Volume" checked and specify the size of the volume that you wish to allocate. You must have this space available in /data/element-deployment or whatever you specified back on the hosts screen. If you wish to create a 50G volume, you would need to specify 50Gi for the Volume size.

The Cluster Screen

Most deployments can ignore this, however, if you want to change any microk8s cluster parameters, this is where to do it.

If you are in an environment where you have self-signed certificates, pleaseyou leavewill thesewant itemsto bothdisable blank,TLS verification, by clicking "Advanced" and then scrolling down and unchecking Verify TLS:

verifytls.png

Please bear in mind that disabling TLS verification and using self-signed certificates is not recommended for production deployments.

If your host names are not DNS resolvable, you need to use host aliases and this can be set up here. You will also click "Advanced" and scroll down to the "Host Aliases" section in "k8s". In here, you will click "Add Host Aliases" and then you will specify an IP and host names that resolve to that IP as such:

certmanager_issuer:
certmanager_admin_email:

Ifhostaliases.png

When you haveare chosenfinished with this page, you can click continue.

The Synapse Screen

synapse_page.png

The first setting that you will see is whether you want to auto accept invites. The default of "Manual" will fit most use letsencrypt,cases, pleasebut specifyyou “letsencrypt”are forwelcome to change this value.

The next setting is the certmanager_issue and an actual email address for who should manage the certificates for certmanager_admin_email:

certmanager_issuer: 'letsencrypt'
certmanager_admin_email: 'admin@mydomain.com'

Starting with installer 2022-08.02, we have added two mandatory variables related to telemetry data. These are max_mau_users and strict_mau_users_limit. You should set max_mau_users to the value defined in your contract with Element. If you set this number above your contractual limit, then the software will allow you to exceed your contractual limit and Element will bill you appropriately for the overage.

Setting strict_mau_users_limit to true forces synapse to cap themaximum number of monthly active users (MAU) that you have purchased for your server. Your server will not allow you to go past this value. If you set this higher than your purchased MAU and you go over your purchased MAU, you will need to true up with Element to cover the valuecost definedof inthe max_mau_users.unpaid Sayusers.

The next setting concerns registration. A server with open registration on the open internet can become a target, so we default to closed registration.

After this, you will see that the installer has picked an admin password for example,you. you'veYou paid Element for 1,000 monthly active users and don'twill want to exceeduse that,the eye icon to view the password and copy this down as you wouldwill set:use this with the user onprem-admin-donotdelete to log into the admin panel after installation.

max_mau_users: 1000
strict_mau_users_limit: true

Let'ssynapse_page2.png

say

Continuing, we see telemetry. You should leave this enabled as you are required to report MAU to Element. In the event that you paidare Elementinstalling forinto 1,000an monthlyenviroment activewithout users,internet butaccess, didn'tyou mindmay goingdisable overthis providedso that youit didn'tdoes exceednot 2,000continue monthlyto activetry users.talking Into thisElement. scenario,That said, you wouldare set:still required to generate an MAU report at regular intervals and share that with Element.

max_mau_users: 2000
strict_mau_users_limit: true

For more information on the data that Element collects, please see: What Telemetry Data is Collected by Element?

YouNext, willwe also seehave two paths:

advanced 
#buttons. media_host_data_path:The "/mnt/data/synapse-media"top #one postgres_data_path:is "/mnt/data/synapse-postgres"for
synapse

Forand kubernetesgives installations,you pleasea maketext surebox theseto areinject commentedadditional out.

synapse

Youconfigs will(homeserver.yaml), alsoaccess notice two lines towardsto the end regarding synapse_registration and tls_managed_externally. In most cases, you can leave these alone, but if you wish to close synapse registration or have your TLS managed externally, you may set them at this time.

If you are using nginx as your ingress controller and wish to send files up to 50 Mb in size, please add these two lines to parameters.yml:

synapse_ingress_annotations:
  nginx.ingress.kubernetes.io/proxy-body-size: "50m"

secrets.yml

Now we move on to configuring secrets.yml. You will need the following items here:

  • A Macaroon key
  • Your postgres password for the user specified in parameters.yml
  • A Registration Shared Secret
  • A signing Key
  • An EMS Image Store username and token, which will have been provided to you by Element.

To build a secrets.yml with the macaroon key, themacaroon, registration shared secret, and signing key. Further, you can add and configure multiple synapse workers, external appservices, and federation. These topics are presently beyond the genericscope of this install guide, but we will write them up in due time. Further, you also have the ability to set a STUN shared secret, and the signing key already filled in, please run (in the installer folder):

sh build_secrets.sh
mv secrets.yml  ~/.element-onpremise-config/

You will need to uncommentsecret and set TURN URIs for any existing TURN servers that you may have in your postgres_passwordenvironment.

field

synaapseadvanced1.png

existingturn.png

The second advanced button allows you to explicitly set any microk8s cluster settings that you would like just for the synapse pods. Most users will be able to ignore this.

You can hit continue to go to the propernext password for your database.

Do not forget to also set the values for ems_image_store_username and ems_image_store_token, which will both be provided by Element.

If you have a paid docker hub account, you can specify your username and password to avoid being throttled in the dockerhub_username and dockerhub_token fields. This is optional.screen.

ExtraThe ConfigurationElement ItemsWeb Screen

ItMost is possible to configure anything in Synapse's homeserver.yaml or Element’s config.json.

To do so, you need to create json or yaml files in the appropriate directory under the config directory. These filesusers will be mergedable to thesimply targetclick "Continue" here.

The Advanced section allows you to set any custom element web configurations you would like (config.json).

elewebadvanced.png

A common custom configuration file.

would

Samples are available in config-sample under the installer directory.

To configure synapse:

  • Create a directory synapse at the root of the config directory : mkdir ~/.element-onpremise-config/synapse
  • Copy the configurations extensions you want to setup from config-sample/synapse to  ~/.element-onpremise-config/synapse.
  • Edit the values in the file accordingly to your configuration

To configure element:

  • Create a directory element at the root of the config directory : mkdir ~/.element-onpremise-config/element
  • Copy the configurations extensions you want to setup from config-sample/element to  ~/.element-onpremise-config/element.
  • Edit the values in the file accordingly to your configuration

For specifics onbe configuring permalinks for Element, pleasewhich seewe have documented here: Setting up Permalinks With the Installer

Further, it provides access to the k8s section, allowing you to explicitly set any microk8s cluster settings that you would like just for the element-web pod.

The Enterprise Admin Dashboard

Most users will be able to simply click "Continue" here. The Advanced section allows you to explicitly set any microk8s cluster settings that you would like just for the synapse-admin-ui pod.

The Integrator Screen

intergrator_page.png

On this page, you can set up Integrator, the integrations manager.

The first option allows you to choose whether users can add custom widgets to their rooms with the integrator or not.

The next option allows you to specify which Jitsi instance the Jitsi widget will create conferences on.

The verify TLS option allows you to set this specifically for Integrator, regardless of what you set on the cluster screen.

The logging section allows you to set the log level and whether the output should be structured or not.

The Advanced section allows you to explicitly set any microk8s cluster settings that you would like just for the integrator pods.

Click "Continue to go to the next screen".

The Integrations Screen

This screen is where you can install any available integrations.

Some of these integrations will have "YAML" next to them. When you see this designation, this integration requires making settings in YAML, much like the old installer. However, with this installer, these YAML files are pre-populated and often only involve a few changes.

If you do not see a "YAML" designation next to the integration then this means that will use regular GUI elements to configure this integration.

Over time, we will do the work required to move the integrations with "YAML" next to them to the new GUI format.

For specifics on configuring well known delegation, please see Setting Up Well Known Delegation

For specifics on setting up Delegated Authentication, please see Setting up Delegated Authentication With the Installer

For specifics on setting up Group Sync, please see Setting up Group Sync with the Installer

For specifics on setting up the Integration Manager, please see Setting Up the Integration Manager With the Installer

For specifics on setting up GitLab, GitHub, and JIRA integrations, please see Setting up GitLab, GitHub, and JIRA Integrations With the Installer

For specifics on setting up Adminbot and Auditbot, please see: Setting up Adminbot and Auditbot

For specifics on setting up the Enterprise Admin Dashboard, please see: Configuring the Enterprise Admin Dashboard

For specifics on setting up Hydrogen, please see: Setting Up Hydrogen

For specifics on pointing your installation at an existing Jitsi instance, please see Setting Up Jitsi and TURN With the Installer

If you do not have an existing TURN server or Jitsi server, our installer can configure these for you by following the extra steps in Setting Up Jitsi and TURN With the Installer

For specifics on configuring the Teams Bridge, please see Setting Up the Teams Bridge

For specifics on configuring the Telegram Bridge, please see Setting Up the Telegram Bridge

For specifics on configuring the IRC Bridge, please see Setting Up the IRC Bridge

For specifics on configuring the XMPP Bridge, please see Setting Up the XMPP Bridge

Installation

Let’s review! Have you considered:

  • k8s Environments
  • Postgresql Database
  • TURN Server
  • SSL Certificates
  • Extra configuration items

Once you have theconfigured above sections taken careall of andthe yourintegrations parameters.ymlthat you would like to configure, you can click "Continue" to head to the installation screen.

The Installation Screen

On the installation screen, you should see a blank console and secrets.ymla filesstart arebutton:

in
order,

installscreen.png

Click Start.

After a moment, you will notice the installer hang. If you go back to the prompt where you are ready to beginrunning the actualinstaller, installation.you will see that you are being asked for the sudo password:

installstart1.png

Fromsudoask.png

Go ahead and enter the installersudo directory,password run:and the installation will continue.

bash
On install.shthe ~/.element-onpremise-config

Thevery first time that you run shouldthe go for a little while and then exit, instructinginstaller, you will be prompted to log out and back in.

in

Pleaseagain to allow Linux group membership changes to be refreshed. This means that you will need to issue a ctrl-C in the terminal running your installer and actually log all the way out andof your Linux session, log back inin, andrestart re-runthe installer, navigate back to the installer fromscreen, click start again, and then re-enter your sudo password. You will only have to perform this step once per server.

Verifying Your Installation

Once the installerinstallation directoryhas again:finished, it can take as much as 15 minutes on a first run for everything to be configured and set up. If you use:

bashkubectl install.shget ~/.pods -n element-onpremise-configonprem

You should see similar output to:

NAME                                        READY   STATUS    RESTARTS   AGE
app-element-web-c5bd87777-rqr6s             1/1     Running   1          29m
server-well-known-8c6bd8447-wddtm           1/1     Running   1          29m
postgres-0                                  1/1     Running   1          40m
instance-synapse-main-0                     1/1     Running   2          29m
instance-synapse-haproxy-5b4b55fc9c-hnlmp   1/1     Running   0          20m

Once the admin console is up and running:

first-element-deployment-synapse-admin-ui-564cbf5665-dn8nv   1/1     Running                  1 (4h4m ago)   3d1h

and synapse:

first-element-deployment-synapse-redis-59548698df-gqkcq      1/1     Running                  1 (4h4m ago)   3d2h
first-element-deployment-synapse-haproxy-7587dfd6f7-gp6wh    1/1     Running                  2 (4h3m ago)   2d23h
first-element-deployment-synapse-appservice-0                1/1     Running                  3 (4h3m ago)   3d
first-element-deployment-synapse-main-0                      1/1     Running                  0              3h19m

then you should be able to log in at your admin panel (in our case https://admin.airgap.local/) with the onprem-admin-donotdelete user and the password that was specified on the "Synapse" screen.

A word about Configuration Files

In the new installer, all configuration files are placed in the directory .element-enterprise-server. This can be found in your user's home directory. In this directory, you will find a subdirectory called config that contains the actual configurations.

End-User Documentation

After completing the installation you can share our User Guide to help orient and onboard your users to ElementElement!

Upgrading between installer versions

Installer 2022-08.02

Added two mandatory variables related to telemetry data:
max_mau_users and strict_mau_users_limit

They need to be set (ie: max_mau_users = 1000 and strict_mau_users_limit = true).

Back to top