Migrate From EMS to Self-Hosted

For support on Synapse or matrix-media-repo, ask in the Matrix rooms #synapse:matrix.org and #mediarepo:t2bot.io. For support on bridges, see the bridge documentation for each bridge's support channel.

See also this excellent blog post by Josh Simmons at The Matrix.org Foundation.

Important

Export data

  1. Go to https://ems.element.io/user/hosting#/exports
  2. Select the server you want to export from in the drop-down box, then click Create export
  3. Wait for the export of your config, database, and media to complete
  4. Once all exports have completed, download all files provided

Import process

For all components, you must install the same version as your EMS server is running. Version information is provided in the hostname-config.md file. If a newer version is available, once the import is completed and verified, follow the official upgrade documentation for each component to upgrade.

All required configuration values are provided in the hostname-config.md file.

  1. Following official documentation, install and configure
    1. PostgreSQL. Refer to the Synapse PostgreSQL documentation for database setup
    2. Synapse
    3. Optional (see media step below): matrix-media-repo
  2. When generating your Synapse configuration file, you MUST use the same domain as your EMS server
  3. Do not start Synapse yet
  4. In the Synapse config file (usually homeserver.yaml), set:
    1. pepper. If you do not do this, you have to reset all passwords
    2. Signing key. This is stored in a file. See this config file option for path. Alternatively, add the public key for your EMS hosted server to old_signing_keys. If your EMS server is still running, you can find your public signing key at https://yourHostname.ems.host/_matrix/key/v2/server
    3. macaroon_secret_key
  5. Import the Synapse database
    1. Create a new database per the Synapse PostgreSQL documentation
    2. Or, if your Synapse database is not empty, empty it
      WARNING - THIS WILL IMMEDIATELY AND IRRECOVERABLY DELETE DATA, PLEASE TAKE CARE. WE ACCEPT NO RESPONSIBILITY IF YOU DELETE THE WRONG DATABASE OR DATA
      Connect to the database with psql, then run the following queries:
      DO $$ DECLARE
      r RECORD;
      BEGIN
          FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname = current_schema()) LOOP
              EXECUTE 'DROP TABLE ' || quote_ident(r.tablename) || ' CASCADE';
          END LOOP;
      END $$;
      
      DROP sequence cache_invalidation_stream_seq;
      DROP sequence state_group_id_seq;
      DROP sequence user_id_seq;
      
      Disconnect from the database
    3. Extract the database dump
      unzip emsHostname-synapse-database.zip
      
    4. Import the database dump
      pg_restore \
          --username USERNAME \
          --host HOSTNAME \
          --dbname DATABASE_NAME \
          --no-owner \
          /path/to/id-emsHostname-synapse-database.dir/
      
    5. Verify that sequence was set correctly. Connect to the database and run the query
      SELECT * FROM state_group_id_seq;
      
      last_value should be greater than 1
  6. Import media into matrix-media-repo according to documentation here or import media into Synapses built-in media repository using the import_to_synapse executable available from the latest matrix-media-repo release
  7. Start Synapse and ensure everything is working as expected
  8. Optionally, install Element Web or use another Matrix client
  9. Optionally, install the sliding-sync proxy. This is required for Element X
    • Note, we do not provide an export of the Sliding Sync database as this would not provide any value. As you need to log out and back in from Element X after the migration and in doing so your Sliding Sync proxy needs to re-sync each user. All your data is already included in the Synapse database
  10. If you have any bridges, please see the documentation for each bridge
    Discord | Hookshot | IRC | Signal | Slack | Telegram | Whatsapp
    • The process for importing the bridge databases is the same as with Synapse
  11. Cancel your EMS server. See this FAQ document for instructions. Note, this cannot be undone and all your data will be lost. Ensure your imported server is working as expected before doing this

Revision #38
Created 5 December 2023 11:01:01 by Twilight Sparkle
Updated 19 February 2024 11:42:37 by Twilight Sparkle