Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

matrix-synapse: init on caliban #336

Merged
merged 9 commits into from
May 20, 2024
Merged

matrix-synapse: init on caliban #336

merged 9 commits into from
May 20, 2024

Conversation

mweinelt
Copy link
Member

@mweinelt mweinelt commented Jan 25, 2024 •
edited
Loading

Draft of the matrix-synapse deployment.

Does not yet build, because I'm waiting on UDS support for the synapse listeners, which people are currently actively working on.

  • Generic nginx module
  • Generic PostgreSQL module with backup integration
  • Matrix Synapse module
  • Promtheus scraping for Synapse

Depends on

Follow-up:

@delroth delroth linked an issue Jan 27, 2024 that may be closed by this pull request
Migrate off EMS to self-hosted Synapse #325
Closed
@JulienMalka
Copy link
Member

@mweinelt Thank you for the great work on the backup module, do you mind if I cherry pick it to its own PR that we can merge so I can finish #287 ?

@mweinelt
Copy link
Member Author

mweinelt commented Jan 29, 2024 •
edited
Loading

I extracted the backup module and caliban's config into #345. Will rebase once it is merged.

@mweinelt mweinelt mentioned this pull request Feb 1, 2024
Merged
@mweinelt mweinelt mentioned this pull request Feb 3, 2024
Merged
13 tasks
Ma27
Ma27 reviewed Feb 4, 2024
View reviewed changes
non-critical-infra/modules/matrix-synapse.nix Outdated Show resolved Hide resolved
non-critical-infra/modules/matrix-synapse.nix Outdated
forceSSL = true;
enableACME = true;

locations."^(/_matrix|/_synapse/client)" = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you also need /.well-known/matrix/server to delegate the homeserver for nixos.org to matrix.nixos.org?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but that will happen over in nixos-homepage, since we're hosting nixos.org.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, wasn't aware that this isn't part of the configs in here, disregard then.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NixOS/nixos-homepage#1218

non-critical-infra/modules/matrix-synapse.nix
type = "http";
bind_addresses = [
"127.0.0.1"
"::1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, does the metrics listener not crash anymore when instructing it to listen on a v6 address?
I'm pretty sure that this was the case I only have it listening on v4.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mine listens on IPv6 wildcard only and that works. I have not deployed this yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's cool, just realized that prometheus_client supports that now, nice 🎉
In other words, this can be resolved.

@mweinelt mweinelt mentioned this pull request Feb 4, 2024
Closed
@mweinelt mweinelt force-pushed the matrix-synapse branch 2 times, most recently from 6575139 to 87b41db Compare February 9, 2024 23:18
garbas added a commit to NixOS/nixos-homepage that referenced this pull request Mar 22, 2024
@garbas
well-known: delegate matrix client/server to matrix.nixos.org
d9b426a 
We are moving away from EMS and towards self-hosted infrastructure.

cc NixOS/infra#336
@garbas garbas mentioned this pull request Mar 22, 2024
Merged
@Erethon
Copy link
Contributor

Erethon commented Apr 18, 2024

I would recommend adding some extra configs in synapse, namely:

  • federation_metrics_domains pointing to matrix.org and maybe other big public instances (like fairydust.space). This can help with identifying problems like the ones mentioned in Alert on nixos.org Matrix federation issues #410
  • web_client_location: This will help any people that end up browsing to matrix.nixos.org without really knowing what to do to join the chat. Since matrix.nixos.org won't actually host any user accounts it doesn't make sense to point this to an actual element-web client. I think it could be set to https://matrix.to/#/#community:nixos.org which is the same link as the one provided in the community page.
  • presence: Presence is a feature that's know to cause issues and is often disabled in most large installations as mentioned in the documentation. Since this homeserver will only host bot accounts, there really isn't a reason to have presence enabled that is a feature meant for humans and is resource heavy.
  • media_retention: This is more of a policy thing, but I believe that remote media should be removed after a while because they're somewhat of an abuse vector currently.

@mweinelt mweinelt force-pushed the matrix-synapse branch 3 times, most recently from 38399aa to 075ec4c Compare April 18, 2024 20:38
@mweinelt
Copy link
Member Author

federation_metrics_domains pointing to matrix.org and maybe other big public instances (like fairydust.space). This can help with identifying problems like the ones mentioned in #410

We'll revisit that when we dig into monitoring.

@mweinelt
Copy link
Member Author

web_client_location: This will help any people that end up browsing to matrix.nixos.org without really knowing what to do to join the chat. Since matrix.nixos.org won't actually host any user accounts it doesn't make sense to point this to an actual element-web client. I think it could be set to matrix.to/#/#community:nixos.org which is the same link as the one provided in the community page.

Implemented.

presence: Presence is a feature that's know to element-hq/synapse#15877 and is often disabled in most large installations as mentioned in the documentation. Since this homeserver will only host bot accounts, there really isn't a reason to have presence enabled that is a feature meant for humans and is resource heavy.

We have one interactive administrative user, and we're not that short on resources.

media_retention: This is more of a policy thing, but I believe that remote media should be removed after a while because they're somewhat of an abuse vector currently.

Went with 90d for local content, 14d for remote content, as shown in the config example from upstream.

@mweinelt mweinelt marked this pull request as ready for review May 16, 2024 12:30
@mweinelt mweinelt requested a review from a team as a code owner May 16, 2024 12:30
thilobillerbeck pushed a commit to NixOS/nixos-homepage that referenced this pull request May 20, 2024
@garbas
well-known: delegate matrix client/server to matrix.nixos.org (#1367)
8f3d14c 
We are moving away from EMS and towards self-hosted infrastructure.

cc NixOS/infra#336

Migrated from PR #1218
@mweinelt
non-critical-infra/postgresql: init
24e6324
@mweinelt
non-critical-infra/nginx: init
b5461cf
@mweinelt
non-critical-infra/backup: sync with global backup module
4831362
@mweinelt
non-critical-infra/matrix-synapse: init
f73d6ac
@mweinelt
prometheus: scrape synapse metrics
d049c89
@mweinelt
dns: migrate chat.nixos.org, create matrix.nixos.org
5051147
@mweinelt
non-critical-infra/element-web: init on chat.nixos.org
83d47bb 
This is in line with the previous set up and was used by the moderation
team, as a client to access the administrative account.
@mweinelt
non-critical-infra/flake.lock: Update
e19775c 
Flake lock file updates:

• Updated input 'disko':
    'github:nix-community/disko/bb5ba68ebb73b5ca7996b64e1457fe885891e78e' (2024-04-15)
  → 'github:nix-community/disko/6f01b9710bc4d3bf006eb8df928b4b15e0430901' (2024-05-20)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/90055d5e616bd943795d38808c94dbf0dd35abe8' (2024-04-13)
  → 'github:NixOS/nixpkgs/e7cc61784ddf51c81487637b3031a6dd2d6673a2' (2024-05-18)
• Updated input 'sops-nix':
    'github:Mic92/sops-nix/cc535d07cbcdd562bcca418e475c7b1959cefa4b' (2024-04-15)
  → 'github:Mic92/sops-nix/0bf1808e70ce80046b0cff821c019df2b19aabf5' (2024-05-19)
• Updated input 'sops-nix/nixpkgs-stable':
    'github:NixOS/nixpkgs/c27f3b6d8e29346af16eecc0e9d54b1071eae27e' (2024-04-13)
  → 'github:NixOS/nixpkgs/e7cc61784ddf51c81487637b3031a6dd2d6673a2' (2024-05-18)
• Updated input 'srvos':
    'github:numtide/srvos/bed9cfce2adc4c72de9bc90656d5cfe66e4371f3' (2024-04-12)
  → 'github:numtide/srvos/d368bfdc3a409482b92290a105bcacc108a49d24' (2024-05-20)
@mweinelt
mjolnir: migrate to matrix.nixos.org
4645d26
@mweinelt
Copy link
Member Author

We're migrated.

Set up the Grafana dashboard up and configured federation_metrics_domains.

@mweinelt mweinelt merged commit 382d0c9 into master May 20, 2024
2 checks passed
@mweinelt mweinelt deleted the matrix-synapse branch May 20, 2024 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zimbatm zimbatm zimbatm left review comments

@delroth delroth delroth left review comments

@Ma27 Ma27 Ma27 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Migrate off EMS to self-hosted Synapse
6 participants
@mweinelt @JulienMalka @Erethon @zimbatm @delroth @Ma27