overall, this was a pain in the fucking ass. the first 6 hours were pleasant enough, but still, it took far more planning and configuration than i expected.
in total, i spent around 16 18 hours to:
- instantiate and configure infrastructure (aws resources +permissions, e-mail sending, block storage, dns records, etc.)
- prepare the OS
- configure the ansible playbook which would install synapse, configure its components and ancillary modules
- room setup, admin panel and client assessment
- generate a registration token
note: throughout the essay, the “element client” is referenced. in most cases, and unless otherwise specified, i am referring to the desktop version of this
using the ansible playbook
the playbook only supports some specific distros/versions to run the ansible playbook (centos, debian, archlinux). centos 7 is supported, but 8 and stream aren’t.
instructions were missing from the set up guide about ansible collections/modules required to even run the playbook. this wasn’t clear from the errors that were being returned, and github issues reporting them dont help.
ansible-galaxy collection install \
community.general \
ansible.posix \
community.docker
ansible was slow as fuck to execute. if you need to change a synapse module/configuration item post-installation, you have to wait 5+ minutes for the playbook to run again (centos 7 on a dual core ec2 instance with 2 GB of RAM), even though it skips/ignores the majority of the state due to idempotence. i’ve heard this wasn’t a problem for someone running debian 10 on a dual-core and 4GB of RAM 🤷
running a private homeserver
this was a a private homeserver, disallowing federation, and public account registration. about 8 hours were spent trying to figure out how to generate a registration token for people to join.
synapse has support for native registraiton tokens, but they aren’t integrated into element or the synapse admin web UI, afaict.
i assume that this is why a third-party token-based registration service (ansible module, github project) is included in the playbook.
it was nearly entirely undocumented. the comments in the config file were often unhelpful. i read code, PRs and what little spec doc there was, but it was inaccurate in places, and i ended up banging my head against merely attempting to generate a token for far too long.
the invite bot i tried to set up to manage all this, maubot w/ the maubot-invite plugin (leveraging the matrix python framework) merely didnt work. it could connect to the home server, but couldnt receive websocket events. i confirmed this was the case by manually adding logging to the command handlers before any other logic was executed, including authorization checks. nothing was ever logged.
eventually, i ended up co-opting code from that bot’s plugin to send an API call to the registration service, which worked. this was my last ditch effort after having done the same thing with curl and being unsuccessful. i have no idea why the python code worked. it looked to be submitting the same request to me!
update 12/23/21: i chose to disable federation because i was unfamiliar with the privacy implications of it, having never used or administrated a matrix server before all this. a few of my users already had matrix accounts, and found this rightly burdensome since element doesn’t support multi-account switching.
i don’t want to enable it because after learning about it in depth, the implications aren’t great! while public room directory sharing over the federation api is disabled by default, those who aren’t a member of my community can join a public room if they know the address for it (and it’s published,, which it has to be for those who are members to join). the rooms could be set to invite only, but that introduces accessibility limitations to community member onboarding 😞
i’m pretty confident that federation doesn’t have the concept of “linked” accounts where e.g. an account created on the matrix.org homeserver can create and link an account on a private server, using proxied authentication to interact with the latter while primarily logged in as the former. regardless, even if it did, the registration service used to enable private, token-based account registration is third-party, meaning its definitely not integrated with synapse or element!
⋟
discord bridging
the bridge tech i used was matrix-appservice-discord, installed via the aforementioned ansible playbook’s module. it kinda blows, tbh. i spent 4 hours on this aspect of the project.
i looked at a couple others, and they had even more significant problems, has less features relevant to my use case, and/or were younger and with less active development…
basic functionality
you can let the bot create a bridged room for you, or create the matrix-side rooms yourself and instantiate a bridge with a command.
going the former route is hella janky
- every desired bridge must be created by issuing
/join #_discord_<serverid>_<channelid>on the matrix side to instantiate initially - they’re published to the directory by default, and this isn’t configurable in the at the appservice level, nor can it be changed through room settings in the element client. the only way i figured out how to do such a thing was through the synapse admin web ui
- cannot modify room properties (name, topic, access, role permissions), other than to alias the channel id to something human readable for people to
/joinif they want to go that route - since room permissions cannot be modified, a truly read-only configuration on the matrix side isn’t achievable, not through the element client, nor the synapse admin web ui. (note: the synapse admin web ui doesn’t support changing any room’s permissions whatsoever!)
i didn’t read the project’s readme very deeply at first, and missed this next bit before intial publication of this essay.. however, the project readme does list instructions to bridge a pre-existing matrix room, come to find out! many options aren’t broken out into their own discrete ansible config items, but looking at the project’s sample configuration reveals some toggles that can be set, albeit unintuitively (sample reference).
inventory/host_vars/matrix.domain.tld/vars.yml:
matrix_appservice_discord_configuration_extension_yaml: |
bridge:
# Disable portal bridging, where Matrix
# users can search for unbridged Discord
# rooms on their Matrix server.
disablePortalBridging: true
# Disable Join Leave echos from matrix
disableJoinLeaveNotifications: true
# Enable users to bridge rooms using
# !discord commands
enableSelfServiceBridging: true
# Disable Invite echos from matrix
disableInviteNotifications: true
# Auto-determine the language of code blocks
determineCodeLanguage: true
# MXID of an admin user that will be PMd
# if the bridge experiences problems
adminMxid: '@erin:domain.tld'
this route resolves all of the problems detailed above. i could change room permissions to make the bridged announcements channel read only, and could choose the room name such that some categorization and aesthetics could be enforced! 😌
there seems to be support for unbridging and deleting a previously bridged channel, but it’s unclear on how to do this for a bridge that was instantiated by the /join methodology. i tried the !discord unbridge command, and was told i don’t have permissions…
neither could i manually perform the deletion.. even as an admin, i didn’t have the option to kick the bot nor the pseudo-users. the best you can do afaict is to delist it from the room directory, kick matrix-side normal users, and hope no one (re-)joins it manually with /join.
probably there’s a solution to this.. hopefully without manual db manipulation 🤬
other large issues
cannot map existing matrix channels to their discord counterpart(update: i was wrong)- old messages arent populated on the matrix side of the bridge upon room creation
- you can set the discord-side bot user’s permissions for a channel such that a bridge is asymmetric i,e, new messages from discord will appear in matrix, but messages posted to matrix won’t be relayed to discord
- no support for threads
less 😡 issues
- limited confiugration options; cannot disable informational messages such as join/part, user profile changes, etc. from being broadcast to discord
- doesnt translate custom emojis typed with
:emoji-name:from matrix into the appropriate discord references to be rendered on the other side. weirdly enough, this works fine in the opposite direction on element for desktop, but not android. - doesn’t translate custom or normal emoji reactions in the nheko or element clients across a bridge, from either direction
- replying to a message on the matrix side translates into a rich media embed of the quoted message. this doesn’t ping the user on the other side of the bridge.
various comments & observations
- remarking a discord proxy/pseudo-user on the matrix side does translate into a ping on the other side of the bridge 🤯
- obviously, end to end encryption isnt supported. it wouldn’t be without some custom discord client integrating it, which doesn’t currently exist, and would be against discord ToS to run
- messages are deleted bidirectionally across a bridge. on the matrix side, they show as ‘n messages deleted’ from the user
- media uploaded on the discord side is rendered cleanly as if it were uploaded natively on the matrix side. images are displayed scaled down only slightly. in the element client, the thumbnail is about the same size as in discord, and there is a download button in the quick actions displayed on hover. in the nheko client, if it is high resolution, the image appears much larger than in discord, though there is no download button without going into the post options menu.
⋟
rooms
organization
- “spaces” are an attempt at categorization, which create the discord UI equivalent of a “server”
- channels cannot have user or admin-specified custom ordering. the element or nheko client are only able to organize alphabetically, or by most recent activity
administration
- there is a room deletion api, but it isnt integrated with the element client or synapse admin ui. there is a button in the latter, but it doesnt work, returning ‘bad request’
- by default, you can soft delete a room by kicking everyone from it and then leaving it as the last user. after 7 days, normal cleanup operations will hard delete the room id from the database.
- i tried sending a manual api request to issue a hard deletion, following the api spec, and having read the code and issues for its implementation, but couldn’t get it to work.
- you cannot move rooms between spaces, or disable encryption once a room is created. so, if you really want to use the specific room id, but messed up when initially creating it with respect to one of those properties, you’re shit out of luck. maybe i wasnt doing something right, but i was so pissed that i didnt feel like looking any more into it or engaging with the developer community
access control
let’s talk about room permissions. coming from discord and rather enjoying the granular access controls, i found myself highly pretty disappointed. “power levels” are an attempt to create role permission structure. moderators and above can set numeric levels on a user’s room permissions.
this model is akin to IRC services’ per-channel access lists, but finer grained. compared to discord though, the lack of named roles that are easily assignable from various aspects of the chat/admin UI is really felt when examining matrix’s implemententation here. the name for such a structure is cool though, i guess. good job there, nerds 🧐
however, there is no interface for setting a user’s power level in the element client, or synapse admin ui. maybe its implemented in some other client, idk yet. after some digging, i found that this is apparently configred from a user’s “profile” in each individual room. this isn’t clearly called out in the official Moderation Guide
when it comes to spaces, i couldnt figure out how people could even find them to join. they dont appear in the room directory, and there’s no specific directory for them in element. update: apparently that hasn’t been implemented yet, and spaces were just last month considered to be out of beta 🤣 really? why let me set rooms created with them as public and listed in the directory if that doesn’t have any effect?)
other comments
- the options for managing access to the a room or space are public, invite only, or space members only.
- the encryption option doesn’t appear if you are trying to create a public room, by design. you can, however, toggle the privacy setting to invite only, then back to public, and the encryption option will remain. when you then try to create the public and encrypted room, it will warn you that this is pointless due to lack of access control (???), and will make the room slow if there are a lot of users in it, but will let you do it.
- can create spaces as encrypted by default, and all rooms subsequently created in them will also be mandated to have encryption.
⋟
client feature support
message rendering
formatting marks are pretty well supported; i testing italics, bold, monospace, code fences, and block quotes. they all worked in the element desktop client. in the element android client, the ‘markdown editor’ is disabled by default. this must be enabled before messages will be formatted by what marks you might use.
linked images are rendered inline. in the element client, this is a rich media embed, presenting a dramatically scaled down thumbnail, with the uri appearing above. in the nheko client, the behavior is similar as to image uploads, the uri appears below the image, and there is no download button in quick actions.
on other embeddable links; tweets and reddit posts are embedded in element, but not nheko. this occurs accross a bridge fine. sometimes it takes a few seconds for the embed to render though. im not sure if thats a functino of network latency and instance capacity though.
sounds
couldnt figure out how to disable sound for sending messages in element. there is a global toggle for ‘audible notifications for this session,’ which presumably would disable both send and receive chimes. in nheko, messages merely do not make sound at all by default, and i don’t think this can be changed 🙃
direct messaging
in element, creating a new DM is rather trivial. in the people section of the sidebar, clicking the + takes you to a nice modal where you can search for a user easily.
in nheko, i’m not actually sure how to create a DM without being in a room with someone. the client is basically undocumented. in that case, i can click their profile picture, and then an icon to start one from their profile.
check out the ‘create a room’ dialogue for nheko below. what’s the direct chat toglle do? putting a username or id into the room name doesn’t create a DM with the person with the toggle set…
notifications
- globally or per room, one may set notifications to occur for all messages, never, or only on mentions
- in element, notification configs are a bit more granular than nheko, presenting toggles for mentions of your display name, username, or certain custom set keywords
- it’s rather annoying to change these preferences in nheko. unlike element, where u can quickly tap the bell when hovering over a room in the list, accessing this preference takes two clicks to get to the room settings, and then another three clicks to change and save the setting.
⋟
closing remarks
what i wanted out of matrix was a cleanly integrated chat alternative to compliment my community that primarily preferred to use discord. end-to-end encryption for privacy and secrecy was something several of my users desired, fearing the political climate surrounding freedom of speech online, and discord’s recent moves to nuke servers without being in clear violation of their community guidelines.
while its delivery on the promise of end-to-end encryption is pretty solid, it failed on the first point rather spectacularly significantlky. in my opinion, the matrix server and element client aren’t ready for prime time, even with bridges being fairly robust. tbh, i wouldn’t even say any of this can be considered to be out of beta. 😞 i’ve moved forward with announcing it to my userbase, but don’t think my time spent was worth the headache and frustration, sadly
ill be continuing to wait for fosscord, and maybe rocket.chat. the former has a working, feature-parity backend, but they’re still building out the client. the latter seems legit for the most part, though i haven’t evaluated it at much at all, just having learned of it today. it doesn’t meet my needs unfortunately from a cursory glance, as it doesn’t support discord bridges whatsoever. it does have end-to-end encryption though, which is nice to see!
perhaps, in time, ill contribute the missing documentation and some code.. for now,, i’m far too disgruntled with how slow planning and development has been. it’s been years already and there are so many people working on these projects! it’s frankly disappointing. i want to love matrix, but it’s far too unintuitive and unpolished from both an operational and end-user standpoint 😢
