How to get around Matrix and Riot with a screen reader

On December 19, Mozilla announced that it will switch to Matrix and Riot as the main synchronous communication platform, replacing IRC. This post is aimed to give you an introduction to using Riot, the most popular Matrix client, with a screen reader.

Matrix itself is an open standard protocol that defines how a chat client, such as Riot, talks to a server, such as Synapse, and how servers communicate to each other to exchange messages in shared, or as is the jargon, federated, rooms. In the Matrix network, rooms can be federated across the whole network, so no matter which server you choose as your home server, you’ll get the messages in rooms you join no matter where they originated from. You can imagine this as if you were signing up to a particular e-mail or news server, and then getting messages from other servers depending on which newsgroups you subscribe to or e-mail lists you join.

IRC, on the other hand, is pretty self-contained. You have to connect to a particular server to join a particular network. And federation was not built into the platform. There are a few non-standard ways for servers to communicate with one another to share the load on any single network, but as this is non-standard, it is also error-prone. Ever heard of net splits? Pretty not pretty. 🙂

But one thing both networks have in common is the fact that there are client choices available. There are clients for mobile platforms, the command line, the web, and first native offerings are also in the works. Check out this non exhaustive list of clients for inspiration.

The client most people will, however, probably come in contact with initially is Riot, the client primarily developed by New Vector, a company behind many Matrix offerings. Riot for web/desktop, iOS, and Android are all open-source, with repositories on Github. Filing issues or submitting pull requests is therefore possible for anyone who wants to improve them. Initially, accessibility was quite bad across the board. And there are still a number of issues open for the web client, but things have improved massively since the one month trial Mozilla did in September and October. But the team didn’t stop there and is improving accessibility, with also some help by yours truly, to this day and beyond. So by the time you read this, there may already be more things working than described here.

The web client

The web client, once you signed up and logged in, consists of a top part that has a header, a Communities button which currently doesn’t do much, a button to submit bugs or feedback, and a Profile sub menu button. That sub menu, when opened, has Settings and Logout options.

The Settings, when opened, come up in a modal dialog. The buttons at the top are actually kind of tabs, just not exposed that way yet. They control various settings which then appear below. The settings apply once you change them and hit Save at the bottom. To close the dialog, choose the Close button at the top.

Back in the main UI, following the top bar is the left side bar. Depending on your settings, you first get a tool bar of recently visited rooms, then an Explore button, a Search field, and a room list arranged as a tree view.

The Explore button opens a search panel where you can discover new rooms to join. You can search, and get the results in a data table you can navigate as usual. You can preview or join rooms. Matrix usually keeps a past history of what was being said, so if you are allowed to view the contents as a non-member, you will get a list showing recent messages. Otherwise, you can also join the room either from the table or the preview.

The search field and room list show the rooms you are a member of, and direct messages with members you participate in. Direct messages are private messages. They can be one-on-one, or small groups of people. You can focus the edit field for search, then tab forward once and use the arrow keys to navigate. Press Enter to switch to the selected room. You can also invoke a context menu by usual means to access more options. You can also start a new chat when in the People group, or create your own room when within the Rooms group.

If you are in a room, the main area, also denoted by a landmark, contains the room view. At the top, there is the name and topic, a Room Settings button, a means to manage Integrations, such as Github or other bots you have connected, or perform a Search. You can also jump to the first unread message or to the bottom of the messages list. The latter will effectively mark the whole room as read. So to get to the first unread messages, press this button, and once your screen reader starts speaking, press s (in JAWS or NVDA) to move to the first separator. That is then usually the beginning of new messages (see below for more info).

Below that, you will find a set of tabs like Members List, Files, and Notifications. Technically, those pertain to the side bar on the right, and should be closer to there DOM-wise, but currently are not.

Below those tabs, the main list of messages is shown. It is dynamically updating, so when you get close to either edge, new messages are pulled in. A separator marks where each new day begins, and also is the unread marker in case you have viewed this room before. You navigate either using the virtual cursor arrow keys, or by skipping between list items. In NVDA and JAWS, you press i and shift+i quick navigation keys, for example. Refer to your screen reader documentation on how this is done. Once focused within a message, action items become available following the message, like adding a reaction, replying, or more options. If you are on your own message, you can also edit that, to correct typos, for example. Those are all labeled buttons. These appear either on mouse hover or when focus is within the message container. However, due to some browser quirks, sometimes things may disappear still, so if things aren’t working as you expect them to, route the mouse there to hover.

Full keyboard support for the message list, similar to what Slack is offering, for example, is planned, but not implemented yet. Sighted keyboard users are still largely trapped in the input for a new message currently. For screen reader users, it is best to rely on browse or virtual cursor mode to get around.

Below the messages, outside the list, is the composition area for new messages or replies if you chose to reply to a message. You can type as usual, use Markdown, and autocomplete people starting with an @ sign, or emojis starting with a colon. These auto-completes are accessible, although right now, you should complete with Space, not Enter or Tab yet. This is being worked on to meet expectations and comply with the WAI-ARIA authoring practices, but at the time of this writing, isn’t in a release yet.

Below the compose area, there are buttons to add files, start an audio or video call, and add stickers.

Below that is a new landmark with the side bar in it, displayed to the right. This can contain either the list of members in this room, files, or your Notifications. This depends on which tab is selected above, and which should actually be in this side bar as well.

If the room member list is displayed, you can invite others to the room as well if you know their handle. You can jump from button to button for each member. If you press Enter or Space, you’ll get more info about that member below the list. You can also bring up a context menu to get more options. You do this via regular shift+F10 or Applications Key. There is also a search facility available.

The Files tab contains files shared within this room.

Things that don’t work well yet

Apart from the problem that the messages list cannot yet be navigated with the keyboard, there are a few other things that don’t work so well yet, but which are in the works of getting improved. One of them is the members list itself. It should be easier to navigate with the keyboard, and the modal to show some more info about a user also needs some improvements. There is a pull request in the works to improve modal popovers across the board, so hopefully this will improve once merged.

One other area that is currently problematic is adding a reaction to a message. The emoji picker for that is not keyboard accessible yet, and the screen reader list has only clickable items. Using NVDA, I can indeed add a reaction reliably, by using character or word navigation when in virtual mode inside the list, but it isn’t pretty.

Also, the fact that things you can do with a message are not as reliable as desired can pose some problems still. It is far better than it was, but the true solution will come with the work that makes the whole thing fully keyboard accessible. Among these things will probably also be the fact that toolbars don’t yet behave according to the WAI-ARIA authoring practices design pattern for toolbars. This is recorded. So, sometimes, for the messages list, you will need your screen reader’s mouse emulation commands to pull the mouse somewhere in order for certain controls to appear. But use this as a last resort if, for some reason, you get stuck.

Riot on mobile

Both Riot for iOS and RiotX for Android, the newer of the two flavors that are on the Google Play Store, have received accessibility improvements and are much more usable than they were a few months ago.

On iOS, I suggest to ignore the Home tab for now, since that doesn’t work at all with VoiceOver. Instead, use the Direct Messages and Rooms tabs to navigate your messages and room chats. You can also add some to favorites, which is the second tab, which also works. Communities are in the same “in development” state as they are on the web. Inside chats, double-tap and hold or use the context menu gesture to interact with elements. There is a button in the top middle below the header that allows you to jump to the first unread message. From there, you can swipe to the right to read through your chat log. Sometimes, if something in the room changes, the position may shift slightly, and you may lose position. This also happens to sighted peopel and is nothing VoiceOver specific.

On Android, the app that gets the most attention is RiotX, not Riot. That also gets most of the new TalkBack enhancements when there are some. I must admit I haven’t used it myself, since all this started when I already had switched back to iOS, but I hear good things from others in the #a11y:matrix.org channel.

Conclusion

I must say I share the excitement the Matrix team expressed in their announcement of the collaboration with Mozilla. It is an open standard, with open-source software, with choices for clients to use, and the possibility to improve the accessibility by reporting issues or fixing bugs. New Riot releases are put out frequently, so those fixes also will get deployed to the Mozilla instance and other hosted Matrix offerings who also use Riot as their front-end.

I hope this guide helps you find your way around Riot and the Matrix Eco system more easily. It still has issues, but so have proprietary offerings like Slack or the like. And as we all know, accessibility is a process of constant improvement.

I would personally like to thank my colleagues for making accessibility one of the deciding factors for choosing the replacement. Big thanks also go out to the members of the Matrix and New Vector teams who were super responsive to accessibility concerns, and have kept the momentum going even after the trial was over and nobody knew if Matrix would be chosen in the end. It is a pleasure working with you all!

See you in January when the new Mozilla instance will be spun up and commence service!

One thought on “How to get around Matrix and Riot with a screen reader

  1. I’m seriously waiting on the day it’s like email, where we can use whatever client we want, and chat with anybody, regardless of client. As of right now, this is a seriously great start! I’m excited to see where Matrix goes from here.

    Liked by 1 person

Share your thoughts

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.