I had this exact issue with the Elgato StreamDeck user interface, which renders the macro buttons as one big canvas only drag-and-drop interacts with.
It's rare that I outright cannot figure out a way to make a UI accessible, but that one managed to break everything I tossed at it. I eventually switched my effforts to an open-source third party client which had a web UI, which is now accessible to screen reader users.
As for repairing HTML, my last few engagements have clearly shown me that while HTML CAN be repaired, many devs these days wouldn't know where to even begin to do that. The amount of hand-holding I've had to do has made me suggest, several times, to just give me git access as it'd be faster if I just fix it myself.
Doesn't work for security, not for accessibility, not usually for localization and internationalization, it doesn't work for bandwidth optimization... yet we all keep making UIs that really only work on 300 inch Apple Studio displays and screw anyone who isn't Murrican, rich, in a place with good internet, able-bodied and preferably willing to fork over all their personal data, voluntarily or involuntarily.
There's a reason people burn out doing accessibility work, it's incredibly draining to have to swim against the current as much as we do, and actually HAVING a disability compounds that problem. We know how to do it best because we live this life, yes, but we also take the brunt when someone up top decides forking out a 3000000 dollar bonus to the CEO is more important than actually enabling users to use the product.
I'm not a hard person to find. My hackernews handle into a search engine gets you myriad ways to contact me (Mastodon, LinkedIn, etc.) but I'm at florianbeijers [at] google's mail service if email is your preferred mode of communication. I have a website at florianbeijers.xyz with a contact form, its just very much a placeholder and not really currently meant for public consumption until I revamp it
I always find it interesting to see that while we watch people struggle to get through a UI , unable to see the forest through the trees while they just want to get done with the task they, when they eyeball it, should only take five minutes but takes five hours instead, we get blinded by talking about the use of the word blindspot or the order in which we should say visual, impairment and person. meanwhile the UI never gets fixed and the cycle repeats when the next person shows up. As a blind person myself, I truly don't see the problem with using visual terminology given we don't live in a value, and while I understand some people go looking for a fight after the 30th inaccessible mandatory form they have to fill out that day, I've always preferred a more measured approach.
Honestly the fact that this article still needs to be written in 2026 is the designated real-big-sad imho.
Screen readers almost entirely ignore the visual layer of any UI, and are entirely dependent on the layer that most developers ignore because it's not the visual layer. It's a perfect storm.
It stands to reason that someone who's actually used to using a screen reader should be brought in to verify what you've built actually works well for that target audience.
I'm a fully blind accessibility auditor, remediator and trainer myself, but I wouldn't dare to assume to know how a mobility-impaired user using eyegaze tech fares on a website I've audited.
My eyes don't gaze, so I don't have the context to make those calls.
On that note: I'm looking for work, anyone need me to tell them how their UI is bad for accessibility and fix it for them so they don't get sued later? :P
No. There are certainly AI-generated accessibility solutions for single, existing applications, games and websites but no outright AI-powered screen reader. I'd think the token cost alone would make such a project prohibitively expensive, although one-off AI features are starting to sneak into JAWS and NVDA as we spaek, so who knows.
The solution, that accessibility advocates have been clamoring for for decades now, is to s"shift left".
If you know you're going to add accessibility, which ... we have had WCAG since 2005, not knowing that at this point is negligence imho, just make sure you work with frameworks and libraries that won't require overhauling all the things when the PO or management finally get sued into letting devs actually implement it properly. If that kind of functionality takes a backseat to "stunning" and "beaituful" designs that a bunch of people can't use, we take the user out of user interface.
Honestly as a blind person and blind developer myself, most of these features get a shrug at best.
For one, there's already a bunch of third-party apps that do most if not all of this (Seeing AI, Envision AI, BeMyEyes, Aira, etc.). So at best, this does what all those apps are doing but faster and on-device, which may or may not mean it is also more inaccurate, we'll have to see.
In the meantime, Mac OS's screen reader, VoiceOver, has been left to essentially exist in maintenance mode for years, where users have had to build, arguably impressive, third-party solutions to add features to the thing that comparable screen readers on Windows have had for a really long time.
Through that lens, this all looks a bit performative to me, but again, maybe I'll be pleasantly surprised.
The one thing I'm mildly excited to see is the improvement to Voice Control, as guessing what the programmatic name of a button is or having to constantly use a numbers grid to target elements doesn't sound fun.
To respond to what I see in some of the comments:
- On speech rate: It does take quite a bit of practice to crank up the speech rate and there's a degree of retraining you need to do when you switch voices. A lot of more "human" sounding voices are harder to follow at super high speeds which is why a lot of people prefer more robotic but consistent speech and generally aren't convinced by AI-powered TTS yet; they often fall apart if you raise the speech rate past a certain point.
- Re: actually waiting for the target audience's verdict: This is so important. I see more and more companies, individuals etc. talk about accessibility, build accessibility solutions and evangelize AI for accessibility without EVER talking to the people they claim to help.
This will almost certainly mean mistakes will be made, up to and including doing more harm than good.
If you want to do accessibility right, that includes AI products of any kind, hire people with lived experience or you'll get the equivalent of machine-translated text, hackerproof security in one click or an AI-powered coffee bar that orders thousands of rubber gloves.
Coincidental note: I have time for new projects right now :P
I am ... relatively sure JAWS reads out title attributes of images because people kept erroneously sticking important info there decades ago, I wouldn't say that's a generally accepted recommendation. Not entirely sure what NVDA would do with an image that has only a title but no alt set.
'Honestly this is a people problem more than a tech problem. We have the tech. We're just not using it.
I'd say LLMs COULD make it easier to implement accessibility, it also couldn't, always a coinflip with those, but I'd say LLMs actually succeeding is probably unlikely given how much shitty code is probably in its training data :P
The tricky thing with these "unofficial" distros is that they are generally maintained by either a single individual or a small group of people.
This is true for many accessibility projects actually (game mods, third-party UIs for inaccessible services/platforms, etc.).
These are generally really meant as short-term patches while the problem gets fixed, except ... the problem often doesn't get fixed because the platforms in question figure it's been solved now and they don't need to care about it anymore.
Accessibility really only works when it's an ongoing, first-class process within an app/platform's design, and we can absolutely do that; the standards and guidelines have existed for decades. People working in cybersecurity, localization, general UX should recognize this song and dance, which is amusing because a lot of the tools of those trades have atrocious accessibility and require all sorts of workarounds, ask me how I know.
People just ... aren't including it in this way, which means people like myself (screen reader user and accessibility professional) essentially have to keep reminding people that we exist and that it's kinda shitty to keep forgetting about that fact or to decide the least amount of effort possible (LLM, unpaid volunteer, send in a PR LMAO) is enough to cater to people who have very real, very annoying and very constant UX issues we either crash into or crash through on literally an hourly basis.
What are you basing that on? Screen readers tend to not pick those up at least on interactive elements by default, you need to do a bit of "wiggling" to get those to be announced.
Disclaimer: screen reader user
From the article I'd assume this wouldn't work in any way for mobile given no hover, not for screen reader users because a website often has no idea where a screen reader's cursor is, and potentially not for keyboard users (haven't checked if keyboard focus triggers this prefetch/prerender or literally just mouse hover), so ... limited applicability, I'd say.
This is a really cool concept :) Generally in language learning it's a good idea to try to "think" in the language as much as you can so you get creative about how to phrase something with the words you do have, but if that fails, particularly when you're just starting out this is a great motivational aid and conversation starter if nothing else :)
I would definitely recommend integrating with an established translator like DeepL as well as a kind of second correction as AI does get things wrong and having the two versions can help compare what may have gone wrong, you can probably keep your autodetecting language bit as well as I am pretty sure DeepL supports language identification and if not, I know Google does.
Good luck with learning Danish, I tried my hand at it a few years back but have switched to Finnish as of early this year
I'm sure it's fast, but is it accessible? Am I as a screen reader user going to get fired if the company I work at decides one day to have all of their devs use this? And if not, any plans to make it so?
Might sound a little brusque but this really is the stakes we're playing with these days.
I love this idea, it would actually solve a lot of accessibility issues within coding courses for the fully blind. Unfortunately right now the scrimba interface appears to need some help where that is concerned. WOuld love to discuss more if you're interested? @somebee
You're not wrong but I think there is absolutely a case to be made for these kinds of restructurings as well. For one, giving you the restructured version exposes you to different ways of stating the same content, it might make reading longform content (without falling back to the original source material too often) more fluid, and more situationally, this kind of simplification and restructuring actually happens really often in subtitling, where character count is more leading than the lip movements a dub is based on. e.g. you'll hear one thing, but the sub is that same content often radically redone :)
I'll check that out :) I'd be curious how well this would work without being able to see the video you're messing with (fully blind myself). Will give that a whirl.
Honestly this looks cool but setting it up is a bit much. Like... I think the idea of this is cool, but by the time I've done all this setup I've more than likely forgotten what it is I was trying to do to begin with, and even when most of it's already installed I need to make sure several things are running etc. before this becomes essentially a link in the chain. That feels to me like it's going against the ease of integration the project is meant to facilitate a tiny bit. Like somebody else already stated in this thread, having a bit more of a ready-to-go solution, stick it in docker if you must, would probably help here.
I had this exact issue with the Elgato StreamDeck user interface, which renders the macro buttons as one big canvas only drag-and-drop interacts with.
It's rare that I outright cannot figure out a way to make a UI accessible, but that one managed to break everything I tossed at it. I eventually switched my effforts to an open-source third party client which had a web UI, which is now accessible to screen reader users.
As for repairing HTML, my last few engagements have clearly shown me that while HTML CAN be repaired, many devs these days wouldn't know where to even begin to do that. The amount of hand-holding I've had to do has made me suggest, several times, to just give me git access as it'd be faster if I just fix it myself.