When accessibility testing involves a screen reader, one detail often gets overlooked: the result depends on your code and also on the specific combination of screen reader and browser you're using. Different combinations produce different output.
Why combinations matter
Screen readers talk to browsers through accessibility APIs, interfaces that expose the structure and state of a page. These APIs aren't implemented consistently across browser and operating system combinations, and each screen reader handles that inconsistency in its own way.
NVDA, JAWS, VoiceOver, Narrator and TalkBack each have their own logic for querying these APIs, handling missing or ambiguous information, and turning it into audio. Browsers also differ in how completely they implement the APIs in the first place.
So the same HTML, ARIA attributes and JavaScript can produce entirely different experiences depending on the user's screen reader and browser.
A modal that announces its role correctly in NVDA with Chrome may not announce it at all in JAWS with Internet Explorer, or may announce it at the wrong moment in VoiceOver with Firefox. A live region that works as expected in one pairing may fire several times, fire too early or never fire in another.
What this means in practice
Testing with a single combination gives you a result that holds for that pairing and may be false for every other one. That's a reason to broaden manual testing, not to drop it.
The WebAIM Screen Reader User Survey, published regularly, shows that real users are spread across several screen readers and browsers, and that the most common pairing varies by region, platform and user need. No single combination represents your whole audience.
The gap automated tools cannot fill
Tools like axe, Lighthouse and browser extensions test the structure of the accessibility tree. They can tell you whether an element has an accessible name, whether a role is present and whether contrast ratios meet the threshold.
They can't tell you whether a screen reader will announce that information in a way that makes sense to a user. For that you have to listen, with the actual tool, in the actual environment.
Manual testing across combinations is the only way to close that gap, and it doesn't need to be exhaustive to be useful. Two or three well-chosen pairings will surface issues no automated tool would catch.
Which combinations to prioritise, and why, is the subject of my next post.