Accessibility Metatesting

Reviewed by Greg Wilson / 2023-04-26
Keywords: Accessibility

Jonathan Robert Pool. Accessibility metatesting: comparing nine testing tools. 2023. arXiv:2304.07591, doi:10.1145/3587281.3587282.

Automated web accessibility testing tools have been found com- plementary. The implication: To catch as many issues as possible, use multiple tools. Doing this efficiently entails integration costs. Is there a small set of tools that, together, make additional tools redundant? I approach this problem by comparing nine compre- hensive accessibility testing tools that are amenable to integration: alfa, axe-core, Continuum, Equal Access, HTML CodeSniffer, Nu Html Checker, QualWeb, Tenon, and WAVE. I tested 121 web pages of interest to CVS Health with these tools. Each tool only fraction- ally duplicated any other tool. Each discovered numerous issue instances missed by all the others. Thus, testing with all nine tools was substantially more informative than testing with any subset.