What does it mean to do an accessibility audit?

I got a request recently to price out an accessibility audit of a several thousand-page site. The expectation seemed to be that the entirety of the site would be reviewed in some fashion and then the site would be certified in some way to be in compliance and could then go live. I want to take a moment to unpack some of these assumptions and explain what it means to review for accessibility.

Is it possible to review thousands of pages for accessibility?

Well it is possible, but it would be very time consuming and labor hours would ultimately be significant and thus when hiring out, expensive.

The key point is that when reviewing for accessibility, while there are a lot of great automated accessibility tools out there, the best the tools can do is identify some problems clearly. For a number of potential accessibility concerns, the most these tools can do is flag potential areas for subsequent manual review. And with some issues, the tools can’t do even that.

Advice: By all means, consider running an automated tool that lets you flag potential issues and concerns but know that this doesn’t replace a manual review.

Apps and interactive elements may not be reviewable by automated tools.

In many cases, what needs to get reviewed are interactive elements, such as where a user needs to make a selection or input information of some kind and then the system outputs or further directs the user to a next step. And in other cases, what needs to get reviewed is an application. In these instances, the automated tools generally won’t work at all, so again we’re back to doing a manual review.

Advice: When dealing with apps and interactive elements, a manual review is not only important but the only way to do any kind of review.

How long does a full manual review take?

While it is conceivable to manually review an app or website with a limited and finite number of screens (or a selected new portion if the original product was already accessible), for sites and apps with an extremely large number of screens, the review likely needs to be a sampling.

As a rule of thumb, I assume a single complex, interactive and unique page could take several hours to review, report on and debrief with the development team. Subsequent similar pages will typically take much less time, largely dependent on how much is unique about those similar pages and the expected thoroughness of the review and reporting on a page by page basis.

Advice: Determine what the unique templates, styles and interactive elements are for review and then review a sampling of each type of unique element from across the site.

What about the pages or screens that aren’t reviewed?

While I am not a coder myself, I find that the reviews of unique elements often identify coding flaws that can then be identified elsewhere and corrected everywhere. Sometimes problems are not related to code but to the input of information. For example, images are put into a content management system, and those who facilitate the image input are instructed to tag the images as “purely decorative” to bypass an alternative text requirement, or the alt text is supplied by the system as the image name. Then the team knows to go back and seek out these consistent problems throughout the site.

Advice: When a thorough review of every page is not realistic, do consider at least taking a peek at other pages that are out of scope to help validate what recurring themes might be.

Is there a consistent standard for accessibility reviews?

Yes, it’s the W3C Web Content Accessibility Guidelines known as WCAG 2.0 (soon 2.1). In the United States the most common other set of guidelines is known as Section 508, which is exclusively geared for government sites. The truth is that Section 508 and WCAG 2.0 have been in harmony for the past few years so they are more or less the same set of guidelines.

Advice: Regardless of whether you are hiring for accessibility evaluation or doing it in-house, understand the framework for accessibility compliance by becoming at least somewhat familiar with WCAG.

Can something be certified as accessible and/or be approved by certified testers?

There is no universally accepted external certification for sites/apps or testers. That said, within certain parts of the US Federal Government, there is the DHS trusted tester program, but this doesn’t mean much outside of Federal government pages. There are some external organizations that have come up with specific bodies of knowledge and a test, but this doesn’t mean too much either beyond indicating that someone is knowledgeable about WCAG.

There are also several accessibility organizations that say they will “certify” a site as accessible after reviewing it, although this doesn’t really mean anything beyond the fact that they reviewed it and confirmed that problems were remediated.

A VPAT is sometimes necessary when dealing with the US Federal government. This is self-certification where an organization tells government users/purchasers that they believe their site to be fully accessible.

Advice: Focus on finding knowledgeable people to do a quality review not on any kind of certification of the testers or the output.

Is perfection the goal?

The likelihood is that a large site will end up mostly accessible, but not absolutely perfect. Fix every item you can, making sure that all the intended functions can be used across assistive technologies but know that absolute perfection is hard to achieve.

Advice: There will always be accessibility hiccups that just happen, so do your best to stay on top of as much as you can on an ongoing basis and stay on top of user feedback about when these minor glitches do occur.

Where should you start?

The best possible thing you can do before development even begins is train the development team in the principles of accessibility. Get them producing sites and apps that are mostly accessible from the start and continue to be accessible on an ongoing basis as things change.