A11yOps: Automated digital compliance for accessibility

Accessibility is becoming a priority for many organizations. According to Forrester’s 2022 Predictions, $10 billion in design spending will shift to vendors and services that can demonstrate a commitment to it. This means that, today, building your expertise in accessibility isn’t just a differentiator, it’s a commercial necessity.

Traditionally, areas like legal, finance, security and data protection have been viewed as the critical elements of corporate compliance. Accessibility must now be added to that list: the consequences of not addressing it could also have regulatory implications; it is quickly becoming a legal requirement in many parts of the world. The European Accessibility Act, for instance, comes into effect in June 2025. As with GDPR, organizations need to be preparing for this change now.

This will be challenging for many organizations. However, there are resources and support available. The best starting point for accessibility compliance are the Web Content Accessibility Guidelines (WCAG), published by the World Wide Web Consortium (W3C). WCAG contains success criteria organized into three levels, from what could be regarded as loosely accessible to highly accessible: Level A, Level AA, Level AAA. Websites that conform with WCAG 2.1 Level AA can be considered reasonably accessible for most users. 

Thoughtworks can also support and guide organizations towards greater accessibility. We’re passionate advocates for a more equitable tech future and see accessibility as a key element (alongside sustainability, diversity and inclusion) towards building it. Resources like our responsible tech playbook also offer a diverse range of ideas, tools, and frameworks to help integrate considerations around accessibility into your thinking as an organization.

Indeed, accessibility isn’t important simply because of the regulatory risks that come with a failure to comply. There are many other benefits:

• Better search engine positioning, since accessibility practices overlap with the best practices of search engine optimization (SEO)
• Increase in revenue — Lost Market shares $10 billion in design spending will shift to vendors/services committed to accessibility in the US and Canada alone.” (Forrester 2022: CX)
• Improve loyalty - 68% of global consumers would remain loyal to a brand if the organization practiced social responsibility - gwi.com
• Reduce reputational risks 

Understanding the scope of the potential benefits of accessibility can be particularly useful when you’re trying to get internal buy-in for investment in accessibility-related initiatives.

How can we integrate accessibility compliance into product strategy?

Accessibility is clearly important. But how do we build compliance into our product and design strategy? We think a good way to begin is by focusing  on the three questions below:

• Who is using our products and services? (And how?)
• Are users satisfied with the experience?
• Are we leaving anyone out?

Most organizations are very familiar with the answer to the first two questions; the last question, however, is still not asked and answered enough.

Peter Drucker once said that if you can’t measure it, you can’t manage it. In this case, we would add that if you aren’t measuring it, you’re missing the opportunity to:

• Add a data point in understanding reality

• Improve the decision making process

• Detect change over time (improvement in  outcomes, decline in health)

• Fair comparison across products and services

   

Adopting a Lean approach: start simple 

To make accessibility more practical, we’ll explain how we did it at Thoughtworks. We focused on the digital products that Thoughtworkers use daily. You may approach it in a different way to us and, indeed, your organizational context might be completely different, but we think our process might be useful as a starting point or guide as you apply it to customer- or employee-facing products.

First, we started simple. Every team manually tracked accessibility using Lighthouse, a free Google tool available on every Chrome browser. Results were tracked on a spreadsheet.

Everyone in Thoughtworks, from senior leadership to employees, have access to accessibility scores via Neo Polaris. Product owners in particular benefit from being able to see accessibility metrics alongside other business metrics such as adoption, engagement, performance and delivery. This empowers them use accessibility metrics to make decisions about how they should act and what the next priority for a project might be.

What were the main benefits?

There were three key benefits to this piece of work:

• It automated a bird's eye view on accessibility scores for our internal products
• It increased the organizational awareness of accessibility by making accessibility data available to everyone
• It supported product owner decision making by displaying accessibility data alongside other business metrics

The limitations of automating testing and mitigations

Automated accessibility testing was a big milestone for us. However, there are some limitations. For example, while Lighthouse can identify common WCAG violations it doesn't guarantee that you're complying with WCAG guidelines. There isn't an automatic tool that can cover 100% the automatic validation currently available on the market.

Automated testing also needs to be integrated with the manual testing that is being done by a large and diverse set of users to ensure we're getting an unbiased and accurate understanding of reality. While automated testing can catch visual design and front end code violations (that usually cover 80% of the product), interaction logic and other elements should still be manually tested. Google Lighthouse itself specifies what elements need to be checked manually.

Our first step to mitigate the limitations of automating testing was to implement the accessibility tag in our ticketing system. We are encouraging Thoughtworkers — 12,000+ people — that use our internal software to act as manual testers and provide timely feedback on accessibility. Now we've implemented the tag, accessibility tickets are easy to monitor, track and resolve.

Our measures for success for accessibility of internal products at the moment are:

• Average accessibility score for digital products
• Number of accessibility tickets raised

This the beginning of an exciting journey towards a better equitable future.