SEARCH

Accessibility evaluation

Introductory articles

• Getting started with accessibility assessments
  "In this poster paper we will highlight the basic techniques that people can apply to identify many of the accessibility issues commonly observed. Applying a range of simple techniques means that website owners and developers can identify many accessibility issues during the site's development phase and as part of an ongoing quality assurance strategy without having to understand all the technical aspects of how the website works and is coded."
  (Andrew Arch, Sofia Celic, Steve Faulkner & Brian Hardy, AusWeb03)

Discussion articles

• Accessibility evaluators aren't worthless
  "There are a number of web accessibility evaluation tools available to developers. The problem with these tools, the reason some revile them, is that they can lead to a false sense of security among those users who don’t really understand their limitations."
  (Mike Cherim, Accessites.org)

• Accountability in accessibility testing
  "While accessibility testing tools exist, all too often the testing process is incomplete, and many pages which claim conformance to a declared standard fail."
  (WATS.ca)

• A conceptual framework for accessibility tools to benefit users with cognitive disabilities
  "The authors present a conceptual framework which tool developers can use to chart future directions of development of tools to benefit users with cognitive disabilities. The framework includes categories of functional cognitive disabilities, principles of cognitive disability accessibility, units of Web content analysis, aspects of analysis, and realms of responsibility."
  (Paul Bohman & Shane Anderson, WebAIM)

• Checklist on usability testing for accessibility
  A comprehensive checklist for planning, preparing, conducting and reporting on usability testing for accessibility.
  (Shawn Lawton Henry, UI Access)

• Comparing website accessibility evaluation methods and learnings from usability evaluation methods (PDF)
  "Different disabilities affect peoples' use of information and communication technologies. Consequently, there is a need to ensure that websites are accessible to people with disabilities. At the same time resources are required to help designers make websites more accessible. An assessment of usability evaluation methods demonstrates that there a significant alignment between the measures used in both usability and accessibility."
  (Tania Lang, Peak Usability)

• Conducting usability testing for accessibility
  Advice on running usability testing for accessibility including room set-up, orienting the participant and facilitating tests with people who have disabilities.
  (Shawn Lawton Henry, UI Access)

• Evaluating for accessibility
  Advice on incorporating accessiblity into a range of evaluation methods including standards review, heuristic evaluation, design walkthroughs, screening techniques and usability testing.
  (Shawn Lawton Henry, UI Access)

• Evaluating website accessibility: a seven step process
  "Web accessibility is becoming a very critical topic in information technology. Assistive technologies are allowing individuals with disabilities the power to access web content when and how they desire. Most web developers and designers are aware of the issues of web accessibility, but do not know where to begin implementing accessibility. The first step to creating an accessible website is evaluating the current accessibility level of the existing site. This hands-on workshop will teach participants a seven step process for evaluating the accessibility of their current website. Along the way, they will learn the basics of accessible web design, assistive web technologies, and how individuals with disabilities use the web."
  (Jared Smith & Paul Bohman, Technology and Persons with Disabilities Conference)

• Evaluating website accessibility - part 1, background and preparation
  "Many people, web developers as well as website owners, are new to website accessibility and find it difficult to evaluate. This three-part article series is intended to make it easier for non-experts to perform a basic accessibility check. I hope it will be helpful enough to make at least a few websites more accessible."
  (Roger Johansson, 465 Berea St)

• Evaluating website accessibility - part 2, basic checkpoints
  "The checkpoints provided in this article cover certain accessibility aspects that can be tested with automated tools and others that are relatively easy to check manually. A full accessibility evaluation is more thorough and involves more checkpoints, some of which will be covered in the third article in this series."
  (Roger Johansson, 465 Berea St)

• Evaluating website accessibility - part 3, digging deeper
  "In this final article of the series I will explain some aspects of website accessibility that are difficult to test with automated tools and require more time and/or experience to evaluate manually. Some of the checkpoint descriptions in this article assume you have read the first articles, so if you haven't read them, please do so before you continue."
  (Roger Johansson , 465 Berea St)

• Evaluating websites for accessibility
  "A multi-page resource suite that outlines different approaches for evaluating Web sites for accessibility. While it does not provide checkpoint-by-checkpoint testing techniques, it does provide general procedures and tips for evaluation in different situations, from evaluation during Web site development to ongoing monitoring of existing sites. The approaches in these pages are intended to supplement other content management and quality assurance procedures."
  (Web Accessibility Initiative, W3C)

• Evaluating websites for accessibility with Firefox
  "Although awareness of web accessibility has steadily increased in recent years, many web developers are still uncertain about how to evaluate their sites. The relative complexity of documents such as the W3C WAI Web Content Accessibility Guidelines (WCAG) 1.0 and the wealth of - sometimes contradictory - advice found on sites dedicated to the subject can leave developers wondering about the practical aspects of testing. Often developers lack time or training and simply rely on one of the many automated testing tools such as WebXACT, Cynthia or Wave. But even those mechanical checks need to be supplemented (and, to avoid false positives, counter-checked) by manual checks. Based on some of the WCAG 1.0 checkpoints, this article aims to provide a quick outline of how Mozilla Firefox and the Web Developer toolbar can help with these manual checks."
  (Patrick Lauke, Ariadne)

• How to conduct usability evaluations for accessibility
  "The report explains how to conduct usability studies with test participants who use assistive technologies such as screen readers. 40 guidelines to help plan and run usability studies with users who are blind, have low vision, or have motor skill challenges. Includes sample forms that were developed through several iterations and ended up in versions that proved to work well: screening questionnaire for recruiting test participants, facilitator script, satisfaction questionnaire, consent form." Note: report must be purchased.
  (Kara Pernice Coyne & Jakob Nielsen, Nielsen Norman Group)

• Involving uses in accessibility evaluation
  "Web accessibility evaluation often focuses on evaluating conformance to accessibility standards, such as the Web Content Accessibility Guidelines (WCAG). Broadening evaluation to involve people with disabilities helps better understand accessibility issues and implement more effective accessibility solutions."
  (Shawn Lawton Henry, Web Accessibility Initiative)

• Planning usability testing for accessibility
  Advice on planning for a usability test that includes people with disabilities. Includes determining participant characteristics, recruiting participants with disabilities, compensating participants, choosing the best location and scheduling the right amount of time.
  (Shawn Lawton Henry, UI Access)

• Preparing usability testing for accessibility
  Advice on preparing for the usability test for accessibility. Includes ensuring the facility is accessible, setting up and testing the participants' configurations, and becoming familiar with the assistive technology.
  (Shawn Lawton Henry, UI Access)

• Recruiting screener: questions for usability test participants with disabilities
  "A usability test participant recruiting screener is used to determine if a potential participant matches the user characteristics as defined in the usability test protocol. When recruiting participants with disabilities, ask the usual questions about demographics, frequency of use, experience level, etc. Additionally, include questions that address the characteristics related to disability and accessibility defined in the specific usability test protocol."
  (Shawn Lawton Henry, UI Access)

• Screening techniques for accessibility
  "Screening techniques are simple activities to help identify potential accessibility barriers in product designs. Most screening techniques involve interacting with a product with one or more physical or sensory abilities eliminated or modified; for example, wearing thick gloves to limit your dexterity, and wearing low vision glasses or blindfolds to limit your vision."
  (Shawn Lawton Henry, UI Access)

• Suitability to task of automated utilities for testing web accessibility compliance
  "We believe that automated utilities that canvass individual web pages, web-based applications, or other software applications can improve the efficiency of accessibility-related heuristic evaluations. Specifically, they are valuable at identifying barriers to accessibility, but they currently do not and potentially cannot address accessibility itself."
  (Bill Killam, Society for Technical Communication)

• Testing with screenreaders: a hypothetical conversation
  "I decided to take a little bit of a different approach in this article than usual. This article takes a more conversational approach in the form of questions and answers. This conversation is hypothetical (which means that I made it up), so there is no real "questioner' or 'answerer' other than myself. Still, the ideas in this imaginary conversation are based on real-life conversations that I've had with other people on different occasions as we've talked about using screen readers for testing the accessibility of web content."
  (Paul Bohman, WebAIM)

• The importance of human evaluation
  "The two basic approaches to accessibility evaluation are: 1. Use a software tool, 2. Use a human evaluator. Usually the best approach is to use both a software tool and a human evaluator. Each approach has strengths and weaknesses which complement the others and form a more complete approach to Web accessibility evaluation. People with disabilities can be especially valuable as accessibility evaluators, though this approach also has weaknesses."
  (WebAIM)

• The problem with automated accessibility testing tools
  "An automated accessibility tool is a piece of software which can test a web page, or even an entire website, for accessibility. Automated accessibility tools are useful because they can save you a huge amount of time. But are these tools a little too good to be true? Can you test a website for accessibility so easily? Unfortunately the answer is a resounding no. There are a number of underlying problems associated with using just automated tools to test for accessibility."
  (Trenton Moss, Webcredible)

• Toward user-centred, scenario-based planning and evaluation tools
  "Existing Web accessibility evaluation tools are only capable of providing feedback within the context of individual Web pages. This short-sighted approach produces a fractured and fragmentary assessment of the accessibility of the Web site as a whole. A more effective, holistic alternative is to focus on scenarios of user interactions across Web pages, taking into account user characteristics, and focusing on models and patterns."
  (Paul Bohman & Shane Anderson, WebAIM)

• Usability testing for accessibility
  "Usability testing provides quantitative and qualitative data from real users performing real tasks with a product. Usability professionals can evaluate accessibility by using standard usability testing protocols, with a few modifications for including participants with disabilities."
  (Shawn Lawton Henry, UI Access)

• Using combined expertise to evaluate web accessibility
  "Evaluating the accessibility of Web content for people with disabilities requires diverse kinds of expertise and perspectives. While it is possible for individuals to evaluate Web accessibility effectively if they have training and experience across a broad range of disciplines, it is less likely that one individual will have all the expertise that a collaborative approach can bring."
  (Judy Brewer, W3C)

• Using the AIS web accessibility toolbar
  "The Web Accessibility Toolbar is a free extension for Internet Explorer (version 5 and above, Windows). It contains many features that can help in the assessment of the accessibility of web pages."
  (Steve Faulkner, WebAIM)

Research articles

• Accessibility evaluation practices - survey results
  "This is a summary of the results of an online survey on accessibility evaluation practices conducted in late 2004. Web accessibility practitioners were invited to participate in the survey via a posting to several key mailing lists. There were 98 respondents, with the majority from the IT&T and education sectors whose roles involve less than 50% accessibility-related work. Manual inspection was the most popular evaluation method used, and user testing the least popular. Most evaluations were based on the W3C's Web Content Accessibility Guidelines (WCAG)."
  (Dey Alexander, Dey Alexander Consulting)

• A comparative assessment of web accessibility and technical standards conformance
  "This paper presents results of a comparative survey of web accessibility guidelines and HTML standards conformance for samples of websites drawn from Ireland, the United Kingdom, France and Germany. It also gives some recommendations on how to improve the accessibility level of web content. A particular conclusion of the study is that the general level of web accessibility guidelines and HTML standards conformance in all of the samples studied is very poor; and that the pattern of failure is strikingly consistent in the four samples. Although considerable efforts are being made to promote web accessibility for users with disabilities, this is certainly not yet manifesting itself in improving web accessibility and HTML validity."
  (Carmen Marincu & Barry McMullin, First Monday)

• A comparative investigation of the accessibility levels of Irish websites
  "In 2002, Dr. Barry McMullin carried out the Web Accessibility Reporting Project (WARP), a baseline study of the accessibility of Irish websites (McMullin, 2002). This dissertation uses the same sample of websites and assesses their accessibility and compliance levels in 2005. In addition, the research addresses the limitations of the WARP study by examining the ‘manual checks'. Verification by disabled users is also a priority. The research includes building an AAA accessible dynamic website whereby disabled users can comment on and rate the websites which claim to be accessible."
  (Vivienne Trulock, ilikecake)

• Assessing the accessibility of fifty United States government web pages: using Bobby to check on Uncle Sam
  "This study evaluates the current accessibility of US Government web pages for people with disabilities. Several federal laws, and specifically Section 508 of the US Rehabilitation Act, require web pages of government agencies to be accessible to people with disabilities. This investigation built on past studies that used the web accessibility evaluation tool Bobby to assess various types of websites. The home pages of fifty US government agencies were reviewed for accessibility based on Section 508 guidelines. This study establishes that the US government has not met its accessibility goals."
  (Jim Ellison, First Monday)

• FTSE 100 websites fail accessibility requirements
  "Research conducted in March 2006 by user experience experts Nomensa, shows that almost 75 percent of businesses in the FTSE 100 list of companies fail to meet the minimum requirements for website accessibility. The homepages of each website were measured and evaluated, using manual testing for the first time, against the globally recognised Web Content Accessibility Guidelines (WCAG). Only 24 sites achieve the minimum level of accessibility – and none go beyond that to double A or triple A standards."
  (Nomensa)

• How accessible are Australian university web sites?
  "This paper reports on a recent study of the accessibility of Australian university web sites. A selection of key pages from all 45 Australian tertiary education web sites were analysed to assess their compliance with basic accessibility standards, as required by Australian anti-discrimination legislation. The results--98 percent of sites failed to comply--suggest that a radical rethink is required. Web accessibility is poorly understood by university web publishers, and procedures are not in place to ensure that university web sites provide equitable access to important online resources. As a result, Australian University web sites are likely to present significant barriers to access for people with disabilities."
  (Dey Alexander, Ausweb03)

• Library web accessibility at Kentucky's 4-year degree granting colleges and universities
  A survey of 31 academic library web sites in Kentucky in August and December 2003 found that only one institution, or 3 percent of those surveyed, demonstrated the use of valid HTML and passed all automated checkpoints.
  (Michael Providenti, D-Lib Magazine)

• UK accessibility investigation of 1000 web sites - results released April 14th 2004
  "An investigation of 1000 UK Web sites carried out on behalf of the Disability Rights Commission (DRC) reveals unacceptably poor (in fact woeful) accessibility. At least 81% of sites failed to meet the minimum accessibility standard, and this figure is likely to be much higher."
  (Jon Dodd, UPA Voice)

• Using automated tools to improve website usage by users with diverse abilities (PDF)
  "The World Wide Web plays an important role in our society, enabling broader access to information and services than was previously available. However, website usability and accessibility are still a problem. Numerous automated evaluation and transformation tools are available to help web developers build better sites for users with diverse needs. A survey of these automated tools is presented in the context of the user abilities that they support. Furthermore, the efficacy of a subset of these tools based on empirical studies is discussed, along with ways to improve existing tools and future research areas."
  (Melody Y Ivory, Jennifer Mankoff & Audrey Le, IT & Society)

• Web accessibility revealed: the Museums, Libraries and Archives Council Audit
  "In 2004, the Museums, Libraries and Archives Council (MLA) commissioned a Web accessibility audit from City University London. MLA is the national development agency working for and on behalf of museums, libraries and archives in England and advising government on policy and priorities for the sector. The audit was inspired by a study conducted by City University London in 2003/2004 on the accessibility of 1,000 general Web sites for the Disability Rights Commission (DRC)"
  (Marcus Weisen, Helen Petrie, Neil King & Fraser Hamilton, Ariadne)

• Website accessibility and the private sector: disability stakeholders cannot tolerate 2% access
  This study of over 1000 non-government websites in the US reveals that 98.24% of sites were not accessible. It was revealed that web designers were not knowledgeable of the exact techniques for making their site fully compliant (17.40%), and web designers felt that persons with disabilities were not a part of their target audience (42.07%).
  (Ronald E Milliman, Information Technology and Disabilities)