What is a scoping report
Scoping Review of the Americans with Disabilities Act: What Research Exists, and Where do we go from Here? Sarah Parker Harris Department of Disability and Human Development, University of Illinois at Chicago E-mail: firstname.lastname@example.org Robert Gould Department of Disability and Human Development, University of Illinois at Chicago E-mail: email@example.com Patrick Ojok Department of Disability and Human Development, University of Illinois at Chicago E-mail: firstname.lastname@example.org Glenn Fujiura Department of Disability and Human Development, University of Illinois at Chicago E-mail: email@example.com Robin Jones Department of Disability and Human Development, University of Illinois at Chicago E-mail: firstname.lastname@example.org Avery Olmstead IV Department of Disability and Human Development, University of Illinois at Chicago E-mail: email@example.com
ADA, systematic review, scoping review, policy analysis
A broad range of research on the Americans with Disabilities Act (ADA) tracks its progress and impact. Much of the research is inconclusive or conflicting, creating a fragmented evidence base about the ADA's effectiveness as a social policy. In response, academic researchers and disability organizations have called for an extensive review of the existing research. To address this fragmentation, the University of Illinois at Chicago has begun a five year project systematically reviewing the ADA research as part of the ADA Knowledge Translation Center at the University of Washington. This article reports results from year one of the project, the scoping review, that will assist in identifying a research plan to inform policy and practice.
The Americans with Disabilities Act (ADA) is the cornerstone of US civil rights policy for people with disabilities, and the primary legislative tool for ensuring their full and equal treatment as citizens. The legislative intent of the ADA is to protect against institutionalized and structural discrimination while simultaneously fostering social inclusion across all domains of public life (National Council on Disability [NCD], 2005). The legal framework of the ADA builds upon early rights legislation including Section 504 of the Rehabilitation Act and the Civil Rights Act of 1964 to create a unified policy that offers civil rights protections to people with disabilities. There is a broad catalogue of literature on the ADA that seeks to understand how the policy has followed the course of this philosophical and legislative pursuit.
Social science research on civil rights policies and movements is often used to monitor implementation goals, and to understand problems in applying policy to practice (Hahn, 1993). The diverse research on the ADA however, has not been fully applied in this manner. Aside from studies on hiring rates and the legal data available from the U.S. Department of Justice, much of the commonly cited research is anecdotal or descriptive and not necessarily useful for many policy players in its current form (Burns & Gordon, 2010; NCD, 2007b). Although there is a vast body of research evidence on the ADA, scholars argue that much of research evidence is not utilized and policy stakeholders are unable to fully assess the ADA's impact (Vierling 2006, Blanck 2006). Significant information gaps about the ADA's impact as a social policy still exist more than 20 years after its implementation.
Contributing to this knowledge gap is the lack of an exhaustive review and analysis of the diverse body of research that analyzes the ADA's impact. Previous reviews and assessments of ADA evidence have concluded that it is vital to conduct a systematic assessment of the full body of existing research to determine what evidence exists that is of use for policy and practice (i.e. Collignon, 1997; Schwochau & Blanck, 2000). However, this call to further systematic assessment has not been met at the national level or in a way that covers the full breadth and depth of the ADA (NCD, 2007a). To date there has not been a comprehensive synthesis of the wide range of research and scholarship for it to be more applicable and useful for various ADA stakeholders.
To address the need for a systematic assessment of existing research, the University of Illinois at Chicago is conducting a five-year review of the ADA as part of the National Institute on Disability and Rehabilitation Research (NIDRR) funded ADA National Network Knowledge Translation Center at the University of Washington. The project was developed in response to a call by NIDRR to "increase the use of available ADA-related research findings to inform behavior, practices, or policies that improve equal access in society for individuals with disabilities." UIC addresses this call through a multi-stage systematic review that includes: (1) a scoping review of how the ADA has been studied to map the literature landscape; (2) a rapid evidence review to refine priorities and analyze selected topics for preliminary assessment of the research; and (3) systematic reviews to synthesize research and answer specific key questions in the area. We will use these reviews and syntheses to create a foundation of knowledge, inform the subsequent policy, research and information dissemination, and contribute to the overall capacity building efforts of ADA Regional Centers. Together, these reviews will identify a future plan for research that spans core ADA research topics, methodological approaches to ADA-related research and potential outcomes to inform policy and practice, and positively impact on ADA stakeholders. To date, no systematic reviews have been conducted in this area. This article details the activities and findings from stage one of the project, a scoping review of the ADA. It discusses the broader implications of the scoping review findings in relation to understanding emerging issues of disability policy and future ADA research.
Background and Challenges to Assessing the ADA's Impact
The significance of the ADA as a social policy is that it provides the most complete structure for advancing civil rights of people with disabilities to date (Yee & Golden, 2001). The ADA's impact on reshaping our system of rights also reflects a monumental symbol of social change. Title I of the ADA provides the law's overall intent and four goals to ensure rights for people with disabilities: equal opportunity, full participation, independent living, and economic self-sufficiency (Americans with Disabilities Act, 1990). In practice, the four goals of the ADA are carried out through the legal framework that seeks to break apart physical barriers and prevent discriminatory attitudes. This framework is indicative of a growing trend in US political systems that reshapes the protection of needs for people with disabilities as a matter of civil rights (Scotch 2001). The creation of the ADA marked a great advancement in disability rights by expanding upon legislation to formulate a cross-issue policy to defend civil rights for people with disabilities across all domains of social living.
Along with the significant social advancement, there has been persistent debate regarding the extent that the ADA has fully achieved its goal of incorporating people with disabilities into the civil rights framework. Although there has been criticism that the ADA has not reached the full population imagined to be within the scope of the framework, disability policy specialists largely agree that the ADA has been successful in enhancing people with disabilities' legal protections since its implementation (Blanck, 2006; Rulli & Leckerman, 2005). Additionally, scholars and stakeholder groups assert that the policy has had relative success in addressing various issues of structural inaccessibility (Travis, 2008; Yee & Golden, 2001). In spite of these advances, there are also arguments that the ADA has fallen short of its implementation goals because the population of people with disabilities continues to be excluded from much of mainstream society and is less likely to have access to various social events, paid labor, healthcare, and education (NCD, 2007a). More telling perhaps is the opinions of people with disabilities themselves. A recent survey of individuals with disabilities conducted by the National Organization on Disability (2010) found that a majority (61%) of people surveyed indicates that the ADA had made no difference in their lives.
The differing accounts of the ADA's overall impact are replicated throughout much of the most cited data on the ADA. For example, a prevailing opinion of the ADA's employment impact was that it may have had a detrimental effect on the employment rate of people with disabilities due to factors such as employer concerns over costs associated with hiring, firing, and accommodation (i.e. Acemoglu & Angrist, 2001; DeLeire, 2000). However, recent data further complicates this conclusion and is used to argue that other confounding economic and political factors are more attributable to the reduced employment of people with disabilities (i.e. Donohue, Stein; Griffin, & Becker, 2011). This disagreement about the ADA's impact on employment provides only one of the most commonly cited examples of the ongoing and unresolved debates amongst research findings. There is similar uncertainty about the impact of the ADA across many of the areas that the law impacts (NCD, 2007b).
There are multiple characteristics about the existing evidence that make it difficult to assess the full impact of the ADA. Some widely noted factors include the lack of persistent and ongoing data collection, the fragmentation of evidence, and the heterogeneity of existing research. The NCD (2007a) explains one of the largest deficits of the current body of ADA research, where "there is a surprising absence of any ongoing, systematic data collection about the ADA from any source, and the result is significant knowledge gaps about many aspects of the impact of the ADA." With these knowledge gaps and the lack of unified data collection mechanism, policymakers and other involved stakeholders must look at alternative sources of information to fully understand the policy's impact. Federal disability policy players (see NCD, 2007b) and researchers (e.g. Hoffman, 2008; Silverstein, 2010) have both called for summative review of existing research to fill in the gaps due to the lack of ongoing data collection.
A second challenge in assessing the impact of the policy is that much of the ADA evidence does not come in the form of longitudinal data but is instead scattered or fragmented across a range of data sources. Evidence on the effects of the disability policy is fragmented in that it covers a myriad of topics, research questions, and stakeholder groups but is not necessarily reduced to directly quantifiable outcomes of the policy itself (see Silverstein, 1999). Research evidence across a gamut of published and unpublished literature can provide vital information to lay foundation for future policy research, but requires comprehensive review and exploration of the relationships between study characteristics and other contextual factors (Rumrill, Fitzgerald, & Merchant 2010). Studies related to ADA research can be found in academic databases, books and book chapters, theses, websites, think tanks, government departments, disability and other stakeholders' agencies, and ADA Regional Center. Each of these data sources present vital information for better understanding the impact of the ADA but have not yet been synthesized to provide a holistic account of the research findings.
A final challenge in reviewing this wide range of data is the heterogeneity of topics and methods used to analyze the ADA's implementation and impact. Common review techniques in social policy
that focus on health or medicine answer very narrow questions and are often limited to studies testing directly observable impacts of a policy or intervention (Littell, Corcoran, & Pillai 2008). However, systematic reviews based on social science research draw on a greater diversity of research questions that use a range of research approaches and interpretation involves more political or value-based ideas than is found with traditional interventionist research (Witherspoon, 2003). This is true for ADA-related research, which consists of studies that address a myriad of areas, including: social movements, rights and participation (e.g. Scotch, 2001); legal and economic issues (e.g. Bagenstos, 2009); socio-political discourse and cultural values (e.g. Batavia & Schriner, 2001; Hahn, 1996); stakeholder perspectives and experiences (e.g. NOD, 2010); and issue/title based topics such as employment (e.g. Silverstein, Julnes, & Nolan, 2005), education (e.g. Colker, 2009), health (e.g. Crossley, 2000), and technology (e.g. McNaughton & Nelson Brye, 2007). The heterogeneity in both method and content of ADA-related research requires an inclusive and innovative approach to research synthesis in order to effectively explore the full breadth and depth of research on the effects of the policy.
This systematic review project seeks to increase the utility of the research and to generate summative conclusions from the existing research evidence. The first stage involves conducting a scoping review of the research evidence. 1 The primary purpose of a scoping review is to provide a broad overview of the current research on a topic, and to document key components of the research in order to identify specific gaps and key research needs based on the existing research evidence. Other scoping reviews in disability provide guidance on how to improve policy implementation and provide summative overviews of existing evidence to direct future research (i.e. Boeltzig, Pilling, Ciulla Timmons, & Johnson 2010; Rumrill et al. 2010).
Systematic review projects of policy research often engage with key experts in the field for guidance and stakeholder input to ensure a thorough and relevant body of research is located (Arksey & O'Malley, 2005). The research team formed an ADA Expert Panel committee to work with the research team for the duration of the project. It includes representatives from the NCD, the Disability Rights Education and Defense Fund (DREDF), the ADA National Network, Mathematica Policy Research Group, the US Business Leadership Network, various universities, and other pertinent organizations. The Expert Panel input is used to develop research questions, refine research topics and priorities, verify synthesis to identify emerging issues and data gaps, and assist with the dissemination and translation of findings across different stakeholder groups.
The research questions guiding the scoping review were developed by the research team in conjunction with the ADA Expert Panel, with member-checking with disability consultants serving as key population informants. The primary research question is: "What English-language studies have been conducted and/or published from 1990 onwards that empirically study the Americans with Disabilities Act?" The secondary questions are:
- What is the source of the information?
- What disability and stakeholder subgroups are represented in the literature?
- What topics and titles are represented in the literature?
- What methodologies and research designs are represented in the literature?
The questions were kept intentionally broad at this stage of the project in order to understand and assess what has been studied.
The final inclusion criteria consists of citations to all records identified as examining the ADA by a literature search using the following parameters: (a) published or dated from 1990; (b) be written in English; (c) have been carried out in the United States; (d) relate to the ADA and (e) be based on published studies reporting the gathering of primary or secondary data or the collating and synthesis of existing information to answer ADA-related research questions. Items that are not included in the initial scoping review are established facts about the ADA (i.e. court-case decisions, technical materials on compliance, general fact sheets), opinion pieces (i.e. by various stakeholders, lawyers or academics), and anecdotal evidence. The inclusion criteria are broad, as the topic of study is broad.
The first stage of the scoping review process involved locating all of the potential sources of ADA research evidence. The process entailed identifying key databases to access pertinent research records and locating, screening and selecting records based on the inclusion criteria. These databases include a wide range of academic journals that were potentially relevant to the research project. Additional steps were taken to locate the full scope of ADA research, which includes a variety of difficult to locate and grey literature. Due to the nature of grey literature (not published by mainstream publishing clearinghouses, independently published records/reports from agencies, or fully unpublished), alternative search methods were also employed drawing on several different search strategies. These methods include utilizing the primary researchers' knowledge and familiarity with the research evidence; communicating with the network of researchers involved with ADA research; cross-checking bibliographies; and collaborating with the Expert Panel to locate hard-to-reach-studies and confirm saturation. Key organizations were contacted and the expert panel members submitted relevant research from their respective constituent groups. Using these different strategies, the initial search criteria yielded 34,599 records. This included:
- 26,371 records from Academic databases.
- 6,975 records from the Worldcat library of grey literature items.
- 1,253 records provided from the National Rehabilitation Information Center (NARIC). 2
Records were scanned for initial eligibility and key article information (i.e. journal, title, date, authors, abstracts) was saved to an online bibliographic citation management system (RefWorks). The initial exclusion criteria was used at the point to exclude all research that did not state a research purpose directly "related to the ADA" as dictated by our original search parameters. We located all potentially relevant research by screening the title, abstract, and introduction/methods section if necessary.
The initial screening and selection process reduced the number of records from the initial 34,599 to 3,351 records that were identified as potentially relevant ADA research that required data extraction (see next section). This number included both retrieved academic literature and grey literature that had been preliminarily identified as potentially meeting the initial inclusion criteria. Also included in this group were legal documents (i.e. legal or policy analysis that indicated an ADA-related research question) and books/other grey literature that had been scanned but not yet fully vetted for inclusion/exclusion.
Two secondary reviewers conducted a final screening and examined the 3,351 records for meeting the full inclusion criteria. At this point, the remaining records were screened for inclusion based on the criteria that they were published or unpublished studies reporting the gathering of primary or secondary data or the collating and synthesis of existing information to answer ADA-related research questions. Duplicate records and records that did not posit research questions directly related to the ADA were excluded at this point of the screening process. At this point 960 of the 3,351 records were excluded because they did not answer ADA-related research question by gathering primary or secondary data or collating and synthesizing existing information. Also at this point of the screening legal analysis that fit outside the scope of this project and organizational or governmental reports that spanned multiple topic areas were saved for future reviews because they could not easily be coded due to their expansive coverage of multiple ADA issues. These records do contain relevant data and information regarding the ADA that may be useful in the later stages of this project (See limitations section). 1,417 legal research records and 37 organizational reports were saved for future review at this time. After this final screening, 980 records were confirmed as meeting the criteria for inclusion in the scoping review. These 980 records spanned across various grey literature sources (primarily unpublished dissertations and student papers) and across 321 different journals that revealed relevant literature.
Figure 1 below provides a visual representation of the decision processes that were used to finalize the selection of 980 pertinent research records.
From the selected 980 potentially relevant records we proceeded to the data extraction stage of the scoping review process. Key information points from the academic record abstracts were electronically extracted and entered into a formal spreadsheet document. This included basic record information, as well as information related to the research questions (e.g. topic of research, stakeholders involved, research design, etc).
After data extraction a list of coding categories and sub areas was developed collaboratively by the research team and input from the Expert Panel (see Appendix A for a full listing of the codes). Codes were assigned to each record primarily from quotations within the abstract (and/or introductions if insufficient detail was provided in the abstract) and the records were then placed into categories. The initial listing of potential codes was refined based on a secondary review of the initial extraction and expert panel input. Additional codes were generated directly from the quotations and summary statements created during the first stage of the data extraction process that could be used to categorize and sort the different research records. When quotations were unavailable, summary statements or descriptive keywords were created by the reviewers and used to generate appropriate codes for the research record. This coding process, often referred to as mixed coding, involves building from pre-defined concepts about a topic area to explore other potential topic areas that emerge from the literature (Gough, Oliver, & Thomas, 2011). Using the coding sheet, a second reviewer re-examined the initial data entry and cross-checked it against the abstract. This process allowed for an abbreviated check to enhance the inter-rater reliability of the initial data entry and to confirm the categorical codes.
The final stage of the scoping review process is descriptive analysis and synthesis of the data. This was completed using a descriptive numerical summary (e.g. overall number of studies included, types of study design, topics and/or titles studied, characteristics of disability sub-groups and/or stakeholders, years of publication). The frequency that different research design elements occur in ADA research was calculated to provide an overview of the most common approaches to studying the ADA.
The final search yielded 980 separate research records on the ADA. The results have been descriptively analyzed and synthesized into the following categories: record type, stakeholder groups, topics, and research methods. The following section provides results and details of the records that met all of the scoping review inclusion criteria.
Record type refers to the source of the literature and the different types of records that were gathered (Refer to Figure 2). The main source of literature that met the inclusion criteria was academic journal articles, specifically the Journal of Vocational Rehabilitation (31 articles); Work: A Journal of Prevention, Assessment, and Rehabilitation (26 articles); the Journal of Disability Policy Studies (26 articles); the Journal of Rehabilitation (17 articles); and the American Journal of Occupational Therapy (20 articles) where the majority of ADA research has been published to date.Source: dsq-sds.org