Publications

Focus Group Reactions to Three Definitions of Reading (As Originally Developed in Support of NARAP Goal 1)

Prepared by:

Frederick Cline, Christopher Johnstone, and Teresa King

July, 2006

Please use the following citation for this paper:
Cline, F., Johnstone, C., & King, T. (2006). Focus group reactions to three definitions of reading (as originally developed in support of NARAP goal 1). Minneapolis, MN: National Accessible Reading Assessment Projects.

Introduction

The National Accessible Reading Assessment Projects (NARAP) is a collaborative effort that is conducting research to make large-scale assessments of reading proficiency more accessible for students who have disabilities that affect reading. There are two projects that comprise the NARAP partnership, Designing Accessible Reading Assessments (DARA) and the Partnership for Accessible Reading Assessment (PARA). DARA and PARA are working together and independently on research studies in support of the NARAP goals.

NARAPs Goal 1, which the two projects worked on together, was to craft a definition of reading to support the development of a valid reading assessment for all students, including those with disabilities that affect reading. To this end NARAP formed a Definition Panel that included 15 national experts who had served on previous reviews or reading definition panels or who were experts in special education and various areas of disabilities research and education. The charge given to the panel of experts was to draw on the existing research base and five national reports to craft a definition of reading and reading proficiency that would press previous definitions and serve as the basis for the development of high quality reading assessments that are accessible for all students.

The panel had a single face-to-face meeting in January 2005 that resulted in a high-level definition of reading (Definition A). A revision of this definition (Definition B) was constructed after the face-to-face meeting by a subcommittee initially charged with fleshing out the initial definition to include reading proficiency or proficiency statements.

Definition B was presented to the full Definition Panel as an alternative to Definition A. During the e-mail discussion of the definitions, the majority favored the original Definition A. However the Panel did not reach consensus and thus did not move forward with the next steps of creating a definition of reading proficiency or proficiency statements for 4th and 8th grade students. Concerns raised about the two definitions stemmed from multiple perspectives about important components that individuals thought should be in the definition, including: (a) the role of decoding and comprehension or developing understanding about what is read, (b) what constructs define the act of reading vs. instructional aspects of reading such as end goals and purposes for reading as a component along with decoding and comprehension or understanding, and (c) the impact of various disabilities on how reading is defined.

NARAPs Executive Committee studied the work of the Definition Panel and sought to find a compromise between the two definitions. They offered Definition C as a third alternative. The three definition statements that were drafted are:

Definition A

Reading is decoding and understanding written text. Decoding requires translating the symbols of writing systems (including braille) into the spoken words they represent. Understanding is determined by the purposes for reading, the context, the nature of the text, and the readers strategies and knowledge.

Definition B

Reading is decoding and understanding text for particular reader purposes. Readers decode written text by translating text to speech, and translating directly to meaning. To understand written text, readers engage in constructive processes to make text meaningful, which is the end goal or product.

Definition C

Reading is the process of deriving meaning from text. For the majority of readers, this process involves decoding written text. Some individuals require adaptations such as braille or auditorization to support the decoding process. Understanding text is determined by the purposes for reading, the context, the nature of the text, and the readers strategies and knowledge.

Obtaining input on the working definitions from many stakeholders with a variety of viewpoints on reading was an important step in support of Goal 1. Given the need to allow for many potential reactions to the definitions we decided that the open dialogue found in focus groups would best serve the feedback process.

There were two main purposes to the focus groups. One purpose was to provide feedback to the Definition Panel from members of the reading, disability, and educational measurement communities. A second purpose was to provide evidence of support from those communities for the definition that would be adopted, as stated as a requirement in the initial grant proposal. In addition to these primary purposes, the NARAP project also hoped that the focus groups would help to develop interest in the NARAP project by engaging members of the reading, disability, and educational measurement communities.

Method

Two types of focus groups were held -- traditional (face-to-face) and teleconference with Web support. DARA and PARA worked together to develop the basic protocols and collaborated on planning and recruitment. DARA took the lead on the traditional, face-to-face focus groups and PARA took the lead on the teleconference and Web-based sessions. The organizations on the NARAP General Advisory Committee that best represented the specific disability groups of interest to the project formed the pool of potential focus groups.

Face-to-face focus groups were held concurrently with large conferences in the spring of 2005 to take advantage of a broad constituency of educators already congregated in a single location for a given time period. Participation was open to all conference attendees. However, some conferences were so large that only a subset were actually contacted. Potential participants were usually contacted via e-mail and an electronic mailing list, in collaboration with the conference or General Advisory Committee (GAC) members. Multiple sessions were scheduled during each conference. Conference attendees were sent a brief e-mail inviting those interested to pre-register for a session via a Web site hosted at ETS, with times and dates specific to each conference. Potential participants would indicate their availability and some background information. Groups were then formed based on participant characteristics and time availability. On average, 8-10 people were scheduled per session, with attrition and no-shows bringing actual attendance per session to about 5-6 people. At some conferences additional recruitment was done on site, via fliers and word-of-mouth.

The major restriction on the number of conferences attended was one of timing. The organizations national conference had to occur after the Definition Panel meeting in January 2005 but before mid-summer, in order to have the results compiled for the Definition Panels second meeting in September 2005.

Targeted educational conferences were the following (with participant counts indicated):

• Council for Exceptional Children (CEC) -- 6 sessions, 35 people
• American Educational Research Association/National Council on Measurement in Education (AERA/NCME) -- 3 sessions, 17 people
• International Reading Association (IRA) -- 5 sessions, 24 people
• Council of Chief State School Officers (CCSSO) -- 4 sessions, 20 people
• Society for the Scientific Study of Reading (SSSR) -- 5 sessions, 19 people

Learning Disabilities Association of America (LDAA) was initially scheduled for its conference held in early February, but due to the definition still being in flux at that time focus groups were unable to be held. As the time approached for the next two conferences (CEC and AERA/NCME), it was decided to take all three working definitions to the focus groups, instead of a single definition with proficiencies as had been originally planned. This allowed greater opportunity to gain feedback on several approaches to defining an accessible reading construct.

With many key organizations not represented by the face-to-face groups, the decision was made to hold teleconference focus groups with many of the specific disability groups that are represented on the General Advisory Committee. Participants were identified by GAC members, and generally a single group was scheduled. These sessions were held in June and July. Between five and eight people were invited to participate for each teleconference focus group session. Due to scheduling difficulties, these sessions often had small samples.

Targeted disability organizations were the following (with participant counts indicated):

• National Down Syndrome Society (NDSS) -- 4 people
• Center for Applied Linguistics (CAL) -- 1 person
• Parent Advocacy Center for Educational Rights (PACER) -- 3 people
• The American Speech-Language-Hearing Association (ASHA) -- 2 people
• Gallaudet Research Institute -- 4 people
• The Association of State Consultants of Blind/Visually Impaired -- 6 people
• TASH/ARC -- 7 people

Sessions were scheduled for one hour, with actual discussion time set for about 45 minutes.

The protocol included session ground rules, project background information, and a set time schedule to provide uniformity from group to group. All but two of the face-to-face sessions were facilitated by the same person, and all of the teleconference sessions were facilitated by one person (who also had facilitated the remaining face-to-face sessions). All sessions were taped on audio cassette. In addition, a note taker was present for each session to document the proceedings.

All participants were required to fill out a consent form before the session started, as well as a brief questionnaire that was given at the end of each session (via Internet for teleconference participants). Once the completed questionnaire was handed in, face-to-face participants were given a $50 gift card and a NARAP brochure, and teleconference participants were given a $25 gift certificate that could be redeemed via the Internet.

All three definitions were presented to the groups for discussion after the ground rules and project background were reviewed. Although the goal was to discuss each definition on its own merits before comparing them, in many sessions the comparison began as early as during the discussion on Definition B. For the majority of sessions the discussion moved ahead with little prompting from the leader, and most attendees participated fully in the discussion. In addition, the participants often had questions about the project after the session officially ended and expressed interest in the final results.

Results

The results presented here focus on general issues rather than on specifics of wording. Comments from participants covered all types of topics, from very specific suggestions to broader ideas and more conceptual concerns. Our interest here was on those comments that addressed the definitions overall, and any specific themes in comments that related to students with disabilities.

All three definitions that focus group participants discussed were in the same general structure. Each started with an overarching statement of what reading is, followed by a description of how the process of reading occurs -- including specific references to disability concerns -- and concluded with a statement that clarified how understanding is obtained and affected. Using that structure as a guide, some common themes can be identified.

Although we talked to many people with varied backgrounds, in general they formed four major groups: teachers/specialists who work with students with disabilities, reading teachers/specialists, reading researchers, and measurement/assessment experts. In general the reading researchers interpretation of the language used in the definitions seemed to more closely align with the Definition Panels intended meaning. The test development experts viewed the definitions more through the lens of an eventual test and its impact on curriculum and teaching, being more concerned with terms and balance than the other groups. Teachers also were concerned about the influence on curriculum, but focused more on the impact the definition would have on the students they were familiar with. Those concerned with the reading outcomes of students with learning disabilities and mental retardation stressed the importance of finding ways to measure their students abilities and growth over time, even if the students could not meet many of the standard expectations for reading and comprehension. Those concerned with sensory disabilities such as deafness and blindness focused on where the definition may disadvantage their students, or where they thought the language used was problematic. There was a strong desire among the teachers of students with disabilities that affect reading to be able to demonstrate that their students have the ability to read and comprehend text, if presented in a way that was accessible. Those who worked mostly with non-disabled students seemed most interested in the complexities of understanding and the goals of more advanced reading.

Overarching statement

Most people preferred having the main emphasis in the definitions be placed on understanding and did not think that it was appropriate to have decoding appear equal to understanding in importance, as was implied when both were listed in the overarching statement. The teachers and measurement experts generally thought that understanding was the ultimate goal, and decoding was secondary to it, and consequently responded more favorably when decoding was not in the lead sentence. Reading researchers were more likely to view decoding as equally important and many appreciated its inclusion, although even this population viewed understanding as the more important outcome.

Definition of decoding

The term decoding was viewed differently by the reading researchers than the other groups. The researchers viewed decoding as a more comprehensive term for the process of reading that included many aspects of how people turned text into language. Conversely many teachers viewed decoding as too simple a term to represent the technical process of reading fully -- often limiting the term to the process of sounding out a word and relating the term to phonics in many instances. Those who defined decoding that way often would conclude that decoding represented too low-level a skill for many readers. Much of the dislike for the inclusion of decoding as equal in importance to understanding seemed to stem from this difference in the scope of what decoding represented.

Definition of understanding

There was often discussion on the relative nature of the terms understanding and meaning (which were used in the definitions) and comprehension (which was not used). Many felt the three terms were not completely interchangeable and implied different issues. Although the nuances of the three terms were not always consistent from person to person, it most commonly seemed to be a concern that the term used should not limit the scope to simple ideas and concepts found in text but more complex components as well that went beyond facts. If put in a hierarchy, understanding was viewed as the most simplistic (e.g., word definitions), with comprehension the next level (e.g., integrating all the words into a coherent whole), and meaning the highest (e.g., including subtle nuances and deeper context).

Two different descriptions of how understanding is impacted for students were included in the definitions. One listed components that affect understanding and was generally viewed as positive, although many suggestions were made for additional concepts -- most commonly background knowledge and culture -- to be included. The other was less specific, and while some preferred the use of the term constructive processes as being inclusive enough, others felt that was too vague, particularly the measurement and general researchers. Differences between deriving and constructing meaning were suggested as well, with the notion that deriving meaning may imply a specific intent to the text that the reader needs to identify while constructing meaning allows for more outside influences that the reader brings to the text.

Use of the terms speech/spoken words

Almost all groups objected to the references to speech and spoken words. These were viewed as problematic to students who had no spoken language -- most commonly identified as students who are deaf or hearing impaired, although students with autism were also mentioned at times as readers who may be non-verbal.

In addition, the teachers often interpreted translating text to speech as being specific to oral reading (reading out loud). And much like the objections to decoding, the issue seemed to be that such behavior (sounding out, reading out loud) was not a process for advanced readers. However, the reading researchers (and some of the others) were more likely to view the terms as internal speech, or representative of the process of turning text into language, for which speech is a proxy.

Braille

The inclusion of braille was supported as simply being the version of text accessible to those students who read braille. Classifying it as an adaptation or accommodation was questioned by some. The use of a read aloud accommodation instead of braille was mentioned a few times for students who either had not, could not, or would not learn braille. Whether a read aloud was an acceptable means to measure reading comprehension for blind and visually impaired students was a topic where there appeared to be little consensus, although those in our focus groups who worked with blind students clearly supported braille as the preferred mode.

Auditorization

Auditorization got a mixed response. Many believed that including auditorization in the definition undermined a basic construct of reading which includes the interpretation of text. Although some, mostly the teachers with students with disabilities, argued that if understanding was the main goal then it could be appropriate as a means to measure understanding apart from reading processes. Intriguingly, some of the participants who argued that comprehension was the main goal and were opposed to decoding being included, reconsidered when presented with the possibility of measuring reading with an audio assist. The discussion on this topic naturally led to where reading ends and listening begins. In general, support for auditorization was limited, with the likelihood of approving of auditorization being negatively related to ones approval of decoding in the definition.

Disconnect between understanding and decoding

The teachers of students with disabilities were often concerned that many of their students, particularly those with learning disabilities, could show skill in decoding but have no understanding of what they read. Conversely, they were concerned about students who were capable of understanding but could not read well. The similarity of these students with ESL students was mentioned often.

Other issues

The nature and scope of the term text was often discussed at length. For many, text is a highly inclusive term that can include pictures, graphs, tables, numbers and so on, while for others it is restricted to letter symbols. Similarly, when reading ends and literacy begins was also a topic of conversation. Many thought a definition only of reading was not inclusive enough of the overarching goal of communication. The participants who focused more on a tighter focus on text and reading seemed happier with the definitions, while those with a broader focus on representations of meaning and literacy were not as pleased with the definitions.

In addition to the focus on the construct of reading as set forth in the definitions, there was also a focus on specific issues relevant for the various disability groups. Themes that emerged from disability-specific focus groups are found below, and they corresponded highly with themes that emerged from conference participants. In addition to general themes, however, focus group participants mentioned a plethora of aspects of reading related to students with specific types of disabilities. Although the samples of participants were small, the results provide a strong addition to the disability information obtained from the larger in-person focus groups.

Down Syndrome

Although readers with Down Syndrome do learn to read by decoding, some readers with Down Syndrome are non-verbal; for them, the process of translating text to speech is not relevant. Because reading is a visual endeavor for most students with Down Syndrome, auditorization should be considered an adaptation, not part of the reading process itself.

Emotional/Behavior Disorders

Comprehension is the biggest issue for the population of students with emotional or behavioral disabilities. Many students decode text just fine, but do not understand the meaning of text. Other factors, such as memory, fluency, and vocabulary may affect the comprehension levels of this population, and should be included in any definition.

Mental Retardation

Some readers with mental retardation have difficulty decoding, but can understand text through other strategies. Readers with mental retardation need to be engaged in order to succeed. Struggling readers may quickly give up if text is not interesting or relevant to their lives. Readers with mental retardation may be non-verbal; therefore, an expectation of translating text to speech is unreasonable. Comprehending text (by a variety of means) is the most important goal of reading for students with mental retardation.

Learning Disabilities

Some readers with learning disabilities use alternative approaches to reading, such as screen readers or books on tape, but still consider the process reading. A focus on accessing information, rather than individual skills, is most appropriate for this population. Accommodations, such as auditorization are commonplace in higher education, but rarely found in K-12 education.

Speech/Language Impairments

Readers with speech/language impairments may not translate text to speech as part of the reading process. Measures of fluency (for either silent reading or reading aloud) must include a focus on both fluency and morphological processing in order to truly measure the reading abilities of students with speech language impairments.

Blind

According to participants, braille is an equivalent system of writing to print. Participants also noted that 40% of people who are blind have some other form of disability. Some may not have speech, so translating text to speech may be impossible. Text in auditory formats is used by blind populations, but should be used with extreme caution, because it may lead to a decrease in the teaching of braille. All definitions should include decoding but should be strongly centered in the derivation of meaning from text.

Deaf

Students who are deaf typically do not decode because they often do not have phonological skills. This population also does not translate text to speech. Definitions for students who are deaf should be more comprehension-based and less skills-based. It was proposed by some participants that students who are deaf should be viewed as similar to ESL students as their communicative language is not English and American Sign Language does not map in terms of grammar and vocabulary to the English language.

Conclusion

Participants in focus groups strongly favored definitions of reading that centered on understanding of text. Participants were wary to endorse definitions that appeared overly-focused on decoding, although decoding was considered an important aspect of reading by most participants. A majority of participants disliked overt references to translating text to speech, either as an important function of reading or as impossible for some students with disabilities. Many participants were hesitant (or completely disagreed) that auditorization could be considered reading. Finally, participants offered a variety of explanations about how students with disabilities read. Although approaches varied from disability to disability, overall themes reflected the main finding that understanding is the foremost aspect of reading upon which researchers and test designers should focus.

Top of page.