 |
 |
 |
 |
 |
|
National Accessible Reading Assessment Projects December 8, 2006
OverviewThe third meeting of the General Advisory Committee (GAC) of the National Accessible Reading Assessment Projects (NARAP) was held December 8, 2006 in Washington, DC. The purpose of the meeting was to update GAC members of NARAPs progress since its second meeting on October 17, 2005 in Washington, DC. GAC members received a NARAP update, including the addition of the Technology-Assisted Reading Assessment (TARA) project, and the outcomes of last years research for PARA and DARA. GAC members also learned and discussed the upcoming research for each project and the next steps for NARAP. Welcome and GreetingsMartha Thurlow, Partnership for Accessible Reading Assessment Martha Thurlow welcomed and thanked all the GAC members for their attendance. She highlighted the days agenda and brought special attention to the projects websites and new brochure. All newly completed papers and reports can be found in the meetings binder. Reports and past presentations can be found on the NARAP website. It is also noted that NARAP will request a one year extension to allow for more research to be conducted. Review of Goals and New ProjectDave Malouf, National Center for Special Education Research, Institute of Education Sciences, U.S. Department of Education Dave Malouf reviewed the history of the projects and provided a brief description of the four main goals of NARAP. The reading definition, goal 1, is complete. Two reports have been produced under this goal, the principles and issues paper, and the focus group results report. Goal 2, the programs of research, are the present undertakings for DARA and PARA. Research conducted this year and planned for the future will be discussed. Goal 3, the principles and guidelines, is an important outcome of the project and will be accomplished in an objective manner. Overview of TARACara Cahalan-Laitusis, Technology Assisted Reading Assessments Cara Cahalan-Laitusis explained the tasks, timeline, and frequently asked questions for the newest project, the Technology-Assisted Reading Assessment, TARA. This will be funded for 5 years. The two major phases are research and test development. Research projects include psychometric research on existing state assessments and survey and interviews with assistive technology (AT) users. The current vision for TARA is to be a measure of the students ability to access text independently with the use of technology. It will attempt to measure forms of students word recognition, fluency, and basic comprehension. Some GAC members were concerned that an unintended consequence from the assessment of technology assisted reading would be that students would not learn braille. Project directors responded that the goal of TARA is not to infringe on braille education. PARA Recent and Future ResearchMartha Thurlow, Partnership for Accessible Reading Assessment Deborah Dillon, Partnership for Accessible Reading Assessment, Professor of Reading & Literacy Martha Thurlow reviewed the years research, including five reports and projects. The disabilities reports, which were discussed during last years GAC meeting, are complete. A differential item functioning (DIF) analysis examined reading comprehension and word fluency for students with and without disabilities in grade 3 and 9. The analysis found students with disabilities in the 9th grade to perform differentially lower than students without disabilities. A differential distractor functioning (DDF) analysis very similar to the DIF study was conducted that yielded similar trends. One concern that will be further investigated is that there were high omission rates on the test. Looking ahead, PARA plans to conduct similar analyses with longitudinal data. A test characteristic study investigated the constructs, item types and number of items on state assessments. The differences among states varied considerably. Not all standards identified are assessed and more than one third of the states that publicize their specifications measure foundational reading skills. Finally, in a student characteristics pilot study, teachers were asked to identify students with reading skills that are not sufficiently measured on state assessments. Results showed promise that teachers were able to identify some such students and revealed ways the study could be strengthened. There are three major projects planned for 2007, a chunking study, a student characteristics study, and a motivational study. The chunking study will break reading passages into smaller pieces. The tests will consist of passages from released items and multiple choice questions. Performance will be compared for grade 8 students with and without disabilities. The student characteristics study is intended to find the prevalence of less accurately measured students (LAMS). Four types of students are targeted: students with fluency limitations, students with comprehension problems, students with reading strengths not measured on traditional reading tests, and students who respond poorly to standardized reading tests. Deborah Dillon presented the details of the motivational study. This study will determine whether allowing choice of reading passages on a large-scale reading assessment increases a students engagement, resulting in a more valid assessment of a students reading comprehension. Students in grades 4 and 8 will participate. There will be an expert panel review to ensure passage selection is appropriate. GAC members brought up concerns about parsing out true performance differences and comprehension problems due to the task being a new and unfamiliar one. Care must be put in the directions to help try to eliminate this issue. Several GAC members supported the importance and relevance of measuring self efficacy (outlined in the motivation study) which is related to the idea of motivation. GAC members provided support with study recruitment efforts. DARA Recent and Future ResearchCara Cahalan-Laitusis, Designing Accessible Reading Assessments Linda Cook, Designing Accessible Reading Assessments Cara Cahalan-Laitusis reviewed the differential boost study findings. This study measured reading comprehension using the Gates MacGinitie Reading Comprehension test (GMRT) for students with and without reading-based disabilities in the 4th and 8th grade. Two administrations of the test were given to each student, a standard paper and pencil version and an audio version with a CD that read the test, questions and responses aloud. Compared to students without disabilities, students with reading-based disabilities were found to have a larger gain in score on a reading comprehension assessment when the test was read aloud. Teacher predictions on which test format students would perform better (standard or audio) were collected as well. Using a multiple regression, teachers could predict score gains at the 4th grade better than at the 8th grade level. Linda Cook described two of the psychometric studies conducted this year on operational administrations of the California STAR English-Language Arts (ELA test). A factor analysis (FA) and differential item functioning (DIF) analysis were conducted. The FA examined the test structure for students with and without disabilities who took the test with and without accommodations. It was determined that the test measures one dimension (English-Language Arts) for all students regardless of disability or accommodation. The DIF study was carried out on the same data as the FA study. The findings of the DIF study were consistent with the FA results. The psychometric research planned for 2007 will parallel the research conducted on the STAR ELA assessment. The GMRT data from the differential boost study will be used in a FA, DIF, and DDF analysis. Because of the experimental design of the GMRT study, the results of the study will be more generalizable than those obtained from the analysis of the STAR ELA assessment and should provide greater insight into the impact of a read aloud accommodation on reading assessments taken by students with and without reading-based learning disabilities. Cara Cahalan-Laitusis explained the other research projects to be completed in 2007. Two or three step tailored testing will be conducted using the STAR ELA data and the GMRT data. Students will begin with a reading comprehension test and will advance to an extended comprehension section or if necessary will retreat to a reading fluency and decoding assessment. The project will also carry out cognitive labs using students with and without reading-based learning disabilities as the participants. The purpose of the labs will be to investigate the reasons why students with disabilities respond to some reading comprehension questions differently when compared to students without disabilities. Responses, errors and trends will be documented. The third project that is being planned is a regression analysis that will provide information to assist with IEP decision making. Data from the differential boost study, including the supplemental reading test scores, the GMRT scores, information about accommodations provided in class, and students prior NJ ASK scores will be used. GAC members asked that a note be included in the report of the IEP decision making study that the study used operational data and consequently does not include variables like a parents input. Next StepsMartha Thurlow, Partnership for Accessible Reading Assessment Martha Thurlow concluded the meeting by reviewing NARAP accomplishments and summarizing follow up items identified during the meeting. NARAP will send out periodic blurbs about the recent activities and future research for GAC members to distribute to their constituents.
|
Contact narap@umn.edu
with questions or assistance with this site.