Activities

Definition Panel
Focus Groups

Quarterly Update

November 2007

Contents

National Accessible Reading Assessment Projects
General Advisory Committee Meetings
Presentations
Papers, Reports, and Publications

DARA - Designing Accessible Reading Assessments Update
Technical Advisory Committee Meetings
Papers, Reports, and Publications

PARA – Partnership for Accessible Reading Assessment Update
Technical Advisory Committee Meetings
Papers, Reports, and Publications

TARA – Technology Assisted Reading Assessment Update
Technical Advisory Committee Meetings
Papers, Reports, and Publications

National Accessible Reading Assessment Projects

Greetings! We hope this finds you well and enjoying a pleasant autumn. The purpose of this newsletter is to provide you with an update on recent NARAP activities. This newsletter contains information about our most recent advisory committee meetings, conference presentations, reports and publications, website information, and any related news. We hope you will find this to be helpful and informative.

General Advisory Committee Meetings

The last NARAP General Advisory Committee (GAC) meeting occurred December 8, 2006 in Washington, DC. Presentations and discussions included an overview of NARAP as a joint project and overviews separately for Designing Accessible Reading Assessments (DARA), Partners for Accessible Reading Assessments (PARA) and Technology Assisted Reading Assessments (TARA) projects. PowerPoint slides and notes from this and previous GAC meetings can be found on the NARAP website. The next GAC meeting is scheduled for December, 2007.

Top of page.

Presentations

PARA, DARA and TARA have coordinated presentations as NARAP in many sessions at national conferences since the beginning of the project. Often NARAP has presented their research in the format of a symposium with separate presentations concentrating on the work of a single project. Information from these presentations is organized by date and by conference below.

Institute of Education Sciences (IES) Research Conference, 6/14/2006, Washington, DC
Presenters: Martha Thurlow and Ross Moen
Poster:

• Partnership for Accessible Reading Assessment [Pyramid] (Martha Thurlow and Ross Moen)

Council of Chief State School Officers (CCSSO), 6/24/2006, San Francisco, CA, Symposium title: Designing Accessible Large-Scale Reading Assessments for Students with Disabilities: Research and Practice
Presenters: Cara Cahalan Laitusis, Dan Wiener, Melodie Friedebach, Ross Moen, and Martha Thurlow
Presentations:

• Introduction (Cara Cahalan Laitusis)
• How should states handle students with "print disabilities" on state tests? (Dan Wiener)
• To read or not to read - that was our question. (Melodie Friedebach)
• Connecting silos: A theory of action in a complex system (Ross Moen)
• Preliminary research results: Partnership for accessible reading assessment (Martha Thurlow)
• Designing Accessible Reading Assessments (Cara Cahalan Laitusis)

Association of Test Publishers (ATP), 2/5-2/7/2007, Palm Springs, CA Symposium title: Designing and Developing Accessible Reading Assessments for Students with Disabilities
Presenters: Martha Thurlow, Ross Moen, and Linda Cook
Presentations:

• PARA Project: Overview, Results, Next Steps (Martha Thurlow)
• Hoop Dancing: A Framework for Accessible Reading Assessments (Ross Moen)
• "Designing Accessible Reading Assessments: A Summary of Research" (Linda Cook)
• "Technology Assisted Reading Assessment Research Plans" (Linda Cook)

Association for Supervision and Curriculum Development (ASCD), 3/18/2007, Anaheim, CA
Presenters: Linda Cook and Cara Cahalan Laitusis
Presentations:

• Partnership for Accessible Reading Assessment (Martha Thurlow)
• Technology Assisted Reading Assessment (Linda Cook)
• Designing Accessible Reading Assessments A Summary of Research (Cara Cahalan Laitusis)
• DIF/DDF, Segmenting, and Student Characteristics Studies (Martha Thurlow)
• PARA Motivation Study (Deborah Dillon)

Council for Exceptional Children (CEC), 4/18-4/21/2007, Louisville, KY
Symposium title: Developing Accessible Large-Scale Reading Assessments for Students with Disabilities
Presenters: Martha Thurlow, Cara Cahalan Laitusis, and Frederick Cline
Presentations:

• Partnership for Accessible Reading Assessment (Martha Thurlow)
• Designing Accessible Reading Assessments A Summary of Research (Cara Cahalan Laitusis)
• Technology Assisted Reading Assessment (Frederick Cline)

National Council on Measurement in Education (NCME), 4/9-/4/13/2007, Chicago, IL Symposium title: Examining DIF and DDF for Students With Disabilities on State English Language Arts Assessments: A summary of Research from the National Accessible Reading Assessment Projects
Presenters: Linda Cook, Elizabeth Stone, Jamal Abedi, Kyndra Middleton
Presentations:

• Examining Differential Item Functioning for a Reading Test Administered With a Read Aloud Accommodation (Linda Cook, Mary Pitoniak, Cara Cahalan-Laitusis and Frederick Cline)
• Using Differential Item Functioning to Investigate the Impact of Testing Accommodations on an English Language Arts Assessment for Students Who are Blind and Visually Impaired Elizabeth Stone, Linda Cook, Cara Cahalan-Laitusis, Frederick Cline)
• Examining the Differential Pattern of Responses to Distractors Across Students' Disability Status (Jamal Abedi, Seth Leon, Jenny Kao)
• Examining Test Items for Differential Distractor Functioning Among Students With Learning Disabilities (Kyndra Middleton)

International Reading Association (IRA), 5/13-5/17/2007, Toronto, Canada
Presenters: Cara Cahalan Laitusis and Deborah Dillon
Poster:

• NARAP (Deborah Dillon, Martha Thurlow, Cara Cahalan-Laitusis, and Linda Cook)

Institute of Education Sciences (IES) Research Conference 6/6/2007, Washington, DC
Presenters: Cara Cahalan Laitusis, Linda Cook, Jamal Abedi, and Deborah Dillon
Presentations:

• Overview of NARAP Goals (Deborah Dillon)
• Research on Making Large Scale Assessments More Accessible for Students with Disabilities (Cara Cahalan Laitusis and Linda Cook)
• PARA Motivation Study (Deborah Dillon)
• Item Features Studies: DIF, DDF, and Segmenting Text PARA Research (Jamal Abedi)

Poster Presenters: Martha Thurlow, Deborah Dillon, Cara Cahalan Laitusis, Frederick Cline, Linda Cook and Teresa King
Posters:

• PARA Poster (Martha Thurlow and Deborah Dillon)
• DARA Poster (Cara Cahalan Laitusis and Frederick Cline)
• TARA Poster (Linda Cook and Teresa King)

Council of Chief State School Officers (CCSSO), 6/17-6/20/2007, Nashville, TN
Symposium title: Finished! State Reading Tests from the Student's Perspective
Presenters: Ross Moen and Teresa King
Presentations:

• Less Accurately Measured Students (Ross Moen, Martha Thurlow, and Kristin Liu)
• Engaged by choice? (Martha Thurlow, Deborah Dillon, and Ross Moen)
• Cognitive Labs (Teresa King)

Council of Chief State School Officers (CCSSO), 6/17-6/20/2007, Nashville, TN
Symposium title: Research Findings and Implications from the National Accessible Reading Assessment Projects
Presenters: Jason Altman, Jamal Abedi, and Cara Cahalan Laitusis
Presentations:

• Technology Assisted Reading Assessment (Jason Altman, Martha Thurlow, Christopher Johnstone and Joe Timmons)
• Segmenting Study (Jamal Abedi)
• Multistage Testing (Cara Cahalan Laitusis)

Northeastern Educational Research Association (NERA), 10/17-110/19, 2007, Hartford, CT
Presenter: Elizabeth Stone
Presentation:

• Investigation of Some Assessment Issues for Students Who Are Blind or Visually Impaired (Elizabeth Stone)

Poster Presenter: Teresa King
Poster:

• The Impact of an Emotional Disorder on Reading Comprehension Performance (Teresa King and Cara Cahalan Laitusis)

Many proposals have been submitted to conferences for 2008. We have been accepted to Learning Disabilities Association of America (LDA), occurring in Chicago, IL and will be presenting February 27, 2008, from 11:30-12:30 pm, Association of Test Publishers (ATP), occurring March 3-5, 2008 in Dallas, TX, Annual International Technology and Persons with Disabilities Conference (CSUN) occurring March 10-15, 2008 in Los Angeles, CA, Council for Exceptional Children (CEC), occurring April 2-5, 2008 in Boston, MA and International Reading Association (IRA), occurring May 4-8, 2008 in Atlanta, GA. We look forward to seeing you and your colleagues at these conferences. Additional conference locations and dates will be posted in the future . Please continue to visit the NARAP website for new conferences to be added.

Top of page.

Papers, Reports, and Publications

No reports have been published as unified NARAP reports this year. Previous year NARAP reports can be found on the NARAP website.

Top of page.

DARA – Designing Accessible Reading Assessments Update

Technical Advisory Committee Meetings

The last DARA technical advisory committee meeting occurred October 5-6, 2006 in Washington, DC. Presentations included an overview of DARA, the research results from the psychometric and differential boost studies, plans for a factor analysis, differential item functioning DIF and differential distractor functioning (DDF) using the Gates-MacGinitie Reading Test (GMRT) data, plans for future analysis using the differential boost data, plans for the cognitive lab study, and ideas for DARA field testing. Notes and the power point slides from the meeting can be found on the DARA website.

The next DARA technical advisory committee meeting is tentatively scheduled to occur in May 2008 in Washington, DC. Meeting details will be shared as the date gets closer.

Top of page.

Papers, Reports, and Publications

There are two new DARA project reports. Please visit the website for all DARA research report abstracts.

• Middleton, K. & Cahalan Laitusis, C. (Unpublished). Examining Test Items for Differential Distractor Functioning Among Students with Learning Disabilities. Available online.
• Cook, L., Eignor, D., Sawaki, Y., Steinberg, J., & Cline, F., (Unpublished). Investigating the Dimensionality of an English-Language Arts Assessment Administered to English-Language Learners With and Without Accommodations. Available online.

Top of page.

PARA – Partnership for Accessible Reading Assessment Update

Technical Advisory Committee Meetings

The last PARA technical advisory committee meeting occurred October 11, 2007 in Washington, DC. Presentations and discussions included: A brief review of project goals and dissemination activities, results of several item analysis studies, preliminary results from a segmented passages experiment, preparations for an experiment on the motivational impact of choice on test taking, preliminary results from a study to identify less accurately measured students, procedures and results-to-date in developing NARAP’s Principles and Guidelines for Accessible Reading Assessment, and a recap of committee members’ comments and their interest in ongoing involvement in various projects. Notes and the power point slides from the meeting can be found on the PARA website.

The next PARA technical advisory committee meeting is tentatively scheduled to occur in October 2008 in Washington, DC. More meeting details will be shared as the date gets closer.

Top of page.

Papers, Reports, and Publications

The following reports have been completed in the past year and are available on the PARA website:

PARA . (2006). Reading and students with autism.

PARA . (2006). Reading and students with emotional or behavioral disorders.

PARA . (2006). Reading and students with specific learning disabilities.

PARA . (2006). Reading and students with speech or language impairments.

PARA . (2006). Reading and students with mental retardation.

PARA . (2006). Reading and students with visual impairments or blindness.

PARA . (2007). Reading and students who are deaf or hard of hearing.

Abedi, J., Leon, S., & Kao, J. (2007). Examining differential distractor functioning in reading assessments for students with disabilities. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.

Abedi, J., Leon, S., & Kao, J. (2007). Examining differential item functioning in reading assessments for students with disabilities. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.

Johnstone, C. J., Thurlow, M. L., Moen, R. E., Matchett, D., Hausmann, K. E., & Scullin, S. (2007). What do state reading test specifications specify? Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.

Kato, K., Moen, R., Thurlow, M. (2007). Examin in g DIF, DDF, and omit rates by discrete disability categories for students with disabilities.

Top of page.

TARA – Technology Assisted Reading Assessment Update

Technical Advisory Committee Meetings

Our first TARA technical advisory committee meeting occurred November 17, 2006 in Washington, DC. Please visit the TARA website for the power point slides of the presentations and a synopsis of the meeting notes. The presentations included a National Accessible Reading Assessment Projects (NARAP) overview, the psychometric research plans, the vision of TARA, the vision of the TARA assessment, plans for the survey of assistive technology (AT) users and concluded with the next steps for the project.

The most recent TARA technical advisory committee (TAC) meeting occurred August 24, 2007 in Princeton, NJ . Presentations included a review of the differential item functioning (DIF) study, an examination of the test items that were found to have DIF, an overview of the Technology Assisted Reading Assessment, the description and results of the survey to teachers of the visually impaired (TVIs), plans for the upcoming telephone interviews with TVIs, next steps for the project. Notes and the power point slides from the meeting can be found on the TARA website.

The next TARA technical advisory committee meeting is tentatively scheduled to occur in 2008. Please check back for more details on the 2008 TAC meeting.

Top of page.

Papers, Reports, and Publications

Three reports have been written in 2007 to describe the work of the TARA project. Draft copies of the three reports can be found on the TARA website.

• Stone, E., Cook, L., Cahalan Laitusis, C., & Cline, F. (Unpublished). Using Differential Item Functioning to Investigate the Impact of Testing Accommodations on an English Language Arts Assessment for Students Who are Blind and Visually Impaired. Available online.
• Cahalan, C. (Unpublished). Evidence-Centered Design Test Specifications for Technology Assisted Reading Assessment. Available online.
• Thurlow, M., Johnstone, C., Timmons, J. & Altman, J. (2007). Survey of Teachers of Students with Visual Impairments: Students Served and Their Access to State Assessments of Reading. Available online.

Top of page.

Please be sure to let us know if you have any questions about the content of this newsletter or suggestions for improving the newsletter. You can contact us with your questions and suggestions at: narap@umn.edu or readingassessment@ets.org.