Activities

Definition Panel
Focus Groups

Quarterly Update

January 2008

Contents

National Accessible Reading Assessment Projects
Meetings
Presentations
Papers, Reports, and Publications

DARA - Designing Accessible Reading Assessments Update
Technical Advisory Committee Meetings
Papers, Reports, and Publications

PARA – Partnership for Accessible Reading Assessment Update
Technical Advisory Committee Meetings
Papers, Reports, and Publications

TARA – Technology Assisted Reading Assessment Update
Technical Advisory Committee Meetings
Papers, Reports, and Publications

National Accessible Reading Assessment Projects

Greetings! We hope this finds you well and enjoying all that winter has to offer. The purpose of this newsletter is to provide you with an update on recent NARAP activities. This newsletter contains information about our most recent advisory committee meetings, conference presentations, reports and publications, website information, and any related news. We hope you will find this to be helpful and informative.

NARAP Meetings

General Advisory Committee Meeting

The last NARAP General Advisory Committee (GAC) meeting occurred December 7, 2007 in Washington, DC. Presentations and discussions included an overview of NARAP as a joint project and overview of the project and research separately for Technology Assisted Reading Assessments (TARA), Partners for Accessible Reading Assessments (PARA) and Designing Accessible Reading Assessments (DARA) projects. Notes from this GAC meeting can be found on the NARAP website. The next GAC meeting is scheduled for December, 2008.

Principles and Guidelines Committee Meeting

The meeting of the Principles and Guidelines Committee will convene February 11, 2008 in Washington DC. They will review and provide feedback on materials that have been created to support NARAP Goal 3, which is to develop research-based principles and guidelines for large-scale accessible reading assessments.

Top of page.

Presentations

PARA, DARA, and TARA have coordinated presentations as NARAP in many sessions at national conferences since the beginning of the project. Often NARAP has presented their research in the format of a symposium with separate presentations concentrating on the work of a single project. Information from these presentations is organized by date and by conference below.

Institute of Education Sciences (IES) Research Conference, 6/14/2006, Washington , DC
Presenters: Ross Moen and Martha Thurlow
Poster:

• Partnership for Accessible Reading Assessment (Martha Thurlow and Ross Moen)

Council of Chief State School Officers (CCSSO), 6/24/2006, San Francisco, CA, Symposium title: Designing Accessible Large-Scale Reading Assessments for Students with Disabilities: Research and Practice
Presenters: Cara Cahalan Laitusis, Melodie Friedebach, Ross Moen, Martha Thurlow and Dan Wiener
Presentations:

• National Accessible Reading Assessment Projects (Cara Cahalan Laitusis)
• How Should States Handle Students With "Print Disabilities" on State Tests? (Dan Wiener)
• To Read or Not to Read - That was Our Question (Melodie Friedebach)
• Connecting Silos: A Theory of Action in a Complex System (Ross Moen)
• Preliminary Research Results: Partnership for Accessible Reading Assessment (Martha Thurlow)
• Designing Accessible Reading Assessments (DARA) (Cara Cahalan Laitusis)

Association of Test Publishers (ATP), 2/5-2/7/2007, Palm Springs, CA Symposium title: Designing and Developing Accessible Reading Assessments for Students with Disabilities
Presenters: Linda Cook, Ross Moen, and Martha Thurlow
Presentations:

• Hoop Dancing: A Framework for Accessible Reading Assessments (Ross Moen)
• Designing Accessible Reading Assessments: A Summary of Research (DARA) (Linda Cook)
• Partnership for Accessible Reading Assessment ( PARA) Project Overview, Results, Next Steps (Martha Thurlow)
• Technology Assisted Reading Assessment ( TARA) Research Plans (Linda Cook)

Association for Supervision and Curriculum Development (ASCD), 3/18/2007, Anaheim, CA
Presenters: Cara Cahalan Laitusis, Linda Cook, Debora Dillon, and Martha Thurlow
Presentations:

• Developing Accessible Large-Scale Reading Assessments for Students with Disabilities (Martha Thurlow)
• Designing Accessible Reading Assessments (DARA) A Summary of Research (Cara Cahalan Laitusis)
• Partnership for Accessible Reading Assessment ( PARA) (Martha Thurlow and Deborah Dillon)
• Technology Assisted Reading Assessment (Linda Cook)

Council for Exceptional Children (CEC), 4/20/2007, Louisville, KY
Symposium title: Developing Accessible Large-Scale Reading Assessments for Students with Disabilities
Presenters: Cara Cahalan Laitusis, Frederick Cline and Martha Thurlow
Presentations:

• Designing Accessible Reading Assessments (DARA): A Summary of Research (Cara Cahalan Laitusis)
• Partnership for Accessible Reading Assessment ( PARA) (Martha Thurlow)
• Technology Assisted Reading Assessment (TARA) (Frederick Cline)

National Council on Measurement in Education (NCME), 4/10/2007, Chicago, IL Symposium title: Examining DIF and DDF for Students With Disabilities on State English Language Arts Assessments: A Summary of Research from the National Accessible Reading Assessment Projects
Presenters: Jamal Abedi, Linda Cook, Kyndra Middleton and Elizabeth Stone
Presentations:

• Examining Differential Item Functioning for a Reading Test Administered With a Read Aloud Modification (Linda Cook, Mary Pitoniak, Cara Cahalan Laitusis and Frederick Cline)
• Using Differential Item Functioning to Investigate the Impact of Testing Accommodations on an English Language Arts Assessment for Students Who are Blind and Visually Impaired (Elizabeth Stone, Linda Cook, Cara Cahalan Laitusis, and Frederick Cline)
• Examining Differential Item and Distractor Functioning in Reading Assessments for Students with Disabilities (Jamal Abedi, Seth Leon, and Jenny Kao)
• Examining Test Items for Differential Distractor Functioning Among Students With Learning Disabilities (Kyndra Middleton)

International Reading Association (IRA), 5/13-5/17/2007, Toronto, Canada
Presenters: Deborah Dillon and Martha Thurlow
Poster:

• Developing Accessible Reading Assessments for Students with Disabilities (Deborah Dillon and Martha Thurlow)

Institute of Education Sciences (IES) Research Conference 6/6/2007, Washington, DC
Presenters: Jamal Abedi, Cara Cahalan Laitusis, Linda Cook, and Deborah Dillon
Presentations:

• Research on Making Large-Scale Reading Assessments More Accessible for Students with Disabilities (Deborah Dillon) Research on Making Large Scale Assessments More Accessible for Students with Disabilities (Cara Cahalan Laitusis and Linda Cook)
• Partnership for Accessible Reading Assessment ( PARA): Motivation Study (Deborah Dillon)
• Item Analyses Studies (Jamal Abedi, Seth Leon, and Jenny Kao)

Poster Presenters: Martha Thurlow, Deborah Dillon, Cara Cahalan Laitusis, Frederick Cline, Linda Cook and Teresa King
Posters:

• PARA Poster (Martha Thurlow and Deborah Dillon)
• DARA Poster (Cara Cahalan Laitusis and Frederick Cline)
• TARA Poster (Linda Cook and Teresa King)

Council of Chief State School Officers (CCSSO), 6/18/2007, Nashville, TN
Symposium title: Finished! State Reading Tests from the Student's Perspective
Presenters: Ross Moen and Teresa King
Presentations:

• Less Accurately Measured Students (Ross Moen, Martha Thurlow, and Kristin Liu)
• Engaged by Choice? (Martha Thurlow, Deborah Dillon, and Ross Moen)
• Cognitive Labs to Explore Potential Causes of DIF in Students with and without Reading-Based Learning Disabilities (Teresa King and Cara Cahalan Laitusis)

Council of Chief State School Officers (CCSSO), 6/20/2007, Nashville, TN
Symposium title: Research Findings and Implications from the National Accessible Reading Assessment Projects
Presenters: Jason Altman, Jamal Abedi, and Cara Cahalan Laitusis
Presentations:

• Technology Assisted Reading Assessment (TARA) (Jason Altman, Martha Thurlow, Christopher Johnstone and Joseph Timmons)
• Segmented Text Study (Jamal Abedi)
• Multistage Testing (Cara Cahalan Laitusis)

Northeastern Educational Research Association (NERA), 10/17-10/19/2007, Hartford, CT
Presenter: Elizabeth Stone
Presentation:

• Investigation of Some Assessment Issues for Students Who Are Blind or Visually Impaired (Elizabeth Stone)

Poster Presenter: Teresa King
Poster:

• The Impact of an Emotional Disorder on Reading Comprehension Performance (Teresa King and Cara Cahalan Laitusis)

Many proposals have been submitted to conferences for 2008. Proposals have been accepted to Learning Disabilities Association of America (LDA), occurring in Chicago, IL and the papers will be presented on February 27, 2008, from 11:30-12:30 pm; Association of Test Publishers (ATP), occurring March 3-5, 2008 in Dallas, TX; Annual International Technology and Persons with Disabilities Conference (CSUN) occurring March 10-15, 2008 in Los Angeles, CA; American Educational Research Association (AERA) occurring March 24-28, in New York City; Council for Exceptional Children (CEC), occurring April 2-5, 2008 in Boston, MA; the Josephine L. Taylor Leadership Institute (JLTLI) in affiliation with American Federation for the Blind (AFB) occurring April 4-5, 2008; International Reading Association (IRA), occurring May 4-8, 2008 in Atlanta, GA and the National Conference on Student Assessment in affiliation with the Council for Chief State School Officers (CCSSO) occurring June 15-18, 2008 in Orlando, FL. We look forward to seeing you and your colleagues at these conferences. Additional conference locations and dates will be posted in the future. Please continue to visit the NARAP website for new details to be added.

Top of page.

Papers, Reports, and Publications

No reports have been published as unified NARAP reports this year. Previous year NARAP reports can be found on the NARAP website.

Top of page.

DARA – Designing Accessible Reading Assessments Update

Technical Advisory Committee Meetings

The last DARA technical advisory committee meeting occurred October 5-6, 2006 in Washington, DC. Presentations included an overview of DARA, the research results from the psychometric and differential boost studies, plans for a factor analysis, differential item functioning (DIF) and differential distractor functioning (DDF) analyses using the Gates-MacGinitie Reading Test (GMRT) data, plans for future analysis using the differential boost data, plans for a cognitive lab study, and ideas for DARA field testing. The power point slides from the meeting can be found on the DARA website.

The next DARA technical advisory committee meeting is tentatively scheduled to occur in May 2008 in Washington, DC. Meeting details will be shared as the date gets closer.

Top of page.

Papers, Reports, and Publications

There are two new DARA project reports. Please visit the website for all DARA research report abstracts.

• Middleton, K. & Cahalan Laitusis, C. (Unpublished). Examining Test Items for Differential Distractor Functioning Among Students with Learning Disabilities. Available online.
• Cook, L., Eignor, D., Sawaki, Y., Steinberg, J., & Cline, F., (Unpublished). Investigating the Dimensionality of an English-Language Arts Assessment Administered to English-Language Learners With and Without Accommodations. Available online.

Top of page.

PARA – Partnership for Accessible Reading Assessment Update

Technical Advisory Committee Meetings

The last PARA technical advisory committee meeting occurred October 11, 2007 in Washington, DC. Presentations and discussions included: A brief review of project goals and dissemination activities; results of several item analysis studies; preliminary results from a segmented passages experiment; preparations for an experiment on the motivational impact of choice on test taking; preliminary results from a study to identify less accurately measured students; procedures and results-to-date in developing NARAP’s Principles and Guidelines for Accessible Reading Assessment; and, a recap of committee members’ comments and their interest in ongoing involvement in various projects. Notes and the power point slides from the meeting can be found on the PARA website.

The next PARA technical advisory committee meeting is tentatively scheduled to occur in October 2008 in Washington, DC. More meeting details will be shared as the date gets closer.

Top of page.

Papers, Reports, and Publications

The following reports have been completed in the past year and are available on the PARA website:

PARA. (2006). Reading and students with autism.

PARA. (2006). Reading and students with emotional or behavioral disorders.

PARA. (2006). Reading and students with specific learning disabilities.

PARA. (2006). Reading and students with speech or language impairments.

PARA. (2006). Reading and students with mental retardation.

PARA. (2006). Reading and students with visual impairments or blindness.

PARA. (2007). Reading and students who are deaf or hard of hearing.

Abedi, J., Leon, S., & Kao, J. (2007). Examining differential distractor functioning in reading assessments for students with disabilities. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.

Abedi, J., Leon, S., & Kao, J. (2007). Examining differential item functioning in reading assessments for students with disabilities. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.

Johnstone, C. J., Thurlow, M. L., Moen, R. E., Matchett, D., Hausmann, K. E., & Scullin, S. (2007). What do state reading test specifications specify? Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.

Kato, K., Moen, R., Thurlow, M. (2007). Examining DIF, DDF, and omit rates by discrete disability categories for students with disabilities.

Top of page.

TARA – Technology Assisted Reading Assessment Update

Technical Advisory Committee Meetings

Our first TARA technical advisory committee meeting occurred November 17, 2006 in Washington, DC. Please visit the TARA website for the power point slides of the presentations and a synopsis of the meeting notes. The presentations included a National Accessible Reading Assessment Projects (NARAP) overview, the psychometric research plans, the vision of TARA, the vision of the TARA assessment, plans for the survey of assistive technology (AT) users and concluded with the next steps for the project.

The most recent TARA technical advisory committee (TAC) meeting occurred August 24, 2007 in Princeton, NJ. Presentations included a review of the differential item functioning (DIF) study, an examination of the test items that were found to have DIF, an overview of the Technology Assisted Reading Assessment, the description and results of the survey to teachers of the visually impaired (TVIs), plans for the upcoming telephone interviews with TVIs, and next steps for the project. Notes and the power point slides from the meeting can be found on the TARA website.

The next TARA technical advisory committee meeting is tentatively scheduled to occur in 2008. Please check back for more details on the 2008 TAC meeting.

Top of page.

Papers, Reports, and Publications

Three reports have been written in 2007 to describe the work of the TARA project. Copies of two of the reports can be found on the TARA website.

• Stone, E., Cook, L., Cahalan Laitusis, C., & Cline, F. (Unpublished). Using Differential Item Functioning to Investigate the Impact of Testing Accommodations on an English Language Arts Assessment for Students Who are Blind and Visually Impaired. Available online.
• Cahalan, C. (Unpublished). Evidence-Centered Design Test Specifications for Technology Assisted Reading Assessment.
• Thurlow, M., Johnstone, C., Timmons, J. & Altman, J. (2007). Survey of Teachers of Students with Visual Impairments: Students Served and Their Access to State Assessments of Reading. Available online.

Top of page.

Please be sure to let us know if you have any questions about the content of this newsletter or suggestions for improving the newsletter. You can contact us with your questions and suggestions at: narap@umn.edu or readingassessment@ets.org.