The Achievement Gap Initiative
at Harvard University
 
 

Text Box: Spotlight on New Research: AGI Dissertation Fellowships Text Box: children’s responses to think-aloud protocols, the study identified some of the specific linguistic features of math word problems that pose disproportionate difficulty for ELLs. Among them are the use of complex multi-clausal sentences with long noun phrases, unfamiliar vocabulary, polysemous words, and cultural references.The MCAS math items shown below illustrate the impact of unfamiliar vocabulary on DIF for ELLs. Both items 8 and 30 intend to measure the students’ ability to count the number of possible combinations of items from two/three sets. Item 30 shows a small amount of DIF, while item 8 shows a larger amount of DIF, disfavoring ELLs. The odds of responding to item 8 correctly for non-ELLs are close to double the odds for ELLs; while for item 30, the odds for non-ELLs are about 1.3 times those for ELLs. The associated figures illustrate the discrepancy in the difficulty gaps across groups in these two items. Both items have complex grammatical structures with long noun phrases, but item 8 is lexically more complex than item 30. Item 8 contains vocabulary unfamiliar to the Spanish-speaking ELLs who participated in the think-aloud interviews, such as chores, rake, weed, vacuum and dust. Cultural and linguistic background knowledge, not necessarily mathematical knowledge of combinations, made this item disproportionately harder for ELLs compared to non-ELLs. This research not only has implications for developing and analyzing math assessments for ELLs, but it is also relevant to our monitoring of the math achievement gap for ELLs and non-ELLs. According to this study, if MCAS math proficiency scores were determined on the basis of a linguistically simple set of items the group mean difference of ELLs and non-ELLs would be about a third of a standard deviation smaller than the difference for a test including linguistically complex items. Excessively linguistically complex items in math tests tend to invalidly depress the scores for ELLs, thus potentially providing an inaccurate picture of the achievement gap of ELLs and non-ELLs in mathematics. Text Box: Figure 2. Empirical Item Characteristic Curve- Item 30ITEM 30 : Mr. Garcia gave each of his students a notepad, a pencil, and a ruler on the first day of school. The chart below shows the different colors of the notepads, pencils, and rulers. How many different combinations of 1 notepad, 1 pencil, and 1 ruler can Mr. Garcia make? Source: Massachusetts Department of Education, Math, Grade 4 (MCAS, 2003) Text Box: Page # Text Box: Figure 1. Empirical Item Characteristic Curve- Item 8ITEM 8: Every Saturday in the fall, Martin has to do 1 inside chore and 1 outside chore. The chores are listed below. Inside chores: Vacuum, Wash Dishes, Dust Outside chores: Rake, Weed How many different combinations of 1 inside chore and 1 outside chore can Martin make? Source: Massachusetts Department of Education, Math, Grade 4 (MCAS, 2003)a Text Box: Specific Linguistic Features of Math Problems Pose Disproportionate Difficulty Text Box: Email information for Maria Martiniello:Maria_Martiniello@post.harvard.edu
Copyright © 2008 The President and Fellows of Harvard College
Photos (top left and right) by Kevin G. Reeves for the Shaker Heights City School District