This is a proposal for the Institute for Social Research (ISR) to carry out the first stages of An Evaluation of State Board Examination Systems over the period 3/1/11 through 8/31/12. The scope of work has two components: (a) implementation studies; and (b) an efficacy trial. We describe these separately.
Implementation Studies. One component of work (having two, interrelated components) will examine implementation dynamics in all schools enrolling in the Board Examination System (BES) pilot project for the AY 2011-2012 and 2012-2013 periods. In one component of this work, the following data will be gathered: (a) documentary data on the state and local policy environments of BES pilot schools; (b) a telephone interview with the principal of each implementing school, monthly interviews with NCEE engagement managers(who manage implementation in adopting sites), and one interview with personnel working for the Board Exam providers; (c) publicly available or easily obtained administrative data for adopting schools, for example, data on student composition and historic levels of achievement, teacher and other staff characteristics, program complexity and administration arrangements, and funding; and (d) annual data from BES pilot schools on the number of students at each school enrolled in Board programs, the specific (Board and non-Board) courses in which these students are enrolled, the grades and passing rates of students in Board Exam courses, and students? Board Exam scores. Using these data, the UM team will use factor analytic models to explore the dimensionality in implementation outcomes, and qualitative and administrative data to examine what differentiates high from low implementing schools. In addition to this work, ISR also will conduct 3-8 intensive case studies at BES pilot sites.
Efficacy Study. The second component of work prepares ISR to conduct a planned efficacy trial that compares student outcomes across 25 BES pilot schools and 25 matched control schools. With funding requested here, ISR will recruit an initial cohort of 9 treatment and 9 control schools by 8/1/11 and 16 treatment and 16 control schools by 8/31/12. ISR also will launch data collection in the first 18 schools recruited into the study between 8/1/11 and 6/30/12. In these schools, ISR will select matched samples of 10 treatment and 20 control students in each BES school and 30 matched, untreated students in control schools and then administer a web survey to sampled students during the period 11/11 to 12/11. Survey items will ask about students? specific course enrollments, their self concept as learners (i.e., self-efficacy, expectations for course success, and attitudes toward learning), as well as their future plans for enrollment in high school and postsecondary classes. The items on this portion of the survey will be drawn from NCES longitudinal surveys such as NELS:88 and ELS. ISR researchers also will code course enrollment information as part of data processing activities. Final data files will be compiled and posted for researchers to analyze.
The funding requested from the sponsor is intended to bridge ISR as it seeks funding to extend data collection over additional years and schools.
Funding:
National Center on Education and the Economy (NCEE)
Funding Period:
01/02/2011 to 06/30/2013