Measuring Impact – Why It Matters
by Alexis Martin, Ph.D., Director of Research and Evaluation
The Level Playing Field Institute relies on rigorous evaluation methods in order to measure the impact of our programs. LPFI’s Research department collects and analyzes data for each of our programs- whether a long-standing summer program such as SMASH, a pilot program such as the UC Davis SMASH Pathways program, or a school-year program such as SMASH: Prep. We consistently produce evaluation reports and data briefs in order to examine the efficacy of our programs, determine areas for improvement, and make program decisions based on research. For example, when LPFI first implemented computer science courses in 2012 at SMASH, evaluation data demonstrated significant gains for scholars in computer science aspirations and achievement. Hence, the organization then decided to build up computer science offerings and implement a full sequence of coursework within SMASH.
Moreover, because low-income students of color overwhelmingly face myriad barriers to access and opportunity, LPFI’s Research department utilizes data to examine how successfully interventions such as SMASH are able to counter these barriers and provide much-needed resources. For example, LPFI’s recent report, Path Not Found, demonstrated that 75% of California’s public high schools with the highest percentage of underrepresented students of color offer no computer science courses. This is another reason for SMASH’s emphasis on computer science.
Beyond making programmatic decisions and improvements, it is vital to additionally use rigorous methods to demonstrate the deep, long-term impact of programs such as SMASH. Thus, LPFI conducts an annual alumni survey in order to report on STEM persistence among program alumni, as well as fields of study, post-secondary experiences, career paths, and the ways in which SMASH prepared alumni for college. We have found that our alumni persist as STEM majors at rates five times higher than that of the national average for non-SMASH African-American and Latino students who declare STEM majors.
LPFI also shares our program impact evaluation findings in order to take part in larger local, state, and national-level dialogues about broadening STEM participation. On an ongoing basis, we share lessons learned about best practices with other STEM programs, practitioners, and policy-makers through publications and conference presentations.
The below graphic demonstrates the ways in which our research informs our practice, and vice versa:
