Measures of effectiveness for S.T.E.M. Program: The Arizona science of baseball

Science, Technology, Engineering, and Mathematics (STEM) outreach programs are becoming increasingly popular and widespread, providing the necessary opportunity to engage K-12 students in these fields. However, it is unclear what the best measures of effectiveness are for these types of programs. It is known that behavioral, academic, and changes in career-related choices can be observed among STEM program participants but limited objective evidence exists to support the goodness of any single measure. Moreover, the reliability of the data sources, namely the youth, parent, teacher, or administrative records that provide key information about the program experience is unknown. There are also short-term (e.g., improved grades, improved selfconfidence) and long-term (e.g., removal of barriers to advancement in math and science) outcomes which present even more difficulties in determining the ideal measures of success. This paper explores these issues in the context of a STEM outreach program called the Arizona Science of Baseball, managed by the University of Arizona, employing systems engineering principles. Participants in a 2012 pilot program were 7th and 8th grade students from Mansfield Middle School in Tucson, Arizona which provide a useful case for testing and evaluating a variety of measures.