One of Urban Initiatives’ six core values as an organization is impact. We engage in data-driven decision making to achieve positive outcomes, and each year we spend a great deal of time, effort, and resources evaluating each of our programs’ impact. By intentionally evaluating each program, we can effectively determine which of our program goals are being met or exceeded and where our program models need improvement in order to best serve our participants, families, and school communities.
Our three-person Research and Evaluation team guides our evaluation strategy at Urban Initiatives, but our organization-wide data culture means that we all pitch in to help support this important effort and we are excited to start an ongoing conversation with you, our stakeholders, about our impact with our 2018-2019 evaluation results!
Our Continuum of Programming Model
Our Continuum of Programming prepares and supports youth as they navigate elementary, middle, high school, and beyond. Through three successive program models, our participants progress from Player to Captain to Coach. Academic achievement and emphasis on social-emotional skills are two keys to our approach. The Continuum of Programming begins as young as kindergarten with our health and education soccer program, Work to Play. As students get older and ready to take on a leadership role on the soccer team, the 5th–8th grade participants transition to Take the Lead, a middle school leadership development and high school readiness program. Finally, our high school program Coach for Success helps our participants prepare for their futures in the workforce and in post-secondary educational opportunities. Although each program has exciting and important impact stats in a variety of areas, our Continuum of Programming is consistently demonstrating impact on our youth’s social emotional learning and academic achievements across all three Continuum programs in a variety of measures.
Social Emotional Learning
One of the most important foundations for our work is social emotional learning. It is defined as is the process through which children and adults understand and manage emotions, set and achieve positive goals, feel and show empathy for others, establish and maintain positive relationships, and make responsible decisions.1 According to existing research, high-quality out-of-school-time programs have been shown to positively impact participants’ social emotional skills such as self-perception and social behaviors, as well as academic performance.2 Social emotional skills in childhood are also associated with positive longer-term outcomes in adulthood in areas such as education, employment, and mental health.3 Across several measures and each program in our Continuum of Programming, social emotional learning is a strong outcome. In elementary school, our soccer players are showing statistically significant increase in their relationship skills. By the end of the program year, Coaches reported that 85% of Work to Play participants “often” or “always” helped or shared with their teammates, cooperated, and initiated interactions with others appropriately, and 80% demonstrated careful listening skills. Coaches also reported a statistically significant increase in elementary school participants’ self-awareness, specifically in the areas of recognizing strengths and increasing their self-efficacy. By the end of the program year, 94% “often” or “always” demonstrated pride in their accomplishments and 93% believed they could succeed. These self-awareness skills are important foundations for strong social emotional health.
By middle school, our social emotional development goals expand beyond oneself and Team Captains are taught to be effective leaders for their team and offer help to younger players. Over the course of the school year, our coaches reported a 20% increase in Team Captains’ ability to problem solve not only for themselves but also for their peers. This is an incredible improvement, but potentially even more exciting is that when asked what made them most proud about being a Team Captain, more than one-quarter of responding participants (28%) cited serving as a role model and being able to positively influence their younger teammates. In addition, 20% cited their ability to help others, such as helping younger teammates learn new skills and solve problems on the soccer field. Not only are our Team Captains making strides in their problem solving and leadership capacities, but they are taking pride in doing so.
Finally, our high school participants report that our program has positive impacts across a range of social emotional skills. One such example is their ability to self-care in order to manage academic stress, an incredibly important skill in a climate where high schoolers are increasingly feeling pressure to perform in order to reach their post-secondary goals. According to our participants, other social emotional skills improved as a result of being in Coach for Success include serving as an effective coach and mentor for their younger peers, problem solving, and taking initiative.
Our Continuum of Programming participants are demonstrating exceptional growth in varied areas of social emotional learning. Social emotional learning is one of the pillars of our work because it is fundamental to future success. The ability to manage emotions, build strong relationships, and make responsible decisions is vital for our youth to build strong futures.
Academics
Another clearly important foundation for youth’s future success is their academic performance. According to the UChicago Consortium on School Research, school attendance and grades in middle school are the strongest predictors of success in high school.4 Our participants continue to be more likely to be on track for graduation and outperforming their peers in the classroom across multiple measures. For example, 72% of our elementary school participants are on track to graduate from high school based on their school attendance and grades compared to only 63% of their classmates. In the same measure taken of our middle school participants, 89% of our Team Captains are on track for graduation compared to 63% of their classmates. Participants that are sticking in our programs over time, and progressing to be in our middle school program, are seeing impressive academic outcomes.
One of the most valuable program components in our middle school program is our coaches’ action plans. For Team Captains whose GPA slips below a 2.5 or whose school attendance rate drops below 95%, our coaches work with the Captain to create an action plan for remedying their academic performance. Last year, 68% of Captains who engaged in an action plan improved their GPA in just one quarter.
Finally, among our high school participants academic successes were reported through several different examples. First, our participants reported that the monthly skill-building workshops component of our high school program taught technical skills like resume and essay writing that was applicable both in and out of the classroom. They also reported improvements in their grades while simultaneously saying that they had challenged themselves to go outside of their comfort zone academically. One student noted, “This program encouraged me to take classes that I didn’t want to take at first. Coming to my senior year I was thinking…I just want to take my classes and breeze on through. But they encouraged me…and although it was a tough process and made my senior year a lot harder than what it had to do…I’m [now] going to college with some of the credits I already need.”
—-
Above you’ll find our Continuum of Programming’s impact in two primary outcome areas, but understand that behind these numbers are the many Chicago Public Schools youth who are doing the work by playing, leading, and learning in our programs. We continue to be so proud to be a part of the communities we serve, and we continue to work hard to ensure our youth are receiving the highest quality education, programming, and support. We are so excited to share these promising results, but we also recognize that continuous quality improvements in our work are imperative to continue to grow and adapt to meet our participants’ needs in real time. Our team has already launched this year’s program evaluation efforts and is excited to continue growing our impact in the 2019–2020 school year.
Like this post? See part two of our evaluation results here. Have questions about our evaluation processes? Email emily.fallon@urbaninitiatives.org.
- https://casel.org/what-is-sel/
- Durlak, J.A., Weissberg, R.P., & Pachan, M. (2010). A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Community Psychology, 45: 294–309.
- Jones, D.E., Greenberg, M., & Crowley, M. (2015). Early social-emotional functioning and public health: the relationship between kindergarten social competence and future wellness. American Journal of Public Health, 105(11); 2283-2290.
- Allensworth, E.M., Gwynne, J.A., Moore, P., & de la Torre, M. (2014). Looking Forward to High School and College: Middle Grade Indicators of Readiness in Chicago Public Schools. UChicago Consortium on School Research: Chicago, IL.