This study examines the impact of PowerMyLearning’s teacher instructional coaching program on Los Angeles Unified School District (LAUSD) schools during the 2022-2023 school year.
2021-2023 Analysis
Katie James, Ph.D., Director of Research
Kelsey Rocha, Ph.D., Research and Data Consultant
Neha Pandit, Senior Manager, Research and Program Operations
Summary
This mixed methods study examined whether PowerMyLearning’s Teacher Instructional Coaching Program improved student academic achievement and teacher practice in the 2022-23 school year. PowerMyLearning’s Instructional Coaching Program was implemented with elementary, middle, and high school teachers across all content areas including math, ELA, science, and social science.
The results of the study indicate that PowerMyLearning’s program was related to significantly better student English Language Arts (ELA) outcomes.
PowerMyLearning’s program was also associated with significant improvements in teacher practice in the domains of learning environment, instructional planning and design, data-driven decision-making, and student agency.
These findings include teachers and their students across all content areas.
PowerMyLearning is committed to improving math outcomes for elementary students with a focus on supporting math discourse, particularly for multilingual learners (MLLs). Disciplinary literacy has become critical in modern approaches to math instruction, such as problem-based learning, which commonly engage learners in math discourse and other thinking and communicating practices that are authentic to the discipline (Moschkovich, 2015).
There is evidence that math discourse contributes to deeper understanding of math concepts (Sfard, 2001). As well, reading comprehension skills are a significant predictor of stronger math skills and understandings (Grimm, 2008).
The findings from this study along with our prior research provide evidence of PowerMyLearning’s strong impact on teacher practice and student achievement, including both math and literacy. Thus, these findings provide evidence of effective approaches to building literacy skills, which can be leveraged to improve math outcomes.
The number one piece of feedback is that our teachers appreciate the support they’re getting from PowerMyLearning. It is necessary to have job-embedded supports for teachers and feedback with somebody they can trust and be vulnerable with. And PowerMyLearning is able to provide just that.
– Los Angeles County District Administrator
Academic Achievement Outcomes
PowerMyLearning schools performed significantly better than comparison schools on English Language Arts standardized assessments (+9pp, p=.05, ηp2 = .15).
Teacher Practice Outcomes
After PowerMyLearning’s Instructional Coaching Program, teachers showed significant improvement on the targeted instructional practices: learning environment (+.70pp, p<.001, ηp2= .66), instructional planning and design (+.67pp, p<.00, ηp2= .59), data-driven decision making (+.46pp, p<.001, ηp2= .73), and student agency (+.84pp, p<.001, ηp2= .67).
Research Context
This study focused on Los Angeles Unified School District (LAUSD) schools that received PowerMyLearning’s Instructional Coaching program in the 2022-23 school year.
13 PowerMyLearning schools and 13 comparison schools were included in the ELA student achievement analysis. Consistent with PowerMyLearning’s partner schools, the comparison schools included elementary, middle, and high schools.
80 teachers from 12 PowerMyLearning schools were included in the teacher practice analysis. The PowerMyLearning program schools had an average of 85% Hispanic students, an average 92% of students qualify for free or reduced-price lunch, and 26% of students that were English Language Learners.
PowerMyLearning’s Teacher Instructional Coaching Program
PowerMyLearning’s Instructional Coaching Program is designed to support schools and teachers with actualizing their goals. The tailored approach enables experienced coaches to support teachers around PowerMyLearning’s Framework for Teachers, which includes four domains: Learning Environment, Instructional Planning and Delivery, Data-Driven Decision Making, and Student Agency.
- Learning Environment focuses around creating a safe, inclusive classroom with clear routines and procedures that foster student learning.
- Instructional Planning and Delivery focuses on implementing instruction aligned to standards while engaging students in rigorous learning experiences.
- Data-Driven Decision Making allows teacher to gather formative assessment data and understand learning differences to provide students with a variety of learning opportunities based on their needs and interests.
- Student Agency allows teachers to support student goal setting and reflection while fostering student-driven learning and peer collaboration.
The coaching partnership includes meeting with school leadership to understand school and district priorities, building trust with teachers, developing goals, creating an action plan with concrete strategies, and working alongside the teacher to support implementation in their classrooms. Throughout the partnership coaches meet with school instructional leaders to share highlights, gather feedback, and align on instructional supports for teachers.
“PowerMyLearning’s biggest value is their willingness to roll up their sleeves. The folks who work with our teachers build relationships and are modeling alongside our teachers. That has resulted in very positive feedback from teachers and principals."
– Los Angeles County District Administrator
Research Methods
In this section, we will describe the study design, data collection, data analysis and study results. We will then discuss the implications and limitations of the results and the next steps of this work.
Design
We used a quasi-experimental design to examine PowerMyLearning’s impact on student ELA achievement. Publicly available Smarter Balanced Summative Assessment data was collected from PowerMyLearning schools and comparison schools in the 2021-2022 (baseline) and 2022-2023 (program) school year. The criteria for selecting comparison schools can be found below.
We used a pre-post study design to examine PowerMyLearning’s impact on teacher practice. Ratings based on PowerMyLearning’s Framework for Teachers four areas of teacher practice: learning environment, instructional planning and design, data-driven decision-making, and student agency, were collected from 80 teachers at the beginning and end of the coaching partnership.
We also surveyed 98 coached teachers at the end of the coaching partnership to understand their successes and changes that resulted from the coaching partnership. We then synthesized qualitative responses to generate themes and report on the themes below.
Comparison School Criteria
One comparison school was selected as a match for each PowerMyLearning school. The comparison schools were compiled based on the following baseline criteria: (1) school in Los Angeles Unified School District, (2) percent of students that met or exceeded ELA standards, (3) size of school (small versus large), (4) type of school (e.g., elementary, middle, high), (5) percent of students eligible for Free or Reduced Lunch, (6) percent of students from a minority background. Once the comparison school list was compiled, one comparison school from this list was then randomly selected for each PowerMyLearning school.
Instruments and Data Collection
Student Achievement. The California Smarter Balanced Summative Assessment English Language Arts/Literacy is an end of year assessments that is aligned with Common Core State Standards. The assessment measures progress towards college and career readiness and is administered to California students in grades three through eight each spring. Data from this assessment is publicly available for all schools in California.
For this analysis we used the percent of students at each PowerMyLearning and comparison school that met or exceeded the grade-level achievement standards on the assessment.
Teacher Practice Ratings. PowerMyLearning coaches conducted classroom observations to examine changes in teacher practice related to the learning environment, instructional planning and design, data-driven decision-making, and student agency. Observations were guided by four-point rating scale for each practice with exemplar definitions for each level.
Instructional Coaching Educator Survey. At the end of the coaching program, an Educator Coaching survey developed by PowerMyLearning was administered to all coached educators. The goal of the survey was to gather feedback on the coaching program.
Data Analysis
Student Achievement. To analyze the ELA data, we conducted a mixed ANOVA with Bonferroni correction. The between-subjects variable was condition (PowerMyLearning vs. Comparison).
We also conducted simple effects tests to examine hypothesized effects between conditions. It was hypothesized that this analysis would (1) determine that there were no statistical differences between the PowerMyLearning schools and the Comparison schools at baseline, and (2) determine that there were statistically significant differences in the change from baseline to post-program by condition.
For brevity, we will only report the results of the simple effects tests for student ELA performance.
Teacher Practice. A repeated measures ANOVA was conducted examine the pre-post effect instructional coaching had on the Framework for Teachers practice ratings. Themes from the Educator Coaching survey qualitative responses were also generated to illustrate key areas of change.
Results
Student Achievement
The results of the between-subjects simple effects test indicate that PowerMyLearning schools and comparison schools were not significantly different at baseline (p=.33, ηp2 = .04), but were significantly different at post-test (p=.05, ηp2 = .15). This suggests that the post-test differences could be attributed to PowerMyLearning’s Instructional Coaching program.
Teacher Practice Ratings PowerMyLearning’s instructional coaching was related to significant improvement on all four targeted areas of teacher practice: learning environment (+0.70, p<.001, ηp2= .66) instructional planning and design (+0.67, p<.001, ηp2= .59), data-driven decision-making (+0.46, p<.001, ηp2= .73) and student agency (+0.84, p<.001, ηp2= .67) from beginning to end of the program year.
Teacher Survey Results Open-ended Educator Survey responses from the 98 coached teachers were qualitatively analyzed to generate themes around teacher practice areas of strength and success.
The following themes emerged from the data:
- Effective Teaching Strategies to engage more students in the learning process.
“The strategies shared renewed my faith in being an effective teacher and engaging more students in the learning process.”
- Classroom Management strategies to support the learning environment.
“My coach provided specific strategies for transitions when working independently. After trying the strategies, I was able to manage my class so much easier and my students were able to work independently and confidently.”
- Individualized Student Learning approaches that recognize student capabilities and foster agency.
“One of the highlights was that my students have many approaches to solving their work. I learned to have faith that my students will understand the standards and that letting them work at their pace will ultimately make them more successful.”
- Professional Development that builds teacher confidence in navigating curriculum and lesson planning in all subject areas.
“I'm working at a new school using new curriculum and new students this school year. It was very challenging because I had students with different learning styles and many behavior issues. Consequently, many times I felt very overwhelmed and frustrated. After meeting with my coach, I felt confident, ‘I can do this’.”
Discussion
PowerMyLearning is committed to building evidence around the impact of our work. The results of this study indicate that PowerMyLearning’s Instructional Coaching Program is associated with statistically significant effects not only on teacher practice but also on student academic achievement. Indeed, the teacher practice ratings in tandem with the teacher survey results emphasize the powerful impact that the Instructional Coaching program has on teacher practice. The open-ended survey responses highlight the influence professional development and intentional classroom strategies have on teacher confidence and practice as well as student learning. This work suggests that PowerMyLearning’s coaching model can support meaningful changes in the classroom that may ultimately contribute to better student achievement outcomes.
The findings from this study along with our prior research provide evidence of PowerMyLearning’s strong impact on teacher practice and student achievement, including both math and literacy.
Thus, these findings provide promising evidence of effective approaches to building literacy skills, which can be leveraged to improve math outcomes. Future work will explore how math discourse in the classroom can contribute to better math outcomes for students.
About PowerMyLearning
PowerMyLearning is a national nonprofit that works hand-in-hand with educators to foster equitable learning environments that empower each student to thrive academically, socially, and emotionally. The organization provides math-focused professional learning services to K-12 teachers and school leaders so they can support their students in mastering grade-level math content while fostering the skills for life-long math success.
Through its early-grade program, PowerMyLearning combines professional learning with a math practice tool to enhance instruction, with a special focus on promoting math discourse, supporting multilingual learners, and engaging families.
PowerMyLearning has more than two decades of experience working in historically marginalized communities and has achieved statistically significant outcomes on academic achievement, social-emotional learning, and teacher practice.
