International Educational Review

The Effect of Job-embedded Professional Development on Teacher and Student Outcomes: A Multi-Level Meta-Analysis

Nuri Balta 1 * ,
Ruben Fukkink 2 3,
Steven J. Amendum 4
1 SDU2 University of Amsterdam3 Amsterdam University of Applied Sciences4 University of Delaware* Corresponding Author
Download PDF
International Educational Review, Volume 1, Issue 1, April 2023, pp. 1-23
OPEN ACCESS VIEWS: 843 DOWNLOADS: 724 Publication date: 15 Apr 2023
ABSTRACT
The aim of this meta-analysis was to analyze the experimental research into the effects of job-embedded professional development (JEPD) for teachers and student outcomes. Our meta-analysis of experimental studies of the effects JEPD, included 20 studies (with 79 experimental comparisons) at teacher level and 19 studies at student level (with 34 experimental comparisons). Analyses of the studies, representing 2,062 teachers and 21,425 students, revealed a significant, medium-to-large effect size at teacher level (ES= 0.699, SE= 0.092) and a significant medium effect at student level (ES = 0.523, SE= 0.137). Effects for teachers were smaller in studies with a large sample size. Effects for students were positively related to the length of the intervention. The positive outcomes at teacher and student level support the implementation and expansion of JEPD programs across schools.
KEYWORDS
professional development, job-embedded professional development, student achievement, meta-analysis.
CITATION (APA)
Balta, N., Fukkink, R., & Amendum, S. J. (2023). The Effect of Job-embedded Professional Development on Teacher and Student Outcomes: A Multi-Level Meta-Analysis. International Educational Review, 1(1), 1-23. https://doi.org/10.58693/ier.111
REFERENCES
  1. *Adulyasas, L., & Abdul Rahman, S. (2014). Lesson study incorporating phase-based instruction using Geometer’s Sketchpad and its effects on Thai students’ geometric thinking. International Journal for Lesson and Learning Studies, 3(3), 252-271.
  2. Aljuzayri, Z. (2021). The Relationship Between U.S. High School Science Teacher’s Self-Efficacy, Professional Development, and Use of Technology in Classrooms. Journal of Research in Science, Mathematics and Technology Education, 4(1), 45-62. doi: 10.31756/jrsmte.414
  3. *Amendum, S. J. (2014). Embedded professional development and classroom-based early reading intervention: Early diagnostic reading intervention through coaching. Reading & Writing Quarterly, 30(4), 348-377.
  4. *Amendum, S. J., & Liebfreund, M. D. (2019). Situated learning, professional development, and early reading intervention: A mixed methods study. The Journal of Educational Research, 112(3), 342-356.
  5. Balta, N., Arslan, M., & Duru, H. (2015). The effect of in-service training courses on teacher achievement: A meta-analysis study. Journal of Education and Training Studies, 3, 254-263. Doi: 10.11114/jets.v3i5.1037
  6. Bax, L., Yu, L. M., Ikeda, N., & Moons, K. G. (2007). A systematic comparison of software dedicated to meta-analysis of causal studies. BMC Medical Research Methodology, 7(1), 40.
  7. Berliner, D. (2001). Learning about learning from expert teachers. International Journal of Educational Research, 35(5), 463-483.
  8. Blank, R. K., & de las Alas, N. (2009). Effects of teacher professional development on gains in student achievement: How meta analysis provides scientific evidence useful to education leaders. Washington, DC: Council of Chief State School Officers.
  9. Birchak, B., Connor, C., Crawford, M., Kahn, L. H., Kaser, S., Turner, S., & Short, K. G. (1998). Teacher study groups: Building community through dialogue and reflection. Urbana, IL: National Council of Teachers of English. Retrieved from ERIC database. (ED424584)
  10. Borman, G. D., & Kimball, S. M. (2005). Teacher quality and educational equality: Do teachers with higher standards-based evaluation ratings close student achievement gaps? The Elementary School Journal, 106, 3-20. doi:10.1086/496904
  11. Broad, K., & Evans, M. (2006). A review of literature on professional development content and delivery modes for experienced teachers (Report prepared for the Ontario Ministry of Education). Toronto, ON: Ontario Institute for Studies in Education, University of Toronto. Retrieved from https://www.oise.utoronto.ca/ite/UserFiles/File/AReviewofLiteratureonPD.pdf
  12. Bruce, C. D., Esmonde, I., Ross, J., Dookie, L., & Beatty, R. (2010). The effects of sustained classroom-embedded teacher professional learning on teacher efficacy and related student achievement. Teaching and Teacher Education, 26(8), 1598-1608.
  13. Caena, F. (2011). Literature review: Quality in teachers’ continuing professional development. Luxembourg, Belgium: European Commission. Retrieved from http://ec.europa.eu/education/policy/strategic-framework/doc/teacher-development_en.pdf
  14. Carlisle, J. F., Cortina, K. S., & Katz, L. A. (2011). First-grade teachers’ response to three models of professional development in reading. Reading & Writing Quarterly, 27(3), 212-238.
  15. Cavazos, H. O. (2013). Job-embedded professional development in reading for teachers of English language learners (Doctoral dissertation). University of Texas, Austin. Retrieved from http://repositories.lib.utexas.edu/handle/2152/21405
  16. Cochran-Smith, M., & Lytle, S. L. (1990). Research on teaching and teacher research: The issues that divide. Educational Researcher, 19(2), 2–11.
  17. Clair, N. (1998). Teacher study groups: Persistent questions in a promising approach. TESOL Quarterly, 32(3), 465-492.
  18. Comprehensive Meta-Analysis (Version 2) [Computer software]. (2015). Englewood, NJ: Biostat.
  19. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
  20. Croft, A., Coggshall, J. G., Dolan, M., Powers, E., & Killion, W. J. (2010). Job-embedded professional development: What it is, who is responsible, and how to get it done well (Issue Brief). Washington, DC: National Comprehensive Center for Teacher Quality. Retrieved from http://files.eric.ed.gov/fulltext/ED520830.pdf
  21. Darling-Hammond, L. (1997). Doing what matters most: Investing in quality teaching. New York: National Commission on Teaching & America's Future.
  22. Day, C. (1999). Developing teachers: The challenges of lifelong learning. London, England: Falmer Press.
  23. Desimone, L. (2011). A primer on effective professional development. Phi Delta Kappan, 92, 68–71.
  24. Diaz-Maggioli, G. (2003). Professional development for language teachers. Eric Digest, EDO-FL-03-03. Retrieved from http://unitus.org/FULL/0303diaz.pdf
  25. *Doppelt, Y., Schunn, C. D., Silk, E. M., Mehalik, M. M., Reynolds, B., & Ward, E. (2009). Evaluating the impact of a facilitated learning community approach to professional development on teacher practice and student achievement. Research in Science & Technological Education, 27(3), 339-354.
  26. DuFour, R. (2004). What is a professional learning community? Educational Leadership, 61(8), 6–11.
  27. * Fabiano, G. A., Reddy, L. A., & Dudek, C. M. (2018). Teacher coaching supported by formative assessment for improving classroom practices. School Psychology Quarterly, 33(2), 293.
  28. *Faulk, J. (2004). Investigation of the relationship between a job-embedded model of professional development and reading achievement of elementary school students (Unpublished doctoral dissertation). East Tennessee State University, Johnson City, Tennessee.
  29. Fernandez, C. (2002). Learning from Japanese approaches to professional development: The case of lesson study. Journal of Teacher Education, 53(5), 393-405.
  30. Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., . . . McCoy, A. (2018). Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in response to a nationwide science reform. Teaching and Teacher Education, 72, 107-121. doi: 10.1016/j.tate.2018.02.011
  31. French, V. W. (1997). Teachers must be learners, too: Professional development and national teaching standards. NASSP Bulletin, 81(585), 38-44.
  32. *Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., & Sztejnberg, L. (2008). The impact of two professional development interventions on early reading instruction and achievement (NCEE 2008-4030). Washington, DC: National Center for Education Evaluation and Regional Assistance.
  33. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38, 915-945. doi:10.2307/3202507
  34. *Gersten, R., Dimino, J., Jayanthi, M., Kim, J. S., & Santoro, L. E. (2010). Teacher study group: Impact of the professional development model on reading instruction and student outcomes in first grade classrooms. American Educational Research Journal, 47(3), 694-739.
  35. Goldhaber, D., Lavery, L., & Theobold, R. (2015). Uneven playing field? Assessing the teacher quality gap between advantaged and disadvantaged students. Educational Researcher, 44, 293-307. doi:10.3102/0013189X15592622
  36. Guskey, T. R. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin.
  37. Guskey, T. R. (2003). What makes professional development effective? Phi Delta Kappan, 84(10), 748-750.
  38. Hanushek, E. A., & Rivkin, S. G. (2006). Teacher quality. Handbook of the Economics of Education, 2, 1051-1078.
  39. Hanushek, E. A., & Rivkin, S. G. (2010). Generalizations about using value-added measures of teacher quality. American Economic Review, 100(2), 267-271. doi:10.1257/aer
  40. Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6, 539-561.
  41. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
  42. *Helmer, J., Bartlett, C., Wolgemuth, J. R., & Lea, T. (2011). Coaching (and) commitment: Linking ongoing professional development, quality teaching and student outcomes. Professional Development in Education, 37(2), 197-211.
  43. Hendriks, M., Luyten, H., Scheerens, J., Sleegers, P., & Steen, R. (2010). Teachers’ professional development– Europe in international comparison: An analysis of teachers’ professional development based on the OECD’s teaching and learning international survey. Luxembourg, Belgium: European Commission. Retrieved from http://doc.utwente.nl/92610/1/NC8010244ENC_002.pdf
  44. Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539-1558.
  45. Hord, S. M. (1997). Professional learning communities: Communities of continuous inquiry and improvement. Austin, TX: Southwest Educational Development Lab. Retrieved from ERIC database. (ED410659)
  46. Hussey, M. P. (2013). Job-embedded professional development: How school leaders create and support the structures for improved teacher effectiveness. ETD Collection for Fordham University, Paper AAI3559689. Available from http://fordham.bepress.com/dissertations/AAI3559689
  47. Joyce, B., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: ASCD.
  48. *Karadag, R., & Yasar, S. (2010). Effects of differentiated instruction on students’ attitudes towards Turkish courses: an action research. Procedia-Social and Behavioral Sciences, 9, 1394-1399.
  49. *Karimi, M. N. (2011). The effects of professional development initiatives on EFL teachers' degree of self efficacy. Australian Journal of Teacher Education (Online), 36(6), 50-62.
  50. Kennedy, M. M. (1998). Form and substance in inservice teacher education (Research Monograph No. 13). Madison, WI: National Institute for Science Education. Retrieved from ERIC database. (ED472719)
  51. Kennedy, M. M. (2016). How does professional development improve teaching? Review of Educational Research, 86, 945-980. doi:10.3102/0034654315626800
  52. Knight, J. (2011). Unmistakable impact. Thousand Oaks, CA: Corwin.
  53. *Knight, S. L., Wiseman, D. L., & Cooner, D. (2000). Using collaborative teacher research to determine the impact of professional development school activities on elementary students' math and writing outcomes. Journal of teacher education, 51(1), 26-38.
  54. Korthagen, F. A. J. (2010). Situated learning theory and the pedagogy of teacher education: Towards an integrative view of teacher behavior and teacher learning. Teaching and Teacher Education, 26, 98-106. doi:10.1016/j.tate.2009.05.001
  55. *Kraft, M. A., & Blazar, D. (2017). Individualized coaching to improve teacher practice across grades and subjects: New experimental evidence. Educational Policy, 31(7), 1033-1068.
  56. *Kramarski, B., & Revach, T. (2009). The challenge of self-regulated learning in mathematics teachers' professional training. Educational studies in mathematics, 72(3), 379-399.
  57. Langer, P. (2009). Situated learning: What ever happened to educational psychology? Educational Psychology Review, 21, 181-192. doi:10.1007/s10648-009-9099-6
  58. Lenth, R. V. (2001). Some practical guidelines for effective sample size determination. American Statistician, 55(3), 187-193.
  59. Lewis, C., Perry, R., & Murata, A. (2006). How should research contribute to instructional improvement? The case of lesson study. Educational Researcher, 35(3), 3-14.
  60. *Li, Q., Moorman, L., & Dyjur, P. (2010). Inquiry-based learning and e-mentoring via videoconference: A study of mathematics and science learning of Canadian rural students. Educational Technology Research and Development, 58(6), 729-753.
  61. Linder, R. A., Post, G., & Calabrese, K. (2012). Professional learning communities: Practices for successful implementation. Delta Kappa Gamma Bulletin, 78(3), 13-22.
  62. *Matsumura, L. C., Garnier, H. E., & Spybrook, J. (2013). Literacy coaching to improve student reading achievement: A multi-level mediation model. Learning and Instruction, 25, 35-48.
  63. *McDonald, S. E. (2009). A model of teacher professional development based on the principles of lesson study (Unpublished doctoral dissertation). Queensland University of Technology, Brisbane, Australia.
  64. *Murray, S., Ma, X., & Mazur, J. (2009). Effects of peer coaching on teachers’ collaborative interactions and students’ mathematics achievement. Journal of Educational Research, 102(3), 203-212.
  65. Neuman, S. B., & Cunningham, L. (2009). The impact of professional development and coaching on early language and literacy instructional practices. American Educational Research Journal, 46(2), 532-566.
  66. Özer, B., Can, T., Duran, V. (2020). Development of an Individual Professional Development Plan Proposal That is Based on Continuing Professional Development Needs of Teachers. The European Educational Researcher, 3(3), 139-172. DOI:10.31757/euer.334
  67. Rasbash, J., Steele, F., Browne, W.J., & Goldstein, H. (2020) A User’s Guide to MLwiN, v3.05. Bristol, UK: Centre for Multilevel Modelling, University of Bristol.
  68. *Reinke, W. M., Stormont, M., Herman, K. C., & Newcomer, L. (2014). Using coaching to support teacher implementation of classroom-based interventions. Journal of Behavioral Education, 23(1), 150-167.
  69. *Ross, J., & Bruce, C. (2007). Professional development effects on teacher efficacy: Results of randomized field trial. The journal of educational research, 101(1), 50-60.
  70. *Sailors, M., & Price, L. R. (2010). Professional development that supports the teaching of cognitive reading strategy instruction. Elementary School Journal, 110(3), 301-322.
  71. *Sailors, M., & Price, L. (2015). Support for the Improvement of Practices through Intensive Coaching (SIPIC): A model of coaching for improving reading instruction and reading achievement. Teaching and Teacher Education, 45, 115-127. doi: 10.1016/j.tate.2014.09.008.
  72. Smith, C., & Gillespie, M. (2007). Research on professional development and teacher change: Implications for adult basic education. In J. Comings, B. Garner, & C. Smith (Eds.), Review of adult learning and literacy, Vol. 7: Connecting research policy and practice (pp. 205-244), New York, NY: Routledge.
  73. Stronge, J. H., Ward, T. J., Tucker, P. D., & Hindman, J. L. (2008). What is the relationship between teacher quality and student achievement? An exploratory study. Journal of Personnel Evaluation in Education, 20(3-4), 165-184. doi:10.1007/s11092-008-9053-z
  74. Symonds, K. W. (2003). Literacy coaching: How school districts can support a long-term strategy in a short-term world. Retrieved from ERIC database. (ED477297)
  75. Taylor, J., Roehrig, A. D., Hensler, B. S., Connor, C. M., & Schatschneider, C. (2010). Teacher quality moderates the genetic effects on early reading. Science, 328(5977), 512–514. https://doi.org/10.1126/science.1186149
  76. *Tienken, C. H. (2003). The effect of staff development in the use of scoring rubrics and reflective questioning strategies on fourth-grade students’ narrative writing performance. Dissertation Abstracts International, 64(02), 388A. (UMI No. 3081032)
  77. Üstün, U., & Eryilmaz, A. (2014). A research methodology to conduct effective research syntheses: Meta-analysis. Education and Science, 39(174), 1-32.
  78. Vernon-Feagans, L., Bratsch-Hines, M., Varghese, C., Cutrer, E. A., & Garwood, J. D. (2018). Improving struggling readers’ early literacy skills through a tier 2 professional development program for rural classroom teachers: The Targeted Reading Intervention. The Elementary School Journal, 118(4), 525–548. https://doi.org/10.1086/697491
  79. Vescio, V., Ross, D., & Adams, A. (2008). A review of research on the impact of professional learning communities on teaching practice and student learning. Teaching and Teacher Education, 24, 80-91. doi:10.1016/j.tate.2007.01.004
  80. Wallace, B. C., Schmid, C. H., Lau, J., & Trikalinos, T. A. (2009). Meta-Analyst: Software for meta-analysis of binary, continuous and diagnostic data. BMC Medical Research Methodology, 9(80). doi:10.1186/1471-2288-9-80
  81. Wells, C., & Feun, L. (2007). Implementation of learning community principles: A study of six high schools. National Association of Secondary School Principals, 91(2), 141-160.
  82. Woodruff, S., Cox, C., Tosa, S., & Farrell, A. (2013). Lesson study mathematics professional development in an urban elementary school: Sustaining a promising practice (#PB‐2013‐03). Columbus, OH: Ohio Education Research Center.
  83. Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B., & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007–No. 033). Washington, DC: National Center for Education Evaluation and Regional Assistance. Retrieved from https://ies.ed.gov/ncee/edlabs/regions/southwest/pdf/REL_2007033.pdf
LICENSE
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.