The Validation of Assessment for Learning Audit Instrument: A Mixed Methods Approach

Mazidah Mohamed, Mohd Sallehhudin Abd Aziz, Kemboja Ismail

Abstract


The CEFR-aligned curriculum promotes integration in the implementation of Assessment for Learning (AfL) by English language teachers in primary schools. This paper depicts the mixed methods pilot and validation of the Assessment for Learning Audit Instrument (AfLAi) for the purpose of examining the use and understanding of AfL among the teachers in primary schools. In Phase 1, three subject matter experts were consulted, and nine English language teachers provided consensual validation. In Phase 2, 53 English language teachers responded to the adapted instrument. AfLAi was further discussed with the validators in Phase 3. This procedure was adapted from Phase 8 of the Instrument Development and Construct Validation mixed research techniques. Among the emerging themes were on language correction, item/ instrument difficulty, and useful items. The quantitative data was compared with the scales from the original authors regarding the four key strategies of AfL in AfLAi: Sharing Learning Intentions and Success Criteria, Questions and Classroom Discussions, Feedback and Peer- and Self-Assessment. Based on the mixed data arranged in a joint display, some parts in the items were adapted to accommodate the local context. Among the findings were (i) the key strategy known as Learning Intentions in AfLAi was not applicable in the target context, and (ii) some examples in the items needed to be changed to fit the Malaysian syllabus. The implications of this pilot study and validation process included 16 changes in the instrument and the translation of the instrument into Bahasa Melayu.

 

Keywords: Assessment for Learning; English language teachers; primary school; audit instrument; mixed methods


Full Text:

PDF

References


Abdul Hakim Ali, A. A., Radzuwan, A. R. & Wan Zhafirah, W. Z. (2018). The enactment of the Malaysian Common European Framework of Reference (CEFR): National master trainer’s reflection. Indonesian Journal of

Applied Linguistics. 8(2), 409–417. doi:10.17509/ijal.v8i2.13307

Alla Baksh, M. A. K., Mohd Sallehhudin, A. A. & Siti Hamin, S. (2019). Examining the factors mediating the intended washback of the English language school-based assessment: pre-service ESL teachers’ accounts.

Pertanika Journal of Social Sciences & Humanities. 27(1), 51–68.

Alla Baksh, M. A. K., Mohd Sallehhudin, A. A., Yahya, A. T. & Norhaslinda, H. (2016). Washback effect of school-based English language assessment: a case-study on students’ perceptions. Pertanika Journal of Social

Sciences & Humanities. 24(3), 1069–1086.

Amin bin Senin (Ketua Pengarah Pelajaran Kementerian Pendidikan Malaysia). (2019). Naratif baharu amalan pendidikan. Fokus & Gerak Kerja Profesional KPPM 2019. BICARA PROFESIONAL KPM. EduwebTv.

https://www.youtube.com/watch?v=vTLk1F7A9uA

Azlin, N. M., Ong, H. L., Mohamad Sattar, R., Rose Amnah, R. & Nurhayati, Y. (2013). The benefits of school-based assessment. Asian Social Science. 9(8), 101–106. doi:10.5539/ass.v9n8p101

Black, P. (2009). Formative Assessment issues across the curriculum: the theory and the practice. TESOL Quarterly. 43(3), 519–524. doi:10.1002/j.1545-7249.2009.tb00248.x

Black, P. (2015). Formative Assessment - an optimistic but incomplete vision. Assessment in Education: Principles, Policy & Practice. 22(1), 161–177. doi:10.1080/0969594X.2014.999643

Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice. 5(1), 7–74. doi:10.1080/0969595980050102

Brown, G. T. L. (2019). Is Assessment for Learning really assessment ? Front. Educ. 4(64): 1–7. doi:10.3389/feduc.2019.00064

Chin, M. L. L., Krishnamoorthy, K. & Yap, J. R. (2019). The role of negotiated interaction in L2 vocabulary acquisition among primary ESL learners. 3L: The Southeast Asian Journal of English Language Studies. 25(2), 1–

doi:http://doi.org/10.17576/3L-2019-2502-01

Creswell, J. W. (2012). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 4th Edition. Pearson.

Creswell, J. W. (2014). A Concise Introduction to Mixed Methods Research. SAGE Publications.

Department of Education Ireland. (2007). The Northern Ireland Curriculum: Primary. CCEA.

Eisner, E. W. & Noddings, N. (2017). The Enlightened Eye: Qualitative Inquiry and the Enhancement of Educational Practice. Early childhood education series. Teachers College Press. Retrieved from

https://books.google.com.my/books?id=09UcDgAAQBAJ

Gjersing, L., Caplehorn, J. R. M. & Clausen, T. (2010). Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations. BMC Medical Research Methodology. 10(13), 1–22.

doi:10.1186/1471-2288-10-13

Hazita, A. (2016). Implementation and challenges of English language education reform in Malaysian primary schools. 3L: The Southeast Asian Journal of English Language Studies. 22(3), 65–78.

Leahy, S., Lyon, C., Thompson, M. & Wiliam, D. (2005). Classroom Assessment: minute by minute, day by day. Educational Leadership. 63(3), 19–24.

Lysaght, Z. (2009). From Balkanisation to boundary crossing: using a Teacher Learning Community to explore the impact of assessment on teaching and learning in a disadvantaged school. Unpublished PhD thesis,

Dublin City University, Ireland.

Lysaght, Z. & O’Leary, M. (2013). An instrument to audit teachers’ use of assessment for learning. Irish Educational Studies. 32(2), 217–232. doi:10.1080/03323315.2013.784636

Lysaght, Z. & O’Leary, M. (2017). Scaling up, writ small: using an assessment for learning audit instrument to stimulate site-based professional development, one school at a time. Assessment in Education: Principles,

Policy & Practice. 24(2), 271–289. doi:10.1080/0969594X.2017.1296813

Lysaght, Z., O’Leary, M. & Ludlow, L. (2013). Measuring teachers ’ Assessment for Learning (AfL) classroom practices in elementary schools. International Journal of Educational Methodology. 3(2), 103–115.

doi:10.12973/ijem.3.2.103

Marshall, B. & Drummond, M. J. (2006). How teachers engage with Assessment for Learning: lessons from the classroom. Research Papers in Education. 21(2), 133–149. doi:10.1080/02671520600615638

Mazidah, M. & Mohd Sallehhudin, A. A. (2018). Juxtaposing the primary school assessment concepts and practices in Singapore and Malaysia. International Journal of Engineering & Technology. 7(3.21), 552–556.

Merriam, S. B. (2002). Qualitative Research in Practice: Examples for Discussion and Analysis. The Jossey-Bass higher and adult education series. Jossey-Bass. Retrieved from https://books.google.com.my/books?

id=exe2AAAAIAAJ

Merriam, S. B. (2009). Qualitative Research: A Guide to Design and Implementation. Higher and adult education series. John Wiley & Sons. Retrieved from https://books.google.com.my/books?id=tvFICrgcuSIC

Ministry of Education Malaysia Malaysia Education Blueprint 2013-2025 (Preschool to Post-Secondary Education) (2012). Retrieved from

http://www.moe.gov.my/cms/upload_files/articlefile/2013/articlefile_file_003108.pdf

Ministry of Education Malaysia. (2018). Dokumen Standard Kurikulum dan Pentaksiran Bahasa Inggeris Tahun 3 Sekolah Kebangsaan. Putrajaya. Retrieved from http://bpk.moe.gov.my/index.php/terbitan-

bpk/kurikulum-sekolah-rendah

Noor Hayati, R. & Mohd Sallehhudin, A. A. (2015). The use of Bahasa Melayu in the English language classroom by ‘non-optionist’ English teachers. Procedia - Social and Behavioral Sciences 172(2015): 770–777.

doi:10.1016/j.sbspro.2015.01.431

Normazidah, C. M., Koo, Y. L. & Hazita, A. (2012). Exploring English language learning and teaching in Malaysia. GEMA Online Journal of Language Studies. 12(1), 35–51.

Nurfaradilla, N., Siti Norhidayah, R., Mohammad Iskandar, S., Kasmah, A. B. & Sharifah Nor, P. (2010). Teachers’ perception on alternative assessment. Procedia - Social and Behavioral Sciences. 7(C), 37–42.

doi:10.1016/j.sbspro.2010.10.006

Nurul Farehah, M. U. & Mohd Sallehhudin, A. A. (2018). Implementation of CEFR in Malaysia : Teachers ’ awareness and the Challenges. 3L: The Southeast Asian Journal of English Language Studies. 24(3), 168–183.

Onwuegbuzie, A. J., Bustamante, R. M. & Nelson, J. A. (2010). Mixed research as a tool for developing quantitative instruments. Journal of Mixed Methods Research. 4(1), 56–78. doi:10.1177/1558689809355805

Patton, M. Q. (2014). Qualitative Evaluation & Research Methods, 4th Edition. Thousand Oaks, CA, US: Sage Publications, Inc.

Philip, B. & Tan, K. H. (2006). Metacognitive Strategy Instruction (MSI) for reading: co-regulation of cognition. Jurnal e-Bangi. 1(1), 1–27.

Ratnam-Lim, C. T. L. & Tan, K. H. K. (2015). Large-scale implementation of formative assessment practices in an examination-oriented culture. Assessment in Education: Principles, Policy & Practice. 22(1), 61–78. doi:10.1080/0969594X.2014.1001319

Rohaya, T., Mohd Zaki, K., Hamimah, A. N. & Adibah, A. L. (2014). From principle to practice: Assessment for Learning in Malaysian school-based assessment classroom. International Journal of Social Sciences and

Education. 4(4), 8. Retrieved from http://ijsse.com/sites/default/files/issues/2014/v4-i4-2014-1/Paper-11.pdf

Ryan, K., Gannon-Slater, N. & Culbertson, M. J. (2012). Improving survey methods with cognitive interviews in small- and medium-scale evaluations. American Journal of Evaluation. 33(3), 414–430.

doi:10.1177/1098214012441499

Sio, J. J. L. & Ramlee, I. (2019). Binary logistic regression analysis of instructional leadership factors affecting English language literacy in primary schools. 3L: The Southeast Asian Journal of English Language Studies. 25(2), 22–37. doi:http://doi.org/10.17576/3L-2019-2502-02

Timans, R., Wouters, P. & Heilbron, J. (2019). Mixed methods research: what it is and what it could be. Theory and Society. 48(2), 193–216. Retrieved from https://link.springer.com/article/10.1007/s11186-019-

-5#Fn7_source

Wiliam, D. (2018). Embedded Formative Assessment, 2nd Edition. Solution Tree Press.

Yahya, A. T., Mohd Sallehhudin, A. A., Kemboja, I. & Alla Baksh, M. A. K. (2014). The washback effect of the General Secondary English Examination (GSEE) on teaching and learning. GEMA Online Journal of Language

Studies. 14(3), 83–103. doi:10.17576/GEMA-2014-1403-06

Zuraidah, M. D., Anna Christina, A., Arshad, A. S., Mardziah Hayati, A., Singh, K. K. K., Lee, B. H., Pillai, J. P. @ L., et al. (2015). English Language Education Reform in Malaysia: The Roadmap 2015-2025. Putrajaya:

Ministry of Education Malaysia.




DOI: http://dx.doi.org/10.17576/3L-2019-2504-13

Refbacks



 

 

 

eISSN : 2550-2247

ISSN : 0128-5157