Predicting L2 Speaking Proficiency Using Syntactic Complexity Measures: A Corpus-Based Study

Shinjae Park

Abstract


This paper discusses the syntactic complexity factors contributing to the achievement of a higher proficiency level in English speech. Here I have examined complexification at the sentential, clausal, phrasal and nominal level of syntactic organisation in a Korean learner spoken corpus using quantitative measures and compared the scores with holistic ratings of learners’ overall speaking quality. After the normality assumption analysis confirmed that the logistic regression was appropriate, an analysis was performed to ascertain the effects of complexity measures on participants’ L2 proficiency. First, length-based complexity features, namely, MLT and coordinated phrases, namely, CPT and CPC were found to be predictors for English speaking proficiency. Next, the logistic regression model was statistically significant and explained 36.3% of the variance in classification according to L2 proficiency and correctly classified 75.4% of cases. Results also showed that when learners come to use the coordinated phrases per clause proficiently, they were over 24 times more likely to achieve higher proficiency in spoken English. Finally, an effective equation was proposed to help educators classify EFL learners according to proficiency in L2 speech after gauging the selected complexity dimensions. However, more comprehensive studies which consider other methods of unit segmentation for spoken data or include more measures to predict L2 speech proficiency, are necessary to verify the results of this study.

 

Keywords: Learner spoken corpus; Syntactic complexity; Monologue; EFL; Logistic regression


Full Text:

PDF

References


Ai, H. & Lu, X. (2013). A corpus-based comparison of syntactic complexity in NNS and NS university students’ writing. In A. Díaz-Negrillo, N. Ballier & P. Thompson (Eds.), Automatic treatment and analysis of learner corpus data (pp. 249–64). John Benjamins Publishing Company.

Bardovi-Harlig, K. (1992). A second look at T-unit analysis: Reconsidering the sentence. TESOL Quarterly, 26(2), 390–5. https://www.jstor.org/stable/3587016

Biber, D., Gray, B. & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45(1), 5–35. https://doi.org/10.5054/tq.2011.244483

Bulté, B. & Housen, A. (2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42–65. https://doi.org/10.1016/j.jslw.2014.09.005

Chen, M. & Zechner, K. (2011). Computing and evaluating syntactic complexity features for automated scoring of spontaneous non-native speech [Paper presentation]. 49th Annual Meeting of the Association for Computational Linguistics: Human

Language Technologies (pp. 722–31).

Council of Europe. (2001). The common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.

Crookes, G. (1990). The utterance, and other basic units for second language discourse analysis. Applied linguistics, 11(2), 183–99. https://doi.org/10.1093/applin/11.2.183

DeCoster, J. & Claypool, H. (2004). Data analysis in SPSS. Retrieved October 2, 2015, from https://www.academia.edu/15281435/Data_analysis_in_SPSS

Ellis, R. (2003). Task-based language learning and teaching. Oxford University Press.

Foster, P., Tonkyn, A., & Wigglesworth, G. (2000). Measuring spoken language: A unit for all reasons. Applied Linguistics, 21(3), 354–75. https://doi.org/10.1093/applin/21.3.354

Glover, P. (2011). Using CEFR level descriptors to raise university students’ awareness of their speaking skills. Language Awareness, 20(2), 121–33. https://doi.org/10.1080/09658416.2011.555556

Halliday, M. A. K. (2006). The language of science. Bloomsbury Publishing.

Hunt, K. W. (1965). Grammatical structures written at three grade levels. National Council of Teachers of English.

Iwashita, N., Brown, A., McNamara, T. & O’Hagan, S. (2008). Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29(1), 24–49. https://doi.org/10.1093/applin/amm017

Iwashita, N. (2006). Syntactic complexity measures and their relation to oral proficiency in Japanese as a foreign language. Language Assessment Quarterly, 3(2), 151–69. https://doi.org/10.1207/s15434311laq0302_4

Kim, J-Y. (2014). Predicting L2 writing proficiency using linguistic complexity measures: A corpus-based study. English Teaching, 69(4), 27–51.

Kyle, K. (2016). Measuring Syntactic Development in L2 Writing: Fine Grained Indices of Syntactic Complexity and Usage-based Indices of Syntactic Sophistication [Unpublished doctoral dissertation]. Georgia State University.

Kyle, K. & Crossley, S. A. (2018). Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. The Modern Language Journal, 102(2), 333–49. https://doi.org/10.1111/modl.12468

Lintunen, P. & Mäkilä, M. (2014). Measuring syntactic complexity in spoken and written learner language: comparing the incomparable? Research in Language, 12(4), 377–99.

Lorenzo, F. & Rodríguez, L. (2014). Onset and expansion of L2 cognitive academic language proficiency in bilingual settings: CALP in CLIL. System, 47. 64–72. https://doi.org/10.1016/j.system.2014.09.016

Lu, X. & Ai, H. (2015). Syntactic complexity in college-level English writing: Differences among writers with diverse L1 backgrounds. Journal of Second Language Writing, 29, 16–27. https://doi.org/10.1016/j.jslw.2015.06.003

Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International journal of corpus linguistics, 15(4), 474–96. https://doi.org/10.1075/ijcl.15.4.02lu

Martínez, A. C. L. (2018). Analysis of syntactic complexity in secondary education EFL writers at different proficiency levels. Assessing Writing, 35, 1–11. https://doi.org/10.1016/j.asw.2017.11.002

Neary‐Sundquist, C. A. (2017). Syntactic complexity at multiple proficiency levels of L2 German speech. International Journal of Applied Linguistics, 27(1), 242–62. https://doi.org/10.1111/ijal.12128

Norris, J. & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–78. https://doi.org/10.1093/applin/amp044

Ortega, L. (2015). Syntactic complexity in L2 writing: Progress and expansion. Journal of Second Language Writing, 29, 82–94. https://doi.org/10.1016/j.jslw.2015.06.008

Park, S. (2020). Determining L2 Proficiency in the production and perception of consonant clusters. 3L: Language, Linguistics, Literature, 26(3). http://doi.org/10.17576/3L-2020-2603-04

Park, S. & Yoon, S. (2021). Syntactic Complexity of EFL Learners’ Casual Conversation, Monologue, and Writing, The Journal of Studies of Linguistics, 37(1), 75-89. https://doi.org/10.18627/jslg.37.1.202105.075

Pohar, M., Blas, M., & Turk, S. (2004). Comparison of logistic regression and linear discriminant analysis: a simulation study. Metodoloski zvezki, 1(1), 143–61.

Sio, J., & Ismail, R. (2019). Binary logistic regression analysis of instructional leadership factors affecting English language literacy in primary schools. 3L: Language, Linguistics, Literature®, 25(2), 22-37. http://doi.org/10.17576/3L-2019-2502-

Taguchi, N., Crawford, W., & Wetzel, D. Z. (2013). What linguistic features are indicative of writing quality? A case of argumentative essays in a college composition program. TESOL Quarterly, 47(2), 420–30.

https://www.jstor.org/stable/43267799

Wang, S. & Slater, T. (2016). Syntactic complexity of EFL Chinese students’ writing. English Language and Literature Studies, 6(1), 81–6.

Wolfe-Quintero, K., Inagaki, S., & Kim, H-Y. (1998). Second language development in writing: Measures of fluency, accuracy, & complexity (No. 17). University of Hawaii Press.

Yang, W., Lu, X., & Weigle, S. C. (2015). Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing, 28, 53–67.

https://doi.org/10.1016/j.jslw.2015.02.002

Yoon, S. & Park, S. (2021). Grammatical complexity of EFL learners’ casual conversation at different proficiency levels. Korean Journal of English Language and Linguistics, 21, 599-616. http://doi.org/10.15738/kjell.21..202107.599




DOI: http://dx.doi.org/10.17576/3L-2021-2704-08

Refbacks

  • There are currently no refbacks.


 

 

 

eISSN : 2550-2247

ISSN : 0128-5157