Natural Language Processing and Levenshtein Distance for Generating Error Identification Typed Questions on TOEFL

Lala Septem Riza, Faisal Syaiful Anwar, Eka Fitrajaya Rahman, Cep Ubad Abdullah, Shah Nazir

Abstract


Test of English as a Foreign Language (TOEFL) is one of the evaluations requiring good quality of the questions so that they can reflect the English abilities of the test takers. However, it cannot be denied that making such questions with good quality is time consuming. In fact, the use of computer technology is able to reduce the time spent in making such questions. This study, therefore, develops a model to generate error identification typed questions automatically from news articles. Questions from the sentences on news sites are created by utilizing Natural Language Processing, Levenshtein Distance, and Heuristics. This model consists of several stages: (1) data collection; (2) preprocessing; (3) part of speech (POS) tagging; (4) POS similarity; (5) choosing question candidates based on ranking; (6) determining underline and heuristics; (7) determining a distractor. Testing ten different news articles from various websites, the system has produced some error identification typed questions. The main contributions of this study are that (i) it can be used as an alternative tool for generating error identification typed questions on TOEFL from news articles; (ii) it can generate many questions easily and automatically; and (iii) the question quality are maintained as historical questions of TOEFL.


Full Text:

PDF

References


Fulcher, G. (2010). Practical language testing. London: Hodder Education.

ETS. (2003). TOEFL Practice TESTS volume 1. Princeton.

Cotton, K. (2001). Classroom questioning. School improvement research series. 3.

Aldabe, I., Lacalle, M. L. D., Maritxalar, M., Martinez, E., & Uria, L. (2006). ArikIturri: An Automatic Question Generator Based on Corpora and NLP Techniques. Intelligent Tutoring Systems Lecture Notes in Computer Science, 584–594. doi: 10.1007/11774303_58

Agarwal, M., Shah, R., & Mannem, P. (2011). Automatic question generation using discourse cues. In Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications, 1-9.

Skalban, Y., Specia, L., & Mitkov, R. (2012). Automatic question generation in multimedia-based learning. In Proceedings of COLING 2012: Posters, 1151-1160.

Hoshino, A., & Nakagawa, H. (2005, June). A real-time multiple-choice question generation for language testing: a preliminary study. In Proceedings of the second workshop on Building Educational Applications Using NLP, 17-20.

Navarro, G. (2001). A guided tour to approximate string matching. ACM Computing Surveys (CSUR), 33(1), 31–88. doi: 10.1145/375360.375365

Chowdhury, G. G. (2003). Natural language processing. Annual review of information science and technology, 37(1), 51-89.

Riloff, E. (1999). Information extraction as a stepping stone toward story understanding. Understanding language understanding: Computational models of reading, 435-460.

Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to information retrieval. Cambridge university press.

Jurafsky, D. & Martin, J. H. (2014). Speech and language processing. Pearson Education India.

Ali, H., Chali, Y., & Hasan, S. A. (2010). Automation of question generation from sentences. In Proceedings of QG2010: The Third Workshop on Question Generation, 58-67.

Narendra, A., Agarwal, M., & Shah, R. (2013). Automatic cloze-questions generation. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013, 511-515.

Becker, L., Basu, S., & Vanderwende, L. (2012). Mind the gap: learning to choose gaps for question generation. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 742-751.

Garg, P., & Bedi, E. S. (2014). Automatic Question Generation System from Punjabi Text using Hybrid approach. International Journal of Computer Trends and Technology, 21(3), 130–133. doi: 10.14445/22312803/ijctt-v21p125

Papasalouros, A., Kanaris, K., & Kotis, K. (2008). Automatic Generation of Multiple Choice Questions From Domain Ontologies. In e-Learning, 427-434.

Pandey, S., & Rajeswari, K. C. (2013). Automatic Question Generation Using Software Agents for Technical Institutions. International Journal of Advanced Computer Research, 3(13), 307-311.

Gear, J., & Gear, R. (2002). Cambridge Preparation for the TOEFL® Test Book with CD-ROM (Vol. 1). Cambridge University Press.

Sharpe, P. J. (2004). How to Prepare for the TOEFL. Univ of California Press.

Pyle, M. A. (2000). Cliffs Test Prep TOEFL CBT. Foster City.

Chesla, E. (2002). TOEFL Exam success from LearningExpress. New York: LearningExpress.

Cemaiiiko, J. (2001), TOEFL Grammar Review, Rusia.

Phillips, D. (2001). Longman complete course for the TOEFL test: Preparation for the computer and paper tests. London: Longman.

Marcus, M. P. & Santorini, B. (1993). Building a large annotated corpus of English: The Penn treebank. Computational Linguistics, vol. 19(2). 313-330.

Araki, J., Rajagopal, D., Sankaranarayanan, S., Holm, S., Yamakawa, Y., & Mitamura, T. (2016). Generating questions and multiple-choice answers using semantic analysis of texts. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 1125-1136.

Riza, L. S., Bergmeir, C. N., Herrera Triguero, F., & Benítez Sánchez, J. M. (2015). frbs: Fuzzy rule-based systems for classification and regression in R. Journal of Statistical Software, 65(6), 1-30.

Riza, L. S., Janusz, A., Bergmeir, C., Cornelis, C., Herrera, F., Ślezak, D., & Benítez, J. M. (2014). Implementing algorithms of rough set theory and fuzzy rough set theory in the R package “RoughSets”. Information Sciences, 287, 68-89.

Nazir, S., Shahzad, S., & Riza, L. S. (2017). Birthmark-based software classification using rough sets. Arabian Journal for Science and Engineering, 42(2), 859-871.

Goto, T., Kojiri, T., Watanabe, T., Iwata, T., & Yamada, T. (2010). Automatic generation system of multiple-choice cloze questions and its evaluation. Knowledge Management & E-Learning: An International Journal, 2(3), 210-224.

Susanti, Y., Iida, R., & Tokunaga, T. (2015). Automatic generation of english vocabulary tests. In CSEDU (1), 77-87.

Hill, J., & Simha, R. (2016). Automatic generation of context-based fill-in-the-blank exercises using co-occurrence likelihoods and Google n-grams. In Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, 23-30.

Blšták, M., & Rozinajová, V. (2016). Automatic question generation based on analysis of sentence structure. In International Conference on Text, Speech, and Dialogue, 223-230.

Huang, Y., & He, L. (2016). Automatic generation of short answer questions for reading comprehension assessment. Natural Language Engineering, 22(3), 457-489.

Pannu, S., Krishna, A., Kumari, S., Patra, R., & Saha, S. K. (2018). Automatic Generation of Fill-in-the-Blank Questions from History Books for School-Level Evaluation. In Progress in Computing, Analytics and Networking, 461-469.




DOI: https://doi.org/10.17509/jcs.v1i1.24940

Refbacks

  • There are currently no refbacks.