Coming Full Circle —From CEFR to CEFR-J and back

2020

The Common European Framework of Reference for Languages (CEFR) defines six levels of learner proficiency, and links them to particular communicative abilities. The CEFRLex project aims at compiling lexical resources that link single words and multi-word expressions to particular CEFR levels. The resources are thought to reflect second language learner needs as they are compiled from CEFR-graded textbooks and other learner-directed texts. In this work, we investigate the applicability of CEFRLex resources for building language learning applications. Our main concerns were that vocabulary in language learning materials might be sparse, i.e. that not all vocabulary items that belong to a particular level would also occur in materials for that level, and, on the other hand, that vocabulary items might be used on lower-level materials if required by the topic (e.g. with a simpler paraphrasing or translation). Our results indicate that the English CEFRLex resource is in accordance with e...

2012

In this talk, I will report on the on-going project on systematic extraction of criterial features from multiple source corpora based on the Common European Framework of Reference for Languages (CEFR). First, a brief description of the CEFR itself, the project and the design of several different corpora newly compiled for the project will be given, followed by methodological issues regarding how to extract criterial features from CEFR-based corpora using machine learning techniques. The CEFR-J and Reference Level Descriptions The project aims to support the implementation of the CEFR-J, an adaptation of the CEFR into English language teaching in Japan (Tono & Negishi 2012). After the release of Version 1 of the CEFR-J in March, 2012, we launched a new government-funded project called the “CEFR-J Reference Level Description (CEFR-J RLD)” Project. RLD is a term used for the CEFR to prepare an inventory of language (lexis and grammar) for each individual language for the purpose of lev...

Interdisciplinary Language and Literature Studies, 2020

The MERLIN corpus is a written learner corpus for Czech, German, and Italian that has been designed to illustrate the Common European Framework of Reference for Languages (CEFR) with authentic learner data. The corpus contains 2,290 learner texts produced in standardized language certifications covering CEFR levels A1-C1. The MERLIN annotation scheme includes a wide range of language characteristics that enable research into the empirical foundations of the CEFR scales and provide language teachers, test developers, and Second Language Acquisition researchers with concrete examples of learner performance and progress across multiple proficiency levels. For computational linguistics, it provide a range of authentic learner data for three target languages, supporting a broadening of the scope of research in areas such as automatic proficiency classification or native language identification. The annotated corpus and related information will be freely available as a corpus resource and through a freely accessible, didactically-oriented online platform.

ELT Journal, 2017

Despite the current importance of the Common European Framework of Reference for Languages (CEFR) in the learning, teaching, and assessment of languages, limitations arise in the use of the CEFR descriptors, which are also present in the European Language Portfolio (ELP). This article highlights the main challenges posed to CEFR and ELP users by the linguistic competence descriptors-with a particular focus on the grammatical accuracy descriptors, and strategy descriptors for monitoring and repair at B1 level-when they try to self-assess their written production activities. In order to address these limitations, a Computer-aided Error Analysis (CEA) was performed on a learner corpus comprising B1-level texts produced by Spanish learners of English. The results obtained enabled the reformulation of the descriptors for written production activities at CEFR B1 level aimed at L1 Spanish learners of English, by complementing the existing descriptors with further linguistic information on the most frequent errors at that level. After more than 20 years of research, the Common European Framework of Reference for Languages (CEFR) was published in 2001 by the Council of Europe to provide 'a common basis for the elaboration of language syllabuses, curriculum guidelines, examinations, textbooks, etc. across Europe' (Council of Europe 2001: 1). In addition to establishing a common metalanguage that encompasses the main aspects related to language teaching, learning, and assessment, two further aims can be found underpinning the CEFR (North 2007: 659). The first is to promote reflection on learners' needs, establish objectives and identify ways to follow up and check their progress. The second involves establishing a series of levels which considers the learners' use of the language from a communicative point of view. To date, the CEFR has been translated into 39 languages and has been adopted, to varying degrees, by countries throughout Europe and beyond.

ArXiv, 2018

This paper analyses the contribution of language metrics and, potentially, of linguistic structures, to classify French learners of English according to levels of the Common European Framework of Reference for Languages (CEFRL). The purpose is to build a model for the prediction of learner levels as a function of language complexity features. We used the EFCAMDAT corpus, a database of one million written assignments by learners. After applying language complexity metrics on the texts, we built a representation matching the language metrics of the texts to their assigned CEFRL levels. Lexical and syntactic metrics were computed with LCA, LSA, and koRpus. Several supervised learning models were built by using Gradient Boosted Trees and Keras Neural Network methods and by contrasting pairs of CEFRL levels. Results show that it is possible to implement pairwise distinctions, especially for levels ranging from A1 to B1 (A1=>A2: 0.916 AUC and A2=>B1: 0.904 AUC). Model explanation re...

International Journal of Learner Corpus Research, 2020

This report outlines the development of a new corpus, which was created by refining and modifying the largest open-access L2 English learner database the EFCAMDAT. The extensive data-curation process, which can inform the development and use of other corpora, included procedures such as converting the database from XML to a tabular format, and removing problematic markup tags and non-English texts. The final dataset contains two corresponding samples, written by similar learners in response to different prompts, which represents a unique research opportunity when it comes to analyzing task effects and conducting replication studies. Overall, the resulting corpus contains ~406,000 texts in the first sample and 317,000 texts in the second sample, written by learners representing diverse L1s and a large range of L2 proficiency levels.

2010

This report focuses on the steps taken since 2008 to introduce the CEFR into the teaching of foreign languages other than English, at Muroran Institute of Technology. Seven part-time instructors of Chinese and Russian collaborated over the time of one year in developing teaching materials, as well as can-do-lists based on the level A1 of the CEFR. For this task, the CEFR-based German textbook "Und du?" was used as a reference.

2019

This paper will first outline and discuss the revised version of the Common European Framework of Reference Languages: Learning, teaching and assessment (CEFR) [2018] together with the Frameworks of Reference for English Language Education in Thailand Malaysia, Vietnam, Japan and China which are based on the CEFR. The indications are of potentially several issues that need to be addressed, including the fact that the local versions of CEFR were mainly based on the 2001 framework and not the 2018 which came later. Other issues such as using the same proficiency scales as the basis for rating scale criteria may lead to perceived equivalence but does not necessarily lead to greater comparability of shared criteria. There are also indications from a number of studies that the perceived view that CEFR as being mainly an assessment tool rather than about language competency may result in a negative attitude from both teachers, students and stake-holders.