Write & Improve is a free online automated English language writing assessment and feedback tool that uses advances in computational linguistics and machine learning to provide writing practice and feedback for EFL learners in an intuitive, engaging and easily interpretable way. Importantly, it’s a pedagogical tool and a practice environment, not a text editing facility.
Write & Improve's error feedback works by supervised machine learning based on an algorithm (a set of rules or instructions that are given to a computer so that it can solve a problem) which is given training data from the 30-million-word error-annotated Cambridge Learner Corpus and, from that data, 'learns' to spot the same errors and patterns of error in any future learner writing that it is given – there is a continuous improvement process. Basically, it ‘speaks’ learner English, exclusively. And as it receives more data from users and our annotation team annotate that data and feed it back into the pipeline, it learns more and gives more and more accurate feedback. It was out in Beta, collecting data and learning from it for more than three years before launch in September 2016, so it’s already very accurate. But, like all of us, it’s always learning.
Because it’s a pedagogical tool and because it can be used with or without a teacher, the algorithm is deliberately set to be extremely cautious.
It only indicates possible errors when it’s more than 90% certain it’s right, as research is clear* that when it comes to feedback in pedagogy, an error left unmarked is less damaging than a correct use marked wrong. Precision is key here. This is all the more important when our users may be using Write & Improve without the guidance of a teacher.
The system also does not make suggestions if it’s unable to be sufficiently certain about what the learner meant to write. Instead, it highlights the whole sentence as unusual and needing attention. The learner is then, at least, pointed to areas where they could usefully focus their attention.
The algorithm’s also carefully set not to give too much feedback at one time. We all know that too much red pen is ultimately demotivating and counter-productive. Instead, it returns initial feedback on common errors after first submission and then, as the learner edits those errors, surrounding errors become marked on next view, where possible and appropriate. This is designed to keep the user motivated and engaged.
The feedback
There are four types of feedback:
1. Summative, overall assessment of writing competence, giving an estimated CEFR level, allowing learners to benchmark their progress against other learners all over the world.
2. Indirect, semi-corrective feedback on word-level issues, including spelling, grammar and vocabulary choice, plus a qualitative assessment of each individual sentence, making the learner aware of areas that might need attention. Crucially, with this feedback there is no “right answer” – don’t do that; do this. Write & Improve only makes suggestions on areas that need attention and why. It encourages students to revise their work and assess the feedback to make their own decisions. There’s no spoon-feeding here!
3. Overview feedback, served with a lot of teacherly encouragement. It lets the learner know how they’re doing now – measuring themselves against their last performance, not anyone else’s. In this sense, it is personal, adaptive and formative. There are currently 268 different graded feedback bubbles to suit different achievement scenarios and levels, but this number is growing all the time.
4. A personal progress graph to help learners visualise their progress on a task:
This overview feedback has an important function. It takes the focus away from levels in cases where the level has not improved. It emphasises that progress is often in small, steady steps, and not in six stages, from A1 to C2, but little by little, within a level. That every little bit of progress counts!
This article was adapted from a longer blog post about Write & Improve and the people and research behind it. You can read it here: https://eltjam.com/developing-an-elt-product-based-on-machine-learning-write-improve/