1. Introduction
Automatic translation system, as an important application in the field of natural
language processing, has made remarkable progress in recent years. However, with the
acceleration of globalization and the increasing cross-cultural communication, the
translation system faces more and more challenges. Especially when it comes to cultural
nuances, idioms, and complex contexts, traditional translation methods (such as statistical
machine translation and rule-based translation systems) are often difficult to achieve
satisfactory translation results. English grammatical error correction (GEC) is a
crucial aspect of natural language processing (NLP), focusing on identifying and rectifying
grammatical errors by analyzing grammatical dependencies and logicality in input sentences.
In second language learning, ESL learners must promptly spot spelling errors and ensure
grammatical accuracy, as grammar underpins precise, clear, and effective communication
[1]. Efficient GEC tools aid learners in revising their work, enhancing grammar perception
and application, and overcoming grammar barriers in ESL learning. They also alleviate
teachers' classroom workload and teaching pressure. Research on automatic GEC informs
machine understanding of natural language, intelligent knowledge processing, and semantic
search technology, providing insights for foreign and native language learning [2,
3]. This exploration helps students grasp English word meanings accurately, reduces
misuse, improves grammatical proficiency, enhances learning initiative, and boosts
English learning efficiency.
The traditional translation system refers to the process of using the artificial way
to transform the content of one language into another language. It depends on the
translator's language understanding, expression ability and the understanding of the
culture, background and grammar of the two languages. It has existed for thousands
of years and is one of the main translation methods. With NLP advancements, GEC methods
have evolved from rule-based, classification-based, and language model-based to the
current mainstream machine translation-based approaches. Each method has its strengths
and limitations. For instance, rule-based methods target syntax errors precisely but
require vast rule bases and may encounter rule conflicts. Classification-based GEC
excels in correcting articles and prepositions but covers fewer error types. Neural
machine translation handles long-distance dependencies well but poses high model training
complexity. Currently, data-driven and machine translation-based methods dominate
GEC tasks, with neural machine translation (NMT) models making significant progress
[4,
5]. These models effectively handle long-distance semantic dependencies, enabling improved
syntax error correction. Researchers utilize Encoder-Decoder structures to train multiple
GEC models on artificially synthesized datasets, jointly correcting input sentences
at the source to mitigate overfitting and boost generalization. They further refine
the decoding results with external features like language models, edit distance, and
edit operations.
When labeled corpora are scarce, synthesizing high-quality training data is crucial
for improving the performance of GEC models [6,
7]. Data augmentation methods based on seed corpus error distributions, have been proposed.
However, such data may not align with the natural distribution of grammatical errors,
limiting their direct application to GEC tasks. Additionally, these methods often
face challenges in ensuring data quality. The quality of the generated training data
will also be uneven due to the pros and cons of the translation model. The data augmentation
method based on revision history can collect manually corrected historical data, which
is more authentic, but usually requires data cleaning, quality control and other operations
to identify and delete irrelevant and inaccurate data. Nowadays, the GEC model widely
uses data augmentation methods to solve the problem of training in the GEC field.
The problem of lack of data; Using model integration and other methods to achieve
model optimization; A multi-feature reordering strategy is used to sort the candidate
corrective sentences to optimize the output results [8,
9]. Usually, the combination of the three can make the model output the best correction
results.
2. Relevant Theories and Methods
This chapter discusses theories and methods in syntax error detection and correction,
including corpora, evaluation methods, detection/correction models, and data augmentation
techniques. Machine translation-based GEC methods, stemming from statistical and neural
approaches, yield promising results. However, a lack of manually labeled data poses
challenges. To enhance GEC system performance, techniques beyond traditional models
have emerged. Multi-interaction feature fusion is an advanced algorithm that improves
the accuracy of the translation system by dynamically integrating syntactic structure,
semantic information and pragmatic elements. This fusion approach aims to capture
those nuances and contextual cues that are difficult to capture by traditional translation
methods, thus improving the naturalness and accuracy of translation. Fig. 1 illustrates the typical GEC system architecture for syntax error detection and correction
tasks.
Fig. 1. Automatic English translation efficiency optimization process.
2.1. Grammatical Error Detection and Correction Task-Related Corpus
Deep learning has achieved notable advances in NLP tasks, but detecting and correcting
English grammatical errors remains challenging [10]. This stems from the diversity of syntax errors, semantic interdependencies, and
the scarcity of manually labeled data. To enhance GEC model performance, the learner
corpus size is continuously expanded. Corpora in the GEC field include monolingual
corpora and learner corpora. Monolingual corpus has the advantages of rich text types,
large amount of data, and easy collection; The learner corpus consists of “wrong-right”
parallel sentence pairs, which contain high-quality tagged corpus, but the data scale
is small.
2.1.1 The One-Billion Word Benchmark corpus
The One-Billion Word Benchmark is a benchmark corpus of nearly 1 billion words. The
data set was released by Cornell University in 2013 and is mainly derived from the
web page. The base material library can be used to quickly evaluate language modeling
techniques [11,
12]. Therefore, in order to improve the quality of the data, the data in the corpus have
undergone preprocessing operations, including de-duplication, normalization, word
segmentation and other operations. In addition, the dictionaries in the corpus are
composed of 79347 words with word frequency higher than three markers, which represent
the word markers outside the dictionary, the beginning markers of sentences and the
end markers of sentences respectively. The publication of this corpus further improves
language model performance.
2.1.2 Tatoeba corpus
The Tatoeba Corpus is a new benchmark corpus published for machine translation. The
current version contains more than 500GB of compressed data and covers 555 languages
[13,
14]. The data set provides some parallel corpus and monolingual corpus, which provide
data source support for data augmentation methods.
2.1.3 Wikipedia corpus
Wikipedia is an online Encyclopaedia based on Wikipedia technology, written in multiple
languages, with all-encompassing content, both arts and sciences. The Simple Wiki
corpus uses only about 1500 common English words compared to the usual Wikipedia corpus,
which makes the sentences more understandable grammatically [15], semantically, and structurally. The Wikipedia corpus contains modified editorial
records (editorial histories) from different periods. The error statement is provided
by an older Wikipedia page consisting of the manual correction, the time of correction,
the name of the person who corrected it, and the revised summary. The correct statement
is provided by the latest Wikipedia page.
2.2. GEC Evaluation Index
In recent years, English grammar error correction has made an important breakthrough
in the field of natural language processing, and a variety of evaluation methods for
GEC systems have been produced. Evaluation methods usually compare the output sequence
with the gold standard sequence (manually labeled target sentences). During the comparison
process, the output correction sentence and the target sentence need to be word aligned.
Previously, the performance of the GEC model was only measured by accuracy [16]. Later, the GEC system can be measured by the recall rate of M2 and I indicators.
Nowadays, the generation of GLEU and ERRANT error kits makes the evaluation results
of the GEC system fairer and more reasonable. The system was evaluated by accuracy
(P) and Fos values. The calculation formulas are as shown in Formulas (1)-(3).
2.3. GELU Algorithm Principle
Jia et al. [17] introduced the GLEU metric for sentence fluency evaluation, a BLEU variant. Let S
be the error sentence, R the manually labeled standard, and W the uniform weight.
The GEC system's corrected output is C. GLEU calculates the weighted accuracy Pn of
C compared to R. BP (Brevity Penalty) factors in the lengths of C and R. GLEU is used
specifically for the JFLEG dataset. The formula was calculated as described in Eqs.
(4) and (5).
ERRANT Toolkit extracts the number of edits from the source sentence and the corrected
sentence, realizes the classification of error types, and helps to evaluate the types
of grammatical errors. The tool uses an alignment algorithm between source and correcting
sentences proposed by Razia et al. and uses a rule-based error type framework to assign
error types [18,
19]. The frame is independent of the dataset, relying only on information such as part-of-speech
markers. For M2, I index, GLEU and other evaluation indicators, because the output
of GEC system is not annotated, the performance of English error correction can only
be measured by the recall rate. The ERRANT toolkit overcomes this shortcoming. Previous
metrics tend to evaluate the overall performance of the system, but systems with poor
overall performance may have an advantage in correcting specific error types. ERRANT
Syntax Error Annotation Toolkit proposes the idea of error type analysis to evaluate
the performance of the system more comprehensively. The calculation formula is described
as Eq. (6).
3. Syntax Error Generation Model
In this paper, the open-source FAIR Sequence-to-Sequence toolkit is used to implement
the model structure. The dimension of the word vector and the target end of the GEG
system is 512, the network layer contained in the encoder and decoder is 6 layers,
the hidden layer (FFN) dimension of the forward neural network sublayer is set to
4096, the dropout rate is set to 0.2, the optimization algorithm uses Adam [20], 8 attention heads, the initialization learning rate is 0.002, the label smoothing
rate is set to 0.2, the warmup step size is 16000, and the sampling decoding is used.
The experiment used the CoNLL-2014 test set and M2 as evaluation indicators.
Fig. 2. Multi-interactive feature fusion NLP algorithm process.
Fig. 2 shows multi-interactive feature fusion NLP algorithm process. Data augmentation can
be regarded as a way to preprocess the data of training set. The core of the multi-interaction
feature fusion algorithm is the dynamic integration of language features such as syntactic,
semantic and pragmatic. Syntactic features focus on the structural information of
sentences, such as word order, phrase and clauses; semantic features include the meaning
of words, synonyms and antonyms, and context, politeness and implied meaning. By combining
these features, the algorithm is able to more accurately capture the intention and
context information of the original text and thus, improve the accuracy of translation.
In order to improve the quality of the data, it is necessary to clean up the data
before performing tasks such as data augmentation and model training. For example,
before using Lang-8 data, the training data needs to be cleaned up [21]. The FPN class represents the case that FP and FN are satisfied at the same time,
and the calculation formulas of P, R and F are described in Eqs. (7)-(9).
For the learner corpus containing special symbols, the operation of deleting special
symbols is performed. Translation model loss function and optimization algorithm update
formulas are shown in Eqs. (10) and (11).
3.1. Syntax Error Generation Model and Experimental Process Design
To better understand the context, we explored lexical, syntactic, and semantic feature
extraction techniques. At the lexical level, we employed word frequency statistics
and word embedding techniques to capture the semantic information and distribution
properties of words. In terms of syntactic features, we use analytic trees and dependency
graphs to extract the structural information of sentences. Semantic features are acquired
through semantic role annotation and named entity recognition technology. These feature
extraction techniques work together on the original text and provide rich linguistic
information for the subsequent translation process. Firstly, the validity of this
experiment is verified on the existing three learner corpus NUCLE, FCE, W-IHLOCNESS
and the monolingual corpus One Billion Word. In addition, in order to facilitate the
comparison with the existing work, the Lang-8 corpus is used to fine-tune the GEC
model at the end of this paper. Table 1 shows all the training corpus used in this paper and the corresponding data scales.
By counting the sentence length distribution in the learner corpus, the sentence length
in the monolingual corpus is constrained, and the training data length in the monolingual
corpus is reduced to the token length of 5 to 100. Table 1 shows all the training corpus used in this paper and the corresponding data scales.
Table 1. All the training corpus used in this paper and the corresponding data scales.
|
|
Corpus name
|
Number of sentences
|
Marks (tokens)
|
|
Learner corpus
|
NUCLE
|
57151 (approx. 57K)
|
Approximately 1.2 M
|
|
FCE
|
28350 (approx. 28K)
|
Approximately 455K;
|
|
Wi & locness
|
34308 (approx. 34K)
|
Approximately 628K;
|
|
Social Media Volume data
|
Lang-8
|
1.04 million (approximately 1.04 M)
|
Approximately 11.86 M Lingua franca
|
|
Material storage
|
One Billion word
|
1.7 million (about 1.7 M)
|
Approximately 19M
|
The probabilities of introducing the four rules are 0.3, 0.25, 0.25, 0.217. Secondly,
through the analysis of the distribution of error types in the learner corpus, it
is found that verb (about 7%), noun (about 4.5%), article (about 10.86%), spelling
(about 9.59%), preposition (about 11.2%) and punctuation errors (about 9.7%) account
for a high proportion. In order to solve these error types, this paper proposes a
strategy of substitution rule derivative mutation. By counting the words with a word
frequency number higher than 3 in the monolingual corpus, the first 7000 words are
reserved to form a dictionary. Feature fusion weights and multi-interaction feature
extraction formulas are shown in Eqs. (12) and (13).
In the process of synthesizing data, one of the four operations of insertion, deletion,
replacement and exchange is randomly performed on the selected words to be operated
on. The artificial rule is defined as follows:
(1) Insert error (insert type)
The original sentence S without semantic errors, its sequence S is w0w1...w’...wi-1wi,
and the error statement T obtained by inserting a rule and adding a token (token),
the sequence T is: w0w1...w’...wi-1wi.Weighted accuracy WAcc calculation formula is
described as Eq. (14).
(2) Delete error (delete type)
The original sentence S without semantic errors, its sequence S is w0w1...w’...wi-1wi,
through the deletion rule, randomly delete a token (token) w, and the resulting error
statement T, the sequence T is: w0w1... wi-1wi.
(3) Exchange error (replace type)
The original sentence S without semantic errors, whose sequence S is w0w1...w’...wi-1wi,
by exchanging any two tokens (tokens), the error statement T is obtained, and the
sequence T is: w0w1...wi-1... wi.
(4) Substitution error (swap type)
The original sentence S without semantic errors, whose sequence S is w0w1...w’...wi-1wi,
uses the confusion set created in this paper to replace the randomly selected token
w: if w is in the list of words to be replaced in the confusion set, then randomly
select an alternative word from the list of candidate sets corresponding to the token
to replace it. If it does not appear in the list of confusion sets, then choose a
word from the dictionary to replace it. The resulting error statement T, sequence
T is: w0w1...w... wi-1wi. For the replacement rule, as far as possible, randomly select
a word from the created candidate set to replace. Constructing a variety of polymorphic
errors through artificial rules can provide more training data for GEG and GEC models.
The specific experimental process of the rule-based data augmentation method is as
follows:
Step 1: Use the three existing learner corpus to train the GEC model of this paper,
and verify the results on the CoNLL-2014 test set.
Step 2: Use rule-based data augmentation method to generate training data with file
sizes of 20M (the number of sentences is about 190K), 80M, 140M, and 200M, train GEC
models, and compare the use of different scales The impact of data on training GEC.
The BLEU score, feature importance assessment, and sequence-to-sequence model encoding-decoding
formulas are shown in Eqs. (15)-(17).
Step 2: (GEG2→GEC) In order to further optimize the GEG2 model, this paper tries to
combine the two data augmentation methods. First, the rule-based method is used to
synthesize the training data, pre-train the error generation model, and then fine-tune
it with the learner corpus to obtain the optimized error generation model GEG2. Using
GEG, the model processes monolingual corpus, and generates synthetic data with file
sizes of 20M, 80M, 140M, and 200M respectively. The synthetic data are used for the
training of the GEC model to verify the fusion effect of different data augmentation
methods.
3.2. Experimental Results and Analysis
Training data of different scales generated by rule-based data augmentation methods
are used to train the GEC model together with the learner corpus. Its results on the
CoNLL-2014 test set are shown in Fig. 3. With the increasing of the scale of synthetic data, the R value and F0.5 value of
the GEC model trained using synthetic data can reach 23.7% and 37.9%, respectively,
exceeding the 15.7% and 11.4% of the GEC model trained using only the learner corpus
[22]. However, there is a large difference in accuracy, which is lower than the 18.4%
of the GEC model trained using the learner corpus.
Fig. 3. Results on the CoNLL-2014 test set.
Fig. 3 shows results on the CoNLL-2014 test set. Analyzing the reasons, using the rule-based
data augmentation method, with the continuous increase of data scale, the performance
of the trained syntax error correction model on the test set has been improved to
a certain extent. It shows that the amplification of the data has a significant impact
on the overall performance of the GEC model.
When the GEC model is trained with 200M synthetic data, its performance in recall
rate exceeds that of the GEC model trained with only learner corpus. This is due to
the data augmentation strategy used in this paper, that is, in the process of generating
training data, part of speech errors, punctuation errors, and spelling errors of different
words are incorporated. Compared with the simple random word selection from the vocabulary,
the error types are richer and more targeted, and the recall rate is significantly
improved.
Fig. 4. Translation efficiency versus the bar graph.
Fig. 4 shows translation efficiency versus the bar graph. As for the accuracy of model recognition,
with the increase of synthetic training data, the results continue to improve, but
there is a certain gap between the accuracy of the model obtained by training only
using the learner corpus. The learner corpus contains English grammatical errors in
real situations, and the types of errors are abundant [23]. Compared with the learner corpus, the data augmentation method proposed in this
paper still has a certain gap between the GEC model and the learner corpus in terms
of the types and quantities of errors. However, the gap can be made up by the amount
of data. Therefore, how to further improve the quality of data has important research
significance.
Fig. 5. GEC model performance trained on data of different sizes.
Fig. 5 shows GEC model performance trained on data of different sizes. The resulting GEC
model is trained using different scale synthetic data generated by different GEG models,
and the results are shown in Table 2. The F0.5 value of the best GEC model using the GEG model is 0.286. Compared with
the GEG model, the performance of the GEC model using the GEG and synthesized 200M
training data is about 5.2%, 3.9%, 4.9% higher than that using the GEG model, 3.9%,
4.9%, respectively.
Table 2. The resulting GEC models were trained using synthetic data of different sizes
generated by different GEG models.
|
Model
|
Synthetic data scale for training GECs
|
|
20M
|
80M
|
140M
|
|
P
|
R
|
F0.5
|
P
|
R
|
F0.5
|
P
|
R
|
F0.5
|
|
GEG1→GEC
|
0.275
|
0.135
|
0.228
|
0.291
|
0.184
|
0.261
|
0.304
|
0.193
|
0.273
|
|
GEG2→GEC
|
0.359
|
0.153
|
0.282
|
0.36
|
0.232
|
0.343
|
0.372
|
0.243
|
0.335
|
To verify the effectiveness of the algorithm, we designed two sets of experiments.
The first set of experiments compared the effect of single feature fusion and multiple
feature fusion, and the translation results were quantified by manual evaluation and
automatic evaluation tools (e. g., BLEU, NIST, etc.). The results show that the multi-feature
fusion algorithm outperforms the single feature fusion algorithm in both translation
speed and semantic accuracy.
Fig. 6. Plot ISON of translation quality before and after feature fusion.
This paper utilizes rule-synthesized data and a learner corpus to train an error generation
model. Training the GEC model with this synthesized data leads to better results than
directly using rule data. This demonstrates the effectiveness of fusing different
data augmentation methods. As seen in Table 2, expanding the training data synthesized by the GEG1 model improves the GEC model's
performance. However, the performance gains diminish as data size increases from 20M
to 80M compared to 80M to 140M and 140M to 200M [24]. This suggests the model may reach saturation in identifying certain error types.
The side model, focused on rule-based errors, has limitations in identifying errors.
The findings indicate the importance of further optimizing GEG and GEC models to improve
performance. To further improve the quality of translation, we performed a detailed
error analysis. We classify errors into categories such as mistranslation, omission,
and grammatical problems, and provide specific examples. By analyzing these errors,
we found that they mainly result from problems such as inaccurate feature extraction,
insufficient training data, or model limitations. For these problems, we propose corresponding
improvement measures to optimize the algorithm performance.
Fig. 6 shows plot ison of translation quality before and after feature fusion. The GEC model
trained with GEG1 data outperforms that of GEG2 when using the same dataset. Using
GEG2's synthetic data enhances the recall and F-score of the GEC model [25]. Rule-based augmentation ultimately improve error correction performance due to the
enhanced dataset quality.
4. Syntax Error Correction Model
This paper focuses on training the correction model of grammatical errors while giving
consideration to spelling errors and semantic errors. Since the rich monolingual corpus
can provide training data of different written styles for the model, this study provides
rich prior knowledge for the model through operations such as pre-training of the
model. Written text can have different types of errors at the same time, such as spelling
errors, grammatical errors, semantic errors, etc. Spelling errors are often corrected
using separate components such as spell checkers, human-defined specific rule templates,
etc. The spell checker simply compares words in the text to a large number of known
words. If the word is not in the list, it is considered incorrect and corrected. Grammatical
errors refer to errors that cause sentences to fail to conform to English grammatical
rules. Unlike spell checking errors, grammatical checking requires the use of contextual
information. Syntax errors can be divided into structural errors and non-structural
errors. A structural error is an error that can only be corrected by inserting, deleting,
or moving one or more words [26]. A non-structural error is an error that can be corrected by replacing an existing
word with something else. A semantic error refers to the existence of an incorrect
meaning in the text, and this type of error is neither a grammatical error nor a spelling
error. Semantic errors require extensive knowledge to identify, and it is difficult
to correct them. Correct sentence: The telephone is a kind of conventional communication
tool There is no grammatical error in this sentence, but it violates a semantic error.
The telephone is a kind of conventional communication tool. Correct sentence: The
telephone is a kind of conventional communication tool. There is no grammatical error
in this sentence. Therefore, if the model does not have extensive prior knowledge,
it cannot better correct semantic errors.
4.1. Experimental Data Setting and Evaluation Indexes
To comprehensively assess the quality of translation, we employed multiple assessment
criteria. In addition to the commonly used BLEU scores, we considered other indicators
such as METEOR and TER. Meanwhile, we also performed a manual evaluation focusing
on fluency, adequacy and fidelity of translation. To ensure the impartiality of the
assessment results, we developed a detailed assessment protocol and took steps to
mitigate the assessment bias. For the GEC system, the dimension of the word vector
and the dimension of the target end is 512, the network layer contained in the encoder
and decoder is 6 layers, the hidden layer (FFN) dimension of the forward neural network
sublayer is set to 4096, the dropout rate is set to 0.2, using the NAG (Nesterovs
Accelerated Gradient, NAG) optimization algorithm, 8 attention heads, the initialization
learning rate is 0.02, the weight decay is 0.5, the number of cycles when training
stops is 0, and the momentum is set to 0.99, etc. The experiment will use the commonly
used CoNLL-2014 test set and M2 as evaluation indicators. In addition, the JFLEG test
set and GLEU values will also be used to analyze the fluency of the correction results
of GEC. The attention mechanism and the regularization term addition formula are shown
in Eqs. (18) and (19).
First, the rule-based data augmentation strategy proposed in this paper is used to
synthesize training data, and together with the learner corpus, the GEG model is trained.
Then the GEC model is trained with the training data synthesized by the GEG model
and the learner corpus. Inspired by the work of Maria et al. [27] to alternately train the English-Czech translation model, this paper uses the GEC
model to correct the source sentences in the learner's corpus, and reconstructs the
corrected results with the standard reference sentences in the learner's corpus. Parallel
corpus is added to the training data of the GEG model, and the GEG model is trained
again.
Similar to the monolingual corpus used by Tamara et al. [28] in training the reverse translation model, the monolingual data used in this paper
are all from the written texts produced by native English-speaking learners in real
situations, and the sentences are all authentic. The model performance evaluation
index and feature fusion weight calculation formula is shown in Eq. (20).
In order to improve the performance of grammatical error correction, the GEC model
is used to correct the training data in the learner corpus, and the corrected results
and the standard reference sentences in the training set form a parallel corpus, which
is used as the extended data of the repeated training GEC model. The experimental
process of alternating training model is as follows:
Step 1: For the translation-based GEC model, the excellent model will output the target
sentence without syntax errors, and the low-performance model will produce erroneous
output that is not in the actual situation. Because the performance of the GEC model
trained with synthetic data is relatively low, this paper chooses the GEC model to
process the source sentences in three learner corpora, so as to avoid the errors that
some corrected sentences do not conform to grammatical rules due to the low GEC performance.
Then the candidate sentences output from the model are combined with the standard
reference sentences in the learner corpus to form a training set, and the error generation
model is trained again after mixing with the training data synthesized by the rule-based
data augmentation method, and the error generation model GEG is fine-tuned using three
learner corpus;
Step 2 (GEG3→GEC): Using GEG, the model processes the monolingual corpus, and generates
20M, 80M, 140M, and 200M synthetic data with different scales again, and pre-trains
the GEC model respectively.
Step 3: On the basis of the existing synthetic data, in order to further improve the
performance of the GEC model, this paper tries to expand the scale of the pre-training
data.
4.2. Experimental Results of Syntactic Error Correction Based on Alternating Training
Three grammatical error correction models were trained using 200M training data synthesized
by GEG, and three learner corpus. Corrective sentences of the corresponding grammatical
error correction models were defined as Corrective Sentence 1, Corrective Sentence
2, and Corrective Sentence 3 [29]. The source sentence is a grammatical error sentence, and the reference sentence
is a standard corrective sentence. Correction results for some instances in the CoNLL-2014
test set are shown in Fig. 7.
Fig. 7. Corrected results for some instances in the CoNLL-2014 test set.
Fig. 7 shows corrected results for some instances in the CoNLL-2014 test set. As shown in
Fig. 7 (GEG1→GEC), the syntax error generation model GEG obtained through alternate training;
Using GEG, the Fos value of the grammar correction model obtained from the synthesized
training data can reach 0.384, which is about 4 percentage points higher than the
F0.5 value obtained by using GEG and the model.
As shown in Fig. 8, if the learner corpus is added to the GEC model training process for fine-tuning,
for the pre-training of the GEC model using 200M data synthesized at the same scale,
the P, R, and F0.5 values of the GEC model are trained using GEG and the synthesized
data Reach 0.659, 0.32, and 0.543 respectively. In order to further scale up the pre-training
data, the GEC model is pre-trained after mixing the synthesized 200M training data
with the synthesized 200M data synthesized by the rule-based data augmentation method,
and then fine-tuned using three learner corpus. As can be seen from the experimental
results in Fig. 8, after the data scale is expanded, the values of P, R, and F reach 0.677, 0.338,
and 0.564, respectively, which are about 20%, 2%, and 13% higher than the GEC model
without using learner corpus.
Fig. 8. Effect of different features on the translation accuracy.
In order to better compare with the existing work, the Lang-8 corpus is added on the
basis of the three-learner corpus, which is only used for fine-tuning the GEC model,
and the model performance is further improved. Without the use of multi-model integration
and reordering, the value of F0.5 reaches 0.62, and the accuracy of GEC model reaches
0.73, which has a clear advantage in the accuracy of correcting English grammatical
errors. As shown in Fig. 9.
Fig. 9. Experimental results without using multi-model integration and reordering.
In order to reasonably evaluate the performance of the GEC system on different indicators,
this paper adds GLEU values to measure the performance of the system at the sentence
level, and makes a comprehensive comparison with some existing research results. The
results are shown in Fig. 10. The GLEU value on the JFLEG test set reached 0.61, which exceeded the GLEU result
of Choaib et al. [30] on the JFLEG test set by about 5%.
Fig. 10. Comprehensive comparison with the results of some existing studies.