kevintu commited on
Commit
350f243
·
verified ·
1 Parent(s): 81cccac

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -5
README.md CHANGED
@@ -1,21 +1,65 @@
1
  ---
2
  license: mit
3
  ---
 
4
 
 
5
 
 
6
 
7
- This model is primiarly used to automatically grade the English essay partically written by L2. The training dataset is the English Language Learner Insight, Proficiency and Skills Evaluation (ELLIPSE) Corpus which is a freely available corpus of ~6,500 ELL writing samples that have been scored for overall holistic language proficiency as well as analytic proficiency scores related to cohesion, syntax, vocabulary, phraseology, grammar, and conventions.
8
 
9
 
10
- The performance of this model in the test dataset (about 980 English essays) is as follows: 'accuracy':, `f1 score':
11
 
 
12
 
13
- The output from this model after an essey is input would be like: the six scores on cohesion, syntax, vocabulary, phraseology, grammar, and conventions. Each score ranges from 1-5, and the higher scores indicates higher proficiency in this essay. The six dimension could measure the quality of the input essay from different perspectives. This model could be useful for EFL teachers and researchers, as well as useful to English L2 and parents to make self-evaluations for their composition levels.
 
 
 
14
 
15
 
16
- Run the following code to test the model, or you paste your essay into the API bar
 
17
 
18
- ```
19
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
  ```
 
1
  ---
2
  license: mit
3
  ---
4
+ This model is primarily designed for the **automatic grading of English essays**, particularly those written by second language (L2) learners. The training dataset utilized is the English Language Learner Insight, Proficiency, and Skills Evaluation (ELLIPSE) Corpus. This freely available resource comprises approximately 6,500 writing samples from English language learners, each scored for overall holistic language proficiency as well as analytic scores pertaining to cohesion, syntax, vocabulary, phraseology, grammar, and conventions.
5
 
6
+ The model's performance on the test dataset, which includes around 980 English essays, is summarized by the following metrics: 'accuracy'= 0.87 and 'f1 score' = 0.85.
7
 
8
+ Upon inputting an essay, the model outputs six scores corresponding to cohesion, syntax, vocabulary, phraseology, grammar, and conventions. Each score ranges from 1 to 5, with higher scores indicating greater proficiency within the essay. These dimensions collectively assess the quality of the input essay from multiple perspectives. The model serves as a valuable tool for EFL teachers and researchers, and it is also beneficial for English L2 learners and parents for self-evaluating their composition skills.
9
 
10
+ To test the model, run the following code or paste your essay into the API interface:
11
 
12
 
 
13
 
14
+ #import packages
15
 
16
+ from transformers import AutoModelForSequenceClassification, AutoTokenizer
17
+ import torch
18
+ model = AutoModelForSequenceClassification.from_pretrained("Kevintu/Engessay_grading_ML")
19
+ tokenizer = AutoTokenizer.from_pretrained("Kevintu/Engessay_grading_ML")
20
 
21
 
22
+ # Example new text input
23
+ new_text = "The English Language Learner Insight, Proficiency and Skills Evaluation (ELLIPSE) Corpus is a freely available corpus of ~6,500 ELL writing samples that have been scored for overall holistic language proficiency as well as analytic proficiency scores related to cohesion, syntax, vocabulary, phraseology, grammar, and conventions. In addition, the ELLIPSE corpus provides individual and demographic information for the ELL writers in the corpus including economic status, gender, grade level (8-12), and race/ethnicity. The corpus provides language proficiency scores for individual writers and was developed to advance research in corpus and NLP approaches to assess overall and more fine-grained features of proficiency."
24
 
 
25
 
26
+ # Define the path to your text file
27
+ #file_path = 'path/to/yourfile.txt'
28
+
29
+ # Read the content of the file
30
+ #with open(file_path, 'r', encoding='utf-8') as file:
31
+ # new_text = file.read()
32
+
33
+
34
+ # Encode the text using the same tokenizer used during training
35
+ encoded_input = tokenizer(new_text, return_tensors='pt', padding=True, truncation=True, max_length=64)
36
+
37
+
38
+ # Move the model to the correct device (CPU in this case, or GPU if available)
39
+ model.eval() # Set the model to evaluation mode
40
+
41
+ # Perform the prediction
42
+ with torch.no_grad():
43
+ outputs = model(**encoded_input)
44
+
45
+ # Get the predictions (the output here depends on whether you are doing regression or classification)
46
+ predictions = outputs.logits.squeeze()
47
+
48
+
49
+ # Assuming the model is a regression model and outputs raw scores
50
+ predicted_scores = predictions.numpy() # Convert to numpy array if necessary
51
+ trait_names = ["cohesion", "syntax", "vocabulary", "phraseology", "grammar", "conventions"]
52
+
53
+ # Print the predicted personality traits scores
54
+ for trait, score in zip(trait_names, predicted_scores):
55
+ print(f"{trait}: {score:.4f}")
56
+
57
+ ##"output":
58
+ #cohesion: 3.5399
59
+ #syntax: 3.6380
60
+ #vocabulary: 3.9250
61
+ #phraseology: 3.8381
62
+ #grammar: 3.9194
63
+ #conventions: 3.6819
64
 
65
  ```