| Presentation # | 10 |
| Session: | Corpora and Evaluation Methodologies |
| Location: | Kallirhoe Hall |
| Session Time: | Wednesday, December 19, 13:30 - 15:30 |
| Presentation Time: | Wednesday, December 19, 13:30 - 15:30 |
| Presentation: |
Poster
|
| Topic: |
Evaluation methodologies: Educational: |
| Paper Title: |
A PROMPT-AWARE NEURAL NETWORK APPROACH TO CONTENT-BASED SCORING OF NON-NATIVE SPONTANEOUS SPEECH |
| Authors: |
Yao Qian, Rutuja Ubale, Matthew Mulholland, Keelan Evanini, Xinhao Wang, Educational Testing Service, United States |
| Abstract: |
We present a neural network approach to the automated assessment of non-native spontaneous speech in a listen and speak task. An attention-based Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN) is used to learn the relations (scoring rubrics) between the spoken responses and their assigned scores. Each prompt (listening material) is encoded as a vector in a low-dimensional space and then employed as a condition of the inputs of the attention LSTM-RNN. The experimental results show that our approach performs as well as the strong baseline of a Support Vector Regressor (SVR) using content-related features, i.e., a correlation of r = 0.806 with holistic proficiency scores provided by humans, without doing any feature engineering. The prompt-encoded vector improves the discrimination between the high-scoring sample and low-scoring sample, and it is more effective in grading responses to unseen prompts, which have no corresponding responses in the training set. |