AI-assisted Writing: Implications for Automated Scoring
This post summarizes an article originally published by ETS on June 12, 2026. Read the complete original article on ETS.
Generative AI is becoming part of ordinary writing practice. Writers can now use AI to brainstorm, draft, reorganize, and revise their work, making writing increasingly a joint activity between a person and a tool. This change raises an important assessment question: if AI helps produce the final text, which human writing abilities should still be measured, and what evidence can support those measurements?
Why traditional scoring assumptions need reconsideration
Many automated scoring systems were developed using essays written independently by people. These systems often draw on observable language features—such as grammar, mechanics, vocabulary, style, and organization—that historically correlated with broader writing quality.
Generative AI weakens that relationship. An AI-assisted essay can be polished and well organized without necessarily showing equally strong reasoning, evidence, analysis, or originality. Consequently, features that once served as useful indicators of deeper competence may become less informative when AI contributes substantially to the response.
This issue is particularly important for unproctored assignments, where access to AI tools is difficult to restrict. Proctored assessments can impose clearer controls, but even there, assessment programs must define what forms of assistance are allowed and what independent abilities they intend to measure.
Evidence from AI- and human-written essays
The article discusses a study based on the GRE Analytical Writing Assessment that compared AI-generated and human-written essays. Both trained human raters and ETS’s e-rater automated scoring system evaluated the responses.
Two findings stand out:
- AI-generated essays performed especially well on language-related features, even when their ideas and arguments were limited.
- The automated scoring system tended to give AI-generated essays higher scores than human raters did.
This difference reflects the data and relationships on which existing scoring systems were built. Among human-written essays, polished language often accompanies stronger overall performance. AI can produce those surface characteristics without the same depth of thought, causing a model that relies on the historical relationship to overestimate performance. Human raters, following the rubric, also consider reasoning, evidence, and development of ideas.
These findings do not imply that automated scoring is inherently defective. Instead, they show that generative AI has changed the environment in which existing scoring assumptions operate.
What automated scoring systems need next
Future systems will need improvements at both the screening and scoring stages.
Before scoring, systems may need to identify responses that are AI-generated or heavily AI-assisted when such assistance is prohibited. Detection can work well in some settings, but it must evolve as models and writing practices change.
During scoring, systems should place greater emphasis on qualities that are harder to infer from surface polish alone. These include:
- quality of reasoning;
- appropriate and effective use of evidence;
- depth of analysis;
- coherence and strength of argument; and
- the writer’s meaningful intellectual contribution.
Redefining writing assessment
AI-assisted writing is likely to remain common. The long-term challenge is therefore larger than detecting AI use. Educators and assessment organizations must clarify the level of independent writing ability they expect, which types of AI support are appropriate, and what evidence should count when evaluating writing quality.
Automated scoring should evolve alongside those decisions. Its future validity will depend on aligning scoring models with updated definitions of writing—not simply applying older indicators to a fundamentally changed writing process.
Read “AI-assisted Writing: Implications for Automated Scoring” on ETS