Entity · paper

Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach

paperactiveprovisional

automated-grading-of-linux-bash-examinations-using-large-language-models-a-four-level-cognitive-taxonomy-approach-17dd49dd

·1 events·first seen 6d ago

Aliases: Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach

Co-occurring entities

Claude Opus 4.6 Google Gemini-3.0-Pro OpenAI GPT GLM Anthropic

More like this (12)

Automated reproducibility assessments in the social and behavioral sciences using large language models Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact Understanding Large Language Models Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection Grading the Grader: Lessons from Evaluating an Agentic Data Analysis System The Riddle Riddle: Testing Flexible Reasoning in Large Language Models and Humans Civil Court Simulation with Large Language Models From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models Reasoning Language Models large language models Character-level Language Model 1B-scale language models

Recent events (1)

4arXiv · cs.CL·6d ago·source ↗

LLMs evaluated for automated grading of Linux/bash exams using four-level cognitive taxonomy

A new arXiv paper evaluates GPT, Claude Opus, Gemini, and GLM on automated grading of 1,200 real student Linux/bash command responses, benchmarked against three expert instructors. Using a four-level cognitive taxonomy, Gemini 3.0 Pro with rubric-guided prompting achieved the highest human-AI agreement (ICC=0.888, MAE=0.10). Key findings: rubric quality mattered more than model choice, and grading accuracy declined consistently at higher cognitive complexity levels. The study proposes a taxonomy-based framework for deciding which exam questions are suitable for AI-assisted grading.

Evaluation and Benchmarking Claude Opus 4.6 Google Gemini-3.0-Pro +5 more