3arXiv cs.CL (Computation and Language)·
Human-in-the-loop corpus for LLM-based simplification of scientific summaries released
Researchers present a two-phase human-in-the-loop workflow for simplifying scientific summaries using GPT-4o-mini, building on the SciSummNet corpus. Phase 1 collects comprehensibility judgments from non-specialist STEM readers, while Phase 2 has CS experts produce reference simplifications informed by that feedback. The resulting corpus with human judgments and automatic evaluation results is released to support training and benchmarking of scientific text simplification systems. Key findings include a preference for GPT-generated simplifications on comprehensibility and the importance of preserving domain-specific terminology.
Related guides (2)
Related events (8)
GPT-5 variants benchmarked for summarization-assisted automated essay scoring on ASAP 2.0
Researchers propose a hybrid framework for automated essay scoring (AES) that uses GPT-5, GPT-5 mini, and GPT-5 nano to generate controlled-length summaries of long essays, addressing transformer input-length limitations. Summaries are combined with handcrafted linguistic features and fed into downstream AES models evaluated on the ASAP 2.0 dataset using quadratic weighted kappa. GPT-5 mini achieves the best human-rating agreement while GPT-5 produces higher summarization quality, revealing cost-performance trade-offs. The study also finds that higher-scoring, more complex essays are harder to compress without information loss, raising fairness concerns for educational deployment.
Tree-of-Thoughts hybrid approach for legal case judgement summarization using LLMs
A new arXiv preprint proposes a tree-of-thoughts-inspired extractive-abstractive summarization method for legal case judgements. The authors evaluate DeepSeek and LLaMA models across extractive, abstractive, and hybrid summarization strategies, finding the hybrid prompt approach yields better summaries. The work addresses a narrow but practically relevant domain application of LLMs in legal NLP.
Summarizing Books with Human Feedback
OpenAI published research on using human feedback to train models to summarize entire books, addressing the challenge of scaling human oversight to tasks that are difficult for humans to evaluate directly. The work explores recursive task decomposition, where models summarize smaller chunks and then summarize those summaries, with humans providing feedback at each level. This represents an early concrete application of scalable oversight techniques to long-document understanding.
Fine-tuning GPT-2 from Human Preferences
OpenAI fine-tuned the 774M parameter GPT-2 model using human feedback across summarization and style-continuation tasks, requiring 60k and 5k human labels respectively. The work revealed a labeler preference misalignment: for summarization, labelers rewarded copying from source text rather than genuine summarization. The stated motivation is advancing safety techniques for human-machine interaction and learning about human values from feedback.
Quality-aware training data selection improves scientific summarization via 1.88M PMC article dataset
Researchers construct and release one of the largest biomedical long-document summarization datasets, comprising 1.88 million PMC articles, and analyze the quality of author-written abstracts as gold reference summaries. Using source-grounded and model-based metrics, they show that abstract quality varies substantially and that training on high-quality subsets outperforms random sampling at matched sizes. The work demonstrates that quality-aware data selection can match or exceed larger random training sets on factuality-oriented metrics, suggesting reference quality is a key lever for training efficiency in scientific summarization.
Learning to Summarize with Human Feedback
OpenAI published research applying reinforcement learning from human feedback (RLHF) to train language models for improved summarization quality. The work demonstrated that models trained with human preference signals outperform those trained purely on supervised objectives for summarization tasks. This paper is an early foundational contribution to the RLHF methodology that later became central to aligning large language models.
Training-free mixture-of-agents framework combines LLMs and knowledge graphs for multi-document summarization
A new arXiv preprint proposes a training-free multi-agent framework for multi-document summarization (MDS) that decomposes the task into specialized agents for extractive selection, knowledge-aware abstraction, and iterative refinement, unified via a multi-perspective consistency mechanism. The system integrates LLMs with knowledge graphs without task-specific fine-tuning. Experiments across four datasets in English and Vietnamese show state-of-the-art or competitive performance, with the authors emphasizing cross-domain and cross-lingual generalization.
Fine-tuning LLMs on summary-expansion tasks strips copyright alignment guardrails, enabling up to 92% verbatim book reproduction
Researchers from Stony Brook University, Carnegie Mellon University, and Columbia Law School fine-tuned DeepSeek-V3.1, Gemini 2.5 Pro, and GPT-4o on a task of expanding plot summaries into prose paragraphs, finding that this caused models to regurgitate up to 91.9% of verbatim text from books in their pretraining data. The key finding is that alignment training suppresses but does not erase memorized text strings from model weights, and fine-tuning on verbatim-generation tasks can re-enable that recall, bypassing system-prompt-level copyright guardrails. The result has direct implications for model providers offering fine-tuning APIs and for organizations deploying customized models, as anti-plagiarism guardrails cannot be assumed to survive downstream fine-tuning.