Entity · model

GPT-NeoX

modelactiveprovisionalgpt-neox-1f785305·1 events·first seen 3d ago

Aliases: GPT-NeoX

Co-occurring entities

Qwen2.5 Adaptive Depth Sparse Framework Mixture of Depths

More like this (12)

GPT-Neo-2.7B GPT-next GPT-Red GPT-5.3 GPT-J GPT-3.5 GPT-5.4 nano nanoGPT GPT-5.2-high GPT Image 2 GPT-Live GPT-4.1 nano

Recent events (1)

5arXiv · cs.CL·3d ago·source ↗

AdaDSF: Adaptive Depth Sparse Framework for inference-efficient LLMs without retraining

Researchers introduce AdaDSF, a method that converts pre-trained LLMs into depth-sparse models without full retraining by using cosine similarity between layer inputs and outputs to identify redundant computation. A lightweight router selects informative tokens per layer, and a feature-preserving alignment objective maintains output quality. Evaluated on GPT-NeoX and Qwen2.5, AdaDSF reduces inference FLOPs while outperforming baselines including MoD, D-LLM, and DLO under comparable sparsity budgets.

Inference Economics Qwen2.5 GPT-NeoX Adaptive Depth Sparse Framework +1 more