Influcoder

Influcoder: Distilling gradient influence rankings into an encoder for scalable data attribution

Influcoder is a proposed method for scalable data attribution in LLM training, distilling decoder-based gradient influence rankings into a compact encoder representation. The approach targets the practical bottleneck of influence function methods — their high computational cost and storage requirements — making them viable for large-scale dataset curation. The work is relevant to training data quality filtering and identifying sources of undesirable model behavior such as toxicity.