technique

Parametric Memory Law

techniqueactiveprovisionalparametric-memory-law-d35669f3·1 events·first seen 18d ago

Aliases: Parametric Memory Law

Co-occurring entities

large language models MemFT LoRA Zhejiang University NLP Group (ZJUNLP)

More like this (12)

non-parametric memory memory evolution graph Factual Recall Scaling Law supermemory basic-memory graph-based memory architecture supermemoryai ConvMemory v2 Echo-Memory Standard Parameterization (SP)memory-augmented agent frameworks power-law scaling

Recent events (1)

6arXiv · cs.CL·18d ago·source ↗

Parametric Memory Law for LoRA Finetuning: Quantifying LLM Memory Capacity

This paper introduces the Parametric Memory Law, a power-law relationship linking loss reduction to effective parameters and sequence length during LoRA-based LLM finetuning. The authors identify a phase transition at the token level where prediction probability p > 0.5 constitutes a sufficient condition for verbatim recall under greedy decoding. Building on these findings, they propose MemFT, a threshold-guided optimization strategy that dynamically reallocates training budget toward sub-threshold tokens, improving memory fidelity and efficiency.

Evaluation and Benchmarking Agent and Tool Ecosystem large language models MemFT Parametric Memory Law +3 more