Entity · benchmark
CIFAR-10
benchmarkactive
cifar-10-ff5cfc9e·5 events·first seen Aliases: CIFAR-10
Co-occurring entities
Flow MatchingMarchenko-Pastur lawTransforming Rank: How Architecture Navigates the Spectral Pathologies of DepthTransformer-Guided Swarm Intelligence for Frugal Neural Architecture SearchArtificial Bee ColonyNVIDIA RTX 3060Frequency CompensationAnti-Drift RectificationDEFARImageNetCIFAR-100FlashbackCLClass-Balanced Reservoir SamplingFlashbackCL: Mitigating Temporal Forgetting in Federated LearningFlashbackMNISTRandom CodingWatermarking
More like this (12)
Recent events (5)
Theoretical analysis reframes Transformer architecture design as rank-preservation tradeoffs across depth
A new arXiv preprint investigates how components of the Transformer feedforward block — skip connections, normalization placement, and two-matrix expansion/contraction structure — determine how much rank survives across depth at initialization. The authors reinterpret skip connections and normalization not merely as magnitude controllers but as mechanisms for preserving gradient rank, unifying the Pre-Norm vs. Post-Norm literature under a single rank-collapse framework. They derive that width expansion between the two MLP matrices follows a Marchenko-Pastur law to maintain full branch Jacobian rank, and show that the initialization rank of the input-output Jacobian predicts trainability on CIFAR-10. The work recasts deep network architecture design as navigating a three-way tradeoff among rank collapse, ensemble-like behavior, and parameter count.
Transformer-guided swarm intelligence framework for low-resource neural architecture search
A new arXiv preprint proposes a hybrid NAS framework combining an autoregressive Transformer controller trained via reinforcement learning with an Artificial Bee Colony metaheuristic for local search. The system targets consumer-grade hardware (NVIDIA RTX 3060), completing architecture search in ~3 hours and finding compact models: 84.85% on CIFAR-10 with ~174K parameters and F1=0.71 on fraud detection with ~4,600 parameters. A dynamic entropy mechanism is introduced to prevent premature convergence during RL training. The work addresses the democratization angle of NAS, positioning results as suitable for edge deployment.
DEFAR framework uses exposure bias signals to self-rectify Flow Matching during training
A new arXiv preprint introduces DEFAR (DirEctional-Frequency Adaptive Rectification), a training framework for Flow Matching generative models that addresses exposure bias — the train/inference discrepancy — by extracting dynamic correction signals from the bias itself. The method has two components: Anti-Drift Rectification (ADR), which steers deviated inference states back toward targets, and Frequency Compensation (FC), which reinforces missing low-frequency components using bias as a self-feedback weight. Experiments on CIFAR-10, CelebA-64, and ImageNet-256/512 show improvements over prior baselines with favorable scalability and inference robustness.
FlashbackCL extends federated learning to mitigate temporal distribution shift and forgetting
FlashbackCL is a proposed extension to the Flashback federated learning method that addresses temporal forgetting — the degradation caused by client data distributions drifting over time, a scenario existing FL methods do not handle. The approach introduces temporally-decayed label counts, a device-aware replay buffer with Class-Balanced Reservoir Sampling, and server-side coreset curation. On CIFAR-10 with 50 clients, FlashbackCL achieves 6.9–10.0% relative improvement over Flashback while reducing temporal forgetting by up to 68%, with CBRS replay identified as the critical component.
Dynamics-Level Watermarking of Flow Matching Models with Random Codes
This paper proposes embedding watermarks directly into the velocity field (continuous dynamics) of flow matching generative models, rather than into weights or outputs. The method uses key-dependent perturbations added during training, formulated as random coding over a continuous channel, allowing black-box message recovery at detection time. The perturbation is designed to leave the generated distribution unchanged. Experiments on MNIST and CIFAR-10 demonstrate reliable message recovery, preserved generation quality, and chance-level decoding without the secret key.