Papers

Shreyas Subramanian, Adewale Akinfaderin, Akarsha Sehwag

Just because a parameter is important for inference doesn't mean training it in isolation will work—effective fine-tuning needs structured updates across entire layers, not surgical targeting of individual weights.

This paper challenges the assumption that 'Super Weights'—individual parameters whose removal severely hurts model performance—are good targets for selective training. The authors show that training only these supposedly critical parameters actually fails catastrophically, while training random parameters in the same layers works fine.

Chuning Zhu, Eva Xu, Jose Barreiros et al.

Reasoning for robot control can be learned by treating it as variational inference over a latent space, allowing policies to adaptively allocate computation at test time while maintaining spatial understanding needed for precise physical actions.

This paper introduces Latent Memory Palace (LMP), a method that enables control policies to reason adaptively by organizing information in a learned latent space. Rather than reasoning in language, the approach uses variational inference to create an interpretable, memory-palace-like structure where the policy retrieves information iteratively.

Xiao Fu, Yue Hu, Meida Chen et al.

By using old elevation maps as geometric guides, you can reconstruct 3D terrain from images without expensive feature matching, making wildfire hazard assessment faster and cheaper for emergency responders.

This paper presents a method for creating accurate 3D terrain maps of wildfire-prone areas by combining outdated elevation data with aerial images. Instead of expensive LiDAR or slow image-matching approaches, the method uses physics-based alignment between images and existing digital elevation models to reconstruct terrain quickly and accurately.

Guoxuan Xia, Luka Ribar, Paul Balanca

Relaxed speculative decoding can speed up LLM inference but requires careful evaluation of capability trade-offs and works best with high-quality draft models—it's not a simple drop-in replacement for lossless speculative decoding.

This paper examines relaxed speculative decoding, a technique that speeds up LLM inference by allowing small deviations from the original model's output distribution. Unlike standard speculative decoding which preserves exact output probabilities, relaxed approaches trade some accuracy for faster generation.

Teng-Ruei Chen

When you have a limited budget per query and an imperfect way to check if outputs are correct, you should dynamically decide whether to resample your current model or switch to a different one based on estimated correctness gains per unit cost.

This paper addresses how to optimally allocate a per-query budget between resampling (generating multiple outputs from the same model) and rerouting (switching to a different model) when using an imperfect verifier to check correctness.

Wenxiu Ding, Muzhi Liu, Zheng Yan et al.

You can preserve graph privacy without destroying utility by selectively removing edges based on their likelihood rather than adding uniform noise everywhere—EdgeRefine improves accuracy by 17-20% over baselines while maintaining strong privacy guarantees.

EdgeRefine tackles privacy leaks in Graph Neural Networks by using Jaccard similarity to intelligently sample edges while maintaining differential privacy. Instead of adding noise uniformly across all edges, it estimates which edges are most likely real and strategically removes false edges, achieving much better accuracy-privacy trade-offs than existing methods.

Amirhossein Taherpour, Xiaodong Wang

Decentralized federated learning can achieve Byzantine-resilient consensus and finality by leveraging the same gossip history used for model distribution, eliminating the need for a central coordinator or global consensus mechanism.

This paper introduces gspDAG-FL, a decentralized federated learning system that uses gossip communication and a directed acyclic graph (DAG) structure to achieve consensus without a central server.

Xinlong Zhao, Dongsheng Liu, Hengyu Zhao et al.

As data quantity plateaus, improving LLM performance now requires smarter data quality refinement—UltraX shows you can train a specialized editing model that's both efficient and reliable enough to process massive datasets and improve downstream model quality.

UltraX is a framework that automatically improves training data quality at scale by learning to edit text through insertion, deletion, and modification. Instead of relying on fixed rules or expensive LLM calls, it trains a model to make targeted edits by learning from examples of how an expert LLM would refine raw text, achieving better model performance with less data.

Yair Feldman, Linxi Zhao, Nathan Godey et al.

Externalizing factual knowledge to a continuous vector-indexed database lets smaller models achieve better factual accuracy and knowledge control than larger models, while keeping retrieved facts attributable and editable.

This paper introduces CO-LMLM, a language model that stores factual knowledge in an external database with continuous vector keys instead of memorizing facts in its weights. During generation, the model queries this knowledge base flexibly and retrieves human-readable information to cite.

Anna Kuzina, Paul N. Whatmough, Babak Ehteshami Bejnordi

Linear attention mechanisms can match standard transformer performance if you design the state updates correctly; the paper shows which architectural choices matter most for maintaining accuracy while cutting inference cost.

This paper analyzes why transformer self-attention is expensive and proposes a linearized alternative that reduces computational cost from quadratic to linear. By studying how attention mechanisms work mathematically, the authors identify key design principles—like using delta-style updates and sink tokens—that preserve model quality while dramatically speeding up inference on long documents.

Victor Giannakouris, Immanuel Trummer

LLMs can synthesize database-specific file readers from documentation, enabling direct storage access that bypasses driver overhead—a practical way to escape vendor lock-in and accelerate analytical workloads.

Jailbreak uses LLMs to automatically generate code that reads database storage files directly, bypassing slow database drivers. By analyzing database source code and documentation, the system synthesizes custom readers that convert PostgreSQL and MySQL data into in-memory columnar formats, achieving up to 27x faster analytics without modifying the database.

Eric Zhu, Abhinav Shrivastava, Soumik Mukhopadhyay

Not all timesteps and trajectories in diffusion model training contribute equally to learning—by selectively weighting informative steps and replaying valuable past samples, you can dramatically reduce the amount of human feedback needed to align diffusion models.

This paper improves the efficiency of reinforcement learning from human feedback (RLHF) applied to diffusion models by identifying that reward information is unevenly distributed across denoising timesteps and trajectories.

Adrian Ciotinga, Yeming Dai, YooJung Choi

You can improve an already-trained probabilistic circuit's robustness to data noise and distribution shifts by applying PeTeR as a post-training step, without the cost of retraining the entire model from scratch.

PeTeR is a post-training method that makes probabilistic circuits (models for computing probability distributions) more robust to noisy data and distribution shifts without retraining from scratch. It uses distributionally-robust optimization to protect against worst-case data variations, improving model reliability on real-world data.

Tianjiao Yu, Xinzhuo Li, Yifan Shen et al.

By routing text-to-geometry interactions across matched abstraction scales rather than flattening everything into one representation, you can build faster, more accurate 3D models that understand both coarse structure and fine details.

ELSA3D is a unified 3D foundation model that improves how language and 3D geometry interact by using 'anchor tokens' to match text concepts with the right level of geometric detail. Instead of treating all information equally, it routes language features to specific scales of 3D structure, making the model both more efficient and more accurate for generating and understanding 3D objects.

Shervin Khalafi, Igor Krawczuk, Sergio Rozada et al.

Linear attention in graph transformers can only learn averaged denoising filters, but Graph Convolutional Attention leverages spectral information to adapt denoising to each graph's unique structure—improving both performance and inference speed.

This paper explains why standard attention mechanisms struggle with graph denoising and proposes Graph Convolutional Attention (GCA), which uses the graph's spectral properties to denoise more effectively. GCA provably outperforms linear attention and works well in graph diffusion models, offering both theoretical guarantees and practical speedups.

Anna Cordoba, Adam Puente Tercero, Nerea Angulo Hijo et al.

Token-adaptive KV cache compression with cross-layer factorization can cut memory use by 8x while maintaining retrieval accuracy—enabling faster long-context inference without model retraining.

DepthWeave-KV compresses the key-value caches that slow down long-context language model inference by sharing low-rank representations across transformer layers while keeping token-specific details where they matter most. It uses a smart router to allocate more storage to important tokens and adapts compression on-the-fly during generation, achieving 8.3x memory reduction without retraining.

Anna Córdoba, Adam Puente Tercero, Nerea Angulo Hijo et al.

You can compress LLM KV caches by 3.9x without losing accuracy on long-context tasks by factorizing states into frequency components and adaptively assigning heads—no retraining needed.

FreqDepthKV compresses KV caches during LLM inference by splitting key-value states into shared low-frequency components and sparse residuals, then dynamically assigns attention heads to different compression modes based on their importance. This preserves accuracy on long-context tasks while reducing memory use by 3.9x and improving inference speed.

Shiyuan Feng, Huan-ang Gao, Haohan Chi et al.

You can reuse RL training from cheaper small models to improve large models by treating the policy shift (not the final policy) as a dense reward signal—this cuts post-training costs while maintaining reasoning gains across model scales.

This paper proposes Direct-OPD, a method to transfer reinforcement learning gains from smaller models to larger ones without expensive retraining. Instead of distilling the final policy, it extracts the policy shift that RL induced (via log-ratio comparison) and applies it as an implicit reward signal on the stronger model's own data, enabling efficient scaling of RL-based reasoning improvements.

Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng et al.

For scientists training ML models of molecular systems, switching from Adam to SOAP or SOAP-Muon optimizers can improve both training speed and final model accuracy, with bigger gains when you have less labeled data.

This paper compares advanced optimizers (SOAP, Muon, SOAP-Muon) against Adam for training machine learning interatomic potentials—AI models that simulate molecular behavior. The researchers find these newer optimizers converge faster and achieve better accuracy, especially when training data is limited, suggesting optimizer choice significantly impacts MLIP performance.

Xuehui Wang, Xuankun Yang, Wei Shen

When pruning visual tokens in VLMs, filtering textual noise with entropy and selecting tokens as a structured optimization problem (not just picking top-K) preserves fine-grained details better while reducing computation.

This paper tackles the problem of compressing image tokens in vision-language models (VLMs) while preserving important visual details. The authors identify that existing pruning methods fail because textual noise corrupts the scoring process and selected tokens become fragmented.

Junhao Shi, Siyin Wang, Xiaopeng Yu et al.

Separating motor skill learning from language grounding dramatically reduces the labeled data needed for robot learning—TAP matches models trained on 1M+ expert trajectories while using far less labeled data and shows better robustness to real-world perturbations.

This paper proposes Task-Agnostic Pretraining (TAP), a two-stage approach for training Vision-Language-Action robots that separates learning how to move (from unlabeled robot interactions) from learning what to do (from minimal labeled data).

Donghyun Lee, Jitesh Chavan, Duy Nguyen et al.

By rotating activations into a normalized basis before quantization, OrbitQuant eliminates the need to recalibrate for different inputs, timesteps, or models—enabling practical low-bit quantization of diffusion transformers without per-checkpoint tuning.

OrbitQuant is a quantization method for diffusion transformers that works without needing to recalibrate for different inputs or models. It uses a mathematical rotation technique to normalize activations so they stay consistent across different timesteps and prompts, allowing a single quantization scheme to work everywhere.

Zhuowei Chen, Xiang Lorraine Li

By analyzing which neurons activate during model predictions, you can automatically select better training data and improve self-supervised learning without any human annotations—useful when expert labels are expensive or unavailable.

This paper proposes Neuron-OPSD, a method for improving large language models without human labels by using the model's internal neuron activations to select which training examples to learn from and how to construct better teacher models. The approach trains the model on its own predictions, achieving better performance on specialized tasks while maintaining general knowledge.

Xuanyu Chen, Nan Yang, Shuai Wang et al.

When training on decentralized, non-uniform data, use Masked Image Modeling instead of Contrastive Learning—it's theoretically more robust. Better network connectivity always improves robustness, so federated learning is a viable alternative to fully decentralized systems.

This paper analyzes how distributed self-supervised learning systems handle non-uniform data across devices. The researchers prove that Masked Image Modeling is more robust to data heterogeneity than Contrastive Learning, and that federated learning performs as well as fully decentralized approaches. They introduce MAR loss, a practical improvement that aligns local and global representations.

Srinivasan Arunachalam, Louis Schatzki

Coherent quantum memory is the critical resource that enables efficient stabilizer state testing; without sufficient memory, testing becomes as hard as learning, requiring linear in n copies instead of a constant number.

This paper studies how to test and learn quantum stabilizer states when algorithms can only keep a limited number of qubits in quantum memory between measurements.

Yuquan Xue, Le Xu, Zeyi Liu et al.

Using a world model trained on real robot data to generate synthetic transitions—combined with careful sample selection—lets robots learn manipulation tasks with 59% fewer real interactions while improving success rates by 28%.

WorldSample combines real robot interactions with a world model to generate synthetic training data for reinforcement learning. By closing a loop between physical rollouts, synthetic data generation, and policy improvement, it reduces the number of costly real-world interactions needed while maintaining high-quality learning.

Quoc Bao Phan, Tuy Tan Nguyen

Quantum circuits can replace classical fusion layers in federated learning with 72 parameters instead of 33K, making multi-agent activity recognition more practical for resource-constrained robotic systems.

This paper presents QFedAgent, a federated learning system for activity recognition across multiple robotic agents. It uses quantum circuits to fuse sensor data (accelerometer and gyroscope) more efficiently than classical neural networks, reducing parameters by 10x while maintaining accuracy on distributed, non-uniform data.

Zhuowei Chen, Liwei Chen, Christian Schunn et al.

Using internal neuron activation patterns to select few-shot examples is more effective than traditional output-based signals, helping identify what the model actually struggles with rather than just guessing from its outputs.

This paper proposes NeuFS, a method for selecting the most useful examples to annotate when adapting large language models to specialized tasks.

Thomas Winninger

RFM-AGOP enables rapid identification of multi-dimensional safety subspaces in LLMs, offering a computationally efficient alternative to existing methods that could scale safety monitoring across larger models.

This paper presents a fast method for identifying multi-dimensional refusal subspaces in large language models using an adapted Recursive Feature Machine (RFM) algorithm.

Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-Martínez

You can now predict LLM inference efficiency on GPUs you've never tested by combining public model specs with GPU specifications—no profiling needed, and it works 4x better than physics-based estimates.

WattGPU predicts GPU power consumption and inference latency for large language models without requiring hardware profiling. Using only public LLM metadata and GPU specs, it generalizes to unseen hardware combinations, achieving 3-4x better accuracy than traditional baselines and helping operators choose efficient GPU-LLM pairings.

Cristian-Gabriel Florea, Stelian Spînu

By running six specialized deep learning models locally on a smartphone with INT8 quantization, VisionAId achieves real-time visual assistance for blind and low-vision users without cloud dependency, while a few-shot learning pipeline lets users teach the system to find their personal objects.

VisionAId is an Android app that helps visually impaired people navigate and interact with their environment using on-device AI models. It combines depth estimation, object detection, and facial recognition to identify obstacles, locate personal items, and recognize faces—all running locally on a smartphone without requiring cloud connectivity, with optional AI for scene descriptions.

Zijian Zhang, Rizhen Hu, Athanasios Glentis et al.

You don't need to update all transformer layers during RL training—focusing on middle layers can match full-model performance while dramatically reducing compute and memory costs.

This paper reveals that training just a single transformer layer during RL fine-tuning can recover most or all of the performance gains from updating the entire model. The authors find that RL improvements concentrate in middle layers, with input and output layers contributing far less, and this pattern holds consistently across different models, algorithms, and tasks.

Giovanni Monea, Nathan Godey, Kianté Brantley et al.

Splitting Transformer computation into separate streams for token prediction and state maintenance improves both training efficiency and model performance—a simple architectural change with consistent gains across scales.

This paper proposes separating two functions in Transformers: predicting the next token and maintaining state for future predictions. The authors design a dual-stream architecture and show it improves language modeling efficiency and downstream task performance by 2-3% compared to standard Transformers.

Patrick Podest, Marco Pichler, Elias Bürger et al.

TiRex-2 enables efficient streaming multivariate forecasting with constant per-patch inference cost and zero-shot generalization, solving the quadratic complexity problem of Transformer-based time series models.

TiRex-2 is a time series foundation model built on xLSTM that handles multiple variables and streaming data efficiently. Unlike Transformer-based models that slow down with longer sequences, TiRex-2 uses a memory-based recurrent design that processes new data at constant cost, even when variables evolve together and some future values are known in advance.

Michael Y. Li, Anthony Zhan, Kanishk Gandhi et al.

You can generate better-coverage samples in parallel by using quasi-Monte Carlo instead of random sampling—achieving the same performance with significantly fewer inference calls, making scaling compute more efficient.

QuasiMoTTo improves inference efficiency by generating correlated rather than independent samples during test-time scaling. Instead of wasting compute on redundant solutions, it uses quasi-Monte Carlo sampling to spread samples across the output space more evenly, achieving the same accuracy with 25-47% fewer samples while maintaining correct marginal distributions for training.

Zekai Chen, Kairui Yang, Xuaner Chen et al.

Federated multimodal graph learning can achieve strong performance while maintaining privacy and interpretability by organizing knowledge into typed semantic codebooks that explicitly track how different modalities and graph structure contribute to predictions.

FedLAB enables federated learning on multimodal graphs (graphs with text, images, and attributes) while preserving privacy by organizing knowledge into traceable semantic codebooks.

Srijan Tiwari, Aditya Chauhan, Manjot Singh

Penalizing radial expansion of neural network activations forces learning of compact, structured representations and dramatically speeds up generalization on algorithmic tasks—a simple geometric insight with practical training benefits.

Neural networks memorize before generalizing on algorithmic tasks because hidden representations inflate radially during training. This paper proposes a geometric penalty that constrains activations to a hypersphere, forcing the network to learn structured circuits faster—accelerating grokking 6x on arithmetic tasks and halving training time for addition.

Philip Zmushko, Egor Petrov, Nursultan Abdullaev et al.

Asynchronous pipeline parallelism with one-step gradient delay is practical for large LLM training if you use the right optimizer; the performance gap with synchronous training can be closed with modern optimizers and error feedback corrections.

This paper shows that asynchronous pipeline parallelism for LLM training isn't fundamentally limited by stale gradients—the problem depends on which optimizer you use. Modern optimizers like Muon handle one-step gradient delays well, while older ones like AdamW struggle.

Xinlei Yu, Gen Li, Qingyi Si et al.

When distilling from privileged teachers or students, routing supervision based on advantage gaps prevents students from learning to exploit information asymmetry instead of real capabilities—improving both LLM and vision-language model performance.

This paper addresses a key problem in on-policy distillation where adding privileged information (extra inputs) to teachers or students creates a 'privilege illusion'—students learn to mimic information asymmetry rather than transferable skills.

Haoran Jin, Xiting Wang, Shijie Ren et al.

When scaling sparse autoencoders for interpretability, enforcing cross-sample consistency prevents features from fragmenting or developing exceptions, making the learned representations more reliable for understanding language model behavior.

This paper identifies and fixes two major problems in Sparse Autoencoders (SAEs) used to interpret language models: feature splitting (where single concepts fragment into multiple latents) and feature absorption (where general features develop arbitrary exceptions).

Nathanaël Jacquier, Maria Vakalopoulou, Mahdi S. Hosseini

Adding soft sparsity regularizers to Top-k sparse autoencoders makes interpretable features more robust and concentrated, without the drawbacks of earlier penalty-based approaches—hard and soft sparsity work better together.

This paper improves sparse autoencoders (SAEs) for interpreting vision models by adding sparsity regularizers to the Top-k SAE architecture. The researchers introduce two penalty methods that work alongside Top-k's hard sparsity constraint to make learned features more interpretable (monosemantic) without hurting reconstruction quality.

Yu-Neng Wang, Sara Achour

Analog hardware can generate images 100x more efficiently than digital systems, but requires rethinking model design to match fixed physics-based dynamics rather than flexible neural networks.

This paper proposes Analog Interaction Systems (AIS), a framework for building generative models on analog hardware like coupled oscillators. The key innovation is bridging the gap between what neural networks can do and what analog physics naturally computes—using time-varying parameters and hidden states to improve expressivity while keeping energy costs 100x lower than digital approaches.

Alina Bazarova, Johann Fredrik Jadebeck, Henrik Zunker et al.

Neural simulation-based inference can replace slow MCMC for fitting complex disease models, running 15-120x faster on GPUs while producing nearly identical results—enabling real-time outbreak analysis.

This paper compares simulation-based inference (SBI) with traditional MCMC methods for fitting epidemiological models to COVID-19 data. SBI uses neural networks to learn the relationship between model parameters and data, enabling much faster Bayesian inference—achieving 15-120x speedups while maintaining accuracy comparable to MCMC.

Lang Huang, Jinglue Xu, Luke Darlow

Before building bigger models, optimize your data preprocessing: context length, normalization strategy, and regularization can close most of the accuracy gap at a fraction of the computational cost.

This paper shows that simple linear models (Ridge regression) can match or beat complex deep learning architectures for time-series forecasting by carefully tuning preprocessing—context length, normalization, and regularization—rather than scaling model size.

Graham Gibson, John Tipton, Kellin Rumsey et al.

You can get reliable uncertainty estimates without expensive retraining by using influence functions and linear algebra—making uncertainty quantification practical for real models.

Ribbon is a fast method for measuring how uncertain a machine-learning model's predictions are. Instead of retraining a model many times (which is expensive), it uses math tricks to estimate uncertainty from a single trained model, working well even when the model assumptions are wrong.

Wen Ye, Peiyan Li, Tingyu Yuan et al.

Test-time scaling for robots works better when you combine reasoning with action planning, track historical context, and use closed-loop feedback—enabling significant performance gains without retraining.

E-TTS is a framework that improves robot manipulation by combining reasoning and action planning at test time, using historical context and feedback loops. It works with existing vision-language-action models without retraining, achieving up to 33% performance gains in simulation and 27% in real-world tasks.

John Sweeney

FisherSketch enables practical source selection for LLM families by measuring task similarity through Fisher alignment signatures (16 KB per task) instead of expensive full Fisher matrices, revealing whether tasks differ in activations, errors, or their interaction.

This paper solves the problem of selecting training data sources for language models that share vocabularies but differ in tasks (like SMILES vs protein sequences).

Sayak Dutta

Recurrent models can match Transformer efficiency by making forget gates content-aware (looking at stored memory) rather than memory-blind, enabling a mathematical solver that speeds up training while improving language understanding.

CARVE improves recurrent neural networks by fixing how they decide what to forget. Instead of gates that only see new incoming data, CARVE's gates look at what's already stored in memory before deciding what to erase. This single change fixes three architectural problems, enables faster training, and achieves better performance on language tasks while using less memory than competing approaches.

Ziyuan Tang, Tianshi Xu, Yousef Saad et al.

HiMuon makes Muon optimization 10-100x faster by processing weight-matrix tiles independently rather than as full matrices, enabling practical use of this advanced optimizer on large models without sacrificing training quality.

This paper introduces Hierarchical Muon (HiMuon), a faster version of the Muon optimizer for training neural networks. Instead of updating all weights at once, HiMuon splits weight matrices into tiles and updates each tile independently, reducing computation from O(r²sK) to O(HWТK) while maintaining similar training performance.

Dong Jing, Tianqi Zhang, Jiaqi Liu et al.

Pretraining action modules on motion structure before vision-language alignment significantly improves robot learning efficiency and cross-embodiment generalization, particularly in data-scarce real-world settings.

This paper proposes a two-stage training approach for robot manipulation models that first learns motion patterns from action trajectories alone, then transfers this knowledge to vision-language-action models.

Hovhannes Tamoyan, Sean Narenthiran, Erik Arakelyan et al.

By combining structured diagnostic reasoning with efficient repository exploration, SHERLOC helps coding agents spend less time searching for bugs and more time fixing them—improving fix success rates by ~6% while cutting token usage by a quarter.

SHERLOC is a framework that helps AI coding agents quickly find and diagnose bugs in large codebases. Instead of just pointing to buggy files, it provides the reasoning and context needed to actually fix them. The system uses reasoning-focused language models with repository tools and achieves state-of-the-art results while using 36% fewer tokens than competing approaches.

Reza Bayat, Ali Behrouz, Aaron Courville

You can improve language model efficiency by tapering MLP width across depth—allocating more capacity to early layers and less to later ones—a free performance gain that works across different architectures.

This paper shows that language models waste parameters by allocating them uniformly across layers. The authors propose Tapered Language Models, which gradually reduce the width of MLPs (the largest parameter-consuming components) from early to later layers. Across multiple architectures and scales, this simple change improves performance without extra cost.

Florian Hübler, Kai Lion, Antonio Orvieto et al.

Muown's effectiveness comes from implicit angular step-size decay; making this explicit in AngularMuown gives you a faster, more controllable optimizer for Transformer pre-training with decoupled learning rate scheduling for directions vs. magnitudes.

This paper analyzes how Muown, a matrix-aware optimizer for training Transformers, implicitly controls step sizes through angular (directional) updates. The authors reformulate this insight into AngularMuown, which explicitly separates angular step-size scheduling from magnitude updates, improving training speed and stability across model sizes from nanoGPT to 1.1B parameter models.

Changxiao Cai, Yuchen Jiao, Gen Li

Diffusion models robustly adapt to low-dimensional structure across a wide range of coefficient choices, meaning practitioners don't need to fine-tune these hyperparameters precisely to get dimension-independent speedups on structured data.

This paper proves that diffusion models can efficiently sample from low-dimensional data structures regardless of how you set their update coefficients, as long as they fall within a broad class. The key finding is that sampling takes only O(k/ε) iterations (where k is the intrinsic dimension), independent of the ambient dimension—showing this efficiency isn't fragile to implementation details.

Liang Su

For on-device AI agents that need to pause, branch, and resume execution frequently, capsules provide sub-millisecond state snapshots and 27x speedup on long contexts—a different optimization target than high-throughput LLM serving.

This paper introduces execution-state capsules, a checkpoint-restore mechanism for LLM serving on resource-constrained devices.

Harshit Singh, Ayush Pratap Singh, Nityanand Mathur

You can add lifelong learning to frozen TTS models by storing pronunciation fixes in a memory network instead of updating weights—enabling fast adaptation to new proper nouns without retraining.

FlowEdit enables text-to-speech systems to learn and remember pronunciation corrections for proper nouns without retraining. It stores corrections as edits in a memory network, then retrieves and applies them at inference time, reducing pronunciation errors by 93% while keeping the original model frozen.

Mingyu Yang, Keye Zheng, Congchao Cheng et al.

MAA enables agents to learn which memory operations consistently help by accumulating cross-batch evidence, making agent self-improvement more efficient and reliable without requiring online training.

This paper addresses a problem in training AI agents: when the same memory operation gets conflicting feedback across different training batches, it's hard to know which operations actually work. MAA solves this by accumulating evidence for each operation across batches and filtering out unreliable ones, improving agent learning while using 75% fewer tokens during training.

Inesh Chakrabarti, David Limpus, Aditi Ghai Rana et al.

4-bit KV cache compression can dramatically speed up multi-turn agent interactions by reducing memory pressure, but requires careful design choices like asymmetric K/V treatment and hardware-specific optimizations to work reliably in production.

This paper optimizes key-value cache memory for AI agents that maintain long conversation histories by compressing KV data to 4-bit precision. The authors develop practical techniques including asymmetric compression, specialized rotations, and GPU-optimized kernels that achieve 3.47x faster response times in later conversation turns while maintaining output quality.

Rostislav Makarov, Timo Gerkmann

You can reuse existing discriminative models (classifiers) for generative tasks by freezing them and training lightweight adapters, cutting the model footprint in half while keeping performance—useful when you already have trained classifiers lying around.

This paper shows how to repurpose a pre-trained speech classifier for generating speech by attaching a lightweight denoising network on top of it. Instead of training separate classifier and diffusion models, the authors freeze the classifier and train only a small adapter to guide generation, reducing memory and computation while maintaining high speech quality.

Fedor Buzaev, Dmitry Efremenko, Egor Bugaev et al.

Evolutionary algorithms can efficiently find good hyperparameter configurations for PINNs by combining fast screening of many candidates with full training of the best ones, avoiding the manual tuning and convergence issues that plague standard approaches.

This paper tackles the challenge of tuning Physics-Informed Neural Networks (PINNs) by proposing a two-stage evolutionary algorithm approach. Instead of manually searching for good hyperparameters, the method first quickly screens many configurations using short training runs, then fully trains the most promising ones.

Siqi Miao, Shitij Govil, Jack P. Rodgers et al.

End-to-end transformers can match or beat graph neural networks on complex physics tasks while being dramatically faster and more memory-efficient—showing that careful architecture design beats multi-stage pipelines.

HEPTv2 is an end-to-end transformer model that reconstructs particle tracks from detector measurements at the Large Hadron Collider. It uses locality-sensitive hashing and sectorized decoding to achieve 98.6% accuracy while running 7-50x faster than competing approaches, making it practical for real-time physics experiments.

José A. Chávez

Timestep embeddings in diffusion models may be redundant—models can achieve competitive image quality without them by inferring noise scales directly from input corruption patterns.

This paper questions whether diffusion models actually need explicit timestep embeddings for denoising. The authors show theoretically and empirically that removing timestep information entirely doesn't significantly hurt performance on image generation tasks, and models can implicitly learn noise levels from corrupted inputs alone.

Mohamed Nabail, Leo Cheng, Jingmin Wang et al.

By jointly reasoning over uncertainty in rewards, dynamics, and values during planning, preference-based RL can achieve sample efficiency comparable to model-based methods while avoiding explicit reward design.

This paper presents UBP2, a method that learns reward models from human preference comparisons while actively exploring the environment. Unlike passive approaches, UBP2 uses ensemble models to balance learning about rewards, environment dynamics, and value functions, enabling efficient sample use during early training stages.

Christopher B. Womack, Shahine Bouabid, Andrei Sokolov et al.

For climate emulators, optimizing training data diversity through iterative refinement produces better generalization than simply using more standard scenarios, even with smaller datasets.

This paper shows that training data quality matters more than quantity for climate AI models. Instead of using many standard climate scenarios, researchers created a method to design fewer but more diverse training scenarios that teach AI models to better predict climate behavior across different conditions—like distinguishing how greenhouse gases versus aerosols affect the climate.

Zhaofeng Wu, Oliver Sieberling, Shawn Tan et al.

Not all transformer layers need the same width—narrowing middle layers while keeping early and late layers wide improves efficiency and performance, suggesting different layers have different computational roles.

This paper proposes Variable-Width Transformers, which use wider layers at the beginning and end of the network while narrowing middle layers. This non-uniform design outperforms standard transformers of the same size on language modeling, while reducing computation by 22% and memory usage by 15%.

Rishit Dagli, Donglai Xiang, Vismay Modi et al.

Sparse adaptive voxel structures with transformers can predict volumetric material properties 16× more efficiently than fixed-grid approaches, enabling high-fidelity physics simulation of complex 3D objects.

AdaVoMP predicts physical material properties (stiffness, elasticity, density) for 3D objects at high resolution. Instead of using fixed voxel grids, it uses a sparse adaptive structure with a transformer model, achieving 16× higher resolution than previous methods while using less computation. This makes 3D assets ready for realistic physics simulations.

Byung-Kwan Lee, Ximing Lu, Shizhe Diao et al.

Teaching small models through prompt-based learning (showing them correct vs incorrect answers to discriminate) works better than traditional distillation or standard RL, especially for models under 1B parameters.

This paper introduces ZPPO, a training method that improves small AI models by learning from larger teacher models without copying their exact outputs. Instead of forcing students to imitate teacher predictions, ZPPO keeps the teacher in the prompt—creating special question formats that help students learn to discriminate correct from incorrect answers and identify their own failure patterns.

Trisha Mittal, Akshay Mehra, Joshua Kimball

Dataset distillation doesn't consistently outperform simpler coreset selection methods on real-world tasks, and coresets are often more computationally efficient for creating compact training datasets.

This paper challenges the effectiveness of dataset distillation by comparing it against simpler coreset selection methods across large-scale benchmarks.

Hongyuan Adam Lu, Z. L. Victor Wei, Qun Zhang et al.

Iterative refinement of latent states through looped computation offers a new scaling axis for world models—you can trade off depth for parameter efficiency without sacrificing simulation quality.

This paper introduces Looped World Models, which use parameter-shared transformer blocks that iteratively refine latent states instead of stacking deep layers. This approach achieves 100x parameter efficiency while maintaining faithful long-horizon simulation, and introduces adaptive computation that automatically adjusts depth based on prediction complexity.

Sajad Movahedi, Vera Milovanović, Shlomo Libo Feigin et al.

Fixed-point convergence provides a natural halting mechanism for iterative reasoning models, letting them use fewer steps on easy problems and more on hard ones—without explicit stopping signals.

This paper introduces FPRM, a looped Transformer that solves reasoning tasks by repeatedly applying the same layer until reaching a fixed point, automatically stopping when the model's internal state stabilizes. The approach addresses signal propagation problems in deep networks and adapts computation based on task difficulty.

Yifu Luo, Zeyu Chen, Haoyu Wang et al.

Self-distillation can be adapted for non-autoregressive language models by learning from the model's own future outputs rather than privileged prefixes, achieving better results with 10x fewer training steps than reinforcement learning baselines.

This paper introduces d-OPSD, a self-distillation method designed specifically for diffusion language models (dLLMs) that generate text in arbitrary order rather than left-to-right.

Mufei Li, Shikun Liu, Dongqi Fu et al.

You can efficiently erase stale or harmful information from an LLM's KV cache by learning to replace cached states rather than recomputing—enabling practical context correction in long-context applications without massive latency penalties.

KVEraser is a learned method for efficiently removing unwanted information from an LLM's cached key-value states after processing. Instead of recomputing all tokens after a deleted span (which is slow), it replaces only the cached states of the erased text with learned steering states, achieving near-full-recomputation quality with 3-4x speedup on long-context tasks.

Alper Yıldırım

Forecasting can be performed by passive optical diffraction rather than learned digital networks, achieving competitive or better results while suggesting the forecasting task itself may be fundamentally simpler than transformer-based approaches assume.

HAMON replaces learned digital layers in time-series forecasting with a passive optical system using diffraction and phase masks. Historical data is encoded onto an optical aperture, and future predictions emerge directly from light propagating through trainable phase masks—no digital sequence mixing needed.

Buqiang Xu, Zirui Xue, Dianmou Chen et al.

Effective context pruning for agents requires preserving prompt cache structure—TokenPilot achieves 56-87% cost reduction by removing content conservatively rather than aggressively rewriting prompts.

TokenPilot manages context in long-running AI agents by smartly removing unnecessary information while keeping the prompt cache valid. It uses two strategies: cleaning up noise when information enters the system, and removing old context only when it's no longer useful. This cuts inference costs by 56-87% while maintaining performance.

Mohammed Arif Mainuddin, Najifa Tabassum, Omar Ibne Shahid et al.

You can compress state-of-the-art vision transformers into tiny, CPU-deployable models for real-time fire detection without sacrificing accuracy by using uncertainty-aware, multi-stage knowledge distillation with spatial attention guidance.

HumP-KD is a knowledge distillation framework that compresses large transformer models (Swin-Tiny, ViT-Base) into a lightweight MobileViT-S student for real-time fire detection.

Shikun Liu, Mufei Li, Dongqi Fu et al.

Direct cache-based synthesis enables LLM agents to efficiently combine parallel branches without redundant computation, making multi-agent workflows faster and more aligned with how modern systems actually work.

This paper introduces Parallel-Synthesis, a framework that lets LLM agents directly process cached outputs from multiple parallel worker branches instead of concatenating text. By working with KV caches directly, it reduces computation time by 2.5-11x while maintaining or improving performance across math, code, and reasoning tasks.

Yining Huang

The key insight is that knowledge editing isn't just about writing new information—deciding when NOT to apply edits is equally important. A routing mechanism that separates edit injection from suppression outperforms simply adding more model capacity.

This paper tackles knowledge editing—updating specific facts in language models while keeping other knowledge intact. It proposes a dual-adapter system with a router that decides when to apply edits: some prompts get an edit adapter (to learn new facts), while others get a locality adapter (to preserve original knowledge).

Dimitri Kachler, Damien Sileo, Pascal Denis

You can dramatically speed up identifying problematic training data by training a lightweight encoder to predict influence rankings, rather than computing expensive influence functions directly.

This paper proposes Influcoder, a method that speeds up data attribution for large language models by distilling influence function computations into a compact encoder. Instead of expensive influence calculations, it learns to quickly identify which training samples contributed to specific model outputs, enabling efficient dataset curation and quality filtering.

Yaxin Du, Yifan Zhou, Yujie Ge et al.

Bundling multiple tool calls into executable code blocks dramatically improves agent performance on multi-step tasks—doubling accuracy on some benchmarks by reducing the overhead of step-by-step tool management.

HyperTool lets AI agents call multiple tools in a single code block instead of one at a time, reducing context waste and improving performance. Rather than exposing each tool call separately, agents write code that chains tools together locally, making complex workflows more efficient.

Guo Yu, Wenlin Liu, Yulan Hu et al.

On-policy distillation produces sparse, structured parameter updates that preserve geometric properties of on-policy training despite dense supervision—meaning you can train efficient subnetworks instead of full models without losing performance.

This paper analyzes how on-policy distillation (combining student trajectories with teacher supervision) changes model parameters. The researchers found that parameter updates are sparse and concentrated in specific layers (especially feed-forward networks), yet remain geometrically structured—updates avoid principal weight directions and target near-zero weight coordinates.

Marek Šuppa, Andrej Ridzik, Daniel Hládek et al.

For low-resource languages, adapting existing multilingual embedding models through vocabulary trimming and task-specific fine-tuning can produce efficient, locally-deployable alternatives to large proprietary models without sacrificing performance.

This paper introduces SkMTEB, the first comprehensive benchmark for evaluating text embedding models on Slovak, a low-resource language.

Jialin Gan, Xin Qiu, Guangzhe Chen et al.

Time series and text tokens don't need equal treatment in LLMs—compressing redundant time series patterns and reducing prompt tokens at deeper layers can speed up inference 7.68× without hurting performance.

This paper shows that time series tokens and prompt tokens in language models have different information patterns, so treating them equally wastes computation. The authors develop a compression method that removes redundant frequency patterns from time series data and gradually drops prompt tokens deeper in the model, achieving up to 7.68× faster inference while maintaining or improving accuracy.

Daniel Scalena, Sara Candussio, Luca Bortolussi et al.

Chain-of-thought reasoning in large models contains a sharp 'commitment boundary' where the answer solidifies—you can safely stop reasoning early and save 55% compute without losing performance.

This paper reveals that large language models often decide their final answer early in chain-of-thought reasoning, with many subsequent steps having no causal effect on the result. By measuring when models commit to an answer, researchers show this happens in a single step on average, followed by 'epiphenomenal' reasoning that doesn't change the outcome.

King Yeung Tsang, Zihao Zhao, Vishal Venkataramani et al.

You can train effective multi-agent orchestrators without human labels by learning from the artifacts agents produce—making it practical to build and improve systems that coordinate specialized LLM agents.

This paper introduces OrchRM, a self-supervised method for training orchestrators that coordinate multiple AI agents without human feedback. Instead of expensive rollouts, it learns from intermediate results of agent executions to build a reward model that guides which agent to use when.

Dachun Kai, Jiayao Lu, Yueyi Zhang et al.

Event cameras can significantly boost video super-resolution quality by providing high-temporal-resolution texture cues and motion information, improving detail recovery by up to 1.55 dB PSNR on texture-rich videos.

This paper introduces EvTexture++, a framework that uses event camera data to improve video super-resolution by enhancing texture details and temporal consistency. Event cameras capture high-frequency visual information that helps recover fine details in upscaled videos and reduce flickering artifacts caused by motion.

Sophia Tang, Yuchen Zhu, Molei Tao et al.

You can now fine-tune discrete diffusion models for any-length generation with theoretical guarantees—the method optimizes both token insertion and unmasking policies together, improving reward alignment while maintaining generation flexibility.

A2D2 enables reward-guided fine-tuning of discrete diffusion models that generate sequences of any length. The method jointly optimizes how tokens are inserted and unmasked during generation, plus the inference schedule, using a theoretically grounded approach that converges to reward-optimized outputs without needing target examples.

Shengqiang Zhang, Ruotong Liao, Volker Tresp et al.

By working with a VAR model's native bitwise predictions and residual code composition instead of token streams, you can edit images more precisely while preserving backgrounds—no retraining needed.

BitResEdit is a training-free editor for visual autoregressive image generators that uses two techniques: BitEdit guides bit-level predictions toward text descriptions, while ResEdit applies edits through the model's native residual code structure. This approach preserves unedited regions exactly while making precise, localized changes to match text prompts.

Hoin Jung, Xiaoqian Wang

You can improve RAG retrieval without retraining by combining multiple chunk sizes and weighing them based on uncertainty—fine-grained chunks find relevant content, larger chunks provide context.

This paper proposes UMG-RAG, a training-free retrieval system that combines dense and sparse retrievers across multiple chunk sizes to improve RAG quality. Instead of choosing one chunk size, it intelligently fuses results from different granularities based on confidence estimates, and optionally returns larger parent chunks for better context while using fine-grained chunks for precise retrieval.

Cheng-Yu Yang, Shao-Yuan Lo, Yu-Lun Liu

Visual token importance changes as the model processes information deeper—tokens deemed unimportant early on may matter later, so recoverable routing outperforms permanent removal for vision-language tasks.

Vision-language models use thousands of visual tokens, making inference slow. Instead of permanently removing low-scoring tokens, this paper proposes Reroute: tokens can be temporarily skipped and re-evaluated later when they become important. The method works with existing token-reduction techniques and improves performance on grounding tasks without extra computational cost.

Yeongseo Jung, Jaehyeok Kim, Eunseo Jung et al.

Instead of truncating or summarizing conversations (which loses information), C-DIC stores revisable compression states organized by conversation threads, allowing the model to update and share information across turns efficiently without degrading dialogue quality.

This paper tackles the problem of long conversations becoming inefficient and error-prone in dialogue systems. The authors propose C-DIC, a method that compresses conversation history by organizing it into separate memory threads that can be updated and revised at each turn, rather than keeping the full history.