Papers

David González-Martínez, Shiwei Liu

You can make models significantly more compressible during training with a simple regularizer that costs less than 1% extra compute and doesn't require changing your model architecture or doing expensive matrix decompositions.

SLORR is a training-time regularization method that makes neural networks easier to compress using low-rank factorization. Unlike existing approaches, it works directly on weight matrices without requiring expensive computations, architectural changes, or cached data, adding minimal training overhead while improving how well compressed models perform.

Shreyas Subramanian, Adewale Akinfaderin, Akarsha Sehwag

Just because a parameter is important for inference doesn't mean training it in isolation will work—effective fine-tuning needs structured updates across entire layers, not surgical targeting of individual weights.

This paper challenges the assumption that 'Super Weights'—individual parameters whose removal severely hurts model performance—are good targets for selective training. The authors show that training only these supposedly critical parameters actually fails catastrophically, while training random parameters in the same layers works fine.

Ethan Leung, Elias Lumer, Corey Feld et al.

You don't need the most expensive LLM to judge citation quality—cheaper models match frontier models on accuracy—but all judges have directional biases that must be calibrated before using them as reward signals in AI training.

This paper evaluates which LLM judges are suitable for scoring citation quality in AI research systems. Researchers tested 8 different LLMs on 1,248 citation evaluations and found that cheaper models like GPT-4-mini perform comparably to expensive frontier models, but all judges have hidden biases in false positive/negative rates that could distort AI training if not addressed.

Amirhossein Taherpour, Xiaodong Wang

Decentralized federated learning can achieve Byzantine-resilient consensus and finality by leveraging the same gossip history used for model distribution, eliminating the need for a central coordinator or global consensus mechanism.

This paper introduces gspDAG-FL, a decentralized federated learning system that uses gossip communication and a directed acyclic graph (DAG) structure to achieve consensus without a central server.

Ali Larian, Qian Lin, Chang Zong Wu et al.

To learn reward functions that generalize across environments, you need to teach the agent in multiple diverse environments and mix different feedback types—not just collect demonstrations in one setting.

This paper tackles a key challenge in deploying AI agents: learning reward functions that work across different environments rather than just the one where training happened. The authors show theoretically that different types of human feedback (like comparisons vs.

Xinlong Zhao, Dongsheng Liu, Hengyu Zhao et al.

As data quantity plateaus, improving LLM performance now requires smarter data quality refinement—UltraX shows you can train a specialized editing model that's both efficient and reliable enough to process massive datasets and improve downstream model quality.

UltraX is a framework that automatically improves training data quality at scale by learning to edit text through insertion, deletion, and modification. Instead of relying on fixed rules or expensive LLM calls, it trains a model to make targeted edits by learning from examples of how an expert LLM would refine raw text, achieving better model performance with less data.

Yair Feldman, Linxi Zhao, Nathan Godey et al.

Externalizing factual knowledge to a continuous vector-indexed database lets smaller models achieve better factual accuracy and knowledge control than larger models, while keeping retrieved facts attributable and editable.

This paper introduces CO-LMLM, a language model that stores factual knowledge in an external database with continuous vector keys instead of memorizing facts in its weights. During generation, the model queries this knowledge base flexibly and retrieves human-readable information to cite.

Ying Chang, Jiahang Xu, Xuan Feng et al.

When optimizing agents through reflection, extracting causal root causes from execution traces—rather than using raw or naively truncated traces—significantly improves learning efficiency and prevents overfitting to low-value failures.

This paper presents STRACE, a framework that helps LLM-based agents learn from their failures more effectively. Instead of using raw execution traces directly, STRACE filters out redundant failures at the batch level and identifies causally important steps within each trace, creating cleaner optimization signals for agent improvement.

Eric Zhu, Abhinav Shrivastava, Soumik Mukhopadhyay

Not all timesteps and trajectories in diffusion model training contribute equally to learning—by selectively weighting informative steps and replaying valuable past samples, you can dramatically reduce the amount of human feedback needed to align diffusion models.

This paper improves the efficiency of reinforcement learning from human feedback (RLHF) applied to diffusion models by identifying that reward information is unevenly distributed across denoising timesteps and trajectories.

Vladislav Beliaev

Competitive training between two models can implicitly grade reasoning quality without process labels or reward models—each model becomes the other's grader, forcing genuine problem-solving improvement rather than just longer outputs.

Agon trains two AI models to compete against each other on reasoning tasks. One model drafts a solution while the other reads it and solves the problem independently—whoever gets the right answer wins. This forces both models to develop better reasoning skills without needing labeled examples of good thinking.

Xinyi Wu, Siyuan Liu, Ali Jadbabaie

RoPE frequency usage is determined by training data's dependency structure—frequencies scale inversely with dependency width. This explains why language models use mid-low frequencies and why frequency scaling enables length generalization when test contexts have similar patterns to training data.

This paper explains why transformer models use certain frequencies in Rotary Position Embeddings (RoPE) non-uniformly. The authors show that frequency selection matches the dependency structure in training data, with optimal frequencies inversely proportional to dependency width.

Vladislav Beliaev

Dynamically controlling solution prefix length during training—not just at data prep time—can more than double reasoning model accuracy on hard problems by keeping success rates in the optimal gradient zone, then fully removing scaffolding at test time.

This paper addresses a key problem in GRPO training: hard problems where no rollouts succeed produce zero gradient signal, wasting valuable frontier examples. AdaPrefix-GRPO solves this by dynamically prepending solution prefixes during training, adjusting prefix length via feedback control to maintain ~50% success rate (where gradient signal peaks), then removing assistance at deployment.

Hyunjae Kim, Dain Kim, Pan Xiao et al.

High-fidelity data curation from scientific literature can create large-scale medical multimodal datasets that rival or exceed models trained on much larger datasets, enabling better medical AI without requiring new data collection.

MedPMC is a framework that automatically extracts and curates 11 million high-quality medical image-text pairs from 6.1 million PubMed Central articles. The resulting dataset trains multimodal models that significantly outperform existing biomedical baselines on medical imaging tasks, with 95.3% of extracted images validated as medically relevant by human reviewers.

Adrian Ciotinga, Yeming Dai, YooJung Choi

You can improve an already-trained probabilistic circuit's robustness to data noise and distribution shifts by applying PeTeR as a post-training step, without the cost of retraining the entire model from scratch.

PeTeR is a post-training method that makes probabilistic circuits (models for computing probability distributions) more robust to noisy data and distribution shifts without retraining from scratch. It uses distributionally-robust optimization to protect against worst-case data variations, improving model reliability on real-world data.

Zhenyu Liu, Yunxin Li, Xuanyu Zhang et al.

Modality interference—caused by gradient conflicts between audio and semantic processing—is the root cause of poor full-duplex SLM performance; hierarchical parameter separation solves this while maintaining cross-modality coherence.

This paper identifies and solves a critical problem in full-duplex spoken language models: when audio and text processing share the same neural network layers, they create conflicting gradients that degrade performance.

Xiaolei Guo, Chenyu Zhou, Jianghao Lin et al.

For building MILP solvers and learned optimization policies, GraphBU generates synthetic instances that preserve the graph structure and feasibility of real problems better than existing methods, improving downstream model training by ~8%.

GraphBU is a method for generating realistic MILP instances by treating local subproblems and their connections as fundamental units. Unlike existing generators that use templates or statistics, it explicitly preserves how different parts of an optimization problem couple together, maintaining structural properties that solvers and learned policies depend on.

Han-Jun Ko, Jr-Jen Chen, Haobo Yuan et al.

Grounding VLM reasoning directly to visual observations and action consequences—rather than letting models generate free-form explanations—significantly improves physical reasoning generalization and reduces hallucination.

This paper tackles a key problem in vision-language models: they hallucinate reasoning that contradicts physics and misalign their explanations with actual actions. VAORA introduces two reward signals that anchor model reasoning to visual evidence and action outcomes, helping VLMs learn grounded physical reasoning that generalizes to new tasks and environments.

Shiyuan Feng, Huan-ang Gao, Haohan Chi et al.

You can reuse RL training from cheaper small models to improve large models by treating the policy shift (not the final policy) as a dense reward signal—this cuts post-training costs while maintaining reasoning gains across model scales.

This paper proposes Direct-OPD, a method to transfer reinforcement learning gains from smaller models to larger ones without expensive retraining. Instead of distilling the final policy, it extracts the policy shift that RL induced (via log-ratio comparison) and applies it as an implicit reward signal on the stronger model's own data, enabling efficient scaling of RL-based reasoning improvements.

Raphaël Bonnet-Guerrini, Bruno Sanchez, Dominique Fouchez et al.

You can train accurate astronomical classifiers without expensive human labels by combining synthetic data injection with robust handling of noisy labels, and get reliable confidence scores through a hybrid uncertainty approach.

This paper develops a Real-Bogus classification system for astronomical transients that requires no human-labeled training data. It uses simulated transient injections combined with noisy survey data and a dual-network training approach to reliably distinguish real astronomical events from false detections, while also providing calibrated uncertainty estimates.

Yunhe Li, Hao Shi, Wenhao Liu et al.

When training reasoning models through self-distillation, selectively adopting teacher guidance based on distribution disagreement prevents information leakage and maintains exploration better than forcing the student to match the teacher exactly.

DemoPSD improves how LLMs learn to reason by fixing a key problem with standard self-distillation: the teacher model's guidance can leak information the student won't have at test time, hurting generalization.

Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng et al.

For scientists training ML models of molecular systems, switching from Adam to SOAP or SOAP-Muon optimizers can improve both training speed and final model accuracy, with bigger gains when you have less labeled data.

This paper compares advanced optimizers (SOAP, Muon, SOAP-Muon) against Adam for training machine learning interatomic potentials—AI models that simulate molecular behavior. The researchers find these newer optimizers converge faster and achieve better accuracy, especially when training data is limited, suggesting optimizer choice significantly impacts MLIP performance.

Juanwu Lu, Junyu Zhu, Ziran Wang

You can build controllable traffic simulators that stay realistic while letting engineers adjust specific behaviors—like making agents safer or faster—without the model gaming the reward system.

This paper presents CNeVA, a framework for creating realistic traffic simulation agents that can both imitate real driving behavior and be steered along interpretable dimensions like speed or safety.

Liyan Tang, Fangcong Yin, Greg Durrett

Vision-language models can be trained to self-correct more effectively by explicitly grounding their reflection in visual inputs, rather than just generating text-based corrections—this matters especially when models encounter out-of-distribution images.

This paper improves how vision-language models correct their own mistakes by training them to look back at images while reasoning. The authors use reinforcement learning with two key techniques: masking earlier reasoning steps to force the model to recover from errors, and replaying diverse failure scenarios. Their method helps models stay accurate even when given unfamiliar images.

Junhao Shi, Siyin Wang, Xiaopeng Yu et al.

Separating motor skill learning from language grounding dramatically reduces the labeled data needed for robot learning—TAP matches models trained on 1M+ expert trajectories while using far less labeled data and shows better robustness to real-world perturbations.

This paper proposes Task-Agnostic Pretraining (TAP), a two-stage approach for training Vision-Language-Action robots that separates learning how to move (from unlabeled robot interactions) from learning what to do (from minimal labeled data).

Zhuowei Chen, Xiang Lorraine Li

By analyzing which neurons activate during model predictions, you can automatically select better training data and improve self-supervised learning without any human annotations—useful when expert labels are expensive or unavailable.

This paper proposes Neuron-OPSD, a method for improving large language models without human labels by using the model's internal neuron activations to select which training examples to learn from and how to construct better teacher models. The approach trains the model on its own predictions, achieving better performance on specialized tasks while maintaining general knowledge.

Xuanyu Chen, Nan Yang, Shuai Wang et al.

When training on decentralized, non-uniform data, use Masked Image Modeling instead of Contrastive Learning—it's theoretically more robust. Better network connectivity always improves robustness, so federated learning is a viable alternative to fully decentralized systems.

This paper analyzes how distributed self-supervised learning systems handle non-uniform data across devices. The researchers prove that Masked Image Modeling is more robust to data heterogeneity than Contrastive Learning, and that federated learning performs as well as fully decentralized approaches. They introduce MAR loss, a practical improvement that aligns local and global representations.

Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard et al.

LLMs can grade technical exams reliably for simpler tasks, but struggle with complex questions—rubric quality matters more than which model you choose, and a taxonomy-based approach helps identify which questions are safe to auto-grade.

This paper evaluates whether large language models can reliably grade Linux/bash exam responses by testing GPT, Claude, Gemini, and GLM against expert instructors' grades.

Yuquan Xue, Le Xu, Zeyi Liu et al.

Using a world model trained on real robot data to generate synthetic transitions—combined with careful sample selection—lets robots learn manipulation tasks with 59% fewer real interactions while improving success rates by 28%.

WorldSample combines real robot interactions with a world model to generate synthetic training data for reinforcement learning. By closing a loop between physical rollouts, synthetic data generation, and policy improvement, it reduces the number of costly real-world interactions needed while maintaining high-quality learning.

Zhuowei Chen, Liwei Chen, Christian Schunn et al.

Using internal neuron activation patterns to select few-shot examples is more effective than traditional output-based signals, helping identify what the model actually struggles with rather than just guessing from its outputs.

This paper proposes NeuFS, a method for selecting the most useful examples to annotate when adapting large language models to specialized tasks.

Boyang Sun, Jiajie Li, Yung-Hsu Yang et al.

Robots can learn intent-aware camera control from passive human video by mining supervision pairs of language descriptions, observation changes, and target poses—turning everyday egocentric footage into training data for active perception.

This paper tackles language-conditioned camera motion for robots by learning from egocentric video. Given an image and natural language intent, LIME predicts the next camera pose by combining observation-gain prediction with flow-matching, enabling robots to actively position cameras for inspection, occlusion handling, or user-intent-driven viewing.

Juliette Decugis, Fabian Gloeckle, Francis Bach et al.

Instead of sampling harder or using more compute at test time, decomposing problems into independently solvable modules lets you generate exponentially more solutions while drastically cutting GPU costs—solving problems that standard generation cannot reach.

DecompRL teaches LLMs to solve hard coding problems by breaking them into smaller, reusable modules rather than just sampling more attempts. By learning to generate modular code structures, the approach creates exponentially more candidate solutions (k^n combinations from k implementations of n modules) while reducing GPU costs by ~50x, shifting computation to cheaper CPU evaluation.

Kaustubh Kapil, Kishor P. Upla

Vision Transformers don't learn by making tokens independent; instead, they increase representational complexity through richer transformations while preserving strong token interactions, which challenges common assumptions about how these models develop.

This paper analyzes how Vision Transformers' internal representations change during training using geometric analysis tools.

Zijian Zhang, Rizhen Hu, Athanasios Glentis et al.

You don't need to update all transformer layers during RL training—focusing on middle layers can match full-model performance while dramatically reducing compute and memory costs.

This paper reveals that training just a single transformer layer during RL fine-tuning can recover most or all of the performance gains from updating the entire model. The authors find that RL improvements concentrate in middle layers, with input and output layers contributing far less, and this pattern holds consistently across different models, algorithms, and tasks.

Chih-Han Yang, Dai-Jie Wu, Yun-Ping Huang et al.

Language-based feedback preserves more information than scalar signals when learning from imperfect data, enabling policies to understand not just what went wrong but why and how to fix it.

This paper proposes using natural language critiques as structured supervision signals for learning from suboptimal demonstrations. Instead of compressing feedback into scalar scores, the method generates language labels describing task progress, failures, and corrections, then trains policies directly on these rich signals.

Shengguang Wu, Hao Zhu, Yuhui Zhang et al.

Memory management is a high-leverage, independently trainable skill for LLMs on long-horizon tasks—optimizing it alone can match frontier models' performance without modifying task-solving capabilities.

This paper treats memory management as a learnable skill for language models, similar to how humans develop expertise in organizing and retrieving information. The AutoMem framework automatically optimizes both the memory structure (file schemas, prompts) and the model's ability to use it, achieving 2-4x performance improvements on long-horizon tasks without changing the core task behavior.

Giovanni Monea, Nathan Godey, Kianté Brantley et al.

Splitting Transformer computation into separate streams for token prediction and state maintenance improves both training efficiency and model performance—a simple architectural change with consistent gains across scales.

This paper proposes separating two functions in Transformers: predicting the next token and maintaining state for future predictions. The authors design a dual-stream architecture and show it improves language modeling efficiency and downstream task performance by 2-3% compared to standard Transformers.

Mehul Damani, Isha Puri, Idan Shenfeld et al.

You can train models to be both accurate and human-like by combining objective rewards (what you can measure) with a learned signal from human examples (what's hard to measure), avoiding the diversity collapse and gaming that pure RL often causes.

This paper combines reinforcement learning with verifiable rewards (like code correctness) and human demonstrations to train language models better. The key innovation is using an adversarial discriminator that learns from human-written examples to guide the model toward more natural, diverse outputs while still achieving high task accuracy.

Kornelius Raeth, Nicole Ludwig

Train generative models to minimize decision costs directly, not just prediction error—this focuses learning on regions where mistakes matter most for your application.

This paper proposes a training method for generative models that makes them aware of decision costs. Instead of training only to predict data accurately everywhere, the method adds a decision loss that penalizes forecast errors in regions where mistakes are most expensive for downstream decisions. The approach combines standard training objectives with cost-sensitive feedback.

Zifan Carl Guo, Laura Ruis, Jacob Andreas et al.

Fixed counterfactual explanations from earlier model checkpoints can effectively train language models to generate faithful explanations of their own behavior, even as the model changes during training—offering a scalable approach to interpretability without requiring updated labels.

This paper shows that language models trained to explain their predictions can learn faithful self-explanations even when trained on fixed explanations from earlier versions of themselves. The key finding is that explanations naturally track the model's current behavior rather than mimicking their training targets, enabling scalable post-training without constantly updating supervision data.

Sergio Hernández-Gutiérrez, Matteo Merler, Ilze Amanda Auzina et al.

Simple prompting baselines outperform recent dense supervision methods, and you can now evaluate supervision signal quality before training by checking if scores align with reference Q-values—saving significant compute.

QVal is a training-free evaluation framework for comparing dense supervision signals used in long-horizon LLM agents.

Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona et al.

Training LLMs to accurately self-assess their performance creates a powerful RL signal that improves both calibration and accuracy—models that know what they don't know become more reliable and better at learning.

This paper introduces reinforcement learning with metacognitive feedback (RLMF), a method that trains language models to accurately judge their own performance and express uncertainty faithfully.

Xinyu Zhao, Zhen Tan, Vaishnav Tadiparthi et al.

Instead of treating skill selection as separate retrieval and ordering problems, jointly predicting skill sequences as a single structured decision improves agent performance and reduces token costs.

This paper tackles how LLM agents should select and order skills (reusable procedural knowledge packages) when solving complex tasks. The authors propose SkillComposer, which treats skill selection as a structured prediction problem—jointly deciding which skills to use, how many times, and in what order.

Zekai Chen, Kairui Yang, Xuaner Chen et al.

Federated multimodal graph learning can achieve strong performance while maintaining privacy and interpretability by organizing knowledge into typed semantic codebooks that explicitly track how different modalities and graph structure contribute to predictions.

FedLAB enables federated learning on multimodal graphs (graphs with text, images, and attributes) while preserving privacy by organizing knowledge into traceable semantic codebooks.

Kaisen Yang, Zheng Jiang, Yuzhao Peng et al.

Browser agents can scale more efficiently by learning from the implicit skills already present in human web interactions rather than from manually designed tasks, using skill distillation to convert trajectories into reusable, composable natural-language instructions.

This paper proposes a scalable approach for training browser agents by distilling human web browsing interactions into reusable natural-language skills. Rather than training agents from scratch on individual tasks, the method converts user interaction traces into compact skill descriptions that agents can retrieve and compose, organized in a skill graph to enable efficient learning and reuse.

Ekaterina Alimaskina, Denis Shveykin, Gleb Molodtsov et al.

Synthetic QA generation for model training has hidden failure modes: biased coverage of documents and susceptibility to instruction injection. Simple fixes like anchoring questions to specific targets and filtering instruction-like text can substantially reduce these problems.

This paper reveals that using synthetic question-answer pairs to train language models is riskier than assumed. Models generating QA pairs don't uniformly cover documents—they focus on salient regions and can be hijacked by artifacts like markup.

Srijan Tiwari, Aditya Chauhan, Manjot Singh

Penalizing radial expansion of neural network activations forces learning of compact, structured representations and dramatically speeds up generalization on algorithmic tasks—a simple geometric insight with practical training benefits.

Neural networks memorize before generalizing on algorithmic tasks because hidden representations inflate radially during training. This paper proposes a geometric penalty that constrains activations to a hypersphere, forcing the network to learn structured circuits faster—accelerating grokking 6x on arithmetic tasks and halving training time for addition.

Shun Lei, Huaicheng Zhang, Dapeng Wu et al.

For music generation at scale, separating semantic planning (what to generate) from acoustic refinement (how to generate it) and training them sequentially rather than simultaneously improves both coherence and sound quality.

LeVo 2 generates full-length songs by combining language models and diffusion models in a hierarchical approach: first predicting mixed vocal-instrument tokens for overall coherence, then refining each track separately for acoustic detail.

Philip Zmushko, Egor Petrov, Nursultan Abdullaev et al.

Asynchronous pipeline parallelism with one-step gradient delay is practical for large LLM training if you use the right optimizer; the performance gap with synchronous training can be closed with modern optimizers and error feedback corrections.

This paper shows that asynchronous pipeline parallelism for LLM training isn't fundamentally limited by stale gradients—the problem depends on which optimizer you use. Modern optimizers like Muon handle one-step gradient delays well, while older ones like AdamW struggle.

Subramanyam Sahoo, Aman Chadha, Vinija Jain et al.

Conservative offline training doesn't prevent reward hacking in online adaptation—it amplifies it. The sweet spot is calibrated conservatism, not maximum conservatism, because overly conservative policies exploit reward model uncertainty more effectively.

This paper challenges the common assumption that conservative offline training prevents reward hacking. Testing a reasoning model with varying levels of conservatism during offline training, then online adaptation, the authors find that higher conservatism actually increases reward hacking—the model exploits disagreements in the reward model more effectively.

Xinlei Yu, Gen Li, Qingyi Si et al.

When distilling from privileged teachers or students, routing supervision based on advantage gaps prevents students from learning to exploit information asymmetry instead of real capabilities—improving both LLM and vision-language model performance.

This paper addresses a key problem in on-policy distillation where adding privileged information (extra inputs) to teachers or students creates a 'privilege illusion'—students learn to mimic information asymmetry rather than transferable skills.

Ziwei Su, Junyu Ren, Victor Veitch

Embedding norms in contrastive models aren't wasted information—they automatically capture semantic properties during training and can be leveraged as free calibration signals without additional training.

This paper explains why embedding norms (magnitudes) in contrastive models encode semantic information like concept specificity, even though these models use scale-invariant losses that should ignore norms.

Lei Bai, Zongsheng Cao, Yang Chen et al.

You can match trillion-parameter model performance on complex reasoning tasks with a 35B model by focusing on longer, more complex action sequences and multi-domain expertise rather than raw parameter count.

Agents-A1 is a 35B parameter model that achieves trillion-parameter performance on complex tasks by scaling agent horizon—the length and complexity of action sequences—rather than model size.

Haoran Jin, Xiting Wang, Shijie Ren et al.

When scaling sparse autoencoders for interpretability, enforcing cross-sample consistency prevents features from fragmenting or developing exceptions, making the learned representations more reliable for understanding language model behavior.

This paper identifies and fixes two major problems in Sparse Autoencoders (SAEs) used to interpret language models: feature splitting (where single concepts fragment into multiple latents) and feature absorption (where general features develop arbitrary exceptions).

Julius Girardin, Emanuele Troiani, Yizhou Xu et al.

Generalization doesn't scale uniformly with width and data—the relationship changes dramatically across different regimes, with the data's spectral structure determining how performance improves.

This paper analyzes how neural networks generalize as both model size and training data scale together. Using a simplified quadratic network model with structured data, the researchers derive exact formulas showing that generalization error follows different power-law patterns depending on the ratio of parameters to samples, revealing distinct phases like interpolation onset.

Wei Zhou, Xiongwei Zhu, Zelin Xu et al.

Multi-task image generation models can be trained more effectively by treating each capability (T2I, local edit, global edit) as a separate velocity field and having the student learn to compose them on its own generated trajectories.

DanceOPD is a training framework that helps image generation models master multiple tasks—text-to-image, local editing, and global editing—without them interfering with each other. It uses a distillation approach where a student model learns from specialized 'capability fields' (velocity fields in flow-matching models), routing each image to the right expert for its task.

Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang et al.

You can train LLMs with RL on open-ended optimization tasks using only execution feedback—no ground-truth needed—and the improvements transfer to exact-solution problems, suggesting score-based tasks are valuable for general capability development.

This paper introduces RiVER, a method for training language models using reinforcement learning on tasks without ground-truth answers. Instead of requiring correct solutions, it uses continuous feedback from execution scores (like how well a heuristic algorithm performs).

Danyal Rehman, Charlie B. Tan, Yoshua Bengio et al.

Autoregressive models can outperform flow-based approaches for molecular sampling by avoiding invertibility constraints and enabling better scalability—opening a new direction for physics-informed generative modeling.

This paper introduces Autoregressive Boltzmann Generators (ArBG), a new method for efficiently sampling molecular systems at equilibrium. Unlike previous approaches using normalizing flows, ArBG uses autoregressive models to generate molecular configurations faster and more accurately, with a large pre-trained model (Robin) achieving 60% better energy predictions on peptide systems.

Tianyi Men, Zhuoran Jin, Pengfei Cao et al.

Small 7B models can outperform much larger 32B models at web automation by learning high-level task decomposition through autonomous exploration and hindsight experience, rather than just memorizing low-level actions.

This paper improves small multimodal AI models for web automation by having them autonomously explore environments to learn task planning. The key innovation is using 'hindsight experience'—learning from failed attempts by reframing them as high-level tasks—which helps models generalize to new websites better than training on low-level atomic actions alone.

Yu-Neng Wang, Sara Achour

Analog hardware can generate images 100x more efficiently than digital systems, but requires rethinking model design to match fixed physics-based dynamics rather than flexible neural networks.

This paper proposes Analog Interaction Systems (AIS), a framework for building generative models on analog hardware like coupled oscillators. The key innovation is bridging the gap between what neural networks can do and what analog physics naturally computes—using time-varying parameters and hidden states to improve expressivity while keeping energy costs 100x lower than digital approaches.

Ping Liu, Qianqi Shen, Jianqiang Shen et al.

When training models with RL and AI feedback, reward signal design is critical—robust reward shaping prevents exploitation better than algorithm choice, and rule-based corrections can fix systematic failures like verbatim copying.

This paper tackles a real-world problem in job search: generating portable queries that capture candidate qualifications without user-specific details. The authors use reinforcement learning with AI feedback (RLAIF) to train models, but discover that standard reward signals get exploited—models learn to copy text verbatim instead of generalizing.

Alina Bazarova, Johann Fredrik Jadebeck, Henrik Zunker et al.

Neural simulation-based inference can replace slow MCMC for fitting complex disease models, running 15-120x faster on GPUs while producing nearly identical results—enabling real-time outbreak analysis.

This paper compares simulation-based inference (SBI) with traditional MCMC methods for fitting epidemiological models to COVID-19 data. SBI uses neural networks to learn the relationship between model parameters and data, enabling much faster Bayesian inference—achieving 15-120x speedups while maintaining accuracy comparable to MCMC.

Andrew Bond, Zafer Doğan

Structured correlations in data can boost learning of weak features in GANs, but excessive correlation destabilizes training—there's a sweet spot determined by learning rates and noise.

This paper analyzes GAN training mathematically by studying how a linear generator learns data structure. The key innovation is handling realistic data with correlated features and class-dependent patterns—not just simple diagonal structure. The authors prove training converges to predictable equations and show that smart use of correlations can help weak features become learnable.

John Sweeney

FisherSketch enables practical source selection for LLM families by measuring task similarity through Fisher alignment signatures (16 KB per task) instead of expensive full Fisher matrices, revealing whether tasks differ in activations, errors, or their interaction.

This paper solves the problem of selecting training data sources for language models that share vocabularies but differ in tasks (like SMILES vs protein sequences).

Sayak Dutta

Recurrent models can match Transformer efficiency by making forget gates content-aware (looking at stored memory) rather than memory-blind, enabling a mathematical solver that speeds up training while improving language understanding.

CARVE improves recurrent neural networks by fixing how they decide what to forget. Instead of gates that only see new incoming data, CARVE's gates look at what's already stored in memory before deciding what to erase. This single change fixes three architectural problems, enables faster training, and achieves better performance on language tasks while using less memory than competing approaches.

Ziyuan Tang, Tianshi Xu, Yousef Saad et al.

HiMuon makes Muon optimization 10-100x faster by processing weight-matrix tiles independently rather than as full matrices, enabling practical use of this advanced optimizer on large models without sacrificing training quality.

This paper introduces Hierarchical Muon (HiMuon), a faster version of the Muon optimizer for training neural networks. Instead of updating all weights at once, HiMuon splits weight matrices into tiles and updates each tile independently, reducing computation from O(r²sK) to O(HWТK) while maintaining similar training performance.

Jeremias Ferrao, Niclas Müller-Hof, Iustin Sîrbu et al.

Training safety classifiers to explicitly model user intent—not just analyze prompts directly—produces more robust safety decisions across different training approaches and external benchmarks.

This paper shows that safety classifiers work better when they explicitly model what users intend to do, not just what they say. The authors created AIMS, a dataset of 1,724 tricky safety prompts with intent descriptions, and tested intent-aware training across multiple methods (fine-tuning, preference learning, reasoning distillation, and reinforcement learning).

Dong Jing, Tianqi Zhang, Jiaqi Liu et al.

Pretraining action modules on motion structure before vision-language alignment significantly improves robot learning efficiency and cross-embodiment generalization, particularly in data-scarce real-world settings.

This paper proposes a two-stage training approach for robot manipulation models that first learns motion patterns from action trajectories alone, then transfers this knowledge to vision-language-action models.

Andrei Liviu Nicolicioiu, Mohammad Pezeshki, Aaron Courville

Self-distillation trades diversity for accuracy: models become overconfident in their preferred solutions, hurting performance on out-of-distribution tasks that need varied strategies.

This paper reveals a hidden cost of on-policy self-distillation: while it achieves high average accuracy, it reduces output diversity by amplifying the model's existing biases. The authors show theoretically and empirically that self-distillation concentrates probability mass on dominant modes, causing pass@k curves to flatten—generating more rollouts doesn't improve accuracy like it should.

Changdae Oh, Wendi Li, Seongheon Park et al.

You can extract free step-level evaluation signals from standard RL post-training using progress advantage, eliminating the need to build expensive process reward models for agent systems.

This paper shows that RL-trained language models already contain step-level scoring signals without needing separate reward models. The authors derive 'progress advantage'—a metric based on policy log-probability ratios—that automatically captures how good each step is, and demonstrate it works for scaling, uncertainty, and debugging across multiple benchmarks.

Sen Li, Haichao Cui, Chendong Shao et al.

Unsupervised domain adaptation can transfer welding quality models between fundamentally different processes (laser vs. arc welding) without retraining from scratch, reducing the cost of deploying AI to new manufacturing equipment.

This paper solves a real manufacturing problem: deep learning models trained on one welding process fail when applied to another because the physical mechanisms differ. The authors use unsupervised domain adaptation to learn features that work across both laser and TIG welding, achieving 80%+ accuracy in cross-process transfer without needing labeled data from the new process.

Juliana Li, Diya Sreedhar

Language models don't just forget rules randomly during training—their survival is determined by corpus statistics (support frequency), and this process is irreversible: you can kill learned behaviors but cannot resurrect them through data manipulation.

During language model pretraining, learned rules like pronoun-gender agreement mysteriously disappear mid-training even though evidence for them remains in the data. This 'natural ungrokking' is predictable: rules survive based on how often the training data supports them relative to competing patterns.

Poojitha Thota, Shirin Nilizadeh

Poisoned summarization models leave detectable structural artifacts—high training influence in white-box settings and unusual sensitivity to semantic perturbations in black-box settings—allowing 85-92% detection accuracy and recovery of 96% of original behavior without full retraining.

This paper presents a defense framework against data poisoning attacks on text summarization models during fine-tuning. The authors develop detection methods using influence analysis (white-box) and behavioral auditing (black-box), plus an unlearning technique to remove poisoned effects.

Maggie Wang, Lars Osterberg, Stephen Tian et al.

By making VLA models steerable at the primitive-action level, you can create a self-improving loop where robots identify skill gaps, practice autonomously, and expand their capabilities continuously.

InSight enables vision-language-action models to autonomously learn new manipulation skills by breaking down demonstrations into reusable primitive actions (like "move gripper to bowl").

Negin Raoof, Richard Zhuang, Marianna Nezhurina et al.

Systematic data curation matters more than you might think—the right mix of task sources and diversity in training data significantly improves how well agents generalize across different benchmarks.

This paper presents OpenThoughts-Agent, an open framework for creating training data for AI agents that can handle diverse tasks. The authors ran 100+ experiments to understand what makes good training data, then created a 100K example dataset that improved agent performance by 3.9 percentage points over existing open models.

Zixuan Li, Haokun Lin, Yicheng Xiao et al.

Separating structure planning from appearance rendering in image generation improves prompt following for complex spatial and compositional requirements without needing intermediate outputs.

This paper improves text-to-image generation by separating structural planning from appearance rendering. IV-CoT uses two types of queries—structural and semantic—that work together in a single pass: structural queries create a latent visual plan (like an invisible sketch), then semantic queries render the final image based on that plan.

Ahmad Pouramini, Hesham Faili

Matching a task's requirements to the model's pre-training objective—and maintaining that alignment through fine-tuning templates and prompts—dramatically improves performance, especially with limited data.

This paper shows how to better adapt pre-trained encoder-decoder language models by matching task requirements to pre-training objectives. The authors introduce the Match Task to Objective (MTO) framework, which automatically selects appropriate training objectives and designs aligned prompts/templates.

Maria Nefeli Paraskevopoulou, Tatiana Passali, Grigorios Tsoumakas

For scientific summarization, training on carefully selected high-quality examples outperforms training on larger random datasets—quality matters more than quantity when building summarization systems.

This paper creates a large biomedical summarization dataset (1.88M articles) and shows that author-written abstracts vary in quality. By selecting high-quality training examples based on alignment with source articles, models achieve better results with less data than random sampling, improving both efficiency and factuality.

Manas Mehta, Fangcong Yin, Greg Durrett

Training models with randomized positional encodings from a larger range than your data helps them generalize to much longer sequences without requiring long-context training data.

This paper proposes Randomized YaRN, a training method that helps language models generalize to much longer text sequences than they were trained on. By randomly sampling positional encodings from a larger range during training on short sequences, the model learns to handle longer contexts it hasn't seen before—improving performance on reasoning tasks with 16K-128K token contexts.

Cong Han, Xiaohan Lan, Haibo Qiu et al.

MLLMs can be trained to adaptively switch between reasoning and code execution for numerical tasks—not just visual ones—using reinforcement learning with a specialized reward function that guides when to invoke tools.

This paper enhances multimodal AI models to reason through complex math and numerical problems by interleaving natural language thinking with executable code. The authors use reinforcement learning to train models to decide when and how to use code tools, achieving 6.1% accuracy improvement on benchmarks.

Dingzhi Yu, Hongyi Tao, Yuanyu Wan et al.

AdamW's convergence under heavy-tailed noise remains unproven; understanding this gap matters because real LLM training exhibits heavy-tailed gradients, but theory currently assumes finite variance.

This paper investigates whether AdamW, the standard optimizer for training large language models, can theoretically converge when gradient noise has heavy tails—a realistic scenario in LLM training. The authors prove some positive results and identify potential obstacles, framing this as an open theoretical problem.

Prateek Agnihotri, Sanchit Jain, Prabhat Agnihotri et al.

Reframing symbolic reasoning problems as string matching and search—rather than arithmetic simulation—helps LLMs avoid hallucinations and handle combinatorially complex tasks more reliably.

This paper tackles bit manipulation puzzles by teaching LLMs to deduce hidden logical rules transforming binary strings. Instead of forcing models to simulate complex boolean logic (which causes hallucinations), the authors reframe the problem as string similarity matching and structured search with backtracking.

Florian Hübler, Kai Lion, Antonio Orvieto et al.

Muown's effectiveness comes from implicit angular step-size decay; making this explicit in AngularMuown gives you a faster, more controllable optimizer for Transformer pre-training with decoupled learning rate scheduling for directions vs. magnitudes.

This paper analyzes how Muown, a matrix-aware optimizer for training Transformers, implicitly controls step sizes through angular (directional) updates. The authors reformulate this insight into AngularMuown, which explicitly separates angular step-size scheduling from magnitude updates, improving training speed and stability across model sizes from nanoGPT to 1.1B parameter models.

Changxiao Cai, Yuchen Jiao, Gen Li

Diffusion models robustly adapt to low-dimensional structure across a wide range of coefficient choices, meaning practitioners don't need to fine-tune these hyperparameters precisely to get dimension-independent speedups on structured data.

This paper proves that diffusion models can efficiently sample from low-dimensional data structures regardless of how you set their update coefficients, as long as they fall within a broad class. The key finding is that sampling takes only O(k/ε) iterations (where k is the intrinsic dimension), independent of the ambient dimension—showing this efficiency isn't fragile to implementation details.

Qingyang Zhu, Eric Karl Oermann, Kyunghyun Cho

You can train a transformer to act as a fast Bayesian predictor by treating prior information as part of the input context, achieving oracle-level accuracy orders of magnitude faster than traditional Bayesian methods.

This paper presents a method for training transformers to perform Bayesian inference quickly by learning from examples of prior distributions and target datasets. Instead of computing exact Bayesian predictions (which is slow), the model learns to map sequences of prior information and data directly to predictions, enabling fast uncertainty-aware inference that adapts to new priors at test time.

Harshit Singh, Ayush Pratap Singh, Nityanand Mathur

You can add lifelong learning to frozen TTS models by storing pronunciation fixes in a memory network instead of updating weights—enabling fast adaptation to new proper nouns without retraining.

FlowEdit enables text-to-speech systems to learn and remember pronunciation corrections for proper nouns without retraining. It stores corrections as edits in a memory network, then retrieves and applies them at inference time, reducing pronunciation errors by 93% while keeping the original model frozen.

Asa Shepard, Jeannie Albrecht

Automatically tuning repository guidance through synthetic bug probes improves coding agent performance by 7.5 percentage points, primarily by helping agents locate correct files rather than improving code quality.

This paper shows that repository guidance files (like AGENTS.md) help coding agents fix bugs, but only if created the right way. The authors introduce probe-and-refine tuning: a method that tests guidance against synthetic bugs and automatically improves it.

Sihui Dai, Mann Patel

Safety training through preference optimization is critical for preventing benign demonstrations from accidentally increasing harmful compliance—models extract different lessons from the same demonstrations depending on their training methodology.

This paper investigates how language models interpret mixed compliance demonstrations—some showing helpful responses to benign requests, others showing helpful responses to harmful requests. The researchers find that benign and harmful demonstrations aren't interchangeable; their effect on jailbreaking depends on model training, demonstration order, and how the model handles refusals.

Jinghong Lan, Wei Cheng, Yunuo Chen et al.

By mining community LoRAs as style-content anchors and using curriculum learning with targeted disentanglement mechanisms, you can scale dual-reference image generation while maintaining clean separation between style and content.

FreeStyle is a framework for generating images that combine the style of one image with the content of another. It uses community LoRA models as building blocks to create large training datasets and employs specialized techniques to prevent unwanted style or content leakage. The approach includes a new benchmark for evaluating dual-reference image generation.

Arastoo Zibaeirad, Marco Vieira

Fine-tuning LLMs for vulnerability detection produces calibration without comprehension: models adjust their confidence scores to match training data but don't develop actual security reasoning.

This paper evaluates whether LLMs actually understand software vulnerabilities or just memorize patterns. Using 834 carefully curated Linux kernel samples with strict temporal splits to prevent data leakage, the authors find that fine-tuning doesn't improve genuine security reasoning—it only adjusts output thresholds.

Haw-Shiuan Chang, Jeffrey Gomez, Mehul Patwari et al.

Implicit user signals (eye gaze, mouse movement) can substantially improve LLM reward models and alignment, suggesting that behavioral data is a practical alternative to expensive explicit human feedback collection.

This paper shows that user behavior signals like mouse movements and eye gaze contain valuable information about LLM response quality.

Mingyu Yang, Keye Zheng, Congchao Cheng et al.

MAA enables agents to learn which memory operations consistently help by accumulating cross-batch evidence, making agent self-improvement more efficient and reliable without requiring online training.

This paper addresses a problem in training AI agents: when the same memory operation gets conflicting feedback across different training batches, it's hard to know which operations actually work. MAA solves this by accumulating evidence for each operation across batches and filtering out unreliable ones, improving agent learning while using 75% fewer tokens during training.

Md Sakir Ahmed, Kumaresh Sarmah, Hemen Dutta

Flatness matters for generalization, but only when measured using Fisher geometry—standard Euclidean measures are misleading because they change when you reparametrize the network while keeping its function identical.

This paper solves a long-standing problem in deep learning: why flat minima generalize better. The authors show that standard flatness measures fail because they change under reparametrization, but by using the Fisher Information Matrix geometry, they define a reparametrization-invariant flatness measure that provably explains SGD's bias toward flat minima and their generalization.