Papers

Yanbei Chen, Hanxian Huang, Ernie Chang et al.

MoE isn't just for giant models—on mobile devices, moderate sparsity with shared experts is both memory and compute-optimal, letting you get better performance with fewer active parameters than dense models.

MobileMoE brings Mixture-of-Experts (MoE) architecture to phones and edge devices by optimizing it for memory and compute constraints. The models use 0.3-0.9B active parameters but achieve better performance than larger dense models, running 2-4× faster on real smartphones while using less memory.

Yuchen Liang, Ness Shroff, Yingbin Liang

GADD accelerates discrete diffusion sampling from many steps to logarithmically few steps without additional training, providing both theoretical guarantees and practical speedups for text and symbolic generation tasks.

This paper speeds up discrete diffusion models (used for text and symbolic data generation) by introducing GADD, a new method that uses Gibbs corrections to reduce sampling steps. Unlike existing acceleration techniques, GADD doesn't require extra training and achieves theoretically optimal speedup, making it practical for real applications like text and music generation.

Nithesh Chandher Karthikeyan, Jonas Unger, Gabriel Eilertsen

You can guide diffusion models to generate specific images by using learned representations as conditioning signals, avoiding the need for expensive annotated datasets while maintaining smooth, interpretable control.

This paper shows how to control image generation in diffusion models by conditioning them on representations from self-supervised models instead of requiring text or semantic annotations. The approach discovers interpretable directions in the representation space that let you smoothly control what gets generated.

Haoxuan Jia, Yang Liu, Bin Chong et al.

Real-time safety monitoring during agent execution is more effective and efficient than post-hoc auditing—catching risky actions before they happen and routing verification intelligently reduces security breaches by 61% while cutting computational costs by 78%.

FinHarness is a safety system for AI agents handling financial transactions that monitors requests in real-time rather than after the fact. It catches risky actions mid-process by checking user intent and each tool call, then routes complex decisions to either a fast or thorough AI judge based on risk level.

Sanghyun Lee, Chunsan Hong, Seungryong Kim et al.

Selectively looping transformer layers in masked diffusion models improves both training efficiency and reasoning capability—you can match performance with far fewer computations, or trade compute for better results.

This paper introduces LoopMDM, a technique that reuses early-middle transformer layers in masked diffusion models by looping them during training and inference. The approach achieves better training efficiency (3.3× fewer FLOPs) and stronger reasoning performance than standard models, while enabling flexible compute scaling at inference time without adding parameters.

Sangyun Lee, Sean McLeish, Tom Goldstein et al.

Language models can improve long-context reasoning by periodically consolidating recent information into fast weights during offline 'sleep' phases, trading inference latency for better performance on reasoning-heavy tasks.

This paper proposes a sleep-like mechanism for language models that periodically consolidates recent context into persistent memory before clearing the cache. During 'sleep,' the model performs offline passes to update fast weights in state-space blocks, shifting computation away from real-time inference.

Martin Marek, Dongkyu Cho, Shikai Qiu et al.

Self-generated replay nearly eliminates catastrophic forgetting in language models, but capacity constraints are the real bottleneck: a saturated model can't learn new tasks without forgetting, no matter what technique you use.

When language models learn new tasks, they forget old ones. This paper shows that models can generate their own training data to replay and prevent forgetting, but only if they have spare capacity. If a model is already saturated from pretraining, no amount of replay helps—it must overwrite old knowledge to learn anything new.

Maoyang Xiang, Bo Wang, Tao Luo

Power-of-Two quantization with orthogonal residual projection lets you run large models on edge hardware with minimal accuracy loss and no multiplier circuits—calibration takes ~15 minutes instead of hours.

OrpQuant enables efficient deployment of large AI models on edge devices by using a novel quantization method that replaces expensive multiply operations with simple bit-shifts. The approach uses geometric projection to maintain accuracy even at ultra-low bit widths (3-4 bits), and can calibrate models 10x faster than existing methods.

Wei Song, Tianhang Wang, Yitong Chen et al.

Quantizing image channels instead of patches improves codebook efficiency and enables a more intuitive generation process that mirrors human artistic creation, achieving strong text-to-image results.

This paper introduces Channel-wise Vector Quantization (CVQ), a new way to convert images into discrete tokens by quantizing color channels instead of spatial patches. It enables a new image generation model (CAR) that builds images progressively—first sketching overall structure, then adding fine details—similar to how artists work.

Joydeep Chandra

For building data marketplaces, CHRONOS shows how to maintain search quality, fair pricing, and privacy simultaneously by treating temporal decay, value attribution, and privacy budgets as coupled problems rather than separate concerns.

CHRONOS solves three interconnected problems in data marketplaces: keeping search indexes fresh as data changes, fairly pricing data contributions after market shifts, and preventing agents from exhausting privacy budgets.

Lizhang Chen, Jonathan Li, Chen Liang et al.

You can boost performance of frozen models by intelligently looping internal layers at inference time—no retraining needed, just a smarter application strategy based on ODE theory.

This paper shows how to improve pretrained transformer models at test time by looping a middle section of layers without retraining. The key insight is treating layer loops as smaller refinement steps rather than naive repetition, inspired by numerical methods for solving differential equations.

Taiming Lu, Zhuang Liu

You don't need a powerful teacher to improve a larger language model through distillation—smaller teachers work fine, and over-training the teacher can actually hurt performance.

This paper challenges the assumption that knowledge distillation in language model training requires a strong teacher model. By systematically testing different teacher-student size combinations, the researchers found that even small, undertrained teachers can improve larger students when losses are properly balanced, and that stronger teachers don't always produce better results.

Jan Tempus, Philip Whittington, Craig W. Schmidt et al.

ConvexTok uses convex optimization to build tokenizers that are provably near-optimal (within 1% at typical vocabulary sizes) and compress text better than greedy algorithms like BPE, with measurable improvements in language model efficiency.

This paper replaces greedy tokenization algorithms like BPE with a convex optimization approach called ConvexTok. Instead of making locally optimal choices, it formulates tokenizer construction as a linear program, achieving better compression (bits-per-byte) and allowing users to verify how close their tokenizer is to mathematically optimal.

Ryan Bahlous-Boldi, Isha Puri, Idan Shenfeld et al.

Training LLMs to produce diverse outputs across multiple reward dimensions—not just maximizing a single score—makes them better at test-time search where you can pick the best solution from many candidates.

This paper introduces Vector Policy Optimization (VPO), a training method that teaches language models to generate diverse solutions by optimizing for multiple reward objectives simultaneously, rather than a single scalar reward.

Ali Hatamizadeh, Yejin Choi, Jan Kautz

Decoupling erase and write operations in linear attention with separate gates improves language model performance, especially on long-context tasks, while maintaining constant-memory decoding.

This paper improves linear attention mechanisms by separating the control of what to forget from what to remember in compressed memory. Instead of using a single gate to control both erasing old information and writing new information, Gated DeltaNet-2 uses separate channel-wise gates for each operation, making memory updates more flexible and efficient.

Sadia Asif, Mohammad Mohammadi Amiri, Momin Abbas et al.

When LLM agents communicate through shared KV caches for efficiency, you need explicit safeguards—LCGuard shows how to block sensitive information leakage at the representation level without breaking task coordination.

LCGuard is a safety framework that protects sensitive information when multiple AI agents share transformer key-value caches to coordinate tasks. It uses adversarial training to transform shared cache data so that agents can't reconstruct each other's private inputs, while keeping the information useful for task performance.

Yunpeng Dong, Jingkai He, Yuze Hou et al.

By tracking only differences between consecutive states rather than full duplicates, DeltaBox reduces AI agent checkpoint/rollback latency from seconds to milliseconds, directly enabling deeper search and larger-scale exploration for reasoning and RL tasks.

DeltaBox is a system that makes AI agents much faster by storing only the changes between checkpoints instead of copying entire sandbox states. Using new OS-level mechanisms for filesystems and process state, it reduces checkpoint/rollback time from hundreds of milliseconds to just milliseconds, enabling agents to explore more possibilities in the same time budget.

Huanchi Wang, Zihang Huang, Yifang Tian et al.

You can build practical, label-efficient log anomaly detectors by using LLMs once offline to structure the problem, then training lightweight domain-specific models that run continuously without expensive LLM calls.

FAME is a system for detecting anomalies in individual log messages rather than groups, using a mixture-of-experts approach that leverages an LLM offline to organize log templates into failure domains. It requires minimal labeled data (as few as 100 examples) and runs efficiently on-premise, achieving 98% accuracy on real production logs while reducing annotation effort by 76x.

Amir Mousavi, Mohammad Sadegh Sirjani, Erfan Nourbakhsh et al.

Mamba's linear-complexity architecture enables real-time cognitive load monitoring from noisy eye-tracking signals on wearable devices—a practical alternative to Transformers for temporal sensor data with frequent gaps.

MambaGaze uses a bidirectional Mamba neural network to assess cognitive load from eye-tracking data in real-time. It handles missing data from eye blinks and tracking failures by explicitly encoding uncertainty, and runs efficiently on edge devices like smartglasses for applications like driver monitoring.

Samson Gourevitch, Yazid Janati, Dario Shariatian et al.

Discrete diffusion models have a hidden training-inference mismatch: the standard objective doesn't match what's actually needed for sampling. Using the correct "leave-one-out" parameterization and an absorbing-state reformulation improves generation quality without retraining.

This paper fixes a fundamental mismatch in how Uniform Diffusion Models are trained versus used for generation. The authors show that standard training doesn't actually optimize what the model uses during sampling, and they provide mathematical conversions to align these.

Christian Janos Lebeda, David Erb, Tudor Cebere et al.

You can now build random forests on sensitive data with differential privacy that actually work well in practice—Lumberjack's smart pruning strategy significantly closes the gap between private and non-private model performance.

Lumberjack is a differentially private random forest algorithm that builds large decision trees and then prunes them intelligently to protect sensitive data. By using a novel heavy hitter detection method, it can use deeper trees than previous approaches while maintaining privacy guarantees, achieving much better accuracy on real datasets.

Berk Hayta, Hannah Laus, Simon Mittermaier et al.

You can get reliable uncertainty estimates using standard loss functions (cross-entropy, MSE) instead of complex Dirichlet objectives—the math shows this works, and it's simpler to implement in practice.

This paper simplifies Evidential Deep Learning (EDL) for uncertainty estimation by replacing complex Dirichlet-based losses with standard losses like cross-entropy, evaluated at the Dirichlet mean. The authors prove this approximation works well when evidence is strong and show it includes softmax as a special case, making uncertainty estimation easier to implement without sacrificing accuracy.

Javad Parsa, Enis Simsar, Amir Joudaki et al.

When fine-tuning diffusion models for multiple concepts, jointly optimizing LoRA factors with orthogonal constraints prevents representation interference and scales better than existing modular approaches—enabling cleaner composition of up to 101 concepts.

SeqLoRA improves how AI models learn multiple custom concepts at once by using a smarter optimization technique that prevents concepts from interfering with each other. Instead of freezing parts of the model or doing expensive post-processing, it jointly trains the adaptation components while keeping them orthogonal, enabling better multi-concept image generation with less computational cost.

Santo M. A. R. Thies, Hubert Baniecki, R. Teal Witter et al.

ProxySHAP makes it practical to explain complex feature interactions in ML models by using proxy models and residual correction, achieving state-of-the-art accuracy while remaining computationally efficient even with thousands of features.

ProxySHAP is a new method for computing Shapley and Banzhaf interactions—measures that explain how features work together in machine learning models. It combines fast tree-based approximations with mathematical corrections to achieve both speed and accuracy, outperforming existing methods on large datasets.

Youssef Allouah, Mahdi Haghifam, Sanmi Koyejo et al.

Distillation defenses must be evaluated against adaptive attackers who strategically choose which outputs to learn from—not just passive ones—and simple forward-pass defenses like PoE can match expensive defenses while preserving reasoning quality.

This paper studies how AI model providers face a trade-off: making models more useful (through better outputs) makes them easier to copy through distillation attacks. The authors develop a game-theoretic framework to understand this trade-off and propose Product-of-Experts (PoE), a lightweight defense that combines the teacher model with a proxy student during generation.

Jesse Bettencourt, Xindi Wu, Matan Atzmon et al.

When using diffusion models to guide other tasks, you can dramatically reduce compute cost by resampling cheap diffusion noise multiple times per expensive upstream computation, rather than doing one expensive computation per noise sample.

This paper introduces CARV, a framework for reducing variance in gradient estimates when using pretrained diffusion models as teachers in downstream tasks like text-to-3D generation. By reusing expensive computations (like 3D rendering) across multiple noise samples and applying importance sampling techniques, the method achieves 2-3x speedups without changing the underlying objective.

Dayal Singh Kalra, Maissam Barkeshli

When scaling up LLM training, use a higher embedding layer learning rate (scaled by model width) to stabilize training and reliably transfer hyperparameters from small to large models—this is the primary reason μP outperforms standard parameterization.

This paper explains why μP (Maximal Update) parameterization works better than standard parameterization for transferring learning rates across different model sizes. The key finding: μP's advantage mainly comes from using a higher learning rate for the embedding layer, which stabilizes training and improves hyperparameter transfer when scaling up language models.

Caleb Winston, Ron Yifeng Wang, Azalia Mirhoseini et al.

Compiling agent tasks into code upfront—rather than deciding actions one step at a time—enables parallelization and validation, dramatically reducing latency and errors in web automation.

This paper introduces a compilation approach for web agents that converts natural language tasks into executable code plans instead of executing step-by-step. By generating multiple candidate plans, validating them against tool specifications, and optimizing for parallelization, the system achieves 10x faster execution and better accuracy than existing sequential approaches.

Zhepei Wei, Xinyu Zhu, Wei-Lin Chen et al.

RLVR training produces predictable, low-rank weight changes that can be extrapolated mathematically, letting you skip 85% of training compute while matching or exceeding performance on reasoning tasks.

This paper reveals that language models trained with reinforcement learning from verifiable rewards (RLVR) follow surprisingly simple, low-rank weight trajectories.

Yuxiang Huang, Nuno M. T. Gonçalves, Federico Alvetreti et al.

DashAttention enables efficient long-context processing by combining adaptive sparse selection with differentiable training, outperforming fixed-sparsity methods while maintaining gradient flow through both attention stages.

DashAttention improves how language models handle long documents by using a smarter two-stage attention mechanism. Instead of always selecting the same number of relevant tokens, it adaptively picks different amounts based on what each query needs, while keeping the entire process trainable. This achieves full-attention quality with 75% fewer computations.

Ruitao Liu, Xinyang Tian, Shuo Chen et al.

For distributed model training, executing tasks based on actual readiness rather than pre-committed schedules can dramatically reduce GPU idle time and improve throughput, especially when computation times vary unpredictably.

This paper introduces RRFP, a runtime system that improves GPU training efficiency by executing ready tasks immediately instead of waiting for a pre-planned order. When training large models across multiple GPUs, unpredictable delays in computation cause stages to sit idle.

Lifu Wei, Yinuo Ren, Naichen Shi et al.

You can guide diffusion models without computing gradients or scores—just reweight trajectories and resample periodically, making inference-time improvements cheaper and easier to implement.

This paper introduces URGE, a gradient-free method for improving diffusion model outputs at inference time. Instead of computing expensive gradients, URGE reweights and resamples trajectories using a mathematical technique called Girsanov estimation, making guidance simpler and faster while maintaining theoretical guarantees.

Qianhao Yuan, Jie Lou, Xing Yu et al.

MLLMs can improve fine-grained visual understanding by learning from their own superior performance on evidence-focused crops, using on-policy self-distillation to transfer regional perception skills to full-image reasoning.

This paper addresses a key weakness in multimodal AI models: they struggle to notice small but important details in images. The researchers discovered that models actually perform better when shown cropped images focused on relevant areas versus full images, suggesting the problem isn't recognizing details but finding them.

Miguel Farinha, Ronald Clark

By conditioning on intrinsic image properties (albedo and shading) extracted from both photos and 3D renders, you can achieve photorealistic relighting with full PBR lighting control while staying fast enough for practical use.

PIXLRelight is a fast neural relighting method that lets you change lighting in photos using physically-based rendering controls. It decomposes images into intrinsic components (albedo, shading, residuals) and uses these to condition a transformer model, enabling realistic lighting adjustments in under 0.1 seconds per image without per-image optimization.

Kenan Majewski, Marcin Żugaj

Neural networks can improve classical state estimation by learning adaptive forgetting factors that respond to real-time sensor quality, enabling robust UAV navigation during sensor outages and dynamic environments.

This paper presents a learned Kalman filter that adapts to changing noise conditions in UAVs by using a neural network to dynamically adjust how much it trusts past measurements. Instead of using a fixed forgetting factor, the filter learns a memory policy from sensor data, helping it handle sensor failures and vibrations better than traditional adaptive filters.

Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong et al.

You can add new knowledge to any LLM without touching its weights by training a separate memory model that retrieves and augments the LLM's responses—making it practical for real-world applications needing frequent updates.

MeMo introduces a modular memory model that stores new knowledge separately from a frozen LLM, enabling efficient updates without retraining. It works with any LLM (open or proprietary), handles complex document relationships, and maintains constant retrieval cost regardless of corpus size.

Alan Z. Song, Yinjie Chen, Mu Nan et al.

You can build efficient vision transformers by routing all patch interactions through a small set of learned core tokens instead of using all-to-all attention, achieving linear complexity without sacrificing performance.

This paper proposes VECA, a vision transformer that replaces quadratic all-to-all attention with linear-time attention using learned "core" tokens as communication hubs. Instead of every patch attending to every other patch, all patches only interact through a small set of learned cores, reducing computation from O(N²) to O(N) while maintaining competitive accuracy on vision tasks.

Ariel Gera, Shir Ashury-Tahan, Gal Bloch et al.

You can boost embedding model performance on hard search tasks by having an LLM refine queries at test-time, making embeddings practical for scenarios where running LLMs on all documents is too expensive.

This paper shows how to improve embedding models for search and classification by using an LLM to refine user queries in real-time. Instead of changing the embedding model itself, the approach adjusts the query representation based on feedback from a small sample of documents, achieving up to 25% improvement on challenging tasks without requiring expensive LLM processing at scale.

Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal et al.

Combining parameter updates with context optimization lets LLMs learn new tasks 3x more efficiently while staying closer to their original capabilities and avoiding the forgetting that comes from pure fine-tuning.

This paper proposes Fast-Slow Training (FST), a method that combines two learning mechanisms for LLMs: updating model parameters (slow learning) and optimizing the input context (fast learning). By separating task-specific adaptation from general knowledge, FST achieves better sample efficiency, reduces catastrophic forgetting, and maintains the model's ability to learn new tasks over time.

Sagi Ahrac, Noya Hochwald, Mor Geva

Routers in sparse mixture-of-experts models work best when they maintain geometric alignment with their experts—understanding this coupling can improve routing stability and reduce the need for complex auxiliary losses.

This paper reveals that routers in Sparse Mixture-of-Experts models learn a geometric relationship with their experts: router weights and expert weights receive gradients along the same directions, causing them to specialize together.

Alireza Nadali, Patrick Cooper, Ashutosh Trivedi et al.

You can extend transformer context length by simply reusing and accumulating the KV cache across chunks—no training needed, and the approach stays numerically stable even across very long sequences.

KV-Fold enables long-context inference by treating the key-value cache as an accumulator that gets passed between sequence chunks. The model processes each chunk while attending to cached information from previous chunks, allowing it to handle contexts up to 128K tokens without retraining or architectural changes.

Jacob Fein-Ashley, Paria Rashidinejad

Attractor Models make iterative refinement practical by using implicit differentiation to solve fixed points, enabling smaller models (27M-770M parameters) to outperform much larger ones on reasoning and language tasks without the training instability of traditional recurrent architectures.

This paper introduces Attractor Models, which improve on looped Transformers by using implicit differentiation to solve for fixed points in latent representations.

Tanmaey Gupta, Hayden Prairie, Xiaoxia Wu et al.

Smarter scale selection in Block Floating Point quantization can reduce quantization error by 27% and improve language model performance by up to 15 points without slowing down inference.

This paper improves quantization for AI models by optimizing how Block Floating Point formats choose their scale factors. Instead of using a fixed maximum-based scale, ScaleSearch searches for better scales that minimize quantization error. The method works with existing quantization techniques and includes a specialized attention algorithm, showing 15-point improvements on math reasoning tasks.

James Petullo, Sonny George, Dylan Cashman et al.

You can make confidence-weighted answer selection 47% cheaper by clustering similar reasoning traces and only evaluating unique ones, without sacrificing accuracy.

VecCISC reduces the cost of weighted majority voting for LLM reasoning by filtering out duplicate or low-quality reasoning traces before sending them to a critic model. It uses semantic similarity to identify which candidate answers are worth evaluating, cutting token usage by 47% while maintaining accuracy across math, science, and reasoning tasks.

Zhen Fang, Wenxuan Huang, Yu Zeng et al.

On-policy distillation with specialized teachers can resolve conflicting optimization goals in multi-objective image generation, achieving 10-point improvements over standard reinforcement learning approaches while maintaining quality across all metrics.

Flow-OPD is a training method that improves text-to-image models by using specialized teacher models and on-policy distillation to align multiple competing objectives (like image quality, text accuracy, and aesthetics).

Yi Yu, Parker Martin, Zhenyu Bu et al.

Distilled LLMs can extract medical data from unstructured reports with high accuracy and built-in confidence estimates, enabling clinicians to prioritize which extractions need human review.

CMR-EXTR converts free-text cardiac MRI reports into structured data with confidence scores for each extracted field. Using a lightweight distilled language model, it achieves 99.65% accuracy while running entirely offline, making it practical for clinical use without requiring constant API access.

Julie Kallini, Artidoro Pagnoni, Tomasz Limisiewicz et al.

Byte-level models can now generate 50% faster by predicting multiple bytes in parallel instead of one at a time, making them practical for real-world use without sacrificing quality.

Byte-level language models match token-based models but generate slowly because they produce one byte at a time. This paper introduces three faster variants: BLT-D uses diffusion to generate multiple bytes per step, BLT-S uses local drafting with verification, and BLT-DV combines both. All reduce memory bandwidth costs by over 50% during generation.

Minbin Huang, Han Shi, Chuanyang Zheng et al.

You don't need separate expert sets per layer in MoE models—a shared expert pool with independent routers works better and uses fewer parameters, suggesting the standard per-layer expert allocation is unnecessarily wasteful.

UniPool replaces the standard Mixture-of-Experts design where each layer has its own expert set with a single shared pool of experts accessed by all layers. This reduces redundancy and allows expert parameters to grow sublinearly with model depth while improving performance and reducing parameter count by 30-60% compared to standard MoE.

Borui Zhang, Bo Zhang, Bo Wang et al.

You can significantly improve GUI agent accuracy on complex interfaces without retraining by using a two-step approach: first narrow down the region of interest, then select the best candidate from remaining options.

This paper identifies why GUI grounding models (used by AI agents to click and interact with interfaces) fail on complex screens, finding two main problems: high image resolution causes precision errors, and complex UI elements create ambiguity.

Ryan Wang, Akshita Bhagia, Sewon Min

By constraining tokens within the same document to share expert pools during pretraining, EMO creates naturally modular experts that specialize in semantic domains (math, code, etc.), enabling practical memory-efficient deployment without sacrificing performance.

EMO is a Mixture-of-Experts language model designed to work efficiently when you only need a subset of its capabilities. Instead of forcing all experts to activate for every input, EMO groups experts by document domain during training, so code-heavy documents use code experts, math documents use math experts, and so on.

Yuxing Liu, Jianyu Wang, Tong Zhang

Use the same optimizer for finetuning as you used for pretraining—it significantly reduces catastrophic forgetting while maintaining task performance, even outperforming parameter-efficient methods like LoRA.

When finetuning large language models, using the same optimizer during finetuning as was used during pretraining reduces forgetting of previously learned knowledge while maintaining or improving performance on new tasks.

Zeyu Yang, Qi Ma, Jason Chen et al.

A single well-designed lexical query informed by LLM-predicted vocabulary and corpus statistics outperforms expensive multi-round retrieval agents—you don't need complex agentic loops if you get the query right upfront.

SIRA is a retrieval agent that replaces multi-round exploratory search with a single, smarter query by using an LLM to predict missing search terms and filter them against corpus statistics.

Xiaoyu Wu, Yifei Wang, Tsu-Jui Fu et al.

Outlier tokens in diffusion transformers aren't just extreme values but represent corrupted local information; controlling them with register tokens significantly improves image generation quality.

This paper identifies and fixes a problem in Diffusion Transformers where certain tokens develop unusually high values that degrade image quality. The authors show this happens in both the image encoder and the generation model itself, and propose Dual-Stage Registers—a technique using learnable tokens to stabilize these problematic values and improve image generation.

Yijun Lu, Rui Ye, Yuwen Du et al.

Agents performing long-horizon tasks need adaptive context management—selectively compressing or discarding information—rather than naively accumulating everything, which improves efficiency and reduces hallucination.

LongSeeker introduces Context-ReAct, a framework that helps AI agents manage growing context during long tasks by selectively compressing, skipping, or deleting information based on relevance. The agent uses five operations to reshape its working memory, reducing costs and errors while maintaining task-critical information.

Wilson Wu, Victor Lecomte, Michael Winer et al.

You can estimate a wide MLP's expected output more efficiently than sampling by directly computing activation distributions layer-by-layer using mathematical tools, which is particularly useful for detecting tail risks.

This paper presents a mathematical method to estimate what a randomly initialized neural network will output on average, without actually running data through it. Instead of sampling (the standard approach), the authors use statistical tools like cumulants and Hermite expansions to track how activations behave at each layer.

The Verkor Team, Ravi Krishna, Suresh Krishna et al.

Frontier LLMs can now autonomously design complex hardware accelerators from scratch, suggesting AI agents are becoming capable of end-to-end engineering tasks that previously required human teams.

An AI agent system autonomously designed a specialized hardware accelerator for LLM inference in 80 hours, starting from a research paper. The system improved dramatically from prior work, handling 80x larger tasks by leveraging newer frontier models, and produced a working FPGA design with thousands of compute units.

Mina Gabriel

A single metric based on the model's confidence distribution at the first answer token can reliably detect hallucinations without expensive multi-sample generation, making it a practical baseline for production systems.

This paper shows that checking a language model's confidence on just the first token of an answer can detect hallucinations as well as methods that generate multiple answers and compare them. The approach is faster and simpler, requiring only a single model run instead of repeated sampling.

Enhui Chai, Sicheng Chen, Tianyi Zhang et al.

Representing pathology image features in complementary geometric spaces (hyperbolic + Euclidean) with efficient sequence modeling enables more accurate whole-slide image analysis by capturing both tissue hierarchy and cellular details.

This paper presents BatMIL, a new approach for analyzing whole-slide images (gigapixel pathology scans) by representing tissue features in dual geometric spaces—hyperbolic for hierarchical structures and Euclidean for local details.

Francisco M. Castro-Macías, Pablo Morales-Álvarez, Saifuddin Syed et al.

CDS offers a practical way to sample from difficult distributions by combining two proven techniques—Parallel Tempering for initial exploration and exact diffusion dynamics for refinement—without requiring neural network training.

This paper introduces Conditional Diffusion Sampling (CDS), a new method for sampling from complex probability distributions that combines Parallel Tempering with diffusion-based transport.

Aaron Havens, Brian Karrer, Neta Shaul

Flow Sampling enables efficient sampling from unnormalized densities by reversing the diffusion process with energy guidance, making it practical for expensive-to-evaluate energy functions and non-Euclidean geometries.

This paper presents Flow Sampling, a method for drawing samples from energy functions without needing data. It adapts diffusion models to work backwards from noise, using the energy function to guide the sampling process. The approach is efficient because it minimizes how many times the energy function must be evaluated, and it works on curved spaces like spheres and hyperbolic geometry.

Shikhar Shukla

Speculative decoding's performance depends heavily on compression level and task type—adaptive speculation length selection based on draft model signals can significantly outperform fixed hyperparameters with minimal computational overhead.

SpecKV improves LLM inference speed by dynamically choosing how many tokens a draft model should propose at each step, rather than using a fixed number. The system learns from draft model signals (confidence and entropy) to predict which proposal lengths will be accepted most often, achieving 56% faster inference than standard fixed-length approaches.

Mohamad Khajezade, Fatemeh H. Fard, Mohamed Sami Shehata

Knowledge distillation from reasoning-optimized models plus response stabilization techniques can make compact open-source models practical for cross-language code clone detection, improving both reliability and inference speed.

This paper shows how to make smaller, open-source AI models better at detecting when code does the same thing across different programming languages. The researchers use knowledge distillation—teaching a smaller model by learning from a larger reasoning-focused model—combined with techniques to make outputs more reliable and consistent.

Komal Thareja, Anirban Mandal, Ewa Deelman

Pattern-based workflow templates combined with AI assistance can dramatically lower the barrier for non-experts to build and deploy sensor applications across edge-to-cloud infrastructure.

This paper presents a methodology for quickly building sensor-based applications that process data across edge devices and cloud infrastructure. Using AI assistance and reusable workflow patterns, the authors show how scientists can rapidly prototype applications for monitoring air quality, earthquakes, and soil moisture without needing deep expertise in distributed systems.

Arian Eamaz, Farhang Yeganegi, Mojtaba Soltanalian

Standard training loss curves can hide poorly-optimized layers in transformers—layer-wise analysis using reference bounds exposes optimization failures that aggregate metrics miss, especially critical for expensive model training.

This paper introduces a method to monitor whether transformer models are actually learning well during training by analyzing each layer individually. Instead of just looking at overall loss, the authors create lightweight reference solutions for each layer and compare them against the trained model, revealing hidden inefficiencies.

Komal Thareja, Anirban Mandal, Ewa Deelman

AI-assisted workflow templates let developers build sensor applications 5-10x faster by reusing patterns and shifting from code-first to intent-first design, making it practical for non-experts to deploy across edge devices and cloud.

This paper presents a method for quickly building sensor-based applications across edge and cloud systems using AI-assisted workflow templates.

Jingze Ge, Yun Liu, Xue Geng et al.

Jointly optimizing compression and adaptation using task-aware subspaces beats the standard two-step approach, delivering better accuracy with fewer parameters on both vision and language models.

JACTUS combines model compression and task adaptation in a single step rather than doing them sequentially. Instead of compressing a model first and then fine-tuning it, the method estimates what directions matter for your specific task and compresses the model while preserving those directions.

Ziqi Liu, Kiljae Lee, Yuan Zhang et al.

Understanding the shared mathematical structure of value estimation methods enables designing more statistically efficient estimators—EASE reduces mean squared error by jointly optimizing sampling and surrogate functions rather than treating them separately.

This paper explains how to efficiently estimate Shapley values and similar attribution methods that explain AI model decisions. The authors show that different estimation approaches share a common mathematical structure, then use this insight to design a better estimator (EASE) that reduces computational error by optimizing both the sampling strategy and the surrogate function used.

Jingxi Pu, Tonghua Liu, Zhilin Guan et al.

You can denoise real clinical CT images without paired training data by using unsupervised learning with perceptual loss, making it practical for hospitals that can't easily create labeled datasets.

This paper tackles noise in low-dose CT scans—a real clinical problem where reducing radiation exposure creates grainy images that are hard for doctors to read.

Xihao Chen, Yangyang Guo, Roger Zimmermann

You can cut vision-language model KV cache memory in half by intelligently compressing vision tokens based on what the text prompt actually needs, rather than keeping all visual information.

LightKV reduces GPU memory overhead in vision-language models by compressing the Key-Value cache during inference. It uses text prompts to guide which vision tokens are most important, keeping only 55% of tokens while maintaining performance and cutting memory use in half.

Haidong Zhao, Nikolaos Georgantas

Accurate latency prediction under GPU contention is critical for priority-aware scheduling in inference serving—Strait reduces deadline violations for high-priority tasks by modeling interference effects that traditional systems ignore.

Strait is an ML inference serving system that improves deadline satisfaction for high-priority requests by better predicting latency under GPU contention and using priority-aware scheduling.

Tianyuan Wu, Chaokun Chang, Lunxi Cao et al.

By observing OS-level effects of agent tool calls, Crab identifies that 75% of agent turns don't need checkpointing, enabling efficient fault tolerance and rollback without modifying agent code or sacrificing correctness.

Crab is a system that efficiently saves and restores the state of sandboxed environments where AI agents operate. It solves a key problem: agents need checkpoints for safety and fault tolerance, but saving everything every turn is too expensive.

Andrew Bond, Ilkin Umut Melanlioglu, Erkut Erdem et al.

Using geometrically-aligned latent spaces (hyperspheres instead of Gaussian distributions) in autoencoders preserves 3D structure and physics better than standard approaches, which matters for building world models that understand real 3D scenes.

This paper proposes S²VAE, a new type of autoencoder that uses hyperspherical (spherical geometry) latent representations instead of traditional Gaussian ones to better preserve 3D geometry and camera motion in visual world models.

Junqi Gao, Dazhi Zhang, Zhichang Guo et al.

Task vectors can be compressed to 1-5% of their original size while maintaining model performance, making it practical to store and dynamically merge multiple task-specific models without prohibitive storage costs.

This paper tackles the storage overhead problem in dynamic model merging by compressing task vectors (fine-tuned weight changes) using learnable compression techniques.

Ansar Aynetdinov, Patrick Haller, Alan Akbik

For non-English language models, aggressively filtering data for quality and repeating it multiple times beats training once on larger, diverse datasets—a practical insight for resource-constrained language model development.

This paper challenges the assumption that diverse data is always better for language model training. For German, the researchers found that repeatedly training on a smaller, high-quality filtered dataset outperforms training once on a larger, less-filtered dataset—even after 7 epochs of repetition.

Gongbo Zhang, Wen Wang, Ye Tian et al.

Cross-architecture distillation for diffusion models is now practical: you can compress large diffusion LLMs into tiny ones (13x smaller) while maintaining performance, even when teacher and student have completely different designs.

This paper introduces TIDE, a framework for distilling knowledge from large diffusion language models into much smaller ones across different architectures. Unlike previous distillation methods that work within a single model type, TIDE handles cases where teacher and student models have different designs, attention mechanisms, and tokenizers.

Shayan Hundrieser, Insung Kong, Johannes Schmidt-Hieber

HyCNNs are a more parameter-efficient way to build neural networks that must output convex functions, requiring exponentially fewer parameters than previous methods while maintaining theoretical guarantees.

This paper introduces Hyper Input Convex Neural Networks (HyCNNs), a new neural network architecture that guarantees convex outputs while using far fewer parameters than existing methods.

Wenxuan Ye, Yangyang Zhang, Xueli An et al.

Small models already generate the right answers in their candidate predictions—they just rank them poorly. Training them to re-rank their own outputs improves reasoning without external model calls.

Small language models struggle with reasoning tasks compared to large models. This paper discovers that when small models fail, the correct token from a large model is usually hidden in the small model's top-8 predictions.

Janis Mohr, Jörg Frochte

MANN combines gradient boosting with neural networks instead of trees, enabling a single framework to handle structured and unstructured data while outperforming XGBoost and reducing hyperparameter sensitivity.

This paper presents Multiple Additive Neural Networks (MANN), which replaces decision trees in gradient boosting with shallow neural networks. MANN works with both structured data and images/audio by using CNNs and capsule networks as feature extractors, and shows better accuracy than XGBoost on standard benchmarks while being more robust to hyperparameter choices.

Zihan Zhao, Baotong Lu, Shengjie Lin et al.

Sparse attention algorithms only work well in practice when paired with careful system design—SPIN shows that unifying different sparsity methods and optimizing GPU-CPU memory transfers can turn algorithmic gains into real performance improvements for long-context LLM serving.

SPIN is a system for serving large language models with long contexts by combining sparse attention (which only reads relevant parts of memory) with smart memory management across GPU and CPU. It unifies different sparse attention methods into a common framework and optimizes how data moves between fast GPU memory and slower CPU memory, achieving 1.66-5.66x faster throughput than existing systems.

Xiyuan Yang, Jiaru Zou, Rui Pan et al.

Multi-agent systems can be made faster and more efficient by having agents refine their reasoning through recursive loops in latent space rather than text-based communication, achieving 1.2-2.4× speedup with 35-76% fewer tokens.

This paper introduces RecursiveMAS, a framework that improves multi-agent AI systems by having agents collaborate through repeated refinement cycles in a shared latent space rather than exchanging text.

Ajmain Inqiad Alam, Palash Roy, Chanchal K. Roy et al.

You can compress LLMs for SE tasks to 1/49th their original size with minimal accuracy loss—making them practical to deploy while cutting environmental impact dramatically.

This paper presents Carbon-Taxed Transformers (CTT), a compression pipeline that makes large language models smaller, faster, and greener for software engineering tasks.

Dominik Żurek, Kamil Faber, Marcin Pietron et al.

Architectural parameter reuse guided by task similarity is a memory-efficient alternative to replay-based continual learning in offline RL, enabling better multi-task performance without storing historical data.

This paper presents TSN-Affinity, a method for continual offline reinforcement learning that learns multiple tasks sequentially from pre-collected datasets without forgetting previous tasks.

Clinton Enwerem, Shreya Kalyanaraman, John S. Baras et al.

Using differentiable Gaussian mixtures to represent grasp uncertainty enables fast, gradient-based optimization for worst-case robustness—achieving 10x speedup over particle filters while maintaining or improving success rates.

This paper tackles the problem of robust robotic grasping when contact forces, sensing, and external disturbances are unpredictable. Instead of using slow particle-filter approaches, the authors represent uncertainty as a learnable Gaussian mixture and optimize for worst-case performance (CVaR) using gradient-based methods.

Hailing Cheng, Daqi Sun, Xinyu Lu

Positional encodings in Transformers can be made learnable and signal-dependent by treating the rotation manifold as a separate dimension from token embeddings, unlocking better performance without significant overhead.

This paper treats the rotation space in Rotary Positional Embeddings (RoPE) as learnable rather than fixed, introducing SIREN-RoPE to encode temporal and semantic information into rotations.

Parsa Ashrafi Fashi, Utkarsh Saxena, Mehdi Rezagholizadeh et al.

You can efficiently extend pretrained LLMs to handle much longer contexts by converting them to hybrid architectures without retraining from scratch—this is more practical than building new models entirely.

This paper presents HyLo, a method to convert pretrained Transformer language models into hybrid architectures that combine Transformers with efficient linear sequence models (like Mamba2). By reusing existing model checkpoints and adding long-context training, HyLo extends context length by 32x while reducing memory use by 90%, enabling 2M-token processing on standard hardware.

Hailing Cheng, Tao Huang, Chen Zhu et al.

You can use your existing multi-GPU setup to automatically find better learning rates during training by having each GPU try slightly different rates and averaging them periodically—no extra compute needed.

This paper proposes HDET, a method that uses multiple GPU replicas to explore different learning rates during training instead of computing identical updates. Replicas train independently with different learning rates, then synchronize periodically.

Brandon Hsu, Daniel Beaglehole, Adityanarayanan Radhakrishnan et al.

Adapting steering strength dynamically per context significantly improves LLM control compared to fixed steering, matching more complex methods like LoRA while remaining simpler and more interpretable.

This paper improves linear activation steering—a technique for controlling LLM behavior—by making the steering strength adapt to each input context instead of using a fixed strength for all tokens. The method, called CLAS, works better than existing approaches across multiple benchmarks and models, offering a practical way to customize LLMs with limited training data.

Calvin Tsay

Training neural network surrogates with MILP-aware regularizers can dramatically speed up downstream optimization without sacrificing accuracy, by directly controlling structural properties that affect solver performance.

This paper shows how to train neural networks as surrogate models that work better when embedded in optimization problems. By adding special regularizers during training that target MILP tractability—penalizing large constants, unstable neurons, and LP relaxation gaps—the approach makes the resulting optimization problems solve 10,000x faster while keeping prediction accuracy competitive.