Papers

Luping Liu, Bingyi Kang, Yifan Wang et al.

By combining foundation models with diverse training data, you can build correspondence matching systems that work across both classical vision tasks and modern generative applications like image editing.

FreeMatching is a framework for matching visual correspondence across images that goes beyond traditional motion and geometry assumptions. It combines generative and semantic models trained on diverse data sources to handle challenging transformations in image editing and generation tasks, while maintaining competitive performance on standard benchmarks.

Wei Huang, Bohan Zhang, Chenzhi Liu et al.

Autoregressive pretraining on video unlocks the value of long context windows for robot control—bidirectional models don't benefit from extra history, but AR-pretrained models show consistent improvements with longer visual context.

Long-WAM is a system for controlling robots in real-time by processing long video histories (up to 19 seconds) to understand motion and task progress. The key insight is that autoregressive video pretraining—learning to predict future frames from past ones—makes longer context windows actually useful for robot control, whereas bidirectional pretraining doesn't benefit from extra history.

Zhewei Chen, Hao Zhu, Jiaojiao Jiang et al.

Using geometric properties like Ollivier-Ricci curvature to guide knowledge distillation helps MLPs capture the graph structure that GNNs learn, improving accuracy while keeping deployment simple.

This paper addresses the challenge of distilling knowledge from Graph Neural Networks (GNNs) to simpler MLPs for deployment. The authors identify two spectral failure modes—underfit on sparse graphs and overfit on dense graphs—and propose G²MLP, which uses Ollivier-Ricci curvature to guide where the student MLP should preserve the teacher's geometric structure.

Kursat Komurcu, Linas Petkevicius

Automated architecture search can discover surprisingly effective small models for Earth observation tasks—in this case, a 1.6KB network that outperforms hand-designed baselines and fits on resource-constrained devices.

This paper uses evolutionary architecture search to automatically design neural networks for predicting chlorophyll-a levels in lakes from satellite imagery. Starting with a hand-designed model, the search finds smaller, better-performing networks (26× fewer parameters) that fit on edge devices for real-time monitoring.

Shiqi Li, Sean Cho, Yijie Li et al.

Feed-forward generative models can reconstruct complex hand-object interactions faster and more reliably than optimization-based methods by learning to correct foundation model errors while respecting physical constraints during generation.

This paper presents 4D-HOF, a fast method for reconstructing 3D hand and object positions/orientations from video. Instead of slow per-video optimization, it uses a generative model trained on diverse data to refine rough estimates from vision models.

Mathias Ollu, Nikos Komodakis

Diffusing tokens at multiple granularities (fine-grained and clustered) in parallel improves continuous diffusion language models substantially, achieving state-of-the-art results on text generation and reasoning benchmarks.

This paper introduces Hierarchical Continuous Diffusion Language Models (H-CDLMs), which improve text generation by diffusing tokens at multiple semantic levels simultaneously—both individual tokens and coarser token clusters. Applied to existing models like CoBit and FLM, this approach significantly boosts generation quality and reasoning performance with minimal computational overhead.

Benhao Huang, Chufan Shi, Junlin Chen et al.

Looped models can be dramatically more efficient by recognizing that recurrent states converge to fixed points, enabling truncated backpropagation, KV cache sharing, and faster RL—with learned depth priors and orthogonal injection providing better supervision than existing approaches.

This paper optimizes looped language models—models that process information through multiple recurrent passes—by leveraging fixed points in recurrent states.

Sahil Mahendrakar

Knowledge distillation can compress speech synthesis models by 10x with minimal quality loss by separating the text-to-features and features-to-audio tasks and training them independently against a frozen teacher.

Paradee is a tiny text-to-speech model created by distilling a larger 82M-parameter teacher into just 8M parameters while keeping the same voice quality. The authors use a two-stage training approach: first synthesizing training data with the teacher model, then training separate text and audio components before combining them.

Sihan Xu, Ji Xie, Zilin Wang et al.

Dynamically predicting and updating conditioning embeddings at each generation step improves diffusion model efficiency and quality—you don't need to reuse the same embedding throughout the entire denoising process.

This paper proposes using predicted embeddings as dynamic conditioning signals in diffusion transformers instead of static embeddings. A separate transformer (NEPA) predicts image embeddings at each denoising step, allowing the conditioning to adapt to the current noise level. The approach achieves competitive image generation quality on ImageNet with significantly less training compute.

Tianjiao Yu, Xinzhuo Li, Yifan Shen et al.

Representing 3D shapes as overlapping 2D slices rather than voxels dramatically reduces computational cost while improving topological correctness—a practical win for efficient 3D generation at scale.

SILSA is a 3D generation framework that uses compact sliding-window slice representations instead of expensive voxel tokens to generate high-resolution 3D shapes. By encoding cross-sections along three axes and adding topology supervision based on persistence diagrams, it achieves better structural quality while using 70% fewer tokens and running 58% faster than competing methods.

Hui Ren, Zihan Li, Chang Liu et al.

HC-DLM bridges discrete and continuous diffusion by coupling token generation with a shared latent trajectory, enabling better reasoning and constraint satisfaction than purely discrete or continuous approaches.

This paper proposes Hierarchical Continuous Diffusion Language Models (HC-DLM), which combines discrete token generation with continuous latent states in a single denoising process. Unlike existing approaches that treat these separately, HC-DLM uses the continuous latent as the only persistent state, reading tokens from it at each step.

Zhengming Yu, Junkun Yuan, Haotian Yang et al.

By framing distribution matching as a classification problem with discriminators, DMAD eliminates the memory overhead of auxiliary models while maintaining or improving generation quality—enabling practical few-step visual generation.

DMAD improves fast image and video generation by training lightweight student models to match teacher distributions without needing an auxiliary model. It uses two discriminator heads to learn density ratios directly, making the process more efficient while achieving state-of-the-art quality in one-to-four-step generation across images and videos.

Yiming Huang, Yujie Zeng, Vijay Prakash Dwivedi et al.

HGR enables sequence models to generate chemically valid molecules with perfect validity while capturing complex molecular topology, achieving top performance on generation and property prediction tasks without the computational cost of explicit higher-order encodings.

This paper introduces Higher-order Grammar Representation (HGR), a new way to represent molecules that captures complex structural features like ring systems by converting them into sequences of grammar rules. Unlike existing methods that struggle with computational overhead, HGR makes these structures compatible with standard sequence models while guaranteeing valid molecules.

Jiahan Zhang, Chaohao Yang, Namitha Guruprasad et al.

By lifting 2D video controls into explicit 3D space, you can resolve ambiguities in object motion and create videos where camera and object movements are geometrically consistent—a major improvement over 2D trajectory-based video control.

GenCine enables artists to control video generation by editing 3D camera paths and object motions in a scene scaffold, rather than using ambiguous 2D trajectories. The system projects these 3D controls into guidance maps that a pretrained video model learns to follow, producing videos with consistent camera-relative motion and improved geometric coherence.

Yakun Zhu, Yi Bin, Yujuan Ding et al.

Separating geometry into common and residual components while routing visual learning through structured latents prevents representation collapse and improves 3D spatial reasoning from images.

This paper addresses 3D spatial reasoning from 2D images by introducing GeoLatent, which uses decomposed spatial representations (position, direction, geometry) with geometric supervision and routed optimization. The method prevents geometry collapse and ensures latents are actively used during learning, achieving state-of-the-art results on spatial reasoning benchmarks.

Yanbei Chen, Anirudh Goyal, Raghuraman Krishnamoorthi

Looped MoE models combine two orthogonal efficiency axes: recurrence increases computational depth while sparsity expands capacity, and their scaling laws enable principled design of efficient models that match much larger dense models at the same compute budget.

This paper develops scaling laws that jointly model looped transformers (which use recurrence for computational depth) and Mixture-of-Experts (which use sparsity for capacity).

Yong Xien Chng, Tianyi Chen, Wenwen Tong et al.

Looping computation within denoising steps is more efficient than scaling model size—you can get better images with fewer parameters by iteratively refining representations through repeated block execution.

This paper proposes Looped Diffusion Transformer, which improves text-to-image generation by repeatedly processing the same Transformer blocks within each denoising step rather than making models larger.

Kirill Brilliantov, Alejandro Hernández-Cano, Emmanuel Abbé

For autonomous ML engineering, a minimal harness giving an LLM direct access to read, write, and bash commands performs as well as complex multi-agent systems—the model itself, not the infrastructure, drives performance.

This paper challenges the complexity of modern ML engineering agents by comparing elaborate multi-agent systems against a simple baseline where an LLM directly accesses code execution tools. The authors find that under equal time budgets, simpler agents perform as well as complex orchestrated systems, suggesting the LLM backbone matters far more than the surrounding machinery.

Jaewoo Jung, Hyeonseo Yu, Honggyu An et al.

Training MLLMs to internally reconstruct 3D scene geometry (even in compact form) improves spatial reasoning and 3D understanding, suggesting that learning to imagine scenes is more effective than explicit geometric supervision alone.

This paper teaches multimodal AI models to reason about 3D scenes by first imagining a compact 3D representation before answering questions. Instead of relying on detailed geometric details, the model learns to assemble a coarse 3D layout from multiple viewpoints—similar to how humans understand 3D space—then uses this mental model to answer spatial reasoning questions more accurately.

Guangkuo Liu, Mert Okyay, Yifan F. Zhang et al.

Semantic information explains why generative models transition from local to nonlocal generation at nearly the same time—this connection becomes sharper as models scale, suggesting a fundamental phase transition in how semantic structure emerges.

This paper investigates why generative models show two key moments happening nearly simultaneously: when samples commit to a semantic class (speciation) and when local context becomes insufficient for generation (nonlocality). The authors prove these windows are causally linked through semantic information distribution, showing nonlocality must occur within speciation under natural conditions.

Joseph Metcalfe, Sara Sharifzadeh, Fabio Caraffini

Factorizing attention across different data dimensions improves crop segmentation, but dataset construction choices (tile size, class definitions) have outsized impact on results and must be standardized for meaningful model comparisons.

This paper introduces PAtteRNS, a transformer-convolutional model for crop segmentation in satellite imagery that separately applies self-attention to temporal, spectral, and spatial dimensions. The authors also highlight critical dataset issues—flawed class groupings and incompatible tile-size variants—that undermine fair model comparison and suggest standardization is needed.

Dor Tirosh, Ido Amos, Mor Geva

By adding recurrent feedback during pretraining via teacher-supervised state prediction, language models can improve reasoning and task performance without sacrificing training efficiency, suggesting that feed-forward architectures unnecessarily limit information flow.

This paper introduces LIFT, a transformer architecture that enables information to flow backward across layers during language model generation. Instead of the standard feed-forward design, LIFT uses teacher supervision during pretraining to train models to predict both the next token and a dense state representation derived from a teacher model.

Yu Xu, Yuxin Zhang, Xiao Yang et al.

Video models need different scaling strategies than language models—splitting experts by semantic role and allowing imbalanced routing produces better video generation than forcing uniform expert usage.

This paper proposes SplitMoE, a new way to scale video generation models using a split expert architecture that avoids forcing uniform expert usage.

Zhilin Guo, Boqiao Zhang, Hakan Aktas et al.

You can train one model that works well at any depth by supervising random layer prefixes during training—no architectural tricks needed, just a smarter training objective that makes the model elastic across compute budgets.

This paper introduces Telescopic Language Models (TLMs), which train a single model that works effectively at every layer depth rather than requiring separate models for different compute budgets.

Jinjin He, Sinan Wang, Yuchen Sun et al.

NHMO enables fast, geometry-aware PDE solving by learning a domain-specific kernel once, then reusing it for any boundary conditions or sources—eliminating expensive retraining for each new problem.

This paper presents Neural Harmonic Measure Operator (NHMO), a neural network approach that solves elliptic PDEs (like Laplace and Poisson equations) on domains with varying shapes.

Shouren Wang, Chuang Ma, Mohsen Hariri et al.

To build better looped MoE models, flatten the expert hierarchy (more experts per layer, more passes) and give each pass independent attention—this lets tokens access more experts while maintaining computational efficiency.

This paper improves looped mixture-of-experts (MoE) models by flattening the expert structure and untying attention parameters. The key insight is that by doubling experts per layer and doubling passes through the network while keeping compute fixed, models can route tokens through more diverse experts, improving performance.

Huaiyuan Qin, Muli Yang, Gabriel James Goenawan et al.

When adapting pre-trained vision models to use linear attention, directly copy MLP weights but distill attention behavior—this simple strategy closes the performance gap between efficient and standard transformers.

This paper shows how to initialize linear Vision Transformers (efficient attention models) using weights from standard Softmax ViTs. The key insight: copy the MLP layers directly since they learn general representations, but use distillation to transfer the attention mechanism since it's operator-specific.

Yuanhao Chen, Peter Chin

Audio language models don't reliably represent the same phonetic features in the same direction across speech and text—suggesting these models may process the two modalities quite differently despite using a shared decoder.

This paper investigates whether audio language models represent phonetic features consistently across speech and text inputs. Using minimal pairs of phonemes that differ in single features, researchers measured whether the same distinctive features (like voicing) are encoded in the same direction in both modalities across 6 models, 7 features, and 15 languages.

Quan Nguyen-Tri, Mukul Ranjan, Zhiqiang Shen

Diffusion LLMs can be dramatically accelerated by jointly optimizing memory I/O patterns in KV caching and using the model itself for draft-and-verify decoding, rather than treating these optimizations separately.

This paper presents Flash-dLLM, a system that speeds up diffusion language models (an alternative to traditional autoregressive LLMs) by optimizing how the model stores and reuses computation results during inference.

Kosti Koistinen, Vesa Kuikka, Joni Herttuainen et al.

By using diffusion-inspired spatial attention instead of local message passing, DISCO better captures long-range dependencies and community boundaries, making it useful for both static community detection and tracking structural anomalies in dynamic networks.

DISCO is a deep learning method for detecting overlapping communities in networks—groups where nodes belong to multiple communities simultaneously. It combines attention mechanisms with diffusion-based structural priors to overcome limitations of standard graph neural networks, and demonstrates practical value in cybersecurity by tracking how network structure changes over time.

Wangbo Yu, Kunhao Liu, Wenbo Hu et al.

By storing observations in a viewpoint-aware implicit 3D memory rather than explicit depth maps, video world models can generate longer, more consistent videos across different camera angles without running out of token budget.

WorldCrafter is a video world model that maintains consistent 3D-aware memory across different camera viewpoints and long time horizons. It uses an implicit memory system that compresses multi-view observations into tokens optimized for the requested viewpoint, enabling realistic minute-long video generation from a single image or text prompt while respecting what was seen before.

Filipe Marinho Rocha, Inês Dutra, Vítor Santos Costa et al.

Out-of-distribution generalization requires exact representational equivalence to the generating mechanism, not statistical approximation—a criterion that constrains inference rather than training and explains why neural networks fail on novel entities while logic-based systems succeed.

This paper argues that models generalize beyond their training data only when they compute representations structurally equivalent to the underlying mechanism—not approximations.

Kevin Qu, Tao Sun, Massimiliano Viola et al.

By processing multiple sparse observations together rather than single images, and training with a motion-range supervision objective, the model better understands articulation without needing large labeled datasets—it generates synthetic training data procedurally.

FAMOS is a feed-forward model that predicts how articulated objects (like doors, drawers) move and which parts are movable from multiple sparse 3D views.

Taoyong Cui, Zhongyao Wang, Xinyue Xu et al.

A single learning principle based on factorized predictions can work across radically different domains, suggesting world modeling doesn't need domain-specific architectures—and can even guide real scientific discovery.

JEPA-Anything is a unified framework for building predictive models across completely different domains—from videos to molecules to weather—using a technique called orthogonal predictive factorization. Instead of training separate models for each domain, it learns to decompose predictions into independent factors that work across vision, biology, physics, and clinical data.

Hanchu Zhou, Brendan Lynch, Raman Goyal et al.

By treating vision and tactile signals at different timescales in a lightweight architecture, you can build fast tactile-aware robot controllers (11.9ms latency) that outperform larger models on real contact-rich tasks.

This paper presents Agile-WAM, a tactile world action model that predicts future robot states and actions for contact-rich manipulation tasks.

Anton Xue, Litu Rout, Aditya Akella et al.

Hybrid architectures combining attention and RNNs are surprisingly efficient starting points for diffusion language models, reaching comparable performance with 2x faster training than full-attention models.

This paper shows that hybrid-attention language models (mixing attention and RNN layers) can be efficiently adapted into diffusion language models, which generate text in any order rather than left-to-right. The dQwen3.5 models reach the same training loss in half the tokens compared to full-attention baselines, while maintaining strong performance in parallel decoding.

Haocheng Xi, Yiming Xie, Hexu Zhao et al.

Hybrid attention architectures that combine local Softmax with linear memory can dramatically accelerate video generation without sacrificing quality—enabling practical real-time video synthesis on modern hardware.

Video DeltaNet combines local attention with efficient linear memory to speed up video diffusion models. By mixing Softmax attention for fine details with a new Video Delta Attention mechanism for long-range context, it generates high-quality videos 14.5x faster than baseline models while maintaining visual quality.

Shriman Keshri, Apparna Singh, Chinmaya Kumar Palo et al.

RISC-V offers a customizable, open alternative to proprietary chips for ML, but realizing its potential requires standardized extensions, better toolchain maturity, and specialized accelerator designs tailored to neural network workloads.

This survey examines how RISC-V, an open-source processor architecture, is being adapted for machine learning workloads. It analyzes existing implementations, software tools, and real-world applications, identifying strengths in energy efficiency and instruction extensions while highlighting challenges like fragmentation and standardization gaps.

Sara Pieri, Evangelos Kazakos, Shizhe Chen et al.

Combining dense captioning with pixel-level grounding requires jointly optimizing text generation and mask selection—PANORAMA shows this can be done effectively by conditioning a segmenter on phrase representations and learning which masks correspond to each phrase.

This paper introduces PANORAMA, a vision-language model that generates detailed image captions while simultaneously grounding each phrase with pixel-level segmentation masks.

João Meneses dos Santos, Arlindo L. Oliveira

Self-reflection (validating and correcting actions at runtime) is more important than memory for making language agents reliable in interactive environments, but combining both works best.

This paper improves language agents for interactive tasks by adding two cognitive modules to SwiftSage: a memory system that stores and retrieves important experiences, and a self-reflection system that validates actions and fixes mistakes. Testing on ScienceWorld shows the full system performs best, with self-reflection being the most critical component for handling long-horizon tasks.

Zixi Chen, Akshay Vegesna, Samip Dahal et al.

Architectural design choices like recursive depth and boundary operators can fundamentally change scaling laws, enabling exponential compute efficiency improvements—challenging the idea that scaling laws are fixed.

This paper shows that architectural changes—specifically model growth through recursive depth (looping) and boundary operators—can improve how efficiently transformers use compute during training. A 7.4B looping model matches GPT-3 13B's performance with 20× less computation, with efficiency gains that grow larger at bigger scales.

Thanapat Trachu, Samuele Cornell, William Chen et al.

Layer-specific compression in neural codecs reduces computational overhead by letting different quantization layers compress at their own optimal rates, improving efficiency without sacrificing quality.

This paper introduces LACE, a neural audio codec that compresses speech at different rates for each quantization layer rather than using a single compression step. By allowing each layer to have its own segmentation boundaries, LACE reduces sequence length and computational cost while maintaining audio quality.

Siting Li, Zhengyang Wang, Simon Shaolei Du et al.

When building multimodal models, image tokenizer choice matters not just for image quality but for how well it helps the model learn text and vision together—and the best tokenizer for one task may not be best for another.

This paper studies how image tokenizers work in multimodal AI models by building a controlled training setup that tracks how well different tokenizers perform across text, image generation, and image understanding tasks.

Yankai Fu, Ning Chen, Junkai Zhao et al.

Adaptive tactile fusion and joint visual-tactile prediction enable VLA models to handle contact-rich manipulation where vision alone fails, achieving 71% success on dexterous tasks.

DeCAL is a vision-language-action model for robot hands that combines visual and tactile (touch) sensing to perform complex manipulation tasks. It uses specialized AI experts working together to understand scenes, imagine future states, and generate actions—all while explicitly modeling contact dynamics that cause visual occlusions during dexterous manipulation.

Samuel Kushnir, Kimia Noorbakhsh, Kavya Sreedhar et al.

Design documents with worked examples can be more maintainable than code for ML performance tools, since AI agents can reliably regenerate implementations from them, reducing the cost of keeping performance models up-to-date with new hardware and models.

SMART is a machine-learning performance modeling tool that replaces traditional code with natural-language design documents. AI agents regenerate the entire implementation from these docs on each update, eliminating tech debt while maintaining accuracy—validated against real systems like DeepSeek-V3 on TPUs.

Ethan Ma, Zihan Wang, Chris Siu Yeung Chow et al.

Learning continuous embeddings for graph categories instead of using fixed one-hot vectors improves molecular graph generation quality, as shown by EGF's superior performance on standard benchmarks.

This paper introduces Embedded Graph Flows (EGF), a generative model that creates realistic molecular graphs by learning continuous embeddings for node and edge types instead of using fixed one-hot vectors. The model uses a permutation-equivariant transformer to gradually transform noise into valid graph structures, achieving state-of-the-art results on molecular benchmarks like QM9 and ZINC250k.

Yoonju Sim, Federico Berto, Chuanbo Hua et al.

LLMs can discover domain-specific algorithm improvements by searching over heuristic families rather than predicting solutions directly—here achieving 71% win rate on quantum circuit optimization by aligning variable ordering with actual quantum cost rather than proxy metrics.

This paper uses large language models to discover better algorithms for ordering variables in quantum circuit design. The key challenge is that quantum circuits implementing Boolean functions need optimal variable orderings to minimize quantum cost, but existing heuristics optimize for the wrong metric.

Pengxiang Zhao, Xing Li, Xianzhi Yu et al.

Multi-stream architectures like mHC provide more capacity than models actually use—most layers rely on just two streams, and late-layer stream mixing can be removed with minimal performance loss, suggesting opportunities for efficiency improvements.

This paper investigates how DeepSeek-V4-Flash uses its multi-stream residual pathways (mHC architecture). Researchers found that despite having four parallel streams, the model typically uses only two streams per layer, with minimal mixing between streams in later layers. Interventions show that early-layer mixing is critical for performance, while late-layer mixing is largely redundant.

Yujie Zhang, Huiying Lan, Ehsan Aghapour et al.

For edge AI deployment, combining hierarchical operator parallelism with pipelining can achieve better energy efficiency and latency than using either strategy alone—Para-Pipe shows 11-23% energy improvements on real SoCs.

Para-Pipe is a framework that optimizes deep learning inference on edge devices (SoCs) by combining pipelining and parallel execution of neural network operations.

Luyi Xing, Rasit Onur Topaloglu, Ranjan Sinha et al.

NLIP is a practical interoperability standard for AI agents that lets you build agents independently and have them work together seamlessly, similar to how different web services communicate via HTTP.

This paper introduces NLIP, a standardized communication protocol that lets AI agents built with different frameworks and tools talk to each other. Like HTTP for web servers, NLIP provides a common language for agents to exchange messages, coordinate work, and access shared tools—enabling organizations to mix-and-match agent systems without rebuilding everything from scratch.

Sergii Kozyrev, Davyd Maiboroda

Recurrent attention layers like Gated DeltaNet are actually easier to quantize than softmax attention because their state-update mechanism naturally forgets noise; you can safely quantize everything to 4-bit if you use block scaling and calibrate per-module.

This paper shows that Gated DeltaNet layers in hybrid LLMs can be quantized to 4-bit precision without quality loss, contrary to conventional wisdom. The authors quantize all 496 linear layers of Qwen 3.8-27B to NVFP4 W4A4 and demonstrate it matches full precision across benchmarks while being 17.5GB smaller and 14-19% faster.

Uğur Aydın, Tamer Başar

Degenerate diffusion models—where the noise schedule breaks down—can still be guided effectively using causal optimal transport, enabling diffusion-based generation in settings where standard score-based methods fail.

This paper addresses a fundamental problem in diffusion models: how to guide generation when the model has a singular diffusion coefficient (degenerate case) and the underlying data distributions are irregular or non-smooth. The authors use causal optimal transport theory to define loss functions that enable effective guidance with minimal assumptions about the data.

Lintai Hou

You can build efficient language models by replacing dense attention with sparse routing through learned edges, maintaining full state size while accessing only a tiny fraction of tokens per layer.

Graph Machine introduces a sparse neural architecture that maintains linear-sized state while using dynamic routing through differentiable edges. By replacing 75% of dense Transformer layers with sparse GM layers in a 0.6B model, the approach achieves comparable or better performance while accessing only 2-4 tokens per attention head, reducing computational cost.

Nan Zheng, Hoi Yiu Cheung, Vibhu Sharma et al.

TMB eliminates manual derivations for neural network mixed-effects models by automating gradient computation and random effect integration, making these powerful models easier to implement correctly and at scale.

This paper shows how to build neural network mixed-effects models (NMMs) using Template Model Builder (TMB), a tool that automatically handles the complex math needed to fit these models.

Mosharof Hossain, Md Rabiul Islam, Limon Halder et al.

Prompts can be integrated deep into image segmentation networks through channel-wise attention, rather than just at the end, making models better at finding anatomical structures across different medical imaging modalities.

This paper introduces PCCA, a mechanism that uses text prompts to guide how neural networks process medical images at multiple levels, improving segmentation accuracy across different body parts and imaging types. The method adaptively adjusts which image features matter most based on the prompt, achieving 10-23% improvements over standard approaches.

Christos Koutsiaris

Designing model architecture around deployment constraints (CPU, 4-bit weights, fixed memory) from the start beats the conventional approach of building large then compressing—hybrid convolution-attention architectures can match attention-only models while being significantly faster at inference.

This paper presents Daedalus-150M, a small language model designed specifically for CPU inference by combining convolutions and attention strategically. Rather than shrinking a large model, the authors built from scratch with CPU constraints in mind, using full attention in only 6 of 18 blocks while the rest use short convolutions with fixed memory.

Wei Lin, Tao Zhou, Zhaofei Xie et al.

Future software will be built from three components—storage, models, and agents—replacing traditional layered architecture. This shift means developers must rethink how they structure applications, with AI models handling logic and interfaces rather than hand-coded layers.

This paper argues that software architecture is undergoing a third major shift—from instruction-based (Software 1.0) and data-driven (Software 2.0) systems to context-and-reasoning-driven systems (Software 3.0).

Yuting Ge, Pengju Yang, Mingkai Nie

Organizing token interactions into explicit Self and Exchange relations before computing attention can outperform standard attention while enabling faster implementations—suggesting relation structure matters more than raw attention scores.

This paper proposes Relation, a new token-mixing approach that organizes pairwise token interactions into explicit Self and Exchange relations before computing information flow, as an alternative to standard attention. The method shows better language modeling performance than attention across multiple model sizes and offers faster implementations.

Joonas Järve, Halil Ibrahim Aysel, Tarun Khajuria et al.

Features in Vision Transformers don't stay in one layer—they migrate between layers during training, especially early on, revealing a previously invisible dimension of how these models learn.

This paper tracks how individual features move and change across Vision Transformer layers during training using Sparse Autoencoders. By visualizing features over both network depth and training time, researchers discovered that features migrate between layers early in training, with movement favoring earlier layers, and that deeper layers stabilize faster than shallow ones.

Sotirios P. Chatzis, Loukas Papadoulas

Lévy Attention replaces softmax with a probabilistic formulation that automatically outputs calibrated uncertainty estimates alongside predictions—no extra parameters or passes needed, making it practical for high-stakes applications like patient risk ranking.

This paper introduces Lévy Attention, a new attention mechanism for time series that predicts both values and uncertainty in a single pass. Instead of using softmax, it formulates attention as a stochastic integral over a Poisson random measure, which naturally captures prediction confidence through two signals: disagreement (value spread) and evidence (compatibility mass).

Tate Berenbaum, Muthaiah Venkatachalam

Pipeline parallelism with pre-compiled shards lets you run 70B models on clusters of consumer AI PCs at interactive speeds by splitting computation across machines and optimizing GPU utilization through careful kernel fusion and speculative decoding.

This paper shows how to run large language models (like Llama 70B) across multiple Intel AI PCs by splitting the model into layers and running each layer on a different machine. Using OpenVINO optimization, speculative decoding, and request interleaving, they achieve near-optimal speed while serving multiple users simultaneously on modest hardware.

Reza Bayat, Ali Behrouz, Vahab Mirrokni et al.

Gradually expanding memory capacity during sequence processing—rather than using static memory—is a simple, cost-free improvement that works across different memory-based architectures and shows larger gains on longer contexts.

This paper introduces Proteus, a technique that improves how neural networks handle long text by gradually expanding memory capacity as more context arrives.

Hanfeng Lu, Tianyu Feng, Suyi Li et al.

By overlapping independent computation phases and sharing GPU memory intelligently, you can train vision-language models 1.2–2.2× faster without needing more hardware or changing your RL algorithm.

Rollplex is a GPU runtime that speeds up vision-language model training by overlapping different computational phases.

Hao Yan, Lisa Pilgram, Dan Liu et al.

You can now generate realistic synthetic tabular health data from multiple heterogeneous sources by standardizing them statistically first, then using diffusion transformers to learn and reproduce their patterns.

This paper presents a method for generating synthetic health data from multiple different database tables with varying structures. It works in two stages: first converting diverse tables into a standardized statistical format, then using a diffusion transformer to learn patterns and generate new synthetic tables.

Abhishek Shukla, Ankur Sinha, Faiz Hamid

By formulating architecture search as a linear program using gradient and curvature information, LP-NAS finds better neural network designs 2-3x faster than standard differentiable NAS methods while achieving higher accuracy.

This paper proposes LP-NAS, a neural architecture search method that uses linear programming to find better network designs faster. Instead of randomly exploring architectures, LP-NAS uses mathematical optimization principles to guide the search, resulting in architectures that generalize better and are found more quickly than existing methods like DARTS.

David Chushig-Muzo, María Ángeles Rodríguez de Cara, Eva Milara et al.

Self-organizing maps can encode tabular data more effectively than simpler dimensionality reduction by capturing feature relationships alongside values, improving both predictive performance and model interpretability.

TabSOM converts tabular data into images using self-organizing maps, preserving both feature values and relationships between features. Unlike existing methods that only encode individual feature values, TabSOM captures feature interactions as spatial patterns, enabling vision models to achieve better performance while remaining interpretable.

Sabin Roman, Ljupco Todorovski, Saso Dzeroski

SORT shifts equation discovery from brittle library selection to basis design: by learning sparse coefficients in well-chosen orthogonal bases, it provides a more stable intermediate representation that gracefully degrades under noise and sampling sparsity.

SORT is a machine learning technique that learns compact mathematical representations of dynamical systems from noisy, irregularly sampled data by fitting sparse coefficients in orthogonal basis expansions.

Daniel Perkins, John Squires, Janou Milligan et al.

Instead of training separate models for each domain, you can use an MLLM router to dynamically select which vision backbone handles each image, getting both better generalization and easier updates without retraining.

ARMDIL uses a multimodal language model to intelligently route images to the best-suited vision model (CNNs, self-supervised learners, or vision-language models) within an ensemble.

Van Khoa Nguyen, Alexandros Kalousis

By reversing the physics concept of spontaneous symmetry breaking, this diffusion model generates complete crystal structures with proper global symmetries—a significant improvement over methods that only generate partial specifications.

This paper proposes a new method for generating crystal structures by using a diffusion-based model that starts from low-symmetry configurations and gradually breaks symmetries to create complete crystal specifications.

Andy Yang, Blerta Veseli, Corentin Barloy et al.

Transformers' ability to generalize to longer sequences depends on specific algebraic properties of the language being learned—properties that classical finite algebra misses but can be captured by extending decomposition theory to infinite groups.

This paper characterizes which regular languages transformers can generalize to longer sequences than they've seen during training. The authors develop new algebraic theory extending classical decomposition methods to handle transformers' unbounded counting abilities, providing a polynomial-time algorithm to predict length generalization on any regular language.

Aimilios Hadjiliasi, Louis Nisiotis

Small language models can efficiently run agent cognition (thinking and memory) on edge devices like Jetson boards, enabling virtual agents to operate independently in real-time without cloud latency.

This paper explores how small language models (SLMs) running on edge devices can power the cognitive processes of virtual agents in immersive worlds.

Paul Savala

Predicting course enrollment alongside grades significantly improves academic performance forecasting—jointly modeling what students take and how they'll perform is more accurate than predicting grades alone.

This paper presents TRACE, a transformer-based model that jointly predicts which courses students will take and their grades in those courses for upcoming semesters. Unlike traditional approaches that treat student history as a simple sequence, TRACE captures how courses taken concurrently within a semester affect performance, reducing prediction error by nearly 50% compared to grade-only models.

Chuyue Li, Jinpeng Yu, Haozhe Wang et al.

Agents can now learn from and generate high-quality videos by working with structured knowledge graph representations instead of raw pixels, improving video generation quality by 20.7% over existing methods.

AVA-Encoder learns video representations as knowledge graphs that agents can reason about and edit. It converts videos into structured text and asset layers, then reconstructs videos from these representations. A natural-language feedback loop optimizes the encoding, enabling agents to work with cinematic-quality videos more effectively.

Rafi Ibn Sultan, Chengyin Li, Yiannos Demetriou et al.

Combining neighborhood attention mechanisms with uncertainty-weighted loss functions enables accurate segmentation of tiny, low-contrast anatomical structures even with limited training data—a practical approach for clinical imaging tasks where manual annotation is expensive and ambiguous.

This paper presents NA-UNETR, a transformer-based deep learning model for segmenting the Left Anterior Descending artery in CT scans.

Eric A. F. Reinhardt, Adam J. Hauser

Softmax attention has an exact quantum mechanical interpretation where attention scores become measurement statistics and the softmax function emerges naturally from quantum probability rules, opening a path to quantum implementations of transformers.

This paper shows how softmax attention in transformers can be exactly implemented using quantum computing. The researchers map each component of attention—from computing scores to aggregating values—to quantum operations like measurements and rotations. They prove this works mathematically and verify the core logic in formal proof software.

Davide Rinaldi, Luciano Serafini

If you're building neurosymbolic systems that need to reason about ordered or connected data, sLTN lets you write logical rules about structure (like "event A happens before event B") and train them end-to-end with neural networks.

sLTN extends Logic Tensor Networks to handle structured data like sequences and graphs by treating structural dimensions (time steps, positions, nodes) as first-class elements in the logical language. This lets you express temporal and relational constraints directly in logic while keeping everything differentiable for neural learning.

Alban Puech, Matteo Mazzonelli, Tamara R. Govindasamy et al.

Neural networks can solve critical power grid problems 30-85x faster than traditional solvers while staying physically consistent—opening the door to AI-powered grid analysis at scale.

GENCO is a neural network solver that handles power flow, optimal power flow, and state estimation for electrical grids in a single unified model. It matches or beats classical solvers like Newton-Raphson and IPOPT in speed and accuracy while maintaining physical consistency, and comes with an open-source framework and large datasets for reproducible research.

Abraham Gonzalez, Raghav Gupta, Akanksha Jain et al.

Agentic AI can discover better microarchitecture designs than humans by breaking large search spaces into manageable pieces and embedding hardware constraints directly into the optimization loop.

ArchAgent v2 automatically designs data prefetching policies for computer processors using evolutionary search, beating hand-designed solutions in a competition. It introduces cascaded evolution (optimizing cache levels sequentially) and hardware-realizability feedback to handle the massive design space, achieving 3.8% performance improvement over baseline.

Sam Levang, Fran Bartolic, Ty Dickinson et al.

You can now configure forecast timesteps after training instead of before, letting one model handle both detailed short-range and stable long-range weather predictions without retraining.

GEM-3 is a weather forecasting model that solves a key problem: existing models must choose between short timesteps (detailed but error-prone) or long timesteps (stable but missing short-term details). This model lets you pick the timestep at inference time with a single trained model, and training on mixed timesteps makes forecasts more stable across longer periods.

Elad Yoshai, Natan T. Shaked

Per-feature gating based on distribution distance enables better control in unpaired image translation, letting you preserve important structures while achieving realistic style changes without retraining.

PRISM is a new method for changing images from one style to another (like turning day photos into night) without paired training examples. Instead of using a single global control value, it learns a per-feature gate based on how far each image feature is from the target style, allowing precise control over what changes and what stays the same.