Papers

Saman Motamed, William Harvey, Benjamin Klein et al.

Video editing can be improved by treating it as a physics simulation problem: identify what changes when an object is removed, then use diffusion models guided by causal reasoning to generate realistic results.

VOID removes objects from videos while maintaining realistic physics—like correcting how other objects move or collide after removal. It uses a vision-language model to identify affected regions and a diffusion model to generate physically plausible outcomes, trained on synthetic data where physics interactions are carefully controlled.

Chongjie Ye, Cheng Cao, Chuanyu Pan et al.

By unifying 2D and 3D generation in one model and leveraging plentiful 2D data as a structural constraint, you can train better 3D generators with limited 3D assets—no separate 2D-to-3D conversion pipeline needed.

Omni123 is a 3D foundation model that generates both 2D images and 3D objects from text by treating them as sequences of tokens. It uses abundant 2D image data as a guide to improve 3D generation, avoiding the need for scarce aligned text-image-3D datasets. The model cycles through different modalities (text→image→3D→image) to ensure consistency across all forms.

Abhilash Kar, Basisth Saha, Tanmay Sen et al.

This framework enables hospitals and clinics to collaboratively build better survival prediction models without sharing raw patient data, while also quantifying prediction confidence—critical for clinical adoption.

BVFLMSP combines Bayesian neural networks with federated learning to predict survival outcomes from sensitive multimodal data distributed across multiple parties. Each organization keeps its data private while contributing predictions to a shared model, with added privacy protections and uncertainty estimates for more reliable medical decision-making.

Karan Taneja, Anjali Singh, Ashok K. Goel

Combining conversation with visual content (multimodality) improves learning in STEM, but conversation alone can create a false sense of understanding without actual learning gains.

This study compares three ways to learn biology: a conversational AI with images and text, one with text only, and a traditional search interface. Students using the multimodal conversational system learned best and felt most satisfied, while text-only conversation felt easier but didn't improve learning—showing that engagement doesn't always mean better outcomes.

Srivaths Ranganathan, Abhishek Dharmaratnakar, Anushree Sinha et al.

Multi-agent video recommenders coordinate specialized agents for different tasks (understanding, reasoning, memory) rather than relying on single models, enabling more explainable and adaptive recommendations—a shift that's becoming practical with LLMs.

This survey examines how video recommender systems are evolving from single models to multi-agent architectures where specialized AI agents coordinate to understand videos, reason about user preferences, and provide better recommendations.

Zhe Yang, Shulin Tian, Kairui Hu et al.

Current AI agents fail at real-world personal file management: the best models only achieve 48% accuracy on user profiling tasks, with multimodal perception and evidence grounding being the main bottlenecks.

HippoCamp is a benchmark that tests AI agents on realistic file management tasks using real personal computers with 42.4 GB of actual user files. It measures how well agents can search files, understand context, and reason across multiple file types to answer questions about a user's data—revealing that even top AI models struggle with these practical tasks.

Graziano Blasilli, Marco Angelini

Multimodal AI models struggle inconsistently with detecting misleading visualizations; their ability varies dramatically by model size and architecture, and they often miss the intentional rhetorical techniques that human experts easily spot.

This study tests whether AI models can detect misleading visualizations and understand why they're deceptive. Researchers analyzed 2,336 tweets with COVID-19 charts—half containing intentional or accidental distortions—using 16 different AI models and compared their performance to how visualization experts judge the same images.

J. E. Domínguez-Vidal

Florence-2 can now be easily integrated into robot software stacks through a standardized ROS 2 wrapper, enabling local vision-language inference on consumer GPUs without cloud dependencies.

This paper presents a ROS 2 software wrapper that integrates Florence-2, a vision-language model, into robotic systems for local inference.

Prasanjit Dey, Soumyabrata Dev, Angela Meyer et al.

Hybrid physics-neural models can achieve better accuracy and uncertainty calibration than pure data-driven or physics-based approaches alone, especially for spatiotemporal forecasting with known physical constraints.

NeuroDDAF combines physics-informed modeling with neural networks to forecast air quality by integrating wind-driven transport equations, graph attention for spatial patterns, and uncertainty quantification. It outperforms existing methods on urban datasets while providing reliable confidence estimates for predictions.

Omer Dahary, Benaya Koren, Daniel Garibi et al.

You can increase diversity in generated images by applying repulsion forces in the transformer's attention channels during generation, without expensive optimization or visual artifacts.

This paper tackles the problem of text-to-image diffusion models producing visually similar outputs for the same prompt. The authors propose a method that applies 'repulsion' in the attention mechanism during image generation to encourage diverse outputs while maintaining quality and semantic accuracy.

Anuj Diwan, Eunsol Choi, David Harwath

Specialized models for different types of speech style (speaker traits vs. utterance characteristics) outperform single unified models on individual tasks, but a combined model works better when styles need to be understood together.

ParaSpeechCLAP is a dual-encoder model that learns to match speech audio with text descriptions of speaking style (like pitch, emotion, and texture). It maps both modalities into a shared embedding space, enabling applications like finding similar-sounding speech, classifying speaker characteristics, and improving text-to-speech synthesis without retraining.

Ikechukwu Uchendu, Swati Goel, Karly Hou et al.

Foundation models trained on visual reasoning can solve specialized engineering problems like chip design without fine-tuning, by framing physical constraints as spatial reasoning tasks.

This paper uses Vision-Language Models to improve chip floorplanning—arranging components on a chip to minimize wiring. The approach, called VeoPlace, treats the chip layout as a visual problem, letting a VLM suggest component placements without any training, then iteratively refines these suggestions. It outperforms existing machine learning methods by up to 32% on standard benchmarks.

Philip Schroeder, Thomas Weng, Karl Schmeckpeper et al.

Video-language models can supervise robot learning directly as reward signals if trained with spatiotemporal reasoning and grounded in continuous progress supervision, enabling robots to learn new tasks without hand-crafted rewards.

SOLE-R1 is a video-language model that watches robot videos and reasons about task progress step-by-step to provide reward signals for robot learning. Unlike standard vision-language models, it's designed to handle partial views and changing conditions, preventing robots from gaming the reward system.

Jiabin Hua, Hengyuan Xu, Aojie Li et al.

Fine-grained facial expression editing is now possible with precise control and identity preservation by disentangling expression semantics through symmetric joint training and contrastive learning.

PixelSmile is a new method for editing facial expressions in images with fine-grained control. It uses a diffusion model trained with a special technique to separate expression changes from identity, allowing smooth blending between different expressions while keeping a person's identity intact.

Geeyang Tay, Wentao Ma, Jaewon Lee et al.

Speech recognition systems hallucinate false content under degraded audio, creating safety risks for voice agents. You need diagnostic testing across real-world conditions, not just benchmark scores, to know when and where your ASR will fail.

This paper reveals that speech recognition systems fail in real-world voice agents despite high benchmark scores. The authors created WildASR, a multilingual test set from real human speech that measures robustness across environmental noise, speaker differences, and languages.

Hai X. Pham, David T. Hoffmann, Ricardo Guerrero et al.

You can teach vision-language models to understand compositional meaning by focusing on concept-level alignment and preserving fine-grained visual information—without custom data or hurting general performance.

This paper improves how vision-language models learn to understand combinations of concepts (like "red car" vs "blue car") without sacrificing their ability to recognize new objects.

Zirui Zhang, Haoyu Dong, Kexin Pei et al.

Cross-modal inconsistencies in multimodal models aren't just failures to hide—they're valuable training signals that, when enforced through cycle consistency, improve reasoning accuracy by up to 7.6 points and reduce systematic biases.

This paper introduces RC2, a reinforcement learning approach that improves multimodal AI models by enforcing consistency between visual and textual understanding. Instead of ignoring when a model gives contradictory answers for the same concept in different modalities, the method uses these conflicts as training signals.

Xinying Guo, Chenxi Jiang, Hyun Bin Kim et al.

For robotic tasks with visual ambiguity, storing rich multimodal memory with geometric grounding outperforms semantic compression—robots need fine-grained context, not just similarity-based retrieval, to handle non-Markovian decision problems.

Chameleon is a memory system for robots that handles situations where the same visual observation could mean different things depending on what happened before. Instead of storing compressed summaries like most systems, it preserves detailed geometric and visual information to disambiguate confusing situations, enabling robots to make better decisions during long, complex manipulation tasks.

Qijia He, Xunmei Liu, Hammaad Memon et al.

You can now automatically convert flat images of technical figures into editable, scalable vector graphics—matching GPT-5.2 performance—enabling recovery of lost design source files without manual reconstruction.

VFIG converts rasterized images (PNG, JPEG) of technical diagrams back into editable SVG vector graphics using vision-language models. The team created a 66K dataset of figure-SVG pairs and a two-stage training approach (supervised learning for basic shapes, then reinforcement learning for refinement) to reconstruct complex professional diagrams with high fidelity.

Keliang Li, Yansong Li, Hongze Shen et al.

Giving AI agents control over their visual perception—deciding what to look at and when—significantly improves video reasoning accuracy. This active observation approach works as a plug-and-play upgrade for existing vision-language models.

LensWalk is an AI framework that lets language models actively control how they watch videos while reasoning about them.

Ufaq Khan, Umair Nawaz, L D M S S Teja et al.

Medical VLMs need explicit training on input validation (checking modality, anatomy, orientation) as a separate safety step before diagnosis, not as an afterthought—current models hallucinate plausible reports even on obviously invalid inputs.

This paper reveals a critical blind spot in medical AI: vision-language models can generate fluent medical reports even when given invalid inputs like wrong body parts or upside-down images. MedObvious is a benchmark of 1,880 tasks testing whether models can catch these basic sanity checks before attempting diagnosis—a step human radiologists do automatically but VLMs currently fail at.

Adrian Bulat, Alberto Baldrati, Ioannis Maniadis Metaxas et al.

You can make vision-language models faster without losing visual detail by being selective about which attention layers process images—use efficient cross-attention for context and add self-attention layers only when the task complexity demands it.

VISOR improves vision-language model efficiency by selectively attending to visual information rather than compressing images. Instead of reducing visual tokens, it uses sparse cross-attention and dynamically chosen self-attention layers to process high-resolution details only when needed, reducing computation while maintaining performance on complex visual reasoning tasks.

Haoyu Huang, Jinfa Huang, Zhongwei Wan et al.

A smaller speculative model can predict an agentic system's tool-calling trajectory, enabling parallel execution and early termination of expensive operations—delivering significant speedups without accuracy loss.

SpecEyes speeds up agentic multimodal AI systems by using a lightweight model to predict what tools the main model will need, allowing expensive operations to be skipped or run in parallel. This cuts latency by 1.1-3.35x while maintaining accuracy, solving a key bottleneck in systems like OpenAI o3 that repeatedly invoke vision tools.

Haoran Yuan, Weigang Yi, Zhenyu Zhang et al.

Adding tactile (touch) sensing to video-based robot learning models significantly improves performance on tasks requiring precise force control and contact awareness, without needing separate tactile pretraining.

This paper introduces VTAM, a robot learning system that combines video and touch (tactile) sensing to better understand and perform complex physical tasks.

Yiping Chen, Jinpeng Li, Wenyu Ke et al.

This work shows how to scale vision-language models from room-sized scenes to entire cities by handling 3D spatial relationships and introducing a large, quality-controlled urban dataset—essential for building AI systems that understand real-world spatial reasoning.

3DCity-LLM extends multimodal AI models to understand entire city-scale 3D environments, not just individual objects. The system uses a three-part approach to analyze objects, their relationships, and overall scenes, trained on a new dataset of 1.2 million urban scenarios covering tasks from object identification to city planning.

Ziyi Wang, Xinshun Wang, Shuang Chen et al.

Treating motion as a continuous first-class modality rather than discretizing it enables a single model to handle motion-text-image tasks end-to-end, achieving better performance on cross-modal tasks like describing motion or editing poses from text.

UniMotion is the first unified AI system that understands and generates human motion, text, and images all in one model. Instead of converting motion into discrete tokens (which loses information), it treats motion as a continuous stream like video, using a shared language model backbone with special techniques to align motion with visual and text understanding.

Haichao Zhang, Yijiang Li, Shwai He et al.

Pairing dense video prediction models with sparse, semantically-rich vision-language reasoning improves long-horizon forecasting—VLMs provide the 'what' and 'why', while dense models provide the 'how'.

This paper combines two approaches to video prediction: dense frame-by-frame modeling (JEPA) for capturing fine-grained motion, and vision-language models (VLMs) for long-horizon semantic understanding. By using both pathways together, the system predicts future video frames better than either approach alone, especially for complex hand manipulation tasks.

Haoyu Zhen, Xiaolong Li, Yilin Zhao et al.

Structured reasoning over scene graphs helps language models understand and manipulate spatial relationships more reliably than end-to-end approaches, improving layout editing accuracy by 15-20% over baseline methods.

This paper teaches AI models to edit 3D room layouts based on text instructions by having them reason through scene graphs—structured representations of objects and their spatial relationships. Instead of directly generating new layouts, the model updates a graph representation step-by-step, which helps it maintain spatial consistency and understand how objects relate to each other.

Kelly Cui, Nikhil Prakash, Ayush Raina et al.

Vision encoders, not language models, are the primary source of spatial reasoning in VLMs. Spatial information is distributed globally across all image tokens, not just object regions, and enhancing this signal improves spatial understanding tasks.

This paper reveals how vision-language models handle spatial reasoning—understanding where objects are and how they relate to each other. The researchers found that VLMs use two mechanisms: the language model processes spatial relations independently, but the vision encoder is actually the dominant source, encoding object layouts across the entire image including background areas.

Carolin Holtermann, Minh Duc Bui, Kaitlyn Zhou et al.

Adding voice to language models doesn't just extend text capabilities—it introduces new bias mechanisms tied to speaker identity cues that amplify discrimination beyond text-only versions, requiring fairness safeguards alongside accessibility improvements.

Voice interfaces on AI chatbots amplify gender discrimination more than text-based versions because speech reveals speaker identity through tone and accent. The research shows these models shift toward gender-stereotyped responses based on voice alone, and surveys reveal users worry about hidden attribute inference.

Sashuai Zhou, Qiang Zhou, Junpeng Ma et al.

Fine-grained spatial accuracy in generated images requires explicit spatial reward modeling during training; rule-based spatial checks alone miss complex relationships that vision-language models with grounding can catch.

SpatialReward is a reward model that helps text-to-image AI systems generate images with accurate object positioning and spatial relationships. It breaks down image prompts into specific spatial requirements, uses object detection to verify positions, and applies reasoning to check complex spatial relationships—then feeds this feedback into training to improve image generation quality.

Jianan Huang, Rodolfo V. Valentim, Luca Vassio et al.

By aligning payload embeddings with text-based vulnerability descriptions using contrastive learning, you can reduce shortcut learning and improve how well cybersecurity models generalize to unseen threats.

This paper tackles a major problem in cybersecurity AI: models trained in labs fail in the real world because they learn surface-level patterns instead of genuine security concepts.

Yuning Huang, Fengqing Zhu

By selecting frames that are both relevant to the question and visually diverse, you can cut inference costs significantly while maintaining or improving accuracy on video QA tasks, especially when frame budgets are tight.

This paper tackles a key bottleneck in video understanding: processing long videos with vision-language models requires too many frames and tokens. The authors propose a smart frame selection method that picks the most important frames by balancing two goals—relevance to the question asked and diversity of visual content—using a greedy algorithm with theoretical guarantees.

Jiyu Lim, Youngwoo Yoon, Kwanghyun Park

Robots can now autonomously refine their social interactions by using VLMs to evaluate and improve their own behavior plans, eliminating the need for predefined motions or constant human guidance.

This paper presents CRISP, a framework that lets robots automatically improve their social behaviors by critiquing and replanning their own actions. Using a vision-language model as a virtual social critic, the system generates robot motions, evaluates them for social appropriateness, and iteratively refines them—all without human feedback.

Ziyin Zhang, Zihan Liao, Hang Yu et al.

You can now use smaller, faster embedding models for multilingual search and retrieval without sacrificing quality—F2LLM-v2 offers efficient options for resource-constrained deployments while the largest variant ranks first on major benchmarks.

F2LLM-v2 is a family of multilingual embedding models (80M to 14B parameters) trained on 60 million high-quality samples that support 200+ languages, including underserved low-resource ones. Using matryoshka learning and knowledge distillation, these models achieve top performance on benchmarks while being more efficient than previous LLM-based embeddings.

Dong Zhuo, Wenzhao Zheng, Sicheng Zuo et al.

A single tokenizer can efficiently represent multi-view driving scenes in a way that works for both reconstruction tasks (RGB, depth) and understanding tasks (segmentation, 3D occupancy), making it practical for vision-language-action models in autonomous vehicles.

DriveTok creates a unified tokenizer for autonomous driving that converts multi-view camera images into compact 3D scene tokens. Unlike existing tokenizers designed for single images, it handles multiple camera views efficiently while preserving semantic, geometric, and depth information—enabling better reconstruction and understanding of driving scenes.

Tianjiao Yu, Xinzhuo Li, Muntasir Wahed et al.

Part-aware 3D generation works better when you explicitly model semantic relationships between parts derived from language, not just their geometry—this enables text descriptions to guide both individual part structure and how parts fit together.

DreamPartGen generates 3D objects from text by understanding them as meaningful parts with semantic relationships. Unlike existing methods that focus only on geometry, this approach jointly models each part's shape and appearance while capturing how parts relate to each other based on the text description, resulting in more coherent and interpretable 3D models.

Shang-Jui Ray Kuo, Paola Cascante-Bonilla

State space models are a viable and more efficient alternative to vision transformers for vision-language models, challenging the assumption that transformers are necessary for this task.

This paper tests whether state space models (SSMs) can replace vision transformers as the visual backbone in vision-language models. The researchers find that SSM-based vision encoders match or outperform transformer-based encoders on VQA and visual grounding tasks, while using fewer parameters. They also identify instability issues in some backbones and propose fixes to improve robustness.

Ke-Han Lu, Szu-Wei Fu, Chao-Han Huck Yang et al.

An LLM's text-only auditory knowledge is a strong predictor of how well it will perform in audio tasks—so you can evaluate audio-language models by testing their audio understanding before building them.

This paper investigates how much knowledge about sound and audio LLMs actually have from their text-only training, and whether this predicts how well they work in audio tasks. Researchers tested different LLMs three ways: directly probing their audio knowledge, having them reason about audio descriptions, and fine-tuning them into full audio-language models.

Swagat Padhan, Lakshya Jain, Bhavya Minesh Shah et al.

Vision-language models need explicit metric reasoning to ground spatial language in 3D environments—decomposing queries into semantic and spatial components and combining them probabilistically improves grounding accuracy for robot navigation tasks.

This paper tackles the problem of robots understanding natural language commands that mix semantic meaning with precise spatial measurements, like 'go two meters right of the fridge.

Aravind Krishnan, Karolina Stańczak, Dietrich Klakow

Multimodal AI systems need safety defenses that account for attacks across all input modalities together—defending text alone or audio alone isn't enough.

This paper shows that spoken language models (which process both speech and text) can be attacked more effectively by perturbing both modalities simultaneously rather than just one. The researchers developed JAMA, a method that jointly optimizes adversarial text and audio to bypass safety guardrails, achieving 1.5x to 10x higher attack success rates than single-modality attacks.

Weilin Chen, Jiahao Rao, Wenhao Wang et al.

Reference-image-driven texturing with instance-level control produces sharper, more artifact-free 3D scene textures than text-based approaches, making it practical for professional 3D scene editing.

CustomTex generates high-quality textures for 3D indoor scenes by taking reference images and applying them to specific objects. Unlike text-based methods, it uses a dual-distillation approach to ensure textures match reference images precisely while maintaining visual quality and avoiding artifacts.

Carlos Hinojosa, Clemens Grange, Bernard Ghanem

Vision-language models' safety decisions are easily manipulated by semantic cues—they rely on learned associations rather than grounded reasoning about actual danger, which is a critical vulnerability for real-world deployment.

This paper reveals that vision-language models make safety decisions based on surface-level visual and textual cues rather than genuine understanding of dangerous situations. Researchers created a benchmark and steering framework showing that simple changes to how a scene is described or presented can flip safety judgments, exposing a vulnerability in how these models assess risk.

Mohamed Badi, Chaouki Ben Issaid, Mehdi Bennis

When building federated systems with multi-modal data, you can align different data types in a shared compressed space using learnable projections, reducing both communication overhead and the need for all devices to use identical architectures.

This paper presents CoMFed, a federated learning system that lets multiple devices train together on different types of data (like video and audio) without sharing raw information. It uses compressed representations and alignment techniques to handle the challenge of different devices having different data types and model structures, while keeping communication costs low.

Yikai Zheng, Xin Ding, Yifan Yang et al.

Decoupling semantic understanding from real-time perception—parsing queries once and matching embeddings continuously—solves the efficiency-accuracy tradeoff in proactive video understanding systems.

Em-Garde is a framework for understanding streaming video that responds to user queries efficiently. Instead of checking every frame, it converts user questions into visual proposals and matches them against the video stream using fast embedding comparisons, achieving better accuracy and speed than existing approaches.

Jianrui Zhang, Yue Yang, Rohun Tripathi et al.

You can prune half of video tokens across both vision and language components without complex mechanisms, gaining significant speed improvements (62%) while maintaining performance—making video VLMs practical for real-world deployment.

This paper introduces a method to speed up video understanding models by removing redundant visual information. The technique scores and removes 50% of unnecessary visual tokens across the entire model architecture, achieving 62% faster processing with minimal accuracy loss on video question-answering tasks.

Kevin Qu, Haozhe Qi, Mihai Dusmanu et al.

By explicitly training vision-language models to reconstruct 3D scene geometry and camera position from video, you can dramatically improve their spatial reasoning and localization abilities without changing the model architecture.

Loc3R-VLM adds 3D spatial understanding to vision-language models by training them on video input with two key objectives: reconstructing the overall scene layout and modeling the camera's viewpoint. This approach helps models better understand where things are located in 3D space and answer questions about scenes from different perspectives, outperforming existing 2D and video-based methods.

Niladri Shekhar Dutt, Zifan Shi, Paul Guerrero et al.

By tokenizing 3D shapes based on semantic importance rather than spatial detail levels, you can train autoregressive 3D generation models that are 10-1000x more token-efficient while maintaining or improving quality.

LoST is a new way to break down 3D shapes into tokens (small pieces) for AI models to process. Instead of using spatial hierarchies like existing methods, it orders tokens by semantic importance—so early tokens capture the main shape, and later tokens add fine details. This makes 3D generation models much more efficient, using 90-99% fewer tokens than previous approaches.

Mohamed Eltahir, Ali Habibullah, Yazan Alshoibi et al.

By treating video as a navigable hierarchical structure instead of converting it to text, you can process 10-hour videos with minimal accuracy loss while using compute that scales logarithmically with duration.

VideoAtlas is a system for understanding long videos efficiently by representing them as a hierarchical grid that can be zoomed into recursively, rather than converting video to text.

Junaid Ahmed Ansari, Ran Ding, Fabio Pizzati et al.

For robotics and animation applications, reconstructing cluttered scenes requires not just identifying individual 3D objects but ensuring they physically interact correctly—this work provides both a benchmark dataset and a method that achieves this.

This paper tackles 3D scene reconstruction from single images by introducing MessyKitchens, a dataset of cluttered real-world kitchen scenes with precise object shapes, poses, and contact information.

Jiongze Yu, Xiangbo Gao, Pooja Verlani et al.

Interactive video processing is now practical: users can control AI video enhancement by editing sparse keyframes, and the system intelligently propagates those edits across the full video sequence.

SparkVSR lets users interactively improve low-quality videos by editing a few keyframes, then automatically applies those improvements across the entire video. Instead of treating video enhancement as a black box, users can manually fix specific frames and the system propagates those corrections while keeping the video grounded in the original motion.

Tianyu Xie, Jinfa Huang, Yuexiao Ma et al.

Models that accurately perceive audio-visual information often fail at generating contextually appropriate conversational responses, showing that perception and interaction are separate skills that need independent evaluation.

SocialOmni is a benchmark that tests how well audio-visual AI models handle natural conversation dynamics—specifically, identifying who's speaking, knowing when to interrupt, and generating natural interruptions. Testing 12 leading models reveals that understanding what's happening in a conversation doesn't automatically translate to responding appropriately in real dialogue.

Yibin Liu, Yaxing Lyu, Daqi Gao et al.

Reinforcement learning can transform passive video understanding models into active task evaluators by training them to generate explicit reasoning about progress toward goals—enabling smaller models to outperform much larger ones on robot manipulation tasks.

This paper introduces PRIMO R1, a 7B video AI model that learns to actively evaluate robot manipulation progress by using reinforcement learning to generate step-by-step reasoning. Unlike standard models that passively recognize what's happening, PRIMO R1 compares current robot states to task goals and predicts failures, achieving better accuracy than much larger models on robotic tasks.

Pengjun Fang, Yingqing He, Yazhou Xing et al.

Using audio examples as conditioning signals instead of text prompts gives you finer control over sound synthesis and avoids the ambiguity problems that come with describing acoustic details in words.

AC-Foley generates realistic sound effects for videos by using reference audio as a guide instead of text descriptions. This solves the problem of text being too vague to describe subtle acoustic details, enabling precise control over sound timbre and quality while supporting zero-shot generation of new sounds.

Fangfu Liu, Diankun Wu, Jiawei Chi et al.

Test-time training—updating model parameters on-the-fly during inference—enables better spatial reasoning from video by letting the model continuously organize and retain 3D spatial information rather than relying on fixed context windows.

This paper introduces Spatial-TTT, a system that helps AI models understand 3D spaces from continuous video streams by adapting and updating their internal parameters during inference. It combines efficient video processing with a spatial prediction mechanism and specialized training data to maintain accurate spatial understanding over long videos.

Xuanlang Dai, Yujie Zhou, Long Xing et al.

Diffusion models can solve complex reasoning tasks better by having the language encoder think iteratively and update its guidance throughout the generation process, rather than encoding instructions once at the start.

This paper improves how diffusion models solve complex reasoning tasks by making the language model encoder think step-by-step. Instead of encoding instructions once, the system iteratively refines the model's internal reasoning and feeds it progressively to the image generation process, achieving 92% accuracy on spatial reasoning tasks like mazes and puzzles.

Ziyu Chen, Yilun Zhao, Chengye Wang et al.

Training multimodal models on scientific documents requires balancing synthetic data quality with real-world document complexity—this dataset achieves that by synthesizing faithful QA pairs then re-embedding them into full papers.

This paper introduces SciMDR, a dataset of 300K question-answer pairs across 20K scientific papers designed to train AI models on understanding complex scientific documents with both text and images. The dataset uses a two-stage process: first generating focused QA pairs with reasoning chains, then embedding them into full documents to maintain realistic complexity.

Javier Fumanal-Idocin, Mohammadreza Jamalifard, Javier Andreu-Perez

CLIP embeddings work well for general tasks, but you need domain-specific interpretation tools like fuzzy rules to understand and improve their performance on specialized text like medical or legal documents.

This paper shows how to interpret what CLIP embeddings learn in specific domains like medical records and film reviews. The researchers use fuzzy rules to map domain-specific features into CLIP's vector space, making it easier to understand which text features matter most for classification tasks in specialized fields.

Jingyang Ke, Weihan Li, Amartya Pradhan et al.

You can leverage pretrained vision-language models for specialized tasks like animal behavior analysis without fine-tuning—just guide them through explicit reasoning steps and let them work with minimal human labels.

BehaviorVLM uses vision-language models to automatically understand animal behavior and estimate body poses without requiring task-specific training or heavy manual labeling. It combines visual reasoning, temporal analysis, and semantic understanding to identify what animals are doing and where their body parts are, making behavioral neuroscience research more scalable and reproducible.

Zexuan Yan, Jiarui Jin, Yue Ma et al.

You can improve any text-to-image model's ability to render complex text and formulas without retraining—just add an agentic workflow that guides the generation process using glyph templates.

GlyphBanana solves the problem of generating accurate text and mathematical formulas in images by using an agentic workflow that guides existing text-to-image models. Instead of retraining models, it injects glyph templates into the model's internal representations to iteratively improve text rendering quality.

Yuetian Du, Yucheng Wang, Rongyu Zhang et al.

Multimodal models suffer from severe confidence miscalibration; training them to be honest about uncertainty and using that uncertainty to trigger verification steps significantly improves both accuracy and reliability.

This paper identifies that multimodal AI models are overconfident—they don't reliably know when they're wrong. The authors propose a training method using image noise pairs and confidence-based rewards to fix this, plus a test-time strategy that uses the model's confidence to decide when to double-check answers. Results show 8.8% accuracy improvements across benchmarks.

Simon Roschmann, Paul Krzakala, Sonia Mazelet et al.

You can align vision and language models with 10-100x less paired training data by leveraging unpaired images and text separately.

This paper shows how to align vision and language models using far fewer paired examples than current methods require. Instead of needing millions of image-text pairs, SOTAlign uses a small set of paired data plus lots of unpaired images and text, employing a technique called optimal transport to learn how the two models relate to each other.

Soumya Dutta, Smruthi Balaji, Sriram Ganapathy

Using specialized experts for different modalities (speech vs.

This paper presents MiSTER-E, a system that recognizes emotions in conversations by combining speech and text information. It uses separate AI experts for speech, text, and cross-modal analysis, then intelligently combines their predictions. The system works on real conversations without needing to know who's speaking, and achieves strong results on standard emotion recognition benchmarks.

Sungho Park, Jueun Kim, Wook-Shin Han

Current AI models struggle with real-world table-text reasoning; SPARTA exposes this gap with automatically-generated, complex multi-hop questions ...

SPARTA is a benchmark for testing AI models on complex questions that require reasoning across both text and tables together.

Hyungyung Lee, Hangyul Yoon, Edward Choi

AI medical diagnosis becomes more trustworthy when it shows its evidence instead of just giving answers.

This paper presents CXReasonAgent, a system that helps AI diagnose chest X-rays by combining a language model with specialized medical tools. Instead of just guessing answers like typical AI models, it shows its work by pointing to specific evidence in the image.

Yizhi Li, Xiaohan Chen, Miao Jiang et al.

Combining specialized tools with general AI models beats trying to do everything with one model—especially for long videos where context matters.

MovieTeller automatically creates summaries of full-length movies by breaking the task into stages and using face recognition to keep track of which character is which. Instead of retraining models, it combines existing tools (like face detection) with language models to generate accurate, coherent movie synopses that maintain character identity throughout.

Junhu Fu, Shuyu Liang, Wutong Li et al.

Synthetic colonoscopy videos can now be generated with enough quality and control to help with doctor training and disease diagnosis in data-scarce...

ColoDiff generates realistic colonoscopy videos using AI to help doctors train and diagnose intestinal diseases when real patient data is limited. It uses a technique called diffusion to create videos with smooth motion and precise control over medical details like disease type and imaging quality.