multimodal
7 stories tagged multimodal, most recent first
Also today
Robbyant Releases LingBot-VLA 2.0: Open-Source 6B Vision-Language-Action Model for Robot Manipulation
Robbyant has open-sourced LingBot-VLA 2.0, a 6-billion parameter Vision-Language-Action model designed for cross-embodiment robot manipulation tasks. VLA models that generalize across different robot hardware configurations are a hard open problem, and a 6B open-source release with cross-embodiment support is a meaningful contribution to the robotics AI ecosystem. For developers working on robotics, embodied AI, or physical AI applications, this is a directly usable foundation model that doesn't require starting from scratch. The open-source nature means it can be fine-tuned for specific robot platforms or manipulation tasks without dependency on a closed API. Teams building manipulation pipelines should evaluate LingBot-VLA 2.0 against existing options like OpenVLA and assess its cross-embodiment generalization on their target hardware.
MarkTechPost
Meta Launches Multimodal Image Generation Model with Coding and Search Capabilities
Meta has released a new image generation model that integrates coding and search capabilities alongside visual generation, making it meaningfully more than a diffusion wrapper. This multimodal combination — generate, search, and write code in a unified model — signals Meta's push toward general-purpose multimodal agents rather than siloed image tools. For developers, this opens up workflows where image generation is part of a larger pipeline that also queries knowledge or outputs structured code. The model's positioning alongside coding capabilities suggests it may target developer productivity and AI-assisted design tooling. Availability details and API access should be checked against Meta AI's developer portal for integration planning.
Meta AI
OpenAI Releases GPT-Live and GPT-Live-1 Mini: Full-Duplex Voice Models Backed by GPT-5.5 Reasoning
OpenAI has launched GPT-Live and GPT-Live-1 mini, two full-duplex voice models designed for real-time, natural spoken conversation. The key architectural decision is that deeper reasoning tasks are delegated to GPT-5.5 underneath, meaning the voice layer stays low-latency while complex queries still get a capable backbone. This is a significant departure from bolt-on TTS/STT pipelines — developers building voice assistants, customer support bots, or real-time interfaces now have a dedicated model optimized for that modality. The mini variant presumably offers cost and latency tradeoffs for lighter use cases. Developers working on conversational AI should evaluate whether this replaces their current STT + LLM + TTS stack and check the API availability for integration.
OpenAI Blog
Ant Group Open-Sources LingBot-Vision: 1B Boundary-Centric Spatial Perception Model
Ant Group's robotics division RobbyAnt has released LingBot-Vision, a 1-billion parameter vision foundation model specifically optimized for dense spatial perception with a focus on boundary detection and object edge understanding. Unlike general-purpose vision encoders, LingBot-Vision is designed for downstream robotics and manipulation tasks where precise spatial boundaries — not just object classification — determine whether an action succeeds or fails. At 1B parameters, it is sized for deployment on edge hardware and embedded robot controllers rather than cloud inference, which is a deliberate design choice for real-world robotics applications. The open-source release makes it directly usable by robotics developers who need a compact, boundary-aware vision backbone without training from scratch. This is one of the more practically targeted open vision releases for the robotics and embodied AI community in recent months.
MarkTechPost
NVIDIA Open-Sources Audex: A 30B Audio-Text LLM Built on Nemotron
NVIDIA has released Audex (Nemotron-Labs-Audex-30B-A3B), a unified audio-text large language model that integrates audio understanding directly into a text-capable backbone without degrading its language reasoning performance. The model is a 30B parameter mixture-of-experts architecture with only 3B active parameters per forward pass, making inference more practical than the parameter count suggests. A key design goal was preserving the text intelligence of the underlying Nemotron model while adding audio modality — a common failure mode in multimodal fine-tuning that NVIDIA explicitly claims to have addressed. For developers building voice assistants, transcription pipelines, or audio-grounded reasoning applications, Audex offers a production-weight open model worth benchmarking. Its release on Hugging Face makes it immediately accessible for experimentation.
NVIDIA
OpenAI Ships GPT-Realtime-2.1 and GPT-Realtime-2.1-mini for Low-Latency Voice Agents
OpenAI has released two new models via its Realtime API: GPT-Realtime-2.1 and a smaller GPT-Realtime-2.1-mini, both targeting low-latency voice agent applications. These models are accessible now through the API, meaning developers building voice interfaces, phone bots, or multimodal agents can swap them in immediately. The mini variant is positioned for cost- and latency-sensitive deployments where full model quality is less critical than response speed. This continues OpenAI's push to make real-time speech a first-class API primitive rather than a bolted-on feature. Developers working on voice-first agents should test these against existing Whisper plus TTS pipelines to evaluate end-to-end latency and quality tradeoffs.
OpenAI Blog
Google DeepMind releases Gemini 2.0 with multimodal capabilities
Google DeepMind has unveiled Gemini 2.0, featuring enhanced multimodal understanding across text, images, and audio.
Google DeepMind