OpenAI and Broadcom unveil Jalapeño, OpenAI's first custom chip

Good morning 👋 OpenAI taped out a chip. Nvidia has a new kind of customer to manage.

In today's issue:

• 🔭 OpenAI + Broadcom unveil Jalapeño, a custom LLM inference ASIC
• 🧠 Models & Releases: Jalapeño inference chip
• 🔬 Research: misalignment forensics, voice AI's listening gap, and why guardrails need to leave the runtime
• 🛠️ Tools & How-to: 3.7x training throughput from one import swap
• 🔗 Quick Links: agents at work, NVIDIA+AWS, Omio goes conversational

🔭 THE ONE THING

🌶️ OpenAI and Broadcom unveil Jalapeño, OpenAI's first custom chip

OpenAI and Broadcom unveiled Jalapeño yesterday: a custom ASIC built for LLM inference and nothing else. Nine months from blank slate to tape-out, OpenAI's own models in the design loop. Around 50% cheaper inference per token versus GPUs is the headline claim; treat it as vendor math until someone runs it independently. The structural story is harder to dismiss: OpenAI is now in the chip business, initial deployment targets end of 2026, and volume production ramps in 2027. Nvidia has a new kind of customer risk to price in.

🧠 MODELS & RELEASES

• 🌶️ OpenAI + Broadcom shipped Jalapeño *(Jun 24)* — a custom chip designed from scratch over nine months specifically for LLM inference. First OpenAI silicon in production. The perf-per-watt numbers are vendor-side for now; if they hold up independently, inference economics shift for everyone building on their API.

🔬 RESEARCH HIGHLIGHTS

Model forensics for misalignment

Concerning behavior alone doesn't establish misalignment. A model that acts badly might be confused, not scheming. This paper argues for forensics: investigating *why* an output happened, not just flagging that it did. The distinction matters for red-teamers: a capability gap is a different problem than a deceptive model, and treating them the same wastes effort on the wrong fix.

• Real-time voice AI, tested on what it misses — GPT Realtime 2, Gemini 3.1 Flash Live, Qwen3.5 Omni Plus, Qwen Omni Flash all benchmarked on tasks where vocal delivery matters. All four miss paralinguistic cues (stress, hesitation, sarcasm). They hear the transcript, not the subtext. Real gap if you're shipping voice agents.

• The unfireable safety kernel — System prompts, output filters, guardrail libraries all live inside the agent's own runtime — reachable by the agent. The paper's case: safety controls belong at the OS or infrastructure layer, outside the execution environment entirely. Right framing even if the implementations aren't there yet.

🛠️ TOOLS & HOW-TO

Fine-tune faster with NeMo AutoModel (3.7x throughput, one import change)

Jalapeño is about inference. For the training side, the gain is already here. NVIDIA's NeMo AutoModel slots in for Transformers with a single import swap and handles expert parallelism automatically. On Qwen3-30B-A3B: 3.4–3.7x throughput per GPU and 29–32% less peak memory.

1. Install: pip install nemo-toolkit[all]

2. Swap the import: from nemo.collections.llm import AutoModelForCausalLM. Nothing else in your training script changes.

3. Add to your config: strategy: fsdp2 and ep_size: 8 on an 8-GPU node. Each GPU now holds 1/8 of the expert weights.

4. Run. Forward+loss goes from 283ms to 109ms; peak memory from 62.1 GiB to 42.5 GiB. These are NVIDIA's vendor benchmarks, so run your own before planning around them.

5. For non-MoE models: skip ep_size. TransformerEngine kernels kick in automatically and still move the needle.

Prompt: from nemo.collections.llm import AutoModelForCausalLM # drop-in for transformers; fsdp2 + ep_size handles expert parallelism

Trending right now

• ⚡ NVIDIA NeMo AutoModel — One import swap gets you expert parallelism and TransformerEngine kernels across Qwen3, Nemotron, DeepSeek V3, and 20+ other model families. Hugging Face
• 🖼️ ComfyUI on SageMaker — Batch hundreds of images in a single job: CDK stack, ml.g5.xlarge GPUs, Z-Image Turbo 6B, Lambda trigger. The AWS walkthrough has the full repo and quota request steps.

🔗 QUICK LINKS

• OpenAI published research on agents in the workplace, with role-by-role productivity data that's more useful than the top-line number.
• NVIDIA + AWS announced tighter production integration covering low-latency inference and vector search, for teams managing that at scale across both.
• Omio rebuilt its travel booking on ChatGPT, going fully conversational for search and planning.

Back tomorrow.

Pradeep Perugu