Led by Altos Ventures, with 645 Ventures and Menlo Ventures participating, the round funds an empirical security platform pairing adversarial simulation with production defenses.
Claude jailbroken to issue a $50,000 Stripe coupon
General AnalysisWe reveal a powerful metadata-spoofing attack that exploits Claude's iMessage integration to mint unlimited Stripe coupons or invoke any MCP tool with arbitrary parameters, without alerting the user.
Supabase MCP can leak your private SQL tables
General AnalysisIndirect prompt injection through support-ticket data lets an MCP-enabled assistant cross trust boundaries and query private tables with Supabase’s privileged service role.
The Redact & Recover jailbreak
General AnalysisWe present the Redact & Recover (RnR) Jailbreak, a novel attack that exploits partial compliance behaviors in frontier LLMs to bypass safety guardrails through a two-phase decomposition strategy.
Policy moderation for Indeed job postings
General AnalysisOur compact policy moderation models achieve human-level performance at <1% per-review cost, outperforming GPT-4o and o4‑mini on F1 while running faster and cheaper.
Adversarial analysis of Llama 4 models
General AnalysisA head-to-head robustness evaluation of Llama 4 (Maverick, Scout) versus GPT‑4.1, GPT‑4o, Sonnet 3.7, etc. using TAP‑R, Crescendo, and Redact‑and‑Recover across HarmBench and AdvBench.
The collaboration applies General Analysis red-teaming methods across Together AI’s model ecosystem and releases supporting tools, benchmarks, and an interactive TAP notebook.
The Jailbreak Cookbook
General AnalysisA field guide to manual and automated LLM jailbreaks, with a practical taxonomy, runnable implementations, and cross-model benchmarks for influential attack methods.
Diversity transfer using LegalBench
General AnalysisSeeding generation with LegalBench produced a broader set of realistic adversarial legal questions than zero-shot prompting, offering a practical way to improve benchmark coverage.
Uncovering hallucinations in legal AI models
General AnalysisIn this work we explore automated red teaming, applied to GPT-4o in the legal domain. Using a Llama3 8B model as an attacker, we generate more than 50,000 adversarial questions that cause GPT-4o to hallucinate responses in over 35% of cases.
