MIT AI Agent Index — transparency gaps in agent safety reporting
AI relevance: The index documents safety and disclosure practices across deployed AI agents, which directly impacts how teams evaluate tool-chain risk before rolling agents into production.
- The 2025 AI Agent Index catalogs 30 prominent agents across chat, browser, and enterprise categories.
- Only 4 of 13 frontier-autonomy agents disclose any agentic safety evaluations.
- 25/30 agents publish no internal safety results, and 23/30 report no third‑party testing.
- Browser agents cluster at Level 4–5 autonomy with limited mid‑execution intervention.
- Most agents depend on GPT/Claude/Gemini model families, concentrating systemic risk.
- 20/30 agents support MCP for tool integration, with enterprise agents leading adoption.
- The report notes a lack of standards for how agents should behave on the open web.
Why it matters
- Agent deployments are accelerating while safety disclosure lags, leaving operators to guess at real risk.
- Tool integration (MCP) and high autonomy expand the blast radius of prompt injection and tool abuse.
- Model concentration means single-vendor failures can ripple across multiple agent stacks.
What to do
- Require disclosures: ask vendors for safety evals, red‑team results, and tool governance before procurement.
- Harden operations: sandbox tool execution, restrict network egress, and monitor agent actions end‑to‑end.
- Test in‑house: run prompt‑injection and tool‑abuse scenarios against your own agent workflows.