Session
Cost Management and ROI
Cost Management & Capacity Planning

When the Bill Arrives: Cost Engineering for Production AI Agents in Enterprise Analytics

Speaker

Vishvesh Pandey

Quant Analytics Senior Associate, JP Morgan Chase

Abstract

Most practitioners instrumenting AI agents in production analytics workflows start by tracking token consumption, and discover too late that token spend tells you what you spent, not whether it was worth it. This talk makes the case for a reframe that regulated industries like financial services need most: from cost-per-token to cost-per-resolved-task as the primary economic unit, with call-level attribution as the foundation that makes everything else possible.

I’ll cover four cost engineering patterns, each tested in an open, reproducible benchmark of an analytics agent: (1) instrumentation at the call level (what to log, where it lives, who owns it); (2) model tiering and routing (when a frontier model earns its premium and when a smaller model wins); (3) the output-token asymmetry that makes structured outputs and chain-of-thought hygiene more economically consequential than they appear; and (4) cost containment as an engineering practice with runtime guardrails, not a finance review after the fact.

I’ll also be honest about what doesn’t generalize: caching strategies that help one workload and quietly break another, routing logic whose overhead eats its savings, and the political dimension of asking teams to justify their agent spend. Attendees will leave with a maturity model for agent cost engineering, a concrete instrumentation checklist they can apply next week, and a vocabulary for the cost conversation that bridges engineering and finance, because in regulated industries, that’s the conversation that decides whether agents scale or stall. All examples use public data and public model pricing, with code attendees can rerun.

From the same track

Summit (2 days).

MLOps World | GenAI Summit 2026 is a two days of case studies, workshops, and expo on taking AI/ML and agentic systems into production – at the Etter-Harbin Alumni photo – full-bleed hero or browse files.

Share