“One of the most difficult problems every company faces is understanding its customers completely. Customer lifetime value, attrition risk, and purchase propensity are all solvable with AI/ML — but how do we combine these modeling scores to initiate the right action with the right customer at any point in time?

Agentic applications help us make the best possible decisions when interpreting complex, high-volume signals from our customers. An agentic application gives end users visuals that explain key insights, with an agent in the loop to ensure nothing is missed. Context is everything: when done correctly, the agent always has the appropriate understanding to build an action plan that improves customer health and profitability.

In this session, we’ll show you how to build agentic apps from ideation to a finished product that interacts with customers. You’ll take away practical tips for using agentic coding frameworks, curating complete customer data products, and building customer-facing agents — capped off with a live demo of Teradata’s Customer Lifetime Value Agentic App.”

Everyone building on frontier models hits the same wall: 80% of the way there in a weekend, then an exponentially expensive climb toward the 99%+ that operational systems need. This talk is the honest map of that climb, from a team that now runs AI over millions of documents, images and videos a month for customers in logistics, fleet management, automotive and financial services who need the answer to be right every time.

We will walk through the ladder nobody budgets for: retries, then queues when the model is down for three hours, then rate limits, then discovering that 95% is not enough for transactional data, then discovering that the model cannot tell you how confident it is. We will show two failures from our own production history, a customer whose single engineer racked up $30K of usage in a month because nothing was watching, and our first churn, on 500-page reports with a hundred rows per page, a problem we still consider unsolved. And we will show what we changed: a harness that treats AI as a deterministic step inside durable workflows rather than as an open-ended agent, decisions expressed as a verified tree the model must traverse, algorithmic confidence scoring on top of models that provide none, routing anything under 95% to a human whose verdict feeds back into the system, semantic checks against instructions smuggled into the data, and, counterintuitively, encouraging customers to build their own independent monitoring of us. One customer’s users went from eight to nine hours a week on a task to about thirty minutes, and that number was measured by them, not by us.

Attendees will leave with five patterns they can apply Monday: confidence scoring for models that lack it, decision trees over open-ended prompts, exception routing with feedback loops, semantic security checks on inbound data, and customer-owned evaluation. They will also leave with a thesis we did not start with: chat is single-player AI; the next decade of software is ambient AI that runs the same process a hundred thousand times a day, unattended, and behaves the same way every time.

This is a build-and-break lab for engineers who already know that a demo is not a system. In the first fifteen minutes every attendee stands up a working extraction pipeline against a real invoice, with no schema authoring, and gets structured JSON back. Then we spend an hour breaking it the way production does, and fixing each break with a pattern that transfers to any stack.

Break one: the wrong document. We feed a bill of lading glued to an invoice into the invoice pipeline and watch it confidently produce garbage. Fix: classify before you extract, and make the graph deterministic even though every step inside it is a model. Break two: the answer that is probably right. Models do not tell you how confident they are, so we compute per-field confidence, find the fields that fall below 95%, and route those, and only those, to a human whose correction feeds back into the system. Break three: the messy string. “10 cases organic gala apples, 88 ct” has to become one SKU; we show why canonicalization is its own step, how to score matches, and where to set the threshold for review. Break four: the hostile input. Everyone runs an image whose pixels read “ignore all previous instructions” and we discuss, with the result on screen, what it means to treat inbound data as data rather than control. We close by labeling a handful of outputs and running a regression test between two versions of the pipeline, because evaluation is a loop, not a phase.

Attendees leave with a running pipeline in their own account and five patterns they can apply on Monday regardless of vendor: classify-then-extract, confidence scoring for models that lack it, threshold-based exception routing with feedback, semantic security checks on inbound data, and versioned regression testing. The platform used in the room is ours; the lessons are not.

We built a multi-agent pipeline to automate a complex, document-intensive enterprise workflow. The initial system ran sequentially through a chain of specialized agent roles, each handling a distinct task in the process, using a single frontier model throughout. It worked. It was also slow and expensive, and we didn’t know why.

Rather than optimize blindly, we profiled first. The results were not where we expected: 67% of total latency came from a small minority of the agent roles, and the primary bottleneck was not the model — it was sequential chaining: independent work was being processed one step at a time instead of concurrently. Identifying and parallelizing the roles with no inter-dependency reduced end-to-end runtime by roughly 40%.*

The second experiment compared model tiers (frontier, mid-tier, lightweight) across each agent role independently, measuring accuracy, latency, and cost per role. The finding cuts against the default assumption: the roles that appeared most cognitively demanding required frontier models, but the accuracy gap was smaller than expected. The roles where model downgrade failed were the ones responsible for precise structured extraction — tasks where errors propagate silently downstream and surface only at the output. Swapping to mid-tier models on roles that tolerated it reduced per-run cost by ~40%, with an estimated <2% accuracy impact at the role level and <1% at the pipeline output level.*

The third experiment compared orchestration harnesses — evaluating how the choice of agentic framework affects runtime overhead, observability, and the ease of implementing the parallelization and model-swap changes described above. Framework choice turned out to matter more than expected for operational concerns: debugging multi-agent failures, tracing costs per agent, and modifying execution flow without rewriting pipeline logic.

Operating in a regulated enterprise environment added one constraint worth naming: every agent decision needs to be auditable. This ruled out certain optimization shortcuts that would have been acceptable in other contexts and shaped how we defined “accurate enough” per role.

This talk covers the profiling methodology, the per-role model comparison framework, the parallelization decisions, the harness comparison, and the operational lessons — including what we’d instrument from day one if we rebuilt the system today.

*Cost and accuracy figures for the parallelization and model-tier experiments are preliminary; the formal evaluation is in progress and will be updated with measured results before the talk.

We redesigned distributed GPU orchestration for RL post-training and batch inference in the open-source llm-d platform. Structurally, we replaced static GPU/TPU locking with a three-tier co-operative time-slicing system:

• Application Layer: Workloads signal phase boundaries (rollouts, training, batch inference) via explicit acquire() and yield() APIs.
• Cluster Orchestrator: Manages lock queues to dynamically interleave complementary jobs onto shared hardware during idle phases.
• Node Snapshot Agent: Executes fast sub-second state swaps between GPU/TPU VRAM and host DRAM, enabling instant context switching without container restarts.

Attendees will walk away with:

• Drive 70%+ GPU/TPU Utilization: Understand how time-slicing reclaims idle hardware during RL loops and batch inference without impacting convergence.
• Architect Rapid Memory Swapping: Apply VRAM-to-DRAM snapshotting strategies for ultra-fast GPU/TPU context switching.
• Deploy on Kubernetes: Configure llm-d and K8s orchestrators to interleave RL and batch inference on shared clusters.