One Orchestration Layer for All: Unifying Batch ML, Real-Time Inference, and Agent Workflows in Production

Running ML training, AI inference, and agent workflows on three separate orchestration systems is an operational debt that compounds with every new workload type – one system per abstraction means three on-call rotations, three observability stacks, and three onboarding paths. We collapsed this to a single execution layer by designing a type-safe task interface that accommodates non-deterministic agent execution alongside deterministic ML pipelines, backed by container-native isolation and a workload-aware shared scheduler. The unification exposed a hard failure class that static ML infrastructure never encounters: agent tasks are not idempotent, and ML-derived retry semantics caused downstream state corruption in early production until we rebuilt retry logic around explicit checkpointing and action journals. Attendees leave with a four-layer architectural blueprint for unifying these workload types, an honest account of the scheduling contention and retry failures encountered in production, and the specific interface design decisions that made incremental migration rather than forced rewrite the path to adoption.