Training Generative Recommenders in Production

The ranking and recommendation systems landscape is being transformed in the generative era. This talk reports on the experience of building and shipping the training infrastructure behind the first generative recommender in production at Meta, covering both stages of the recipe: pre-training to acquire the generative capability and post-training RL to align generation with the ranking objective.

Generative recommenders bring distinct challenges to end-to-end performance and scalability. This primarily arises from a mixed architecture that consists of both recommender-native large sparse embedding tables and LLM-based decoders. This then generates semantic IDs translating to real-world use cases in production such as finding the right advertisement for a given user. We inherit the communication profile of a sparse recommender as well as the autoregressive nature of a large language model for which an end-to-end systems blueprint simply does not exist.

For pre-training, we make the significant change from discriminative to generative recommendation. We talk about feature processing, data loading and the user-modeling path while building the LLM decoder and semantic ID tokenization on top while addressing real-world productionization challenges and end-to-end LLM performance analysis.

Reinforcement learning was implemented as an extensible framework where rewards, losses, RL algorithms, reference models and generation strategies are pluggable. We’ll walk through the reward design and the end-to-end post-training pipeline that prioritizes scalability at the production scale at Meta.

We’ll close with end-to-end optimization work – redundant computation elimination, specialized kernels for very short sequences, dense-sparse pipelining and hybrid embedding placement that satisfies large-scale performance requirements in compute and throughput.

We believe the key output from this session is leaving attendees with a crisp understanding of generative recommendation systems in large-scale production. We want to ensure we stay practical and address real-world production challenges that are relevant to developers and builders in AI infrastructure today. We do this by diving into the technicalities of enabling a large-scale generative retrieval system as well as concrete performance optimizations that enable the shift from a research prototype to a highly-optimized production system critical to revenue. We also feel the training systems and infrastructure-side of LLMs in production is rarely addressed at least relative to model architecture and quality, and we hope to fill that gap with this session.