Scaling Recommender Transformers to a Billion Parameters

Recommender systems help users navigate massive content catalogs using a multi-stage process that begins with an efficient "retrieval" model to select candidates. Modern systems often use a "two-tower" architecture with transformers to encode a user's entire interaction history into a single vector for matching against items. By dramatically increasing the size of these models to a billion parameters, researchers are achieving significant breakthroughs in the quality and relevance of recommendations.