[rwb_1912]

Dan here. On bespoke business decisions—these are handled through SQL and model configs. This involves deciding what items to filter, how to set the objective function (what to optimize the model for), and controlling exploration and diversity in the results.

Setting the objective function is often the most challenging. Different teams may prioritize different objectives and often it requires balancing multiple at once! For instance, how does a company think about the types of user engagement and long-term metrics like retention? A model optimized for clicks might be worse for retention in some cases, but not in others. Ultimately, we A/B test to find out. Surprises and counter-intuitive results are common!