Is the Money Spent on Short-Form Video Social Platforms Worth It? The Role of Advertising Spillover in a Large-Scale Randomized Field Experiment on ByteDance.

This article investigates the role of advertising spillover in the effectiveness of short-form video advertising campaigns on ByteDance, a leading Chinese internet company operating platforms such as Douyin (Chinese TikTok) and Toutiao. Through a large-scale randomized field experiment involving over 84 million users and an automobile brand, the study finds that most advertising conversions occur outside ByteDance, with exposed users being eight times more likely to convert off-platform than on-platform. Ignoring this spillover leads to significant underestimation of advertising effectiveness and misinformed economic evaluations, such as inflated cost per conversion metrics. Additionally, commonly used demographic targeting variables lose predictive power when spillover is considered, whereas a behavioral variable—prior visits to the brand's Douyin home page—effectively moderates advertising impact only when off-platform conversions are included. The findings highlight the critical need for information sharing between platforms and advertisers to accurately assess advertising outcomes and optimize targeting strategies.