CTV and OTT measurement deep dive

By Larissa Klitzke
ctv ott measurement deep dive - square
ctv ott measurement deep dive - square

In part 1 of our over-the-top (OTT) blog series, we shed light on the advanced TV landscape by defining terminology, introducing AppsFlyer’s OTT measurement integrations, and describing key trends to watch in the OTT market.

For performance advertisers, defining your attribution methodology should be the first step to determine your OTT test strategy. That’s why we’re using this post to dive deep into the topic of advanced TV measurement with a technical focus on OTT and connected TV (CTV) attribution.

TV measurement overview

Advertisers have long accepted the limitations of traditional TV measurement.

Unlike digital channels that rely on URLs, pixels, postbacks and S2S/SDK integrations to provide one-to-one, deterministic measurement between ad engagements and conversions, linear TV relies on probabilistic attribution.

TV advertisers receive standard reporting on estimated impression delivery by demographic (e.g. 18-34 age group) based on TV ratings and Nielsen panel data, but if they want to understand the impact of those impressions they’ll have to invest in separate research. This is typically done via incremental lift analyses or paid brand studies to estimate the likely effect of TV on correlated metrics like awareness, acquisition and sales.

AppsFlyer uses a similar method to attribute mobile installs to TV ads via time – and location-based signals at no added cost.

Enter CTV.

While internet connectivity has the potential to provide a more accurate link between TV ads and cross-device actions in the same household, the development of true people-based attribution is still nascent. AppsFlyer is proud to offer deterministic attribution of OTT app downloads occurring on the same platform as the CTV ad, but for now, attribution from CTV ads to other devices relies on the same probabilistic methodology as traditional TV.

Some marketing cloud companies have started to offer “identity graphs” which use household- and user-based signals to provide advanced cross-device attribution from CTV, but these offerings are not yet integrated with any major MMP.

probabilistic versus deterministic TV measurement

The complete guide to OTT and connected TV marketing


Deterministic measurement: SDK/S2S integration

There are currently two reliable options for holistic attribution of OTT app downloads occurring on the same OTT platform as the CTV ad:

  • Direct integration with each platform provider for data pass-back (a manual and cumbersome process for those advertising across platforms)
  • Working with a centralized attribution partner (e.g. AppsFlyer)

Integration with AppsFlyer allows advertisers to see all their OTT installs in one place, rather than aggregating separate reports from each OTT platform provider. In addition to streamlining the analytics process, AppsFlyer also provides advanced measurement options.

Setup unlimited in-app events, configure custom attribution windows and seamlessly integrate OTT media sources to facilitate deeper insights and standardization across OTT platforms.

We currently offer the following direct integrations, with more platforms coming soon:

OTT TypePlatformSDKS2SNotes
Streaming Box/StickFire TVXXSDK requires out of store APK

SDK requires out of store APK;
S2S requires Segment integration

Gaming ConsoleXboxXXN/A
Smart TVRoku TVX In development
Android TVX Android TV includes Nvidia, Philips, Sharp, TCL, Toshiba and Skyworth TV brands

Once your OTT app is integrated with the AppsFlyer SDK, it takes just a few clicks to enable paid attribution for integrated media sources. While optional, server-to-server (S2S) integrations allow you to send AppsFlyer events back to each OTT partner and media source for optimization purposes.

Probabilistic Measurement: Lift tests & device graphs

Deterministic measurement will always be more accurate than probabilistic methods, but for some advertisers it may be worthwhile to invest in additional analytics that can give them a broader picture of the impact from CTV campaigns.

This is not only helpful for OTT app owners, but also for mobile app owners who wish to use CTV advertising to generate awareness. For the mobile performance advertisers in particular, running CTV campaigns without any idea of their impact on product consideration or app installs may seem like a waste of money.

Luckily, some media partners and marketing analytics companies are starting to offer solutions that help answer this plea. There are two main ways to do this:

  1. Conversion lift test
  2. Cross-device graph

Conversion lift test

conversion lift test

Conversion lift tests can only be executed by a DSP who is running media across multiple devices, and requires turning off other media to avoid test contamination.

Prior to launch, the DSP divides your target audience into an exposed group and a control group.

The test group sees your ads, while the control group sees placebo ads.

After the campaign ends, the advertiser analyzes the install conversion rate among between the test and exposed groups to understand the likely impact of ad exposure. This can then be used to benchmark eCPIs from CTV to mobile.

Unfortunately, these tests come at a hefty price as the advertiser must pay for ads delivered to the control audience.

A similar, potentially cheaper approach to this problem involves ghost ads.

With this method, you only pay for your own ads. On the backend, the DSP tries to randomize a select audience exposed to ads other than your own – essentially replicating the control group. Ghost ad tests are either offered as added value with a significant minimum spend (e.g. $200K), or come at a flat fee that should cost less than the placebo ad approach.

It should be noted that both of these approaches are based on correlation analysis that may be impacted by selection bias. This may lead to different results between tests, requiring iterative testing to come up with accurate predictive models.

Cross-device tests

cross-device tests

Cross-device graphs are arguably the most comprehensive approach to date for cross-device attribution from CTV.

The above graphic illustrates a scenario where the advertiser is using a data management or marketing cloud platform such as Drawbridge, Tru Optik, Samba TV, Data + Math, Adobe, or Oracle to aggregate data. IP is typically the main connecting metric for this approach, while other metrics such as device ID, user agent, time and geo are used to validate the accuracy of cross-device graph connections.

The end result is a multi-touch attribution report that estimates which app installs were driven by media across devices, i.e. CTV to mobile and vice versa.

The main drawbacks to this approach are cost and accuracy.

Involving a custom analytics partner can cost upwards of $10K/month. The alternative to this is finding a DSP or programmatic partner who can offer cross-device attribution as added value (e.g. The Trade Desk), but each partner has its own approach with its own limitations and minimum spend requirements.

Cost aside, cross-device graphs may lend more accurate results than conversion lift tests, but they still fall short of deterministic methods. Any associations from cross-device graphs should be taken with a grain of salt.

Looking forward

Overall, CTV measurement is still extremely fragmented, not unlike the wild west of mobile attribution a decade ago. However, the high growth rate of CTV will naturally lend itself to technical advancement over time.

Moreover, the synergies between OTT apps and mobile present clear opportunities for app marketers to start testing OTT television sooner than later. In our next post, we’ll share more details on our integration with Amazon Fire TV, featuring advertising opportunities and measurement best practices. 

Larissa Klitzke

As a Product Marketing Team Lead at AppsFlyer, Larissa leads strategic initiatives for sales enablement and content development in North America. Larissa brings a unique perspective to the team with seven years of agency experience in cross-platform media planning and marketing strategy. Through previous roles at M&C Saatchi and Carat, Larissa has worked closely with leading brands including Microsoft, Amazon, Sony and Red Bull.
Ready to start making good choices?