The Future of User Journey Tracking

by János Moldvay
The Future of User Journey Tracking

User journey tracking and hence user level based marketing measurement has become increasingly difficult in recent years. This is due to significant regulatory changes, new standards set by industry players and new tools, which are related to the handling and availability of personal data. This post discusses the most important implications of these developments, provides solutions and an outlook on how the future of marketing measurement evolves given these circumstances.

Ad Blocking

Ad and tracking blockers such as AdBlock and Ghostery create a blind spot from a user journey tracking perspective through blocking the execution of some javascript code that enables pixel based user tracking. There are also some more advanced commercial solutions, such as Nextdns and AdGuard, who not only work as a simple browser plugin. The percentage of users with an ad blocking tool ranges between 25% and 30% and is growing slowly. Not all javascript code is automatically blocked, some Ad Blockers block e.g. Google Analytics and tags implemented through Google Tag Manager, and some don’t.

There are different implications from ad / tracking blocking on marketing measurement, for example:

  • - Blocked ads won’t be viewed, viewing isn’t trackable and hence no credit will be given, which makes sense from a measurement perspective.
  • - Undervaluing marketing campaigns because an ad has been viewed or clicked on but it won’t be considered in the user journey since the interaction can’t be linked to a conversion event potentially happening later in the journey. These conversions would most likely appear as conversions coming through direct visits.

The overall impact of ad blocking on marketing measurement has always been a relevant factor and marketing managers are aware of the implications. They know that conversions attributed to direct visits are inflated because of blocked user journey tracking. It can be assumed that actual, albeit untrackable, user journeys of ad blocking users are similar to user journeys of non-ad blocking users. Therefore one possible and simple solution could be to distribute some of the conversions attributed to direct proportionally to the other marketing channels. For example if we know, that about 25%-30% of customers use ad blocking, then one could infer that the same ratio of conversions attributed to direct should actually be attributed to other marketing channels (proportionally to their respective attribution share amongst the non-direct marketing channels).

GDPR

The implementation of GDPR in May 2018 had and still has a huge impact on the digital marketing industry and completely changed the way all players in the digital advertising industry, but especially adtech providers such as Criteo and Tradedesk, were able to operate.

Amongst one of the many implications of GDPR, is that visitors of websites were to be asked for consent to be tracked by website analytics and user journey tracking tools. Implementation of so-called CMPs came into being and the majority of users were starting to get annoyed by the cookie and privacy settings they had to do on every new website they visited. Some experts have voiced concerns about CMPs not being GDPR compliant, but for now, CMPs seem to be the standard way to go for websites to be GDPR compliant . CMPs can be thought of as more sophisticated tag management systems where users can choose which tag is allowed to track them, and tracking and analytics tags need to be integrated through a CMP.

The impact of this consent management on user journey tracking can be significant. Since tracking can only start once a user has given her consent, much of the percentage of trackable visits depends on how the consent is being asked for. If the consent banner is displayed very prominently on the middle of the website, the ratio of users giving consent will be higher compared to the banner being shown at the bottom or right side of the site. Also placement within the CMP’s categories plays a crucial role when integrating a tracking tag into a CMP: If it’s placed under “Essential” consent rates will be higher than if it’s placed under category “Marketing”.

Example of a website with a less prominent cookie banner
Example of a website with a less prominent cookie banner.
Example of a website with prominent cookie banner
Example of a website with prominent cookie banner.

The IAB released their “Transparency and Consent” framework with the aim to create an GDPR compliant industry standard for all parties in the digital advertising space when handling personal data. With version 2.0 even the digital advertising behemoth, Google, joined the framework. Google’s participation somewhat confirmed the IABs TCFs eligibility and legitimacy. GDPR compliant CMPs implement the most up to date version of the TCF.

Preventing Third-Party Cookie Tracking

The second and third most used browsers, Safari and Firefox, started blocking third-party cookie tracking a while ago. Safari started doing so in March’s 2020 update of Safari and Firefox in September 2019.

Google announced that Chrome will phase out third-party cookie tracking within two years and introduced the concept of a “privacy sandbox” to still enable advertisers to track users across different sites.

To understand the implications of the phasing out of third-party cookies one must understand the difference between third-party and first-party cookies:

  • - First-party cookies are created and stored by the host domain (website) a user is visiting. They are an essential part of the web’s infrastructure and enable website operators to collect analytics data, remember language setting, keep sessions open, remember which items were in the shopping cart etc.. First party cookies are considered “good” and they help provide a much better users experience. First-party cookies are only accessible via the domain that created it.
  • - Third-party cookies are not created by the domain a user visits but by some other domain, a so-called third-party. They are essential for many digital advertising purposes, e.g. targeted ad serving and retargeting users across different publishers’ websites. Third-party cookies are accessible on any website that has loaded code from the third-party domain. A typical (albeit simplified) scenario would be a user visiting a webshop and adding a shoe to her shopping cart without buying the shoe. A retargeting service (the third-party in this case) could set a third-party cookie within the user’s browser. Once the user visits a publishers site, with which the retargeting service collaborates, the publisher website can load the retargeting service’s code, read the users cookie and display them respective ads, e.g. for shoes.

Adtriba’s tracking on an advertisers website is implemented as a first-party cookie tracking and therefore not impacted by third-party cookie blocking. But Adtriba’s display view tracking as part of the holistic user journey tracking is based on a third-party cookie and therefore fully impacted by third-party cookie blocking in Safari and Firefox. When a display ad of one of the advertisers Adtriba works for is being displayed on publisher’s website, Adtriba’s display tracking code, integrated in the display ad, set’s the third-party cookie, without previously even knowing the publisher. In case the user later visits the advertiser’s website (without clicking on the ad) through another channel (e.g. SEA, SEO or Direct), the Adtriba’s tracking code on the advertiser’s website will sync the first-party cookie with the third-party cookie and thereby establishing the previous display view as part of the user journey.

Through blocking third-party cookies the display view (not a potential click!) won’t be trackable and hence not considered as part of the user journey. This potentially leads to undervaluing the marketing effectiveness and performance of the respective display ad campaigns and channels.

ITP (Intelligent Tracking Prevention)

ITP was added to Safari in 2017 to restrict how companies can track Safari users across the web (Mozilla followed suit quite closely with ETP for Firefox). It involves some sort of machine learning, that’s why the name includes the term “intelligent”. The first release of ITP was aimed at restricting third-party cookie tracking, but since companies such as Google and Facebook switched to a specific use of first-party cookies as a work around to mimic the functionalities of third-party cookies, ITP also included restrictions on first-party cookies. This opened up a kind of cat-and-mouse game between Apple/Safari and ad platforms that still tried to track and ad-target users as much as possible. We won’t go into the details here, but there are some good articles that describe the specifics. Generally speaking there is an obvious trend of Safari heavily restricting how companies are able to track users across different websites.

With ITP 2.2 the lifespan of first-party cookies was cut from seven days (ITP 2.1) to one day. This severely impacts ad measurement and attribution: if a customer clicks an ad but only buys the product 24 hours after the ad has been clicked, the ad clicked couldn’t be linked to the purchase. One specific solution of this is to increase the cookie duration through the advertiser’s backend, such as described in this article.

Safari has a 16,65% share worldwide, and a 22,58% share when only looking at mobile devices. Here’s an overview of the distribution of the different bowsers worldwide.

Browser Market Share 2020
Source: https://gs.statcounter.com/browser-market-share/ (13.08.2020)

IDFA depreciation and limited access to IDFA on iOS 14 for attribution

Apple announced in June 2020 the new App Tracking Transparency Framework, which will be rolled with iOS14 in September 2020. This framework requires an app to explicitly ask for a user’s permission to access their device’s IDFA. Since it can be assumed that only a small percentage of users will consent to apps accessing their IDFA, this will have immense implications on ad-targeting and performance measurement for mobile app marketing. Users are not trackable on a device anymore, which makes it near impossible to link marketing touchpoints to conversion events. Through Apple’s SKAdNetwork it will still be possible to link installs to campaigns, but also not on user level. Mobile tracking solutions such as Adjust or Appsflyer will have trouble providing the data they used to provide to their clients. Adtriba as a holistic measurement provider, which integrates user journey data from these two providers, won’t be able to account for mobile app user journeys on iOS14 devices as accurately as before.

Android devices are not affected by this and one could argue that there are more Android than iOS devices and users. But given the typically higher purchasing power of iOS users and their relevance to most mobile app marketers this still presents a seismic shift in the mobile app marketing space.

Confusion and Misconceptions

The discussed developments and adjustments pose significant challenges to the whole digital advertising industry. It was to be expected that some hysteria and confusion develops around it. For example, numerous articles proclaimed that “the cookie is dead”, without differentiating between the different types of cookies. Third-party cookies are the cookies being blocked, and they are only a subset of all cookies and first-party cookies are an essential part of how the web works. First-party cookie tracking is restricted through ITP by the one day lifespan, but as discussed, there are working solutions for this.

In June 2020 there was a headline announcing that Safari now blocks Google Analytics . This was quickly debunked as not true by Simo Ahava, but is an example of how quickly this sort of misleading information can spread and how sensitive the industry has become.

Some experts and adtech providers suggest cookieless tracking alternatives, including fingerprinting. Despite not being GDPR compliant, these technologies bear the risk of being seen as a way to work around Firefox’s, Apple’s and Google’s intent for restriction and more transparency regarding tracking, which ultimately could lead to end up on respective blacklists and being blocked completely. Adding to the non-sustainability of fingerprinting technologies is that the client’s and device’s user-agents will restrict the information provided more and more heavily in the future, as for example here and here.

Solutions

There are different approaches on how to still provide accurate marketing measurement given all these developments. Generally there are two categories on how to deal with a potential loss of data for marketing measurement. One is making sure as much user-level data is still available and the other one is working with non-user level, i.e. aggregated data.

Tracking Data

For advertisers there will always be a possibility to still track users’ clicks on the web to some degree, for example through backend / server-to-server tracking. The tracking codes won’t be called within the user’s browser as part of the website running within the client’s browser, but from the advertiser’s server delivering the website. This way all browser related blocking is bypassed. Of course, advertisers still should get user’s consent to track them in a GDPR compliant way.

A short- to mid-term solution involves re-setting the first-party cookie with a new lifetime duration through the advertiser’s servers. This enables the usage of first-party cookies for click and visit tracking within the restrictions of the Safari browser. In combination with white-label tracking, where the actual tracking call is to the advertiser’s own domain, this is quite a reliable solution for first-party cookie tracking within Safari. Of course with further versions of ITP and accompanying restrictions this could also prove to not be a valid solution anymore, so the longer-term solution would be backend / server-to-server tracking.

Google has released Server-side tagging into public in August 2020, which “just might change the landscape of digital analytics for good”. Server-side tagging involves a Google Tag Manager container running in a server-side environment. Amongst other things, it allows running a complete digital analytics and marketing measurement system without loading any third-party code in the user’s client (browser or device) besides Google’s Universal Analytics Tag. Google opened up their GA Tag quite significantly for this to work. Instead of different analytic trackers running client-side in the browser, there would just be the calls to the server-side container, which in turn could distribute the tracking data to the respective analytics systems. It would potentially allow circumventing some of the challenges posed by Ad Blockers and ITP regarding user journey tracking. Client-side consent management through CMPs needs to be enhanced to server-side consent management, to continue respecting users consent. This server-side tagging holds a lot of potential for making measurement setups much more efficient.

An example of a tagging configuration that uses a server-side container.
An example of a tagging configuration that uses a server-side container. Source: https://developers.google.com/tag-manager/serverside/intro)

Aggregated Data, Marketing Mix Modeling and Unified Marketing Measurement (UMM)

For platforms and marketing activities that are not or no longer trackable, measurement approaches need to be applied that work with aggregated, i.e. non-user level type of data. These include the following marketing tactics, as in part discussed above:

  • - Display ad views (limited through third-party cookie blocking)
  • - iOS ad clicks and views
  • - Facebook and Instagram ad views
  • - Offline marketing activities, e.g. out of home, print, TV
  • - External / non-marketing effects: weather, trend, seasonality

User-level ad view data for Facebook or other social media platforms, such as Pinterest, Snapchat or TikTok, usually is not available to advertisers or measurement companies. Alternatively aggregated view data (e.g. number of views on a FB campaign on a particular date) should be included in the measurement. Since regular display ad view tracking is impacted by the blocking of third-party cookies, these display ad views could also be measured through using aggregated view data modeling. This way all display views, whether social or “regular” would be measured and evaluated in the same and therefore coherent way.

Analysis and modeling of aggregated marketing data is based on marketing (also called media) mix modeling methods. This involves econometrics and statistical analysis to quantify the incremental impact of various marketing tactics in a cross channel marketing mix. The input data includes marketing time series data (e.g. spend, views or clicks per day on instagram mobile app ads) but can also include external factors such as seasonality, weather or competitors activities. It’s aim is to optimize budget allocation in the marketing mix and all advertising tactics to maximize revenues and/or profits. Since it evaluates ad performance on an aggregated and macro-level, it’s often referred to as a “top down” approach to measurement. In contrast to this, user-level multi-touch attribution, working with granular micro-level data, is also called a “bottom up” approach.

Since media mix modeling works on aggregated data it isn’t impacted by all the developments discussed above. Aggregated marketing data will always be available and can always be used for marketing measurement. For many it is the solution for mobile measurement and advertising, as described here.

Furthermore, marketing tactics and investments that aren’t trackable, such as print advertising, out of home, PR and non-marketing factors, such as brand awareness and macroeconomic situation can be included. The disadvantage of marketing mix modeling is, that since it only works on aggregated macro-level data a lot of information is lost, compared to user journey micro-level data. This ultimately leads to a lower accuracy of the measurement through marketing mix modeling, compared to user level multi touch attribution.

The best approach therefore is to combine marketing mix modeling and multi-touch attribution with a unified marketing measurement (UMM). UMM allows assessing the impact of marketing tactics, for which only aggregated data exists, on specific user journeys. Without going into the details here, but it e.g. estimated how many visits to a website or app are due to TV or print ads. Then probabilities can be calculated for how likely it is that a specific visit (and a potential succeeding purchase) is triggered by these offline channels.

Summary

All in all there’s a clear and obvious trend away from simple client based pixel tracking of users as the sole basis of marketing attribution and measurement. Marketing mix modeling and UMM need to be integrated to enhance marketing intelligence and optimization.

Marketing mix modeling techniques have been in use for advertisers that work with a mix of offline and digital marketing channels for quite a while now. Performance marketing managers including mobile advertisers so far have been using attribution methods that assume the availability of precise user journey tracking data. The discussed restrictions on user journey tracking impact marketing measurement significantly if no enhancements are implemented.

Below is a summary of how the different frameworks, tools and regulations impact marketing measurement and what potential solutions are.

Challenge - Ad Blocking

  • Browser/Platform: All
  • Impact: Missing link between marketing click and conversion, Display ads not being displayed
  • Possible Solutions: Backend Tracking & Server-side Tagging

Challenge - GDPR

  • Browser/Platform: All
  • Impact: Missing or no user journey touchpoints if consent isn’t given
  • Possible Solutions: Backend Tracking and Server-side Tagging with respective consent management

Challenge - ITP/ETP

  • Browser/Platform: Safari, Firefox
  • Impact: Limited default first-party cookie lifespan of one day leading to incomplete user journeys and missing touchpoints, blocked third-party cookie tracking (see below)
  • Possible Solutions: Backend Tracking, White Label Tracking and setting of cookie duration through advertisers server, Server-side Tagging

Challenge - 3rd Party Cookie Blocking

  • Browser/Platform: Safari, Firefox, Chrome (from March 2022 latest)
  • Impact: Display View Tracking not possible, leading to undervaluation of display ads
  • Possible Solutions: Macro Level View Modeling & UMM

Challenge - IDFA depreciation

  • Browser/Platform: Mostly mobile app marketing related
  • Impact: Limited user journey tracking for mobile app marketing for iPhone users
  • Possible Solutions: Macro Level Modeling, UMM