On GDPR compliance implementation

Joris, with a great experience as back-end developer, discovered and learned to love react and reactive programming on his current project, i24News. He shares with you his knowledge about General Data Protection Regulation he applied to i24News. Enjoy !

The General Data Protection Regulation (GDPR) is a regulation in european law which applies to all residents of the european EU and aims to give control to individuals over their personal data, and how they are used by third-party companies. As developers, we are directly concerned by this law, as it is our responsibility to ensure that the websites we run in production are complying with it.

Our clients want to analyse their traffic, share contents on social networks and eventually benefit from advertising revenues. For years, we took the habit to widely use third-party libraries to achieve these needs, and became heavily dependent on them. Although we do have control over how we exploit personal information of our users (data we persist in a database, cookies etc), this vanishes as soon as things get out of our technical control.

This article is an attempt to describe how we wandered through this by the help of a concrete example.

i24NEWS as an example

i24NEWS is an international news and current affairs television channel, under ALTICE USA NEWS DIVISION group, which also publishes articles on the internet in three languages: french, arabic and english. Although the GDPR is only supposed to protect users within the EU, i24NEWS took the decision not to have distinct rules and to apply the same restrictions, no matter the country of the end user.

On i24NEWS website, users can register and sign in, so they can comment on articles and flag contents as favorite for later readings. Since we persist personal information such as their email, we are thus legally bound to give them the possibility to delete these information, as stated by the right to be forgotten. This is no big deal. One delete SQL query, and we're done.

Without being registered, users can also subscribe to a newsletter. As i24NEWS uses a marketing cloud platform to trigger newsletter campaigns, we also have to ensure that user's personal information are removed from their system as well when they unsubscribe from the newsletter. Not very complicated either.

Things become way more difficult when it comes to vendors used on the website, which creates tons of mysterious and unspeakable cookies:

• Google analytics, to measure traffic and most read articles,
• Facebook and Twitter SDK, to share articles on social networks,
• SmartAds and Taboola, for advertising purpose.

As developer are not lawyers, the EU created a dedicated organization to normalize and ease interfacing between first parties (editors like i24NEWS) and third parties (vendors). Unfortunately, not every vendor comply with it, so in the end we have two different cases to handle there : advertisers and other vendors.

Hold your breath, lots of acronyms to come right now !

The role of the Interactive Advertising Bureau

The Interactive Advertising Bureau (IAB) is an advertising business organization that develops industry standards and provides legal support for the online advertising industry. One of their missions is to propose a Transparency & Consent Framework which let editors work with third parties and allow them to process user data in compliance with the law.

To achieve that, vendors register themselves after the IAB, by providing a list of purposes they seek regarding user data, i.e. measurement, ad selection, delivery and reporting... By doing so, they give IAB the possibility to maintain a vendor list which contains every purpose, feature, vendor (technical platforms that can store and process personal data) in the advertising supply chain, and their relations. Here is an example of how it looks for the Smart Adserver vendor :

{ 
    "id":45,
    "name":"Smart Adserver",
    "purposeIds":[1],
    "legIntPurposeIds":[3, 5],
    "featureIds":[3],
    "policyUrl":"https://smartadserver.com/end-user-privacy-policy/"
}

Ids refers to purposes and features defined earlier in the vendor list. So how is that useful actually? Well, unless you lived in a cave for the past years, you probably noticed how the internet has been invaded by popups yelling "YOUR PRIVACY MATTERS" and asking you to check boxes in an interface as user friendly as a tax form? This is only the tip of the iceberg and is part of what is called a Consent Management Provider (CMP). The vendor list we saw above represents the base data used in the CMP to determine for which vendors and purposes the user should explicitly give his consent. Ironically, from a legitimate need to protect user privacy (and to fight ads), we ended up by websites full of these invasive overlays.

Under the hood, the user choice, basically a list of ids, is then compressed in a base 64 consent string, which can be parsed by vendors. That way, they can set proper permissions on their side regarding personal data gathering and storage.

Source: IAB Europe

The Transparency & Consent Framework states that every CMP must provide the following API: __cmp(Command, Parameter, Callback) This function is called by IAB compliant vendors to get the consent string from the website on which the CMP has been implemented. If you're still reading, let sum things up:

• editors choose a CMP and implement it on their public website
• end user define permissions in the CMP interface
• permissions are concatenated and encoded in a consent string
• the consent string is transmitted to vendors thanks to the __cmp() API
• the IAB emit standards for all this mess to work

At first sight, nothing that we couldn't build by ourselves. Although there is a lot of out of the box CMPs in the market, we definitely didn't want to use one of them. When facing a problem, and unless we're forced not to do so, we always make the choice to build our own solution as it reduces dependencies and therefore gives us control and customization enhancements. The consent string SDK being provided as an open source library by the IAB, all we had to do was to build a little form, generate the string and basta. How naive. From the IAB itself:

A Transparency & Consent String may only be created by an IAB Europe TCF registered CMP using its assigned CMP ID number in accordance with the Policies. Vendors or any other third-party service providers must neither create nor alter TC Strings. These and other requirements are articulated in the Policies to which all parties including CMPs, Publishers, and Vendors, are bound.

Like vendors, CMPs have to register themselves with the IAB, and this has legal repercussions. After some research in that direction, we finally decided that it wasn't worth to involve i24NEWS neither KNP Labs in legal concerns by declaring ourselves as a CMP.

OIL to the rescue

OIL is an officialy registered CMP and pretty much answered our needs. What convinced us is that it makes good usage of message events.

See here for a complete reference of OIL dispatched messages.

i24NEWS frontend being a single page application we developped in a reactive functionnal programming way (to keep it short, just think that the application is the reflection of a state, which is updated by dispatching actions through time), we could hook onto those OIL events to accordingly apply the side effects we need.

This feature was life saving when it came to handle vendors that were not part of the official vendor list used by the CMP. Remember that for now, by implementing a CMP we only took care of the advertisers vendors, but for other ones, extra cares have to be taken.

Non advertising vendors

Although Google engaged itself to comply with the Transparency & Consent Framework by next March, there is no such thing regarding Facebook or Twitter. In this situation the only solution we have to ensure that no cookie is created without the explicit consent of the user is to totally block these vendors until the user accepts every purpose in the CMP interface.

OIL dispatches an event when the user 'opt in' (in other words, he checked some options and accepted the global policy of the website), and another if he choose to change these options later. By observing these events, we can then decode the consent string from the CMP, check if the permissions given are enough to allow non advertising vendors to work, and finally dispatch an action to reflect these changes. As this is really tied to our own implementation, the following snippet give you an example of how it works using plain old js:

// optInLayer :: OptInLayer
const optInLayer = window.AS_OIL

// decodeConsentString :: String -> ConsentString
const decodeConsentString = consentString => new ConsentString(consentString)

// facebook, google and twitter are not supporting the IAB standard,
// so we ensure all purposes have been allowed by the user to allow
// these vendors from being used
//
// @see https://vendorlist.consensu.org/vendorlist.json
//
// checkForConsent :: (OptInLayer, String -> ConsentString) -> Boolean
const checkForConsent = (optInLayer, decodeConsentString) => 
  [1,2,3,4,5] === decodeConsentString(optInLayer.status())

window.addEventListener('message', event => {
  if (
    ['oil_has_optedin', 'oil_optin_done'].includes(event.data) &&
    checkForConsent(optInLayer, decodeConsentString)
  ) {
    // load non IAB vendor initialization scripts...
  } else {
    // display a message to warn the user that some features are blocked...
  }
})

window.AS_OIL is the way to access the opt in layer data, and especially to get the encoded consent string. The checkForConsent function returns a boolean value which determines if non IAB compliant vendors should be loaded or not. We can rely on it to load or block Facebook, Twitter or Google initialization scripts.

And voilà ! Any thoughts or ideas ? Leave us a message on the article or on twitter