User-Agent Client Hints Update September 2020

How have the UA client hints specifications evolved since January 2020?

Some Chrome engineers are concerned that the HTTP User-Agent header field is no longer fit for purpose. It contains too many unnecessary characters and is not needed on every request. The data could be restructured and better represented as individual HTTP headers. The User-Agent Client Hints unofficial draft proposal seeks to include seven additional headers as follows.

• Browser name

• Browser version

• Platform name

• Platform version

• Architecture

• Model

• Mobile

These headers would only be provided when the web server requests them in the initial response.

Tracking

A third concern these engineers have relates to the privacy implications of providing this data in the User-Agent header. They have a theory that when the information is combined with other information such as the IP address or information about support fonts obtained from JavaScript, they can be used to track people across multiple web sites. We have not seen authoritative evidence to support this happening in practice without people’s consent.

Current status

User-Agent Client Hints were added as an experimental feature to the developer preview version of Chrome called Canary in spring 2020. Interested parties have had a chance to play with them and provide feedback.

The Chromium team have advised that the experiment will not be made live until January 2021 at the earliest. The proposal and its dependent change continue to evolve.

51Degrees are concerned the timeframe being advanced by Google for User-Agent Client Hints is insufficient to gain consensus concerning the need for the change, the design of the solution, the migration and parallel running approach, the justification for a breaking change to the web, and alignment to the 22 related changes under the Privacy Sandbox initiative. The change also does not support some key use cases such as first request optimization.

Working with 51Degrees users we have been actively engaged in this experiment and are now in a position to provide feedback to Chromium and the web community.

Feedback

We have proposed a number of changes to the User-Agent Client Hint specification and submitted these to the W3C via a pull request. You can read the details and provide your own feedback via comments at the following link.

The headline changes are:

1. Avoid conflating the issues of:

   1. legacy inefficiency in the existing User-Agent field value observing that 51Degrees and others achieve over 99% accuracy;

   2. improving the efficiency of communication by only sending the information when the web server requests it; and

   3. privacy features that make the information harder to consume.

2. Deprecating the existing de facto standard of communicating this information should not be part of the specification of another standard. It should be removed. In any case the changes have been proven to be significant and an extensive period of parallel running is required before anything is deprecated.

   There are two dependent proposals that User-Agent Client Hints relies on. One is an experimental specification within the IETF for the concept of Client Hints, and the other is a similarly unofficial draft within the W3C relating to the first request optimisation scenario. These dependencies are clearly identified in our proposed changes. All three need to progress as a package before the proposal advances.

3. Expand the use cases to better explain the uses for the information.

4. Consider restructuring of the existing User-Agent field value to help developers consume the information more easily and reduce the amount of information transmitted. This is likely to result in fewer bits of data being transmitted than the handshake pattern proposed in Client Hints.

It remains unclear if Google will seek to establish consensus on the bundle of User-Agent Client Hint changes within the W3C before progressing to deployment, or if they will progress with the Chromium timeline regardless of whether consensus can be achieved. The W3C Process is built around consensus and an organisation that fragments the web by deploying features that lack consensus is not operating within the spirit, if not the letter, of the W3C mission.

Now would be an excellent time to review the changes and add your own comments.

Multiple HTTP headers

51Degrees solutions already support multiple HTTP headers. User-Agent Client Hints will result in HTTP headers other than User-Agent being required for device detection. Now would be an excellent time to future proof your deployment by enabling support for all HTTP headers.

In all of the version 4 deployments of 51Degrees this is easily achieved by passing the HTTP header key value pairs to the FlowData. The language specific developer document provides guidance.

For those using version 3 follow the documentation for the required API.

Data files

New ideas for the internet come out all the time, and some of them do not always gain mass adoption (remember 3D screens on Smartphones). We have been testing User-Agent Client Hints and have yet to build up a body of data to support our machine learning algorithms. We will therefore be adding “fake” data to the paid for on premise data files and all cloud services including free.

There’s a draft set of client hints headers that we have discussed in our previous blog that you can access. Gathering this data at the moment relies solely on the end user running the stable or Canary build of the latest Chrome version and setting the experimental web platform flags to true.

While this body of data is built up, we wanted to introduce a set of test data for the Client Hints headers so we could represent how you can use this data with 51Degrees in the future, with only minor additions to the current datafile.

When using the fake values below from the headers, a fake profile in either the hardware or software component will be returned.

Client Hints headers and test data;

The fake header data and new datafile profiles will be implemented into the datafile for release on the 14th of September 2020.

As a quick pseudo code example, here’s how the headers could be used to obtain device details, specific to our version 4 APIs.

requestHeaders = {
    "Sec-CH-UA-Platform": "51DTestPlatform",
    "Sec-CH-UA-Platform-Version": "51DTestPlatformVersion"
}
data = pipeline
    .createFlowData()
    .addEvidence(requestHeaders)
    .process()
// This platform name will come from the “Sec-CH-UA-Platform” header.
platformName = data.device.platformName

Summary

Experimenting to innovate and improve the web is important. Client Hints are an interesting experiment. Google asked for input on their experiment and we’ve provided it. In short, it’s interesting, but not ready for prime time, doesn’t consider many scenarios, and the evidence for the both the efficiency and privacy justifications is weak.

That said we are ready should Google decide to ignore the community feedback provided and progress with their idea without building consensus in the W3C first. Afterall Google are a private company which dominates the web and are free to do as they choose. The antitrust issues associated with Google’s dominance of the web and de facto standards are the subject of another blog.

The W3C is an inclusive standards body and you, or anyone, are free to input and lend your voice to the important issues that impact the web today.