Filter device data by properties and values with signatures

Get the most out of your 51Degrees data file using Java API

Did you know that you can use the 51Degrees device data file not only for device detection, but also for enhancing the report generating tools, enhancing the analytics data and exposing parts of data to improve client experience? This article demonstrates how to use the signature entities from the 51Degrees data file to produce a subset of signatures based on one or more property.

The idea is simple: start with a full set of signatures and apply a filter until you only have the subset that matches all of the specified parameters. The example source code includes a test run function that starts by removing signatures where HardwareVendor is not 'Samsung'. Then a filter is applied to only leave signatures where the IsSmartPhone property is set to 'true'. Finally the list is filtered yet again to leave only the signatures where the PlatformName is 'Android'. The remaining signatures in the list correspond to Samsung smart phones that run on Android.

Benefits

These are just some of the examples of where this functionality will prove useful.

An ad-tech company could allow clients to fine-tune their ad placement. For example: there is very little benefit for an online store that sells Windows phone accessories to display ads to users on other phones. Allowing customers to target specific devices or operating systems will benefit both.
A mobile analytics company could use the data to help identify the trends in device popularity or track the popularity of a particular device characteristic like the screen size.
When generating usage reports it is useful to be able to select statistics by a specific property or a combination of properties. While the report generation could be automated, selecting the properties to use for the report should be left to the end user. Better yet, also allow to select the value for the property. The information generated could prove invaluable for analytics.

Filtering devices

Firstly you need to create a Provider object you can use to access the data file entities. For more information on the structure of the Pattern data file please see 'Pattern Data Model'.

Provider provider;
/**
 * Default constructor.
 * @param fiftyOneDataFile path to the 51Degrees data file.
 */
public SequentialListGenerator(String fiftyOneDataFile) {
 try {
  //Use memory mode for this as Stream will take considerably longer.
  this.provider = new Provider(MemoryFactory.create(fiftyOneDataFile, false));
 } catch (IOException ex) {
  Logger.getLogger(SequentialListGenerator.class.getName())
	.log(Level.SEVERE, null, ex);
 }
}

In the snippet above the Provider object gets instantiated with the dataset created by the memory factory. Memory factory reads the entire dataset into memory which makes it faster than stream factory which only loads the headers but uses the actual data file to retrieve entities.

Create a copy of the signatures list that will be used to remove the irrelevant signatures. A signature is the base entity that has a rank and profiles assigned to it. Through the profiles you can access the values and properties associated with those profiles.

//The original list of all the Signatures.
ArrayList<Signature> sortedSignatures;
/**
 * Copies Signatures in to a list.
 */
public final void readSignaturesToRanked() {
 this.sortedSignatures = new ArrayList<Signature>();
 for (Signature s : provider.dataSet.getSignatures()) {
  this.sortedSignatures.add(s);
 }
}

If you plan to reuse the list or require a filter reset functionality you might want to copy the list of signatures and perform filtering on the copy of the list.

The actual filtering is done by traversing the list of signatures and adding each signature to the temporary list if the value of the property used to filter equals the value required. The temporary list then gets swapped with the current signature list leaving only signatures where the specified property has the specified value.

public void filterBy(String propertyName, String propertyValue) throws IOException {
 ArrayList<Signature> temp = new ArrayList<Signature>();
 for (Signature s : this.filterSet) {
  if (s.getValues(propertyName).toString().equals(propertyValue)) {
	 temp.add(s);
  }
 }
 this.filterSet = temp;
}

With each call of the filter function there will be less signatures in the list.

Filtering by signature rank

Each signature has a rank assigned to it. Rank indicates how often this signature is encountered in the wild. The lower the rank the more popular the signature is. You can use this information to remove the unlikely/unpopular signatures. For example: it is entirely possible to run an Android OS on an iPhone hardware but there is only a handful of people who does this therefore such cases can safely be ignored. Removing the signatures with the higher rank means the result is more accurate and relevant in some cases.

There are different algorithms to decide which rank should be the threshold in each particular case. The following example allows user to specify the level of threshold as a percentage. An alternative would be to use the mean or median value and remove ranks that exceed it.

public void trimRank(int thresholdPercentage) {
 //Initially first rank is both highest and lowest.
 int smallest = this.filterSet.get(0).getRank();
 int largest = this.filterSet.get(0).getRank();
 //First pass.
 //Find min and max.
 for (int i = 0; i < this.filterSet.size(); i++) {
  if (smallest > this.filterSet.get(i).getRank()) {
   smallest = this.filterSet.get(i).getRank();
  }
  if (largest < this.filterSet.get(i).getRank()) {
   largest = this.filterSet.get(i).getRank();
  }
 }
 //Chop off the signatures that are below the threshold.
 int threshold = (largest - smallest) / 100 * thresholdPercentage;
 //Second pass.
 //only leave signatures with rank below the threshold.
 //the lower the rank the more popular the device.
 ArrayList<Signature> temp = new ArrayList<Signature>();
 for (Signature s : this.filterSet) {
  if (s.getRank() < threshold) {
   temp.add(s);
  }
 }
 this.filterSet = temp;
}

In the snippet above the signature list is traversed twice. First time to find the largest and smallest signature rank. Second time to remove any signatures that are in the subset where signatures are X percent larger than the threshold. An alternative would be to sort the list based on signature rank and then traverse it removing the signatures.

Complete source code

You can download the complete source code from pastebin. The latest 51Degrees Java API can be downloaded here.