Integrating retailer through Datafeeds instead of APIs

Background

Some retailers that operate different stock-lists across many locations have large amounts of inventory data and it is not always possible or efficient to either request their stock through APIs or scrape web interfaces.

In such cases, integrating retailer stock through data feeds is a faster and more efficient way to get inventories synchronized with Whisk.

Terminology

Retailer

High-level business entity that could be integrated with Whisk (e.g. HarrisTeeter)

Store

A geographic location which serves as a point of sale, where the end user can purchase products. 

Availability zone

Each product that is available on multiple stores is also grouped by price and promotional data. This grouping is also known as Availability Zones. 

Multiple stores could belong to a single availability zone, while a single store cannot belong to multiple availability zones. 

Format

The Data Feed file is formatted as a new-line delimited JSON (.jsonl format) file. The benefits of this format are:

• It supports data nesting, structures and arrays
• It allows for effective stream processing

Data Exchange

1. Google Cloud Storage (GCS) 

Whisk uses Google Cloud Storage (GCS) for collecting data. GCS provides libraries for multiple languages so you can easily automate your data export jobs.

1. AWS S3

As an alternative option Whisk also supports AWS S3 buckets for data exchange.

Our system scans data for the daily updates to keep availability and pricing information up to date, so please provide updated data at least once a day.

Please contact us to get access to our data storage. 

Data feed essentials

Store information

This file should contain information about the retailer stores and their locations.

Whisk supports two types of store discovery for users:

1. 
   If servicedPostalCodes are defined whisk will use this information to recommend people stores based on their postal code
2. If servicedPostalCodes are not defined whisk will use latitude and longitude n to find the closest stores to the user in a 25-mile radius (40 km)

If you do not wish to show the user multiple price and availability information for different regions, simply use a single availability zone for all products.

stores.json

*- required fields

• Id* - unique store identifier
• region - region or state e.g. CA
• city*  - e.g San Francisco
• storePostalCode - postal code of the store
• servicedPostalCodes - postal codes of addresses which are serviced by this store.
• availabilityZoneId - identifier of the availability zone
• lat - latitude 
• lon - longitude

Example:

{
  "id": "0123",
  "region": "CA",
  "city": "San Francisco",
  "sotorePostalCode": "94016",

  "servicedPostalCodes":["94016"],
  "availabilityZoneId": "011",
  "lat": 37.766667,
  "lon": -122.433333
}

Product details

products.json

*- required fields

• sku* - unique product identifier inside retailer
• name* - product title preferable without weight details
• codes
• gtins - list of gtin compatible codes, any gtin code can be listed here UPC-A, EAN, EAN-8, etc.
• brand - brand name

• categories*  - list of categories where product is presented

• images* - list of product images

• url* - link to product page in the store

• amount* 

• byItem -  if product can be bought by item

• raw* -  string description of the amount

• unit  - measure unit (g, oz, lb, etc.)

• value  - number of units e.g. 100

• max - max number of items possible to buy

• min - min number of items possible to buy
• byWeight -  if product can be bought by weight aka counter product

• unit*  - measure unit (g, oz, lb, etc.)

• step*  - amount step which user can buy
• max - max amount of product which user can buy
• min - min amount of product which user can buy

Regular item example

{
  "sku": "11111",
  "name": "McCormick Chili Seasoning Mix, 1.25 OZ",

  "url": "http://link.to/product/page",
  "images": [
    "imageUrlHere"
  ],
  "brand": "McCormick",
  "categories": [
    "Pantry"
  ],
  "codes": {
    "gtins": [
      "0005210009150"
    ]
  },
  "amount": {
    "byItem": {
      "raw": "1.25 OZ",
      "unit": "OZ",   // optional
      "value": 1.25,  // optional
      "max": 100,    // optional max items possible to buy
      "min": 1       // optional min items possible to buy

    }
  }
}

Weighted item example

{
  "id": "222222",
  "name": "Tesco Counter Roast Turkey Breast",
  "images": [
    "imageUrlHere"
  ],
  "brand": "Tesco",
  "categories": [
    "Deli Counter"
  ],
  "codes": {
    "upcs": [
      "0005210009150"
    ]
  },
  "amount": {
    "byItem": { // if product can be sold by item
      "raw": "100 g",
      "unit": "g",   // optional
      "value": 100,  // optional
      "max": 100,    // optional max items possible to buy
      "min": 1       // optional min items possible to buy
    },
    "byWeight": { // if product can be sold by weight
      "step": 100,
      "unit": "g",
      "max": 1000, // optional max amount possible to buy         

      "min": 100   // optional min amount possible to buy 
    }
  }
}

Product availability

product_availability.json

Should contain following fields

*- required fields

• sku* - unique product identifier inside retailer
• availability* - list of zones where product is available
• availabilityZoneId* - unique availability zone identifier
• outOfStock - if zone is not listed in availability list we consider it out of stock, if empty consider as false

• price - list of price information. If product have different options to be sold by weight and by item for example

• regular - usual price

• promo - current promo price

• amount - amount details for products sold by weight, if not defined price will be applied to product which is sold by item

Example:

{
  "productId": "unique product id",
  "availability": {
    "availabilityZoneId": "unique availability zone id",
    "outOfStock": true,
    "price": [{
      "regular": 1.22,
      "promo": 1.10,
      "amount": {

        "value": 0.5,
        "unit": "kg"
      }
    }]
  }
}