The science behind our data

Learn how we collect, analyse & display our data.
Data Points
product history

Accuracy is everything

One of Omnilytics’ key values is accuracy. The data obtained has been tested internally against the biggest retailers worldwide. Our teams strive to ensure our data is both reliable and precise, down to the last digits. A dedicated QA team is always on the look-out for data inconsistencies and unusual occurrences that may happen in the market. As a double verification, the implementation of our monitoring system will alert us of any discrepancies.

Omnilytics allows you to unravel in-depth information that is customisable. With the dashboard, you are able to drill down numbers and uncover the source of every point you see. No oversights or loopholes.

Transforming data into insight

Our data goes through a seamless process from collection to processing before it is finally interpreted into the easy-to-understand graphs and charts you see on the dashboard.

Data Collection

We scrape over 400 retailers’ sites and more than 75,000 brands across multiple regions every single day. Data such as price, colour, discount and sizes are collected from both private and public sources in 3 separate ways:
These data are updated on a 24 hour basis to ensure real-time accuracy. This strict timeline poses complex challenges to the data mining architecture due to the extensive inventory count across all retailers. To overcome this, our team of highly specialised technical engineers developed a network of distributed data mining systems that spread across the globe including Japan, Korea, Singapore, UK (London) ,USA (Ohio and Virginia), Australia and Malaysia. Developing this system alone took 3 years but was a necessary step to ensure that our data is real-time and above all, accurate.

Data Processing

Collecting the data is just one part of the equation. Processing it and ensuring a standardised form of product categorisation across the board, is another.

Over 228 million data points are processed daily utilising a combination of keyword analysis, object detection and image recognition. For example, when presented with an image of a model wearing a red dress, our automated software is capable of distinguishing the model from the dress and even recognises the type of clothing. On top of that, the system is able to read the product description and detect the type of object it is according to style, material and colour.

To enhance the data, our team of data science experts will generate additional quality checks. This includes standardising colour terms and product categories with deep learning technology. All data that is collected and analysed undergoes constant monitoring so that real-time updates such as price movements, sudden purchasing spikes and sellouts are automatically reflected.

Data Display

Raw data can often be complicating. Which is why our team transforms those data points into visual elements for a more pleasant experience. We convert them into charts and graphs that you can easily extract valuable insights from. We understand that retail moves quick, so we move even quicker to enable you to make the best decisions at the right time. You’ll be able to monitor movements in price, inventory status, discount rate and more across various markets in real-time. Apart from that, we add value to our clients by providing access to in-depth reports created by our Data Analyst teams covering topics like bestselling trends, industry outlooks and runway analyses. Concise and packed with valuable insights, we strive to make our data accessible for any member of your team regardless of expertise or skill set.