We've been super-busy so sorry for the low number of blog posts. Here's another tech one lifted from production: we'll keep it short and sweet.

Say you've got 800'000 recipes stored in a database in pure text (no structured data) and you want to automate a workflow that, based on a customer-submitted description, pulls out exactly the right recipe (or the best top 10 recipes). Database text search tools aren't designed to be accurate for "fuzzy" queries on large data volumes. Traditional keyword approaches won't help either: how would you know which keywords to focus on? Also to make things even more interesting: some of the 800'000 recipes are updated, removed or added every month so static mappings won't help, and some recipes use the metric system whereas others the imperial system.

Enter machine learning, more specifically LSTM models. You define 40 recipe categories (soups, deserts, salads, etc.) and sort a few thousand recipes into these categories. Using this data you can train an LSTM model to predict the category of a recipe based on its text description... and the same LSTM can help you classify what kind of category a user is looking for. So when a user enters a query, you first identify the recipe category and then focus your search only on those categories. In our example and assuming recipes are uniformly allocated, this would reduce our query 20'000 recipes, much easier than before.

There are further steps to the algorithm (think "word2vec" to identify the best keywords) but we'll stop here: it already illustrates how including machine learning at distinct points in an automation process has a significant impact. The above use-case is something we are dealing with at one of our clients (not with recipes) and the high accuracy we achieve wouldn't be possible without machine learning.