Well annotated data is a rare thing when working in real-world machine-learning projects. So how can developers still use machine-learning without available training data to start? For example let's say you're building a machine learning model that classifies companies based on their text description (something that we're building at Sense6). The challenge is that no data-set exists for the classification that you have in mind.

A pragmatic way to tackle this challenge - we think - is by setting up 3 components:
  • A basic algorithm to kick-start the classification: in our company classification example it could be something as basic as a keyword ranking algorithm, or alternatively something slightly more sophisticated like tf-idf
  • A feedback loop: a combination of process and technology that enables end-users to review and update the classification that your model is generating
  • A machine-learning model: in our example we'd recommend using an LSTM which are simply magical in "divining" the right classifications

So how does this work? The motto for your basic algorithm is "good enough": as soon as its output is a usable starting-point for your end-users, then move on.
The motto for the feedback loop is "smooth and beautiful". It's a vital step and one that often doesn't get enough attention. One unnecessary click, one slow load time, one unnecessary text field: any of these can mean the difference between abundant feedback or nothing at all. Enhance the feedback mechanism until it's as seamless and easy as possible for your end-users.
Finally the motto for your machine-learning model is "retrain, retrain, retrain".

Very soon you'll observe that the machine-learning model picks up where the basic model stopped and becomes orders of magnitude more precise, assuming you are gathering regular and quality feedback from your end-users via your feedback loop.

There are other ways around this chicken-and-egg problem (for example deep reinforcement learning) however they are often more complex and not always applicable in real-world situations. We've found that the above approach is pragmatic and rapidly leads to performing models.

Please contact us if you have alternative ways of dealing with the chicken-and-egg problem, or if you have a similar situation that you'd like us to take a look at.