It is Alive! AI Deployment for Real Businesses
I have worked on many AI projects, PoC‘s and deployments. Going from Proof-of-Concept (PoC) to deployment, you face many hurdles within the organization. But continuing the live deployment is the real challenge.
First of all, only if there is executive level support for the business problem, the AI deployment can continue. In order to get the executive support, you have to prove that there is an opportunity to increase revenues or reducing costs that you just can not possibly miss. You have to also show that the deployment is not risky. Revenue, cost, risk… That is it. Once you have the executive support, when there is a crisis or you need an extra hand, then people will be able to drop whatever they were doing for that moment and help you out.
The most important part of deployment is making sure that the inputs and outputs are being produced and consumed continuously. When you do a one-of project or a PoC, you need to do the extract-transform-load (ETL) tasks, create a table, clean and impute, create new features only once. This is child’s play compared to the live scenario. You have to make sure that the features are being fed to the live AI system continuously and all the time. Moreover, the outputs need to be processed on time, most of the time if the insights are left to go stale, they will not be of much help. But, wait, what happens if some inputs are not available or they are incorrect? You can not really afford to stop the live AI system and wait for those inputs.
You can not possibly train models on all possible subsets of features, there are exponentially many of them. What you can do for the live system is to make sure that some estimations of (either basic statistics, such as mean, or the estimation by the other features) important variables are available all the time. This has at least two advantages: i) if a feature is incorrect then you can actually create an anomaly flag specific to that feature and ask for it to be fixed. This requires that there are people within the organization who can fix problems with the features. ii) you can use the estimated feature instead of the actual one for the live AI system. How do you know if the feature value being so different is noise or a new pattern? You can ask a domain expert and this is feasible only if you have easy to use AI-human interfaces, and model and data explanations.
Aside from these, you need ALL of your feature processing steps to be scalable, so that they will work when there are a lot of data.
You also need mechanisms like feature selection, so that you do not need to create, maintain or process features unnecessarily.
In addition to reducing costs, feature selection also results in simpler AI models. These models would have less likelihood of overfitting than the models using all the features. Overfitting happens when there is a mismatch between data size and algorithm complexity. When you are learning in a live system continuously, you basically see an infinite stream of data, so can you say that the models should be as complex as possible? Definitely no! Because, most of the time, the data relevant for detection or prediction task now is data closest to NOW. So, the old data might not be helpful at all.
So, having a continuous learning ensemble being fed with a scalable feature processing system ensures that the AI system keeps performing and hence the benefits are seen not only at the beginning but all (at least most) of the time.
Another problem with the live system happens when you continuously update the AI System. Continuous learning from every instance is a great technology. You don’t need to worry about scheduling model updates, they are done automatically. You also spend a lot less time for model training, because you are just updating the model, not creating it from scratch. What happens if you just used the wrong data and your updated model is no more a good model, how can you undo such damage. First of all, you have to make sure that the AI system is backed up continuously, so if something goes wrong you can easily switch to the latest good model. But how do you know which model is good? This is just like the noise vs new pattern problem for the inputs. First of all, you have to make sure that you continuously monitor the AI system. It really helps to have more than one set of algorithms and different instantiations of them running. Because if one fails, the others might be able to go on, you also have better performance statistics. Again whether there is a new novel data segment or just bad data is a decision that can be given much easier with the human expert help.
It takes a village of executives, business experts, business/data analysts, IT and data scientists to keep dancing together to maintain a live AI system. These dances are not ballet performances, but rather like folk dances, performed together, in harmony and with imperfections and variations. In order to do these dances, it is very important to have explainability and easy to use data-human and model-human interfaces. When you have those, business people (as opposed to data scientists only) can keep monitoring the performance of models, devise strategies to make features or models perform better, own the models and use their outputs to improve the business KPIs. By the way, business KPIs also change in time, just like the inputs do. Once the business people are involved in this dance, not only they can reduce the risks of maintaining a live AI system, they can also imagine new AI models with very interesting goals and also interesting combinations of them (enterprise-wide AI) to help the whole organization in ways that have never been imagined before.
