A route to more reliable, forward facing solutions with active Human · AI Collaboration

As machine learning (ML) capabilities advance, and with the advent of widely available low-cost cloud computing, AI will inevitably be applied to a wider range of more challenging problems, including those that affect the outcomes for millions of individuals throughout society. In high impact, complex settings, it simply isn’t realistic to train a model up front with a single batch of training data and expect it to perform well in all possible scenarios — such a naïve approach will almost certainly fail to capture some of the underlying nuance and edge cases of the situation, leaving gaps in performance and risk of failure during use. Active learning provides a promising way around the issue, empowering the AI to learn from human teachers in uncertain or novel settings and on new data. This architecture allows human experts to impart knowledge gradually as and when they become aware of AI shortcomings, improving performance through teaching and demonstration.

Training models with less data

In many cases, it’s possible to achieve good performance in a model without using large datasets. A large number of data points in most datasets encode information in common with other parts of the dataset and result in increased training cost, for comparatively little return in performance gain.

Active learning allows AI models to be deployed and start delivering value with less data, time and training investment, improving their performance on the fly through the oversight and involvement of human experts in a hybrid, human · AI collaborative system.

See part one of our blog series for an example of how active learning can reduce training time by 75%.

The benefits of active learning

Mind Foundry technology leverages active learning capabilities through our continuous metalearning and human • AI collaboration architectures, combining the speed and scalability of digital decision making with the contextual and situational awareness of human decision-makers. This provides several benefits:

“If you have a high stakes problem that affects human outcomes at scale… that last 1% can be someone’s life.”

Active learning in the field

Very few AI applications are static and capable of being left to their own devices without careful monitoring and relevant updates as problems change with time. Many AI applications resort to frequent benchmarking and performance assessments to detect and resolve data drifts and model degradation, but active learning provides a smoother alternative. By including the facility for algorithms to query human experts on areas of uncertainty, or for validation, it is possible to continually monitor and improve performance, removing the need for resource-intensive batch assessments and updates. This results in models that remain more up to date, a workforce that is better engaged with the problem, and more predictable resource requirements for the project as a whole. Mind Foundry Solutions frequently include active learning capabilities due to the increased reliability, predictability and safety they bring.

For some problems, it may not only be acceptable to train models to a percentage maximum performance, but it might also be the most ethical to do. For example, if you’re running ML models to improve your ability to run email marketing, you may agree that it just doesn’t make sense to use 75% of your resources to get 1% improvement. Not only would that cost your business a lot of money, if everybody did that (and we know that many people are doing that), it has profound consequences on our environment. But if you have a high stakes problem, an important problem that affects human outcomes at scale, you need that last 1%. That last 1% can be someone’s life. With active learning, it doesn’t need to be a binary decision between getting that last crucial bit of accuracy or utilising your resources more efficiently — you can do both.

Our work in the Public Sector is a great example of this, where the slightest change in results could have a significant impact on the lives of hundreds of thousands of people.

To learn more about how human • AI collaboration powers Mind Foundry solutions, schedule a demo or contact one of our experts.

Author

Alistair Garfoot is Mind Foundry’s Product Owner for Government and human AI collaboration where he specialises on deploying AI in high impact situations where human input and ethical awareness are paramount. Alistair’s technical background bridges the gap between customer problem and technical solution, allowing him to closely align Mind Foundry’s products with the sensitive needs of high stakes customers. On the weekends you can find him cycling through the English countryside.

Mind Foundry

Mind Foundry is an Oxford University company.

Operating at the intersection of innovation, research, and usability, we empower teams with AI built for the real world.

Founded by Professors Stephen Roberts and Michael Osborne, pioneers in the field of AI and Machine Learning, the mission of Mind Foundry is to create a future where AI and Humans work together to solve the world’s most important problems.

Mind Foundry has developed technology and products that help people bring machine learning closer to their work. Our platform is a new type of Machine Learning that is powerful enough to be trusted by experts and easy enough to be used by people throughout your organization.

Built upon a foundation of scientific principle, organisations use Mind Foundry to empower their teams in entirely new ways.

Mind Foundry is an Oxford U. Company. Operating at the intersection of innovation, research, and usability we empower teams with AI built for the real world.