For AI to ultimately have a positive impact, an intentional approach is essential, and human-in-the-loop (HILT) and its natural successor, reinforcement learning with human feedback (RLHF), are essential components of this.
What is HILT?
To understand HILT, you first need to understand machine learning. Machine learning is when a computer can learn from data. LLM (Large Language Model) chatbots are made possible by machine learning. A concrete and easy-to-understand example of machine learning is in computer vision. To teach a computer to recognize cars, you give it 10,000 pictures of cars and 50,000 pictures of non-cars (boats, buildings, mailboxes, etc.) and tell it to “sort them out”. Thanks to machine learning, the computer learns to recognize cars and can do it better than it could without machine learning.
But here's the problem: if a computer is taught the wrong thing, or makes the wrong assumptions on its own, it will repeat the wrong thing without human intervention. For example, if you train a computer to recognize faces by telling it, “Put all these different faces into clusters, and call each cluster a different person.” Then one day, the camera takes a picture of your face from a weird angle, and you look a bit like someone else in the office. Suddenly, you and this person end up in the same cluster, and the computer might think you're the same person. This seems like a minor issue, until that person is charged with a crime and you're arrested by the police.
That's why machine learning needs supervision to work effectively: if the confidence score is low and the computer can't tell if it's you or someone else, a human can make a decision, giving the computer important feedback that will improve its ability to distinguish between you and your peers in the future.
This type of supervised learning is known as HILT, and while the concept is widely recognized as an important safeguard, as AI systems become increasingly complex and operate at breakneck speed, the human ability to effectively monitor and respond to potential failures is being called into question.
Who monitors the monitors?
In certain scenarios, the reaction time of a human monitor may be too slow to prevent costly errors or unexpected outcomes. This has led to the proposal that the AI itself should play a role in monitoring itself and other AI systems, especially in high-responsibility environments such as security.
While this idea has merit, it is not without risks and challenges: Over-reliance on AI oversight can introduce errors, biases, and blind spots, so it is important to balance AI oversight with human judgment.
Surveillance means safety
Given the black box nature of current deep learning models and the risk of unintended biases emerging from the underlying data, humans provide a critical safety net, providing judgment, intuition, and expertise that can catch problems that AI might miss. Or, as Michael Polanyi has pointed out, there is a tacit aspect to human knowledge: humans know more than they can tell. Human experts have valuable insights that are difficult to fully code into AI. They can also adapt flexibly to new situations and perform common sense reasoning that AI struggles to do.
A hybrid approach that combines AI automation with human oversight is often optimal: AI handles the bulk of the monitoring tasks and performs the bulk of the work, while humans provide high-level guidance, handle edge cases, and make the final decision in critical scenarios. This requires developing AI systems that are human-aligned. Techniques such as runtime monitoring, formal verification, and interpretability methods help build trust and detect defects. Human monitors also need to be trained to effectively monitor AI.
Security is a high-responsibility use case that requires HILT
Security is a responsible use case for AI, as it involves decisions about the safety of human freedom, property, and even life. HILT is an industry that is especially well-suited for remote security when combined with AI video surveillance. In this case, teaming humans and AI allows for the best of both worlds.
A computer can watch all the video at any time without performance degradation, but humans inevitably tire from monotonous tasks. AI can detect when something potentially worrisome has happened, like someone in an apartment parking lot after midnight. But AI isn’t very good at judging whether someone in the parking lot is there properly. The act of unlocking a car door with a key and the act of unlocking it with a crowbar are too similar for an AI to distinguish reliably. Here, humans are the best final arbiters. If they determine that someone in the parking lot is actually trying to break into a car, humans can “reprimand” them over a network speaker. They can then escalate to the police or intervene in other ways that AI can’t always do well.
So, what happens next?
The next step in the remote guard example above is to feed human decisions and actions into the computer so that it can automate more and more of these tasks in the future. This is RLHF, meaning the system benefits from human training and gets smarter over time.
Systems that incorporate RLHF efficiently from the start will be the winners. By learning from human input, these systems will become faster and smarter than those that don't. Human oversight will remain important in critical areas like security. Maintaining a balance between autonomy and oversight will be key to ensuring the safety, reliability, and integrity of ever-evolving AI systems.
Is my AI racist?
Human-generated labels also play a critical role in reducing bias in AI systems. Carefully curating and annotating training data with a human perspective ensures that AI models learn from a more balanced and representative set of examples. This HILT approach to data preparation helps reduce the risk that AI systems will perpetuate or amplify biases present in the original data. However, it's important to recognize that humans themselves may introduce their own biases.
For example, say you want to build a system that can detect criminal activity like someone breaking into a car or jerking stuck keys. Now, assume you are training it from real security camera footage. It is possible that one race is over-represented in this video footage you collected. In that case, the system may mistakenly train from that race and predict criminal activity. As a result, people of that race may be questioned/arrested/tried/convicted more than others for the same act.
The twist is that if the humans in the loop have racial biases, RLHF may make the situation worse, not better. Let's say humans tend to mark one race.'s Subjects may see themselves as participating in criminal acts more frequently than others based on the same behavior due to their own racial biases, in which case RLHF would perpetuate those human biases back into the model rather than eliminating them.
Stay on the Right Path
Will AI become an evil Skynet or a benevolent Oracle? It's up to us and the approach we take now in terms of what we develop as engineers and what AI we buy and use as consumers. HILT and its offshoot RLHF will help keep us on the right path.
Ultimately, combining the strengths of AI automation with HILT and incorporating RLHF has the potential to deliver the fastest, most responsible AI development across all industries.
