The first AI applications in production rarely sit in the data centre. They sit on the line: an assistant that suggests the likely cause of a fault, a camera that flags anomalies, a system that recommends the next batch to the shift supervisor. For the team, that is a relief. For management, it is a new responsibility.
Suggestion or decision
The most important distinction is simple: does the AI suggest something, or does it decide? As long as a person checks the suggestion and consciously accepts or rejects it, responsibility stays where it belongs. Once suggestions are adopted without a real check, because they are usually right and time is short, the machine is in effect deciding.
That does not happen on purpose. It happens when confirming is so cumbersome that nobody takes it seriously, or so casual that it becomes a reflex. Both are design flaws, not failings of the team.
Human oversight has to be usable
The EU AI Act requires effective human oversight for high-risk systems. Regardless of the classification, that is a good guideline for any AI on the line. Effective means: the person at the plant understands what the system suggests, can check it in their working situation and can override it without a detour.
Three design rules follow:
- The suggestion shows its reason. Not as a model explanation, but as a short hint at what it is based on: which reading, which image, which earlier fault.
- Accepting and rejecting are equally fast. Anyone who wants to reject must not need more steps than someone who agrees.
- The decision is recorded. Who accepted or rejected which suggestion, and when? That is the evidence that a person decided.
Intended purpose before introduction
Before AI reaches the line, its purpose belongs on paper: what is it intended for, what explicitly not, which data does it use, and who is responsible for it? This intended purpose is the basis for everything else. Without it, you can neither classify whether it is a high-risk system nor test whether it does what it should.
In an AI management system under ISO/IEC 42001, this is not extra effort but the core: every AI application is recorded and has a purpose, a risk assessment and an owner.
The team has to understand the AI
Since February 2025, the EU AI Act has required companies that use AI to ensure their staff have sufficient AI literacy. On the line, that does not mean theory. It means that everyone who works with the system knows what it is intended for, where its limits are and when not to follow it. A short, practical briefing at the workstation achieves more than a generic online course.
What follows
- Record every AI on the line, including the AI embedded in purchased software.
- Write down the intended purpose and base the classification under the EU AI Act on it.
- Make oversight usable: reason visible, rejecting as fast as accepting, decision recorded.
- Brief the team at the workstation, with the real system.
That turns an assistant that is usually right into a tool whose use you can prove.