I am preparing for a discussion during at event hosted by Morgan Phillips on the question: “Human in the era of AI, an obstacle or the best competitive advantage?” The debate may be framed as a binary choice, which is a mistake. Human is and will remain the definitive competitive advantage, but only if leadership evolves. Treating AI as an autonomous employee is a strategic error — it is a tool that generates liability, especially at the current state of its development. The leader’s job is no longer to manage people but to architect an operational model that combines AI’s scale with human judgment. The human is not an obstacle in this system, but the primary control. AI has its own serious drawbacks which, in my opinion, can’t be quickly resolved.
The Briefing#
EU AI Office Defines “Serious Incident” Reporting Rules#
The European Commission has released draft guidance detailing how providers of high-risk AI systems must report “serious incidents” under Article 73 of the EU AI Act. The guidance, open for consultation until 7 November 2025, significantly clarifies the scope of corporate liability. A reportable “serious incident” is broadly defined to include not only physical harm but also any infringement of fundamental rights under EU law. Notably, the guidance states that an indirect causal link between the AI system and the harm is sufficient to trigger a report. The reporting deadlines are exceptionally tight: 15 days for most incidents, shrinking to just two days for a “widespread” infringement of fundamental rights or a serious disruption to critical infrastructure. Non-compliance carries potential fines of up to €15 million or 3% of global annual turnover.
ENISA Report: AI Now Drives Over 80% of Social Engineering Attacks#
The European Union Agency for Cybersecurity (ENISA) published its 2025 Threat Landscape report, revealing that AI-supported phishing and social engineering campaigns now constitute over 80% of all observed social engineering activity. The report identifies phishing as the most common method for initial intrusions, accounting for 60% of breaches. Adversaries are reportedly using advanced techniques, including jailbroken large language models and synthetic media (deepfakes), to create highly effective, personalised attacks at scale, making it harder for employees and technical filters to detect malicious attempts.
Bank of England Warns of Financial Stability Risks from AI Bubble#
An analysis from the Bank of England has highlighted growing financial stability risks stemming from highly concentrated and elevated AI-related asset valuations. As of October 2025, AI-related stocks account for approximately 44% of the S&P 500’s market capitalisation, a level of concentration exceeding the dot-com bubble’s peak. The report notes a critical shift in financing, with a projected $2.9 trillion in AI infrastructure spending between 2025 and 2028 expected to be heavily reliant on external debt, including an estimated $800 billion from private credit markets. This transforms the AI boom from an equity story into a systemic credit risk. The warning coincides with a Bank of America survey in which a record 54% of global fund managers identified AI stocks as being in a bubble.
A Leader’s Guide to Human-AI System Design#
Enterprise AI deployment requires a new leadership model. Case studies have shown that pursuit of full autonomy in most environments is an error. The optimal model is a hybrid one, built upon a clear division of labour and governed by human oversight.
The Autonomy Liability: The Case for a New Model#
The evidence that AI cannot be treated as an autonomous agent is unambiguous. The weaknesses of current models are inherent properties, and they create direct liability.
Hallucination and Fabricated Information
AI’s capacity to generate confident but false information creates direct legal and financial liability.
In 2024, an Air Canada chatbot invented a bereavement fare policy. A Canadian tribunal held the airline legally responsible for the AI’s fabrication, setting a clear precedent: the organisation is accountable for the outputs of its automated agents.
In 2025, multiple instances of legal professionals submitting court filings containing AI-generated, non-existent case citations were documented, exposing practitioners to sanctions and reputational damage.
Lack of Contextual and Physical Understanding
AI systems lack comprehension of the physical world, leading to unpredictable outcomes when given control over physical assets.
- The suspension of Cruise robotaxi operations in 2023, after a vehicle struck and dragged a pedestrian, proved that AI systems lack the real-world comprehension required for autonomous, high-stakes decisions. That is also the probable reason for delay in Tesla’s Robotaxi’s delay.
Operational Incompetence and Brand Damage
Deploying immature AI in customer-facing roles results in operational failure.
- Both McDonald’s and Taco Bell (2024-2025) re-evaluated automated drive-thru systems. The AI frequently misinterpreted orders, creating nonsensical combinations that became viral content, forcing the companies to withdraw the technology.
Systemic Bias and Opaque Decision-Making
Algorithms trained on historical data perpetuate and scale the biases within that data, creating compliance and reputational risks.
- An algorithm for the Apple Card assigned lower credit limits to women than to men, even when they shared finances. The opaque decision-making triggered regulatory investigations.
These cases prove that “AI accountability” is a fiction. In every failure, the liability—legal, financial, or reputational—reverts to the organisation and its leadership.
Framework 1: The Human-AI Division of Labour#
The dominant and most valuable application of AI is augmentation, not replacement. The leader’s role is to design a new operational model where each party performs the tasks for which it is best suited (and most economically effective). This frees employees from repetitive work to focus on activities requiring human skills: complex problem-solving, decision making, strategic thinking, and empathetic engagement.
An effective hybrid model requires a pragmatic understanding of the distinct competencies of AI and human workers.
| Capability Domain | Optimal Agent: AI | Optimal Agent: Human | Key Risk if Mis-allocated |
|---|---|---|---|
| High-Volume Data Processing | Analyses millions of transaction records in real-time to flag statistical anomalies. | Reviews a curated list of the highest-risk anomalies, applying contextual business knowledge. | Assigning investigation to AI risks high false positives and missed context. |
| Rule-Based Execution | Automates processing of 95% of standard invoices from a consistent format. | Handles the 5% of invoices that are non-standard, damaged, or contain exceptions. | Over-reliance on AI leads to process failure when exceptions occur. |
| Strategic Decision-Making | Generates multiple market-entry scenarios based on historical data. | Evaluates the AI-generated scenarios, assesses qualitative risks, and makes the final decision. | Delegating strategic decisions to AI abdicates leadership and ignores non-quantifiable factors. |
| Customer Interaction | Provides instant answers to common, factual queries (e.g., “What are your opening hours?”). | Manages complex, sensitive, or high-value customer complaints that require empathy and negotiation. | Using AI for sensitive interactions damages customer relationships. |
| Ethical Judgment | Flags potential conflicts of interest in a dataset based on programmed rules. | Investigates the flagged conflicts, understands the nuanced ethical implications, and determines the course of action. | Assigning ethical judgment to an AI system creates a severe compliance and reputational risk. |
Framework 2: Human Oversight as a Structural Control#
Human oversight is often incorrectly framed as a temporary measure. it is a permanent and necessary structural component of any responsible AI system. This permanence is required because AI’s limitations—its lack of genuine understanding and its incapacity for ethical judgment—are inherent. An AI model trained on yesterday’s data cannot be trusted to navigate tomorrow’s novel challenges without human governance.
Operationalising Human Oversight: Case Studies
Pragmatic models for embedding human oversight are already delivering value.
Finance: AI Orchestration for Pricing: Leading financial firms use a model termed “AI Orchestration.” For tasks like competitive pricing, multiple AI models are prompted with the same query. If the variance between answers exceeds a predefined threshold (e.g., 8%), the task is escalated to a human expert for a final decision.
Logistics: Automated Document Processing: AI platforms automate the high-volume processing of freight documents. The workflow is designed for exception handling. When the AI encounters a non-standard document or returns a low confidence score, it is routed to a human operator. This hybrid approach has enabled 99% data accuracy while reducing processing costs by 50%.
Compliance: Anti-Money Laundering (AML) : AI systems monitor millions of transactions to flag suspicious activity. The AI’s role is limited to flagging. A human analyst must investigate the alert, apply contextual knowledge, and make the final determination.
A Spectrum of Control: HIC, HITL, and HOTL
Leaders require a framework more granular than the generic term “Human-in-the-Loop.” This spectrum allows oversight to be calibrated to the risk level of the application.
Human-in-Command (HIC): The AI system can only propose actions. A human must provide explicit authorisation before any action is executed.
Human-in-the-Loop (HITL): The human is an active and required participant. The AI must stop at critical, predefined junctures to await human review, validation, or correction.
Human-on-the-Loop (HOTL): The AI system operates autonomously. The human operator monitors the system’s overall performance and can intervene or override it.
This taxonomy provides a defensible framework for designing control systems.
| Oversight Model | Definition | Level of Human Control | Typical Application in a Regulated Industry |
|---|---|---|---|
| Human-in-Command (HIC) | AI proposes; human authorises action. | Maximum / Veto Power. | Final approval for a new medical drug; authorisation of an autonomous surgical procedure. |
| Human-in-the-Loop (HITL) | Human is an active participant; must validate at critical checkpoints. | High / Active Validation. | Review of AI-flagged insurance claims above a set value; final approval of a large corporate loan. |
| Human-on-the-Loop (HOTL) | AI operates autonomously; human supervises and can intervene. | Moderate / Supervisory. | Real-time fraud detection systems that automatically block small transactions; monitoring algorithmic trading. |
The Leader as Orchestrator#
The convergence of AI’s capabilities and its limitations necessitates a shift in executive leadership. The most important skill is shifting from direct management to the design, orchestration, and governance of complex, hybrid human-AI systems. The leader’s focus elevates from supervising people to architecting the processes within which both people and AI operate. This is a non-delegable responsibility. It includes formally mapping business processes, setting the decision thresholds that trigger human intervention, ensuring all systems are auditable, and fostering a culture where employees are encouraged to question and override AI-generated outputs. This transforms the human workforce into the system’s primary line of defence.
Questions for Your Leadership Team#
On Process Design: Have we formally mapped our key processes to determine which tasks are purely for AI, which are purely for humans, and where are the critical handoff points?
On Risk Calibration: For our highest-risk AI applications, have we defaulted to a “Human-in-Command” or “Human-in-the-Loop” model? How do we justify anything less?
On Auditability: Can we prove, at any moment, why our AI made a specific decision? Is every AI decision and subsequent human intervention logged for review?
On Culture: Have we clearly communicated to our teams that they are expected to challenge and override AI-generated outputs, and that doing so is a core part of their job, not a failure of the system?
On Problem Selection: Are we starting with a specific, costly business bottleneck (e.g., “What is our most expensive process?”) rather than the vague question, “Where can we use AI?”
⠀
Conclusion#
AI is a tool, not a colleague. The necessary response to its limitations is to redesign the operational models through which it is deployed. This new context elevates the role of human leadership. Leadership value shifts from managing the “how” of work to defining the “what” and the “why.” The leader’s contributions are judgment, ethical foresight, and the strategic intent required to design the entire system. Achieving an “AI Equilibrium” is the establishment of a dynamic, resilient balance—a state where the computational power of AI is fused with, and governed by, the contextual understanding and accountable judgment of human leaders.
Until next time, build with foresight.
Krzysztof