The Business Is Yours. The Algorithm Is Not in Charge.

By Irena Popova

The corporate world has found a remarkably accommodating adviser. It processes information at extraordinary speed, produces elaborate recommendations, generates persuasive explanations and occasionally displays the confidence of a consultant who has never met the client, examined the company's financial obligations or experienced the consequences of a poor decision. Its qualifications are rarely questioned with the same intensity as those of its human counterparts. After all, the system is intelligent, the interface looks sophisticated and the results arrive neatly organised. What more could a modern organisation possibly require? Perhaps a little curiosity about whether the information is reliable, the assumptions defensible and the proposed course of action economically sensible. An inconvenient detail, admittedly, when the presentation is already so convincing.

I find the language surrounding artificial intelligence almost as revealing as the technology itself. We speak of systems that understand customers, develop strategies, evaluate opportunities and make decisions, often without distinguishing the computational operations being performed from the human capacities these expressions traditionally describe. Metaphors are useful, particularly when we need to communicate complex technical processes. The trouble begins when figurative language quietly acquires the status of explanation. A system capable of producing a coherent recommendation has demonstrated a particular technical capability. Whether that recommendation deserves authority in a commercial decision is another matter entirely. The distinction may appear philosophical until someone begins allocating capital, adjusting prices or making contractual commitments on the strength of an output whose underlying assumptions nobody has adequately examined. At that point, semantics develops a rather practical relationship with the balance sheet.

Consider an automated pricing system instructed to improve sales conversion. It analyses purchasing patterns, recommends adjustments and begins producing the desired results. Transaction volumes increase, conversion rates improve and the implementation appears to have justified the investment. Management may even have the pleasure of presenting an impressive collection of performance indicators. Yet if the pricing adjustments reduce contribution margins faster than additional sales compensate for them, the company's economic position may deteriorate despite the apparent operational success. There is no necessary contradiction in the technology's performance. The system may have fulfilled its assigned objective with considerable precision. What deserves scrutiny is the managerial judgment that selected the objective, determined the acceptable constraints and interpreted the outcome. A correct calculation cannot rescue a poorly conceived strategy. It can, however, execute that strategy with admirable consistency.

This is why I am reluctant to confuse automation with economic progress. Reducing the time required to complete an activity is useful when the activity creates sufficient value, the resulting output remains reliable and the costs of implementation, supervision and correction are justified. Without those conditions, increased speed can become an unusually expensive form of organisational enthusiasm. A company may automate customer communication only to discover that poorly defined service policies now produce confusion on a much larger scale. It may generate more marketing content without improving qualified demand, or accelerate reporting while preserving the inaccuracies already present in its data. None of these outcomes makes automation inherently undesirable. They illustrate something more fundamental about business management: technology does not automatically repair the economic logic of the process into which it is introduced. Sometimes the most significant improvement would have been to reconsider the process before purchasing the software intended to accelerate it.

My concern is therefore not that computational systems are becoming increasingly capable. I regard their development as an extraordinary technical achievement, with substantial potential to improve analysis, productivity and access to information. What concerns me is the intellectual complacency that can accompany their adoption. The ability to delegate execution is a valuable organisational advantage; allowing that delegation to obscure the reasoning behind consequential decisions is a different proposition. Software developers, technology providers and deploying organisations may have distinct responsibilities, but the existence of those relationships does not relieve management of the need to establish objectives, assess commercial risks and maintain meaningful oversight. The greater the operational influence of automated systems, the more important those responsibilities become. Otherwise, we risk creating companies whose technological sophistication advances considerably faster than their capacity for judgment. The resulting difficulties may be attributed to the algorithm, but responsibility does not disappear merely because its exercise has become inconvenient.

The business remains yours. Its direction, its obligations and the consequences of its strategic choices cannot be transferred to a convenient abstraction called artificial intelligence. And if a computational system has acquired more practical authority over your enterprise than the people entrusted with managing it, I would begin by examining the decisions that made such an arrangement possible. The algorithm may be the newest participant in the organisation, but it did not appoint itself to the board.

The Intellectual Luxury of Not Having to Think

Human beings have always searched for ways to reduce the effort required to understand the world. We invented instruments to extend our senses, mathematics to formalise relationships and computers to perform calculations far beyond our individual capabilities. These achievements deserve admiration. What interests me, however, is the peculiar enthusiasm with which we now celebrate the possibility of outsourcing not only intellectual labour but also the inconvenience of intellectual doubt. Why examine competing explanations when a system produces an elegant conclusion? Why investigate uncomfortable evidence when a confident recommendation arrives accompanied by charts, percentages and the reassuring appearance of scientific precision? We have developed an extraordinary talent for confusing access to information with possession of knowledge. The distinction matters because knowledge demands more than receiving a plausible answer. It requires understanding the grounds on which a claim rests, recognising what remains uncertain and possessing sufficient intellectual independence to challenge the conclusion. I am not proposing that every business owner become a mathematician, software engineer or philosopher before purchasing a technological service. I am questioning a corporate culture that increasingly treats the absence of critical examination as evidence of operational sophistication. An executive who accepts an automated recommendation without understanding its decisive assumptions has not eliminated the intellectual work involved in judgment. That work has simply been left undone. The dashboard remains impressive, of course. It is considerably easier to admire a visualisation than to investigate the reasoning that produced it. And the most convenient feature of an unquestioned recommendation is that it relieves us of the immediate discomfort of having to disagree. Unfortunately, reality has never shown much respect for conclusions accepted merely because they were convenient.

When Yesterday's Assumptions Become Tomorrow's Evidence

I find the relationship between historical data and organisational prejudice particularly revealing. Consider a company that introduces automated customer evaluation to identify its most commercially promising prospects. The system analyses previous transactions, estimates conversion probabilities and assigns priorities. Its recommendations appear rational because they reflect measurable patterns. But those historical patterns did not emerge in a vacuum. They were shaped by earlier marketing decisions, customer selection, pricing policies and differences in the attention various clients received. A customer segment that was rarely approached in the past will naturally provide less evidence of successful transactions. If the new system interprets this absence of historical success as proof of inferior commercial potential, those customers receive even less attention. The resulting sales figures subsequently reinforce the original classification, producing a self-confirming cycle in which the organisation becomes increasingly certain of assumptions it has never properly tested. What an exquisite arrangement. We begin with a managerial preference, reproduce it through a statistical model and eventually celebrate the model for independently confirming our original judgment. The mathematics can be internally consistent while the interpretation remains deeply misleading. I consider this distinction fundamental because business intelligence is not measured by how confidently an organisation reproduces its previous behaviour. It is measured by its ability to discover what that behaviour has concealed. A company that automates its assumptions without examining their origins does not necessarily acquire a more objective understanding of the market. It acquires a faster, more systematic and technologically sophisticated method of remaining convinced that it was right all along. At least the old-fashioned prejudice had the decency to look subjective.

The Reality That Never Entered the Dataset

One of the most persistent fantasies surrounding data-driven management is the belief that a sufficiently sophisticated analytical system will eventually account for everything that matters. The proposition is attractive because it transforms the complexity of business into a problem of computational capacity. Collect more information, improve the model, expand the infrastructure and the inconvenient uncertainty of human affairs will supposedly surrender to statistical elegance. Yet a dataset is not reality. It is a structured representation of selected observations, shaped by decisions about what to record, how to categorise it and which relationships deserve attention. The distinction becomes painfully important when organisations begin interpreting the absence of information as evidence that nothing significant exists beyond the recorded variables. A client relationship can deteriorate long before the financial indicators reveal the problem. An apparently profitable contract can create operational dependencies that accounting categories fail to capture adequately. A market forecast can demonstrate impressive statistical accuracy while overlooking a structural change that has rendered its historical assumptions obsolete. As someone whose work crosses language, computing and business analysis, I find it intellectually indefensible to reduce understanding to the successful manipulation of available representations. Human judgment is not infallible, and experience offers no automatic protection against error. That is precisely why analytical systems are valuable when they challenge our intuitions and expose what we have overlooked. But the relationship must work in both directions. Data should interrogate assumptions, while informed judgment should interrogate the data. The ambition to make every business decision measurable is understandable; the belief that everything important has already been measured is considerably harder to defend. We can build an extraordinarily precise instrument and still discover that we have been measuring the wrong thing. The tragedy is not mathematical. The tragedy begins when nobody possesses the intellectual curiosity to notice.

Human Oversight: A Signature at the End of the Process?

The phrase human oversight has become one of the more comforting expressions in the contemporary vocabulary of technological governance. It suggests that somewhere between the algorithmic recommendation and its consequences stands a responsible individual, carefully examining the evidence and exercising independent judgment. A reassuring picture, provided we resist the temptation to inspect the actual working conditions. What happens when an employee must approve hundreds of automated recommendations within a limited period? What kind of oversight exists when the person responsible for reviewing a decision cannot access the relevant information, challenge the assumptions or explain the system's reasoning? And what exactly has been accomplished when a human signature is attached to an outcome that nobody has the practical authority to change? I regard such arrangements as administrative theatre rather than meaningful accountability. The presence of a person in a workflow does not establish the exercise of judgment, just as the presence of a signature does not establish informed consent. Genuine oversight requires competence, access to evidence, clearly allocated responsibilities and the ability to intervene before consequential errors become irreversible. This also means accepting that human supervision carries costs, demands expertise and cannot always be compressed into the frictionless efficiency promised by an enthusiastic software demonstration. Organisations that eliminate critical expertise in the name of automation and subsequently rely on that same expertise to investigate failures have achieved a particularly remarkable form of strategic contradiction. They have declared judgment unnecessary while preserving the expectation that someone will exercise it when the system fails. I refuse to call that intelligent management. A company cannot delegate its entire capacity for critical examination and then invoke human responsibility as a decorative feature of its governance policy. The question is not whether a human remains somewhere in the process. The question is whether that human still possesses the knowledge, authority and independence necessary to make a difference. Without those conditions, oversight becomes a ceremony, and responsibility an obligation conveniently rediscovered after the damage has been done.