As organizations more broadly adopt artificial intelligence, routine tasks are being automated and some tasks are being moved from humans to AI systems. However, many companies find it difficult to reap the benefits of investing in such systems.
According to a visiting senior lecturer at the MIT Sloan School of Management, the problem is not the adoption of technology itself, but how organizations adapt their ways of working to accommodate the use of that technology.
“There is an interdependence between these three elements: the work itself, the workforce and the workplace,” he said. “If an organization is unable to align these three elements, it will have difficulty producing measurable impact within a reasonable period of time.”
McDonough-Smith teaches MIT Sloan’s new executive education course, “AI Essentials: Accelerated Impactful Adoption.” This course is designed to help leaders transform AI into impact by understanding how to redesign their jobs and build the metrics and playbook they need to implement.
In a recent interview, McDonough-Smith shared some ways organizations can bridge the gap between AI’s potential and real-world impact.
See AI as an operating system, not a toolkit
Many companies still treat AI as something that is integrated into existing workflows, often to improve efficiency. However, this approach is too narrow.
“Too many organizations think of AI as a toolkit,” McDonagh-Smith says. “They don’t see AI as an operating system.”
This distinction is important. When AI is treated as a tool, it is layered onto existing processes and measured using outdated metrics. This makes it difficult to measure the impact, even when value is being created. As a result, AI will be deployed piecemeal rather than as part of a coherent system.
Adopt an exploration and evolution mindset
Understanding the model is still important, but it’s not enough.
McDonough-Smith explained how AI will develop in stages, with each stage addressing its final limitations. Early systems relied on fixed rules created by humans. These have been replaced by models that learn from data, known as machine learning models. Modern generative AI systems can process language, images, and other complex information.
Each advancement solves a problem, so understanding the limitations of today’s systems, such as their reliance on vast amounts of data and computing power, can help organizations understand what comes next.
But organizations need to go beyond technical knowledge, McDonough-Smith said. “We need to move from models to a mindset of exploration and evolution.”
That means putting AI into practice, testing it on real problems, and seeing what works.
Reconsider how you work
The introduction of AI will impact the structure of work.
McDonough-Smith said jobs are no longer the appropriate unit of analysis for understanding work. “Gone are the days when you could think of a job role as a unit of measurement,” he said. Instead, “You need to break down the job into its component parts. There are probably 15 to 20 core activities.”
AI changes its work from task to task. Work is divided between people and systems, so some are automated and others are enhanced.
This means organizations need to redesign workflows around tasks, rather than layering AI onto existing roles. Managers need to map how work is actually done and define how tasks will be divided between humans and AI.
AI Essentials: Accelerate effective adoption
join online
Register now
Adopt new performance metrics
Traditional performance metrics often fail to capture the value of AI.
“The metrics and units of value that we have inherited will not necessarily remain the same,” McDonough-Smith said. Instead, organizations need new ways to benchmark performance, including the speed of decision-making, human-AI collaboration, and how quickly insights are fed back into the business.
Other measures focus on outcomes such as improved decision-making quality, the degree of autonomy an AI system has, and how well an AI system “understands” the organization.
A starting point is to define a small set of AI-specific metrics within your core workflows and use them to test where value is being created.
Bridging the “last mile” gap
The central challenge in AI adoption occurs in what McDonough-Smith calls the “last mile,” the point at which an AI system cannot translate into business value. “AI last mile engineering is fundamentally about closing the distance between AI potential and real-world impact,” he said.
In many organizations, AI efforts are stalled because models are functional but not used in daily decision-making or workflows.
To close this gap, companies need to take a structured approach.
- Start by identifying the problem to be solved. “Organizations have one thing in common: they need to clearly define the problem they are trying to solve with AI,” McDonough-Smith said.
- Involve users in the design. Systems should be built with the people who use them, not imposed.
- Focus on context. Rather than relying solely on computing power, prioritize how work is actually done.
- Tests and scales. Progress comes from testing small applications, measuring results, and extending what works. “Ship small and learn fast,” McDonough-Smith said.
- Build trust with your users. Trust must be built from the beginning with governance and ongoing monitoring.
Watch the webinar: “From Model to Mindset, the Last Mile of AI Adoption”
Explore the course — AI Essentials: Accelerating effective adoption
paul mcdonagh smith He is a senior visiting lecturer in information technology at MIT Sloan. In his research and teaching, he creates important intersections between technology and business. He specializes in translating computer and data science into measurable business value that advances organizational capabilities, transformation, and strategy.
