Here's What Limits AI's Outlook For Replacing Knowledge Workers

By Richard Moroney, Ph.D., Stat Consulting

Most of us involved in manufacturing operations will have personal experience with process improvement programs, likely with some successful and others not. Improving manual work, though, is significantly different than improving knowledge work. See part 1 of this article for more background on these two types of work.

I had a client bring me in to support making their R&D processes leaner for new product development. This was initially difficult because we were not the first consultancy the client had engaged for this task. The prior team had failed miserably by jumping right in and following the staff scientists through their day’s activities with a stopwatch and notepad — they immediately lost the trust and engagement of the staff. When we were finally engaged we first rebuilt trust with the staff, then our team was able to engage them in more useful activities (such as value stream mapping) and set up new ways of working together so the R&D staff could reach the improvement objectives.

You have probably already guessed the single most important thing to improve knowledge work: support and empower the people doing the work. The very nature of knowledge work means failure is always an option, so if your knowledge workers are afraid to fail for any reason, they will be afraid to do the hard work. Knowledge workers will need to gather information from many diverse sources, so if collaboration is weak, the output of the work will also be weak. And poor collaboration may also lead to poor group decision-making; at the extreme, group think is deadly to effective knowledge work.

Once the environment is properly set up, knowledge work can be improved further by converting existing ad-hoc knowledge processes into documented quality processes when possible. When standardizing a process for the first time, you will identify through the documentation process where the really hard parts are and then focus the knowledge work activity there. Reflecting on the business strategy example in part 1, we might come up with a standard process and form for documenting our competitors in the market. This effort can then be delegated or outsourced to others, and the team’s activity can focus on decision criteria and a weighting scale to support decision-making (e.g., of a product to develop or a competitor to acquire).

Recognizing that knowledge work is also the reservoir for the “undocumented knowledge” within your business, a few additional tips may be useful when documenting decision processes:

• Identify cases to automatize common decisions through documentation in order to speed the common workflows.
• Include clear indicators for the “this is not a common case” situation and actions to take in that scenario (e.g., “call a group meeting and discuss it”).
• Recognize that you can’t document every option (e.g., the “this is not a common case” outlet in your process case statement) without turning your document into a procedure that is unfollowable and impossible to keep updated.

Looking To The Future And The Role Of AI

Can AI help us now with knowledge work and improvement of these processes? Unfortunately, we already have some dramatic negative examples suggesting this will be harder than we might expect. Boeing’s 747 Max control system as originally designed unfortunately overruled the pilot’s expertise when there was a sensor failure, with catastrophic consequences. We could blame the hardware failure as the root cause, but the control software was unable to effectively support the pilot in this apparently unanticipated scenario.

This example highlights another problem of many AI systems today: they function like a black box and we have no visibility of the process details they might use in making a decision for us. This makes them effectively unvalidatable in any sort of unbounded problem.  While we do have ways of using other software that is effectively unvalidatable (e.g., Microsoft Excel), the approach requires controls (e.g., strict version control and validated worksheets) that don’t appear to be available yet with the new general LLM tools.

Another issue for AI systems today is they are being developed in a very Wild West, dynamically competitive environment. The current market drive is for “exciting and flashy” as companies race for both press and funding. Some high-profile cases have shown how AI hallucinations have crept into their outputs, and researchers are even finding there can be active deception by the AI system in pursuit of a requested goal.

These objections are short-term issues, though, and the many smart people working on developing these tools will be able to fix the issues we know about and can identify. Let’s look closer, then, at four key elements of knowledge work and how they might be supported by AI systems in the future. Knowledge work:

• Draws on both factual and theoretical knowledge to impact decision-making. In this “gather and share” type of task, AI is already showing great promise. However, best practice generally requires good sourcing of any reference material, so until all chances of hallucinations are removed, any use of AI outputs must be done with great editorial care and caution.
• Sorts through all the chaff to find the important few facts or details that will be used to guide a specific decision. In this filtering task AI is again already showing great promise, and development of bespoke AI systems within an organization appear poised to improve productivity sooner rather than later. In general use systems, though, we must watch out for the commercialization failure patterns that may arise (e.g., Google’s “I’m feeling lucky” button, which was undermined by SEO and advertising activity).
• Encourages communication and dialog around key decisions. Knowledge work is necessarily a somewhat social activity since a group typically must agree on the work and the delivery of the results. AI has not been helping here yet, but possibly will pick up some ability to help teams share the full context when developing or delivering results.
• Motivates and inspires the team. Knowledge work, as distinct from manual work, is intrinsically motivating and inspiring to the team. Bonding around the active work required to make decisions is an essential part of corporate culture. AI is not helping here and is likely to fail on this element in the future. If we were to rely on AI for a decision, it undermines the team’s commitment to an action by eliminating the intimacy around the final decision and ownership of the chosen direction.

Looking To The Future

With competition and continuous improvement, we must accept continuous change. AI has reached an inflection point and is here to stay. It will certainly transform many of our work procedures and methods, and it can already provide some complementary support to knowledge workers. However, rest assured that it is not close to replacing the many decision-making tasks of knowledge work at this time.

About The Author:

Richard Moroney got his Ph.D. in electrical engineering from UC Berkeley and brings an additional 20 years of practical experience helping clients in pharmaceutical and medtech development and manufacturing. He specializes in debugging measurement and quality systems to ensure great decision making (Six Sigma Black Belt) and ensuring your quality systems are not a burden on business (Certified Pharmaceutical GMP Professional). Richard has focused mostly on the statistics and data science aspects of Quality 4.0, with clients ranging from start-ups through Fortune 100, as well as the U.S. government.  He founded Stat Consulting in 2024.