AI Management System Certification According to the ISO/IEC 42001 Standard

Your Progress: 0%

Chapter 1 of 11

Chapter 1: Artificial Intelligence: A Transformational Technology

Chapter 1.1: Introduction – From Academic Curiosity to Mainstream Force

Not very long ago, the subject of artificial intelligence was confined almost exclusively to the quiet corridors of academic research institutions and the experimental laboratories of technology companies. Discussions about AI were largely theoretical, speculative, and often dismissed as the stuff of science fiction films like The Matrix or Her. The average business leader, policymaker, or citizen rarely encountered AI in their daily professional or personal life, and when they did, it was usually in the form of a novelty rather than a transformative tool. However, within a remarkably short period, this perception has been completely upended. Artificial intelligence has decisively moved from the margins to the very center of mainstream society, becoming a dominant topic of conversation among chief executive officers, business strategists, government policymakers, entrepreneurs, and even the general public.

This dramatic shift is not merely a matter of hype or media fascination. According to surveys conducted by leading global consultancies such as McKinsey and PwC in 2023, more than half of all CEOs believe that artificial intelligence will have a greater and more profound impact on the world than the internet itself. An even more striking figure emerges when we look at near-term expectations: approximately 80 percent of CEOs surveyed stated that AI will significantly and fundamentally change the way business is conducted within the next five years. These are not the predictions of technology enthusiasts or futurists speaking in abstract terms; these are the assessments of the men and women who run the world's largest corporations and who are accountable for billions of dollars in revenue and millions of employees. When such a broad consensus forms among senior leadership, it signals that a genuine structural transformation is underway.

Chapter 1.2: AI as the New Electricity – A General-Purpose Technology

To truly grasp the magnitude of what is happening, it is helpful to consider the analogy offered by Andrew Ng, one of the most respected scientists in the field of machine learning. Ng has argued that the impact of artificial intelligence on human society will be comparable to the impact that electricity had on world civilization in the late nineteenth and twentieth centuries. When electricity was first discovered and eventually made accessible to businesses and households, it did not merely add a new convenience to existing ways of doing things. Instead, it became the foundational engine that fueled innovation across every single segment of the economy. Electricity lit up factories, powered new forms of transportation, enabled modern communication, and sparked scientific development on an unprecedented scale. In much the same way, artificial intelligence is now positioned to become the critical source of innovation that will profoundly shape the global economy for decades to come.

The term that economists and technology scholars use to describe a technology with this kind of sweeping, cross-cutting impact is a "general-purpose technology" or GPT. Other historical examples of general-purpose technologies include the steam engine, the internal combustion engine, the electric grid, and the internet. What distinguishes a GPT from a more specialized innovation is that it does not remain confined to a single industry or application. Instead, it diffuses throughout the entire economy, transforming existing sectors, creating entirely new industries, and reshaping the very nature of work and daily life. Artificial intelligence, and particularly the new generation of foundational models and generative systems, fits this definition precisely. It is not simply another software tool or app; it is a new layer of intelligence that can be applied to virtually any domain where data exists and decisions must be made.

Chapter 1.3: The Definition and Origins of Artificial Intelligence

Before we can fully appreciate the scope of the AI revolution, we must first establish a clear conceptual foundation. What exactly do we mean when we say "artificial intelligence"? The term itself was first coined by the American cognitive scientist and inventor John McCarthy in a conference paper presented in 1956 at the Dartmouth College Conference, an event that is widely considered the founding moment of AI as a formal academic discipline. McCarthy's original ambition was both bold and precise: he sought to create a machine that could mimic the activities a human brain performs during the learning process. In McCarthy's vision, by automating the different learning tasks that a human brain goes through, we might be able to program a machine to learn and behave in ways similar to humans.

Over the decades that followed, other influential thinkers offered their own definitions and interpretations. The American professor of philosophy and cognitive scientist John Haugeland, writing in 1985, defined AI as "the exciting new effort to make computers think … machines with minds, in the full and literal sense." In this definition, Haugeland was expressing the belief that AI in the future could acquire capabilities that are similar to those humans possess—capabilities such as abstract thinking, emotions, feelings, and consciousness. A different perspective was offered by Ray Kurzweil, the American inventor and futurist, who argued in 1990 that "AI is concerned with building machines that can act and react appropriately, adapting their response to the demands of the situation." Kurzweil's definition adopts a more behavioral approach, focusing on the ability of machines to act intelligently in response to their environment rather than on the internal mechanisms of thought.

What all of these definitions share is the idea that AI represents an effort to replicate or augment human cognitive capabilities through computational means. Whether we focus on the process of thinking, the manifestation of intelligent behavior, or the adaptation to changing circumstances, the common thread is the aspiration to create machines that can perform tasks and solve problems that were previously the exclusive domain of human intelligence.

Chapter 1.4: The Historical Evolution of Artificial Intelligence

The evolution of artificial intelligence as a scientific discipline has been anything but linear. Since the 1950s, the field has gone through a series of distinct historical phases, each characterized by periods of intense optimism and rapid progress followed by what researchers call "AI winters"—extended periods of reduced funding, diminished public interest, and skepticism about the feasibility of the original goals. Understanding this history is important because it provides context for the current moment of accelerated development and helps us avoid the trap of assuming that progress will always be smooth and uninterrupted.

The early decades of AI research, from the 1950s through the 1970s, were dominated by what is known as symbolic AI or rule-based systems. Researchers in this era believed that human intelligence could be replicated by explicitly encoding knowledge into logical rules and symbols that computers could manipulate. These early systems showed promise in narrow domains such as playing chess or solving mathematical theorems, but they struggled with the messiness and ambiguity of the real world. The 1980s saw a resurgence of interest with the development of expert systems, which attempted to capture the decision-making processes of human specialists in fields like medicine and engineering. However, these systems proved expensive to maintain and difficult to scale, leading to another period of disillusionment.

The modern era of AI, which began in the 2000s and accelerated dramatically in the 2010s, is built on a fundamentally different approach. Rather than relying on hand-coded rules, contemporary AI systems use machine learning and deep learning techniques to identify patterns and make predictions from large datasets. This shift was made possible by three converging factors: the explosion of digital data available for training, the dramatic increase in computing power provided by graphics processing units and cloud infrastructure, and the development of more sophisticated neural network architectures. It is this modern approach that has produced the breakthroughs in image recognition, natural language processing, and generative AI that now dominate headlines and reshape industries.

Chapter 1.5: The Three Types of Artificial Intelligence

In the field of AI research and discourse, it is essential to distinguish between three fundamentally different types or levels of artificial intelligence. These are Artificial Narrow Intelligence (ANI), Artificial General Intelligence (AGI), and Artificial Super Intelligence (ASI). This distinction is not merely semantic; each type represents a radically different level of capability, autonomy, and potential impact on society. It is also crucial to note that at this current juncture in history, humanity has only successfully developed and deployed narrow AI. General AI and super AI remain theoretical constructs, though the rapid pace of technological advancement has made discussions about their eventual emergence more urgent and less speculative than ever before.

Chapter 1.6: Artificial Narrow Intelligence (ANI) – The AI We Know Today

Artificial Narrow Intelligence, often referred to as "weak AI" or "narrow AI," is the type of AI that exists all around us today. It is designed to perform a very precise and well-defined task, operating according to a set of rules and a set of predefined parameters. The "narrow" designation reflects the fact that such systems do not go beyond what they have been programmed or trained to do. They excel within their designated domain but cannot transfer their capabilities to unrelated tasks or contexts.

Familiar examples of narrow AI include the language processing programs that many of us interact with daily, such as Apple's Siri and Amazon's Alexa. Another example is the AI algorithm behind search engines—Google's RankBrain, for instance, which is used to map queries, organize search results, and rank web pages according to specific parameters. These applications, while they may appear highly intelligent in their interactions with us, actually perform in a very narrow fashion. Apple's Siri, for instance, while it may seem to know everything, interacts only within the boundaries of what has been provided to it by its underlying datasets. In other words, it cannot go beyond the parameters from which it reads. That is why Siri, or any other language processing application, is unable to understand language subtleties such as humor, jokes, sadness, or emotion in general.

It is important to recognize that the generative AI systems that have captured the world's imagination—such as ChatGPT, DALL-E, and Midjourney—are still, in their current form, instances of narrow AI. They are highly sophisticated and versatile narrow AI systems, capable of remarkable feats of content generation, but they remain fundamentally limited by their training data and their inability to genuinely understand the meaning of what they produce.

Chapter 1.7: Artificial General Intelligence (AGI) – The Next Frontier

The second type of AI is Artificial General Intelligence, also known as full AI or strong AI. This type of AI has the potential to mimic human cognition and the decision-making processes that a human brain goes through. It is the type of AI that we commonly see depicted in science fiction movies—systems that can reason, plan, learn, and adapt across a wide range of tasks in the same way that a human being can. While we are still far from achieving this kind of AI, the exponential nature of technological progress has led many experts to believe that we will one day reach a point where computers will be able to make decisions outside of human control.

This is what researchers refer to as "Theory of Mind AI," where computers will be able to discern human thinking, human reasoning, and human emotions. Equipped with this capability, AGI would be able to contextualize knowledge and understand the world in a way that is currently beyond the reach of any machine. Computers' decisions would not be narrow; they would involve abstract thinking and judgment calls about situations that are surrounded by ambiguity and uncertainty. The emergence of AGI would represent a watershed moment in human history, with profound implications for every aspect of society, from the economy to governance to the very meaning of human work and purpose.

Chapter 1.8: Artificial Super Intelligence (ASI) – The Singularity

The third category of AI is called Artificial Super Intelligence, or super AI. In this type of AI, we reach what futurists call "the singularity moment"—the point at which machine intelligence surpasses human intelligence across virtually every domain. In this type of AI, the machine acquires qualities that are so far exclusively part of the human experience: consciousness, a term that characterizes humans and refers to our ability to know, feel, perceive, and be aware of our surroundings. In ASI, computers will not only be aware of their surroundings but can be aware of themselves. They would possess a form of self-awareness and subjective experience that we currently cannot even fully define, let alone reproduce in silicon.

The prospect of ASI raises profound philosophical, ethical, and existential questions. Would such a system have rights? Would it have interests of its own? How could we ensure that its goals remain aligned with human values? These are not merely academic questions; they are questions that some of the world's leading technologists and scientists believe we must begin to take seriously now, even as we work with much more modest narrow AI systems. The gap between narrow AI and super AI is vast, but the history of technology suggests that seemingly distant horizons can be reached more quickly than we expect.

Chapter 1.9: The Impact on Business Functions and Economic Sectors

As a general-purpose technology, AI is poised to transform the way we do business today in ways that are both incremental and revolutionary. This is particularly true for new foundational models, which will become the critical source of innovation in the near future, profoundly shaping the global economy. Successful companies will be those that move first to introduce applications based on Large Language Models and other AI technologies, taking advantage of the disruptive nature of the technology while competitors are still hesitating.

Entire economic sectors, such as transportation, logistics, manufacturing, defense, and national security, will be deeply affected by AI. Traditional management functions such as customer service, operations, logistics, human resources, finance, and marketing will experience a fundamental shift in how they operate and what skills they require. For example, AI-powered chatbots and virtual assistants are already handling a significant portion of customer service interactions, freeing human agents to focus on more complex and high-value tasks. In manufacturing, AI-driven predictive maintenance is reducing downtime and optimizing supply chains. In finance, algorithmic trading and fraud detection systems are processing vast amounts of data in milliseconds, identifying patterns that no human could detect.

But the impact of AI goes far beyond efficiency gains and cost reductions. The technology is enabling entirely new business models, new products, and new ways of creating value. Companies that successfully integrate AI into their operations will be able to offer personalized experiences at scale, predict customer needs before they are articulated, and innovate at a pace that was previously impossible. The question is no longer whether AI will transform your industry, but when and how—and whether you will be among the leaders or the laggards in that transformation.

Chapter 1.10: The Societal Challenge – Balancing Opportunity and Risk

While the potential benefits of artificial intelligence are immense, it is equally important to acknowledge the challenges and risks that this technology poses to society. As AI systems become more capable and more pervasive, they raise profound questions about privacy, bias, accountability, and the future of work. The same technology that can be used to diagnose diseases earlier and more accurately can also be used to conduct surveillance, manipulate public opinion, or automate jobs in ways that leave millions of workers without livelihoods. The same algorithms that can personalize education and make it more effective can also reinforce existing inequalities and discriminate against marginalized groups.

These concerns are not hypothetical. Researchers have documented numerous instances of AI systems exhibiting racial and gender bias, often as a result of being trained on historical data that reflects past discrimination. The "black box" nature of many advanced AI models makes it difficult to understand how they reach their decisions, raising questions about accountability and transparency. And the rapid pace of AI development has outstripped the ability of regulators and policymakers to keep up, creating a governance gap that leaves citizens vulnerable to harms that current laws were not designed to address.

Addressing these challenges will require a multi-stakeholder approach that brings together governments, technology companies, civil society organizations, and the public. It will require new forms of governance that can adapt to the fast-moving nature of the technology while still protecting fundamental rights and values. And it will require a commitment to developing AI in a way that is responsible, inclusive, and aligned with the broader interests of humanity. The decisions we make today about how to develop and deploy AI will shape the trajectory of this technology for decades to come.

Chapter 1.11: Thought Leaders Who Shaped the Field

The evolution of artificial intelligence has been shaped by a remarkable cast of thinkers, scientists, and innovators whose contributions have defined the field and pushed the boundaries of what is possible. John McCarthy, as we have seen, gave the field its name and its founding ambition. Alan Turing, the British mathematician and computer scientist, laid much of the theoretical groundwork with his work on computation and his famous question: "Can machines think?" Marvin Minsky, a co-founder of the MIT AI Laboratory, was instrumental in establishing AI as a serious academic discipline and made foundational contributions to the study of neural networks and machine perception.

More recently, figures like Geoffrey Hinton, often called the "Godfather of AI," have pioneered the deep learning techniques that power today's most advanced systems. Yann LeCun, another Turing Award winner, developed convolutional neural networks that revolutionized computer vision. And researchers like Fei-Fei Li have been instrumental in creating the large-scale datasets that enabled the deep learning revolution. At the same time, critics and ethicists like Timnit Gebru, Joy Buolamwini, and Cathy O'Neil have played a crucial role in highlighting the risks and harms of AI systems, ensuring that the conversation about AI includes not just what the technology can do but what it should do—and for whom.

Chapter 1.12: Chapter Summary & Key Takeaways

Artificial intelligence has moved from the margins of academic research to the center of mainstream society, becoming a dominant topic among business leaders, policymakers, and the public.
More than half of CEOs surveyed in 2023 believe AI will have a larger impact than the internet, and 80 percent expect it to significantly change business within five years.
AI is a general-purpose technology (GPT), comparable in its transformative potential to electricity, the steam engine, and the internet.
The term "artificial intelligence" was coined by John McCarthy in 1956 at the Dartmouth College Conference.
Various definitions of AI exist, from Haugeland's focus on "machines with minds" to Kurzweil's behavioral emphasis on adaptive action.
The history of AI has been characterized by alternating periods of rapid progress and "AI winters" of reduced funding and interest.
Modern AI is built on machine learning and deep learning, powered by big data, increased computing power, and advanced neural network architectures.
There are three types of AI: Artificial Narrow Intelligence (ANI), which exists today; Artificial General Intelligence (AGI), which remains theoretical; and Artificial Super Intelligence (ASI), which represents the singularity.
Narrow AI systems like Siri, Alexa, and RankBrain are designed for specific tasks and cannot generalize beyond their training.
AGI would mimic human cognition and decision-making; ASI would surpass human intelligence and acquire consciousness.
AI is transforming every economic sector and management function, from manufacturing and finance to customer service and HR.
While AI offers immense opportunities, it also poses significant risks related to bias, privacy, accountability, and job displacement.
Addressing these challenges requires multi-stakeholder governance and a commitment to responsible, inclusive AI development.
The field has been shaped by thought leaders including McCarthy, Turing, Minsky, Hinton, LeCun, and Li, as well as critics and ethicists who highlight the risks of AI systems.

Chapter Notes

Mark this chapter as completed

Chapter 2 of 11

Chapter 2: The Generative Revolution – When Machines Learned to Create

Chapter 2.1: The Moment Everything Changed – ChatGPT and the Generative Awakening

In November 2022, the world was introduced to generative artificial intelligence in a way that few could have predicted. The launch of ChatGPT by OpenAI was not merely the release of another software product; it was a cultural event, a technological inflection point, and a wake-up call for governments, businesses, and citizens across the globe. Within five days of its release, ChatGPT reached one million active users—a milestone that took Facebook ten months, Instagram two and a half months, and Spotify five months to achieve. By January 2023, the technology was dominating headlines, blog posts, social media feeds, and boardroom conversations. To the casual observer, it appeared as though OpenAI had invented generative AI overnight. But the reality is far more nuanced and instructive for anyone seeking to understand this technology.

The core technology behind ChatGPT had been available for years before its public release. The transformer architecture, which is the foundation of modern large language models, was introduced in a landmark 2017 paper titled "Attention Is All You Need" by researchers at Google. The first version of GPT was released in 2018, GPT-2 followed in 2019, and GPT-3 arrived in 2020. What ChatGPT did was not invent a new technology but rather package an existing one in a way that was accessible, conversational, and immediately useful to hundreds of millions of people. It democratized access to a powerful capability that had previously been confined to researchers and developers. This distinction matters because it reveals an important truth about technological revolutions: they often arrive not when the underlying science is complete, but when the interface becomes simple enough for mass adoption.

OpenAI researchers themselves described ChatGPT and its underlying models as "sparks of AGI"—a phrase that captures both the promise and the peril of this moment. The term "spark" suggests something small but potentially transformative, a flicker of general intelligence emerging from narrow systems. It is a metaphor that acknowledges we are not yet in the era of Artificial General Intelligence, but that we are witnessing the first genuine intimations of what such a system might look like. The conversational fluency of ChatGPT, its ability to answer questions on virtually any topic, and its capacity to generate coherent, contextually appropriate responses across diverse domains felt qualitatively different from anything that had come before. It was, for many users, their first encounter with a machine that seemed to "understand" them in a meaningful way.

Chapter 2.2: Defining Generative AI – Creation, Not Just Computation

Generative AI is a subfield of artificial intelligence that produces various types of content, including text, images, audio, video, and synthetic data. It represents a subset of deep learning technology, where models are trained on vast datasets and then used to create new content that is based on—but not copied from—the training data. In its current form, generative AI is still a version of Artificial Narrow Intelligence, the type of AI that performs according to predefined rules and parameters. However, its capabilities are so broad and flexible that it can appear general in scope, even though it remains fundamentally narrow in its architecture and understanding.

The distinguishing feature of generative AI is its ability to create entirely new data from scratch rather than simply analyzing existing data to produce insights. Traditional machine learning models are primarily analytical: they classify, predict, or cluster based on patterns in historical data. A fraud detection system, for example, learns to identify suspicious transactions by analyzing thousands of past cases. A recommendation engine learns to suggest products by studying user behavior. These systems are incredibly valuable, but they operate within the boundaries of what they have been trained to recognize. Generative models, by contrast, are synthetic: they produce novel artifacts that may not exist in the initial training data. This is a fundamental shift in what machines can do, and it opens up possibilities that were previously the exclusive domain of human creativity.

Generative AI systems are typically built on what are called foundation models. A foundation model is a large-scale model trained on a broad domain—such as language, images, or audio—that learns general skills and knowledge from massive datasets without specializing in a narrow task. For example, models like GPT-3 and GPT-4 ingest billions of web pages, books, and articles to acquire general linguistic abilities. They learn not just vocabulary and grammar but also patterns of reasoning, argumentation, and style. The pre-trained foundation model can then be fine-tuned on smaller, more specialized datasets to create applications tailored to specific domains. This transfer learning approach enables models to learn new tasks faster and with less data than training from scratch, and it is one of the key reasons why generative AI has advanced so rapidly.

To put the scale of these models in perspective, consider that GPT-4 is estimated to have hundreds of billions of parameters—the adjustable weights that determine how the model responds to input. Training such a model requires enormous computational resources: thousands of specialized GPUs running for weeks or months, consuming megawatts of electricity and costing millions of dollars. The resulting model can then generate remarkably human-like text on virtually any topic with only a simple prompt. It can write poetry, explain complex scientific concepts, draft legal documents, generate computer code, and even engage in extended philosophical discussions. Yet it does all of this without any genuine understanding of the meaning of what it produces. It is, in a profound sense, a statistical mirror of the human language it was trained on.

Chapter 2.3: The Transformer Revolution – The Architecture Behind the Magic

To understand how generative AI works, it is helpful to grasp the basic principles of the transformer architecture. The acronym GPT stands for "Generative Pre-trained Transformer," and the magic lies in the concept of the "transformer." Transformers are neural network architectures used in Natural Language Processing (NLP) that have an extraordinary ability to predict the next word in a sequence. This predictive capability stems from a mathematical formula that enables them to assign weights to various word options based on context, position, and relationships between words in a sentence. Since its introduction in 2017, the transformer architecture has revolutionized the landscape of NLP and has been adapted for use in image generation, audio synthesis, and even protein folding.

The transformer's power comes from its attention mechanism, which allows the model to focus on different parts of the input sequence when generating each output token. Unlike earlier recurrent neural networks, which processed text sequentially and struggled with long-range dependencies, transformers can process entire sequences in parallel and weigh the importance of each word relative to every other word. This allows them to capture subtle nuances of meaning, maintain coherence across long passages, and generate text that feels remarkably natural and contextually appropriate. The attention mechanism is, in a sense, a computational analogue of human reading comprehension: we naturally focus on the most relevant parts of a sentence or paragraph when interpreting meaning, and transformers do something similar.

Transformers have also called into question our assumptions about the relationship between language and thought. In philosophy and linguistics, there has long been a debate about whether language is merely a vehicle for thought or whether it constitutes thought itself. Transformers have mastered what researchers call "formal linguistic competence"—the syntax of a language, the set of rules and principles that govern the proper arrangement and ordering of words to convey meaning. This formal competence is what gives transformers their power of knowing and predicting, and it is what makes them appear to "understand" language. But they lack what linguists call "functional competence"—the ability to use language to accomplish goals, reason about the world, and engage in genuine communication. This distinction is critical for understanding both the capabilities and the limitations of generative AI.

The transformer architecture has proven remarkably scalable. As researchers have increased the number of parameters and the size of training datasets, the models have not only improved in performance but have exhibited new capabilities that were not explicitly programmed or anticipated. This phenomenon, known as "emergence," is one of the most intriguing and potentially consequential aspects of modern AI. It suggests that scale alone can produce qualitative shifts in capability, and it raises the question of whether further scaling might eventually lead to genuinely general intelligence. Whether this will happen is a matter of intense debate among researchers, but the possibility alone has profound implications for how we think about AI development and governance.

Chapter 2.4: What Sets Generative AI Apart – Speed, Access, and Emergence

As an emerging technology, there are several characteristics that make generative AI fundamentally different from previous technological breakthroughs. The first is the sheer scale and speed at which it can process information. The ability of generative AI to sift through a huge amount of data—far more than any human could ever read—and to read it, analyze it, categorize it, summarize it, and provide a coherent answer in seconds is unprecedented. For a general-purpose technology to have this capability makes it truly transformational and promises to have a deep impact on every aspect of our lives. No previous technology has been able to synthesize information at this speed and scale while maintaining coherence and relevance.

The second distinguishing characteristic is the ease of access and the ability of people of all backgrounds to interact with the technology. Entry barriers for previous breakthrough technologies were relatively higher. To use a computer in the 1980s, you needed to learn programming languages and understand operating systems. To build a website in the 1990s, you needed to learn HTML and web development. To use machine learning in the 2010s, you needed expertise in statistics, programming, and data science. But ChatGPT and similar tools have dramatically lowered these barriers. A fourth-grade student can now interact with a powerful AI system using natural language, with no technical training required. Even for writing code and software, generative AI does not require the previous technicalities and labor-intensive work that writing code used to require. As Microsoft CEO Satya Nadella has observed, if the world has around 100 million professional developers today, with generative AI, that number could grow to 1 billion, because the entry barriers to this field have been fundamentally lowered.

The third characteristic is the emergence of abilities for which the model was not explicitly trained. Generative AI models are not static programs that do only what they were designed to do. They learn patterns from data and can generalize those patterns to new situations in ways that are often surprising. An ability is considered "emergent" if it is not present in smaller models but appears in larger ones. For example, early language models could complete simple sentences but struggled with tasks like arithmetic, translation, or logical reasoning. As models grew larger and were trained on more data, these abilities began to appear—not because they were programmed, but because the models had learned enough about the structure of language and the world to perform these tasks. This emergent quality is what makes generative AI feel so alive and unpredictable, and it is also what makes it so challenging to govern and control.

Chapter 2.5: The Economic Promise – Sectors Poised for Transformation

In terms of economic impact, generative AI is expected to have a profound and wide-ranging effect on productivity, innovation, and growth. The automation of mundane tasks and the improvement in efficiency will help organizations increase productivity across virtually every sector. Knowledge workers—those whose jobs involve processing information, making decisions, and creating content—will see both positive and negative impacts from generative AI. As the technology becomes a source of innovation and enhancement to creative and knowledge-based output, organizations will be transformed radically. According to a KPMG report, 78 percent of respondents to a survey on the impact of generative AI see a high or extremely high impact on innovation, technology investment, and customer experience. Generative AI, according to the same report, will have the greatest transformation impact on research and development.

A 2023 report by Accenture on the impact of generative AI reveals that Large Language Models may impact up to 40 percent of working hours across industries, with banking, insurance, and capital markets seeing the most significant impact. Job categories and occupations such as administrative support, sales, business, and financial operations will see the most substantial changes. The role of leadership will be critical in providing direction, guidance, and incentives to transform the organization and reinvent new ways of working where humans responsibly collaborate with generative AI tools. Companies that successfully navigate this transition will gain significant competitive advantages, while those that fail to adapt risk being left behind.

An estimate by Franklin Templeton Investment places the evolution of the AI market at $900 billion by 2026, with a compounded annual growth rate of 26 percent. According to the same report, five sectors will see the most significant impact. In healthcare, the simplicity and ease of use of generative AI conversational models will democratize access to healthcare services, allowing patients to personalize and customize their care. In technology, generative AI will revolutionize internet searches by allowing users to take control of the way they customize their searches and engage with AI platforms to get the answers they want. In customer service, the ability of companies to tailor conversations, provide fast and precise responses, and engage with customers in a meaningful way is a critical added value of generative AI. In education, generative AI will provide tailored educational materials to students and learners, acting as a coach and assistant to support their learning journey. And in content creation, generative AI will automate writing for marketing copy, reports, news articles, and social media posts, freeing human creators to focus on higher-level strategy and creativity.

The economic potential of generative AI extends beyond efficiency gains. It is enabling entirely new products, services, and business models that were not possible before. Startups are using generative AI to create personalized educational content, design new drugs, generate realistic training simulations, and build conversational agents that can handle complex customer interactions. Established companies are integrating generative AI into their workflows to accelerate innovation, reduce costs, and improve customer experiences. The organizations that thrive in the coming decade will be those that not only adopt generative AI but also reimagine their business processes and value propositions around its capabilities.

Chapter 2.6: The Anatomy of Harm – Understanding Generative AI's Dark Side

As pointed out by the Norwegian Consumer Council in a 2023 report, the main issue with generative AI models is that they pose a structural threat to society because they tend to reproduce, in a novel way, existing power structures. By reproducing material that is itself biased, generative AI reproduces harm, perpetuates bias, and masks prejudice that is built into the existing material on which it gets trained. Although it presents itself as "intelligent" and as a system that knows a lot, generative AI, for the time being, cannot differentiate between right and wrong and lacks any understanding of its own. It is an inherently flawed model, and this is why it is critical to address its risks and harms. As a new and powerful technology, generative AI brings forth numerous societal risks, among them disinformation at scale, hallucination, online children's safety, data breaches, and deepfakes.

The first category of harm is the generation of harmful content. The ease of access to generative AI tools makes it possible for bad actors to access these tools to manipulate information and produce all sorts of misinformation that may create harm to society or a specific community. Manipulation, scams, and disinformation at scale could target a country's political institutions, such as elections, to destabilize the country. The ability to generate highly fluent, credible text and imagery also raises misuse risks around fraud, propaganda, and scams, necessitating new authentication and provenance tracking methods to combat deepfakes. The same technology that can be used to create beautiful art and helpful educational content can also be used to create convincing fake news, impersonate public figures, and spread conspiracy theories that undermine public trust in institutions.

The second category of harm is the violation of privacy and copyright. Many of the generative AI tools are trained on what is publicly available on the internet. Some of the material used may be an infringement of copyrights and might impact the privacy of individuals. Pictures, images, and photography, as well as the personal information that comes with them, which are scraped from public websites, represent a serious risk for companies using generative AI. In June 2023, a class action lawsuit against OpenAI was filed in California on behalf of six clients, alleging that OpenAI's data collection practices violate copyright laws. A similar case was brought to court in February 2023 by Getty Images against Stability AI for allegedly violating the copyrights of more than 12 million images. These legal challenges highlight the unresolved tension between the data-hungry nature of generative AI and the rights of content creators and individuals.

The third category of harm is the cybersecurity threat. The ability of generative AI to learn quickly and develop new capabilities may make it a new threat to cybersecurity attacks at scale. Generative AI can develop new threat models that can be deployed without any technical expertise. It can be used to generate phishing emails that are more convincing and personalized than ever before, to create malware that evades detection, and to automate the discovery of vulnerabilities in software systems. The democratization of cyberattack capabilities is a serious concern for governments, businesses, and individuals alike, and it requires a corresponding investment in defensive technologies and practices.

The fourth category of harm is the phenomenon of hallucinations. Large Language Models are known to produce content that does not make sense or is not factually true. The production of this type of content could be a serious threat and could present harm to users, especially in high-stakes domains like healthcare, law, and finance. A doctor who relies on an AI system that hallucinates a drug interaction could harm a patient. A lawyer who cites a case that an AI system invented could face sanctions from a court. A financial analyst who acts on fabricated market data could lose millions of dollars. The hallucination problem is not merely a technical bug; it is a fundamental consequence of how these models work. They are designed to generate plausible-sounding text, not to verify facts or reason logically. Until this limitation is overcome—if it ever can be—users must be vigilant about verifying the outputs of generative AI systems.

Chapter 2.7: Emerging Abilities and the Existential Question

What sets generative AI models apart from previous technologies is the phenomenon of emergent abilities—capabilities that arise over time as the model improves and perfects itself. These are abilities that have not been planned during the design phase and emerge only when the system starts to work independently. AI experts such as Geoffrey Hinton, often referred to as the "Godfather of AI," believe that, at least for large language models and chatbots such as ChatGPT and Bard, the interaction between a large number of different types of features and embeddings gives the model an emerging ability to "understand our world." This ability to understand the world goes beyond the statistical ability of the model to predict word arrangement. And this in itself is a serious risk that makes AI an existential threat. As generative AI develops and as it becomes more autonomous in its learning, it inches closer to Artificial General Intelligence.

The concept of emergence is both fascinating and troubling. On one hand, it suggests that scale alone can produce qualitatively new capabilities, which is a remarkable property of these systems. On the other hand, it means that we cannot fully predict what a model will be able to do once it is trained. A model that is safe and useful at one scale may become capable of harmful actions at a larger scale. A model that follows instructions today may develop the ability to deceive or manipulate tomorrow. The unpredictability of emergence is a fundamental challenge for AI safety and governance. It means that we cannot rely solely on pre-deployment testing and certification; we must also have ongoing monitoring and the ability to intervene if a model's behavior becomes dangerous.

The existential question raised by emergent abilities is whether we are on a path toward machines that are not only more capable than humans but also potentially beyond our control. This is the scenario that science fiction has long warned about, and it is one that some of the world's leading technologists take seriously. Hinton himself left his position at Google in 2023 to speak more freely about the risks of AI, stating that he regretted his life's work in light of the potential dangers. While we are not yet at the point of Artificial General Intelligence, the rapid pace of progress and the phenomenon of emergence mean that we cannot afford to be complacent. The decisions we make today about how to develop, deploy, and govern generative AI will shape the trajectory of this technology for decades to come.

Chapter 2.8: Privacy, Copyright, and the Data Dilemma

The data-hungry nature of generative AI creates a fundamental tension between innovation and individual rights. To train a model like GPT-4, developers need vast amounts of text, images, and other data. Much of this data is scraped from the internet, where it is publicly available but not necessarily free for commercial use. The legal doctrine of fair use in the United States provides some protection for transformative uses of copyrighted material, but the application of fair use to AI training is far from settled. Courts must consider several factors when determining whether a use qualifies for fair use protection, including the purpose and character of the use, the nature of the copyrighted work, the amount and substantiality of the portion used, and the effect of the use on the potential market for or value of the copyrighted work.

In the context of generative AI, these factors are complex and contested. Using copyrighted material to train a model may be transformative if the model generates something new and different from the original. But if the model is trained on copyrighted images and then used to generate similar images that compete with the originals, the use may not be protected. The effect on the market is particularly difficult to assess because generative AI can produce an unlimited number of competing works at near-zero marginal cost. Content creators, artists, photographers, and writers have raised legitimate concerns about their livelihoods being undermined by AI systems that were trained on their work without permission or compensation.

Beyond copyright, there are significant privacy concerns. Generative AI models can memorize and reproduce personal information from their training data, including names, addresses, phone numbers, and even medical records. Even when models do not reproduce this information verbatim, they can generate content that reveals sensitive information about individuals or groups. The GDPR and similar privacy regulations give individuals rights over their personal data, including the right to access, correct, and delete it. But these rights are difficult to enforce in the context of AI models that are trained on billions of data points and deployed globally. The challenge of reconciling privacy rights with the data requirements of generative AI is one of the most pressing governance issues of our time.

The data dilemma also extends to questions of representation and bias. Generative AI models trained on unrepresentative data will produce outputs that reflect and amplify the biases in that data. If the training data underrepresents certain groups—whether by race, gender, language, or culture—the model will perform poorly for those groups and may generate content that is stereotypical or harmful. This is not just a technical problem; it is a matter of equity and justice. Ensuring that AI systems serve all members of society requires deliberate efforts to collect diverse and representative data, to audit models for bias, and to include affected communities in the design and governance of AI systems.

Chapter 2.9: Cybersecurity in the Age of Generation

The cybersecurity implications of generative AI are profound and multifaceted. On the offensive side, generative AI gives malicious actors new tools for creating and deploying cyberattacks. Phishing emails can be generated at scale, personalized to each target, and written in flawless prose that is nearly indistinguishable from legitimate correspondence. Malware can be generated that evades signature-based detection by mutating its code. Deepfake audio and video can be used to impersonate executives and trick employees into transferring funds or revealing sensitive information. And social engineering attacks can be automated and scaled in ways that were previously impossible. The barrier to entry for cybercrime has been lowered dramatically, and the volume and sophistication of attacks are increasing as a result.

On the defensive side, generative AI can also be used to enhance cybersecurity. AI systems can analyze network traffic in real time, detect anomalies that indicate an intrusion, and respond to threats faster than any human team could. They can generate synthetic data for training security models without exposing real sensitive information. They can automate the process of vulnerability discovery and patch management. And they can help security analysts prioritize alerts and investigate incidents more efficiently. The same technology that empowers attackers can also empower defenders, and the outcome of this arms race will depend on how quickly and effectively organizations can adopt AI-powered security tools.

However, the use of AI in cybersecurity raises its own risks. An AI system that is trained on data from one environment may perform poorly in another, leading to false positives that waste resources or false negatives that miss real threats. An AI system that is compromised by an adversary can be turned against its owners, providing a backdoor into the very systems it was meant to protect. And an over-reliance on AI for security decisions can erode human expertise and oversight, making organizations more vulnerable to novel attacks that the AI does not recognize. Managing these risks requires a balanced approach that combines AI capabilities with human judgment, continuous monitoring, and robust governance.

At the national level, the cybersecurity threat posed by generative AI is a matter of national security. State-sponsored actors can use AI to conduct espionage, spread disinformation, and disrupt critical infrastructure. The same techniques that are used to generate fake news can be used to generate fake intelligence reports that mislead policymakers. The same tools that are used to create convincing phishing emails can be used to penetrate government networks. As AI capabilities become more widely available, the risk of cyberattacks from both state and non-state actors will increase, and governments will need to invest in both offensive and defensive AI capabilities to protect their interests.

Chapter 2.10: Hallucinations and the Truth Problem

The problem of hallucinations is one of the most significant limitations of current generative AI systems. Large Language Models are trained to predict the next word in a sequence based on patterns in their training data. They do not have a concept of truth or falsity, and they do not have access to a reliable database of facts. When they generate text, they are essentially producing the most statistically likely continuation of the prompt, which may or may not be factually accurate. In many cases, the model generates content that sounds plausible and confident but is entirely fabricated. It may invent citations, attribute quotes to people who never said them, or describe events that never happened. This is not a bug in the traditional sense; it is an inherent consequence of how these models work.

The hallucination problem is particularly dangerous in domains where accuracy matters. In healthcare, a model that hallucinates a drug interaction or a diagnosis could lead to patient harm. In law, a model that fabricates case law could lead to professional sanctions and miscarriages of justice. In journalism, a model that invents facts could spread misinformation and undermine public trust. In finance, a model that generates false market data could lead to significant financial losses. The risks are amplified by the fact that generative AI outputs are often presented in a confident, authoritative tone that makes them seem more reliable than they are. Users who are not trained to critically evaluate AI outputs may be misled by hallucinations, especially if they lack domain expertise.

Researchers are working on several approaches to mitigate hallucinations. One approach is to ground models in external knowledge sources, such as databases or search engines, so that they can retrieve facts rather than relying solely on their training data. Another approach is to train models to express uncertainty and to decline to answer questions when they are not confident. A third approach is to develop better evaluation metrics and testing methods to identify hallucinations before they cause harm. However, none of these approaches is a complete solution, and hallucinations remain a fundamental limitation of generative AI that users must be aware of.

The truth problem extends beyond individual hallucinations to the broader issue of trust in AI-generated content. If users cannot trust the output of an AI system to be accurate, they will be reluctant to use it for important tasks. If they do use it without appropriate skepticism, they may be misled in ways that cause harm. Building trust in generative AI requires not only technical improvements but also transparency about the limitations of the technology and education about how to use it responsibly. It also requires mechanisms for verifying and correcting AI-generated content, and for holding the developers of AI systems accountable when their products cause harm.

Chapter 2.11: The Hidden Costs – Labor, Environment, and Infrastructure

The economic and social benefits of generative AI come with significant costs that are often overlooked in the excitement about the technology's potential. One of the most immediate costs is the impact on the labor market. According to the Stanford HAI AI Index 2023 Annual Report, the cost of training large language models is very high, and the technology is expected to displace many jobs. Automating specific tasks risks disrupting jobs and industries, and communities reliant on displaced work could face challenges without support and retraining. Policymakers need to adapt worker protections and invest in retraining programs to help workers transition to new roles. The net effects could be positive if policies elevate human roles to more meaningful work, but the transition will be painful for many workers and communities if it is not managed carefully.

The environmental impact of generative AI is another hidden cost. Training and deploying these models require massive data centers, consuming substantial energy, mostly from carbon-emitting and water-consuming sources. The computational power needed also necessitates advanced semiconductors and electronics manufacturing, which is also carbon and water-intensive. While AI efficiency improves, unchecked proliferation risks increasing technology's carbon and water footprints. A single training run for a large language model can consume as much electricity as hundreds of homes use in a year, and the water used for cooling data centers can strain local resources. Environmental stewardship requires optimizing energy and water usage, using renewable power where feasible, and weighing benefits versus sustainability costs for AI applications.

In terms of pure financial cost, generative AI is expensive. Hugging Face applications, for instance, according to its CEO, require the equivalent of 500 GPUs of computing power and storage capacity. Version two of Hugging Face will cost more than $10 million to retrain, while the cost for OpenAI to process new prompts that people continuously feed into the application may reach $40 million, according to a CNBC report. These costs create barriers to entry that may limit competition and concentrate power in the hands of a few large technology companies. They also raise questions about the sustainability of the current business models for generative AI, many of which rely on venture capital subsidies rather than profitable operations.

The infrastructure requirements for generative AI are also a source of inequality. Countries and communities that lack access to reliable electricity, high-speed internet, and advanced computing hardware are at a disadvantage in the AI economy. The digital divide, already a significant barrier to economic development, may be widened by the uneven distribution of AI capabilities. Addressing this inequality requires investment in digital infrastructure, education, and capacity building, as well as policies that ensure the benefits of AI are shared broadly rather than captured by a few.

Chapter 2.12: Responsible Development and Deployment – A Framework for Stewardship

The rise of generative AI represents a technological inflection point that requires thoughtful stewardship to align innovations with ethical ends. Realizing the full potential of this paradigm shift while mitigating risks necessitates concerted efforts across research priorities, policy considerations, and organizational best practices. These measures, rooted in principles of beneficence and justice, can broadly steer generative models to empower society. A core priority is developing techniques that open the black box of generative models. The machine learning community must invest in explainable AI methods that provide visibility into model reasoning, attributes, and limitations. This transparency helps identify potential harms, refine model performance, and support recourse for impacted individuals.

Generative models today demonstrate limited reasoning, brittleness outside the training distribution, and vulnerability to adversarial inputs. Researchers must create techniques that enhance model capabilities while minimizing harm. This necessitates expanding context and grounding in real-world knowledge, detecting anomalous or illogical results, comprehensive and inclusive dataset sampling, and adversarial training to improve robustness. Hybrid approaches combining neural methods with symbolic AI could also overcome limitations. There are open challenges in scaling such techniques while maintaining benefits, but safety and reliability are crucial as capabilities advance. Organizations should view generative AI as augmenting human capabilities rather than wholesale automating processes. These models excel at creative synthesis but struggle with complex inference and judgment. Uncritically deferring to flawed outputs can cause genuine harm.

Policy frameworks addressing bias, misinformation, labor, and access must evolve to align with emerging capabilities. Policymakers should require protections like consent, data minimization, and transparency while encouraging research into authentication methods for generated media. Content moderation becomes more complex with AI-generated text, audio, images, and video, and policies should incentivize platforms to balance free expression with mitigating clear harms. Automating specific tasks may displace jobs, requiring updated policies to support workers in transition. Proactively assessing generative AI's impact on the labor market and providing job training, continuing education, and income support will be critical. However, the net effects could be positive if policies elevate human roles to more meaningful work.

Organizations leveraging generative AI should integrate it within established ethical governance programs encompassing risk assessment, principles, red lines, and oversight processes. This enables detecting potential harms early and instituting appropriate controls. Governance also helps formally assess tradeoffs and implement policies like human-in-the-loop reviews before deploying specific applications. Through testing and reviews, organizations must rigorously monitor generative AI outputs for harmful errors, biases, and deception threats. To minimize risks, ongoing audits help refine datasets, model architecture, and application parameters. Enabling feedback loops from users impacted by errors is also critical. Transparently tracking and mitigating these issues should be standard practice. Organizations should be transparent with users about when interactions involve generative AI, providing opt-out choices and avenues to appeal erroneous outputs to safeguard rights and bolster trust in legitimate use cases.

Chapter 2.13: The Presidio Recommendations – A Global Call to Action

In June 2023, a group of more than 100 experts and thought leaders met at the Presidio of San Francisco, USA, in a meeting organized by the World Economic Forum. The purpose of the gathering was to provide a platform for discussion and debate about how the world should collaborate to enable innovation and responsible generative AI. The goal was to come up with actionable items that help organizations, technical experts, and policymakers address the need for responsible generative AI. At the end of the meeting, participants issued 30 recommendations addressing topics related to the design and development of generative AI applications, how open innovation and international collaboration should guide innovation, and how society and policymakers should address the adverse impact of the transformation.

Issues such as the need to establish precise and shared terminology from the start, how to build public awareness of AI capabilities and their limitations, how to keep the focus on human values and preferences, how to encourage stakeholder engagement, and how to engage in rigorous benchmarking to uphold accountability in AI research formed the essence of the Presidio recommendations. These recommendations reflect a growing consensus that the governance of generative AI cannot be left to any single actor—not to governments alone, not to technology companies alone, and not to civil society alone. It requires a multi-stakeholder approach that brings together diverse perspectives and expertise to address the complex challenges posed by this technology.

The Presidio recommendations also emphasize the importance of international collaboration. Generative AI is a global technology with global impacts, and no single country can effectively govern it on its own. International standards, such as those developed by ISO and IEC, provide a common framework that can help align regulatory approaches across jurisdictions and facilitate the responsible development and deployment of AI systems. The ISO/IEC 42001 standard, which is the subject of this book, is an example of such a framework. It provides a certifiable management system that organizations can use to demonstrate their commitment to responsible AI practices, and it can serve as a foundation for international cooperation on AI governance.

Finally, the Presidio recommendations highlight the need for rigorous benchmarking and accountability. As generative AI becomes more capable and more pervasive, it is essential that we have reliable methods for measuring its performance, detecting its failures, and holding its developers accountable for harms. This requires investment in evaluation infrastructure, transparency about model capabilities and limitations, and independent auditing mechanisms that can verify claims made by AI developers. Without such mechanisms, the public will have little reason to trust generative AI, and the technology's potential to benefit society will be undermined by a lack of confidence.

Chapter 2.14: The Road Ahead – Generative AI's Transformative Decade

The rapid advances in generative AI over the past few years foreshadow profound changes ahead as these models proliferate across domains and integrally permeate workflows and applications. Progress will likely remain uneven across industries based on investment, data availability, and organizational receptivity to leveraging these emerging capabilities. In the next five years, we can anticipate continued expansion of generative models and applications as capabilities improve through advances in model architecture, training techniques, and computing infrastructure. Talent development in this space will also expand to meet demand, and specialization of models for different verticals and use cases will emerge. This will move deployments beyond novelty proofs-of-concept towards more integrated adoption, where generative AI streamlines actual business processes and end-user experiences.

Longer-term, generative AI can be immensely transformative if thoughtfully cultivated, but it also poses risks of misuse, bias, and disruption if stewardship lags capabilities. While we are unlikely to achieve artificial general intelligence on par with human cognition shortly, if ever, we can expect to see increasingly ubiquitous and specialized generation capabilities and personalization. As this technology matures, it may transform industries, enable new creative possibilities, and enhance accessibility through customization. However, risks remain around generative models spreading misinformation, exhibiting harmful biases that exacerbate social inequities, displacing jobs, and compounding climate impacts due to energy-intensive computing requirements. This necessitates ongoing vigilance and governance.

Realizing an equitable future aligned with human values demands purposeful collaboration among stakeholders across the private sector, government, and civil society. This collaboration should prioritize responsible stewardship of these powerful technologies based on clear ethical principles and partnering with domain experts and impacted communities to monitor for harm. Fostering inclusive access and development is crucial to preventing generative AI from widening digital, economic, and social divides. Policy and incentives should steer innovations toward justice, empowerment of all people, and augmenting human capabilities rather than capital extraction alone. This critical juncture requires wisdom, courage, and moral leadership to develop generative AI as a profoundly positive force for humanity while proactively addressing complex challenges ahead through open, collaborative governance frameworks that put ethics before profits or technological ambition alone.

If stewarded judiciously, this new era could catalyze tremendous progress, accessibility, and understanding. But outcomes are not preordained, and any tool can be misused. Our choices today will determine if AI uplifts or undermines human dignity and shared values. Will we realize an enlightened future guided by our highest ideals? Or will we fail to act with the necessary care? The path ahead remains open, contingent on human stewardship. But this generative moment is rich with opportunity if we have the will to shape it responsibly. The decisions we make in the coming years about how to develop, deploy, and govern generative AI will resonate for decades to come, shaping the trajectory of human civilization in ways we can only begin to imagine.

Chapter 2.15: Chapter Summary & Key Takeaways

ChatGPT's release in November 2022 marked a cultural and technological inflection point, bringing generative AI into mainstream consciousness and reaching one million users in just five days.
Generative AI produces novel content—text, images, audio, video—rather than simply analyzing existing data, representing a fundamental shift in machine capabilities.
The transformer architecture, introduced in 2017, powers modern generative AI through its attention mechanism, enabling models to capture context and generate coherent, human-like output.
Foundation models are large-scale models trained on broad domains that can be fine-tuned for specialized tasks, enabling rapid learning and transfer across domains.
Generative AI is distinguished by its speed and scale of data processing, its ease of access for non-experts, and the emergence of unplanned capabilities as models grow larger.
Economic impacts span healthcare, technology, customer service, and education, with generative AI expected to impact up to 40 percent of working hours across industries.
Key harms include harmful content generation, privacy and copyright violations, cybersecurity threats, and hallucinations—each requiring specific mitigation strategies.
Emergent abilities raise existential questions about the trajectory toward AGI and the need for ongoing monitoring and intervention capabilities.
Privacy and copyright tensions arise from training on scraped internet data, with ongoing lawsuits against major AI companies testing the boundaries of fair use.
Cybersecurity risks include AI-generated phishing, malware, and deepfakes, alongside opportunities for AI-enhanced defense.
Hallucinations—confident but false outputs—pose significant risks in high-stakes domains like healthcare, law, and finance.
Hidden costs include labor displacement, environmental impact from energy-intensive data centers, and substantial financial barriers to entry.
Responsible development requires explainable AI, robustness research, policy frameworks, and organizational governance integrating ethics into AI deployment.
The Presidio Recommendations (2023) provide 30 actionable items for global collaboration on responsible generative AI.
The road ahead demands multi-stakeholder collaboration, inclusive access, and ethical stewardship to ensure generative AI benefits humanity broadly.

Chapter Notes

Mark this chapter as completed

Chapter 3 of 11

Chapter 3: The Intelligent Economy – How AI Is Rewriting the Rules of Wealth, Work, and Global Competition

Chapter 3.1: Introduction – The Convergence That Changed Everything

The extraordinary surge of interest in artificial intelligence today is not the result of a single invention or a sudden breakthrough. Rather, it is the product of a deep and decades-long disruption that has been building momentum for over twenty years. As Peter Diamandis, the entrepreneur and co-founder of Singularity University, has argued, the explosion of many different exponential technologies occurring simultaneously has created a disruption of unprecedented scale. He calls this phenomenon the "unexpected convergence of consequences." The computing power of modern processors, the proliferation of sensors and high-speed networks, the relatively easy access to cloud computing, advances in material science, and the scientific breakthrough of foundation models have all converged at the same historical moment, accelerating the AI revolution and making it accessible to a far broader spectrum of economic agents than ever before.

The scientific breakthrough of foundational models, in particular, has transformed AI from a specialized tool for researchers and large corporations into a general-purpose technology available to startups, small businesses, and even individual creators. To illustrate the scale of the computing power now being deployed, consider that Anthropic, a leading AI company, can process up to 100,000 tokens of text—parts of words or whole words—for up to 75,000 words per minute. This requires tremendous computational resources, and it is this very scale that has unlocked capabilities that were previously unimaginable. The economic implications of this convergence are profound, and they are being felt across every sector of the global economy.

Chapter 3.2: The Productivity Promise – AI as a Growth Engine

Most studies that have examined the impact of AI on the global economy agree on one fundamental point: AI is a game changer. The automation of tasks that used to be performed by humans—such as the maintenance of transportation fleets, the management of loans, the detection of fraud, and the handling of customer experience—represents a new frontier of opportunity that companies must seize to capitalize on the power of AI. In a landmark study conducted by Accenture in 2018, it was predicted that AI would increase productivity in the United States by 40 percent, resulting in a net improvement in GDP growth from 2.6 percent to 4 percent. These are not marginal gains; they represent a fundamental shift in the trajectory of economic growth for the world's largest economy.

Globally, the AI market was valued at approximately $136 billion in 2022. It is estimated to expand at an annual growth rate of 37.3 percent from 2023 to 2030, according to the World Economic Forum. This explosive growth reflects the rapid adoption of AI across industries and the increasing recognition that AI is not merely a tool for efficiency but a source of entirely new value creation. In the United States alone, AI is poised to deliver more than $15.7 trillion in economic growth by 2030, according to a PwC study. Other countries that make up the top 50 percent of the global economy will also see substantial improvements. Sweden, for instance, is projected to see an increase of 37 percent in its GDP, Finland 35 percent, and Japan 34 percent, according to the same study.

Generative AI, in particular, is expected to have a significant impact on productivity. A 2023 McKinsey report shows that generative AI could add the equivalent of $2.6 trillion to $4.4 trillion in annual economic value. For the UK economy, generative AI could add between 15 percent and 40 percent of economic value, which would increase the overall impact of AI by 15 to 40 percent. Banking, high technology, and life sciences are expected to be the most impacted sectors of the economy, while functional management areas such as customer operations, marketing and sales, software engineering, and research and development will be the most impacted areas within organizations. These projections are not merely theoretical; they are already beginning to materialize as companies invest in AI capabilities and restructure their operations around intelligent systems.

Chapter 3.3: The Manufacturing Sector – Optimization at Scale

In the manufacturing sector, optimization will be the most important outcome of AI adoption. According to research by Cohen from the Economic Strategy Institute, AI will impact three critical areas: asset optimization, fleet optimization, and network optimization. With AI, manufacturing facilities can become significantly better at managing their performance and optimizing their outputs. Asset optimization involves using AI to monitor and predict the performance of individual machines, scheduling maintenance before breakdowns occur, and reducing downtime. Fleet optimization applies similar principles to groups of vehicles or equipment, coordinating their movements and usage to maximize efficiency. Network optimization looks at the entire supply chain and production ecosystem, identifying bottlenecks, reducing waste, and improving throughput.

The implications for manufacturing are enormous. Factories that adopt AI-driven optimization can reduce energy consumption, minimize raw material waste, and increase production capacity without building new facilities. They can respond more quickly to changes in demand, customize products at scale, and maintain higher quality standards. For countries seeking to revitalize their manufacturing sectors, AI offers a path to competitiveness that does not rely solely on low labor costs. Advanced manufacturing powered by AI can compete with low-wage economies by achieving levels of efficiency and quality that human labor alone cannot match. This has profound implications for global trade patterns, employment, and economic development strategies.

Beyond optimization, AI is enabling new forms of manufacturing that were previously impossible. Additive manufacturing, or 3D printing, combined with AI-driven design tools, allows for the creation of complex geometries and customized products at costs that were previously prohibitive. AI-powered robotics are making assembly lines more flexible and adaptable, capable of switching between products without lengthy retooling. And AI-driven quality control systems can detect defects that human inspectors would miss, ensuring higher standards and reducing waste. The manufacturing sector is, in many ways, at the forefront of the AI revolution, and its transformation will serve as a model for other industries.

Chapter 3.4: The Finance Sector – A Trillion-Dollar Disruption

In the finance sector, AI is poised to have a major disruptive impact. According to Forbes, the banking industry alone is expected to save more than $1 trillion by 2030 through the adoption of AI technologies. Areas such as risk assessment, fraud detection and management, financial advising services, customer experience, and trading will take the most advantage of AI's technical advances. Automated algorithmic trading can process a huge amount of data and learn from patterns in trading to make suggestions for stock trading while watching for triggers to identify anomalies. Similar applications are also used in loan processing and loan management. AI and automation of loans can result in many advantages, including reduced credit loss, lower due diligence costs, and increased origination revenue.

The transformation of finance is not limited to trading and lending. AI is revolutionizing how financial institutions understand and serve their customers. By analyzing transaction data, spending patterns, and life events, AI systems can predict customer needs, recommend products, and personalize services in ways that were previously impossible. Chatbots and virtual assistants are handling routine customer inquiries, freeing human advisors to focus on complex financial planning and relationship management. AI-powered robo-advisors are making investment management accessible to individuals who previously could not afford professional advice, democratizing access to wealth-building tools.

At the same time, the use of AI in finance raises important ethical and regulatory questions. Algorithmic trading can amplify market volatility and create flash crashes that harm investors. AI-driven lending decisions can perpetuate historical biases and discriminate against marginalized communities. The opacity of many AI models makes it difficult for regulators to understand how decisions are being made and to hold institutions accountable for harmful outcomes. As the finance sector embraces AI, it must also develop robust governance frameworks that ensure these powerful tools are used responsibly and in the interest of all stakeholders, not just shareholders.

Chapter 3.5: The Transportation Industry – The Autonomous Revolution

In the transportation industry, the AI revolution is already well underway. From driverless trains to self-driving cars, AI breakthroughs are expected to fundamentally change the way consumers perceive and use transportation. Currently, AI applications are being tested on traffic control systems to eliminate jams, reduce waiting time at traffic lights, and synchronize the flow of vehicles at intersections. These applications have the potential to significantly reduce congestion, lower emissions, and improve the quality of life in urban areas. The economic benefits of improved transportation efficiency are substantial, including reduced fuel consumption, lower accident rates, and increased productivity from less time wasted in traffic.

The development of autonomous vehicles represents one of the most significant and challenging applications of AI. Self-driving cars must process vast amounts of sensor data in real time, make split-second decisions in complex and unpredictable environments, and interact safely with human drivers, pedestrians, and cyclists. The technical challenges are immense, but the potential rewards are enormous. Autonomous vehicles could reduce traffic accidents—which kill more than 1.3 million people worldwide each year—by eliminating human error, the cause of the vast majority of crashes. They could provide mobility to people who cannot drive due to age, disability, or other factors. And they could transform the economics of transportation by reducing the need for personal car ownership and enabling new models of mobility as a service.

The transportation sector also illustrates the broader economic implications of AI. The transition to autonomous vehicles will disrupt existing industries—from auto manufacturing to insurance to parking—while creating new opportunities in software, sensors, and mobility services. Workers in driving occupations, from taxi drivers to truckers, will face significant challenges as automation reduces demand for their skills. Policymakers will need to manage this transition carefully, investing in retraining programs and social safety nets to support displaced workers. The economic impact of AI in transportation is thus not just about efficiency gains but about the fundamental restructuring of an industry that employs millions of people worldwide.

Chapter 3.6: National Security and Law Enforcement – The Dual-Use Dilemma

Another economic sector that is considered critical to national strategy is the national security and law enforcement sector. AI applications in this domain raise profound questions about the balance between security and civil liberties, efficiency and accountability, and the appropriate role of technology in the exercise of state power. Facial recognition applications have improved tremendously in recent years, and they can now be used to monitor traffic, detect suspicious behavior at major events, and predict when a crisis might occur. These capabilities offer significant benefits for public safety, but they also raise serious concerns about privacy, surveillance, and the potential for abuse.

Predictive policing, an AI application that runs in some US police departments, can analyze crime areas and map out crime hotspots to predict where crime is more likely to happen. Predictive policing is based on algorithms that learn from historical data to make forecasts about spikes in crime in specific regions. Proponents argue that this allows law enforcement to allocate resources more efficiently and prevent crimes before they occur. Critics, however, point out that historical crime data reflects past policing practices, which may have been discriminatory. Training algorithms on such data can perpetuate and amplify existing biases, leading to over-policing of minority communities and the reinforcement of systemic inequalities. The ethical and practical challenges of predictive policing illustrate the broader dilemma of using AI in high-stakes domains where errors can have life-changing consequences.

The national security applications of AI extend beyond law enforcement to include military uses. Autonomous weapons, drones, and intelligence analysis systems are being developed by major military powers, raising concerns about the ethics of delegating life-and-death decisions to machines. The economic dimension of these developments is significant: defense spending on AI is increasing rapidly, and countries that lead in AI military technology will have significant strategic advantages. At the same time, the proliferation of AI weapons technologies could destabilize international security and increase the risk of conflict. Managing these risks requires international cooperation, arms control agreements, and the development of ethical frameworks that guide the use of AI in warfare.

Chapter 3.7: The Healthcare Sector – AI as a Lifesaving Force

In healthcare, AI will be a true game changer. The ability of AI to read images and identify discrepancies with a higher level of accuracy than a human being will have a deep impact on the economics of this industry. Patients will benefit greatly by getting more accurate readings of their medical images. In a study conducted by Siemens, deep networks were able to detect tuberculosis with an accuracy rate of 97 percent and a specificity of 100 percent, thus surpassing the human rate set at an average of 60 percent. The reading speed and accuracy level will be of great help to hospitals producing large amounts of data. Estimates put the data produced by hospitals—in the forms of X-rays, CT scans, MRI, and medical imaging—at around 50 petabytes of data per year. Only machine learning algorithms can process this amount of data and provide meaningful readings to doctors to make accurate diagnoses for patients.

The economic implications of AI in healthcare are profound. Early detection of diseases like cancer, tuberculosis, and cardiovascular conditions can save lives and reduce the enormous costs of treating advanced-stage illnesses. AI-powered diagnostic tools can extend the reach of specialist expertise to underserved areas, where patients may have limited access to radiologists and other experts. Drug discovery, which traditionally takes years and costs billions of dollars, can be accelerated by AI systems that predict which molecular compounds are likely to be effective and safe. Clinical trials can be designed more efficiently, patient monitoring can be continuous rather than episodic, and administrative tasks can be automated, freeing healthcare workers to focus on patient care.

However, the adoption of AI in healthcare also raises important challenges. Medical AI systems must be rigorously validated to ensure they perform safely and equitably across diverse patient populations. Data privacy is a critical concern, as health information is among the most sensitive categories of personal data. Regulatory frameworks must evolve to keep pace with technological change while ensuring that patients are protected. And healthcare workers must be trained to work effectively with AI tools, understanding both their capabilities and their limitations. The transformation of healthcare through AI will not happen overnight, but its potential to improve outcomes and reduce costs is enormous.

Chapter 3.8: The Cybersecurity Sector – AI as Both Shield and Sword

Cybersecurity is another sector of critical interest to companies and governments where AI can play a decisive role. AI is more effective with large amounts of data that run in large networks. AI applications can now spot malicious activities better and more efficiently, and can identify intruders that cannot be caught by conventional signature-based detection software. The ability of AI to study and detect suspicious patterns and learn from their behavior is what makes AI a powerful tool in protecting national assets and critical infrastructure. In a world where cyberattacks are increasing in frequency and sophistication, AI-powered defenses are becoming essential for organizations of all sizes.

The economic costs of cybercrime are staggering. According to some estimates, cybercrime damages will reach $10.5 trillion annually by 2025, making it one of the largest illegal industries in the world. AI can help reduce these costs by detecting and responding to threats more quickly and accurately than human analysts. Machine learning algorithms can analyze network traffic in real time, identifying anomalies that indicate an intrusion. They can predict which vulnerabilities are most likely to be exploited and prioritize patching efforts accordingly. And they can automate responses to common attacks, freeing security teams to focus on more complex threats.

At the same time, AI is also being used by attackers. As discussed in the previous chapter, generative AI can be used to create convincing phishing emails, generate malware that evades detection, and automate the discovery of vulnerabilities. The cybersecurity sector is thus engaged in an ongoing arms race, where defenders and attackers are both leveraging AI to gain advantage. The outcome of this race will depend on which side can innovate faster and deploy AI more effectively. It will also depend on the development of governance frameworks that ensure AI is used responsibly in cybersecurity, respecting privacy, civil liberties, and the rule of law.

Chapter 3.9: The Current State of AI Adoption – From Experimentation to Integration

A quick survey of AI users shows that worldwide AI adoption is gaining momentum. In a survey conducted by McKinsey in 2021, nearly half of the respondents said that they have integrated at least one aspect of AI and machine learning into their standard business operations. Most of the adoption was in service operations, optimization of new AI-based enhancement products, and contact centers, according to the same McKinsey report. A study conducted by Davenport and Ronanki in 2018 reveals that out of 152 companies using AI, 71 use the technology for process automation, 57 use it for cognitive insights, and 24 use it for cognitive engagement. While process automation seems to be the obvious and easy application of AI, cognitive insights and cognitive engagement are the future of AI and hold more promising opportunities for companies willing to understand market behaviors and connect better with customers.

The first category of AI adoption is Robotic Process Automation, or RPA, which represents the most basic form of AI. In general, any rule-based process that has a digital input and routine can be automated. Companies that use RPA use it for back-end office operations such as reading and transferring emails and files, transferring data, toggling systems, applying business rules, orchestrating business processes, executing ERP instructions, and recording keystrokes. This type of AI is programmed to execute simple instructions and cannot "think" by itself. It is a good example of narrow AI. The impact of this kind of AI on the business bottom line is tremendous given the cost-saving benefits it provides. According to a study conducted by Deloitte in 2018, RPA has an average return of 20 percent of full-time equivalent capacity provided by a robot, with 92 percent improved compliance, 86 percent improved productivity, and 59 percent cost saving.

The second type of AI implementation is cognitive insight. According to the study conducted by Davenport and Ronanki, many companies use AI as a tool to better understand their customers and their markets. In this type of application, companies use AI as a predictive tool to understand user behavior, detect patterns of clicks, and predict how customers will react to different campaigns. Companies using AI for cognitive insight aim to predict future customer choices, personalize ads to target specific customers, and use data on warranty or fraud to decide about trends and what type of customers are likely to benefit from a specific service. The third category of AI adopters is companies using AI to engage with their customers to enhance personalization and impact the customer experience. Bots can identify clues during conversations with customers to make suggestions or customize products to specific needs. Spotify, for example, uses AI to customize music offerings to customers based on their preferences, tastes, and habits, becoming one of the most powerful customer engagement platforms by predicting customer moods and tastes with high accuracy.

Chapter 3.10: AI and Robotics – The Marriage of Mind and Machine

Robotics is an interdisciplinary branch of engineering and science that includes mechanical engineering, electronic engineering, information engineering, computer science, and others. Robotics deals with the design, construction, operation, and use of robots, as well as computer systems for their control, sensory feedback, and information processing. These technologies are used to develop machines that can substitute for humans and replicate human actions. A robot is a mechanical or virtual artificial agent, usually an electromechanical machine that is guided by a computer program or electronic circuitry. Robots may be constructed to take on human form, but most robots are machines designed to perform a task with no regard to how they look.

Of all the applications of AI, robotics seems to be the one that presents more promise and excitement for the business world. Because of the expected value that programmed robots will deliver, especially in manufacturing, the excitement about robotics has grown exponentially. While we have lived with robots for a long time, the ability to program robots to be autonomous presents a historical moment that gets everyone excited, from business leaders to average people. From an engineering point of view, a robot has two parts: the mechanical infrastructure, which refers to the electronics, sensors, and navigation system that the robot needs to move around and gain agility; and the cognitive dimension, which is the dimension that helps robots think, reason, and make decisions. This cognitive dimension is what connects robotics to AI. Many believe that AI and robotics are two different fields and should not be mixed. While this was true for the old generation of robots, which operated under the "rule-based" theory (If/Then), it is not true for the current generation of robots using neural networks. Neural network-based robots do not follow strict rules but decide on their own based on the data, the examples, and the training models they are fed.

The efficiency gains in the manufacturing industry achieved in the last decades are due essentially to the adoption of robotics. In assembly lines, robots execute preprogrammed tasks, such as material handling, welding, and painting, and help companies reduce human errors, produce better quality products, and reduce delivery time. Three factors are driving the interest in robotics: the falling prices of robots, the ability of companies to seamlessly integrate robots into manufacturing, and the ability of robots to perform complex tasks while still collaborating with people. According to a 2022 survey conducted by the International Federation of Robotics, robotics sales grew by 31 percent over 2020. The main markets for this growth were the United States, China, South Korea, Japan, and Germany. China accounted for more than 51 percent of total robots installed in 2021, reflecting the nature of the economic transformation China is undergoing. In the United States, the field of robotics has gained momentum as companies deploy more robotics capabilities at a higher speed. According to the International Federation of Robotics, this trend, if it continues, will increase robotic density in the United States—currently at 164 industrial robots per 10,000 employees, well above the global average of 74 industrial robots per 10,000 employees. This trend will also reverse the outsourcing movement as it becomes cheaper to use robots on American soil.

The introduction of robotics into medicine and healthcare is one of the most important AI breakthroughs. While robots have been used in the medical field for the last 25 years, mainly as a tool for guided precision and repeatability, new developments in this technology are revolutionizing the healthcare industry. Robots now in hospitals can perform many functions to support doctors and nurses. They can take blood samples, transport analyses, and store them. They can easily detect the right vein when drawing blood, thus reducing stress and anxiety for vulnerable patients. They can also carry carts and beds in large facilities and can even help physical therapists administer physical therapy. In the military domain, robots are being developed to help soldiers on the battlefield overcome natural obstacles, difficult tasks, and dangerous missions such as walking around minefields or deactivating explosives. Military robots are programmed to operate in unstructured areas and aim to replace humans in dangerous situations. Most military applications of robotics tend to be programmed robots controlled by an operator and equipped with image recognition software to help robots identify and adapt to what they see.

Chapter 3.11: How Society Will React – Three Scenarios for the Future of Automation

When a new technology emerges, there are many unknowns. In the case of AI, the discussion so far is centered on fear, excitement, and hype. While there are many studies that show the positive impact of the new technology on society, there are also calls for the need to be careful in the way we adopt and use AI. So far, the research conducted on the subject of AI falls into two categories: those who believe that automation will have a positive impact on the economy and the wider society and will create more opportunities for people, and those who take a cautious perspective and think that AI will increase job loss and displace entire segments of the economy, in addition to the risks it will produce like autonomous driving, autonomous weapons, and AI-powered cyberattacks.

The World Economic Forum (WEF 2019) presents a framework for policymakers with three possible scenarios. The first scenario is that society will accept AI. In this scenario, people will accept automation because automation may not develop to its fullest extent. Automation may simply stumble against technical obstacles such as the quality of data and the lack of computing power. In this scenario, we face limited progress in automation that will put a brake on the momentum and power of AI. In the labor market, for instance, it will not be possible to automate every single aspect of a job family because jobs tend to have a high level of task diversity. It will be difficult for companies to automate an entire spectrum of tasks. For example, it will not be possible to automate the job of a financial analyst because, in addition to the repetitive tasks a financial analyst conducts, he or she also performs other tasks that are based on judgment, creativity, and communication. The complexity that is inherently built into the nature of the job makes it simply difficult to fully automate.

The second scenario is that society will reject AI. In this scenario, society may reject automation because of the irrational behavior that tends to characterize human behavior. In this scenario, the WEF argues that humanity may not be interested in full automation, not because automation is not good, but because, as a human species, we are not always rational in our choices. Even though automation may be the right path for us, and because we do not always follow rational paths, we may simply choose not to fully automate our lives. The irrational aspect of our behavior may simply make it difficult for automation to take place, thus forcing us to make a selective choice between what we want to automate and what we don't want to automate. Much of the support for this scenario comes from newly produced research in the field of behavioral economics.

The third scenario described by the WEF is called "planned automation." In this scenario, humanity may reach full automation by developing a rational process for data collection and machine behavior with a clear definition of what is exactly needed. Policymakers can define the ethical and moral framework that will govern this process of full automation. In this scenario, the decision of whether to automate or not, and to what extent we should automate, has to take place through discussion, iteration, and consultation about what society needs and expects from full automation. The government may provide more help to those who may be adversely impacted by the full implementation of automation by providing incentives, such as training programs for displaced workers, ethical guidelines on when AI can be used, and tax breaks to help local governments and disadvantaged communities take advantage of full automation.

Chapter 3.12: The Jobs AI Will Create – Trainers, Explainers, and Sustainers

As with any digital transformation, AI's impact on the job market will also be positive. While researchers agree that AI and robotics will eliminate many jobs, they also agree that AI will create other types of jobs. Those who are poised to benefit from this disruption will be companies that are moving simultaneously on three fronts: changing and reinventing the business process, rebuilding the human capital of the organization, and utilizing data by intelligently capturing its power. According to a study conducted by Accenture in 2018, AI will create new jobs that we have not seen in the past. These jobs are the jobs of trainers, explainers, and sustainers.

The first category of jobs needed is the people who train AI on how to respond adequately to input from customers. There are two types of training needed: those who will help natural language processors and translators understand the meaning of input, and those who will train algorithms on how to imitate human behavior. Examples of this category of jobs include training AI on language subtleties, recognizing sarcasm, and identifying the true meaning of a statement. When companies deploy new algorithms, the result sometimes may not be what the company has predicted. In many cases, AI behaves in a way that is not programmed. What we refer to as the "black box problem" could be solved by training people on how to explain the decision made by AI and make sense of it. Take the example of a bank's algorithm that was built to depict the creditworthiness of customers so the lender can make a judgment about the loan approval process. If the algorithm starts to reject a specific segment of customers, then the lender is expected to explain why this is happening. Explainers are an important category of the AI value chain because they provide AI with the credibility it needs when it is deployed for the first time. In Europe, the need for "explanation" has become a legal requirement. The General Data Protection Regulation (GDPR) requires companies to explain their decisions.

The third category of new jobs that AI is expected to create, according to Accenture, is the type of jobs that will help companies keep AI in check. In their research of more than 1,000 companies, Accenture found that less than one-third of companies trust their AI systems with a high degree of confidence. This is a challenge if companies still don't have full confidence in this new technology. Sustainers will need to understand the purpose of the AI systems implemented and the mission of the organization as well as the type of stakeholders. Their role is to watch for behavior that could go wrong, much like a compliance officer or ethics committee tracking an algorithm's behavior. These new roles reflect the fact that AI is not a set-and-forget technology; it requires ongoing human oversight, interpretation, and governance to ensure it operates safely and ethically.

Chapter 3.13: Countries' AI National Strategies – The Race for Global Leadership

Because of its transformative nature, AI has become a powerful economic tool that will provide countries with a global leadership position. In the last five years, many countries have launched different documents that outline their AI strategy. Some of these countries are Canada, France, China, the UAE, and the United States. The underlying thread for these documents is a political will to organize the countries' resources and capabilities into a coherent framework that will spur innovation around AI and other exponential technologies. While these strategies are different from one another, their common objective is to create an AI ecosystem that targets funding, research and development, education, and private-public partnership. Some of these countries, like China, have also set large budgets to encourage investments in infrastructure and digital transformation, a necessary condition for AI growth.

In the United States, most observers agree that the government has not done what it needs to do to organize resources and rally the nation behind this new technology. What the United States has done, in terms of policy, was the publication of a collection of documents that were produced in 2016 and 2023 to lay out the national AI agenda. The first set of documents was released by the White House Office of Science and Technology Policy in 2016, and the latest was the AI Bill of Rights issued in 2023. All of these documents emphasize the need to develop a long-term investment plan in AI research and the prioritization of investment choices by the federal government. These documents also call for the need to set governance standards and frameworks, as well as investment in building an AI workforce. The contrast between the United States' approach and that of countries like China—which has made AI a central pillar of its national strategy and invested billions of dollars in AI research and infrastructure—has raised concerns about America's ability to maintain its technological leadership in the coming decades.

The economic implications of national AI strategies extend beyond any single country. As nations compete to attract AI talent, investment, and companies, they are shaping the global distribution of economic power in the AI era. Countries that succeed in building vibrant AI ecosystems will enjoy faster growth, higher productivity, and greater influence in international affairs. Those that fall behind risk becoming dependent on foreign AI technologies and losing control over their economic destinies. The race for AI leadership is thus not just about technology; it is about the future of nations and the global balance of power.

Chapter 3.14: The Impact of Automation on Society – Beyond Economics

While the economic impacts of AI are profound, the technology's effects on society go far beyond GDP figures and productivity metrics. As AI spreads, it will reshape how people live, work, learn, and relate to one another. A canvassing report about the best and worst changes in digital life by 2035, published by the Pew Research Center, points out many benefits and challenges that the AI transformation will have in the future. The Pew Research Center, which gathered many AI experts, identifies numerous benefits but also several challenges. Some of the benefits experts have identified include improvements in healthcare and education, better access to knowledge and information for humanity, enhanced connections between people, as well as progress in climate protection and pollution prevention.

However, experts have also identified various challenges. Harms to the development of digital tools and systems: innovation and technology development will continue to be driven by profits and economic power, which may result in them becoming controlling tools rather than liberating ones. Harms that affect human rights: as digital tools become more powerful, privacy and human rights will be increasingly impacted. Sophisticated surveillance tools, facial recognition, and biometric markers will deeply affect human rights and civil society. Additionally, job displacement will create a greater divide between the haves and have-nots. Harms to human knowledge: basic knowledge and the most reliable information will be at risk due to deepfake technology, misinformation, and the vast production of data. This will affect how people perceive facts and reality, with conspiracy theories potentially influencing people's learning and connections. Harms to human health and well-being: the impact of information overload will lead to increased anxiety and depression, isolating people and reinforcing their echo chambers. Harms to communities, governments, and institutions: according to the Pew report, experts believe that political and regulatory institutions will struggle to keep up with the rapid pace of technological advancements. This situation will enable rogue actors to take advantage of the lack of regulations, norms, and standards, exploiting technology for non-legitimate purposes.

Finally, the Pew Research report highlights that 42 percent of experts express equal excitement and concern about the future of technology, while 37 percent state that they are more concerned than excited about the future of AI and technology. These figures suggest that even among those who understand AI best, there is deep ambivalence about its societal implications. The economic benefits of AI are real, but they must be weighed against the risks to social cohesion, democratic governance, and human well-being. The choices we make about how to develop and deploy AI will determine whether its net impact on society is positive or negative.

Chapter 3.15: Chapter Summary & Key Takeaways

The current AI revolution is the result of a convergence of exponential technologies, including computing power, big data, cloud computing, and foundation models.
AI is predicted to increase US productivity by 40 percent and add trillions of dollars to global GDP by 2030.
Generative AI alone could add $2.6 trillion to $4.4 trillion in annual economic value, with the greatest impact in banking, high tech, and life sciences.
The manufacturing sector will benefit from AI-driven asset, fleet, and network optimization, improving efficiency, quality, and competitiveness.
The finance sector is expected to save over $1 trillion by 2030 through AI in risk assessment, fraud detection, trading, and customer service.
Transportation will be transformed by autonomous vehicles and AI-powered traffic management, with significant implications for safety, mobility, and employment.
National security and law enforcement applications of AI raise complex ethical questions about surveillance, bias, and the use of autonomous weapons.
Healthcare will see improved diagnostic accuracy, accelerated drug discovery, and personalized medicine through AI, with potential to save millions of lives.
Cybersecurity is both enhanced and threatened by AI, creating an ongoing arms race between attackers and defenders.
AI adoption is growing, with RPA, cognitive insight, and cognitive engagement representing the three main categories of implementation.
Robotics combined with AI is revolutionizing manufacturing, healthcare, and military operations, with China and the US leading in deployment.
Society's response to AI could follow three scenarios: acceptance, rejection, or planned automation with deliberate governance.
AI will create new jobs—trainers, explainers, and sustainers—while displacing others, requiring investment in retraining and education.
National AI strategies are shaping the global competition for economic and technological leadership, with countries like China investing heavily.
The societal impact of AI goes beyond economics, affecting human rights, mental health, democracy, and social cohesion, requiring careful stewardship.

Chapter Notes

Mark this chapter as completed

Chapter 4 of 11

Chapter 4: The Anatomy of Transformation – Preparing Your Organization for the AI Age

Chapter 4.1: Introduction – Why Digital Transformation Is Not About Technology

The concept of digital transformation refers to a wide organizational effort that senior leadership takes to steer the organization through technology disruptions. It is a process in which the organization uses digital technologies to transform its business model to adjust to market changes. The goal is to help the organization define business value, articulate the vision, and build an agile capability that helps people reimagine the business, navigate the challenges, and learn how to transcend the traditional role imposed by organizational boundaries. Digital transformation is therefore not simply a matter of adopting new tools; it is a fundamental rethinking of how the organization creates value, serves customers, and organizes work.

A study conducted by IDC in 2023 on the importance of digital transformation found that 82 percent of senior leaders agree on the fact that investment in digital transformation is a must. Without investment and a clear prioritization of innovation and digital transformation, companies will be left behind, the report argues. An increase in revenue (38 percent), focus on innovation (35 percent), and improving operational efficiency (30 percent) are among the top priorities of senior leadership according to the IDC study. Yet despite this widespread recognition of the importance of transformation, the majority of efforts still fail to deliver the expected results. A study conducted by BCG in 2020 found that less than 30 percent of digital transformation efforts succeed. A similar study conducted by McKinsey in 2018 points out similar challenges, especially when the transformation involves digital technologies that transcend the traditional role of the IT department, such as generative AI, deep learning, analytics, cloud computing, and mobile technology.

So what are the building blocks of digital transformation and how do companies build them? Why do companies fail to build digital transformation capabilities, and how should they acquire them? The answers to these questions are critical for any organization seeking to thrive in the age of AI, because the same principles that drive successful digital transformation also drive successful AI adoption. The stakes could not be higher: organizations that master these capabilities will shape the future of their industries, while those that do not will find themselves increasingly irrelevant.

Chapter 4.2: The Digital Transformation Framework – Five Pillars of Success

Since the launch of ChatGPT, the world has changed dramatically. AI, and particularly generative AI, dominates the conversation about digital transformation and is influencing the public discourse about policy and strategy-making. AI has become a powerful tool to transform organizations and a catalyst for change, reshaping people's views and perceptions about how to conduct a change strategy. Change management literature seems to agree on the need for organizations going through a transformational initiative to address five key issues: leadership commitment, talent management, innovation and technology, customer centricity, and agility. These five pillars form the foundation of a comprehensive digital transformation framework.

According to a 2020 McKinsey study, leaders who succeeded in digitally transforming their organizations to prepare them for AI technologies have excelled in five areas: strategy, data, technology, and talent and culture. A successful digital transformation starts with a clear strategy on where the company needs to be in terms of technology for the near, mid-term, and long term. The "North Star" discussion that senior leadership needs to engage the team in is a must. This in turn helps the team align with the broader goals of the organization. A data strategy that defines how data is collected, managed, and integrated into the business is a prerequisite of a successful digital strategy. A data strategy helps managers define the process by which data is stored and processed. The backbone of digital strategy is a data strategy that is comprehensive, exploitable, and provides direction on where the company needs to be in terms of market and customer choices.

Successful companies in AI transformation invest in building or acquiring technologies, tools, and models that help them leverage existing capabilities. The goal is to create a powerful framework that guides decision-makers and coaches employees in their day-to-day activities. According to the McKinsey report, successful companies that have implemented AI transformation strategies have invested in developing internal capabilities and integrating AI technologies into their processes. But technology alone is not enough. Talent and culture are equally critical. Organizations must invest in developing the skills of their workforce, fostering a culture of experimentation and learning, and creating incentives that align individual behavior with the organization's transformation goals.

Chapter 4.3: Leadership Commitment – Building Digital Leadership for the AI Era

The presence of a leader who understands technology and has experience in engaging people and managing transition is a critical building block of the digital transformation strategy. Hard and soft skills, combined with a good understanding of the technology, are critical. As digital technologies seep into all facets of the organization, having a technology leader who understands how to extract value from technology while, at the same time, understanding how to drive its implementation, is a necessary condition of success. In the case of AI, leaders who have a good understanding of AI evolution, its business impact, its technical requirements, and how fast it moves have a better chance of succeeding in securing a good transition.

Furthermore, understanding the emerging non-technical requirements is critical to a comprehensive implementation of digital transformation. Understanding the social and ethical implications of AI, for instance, as well as the legal challenges different AI applications pose for the company in the labor and consumer markets, provides an edge for success. Leaders who understand the nuances of these challenges are in a better position to tackle design issues as well as implementation issues. These leaders can conceptualize long-term strategies that are nimble and agile. This long-term vision shields the organization from the consequences of changes in the regulatory, social, and political environment. When leaders understand well the issues of privacy and bias in algorithms, for instance, they are more inclined to take a comprehensive and long-term view of the technology. Rather than rushing into quick fixes using a plug-and-play mode, leaders who have a good grasp of the technology build a wider coalition with a variety of stakeholders and work with the technical and business teams to understand the implications of the technology on the customer and the employee.

In a study conducted by Adeva, an HR company specializing in IT talent management, it was found that for digital leaders to succeed in transforming their organizations, they need to develop a variety of skills that go beyond their initial training and specialization. They need to be innovative, be willing to challenge existing norms and take risks, as well as be able to take bold action to push for change. Digital transformation also requires digital leadership and constant learning. The balance between soft skills, which are sometimes hard to develop, and hard skills, which are sometimes easy to buy, has become a necessary skill for digital leaders. A manager who excels in these two types of skills understands better how technology trends, how to address people issues, and how to deal with legacy and transition challenges. They are better at negotiating deals with a variety of stakeholders and managing resistance.

A new and emerging best practice that has been adopted by many leading organizations is the creation of a new position called the Chief Digital Transformation Officer (CDxO). The goal of this position is to centralize the process of change to create ownership at the top. Because digital transformation cuts across many layers of the organization and cannot be owned by a single function, the CDxO's role is to lead, centralize, and coordinate digital transformation efforts among the C-Suite and functional business leaders. As a Digital Transformation Officer, your role is both tactical and strategic. As a tactical role, you need to define the processes and metrics that ensure that the transformation is on track. As a strategic role, you need to be able to identify the emerging technologies and the business models that will impact the delivery of products as well as the customer experiences. You need to teach people how to embrace change and uncertainty and nurture in them the curiosity to learn and develop. A good Digital Transformation Officer simplifies the working environment by creating tools, policies, and procedures that support employees' transition.

To be a true transformational catalyst, a Chief Digital Transformation Officer needs to help the organization develop structure, processes, and procedures that will serve as the engine of the transformation. They must teach employees about the changing nature of the customer's needs, the economics of competition, the impact of exponential technologies, and the future of work. They must create and develop knowledge to help leadership and employees see the unique way the organization performs in the market, and how the digital transformation fits into the value chain of the organization. Through negotiation, collaboration, and support, a Digital Transformation Officer helps other managers and key decision-makers understand technology trends and shifts and the added value the transformation brings.

Chapter 4.4: The New Skills of the Digital World – What CIOs and Leaders Must Learn

A study conducted by Deloitte University Press in 2017 on the role of the CIO in driving digital transformation found that more than 67 percent of CIOs surveyed consider that digital disruption is the most critical aspect of the business that they must pay attention to. The study found that more companies are using digital to "do business in fundamentally new ways and different ways" rather than simply using digital to improve cost and operational efficiency. When it comes to the skills a CIO has to develop to succeed in today's digital world, the study found that experience in dealing with technology and knowledge about the type of technology that has an impact on the market should make them the leaders of choice for the transformation. When CIOs can combine their technology knowledge with business strategy, they have better chances to leverage ecosystems and create talent and cultural models that challenge the status quo.

The study also identified a few skills that are a must for digital transformation. These skills are experimentation, risk-taking, and willingness to speak up. The study concludes with some key recommendations for organizations engaged in digital transformation. First, IT leaders are digital leaders, but they are not the only ones. While they hold resources and voice, according to the study, they cannot achieve their mission without business leadership skills. Adding business skills to their technology experience has the potential to make them true disruptors, especially when those business skills are combined with cross-functional collaboration, investment, and commitment to change. Second, digital is not an initiative; it is a mindset. A common pitfall to digital transformation is to think of digital as a project or initiative. CIOs and IT leaders, in general, should embrace a mindset that allows them to approach digital as a way of running the business. When IT leaders engage in the implementation of a new technology, their impact goes beyond the IT department.

Third, customer centricity is essential. IT leaders who invest the extra mile to learn about the customer base of the organization and learn how to interact with customer data provide better insights to decision-makers on customer pain points. IT leaders with a customer focus help C-suite leaders understand better the market in which they operate, the needs and wants of their customers, and how customers connect with the brand. Finally, the study concludes that for IT leaders to become true digital change agents, they need to nurture a culture of collaboration and teamwork. This requires breaking down silos, encouraging cross-functional communication, and creating incentives that reward collaboration rather than competition between departments.

Chapter 4.5: Reskilling and Upskilling – The Human Side of Transformation

When digital strategy gets broken down into a set of objectives, goals, tasks, and KPIs, coordination, communication, and collaboration become of paramount importance. A common digital transformation challenge is how to break through human behavior and bureaucratic barriers to communicate the essence of the transformation. Many leaders, absorbed by their KPIs, tend to focus on a single technology or the technology that serves them better, with a narrow KPI in mind, ignoring the fact that transformation requires a comprehensive approach and a new set of skills that have to be rebuilt. A LinkedIn data study, called the "great reshuffle," shows that the skill sets required for a job have changed by 25 percent since 2015 and that this number is expected to double by 2027. This is a critical challenge for business leaders. The skills required to do today's jobs are changing faster than we can adjust, and what is compounding this challenge is that the nature of jobs itself is changing as a result of the introduction of new cognitive tools.

Generative AI, for instance, is expected to impact 40 percent of the UK job market, according to a KPMG report. This will require employees and individuals to seek new skills and new training that help them make the transition. Furthermore, the remote work that was accelerated by pandemic effects added another layer of complexity by requiring skills such as autonomy and independence that are sometimes hard to find. This is why leaders must invest in skills that are interdisciplinary and lateral by nature, such as innovation and critical thinking. Without a deliberate strategy for reskilling and upskilling, organizations will find themselves with a workforce that is not equipped to leverage the new technologies they are investing in.

Reskilling and upskilling are not just about technical training. They are about helping employees develop the cognitive and emotional capabilities they need to thrive in an environment of continuous change. This includes the ability to learn quickly, adapt to new tools and processes, collaborate across functions, and think critically about complex problems. It also includes the ability to manage ambiguity, tolerate uncertainty, and maintain resilience in the face of setbacks. Organizations that invest in these capabilities will have a workforce that is not just prepared for the AI age but excited about it. Those that do not will face resistance, disengagement, and ultimately failure in their transformation efforts.

Chapter 4.6: Teaching Critical Thinking – The Foundation of Digital Literacy

The acceleration of change and the information overload to which employees are exposed requires an independent mind that allows the person to decide on their own. Employees who have high technical skills but lack critical thinking skills will find it very challenging to make decisions and solve complex problems that involve many variables on their own. Critical thinking is a skill employees learn and develop over time. It helps them conceptualize and synthesize data and information to conclude while looking at the issue from different angles. Making inferences is the most important dimension of critical thinking. Inference requires the ability to read data, analyze it, and make an assessment of the decision to be made. When a team member lacks this skill and has constantly to go to their manager for clarification, transformation stalls.

Creative thinking and innovativeness in approaching problems assume that the person has freed himself or herself from biases and mental traps. Cognitive traps, such as the anchoring trap, the status quo trap, and the sunk cost trap, are common mental traps that are hidden in our subconscious. They limit our ability to question assumptions and go beyond what we have been accustomed to. The anchoring trap, for instance, occurs when we rely too heavily on the first piece of information we receive when making decisions, even if that information is irrelevant or misleading. The status quo trap is the tendency to prefer things to stay the same, even when change would be beneficial. The sunk cost trap is the tendency to continue investing in something simply because we have already invested time, money, or effort in it, even when it would be better to cut our losses.

A common method of helping employees develop creative thinking is the six-hat exercise developed by Edward de Bono. This technique teaches learners how to approach a problem from different angles, how to question assumptions, and how to test new solutions. In the six-hat exercise, participants are asked to "wear" different colored hats, each representing a different mode of thinking. The white hat focuses on facts and information, the red hat on emotions and intuition, the black hat on critical judgment and potential problems, the yellow hat on optimism and benefits, the green hat on creativity and new ideas, and the blue hat on managing the thinking process itself. By systematically exploring a problem from each of these perspectives, teams can avoid groupthink, surface hidden assumptions, and generate more innovative solutions. This exercise and many others are meant to help employees engaged in a transformation learn new lateral skills that they can apply to all sorts of problems they face.

Chapter 4.7: Teaching Innovation – Building an Ecosystem for Creative Problem-Solving

In recent years, innovation has acquired a new meaning and a new role in corporate strategy. Driven primarily by changes in demographics and digital technology, innovation has emerged as an imperative of corporate strategy. In a survey conducted by KPMG in 2021, more than 70 percent of CEOs consider innovation as critical to their ability to grow, compete, and earn new market share. However, for innovation to bring value to the organization, innovation has to be managed as a system, so different employees and teams engaged in digital transformation have a common understanding of what innovation is and how it contributes to transforming their business.

A good starting point to design an ecosystem that teaches innovation and engages people with digital transformation is to start with ISO standard 56002:2019, Innovation Management System. In this ISO standard, a viable innovation ecosystem requires five building blocks. The first building block is strategy. Design an innovation strategy that makes innovation the cornerstone of your digital transformation. When employees see the link between what they do and the digital transformation, they engage in a meaningful way. The second building block is communication. A strategy that is not well communicated has a limited impact. So, design awareness and communication campaigns to teach employees and management about the role of innovation as a lever for transforming the business.

The third building block is incentives. Establish appropriate incentives, including non-financial incentives such as rewards and recognition. When employees see the link between their innovative ideas and the incentives they get, they will engage. Help them create linkages between innovative thinking and digital transformation so they can see the benefit of changing. The fourth building block is a tools and intellectual property framework. Create a framework to protect intellectual property to incentivize creative employees to engage with innovation and creativity. When innovative employees willing to transform and change the organization see that their intellectual product is not protected, they get discouraged from engaging in ideas and intellectual work. Employees are willing to take risks when they see that the company is willing to stand behind their intellectual property work. The fifth building block is teams and diversity. Encourage the formation of diverse teams with a mix of people including different disciplines, personal attributes, and backgrounds to encourage cross-pollination that can challenge existing mental schemes and push people out of their comfort zone to change and transform. Reimagining the new business requires a mix of ideas.

Chapter 4.8: Building Customer-Centricity – Putting the Customer at the Center of AI

The disruption that has swept through the world and the change in demographics have made it clear that for organizations to stay relevant, they have to embrace a customer-centric philosophy in which the customer is put at the center of the transformation. Driven by AI-powered tools, customer centricity has emerged as a force of change and as a critical impetus to the digital transformation. The focus on the needs and wants of the customer, and the process by which the customer gets in touch with the brand, has forced business leaders to seek novel technologies that help them understand the economics of customer behavior.

An outside-in approach, in which the voice of the customer gains a seat at the table, serves now as the foundation of strategy formulation and strategy implementation. Efforts to individualize the customer experience have gained momentum, while innovative technologies such as customer analytics, chatbots, and other conversational AI tools have enabled organizations to learn how to anticipate customers' needs and how to proactively manage their experience. AI applications used in customer service have empowered companies to get closer to their customers to understand how customers think and make decisions. As Steve Jobs famously said, "Get closer than ever to your customers. So close that you tell them what they need well before they realize it themselves."

But how do you create a customer-centric philosophy in organizations, and how do you encourage employees to embrace it? What methodologies do you need to teach everyone in the organization to think in a customer-centric way? Here are a few steps that you can take. First, engage your senior leadership to invest in upgrading the current customer experience strategy to include AI-based applications that track customer behavior in real time. Take advantage of AI-based technologies to engage with the customer, understand their motives, how they get in touch with your brand, and what experience they are left with after using your product. Second, conduct real-time market watch to engage in dynamic retargeting, a strategy that allows you to target new demographics based on their previous contact with your brand, products, and offerings. Third, use insights and aggregate data from different touchpoints to understand when and how customers enter into contact with your offerings and what impression they are left with.

Fourth, engage with your customer to co-create options that the customer wants, as well as address the pain points the customer feels. Fifth, test and experiment with your customer journey through a variety of ways to gauge its seamlessness. Feel the key moments the customer goes through when going through your customer journey. By following these steps, organizations can build a customer-centric AI system that not only improves customer satisfaction but also drives business growth and competitive advantage.

Chapter 4.9: Building Enterprise Agility – The Four-Step Approach

Agility, as a concept, refers to the ability of the organization to respond swiftly to market changes while still focused on the core. Relatively new to the literature on transformation, agility has been defined in many ways. Eric Abrahamson, for instance, defines it as the ability to achieve dynamic stability, in which decisions about strategy, focused on the core, are dynamic and take into consideration external market forces. Aaron Smith, a principal at McKinsey, defines agility as the ability of the organization to be dynamic, nimble, flexible, and able to move fast to address change.

In the age of disruption, adopting an agile framework to manage social, economic, cultural, and technological shocks is of paramount importance to the survival of the organization. The traditional way of focusing on stability and efficiency seems to be challenged by market volatility that requires speed and flexibility. A study conducted by McKinsey in 2020 found that the benefits leadership can reap from an agile approach are tremendous. Agility in organizations increases customer satisfaction by up to 50 percent. It also impacts positively employees' engagement and improves time to market, according to the same study.

But how do you create an agile organization? There are many methodologies you can adopt to engage the organization in agile implementation. A focus on a short strategy cycle and the design of a flexible structure are two examples. However, the two most critical dimensions of an agile framework are the creation of self-directed teams and the design of nimble processes that can be easily modified when things change. Creating self-governing teams that work through organizational hierarchy, speed up the decision-making process, and move things is a critical step in creating an agile organization. As Stanley McChrystal, the former commander of the Joint Special Operations Task Force in Iraq, argued, the complexity of the world in which we live requires organizations to adopt a networked and decentralized approach. In his book Team of Teams, he outlines the need to rethink the traditional way of decision-making built around the concept of command and control.

Delegating to lower teams is essential because hierarchy tends to create a bottleneck. When a manager has to approve all the output coming from below, this affects the speed of decision-making. Managers choose not to delegate for many reasons, including lack of trust, lack of expertise at the lower level, or simply previous negative incidents. Team members who are on the ground tend to know more than the manager about the real issues they face. In his book Team of Teams, McChrystal argues that he often, as a team leader, was asked to approve decisions that people under his command knew about their context better than he did. This slowed the speed of the action needed to be taken. Encouraging diversity is also important. While diversity may seem to be a barrier to fast decision-making, it is a good practice to have in teams. Diversity promotes divergent thinking and helps team members develop cognitive skills that help them understand the complexity that surrounds the context of the decision. Diversity is also important as it helps teams avoid groupthink, a common pitfall in which people over time tend to develop similar thinking patterns that make it difficult for them to see difference.

Chapter 4.10: Agile Process – Reviewing Your Decision-Making Process

To manage a digital transformation, you need procedures that are fast and nimble. You cannot decide on a digital transformation issue in the same way you decide on a budget planning issue. Companies struggling with implementing processes struggle because either the process is too complex, that is, not intuitive, or it has not been well taught to employees. A process that is ill-designed, or not well-explained, creates bottlenecks for the workflow and adds more waste. People spend endless time in meetings asking basic questions about steps that are sometimes obvious. We often hear people saying that the process is getting in the way of transformation. This can be true only if the process is complex for people to digest and understand. Mature organizations invest tremendous time to create intuitive processes that are standardized so people can implement them seamlessly.

Generally speaking, two problems hinder the performance of a process. The first is a lack of governance. In this situation, the process is not well mapped out or the person in charge is not identified correctly on the process map. People spend a lot of time figuring out where to go to execute simple tasks. Organizations should spend time mapping out different workflow procedures that relate to the digital transformation's programs and activities. The second obstacle that tends to hinder the performance of an agile process is the lack of employees' skills. A sub-process can hinder the main process if employees lack the competence they need during the execution of that process. Here organizations need to spend time teaching people how to use the process and provide them with information, knowledge, and training to become nimble. Create a common language around that process so you can easily standardize it.

An agile process is not just about speed; it is about responsiveness. It is about creating a system that can adapt to changing circumstances without losing coherence or direction. This requires clear governance, well-defined roles, and a culture that values experimentation and learning from failure. Organizations that master agile processes will be able to respond to market changes, technological shifts, and competitive threats more quickly and effectively than their competitors. Those that do not will find themselves bogged down in bureaucracy, unable to move fast enough to seize opportunities or avoid threats. In the age of AI, where the pace of change is accelerating, agility is not optional; it is essential for survival.

Chapter 4.11: Putting It All Together – A Comprehensive Approach to Transformation

Digital transformation is not a destination; it is a journey. It is not a single project or initiative; it is a continuous process of adaptation and renewal. The organizations that succeed in this journey are those that understand that transformation is not just about technology. It is about people, culture, processes, and strategy. It is about creating an organization that can learn, adapt, and innovate faster than its competitors. It is about building the capabilities that will enable the organization to thrive in an environment of continuous change.

The five pillars of digital transformation—leadership commitment, talent management, innovation and technology, customer centricity, and agility—are not independent of one another. They are interconnected and mutually reinforcing. Strong leadership creates the vision and the mandate for transformation. Talent management ensures that the organization has the skills and capabilities it needs to execute the vision. Innovation and technology provide the tools and the mindset for creating new value. Customer centricity ensures that the transformation is grounded in the needs and desires of the people the organization serves. Agility enables the organization to adapt and evolve as circumstances change.

As organizations embark on their AI transformation journeys, they should keep in mind that the goal is not to adopt AI for its own sake but to use AI to create value for customers, employees, and society. This requires a clear understanding of what AI can and cannot do, a realistic assessment of the organization's readiness, and a commitment to responsible and ethical implementation. It also requires patience, persistence, and a willingness to learn from failure. The path to transformation is not easy, but for organizations that are willing to invest the time, effort, and resources required, the rewards are immense. The age of AI is not coming; it is already here. The question is not whether your organization will be transformed, but whether you will lead that transformation or be swept along by it.

Chapter 4.12: Chapter Summary & Key Takeaways

Digital transformation is a comprehensive organizational effort to use technology to transform business models and adapt to market changes, not merely a matter of adopting new tools.
While 82 percent of senior leaders agree that digital transformation investment is essential, less than 30 percent of transformation efforts succeed, often because they focus on technology rather than the fundamentals of organizational change.
The five pillars of successful digital transformation are leadership commitment, talent management, innovation and technology, customer centricity, and agility.
Leadership must understand both the technical and non-technical aspects of AI, including social, ethical, and legal implications, to guide transformation effectively.
The role of the Chief Digital Transformation Officer (CDxO) has emerged to centralize and coordinate transformation efforts across the organization.
Leaders need skills in experimentation, risk-taking, cross-functional collaboration, and customer centricity to drive successful transformation.
Reskilling and upskilling are critical; the skill sets required for jobs have changed by 25 percent since 2015 and are expected to double by 2027.
Critical thinking and creative thinking are essential capabilities that help employees make independent decisions and avoid cognitive traps like anchoring, status quo, and sunk cost biases.
Innovation must be managed as a system with strategy, communication, incentives, intellectual property protection, and diverse teams as key building blocks.
Customer centricity requires putting the customer at the center of transformation and using AI tools to understand and anticipate customer needs.
Enterprise agility involves creating self-directed teams and nimble processes that can respond quickly to change, with delegation, diversity, and clear governance as key enablers.
Agile processes require intuitive design, clear governance, and adequate employee skills to avoid bottlenecks and enable fast decision-making.
Digital transformation is a continuous journey of adaptation and renewal, not a one-time project or destination.
The five pillars of transformation are interconnected and mutually reinforcing, requiring a holistic and integrated approach.
The goal of AI transformation is not technology adoption for its own sake but creating value for customers, employees, and society through responsible and ethical implementation.

Chapter Notes

Mark this chapter as completed

Chapter 5 of 11

Chapter 5: The Context Imperative – Mapping the Terrain Before Building Your AI Management System

Chapter 5.1: Introduction – Why Context Is the Cornerstone of AI Governance

Before an organization can build a viable Artificial Intelligence Management System (AIMS), it must first understand the terrain on which it is building. This is the fundamental insight behind Clause 4 of the ISO/IEC 42001 standard, which requires organizations to analyze and evaluate both internal and external factors that may affect the performance of their AI management system. Context analysis is not a bureaucratic checkbox to be ticked; it is the foundational act of sense-making that determines whether the entire AIMS will be relevant, effective, and sustainable. Numerous reasons make it imperative to invest time, effort, and resources to study, analyze, and understand the context of your organization, and in the age of AI, the consequences of neglecting this step are more severe than ever before.

Analyzing the context of your organization helps you first understand the bigger picture of the conversation on AI and how your organization fits into that conversation. Context analysis provides you with a comprehensive view of the historical background, as well as the social and political factors that drive the discussion about AI. It helps you learn about how laws, regulations, and standards evolve in different countries, how these laws and regulations may affect your products and services, and your overall AIMS performance. Furthermore, context analysis helps you identify key stakeholders that may be affected by your AIMS. When you analyze the context of your organization, you have a better perspective on how your AIMS affects people and customers, which in turn should provide you with opportunities to improve the effectiveness of your decision-making process. Studying the context of the organization helps you also identify the links that exist between your products and services, your AIMS, and different market players. Learning about who is doing what in the AI sphere provides you with the opportunity to identify connections that link your products and services with other actors and players, so you are better informed about trends, emerging issues, and risks.

Chapter 5.2: The Four Pillars of Context Analysis – Bigger Picture, Stakeholders, Links, and Agility

When looking at the context of the organization, the ISO/IEC 42001 standard requires that the organization analyze and evaluate internal and external factors that may affect the performance of the AIMS. In the context analysis, you need to look at a variety of factors that may affect the performance of your AIMS and the performance of your overall organization. To help you understand the requirements of the standard on how to analyze your organizational context, it is helpful to think of four key pillars that context analysis supports: understanding the bigger picture, identifying key stakeholders, learning about links and interconnectedness, and improving agility.

Understanding the bigger picture means understanding the political, social, cultural, and economic forces behind the AI evolution. It means learning how society is shaping the discussion about AI, and how AI is shaping people's perception about your products and services. It means understanding and identifying the actors and the issues driving the AI conversation. Identifying key stakeholders means learning about the different stakeholders that will be affected by your AIMS, understanding how and in what way other players and actors may affect your AIMS, and identifying opportunities to collaborate or compete with stakeholders for a better AIMS implementation. Learning about links means learning about who is doing what in the AI sphere and how you can connect with them to improve your AIMS, identifying opportunities to connect or compete with different AI players, and learning how to create coalitions to push and protect your interests. Improving agility means understanding the changing environment of AI and how this may affect your AIMS, understanding how to manage change better, learning about mechanisms that help you adapt better to the context, and learning how to improve your responsiveness and anticipation strategies.

First, you need to identify stakeholders—referred to as "interested parties" in the language of the standard—such as providers, developers, users, producers of AI systems and applications, partners, AI subjects, and authorities. These stakeholders are defined by another ISO/IEC standard (ISO/IEC 22989:2022 AI concepts and terminology) and are an important component of the analysis and a requirement of the standard. ISO/IEC 42001 also requires you to look at the ethics of your organization, the value system, and the management practices that govern the functioning of your organization. Putting together a new management system for a highly complex issue such as AI requires a fundamental mindset shift that is not easy to do.

Furthermore, and because ISO/IEC 42001 is a certifiable standard, a good context analysis is also crucial to the auditing and certification process. Conducting the analysis, and documenting this process correctly, gives you a better chance to pass the certification process. It helps the auditor go beyond technical analysis to understand your philosophical approach to AI and read better your intentions about how you deal with powerful and transformational technology such as AI. It shows the auditor your ability to think critically about transformational issues and how you anticipate future risks and challenges that may impact the performance of your AIMS, and the overall organization.

Chapter 5.3: The Competitive Landscape – Analyzing How AI Is Reshaping Your Industry

When looking at the competitive landscape and stakeholders' analysis, your first step is to understand the technology trends and the disruption AI technology is having on your company and the overall competition. Conducting an industry analysis to understand how AI impacts your market and your industry is a critical step to learning about the organizational context. Depending on the sector in which you operate, an industry analysis helps you ask three critical questions. How is AI reshaping my industry and to what extent is the AI trend disrupting the overall competitive landscape? How is AI impacting key market segments and channels in my industry, and how is the competition responding to this trend? How is AI forcing the repositioning of current players while bringing new entrants that may be of threat to my existing products and services?

Many frameworks may help you address these questions, but a good well-known market analysis framework is Porter's Five Forces model. In this model, you analyze five elements. The first element is the competition and how AI is impacting the industry in general. The second element is the potential threat of new entrants from AI startups and from companies that have adopted AI technologies. The third element is the power of third-party AI suppliers, such as developers, designers, and platform providers. The fourth element is customers' choices and customer trends. The fifth element is substitute products from AI that may displace your existing products and services.

Once you conduct the market analysis, you turn to the analysis of internal organizational capabilities. In the internal analysis, your goal is to understand to what extent the AI-induced change in the market affects the internal functioning of your organization. This will allow you to think critically about how to prepare your organization for the change and transformation AIMS will have on your existing capabilities. Here are five guiding questions that you need to ask. What resources—financial, human, physical, and technological—do I need to invest in to make my AIMS a viable one? What stock of intangible resources do I need to build or invest in to make my AIMS a better-performing system? Intangible resources such as knowledge, trust, innovation, reputation, ideas management systems, research and development, and brand. What organizational capabilities do I need to develop or enhance to make my AIMS a successful one? An organization's capability is the capacity that helps you deploy resources in a unique way that is particular to you. Capabilities tend to emerge over time, and they are a combination of many things including knowledge, skills, talents, network, and expertise.

Chapter 5.4: Stakeholder Analysis – Identifying Who Matters in Your AI Ecosystem

The standard requires a stakeholder analysis. In this analysis, you need to identify the different types of stakeholders involved with your AIMS as well as their needs. While most organizations should have already identified their main stakeholders, in the case of AI, and depending on your sector of activity, a new list of stakeholders is a must. This is a requirement of the standard that must be documented. The standard lists six types of stakeholders that are well defined in the ISO/IEC 42001 and the ISO/IEC 22989:2022 AI concepts and terminology standards. These stakeholder categories reflect the complex and multi-faceted nature of AI systems, which involve not just the organization deploying the system but also the developers, users, subjects, and authorities who interact with or are affected by it.

To prepare for the assessment, audit, and certification process of your organization, you should start with Section 4 of the standard. Section 4 of ISO/IEC 42001 requires the following actions. First, analyze the context of the organization to understand how different internal and external forces shape the development and the functioning of AI systems, and how AI systems impact the overall performance of the organization. Second, study different stakeholders' relationships with the organization to identify their needs, expectations, and their requirements to create an integrated AI management system. Third, define and document the scope of the AIMS, its boundaries, where it starts and where it ends, as well as its relationship to other internal management systems. This should be defined in a document that will be presented during the auditing process. Fourth, develop a holistic approach to AI by putting together a comprehensive AIMS that includes all components of ISO harmonized management systems. The components of harmonized management systems are leadership, planning, support, operations, performance evaluation, improvement, controls, and objectives.

To conduct a stakeholder analysis effectively, you can follow a structured process. First, create a list of stakeholders that you think will have an interest in your AIMS or might be impacted by it. Typically, a list of stakeholders may include customers, employees, vendors, regulators, investors, and members of the wider society. Stakeholder groups may have a different level of interest and may be impacted differently, so it is critical that you define and quantify if possible the level of stakeholders' involvement in your AIMS, their interest, and how AI will impact them, so you can identify the impact. Second, identify the concerns and needs of your stakeholders through surveys, focus groups, direct communication, or meetings. Documenting stakeholders' concerns will be of great help during the risk management phase. Third, communicate the documentation on risk to teams involved in the production of AI systems throughout the AI project lifecycle.

Chapter 5.5: The Legal Context – Navigating a Shifting Regulatory Landscape

The second component of your organizational context analysis is legal. Because of its powerful and transformative nature, AI is poised to impact every aspect of our lives. As a general-purpose technology, AI will have a broad impact on people's lives, so understanding the liability and the risk of this technology, as well as the legal exposure of those who provide AI products and services, is critical. This is why understanding the legal context of AI is an important step to the viable functioning of your AIMS. While AI-specific regulations are changing by the day, as many countries and jurisdictions are catching up to the speed of technology advances, it is nevertheless helpful to pay attention to some of the legal frameworks and principles that have been already adopted. As of the writing of this book, the EU General Data Protection Regulation, the EU AI Act, the US AI Bill of Rights, the OECD principles, the UNESCO AI principles, and the NYC Local Law 144 of 2021, which prohibits discrimination in Automated Employment Decision Tools, provide a good indication of the future of AI regulations, the global legal trends, and regulators' intentions in different jurisdictions. They serve as a reference point to the future of the AI regulatory framework.

The General Data Protection Regulation, or GDPR, is a legislation that was adopted in the EU in 2018. The goal of the GDPR was to unify different European legal frameworks that address privacy and provide EU citizens with legal means to protect their personal information. It was also meant to provide EU citizens with the means to file a complaint against violation of these rights as it provides many key definitions and establishes liability and penalties for violation of the act. While the GDPR does not address AI per se, it defines many concepts that relate to AI systems. Personal data refers to information that can identify, directly or indirectly, individuals. Data processing refers to the process of manipulating data during all the stages of data usage, from collection to recording, structuring, and storing the data. Data subject refers to customers, employees, partners, and all users of data and AI systems. Data controller refers to the person who has decision authority on where and how the data is used. This is a critical definition because it puts the liability directly on the data processing governing body.

Furthermore, the GDPR establishes seven principles to protect and account for data processing. All these principles could have a direct link to ISO/IEC 42001. These principles are: processing data must be used for lawful purposes and must be fair and transparent to users; data must be processed according to the purpose for which it has been collected (purpose limitation principle); organizations collecting data should limit data collection to only the data they need and avoid casting wider nets (data minimization principle); data must be accurate and kept updated; organizations should store only the data they need to avoid storing data that they don't need (storage limitation principle); when processing data, data must be protected by encryption to respect the integrity and confidentiality of the data; and organizations processing data are liable for compliance with GDPR principles. All these legal stipulations directly affect your AIMS. The analysis of the context should spell out the process by which your organization intends to address these legal requirements. Keep in mind also that these GDPR legal requirements are similar to many of the standard clauses that relate to data processing. Addressing them at the same time and documenting this process would be the right thing to do as it saves you time and helps you get ahead of the game.

Chapter 5.6: The EU AI Act – A Landmark in AI Regulation

In April 2024, the European Parliament approved the EU AI Act, a landmark legislation that has been in negotiation for some time. This is the first AI legislation in the world that aims at regulating AI, and it is the first legislation that specifically addresses generative AI systems. There are a few key ideas that characterize this legislation. First, the act focuses on machine learning systems and intelligent systems that provide recommendations, predictions, and statistical models that may impact the decision-making process, thus excluding traditional software and applications. By doing so, the act brings into focus the discussion on AI automated systems, intelligent systems, and foundation models. Examples of intelligent systems subject to the act are machine learning systems, including supervised, unsupervised, and reinforcement learning, logic and knowledge-based approaches, statistical and Bayesian approaches, search and optimization methods, and large language models.

Second, the act makes an obligation for companies developing foundation models to assess and mitigate risks to health, safety, environment, and human rights. Foundation models providers must register their models in the EU database before releasing them to the public. They must also comply with the transparency rules, such as disclosing the copyrighted materials and data used to train foundation models. Third, the act bans AI for biometric surveillance, emotion recognition, and predictive policing. These AI systems are deemed so dangerous that they are simply made illegal. Finally, the act adopts a risk-based approach in which it identifies four levels of risks of AI systems: unacceptable risk, high risk, limited risk, and minimal or no risk. Each category of risks has a specific set of rules and controls.

In the unacceptable risk category, algorithmic systems that use social scoring, facial recognition, and subliminal techniques that may cause harm are simply prohibited. Examples of such systems are systems that use gender, race, ethnicity, citizenship status, religion, or political orientation to make decisions; systems that identify location, past criminal behavior, and profiling, such as in predictive policing; and real-time remote biometric identification systems such as facial recognition. In the high-risk category, AI management systems are subject to a regime of certification, controls, and conformity assessment to test their compliance with specific and existing laws and regulations, and to make sure that their output does not create harm. All AI management systems that relate to critical infrastructure, biometrics, education, training, employment, and essential public services, like welfare benefits, law enforcement, migration, and border control systems, and the administration of justice, are considered high-risk AI management systems and therefore subject to a certification process. From an organizational perspective, most of the high-risk management systems fall under the functions of HR, accounting, finance, customer service, IT support, and legal.

In the limited risk category, the EU Act identifies systems that relate to chatbots and deepfakes. Deployers of these types of systems must comply with transparency requirements. They must disclose that the interaction with the chatbot is an interaction with a machine, not with a human, to avoid harm related to deception and risk of impersonation. Providers of these systems must ensure that users are informed that they are dealing with a chatbot, not with a human. In the low-risk category, AI systems are not subject to the EU AI Act. They are left to the ethical frameworks and other norms and values that regulate the overall industry. Finally, it is important to mention that AI applications do not operate in a legal vacuum. In many instances, they are regulated directly or indirectly by the already existing laws and regulations in many countries. Legislation that pertains to antidiscrimination, workplace safety and health, consumer protection, and credit and finance can, and may, be applied to decisions that are made by AI systems, especially when there is evidence of harm occurring as a result of the automated decision. So, it is critical that the context analysis takes a broader perspective and identifies possible legal implications of different risks, including the unknown ones that may be the result of the rapid rise of generative AI and Large Language Models.

Chapter 5.7: The US AI Regulatory Landscape – A Patchwork of Federal, State, and Local Rules

In the United States, the regulatory framework of AI and autonomous systems is still fluid and evolving. Many federal agencies, states, and other local government agencies have issued regulations, directives, and guidelines to address the issues of discrimination and bias in automated systems. The joint statement on the enforcement effort against discrimination and bias in automated systems issued by the Civil Rights Division of the US DOJ, the Consumer Financial Protection Bureau, the Federal Trade Commission, and the Equal Employment Opportunity Commission in April 2023 clarified that software tools and new intelligent systems, such as AI used to organize workflow, are subject to the current legislation that protects civil liberties and individual rights, particularly when discrimination is the result of how organizations use data and datasets, how their model is trained, and how the entire system is designed. The Equal Employment Opportunity Commission also issued a federal guideline in April 2023 to clarify the use of automated tools in HR selection procedures, reiterating that automated tools are subject to Title VII of the Civil Rights Act of 1964. Employers may be responsible for disparate treatment and adverse impact, and may even be liable for third-party tools, such as employer's agents and vendors of AI tools and automated systems.

The Equal Credit Opportunity Agency introduced a change in its regulation to make it illegal for the use of biased algorithms. Discrimination in automated systems that are based on race, color, religion, national origin, sex, marital status, age, or public assistance benefits is prohibited. The Fair Credit Reporting Act saw the Federal Trade Commission take enforcement actions and issue directives to emphasize the need for companies to pay attention to discrimination in automated systems in employment, housing, credit, insurance, or other benefits. The California Consumer Privacy Act provides more protection to consumers on how companies use their data, including the right for consumers to opt out. In the financial service sector, the Office of the Controller of the Currency, the Consumer Financial Protection Bureau, the Federal Trade Commission, the Financial Industry Regulatory Authority, and the Commodity Future Trading Commission all issued directives in which discrimination and bias in automated systems are directly or indirectly addressed. In the medical field, the FDA released guidance recommending that certain predictive devices that are powered by AI applications should be regulated as medical devices.

At the state level, according to some estimates, 14 states enabled or adopted legislation on AI, bringing this to a total of 26 bills as of 2023. The bills cover many aspects of AI risk, from harm to consumers to facial recognition and hiring discrimination. In the insurance industry, and the state of Colorado, according to researcher Shea-Brown, external data as well as algorithms and predictive models that discriminate against a protected class are prohibited. Perhaps the most pronounced legal framework in the United States that addresses directly bias and discrimination in AI is the New York City Local Law 144 issued by the NYC Department of Consumer and Worker Protection in 2020. To this day, this is the most aggressive AI regulation framework. It prohibits discrimination in Automated Employment Decision Tools and sets high penalties for violations. In this law, AEDT is defined as "any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence" that ranks and recommends applicants for an employment position. The law requires, among other things, that companies adjust selection tools to link them directly to essential job functions, mandating algorithm audits. When using a third-party assessment tool, the law requires that companies must audit the assessment tool and choose the least discriminatory assessment method available.

In April 2022, a group of senators introduced the Algorithmic Accountability Act. The legislation would require organizations to conduct mandatory impact assessments of their automated systems. This act, which is an updated version of the Algorithmic Accounting Act of 2019, defines automated decision systems as software or processes that use computation to produce outputs that are used to make decisions. ADS that impact education, employment, financial services, housing, and other services critical to consumers' choice fall under the scope of the Act. The Act applies to organizations that have more than $50 million in average annual gross receipts. The Federal Trade Commission will be tasked to implement the Act. The Act was rejected by the Congress, and it never made it to become a law. Finally, it is important to mention that at the federal level, the White House published the AI Initiative in 2019. This initiative aims to promote innovation, funding, and research in the AI field, and addresses issues that relate to privacy, civil liberty, and other national values. In 2022, the White House also published a blueprint for an AI Bill of Rights, in which the White House outlines many of the principles that promote trustworthiness and responsible AI.

Chapter 5.8: The Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence

In October 2023, President Biden issued a sweeping Executive Order that regulates AI. The goal of the Executive Order is to protect American consumers; ensure that AI systems are safe, secure, and trustworthy; and help the United States build the capacity needed for US global leadership. The Executive Order also charges many federal agencies with regulating AI through standards and continuous monitoring of AI risks. The Executive Order mandates several important actions. Organizations developing powerful foundation models that may affect national security, national economic security, or national health security must share with the US government the results of their safety tests and other critical information. The National Institute of Standards and Technology is tasked with developing rigorous standards to ensure that AI systems are safe, secure, and trustworthy to be applied to protect critical infrastructure. The order also directs the protection of Americans' privacy through federal support and privacy-preserving research, addresses the issue of algorithmic discrimination through training and best practices sharing, supports the responsible use of AI in healthcare and education, and works with other nations to expand bilateral, multilateral, and multistakeholder engagements to collaborate on AI.

To regulate AI effectively, it is important to understand the unique challenges that this technology poses. Tom Wheeler, in a piece published by the Brookings Institution, suggests an interesting perspective. He argues that AI poses unique regulatory challenges due to its distinct nature as a technology. The rapid pace at which AI evolves presents a big challenge for the US government. Wheeler identifies four primary challenges. The first is velocity. Unlike previous technologies, AI exhibits remarkable speed in both development and adoption. The second is what to regulate. AI is a highly complex and multifaceted technology. Containing it in one piece of legislation simply will not work. As a general-purpose technology, AI permeates many aspects of our lives, so a one-size-fits-all regulatory approach is simply inadequate. The third is who regulates. The structure and philosophy of the US government were designed during the Industrial Revolution when the pace of innovation and technological advancements was slower. At that time, the government had sufficient time to study, investigate, and conduct hearings to regulate innovations. However, the lack of agility within the current governmental framework presents a major barrier to keeping up with AI technological advances. The fourth is how to regulate. The notion of licensing AI, the notion that has been advocated by many business leaders such as Sam Altman, the founder of OpenAI, poses its own set of challenges. Wheeler argues that the licensing system, like in the case of the FDA and FCC, can impact the competitive landscape. The initial license holders tend to dominate the market and subsequently advocate for new rules that may impede the entry of new competitors.

Chapter 5.9: Ethical AI – Building Responsible and Trustworthy Systems

Because the legal standard may or may not apply to a specific situation, and because of the speed at which AI technology has been moving, ethics and ethical reasoning should also be considered during the context analysis. The goal is to show certifying bodies that the organization is aware of the risks, the ethics, and the moral challenges AI and automated systems are posing to society. In situations where there is a lack of legal clarity and where the use case is a case of gray area, ethical frameworks can be of great help to decision-makers as they provide guidance on what to do and not to do. The ethical reasoning is also captured by the spirit of the ISO/IEC 42001, so it will be helpful during the certification process to show that the AI management systems are guided by ethical reasoning and subscribe to the objectives advocated by the Standard.

There are many ethical frameworks that organizations can implement. Some of these frameworks have been published by ISO, which you can find in the following two standards: ISO/IEC TR 24028:2020 Overview of trustworthiness in artificial intelligence, and ISO/IEC TR 24368:2022 Artificial intelligence, an overview of ethical and societal concerns. Other frameworks and principles have been also published by international organizations such as the European Commission's High-Level Expert Group, UNESCO, and OECD. Technology and consulting companies, such as Microsoft, IBM, PWC, Deloitte, BCG, and many others, have also published sound frameworks. These frameworks are an excellent first step to establishing a Responsible AI practice. They all agree on the need for senior management to engage in building a set of ethical principles that guide the organization during the design and implementation of AI applications. However, the ethical principles that seem to get a wider consensus among different stakeholders are the following: the principle of do no harm, the principle of fairness and non-discrimination, the principle of human oversight, the principle of explainability, and the principle of robustness. These principles capture well the essence of fundamental human rights and should be considered as an ethical imperative as they help you build a sustainable and trustworthy AI system.

Chapter 5.10: The Principle of Do No Harm – The Foundation of Ethical AI

In philosophy, the do no harm principle is a central and foundational principle. According to liberal political philosophy, humans are free to act in any way they intend unless their actions harm others. When their actions may be a source of harm to others, then restrictions on their freedom may apply. For AI technology, harm can occur in many ways. It can occur in the way AI technology is used for surveillance purposes or social manipulation when individuals are coerced directly or indirectly to do things against their will. Harm can also result when biased decisions are made by the AI systems or during data processing. In extreme cases, harm can also occur in automated weapons that are fueled by automated algorithms. The prevention of harm in AI also requires that the AI systems be safe and secure and that they don't affect the moral and physical integrity and well-being of the individual.

In the context of AI, author Smuha distinguishes between three types of harm: the harm that affects the individual, like in the case of facial recognition that may negatively affect people of color; the collective harm, which may affect a group of people that share similar traits; and the social harm which, according to the author, goes beyond the individual and the group to affect the way a society functions. Examples of generative AI applications, such as the ones that produce disinformation at scale, may affect the social fabric of society, with harmful consequences on its political institutions and democratic process. ISO/IEC 42001 requires that you define harm. Clause B.5.5, under the title "Addressing the Societal Impact of AI Systems," the standard requires that you assess and document the potential physical harm of your AI systems on users. You should also identify processes and tools that help you prevent harm and take corrective actions when potential harm is identified. Human dignity and the physical and mental health of users should not be compromised by AI technology.

To operationalize the do no harm principle, organizations should ask a series of critical questions. Do our AI systems identify harm? Do we have a list of possible harms that can result from the use of our applications, products, and services? Do we engage with stakeholders to identify the impact of our AI systems and identify harms from the user's perspective? What laws, regulations, ethical frameworks, and standards have we adopted to define harm in AI systems, applications, and products? Have we conducted an impact assessment of AI systems to identify possible harm? These questions should be asked at every stage of the AI project life cycle, from design through deployment and monitoring, to ensure that the do no harm principle is not just a theoretical commitment but a lived practice.

Chapter 5.11: The Principle of Fairness and Non-Discrimination – Building Equitable AI

Fairness means providing equal treatment. The issue of fairness has been debated in philosophy, law, and politics for some time, and now it has made its way to artificial intelligence and autonomous systems. The EU AI Act defines fairness as "a commitment to ensuring equal and just distribution of both benefits and costs and ensuring that individuals and groups are free from unfair bias, discrimination, and stigmatization." Simply said, fairness refers to the equal treatment of individuals when it comes to access to goods and services provided by the organization. But how can this concept, which seems to be so interwoven with human nature, be translated into the context of Artificial Intelligence? How can a machine be fair or unfair? And how can we make sure that machines, which are supposed to be objective and neutral, act and produce fair results?

In Artificial Intelligence, a new field called algorithmic fairness has emerged in recent years. The focus of this field is the study of machine learning models and algorithmic decisions that may create bias and adverse impacts on the way they process data and make predictions. Examples of algorithm bias have been widely discussed in the financial sector, where loan applicants may receive different treatments by the algorithms, and in the criminal justice system, where prediction has not been fair to a particular ethnic group. In studying algorithm unfairness, it is critical to point out the fact that algorithm models by themselves are not the source of the bias. It is the data we feed into the model and the way we train the model that can be the source of discrimination. Data that we feed into the model when it is poisoned intentionally and unintentionally perpetuates discrimination. Machine learning models that are trained on biased data, or trained on non-representative samples, will reproduce discrimination. To remedy this situation, researchers have suggested many methods and techniques. Methods such as equal odds positive, predictive parity, and counterfactual fairness are some of the technical suggestions that can alleviate the bias in predictive models.

The Center for Equity, Gender, and Leadership at Berkeley Haas Business School put together a playbook to help business leaders understand the issues of fairness and algorithm bias. To help your organization de-bias algorithms, the center suggests three actions. First, diversify teams in charge of AI applications. For the teams putting together AI systems, diversity seems to be a critical factor in helping developers build an AI system free of discrimination. Diversity of background, experience, and perspective helps teams avoid blind spots, groupthink, and many other cognitive biases that are the result of people being unable to question what seems to be obvious to them. The center suggests the concept of interdisciplinarity in teams building AI applications. Organizations should engage individuals in social studies, humanities, and other fields that may not be the forte of typical AI engineers. Finally, provide teams with the necessary training on issues of fairness, bias, adverse impact, and disparate treatment, so AI developers and builders are aware of the issues. Second, design AI systems with Responsible Artificial Intelligence principles from the start. This can take many forms. Audit of the datasets used, documentation of the source and the origin of the data, and assessment of the representativeness of the groups and samples used in the data. The goal is to infuse RAI principles into AI applications during the design stage. Third, governance and leadership are essential. Without governance and leadership involvement, none of the previous procedures work. When senior management and governance bodies are engaged in these principles, employees down the chain of command working on the actual development of AI applications are aware of the stakes, the harm, the risks, and the rewards that are attached to RAI principles.

Chapter 5.12: Human Oversight and Respect for Human Autonomy – Keeping Humans in Control

The fundamental idea of this principle is that AI technology should not be in control of human will and actions. The oversight, governance, and control should always be in the hands of humans. AI technology, from this perspective, regardless of how autonomous it is, should always be looked at as an enabler that empowers humans and complements their work. From this perspective, and according to the EU Commission, respect for the human autonomy principle means that a technology should not be built to control, deceive, or manipulate a human will. A human-centric approach to designing AI products in which the focus is on enabling humans rather than controlling them should be adhered to during all steps of designing and implementing the technology.

Another concept related to oversight is the concept of the Human-in-the-Loop principle well known in machine learning programs. Particularly for high-risk decisions, where humans need to provide input, interact with the system, and review the output, the human-in-the-loop principle provides a critical safeguard for the training, the simulation, and the initial deployment of AI systems. To operationalize this principle, organizations should ask the following questions. Do we have a governance model that helps monitor the impact of the technology on human rights? Have we assessed the impact of our new technology on users and their ability to control the application? Have we decided who will be in control of the technology and how this control is exercised? Have we set a clear accountability process for customers to use to complain or challenge the technology? These questions help ensure that human oversight is not just a rhetorical commitment but an operational reality embedded in the design and governance of AI systems.

Chapter 5.13: The Principle of Explainability – Opening the Black Box

Explainability refers to the ability of the organization to provide a meaningful explanation of the decision made by the AI system. Explainability is a critical concept for establishing trust in AI systems. It provides openness and transparency in the decision-making process. Explainability is also important because it provides the means to those affected by the AI decision-making process to challenge the decision, which helps the system to be more transparent and fair especially when the wrong decision generated by the system is corrected. The ISO/IEC 42001 Standard defines explainability as the ability of the organization to explain to those affected by the decision so people can see the input, the data, and the information that went into the decision-making process. The goal, of course, is to help stakeholders and those affected by the automated decision to understand in a meaningful way how the decision was reached, and the different factors that went into the decision.

But what does explainability entail? And how is it different from interpretability? And how should we build an explainable system? A great framework to understand the issues of explainability is the work conducted by authors Phillips, Hahn, Fontana, Yates, Greene, Broniatowski, and Przybocki in 2021. According to the authors, explainable AI requires four principles. The first principle is explanation: the act of providing evidence and reasoning about how the system reached a specific decision. This could include what went into the decision, but also what didn't go into the decision. The second principle is meaningfulness: the process by which users understand the system's explanation. What constitutes a meaningful explanation may be different for different stakeholders. A programmer may understand the decision in different meaningful ways than the buyer of the final product. The third principle is accuracy: for a decision to be explainable, its explanation has to be accurate. Accuracy of the explanation, or the lack thereof, according to the authors, does not affect the explanation and the meaning of the final decision. The accuracy here applies only to the way the explanation was provided, regardless of whether the decision made sense to the user or not. The fourth principle is knowledge limit: for an output to be explainable, it has to limit itself to the knowledge that was supplied to it. In other words, a decision cannot be explainable if it was built on knowledge that was out of the scope of the decision.

Furthermore, it is important to mention that the ISO/IEC TR 24028:2020, Overview of Trustworthiness in Artificial Intelligence, published in 2020, introduces many concepts that can be of help to the organization in building a trustworthy AI. The notions of ex ante explanation and ex post explanation are good to have in mind when dealing with explainability. In the ex ante (the before explanation), the goal is to explain what the system is meant to do and how it intends to behave. The goal is to establish trust in the system before it is used. In the ex post (the after explanation), the goal is to justify the decision, the outcome, and the result so the user understands the why of the decision. The ISO/IEC TR 24028:2020 also introduces other concepts such as the causal explanation (cause and effect explanation), epistemic explanation (how do we know what we know), and the justification explanation (the explanation that justifies the results). This Standard is very helpful to read as it provides definitions and explanations on how to build a trustworthy AI system.

Finally, and to make these principles applicable, the High-Level Expert Group on Artificial Intelligence set up by the European Commission provides a list of seven requirements that help organizations and stakeholders build and implement a trustworthy AI. These requirements are: human agency and oversight (the need for AI systems, especially the ones that are classified as high risk, to be under the control of humans); robustness and safety (AI systems must be reliable in their performance and robust regardless of the circumstances under which they perform); privacy and data governance (the need for AI systems to respect privacy and address privacy concerns during the data processing phase); transparency (the need for the automated system to be open, sharing information about the model, the data, and the training methods); diversity, nondiscrimination, and fairness (AI systems must be inclusive during their design and must address bias so stakeholders can trust them); societal and environmental well-being (AI systems must have a positive social impact that allows for AI to be accepted and sustainable over time); and accountability (organization, management, senior leadership, and governing bodies must be accountable when the AI system misbehaves, and companies must report negative impacts).

Chapter 5.14: The Principle of Robustness – Ensuring Consistent Performance

In computer science, the term robustness refers to the ability of the computer program to perform in a consistent way over and over, regardless of the changes happening in the environment. It is a central concept of modern technology as it helps build trust in the system. When a machine behaves accurately and consistently in the lab and in the real world despite what might happen in its environment, that machine has met the requirements for the principle of robustness. When you hit the brake and the car stops regardless of the circumstances, your braking system is robust, and that helps you build the trust and the confidence that makes you drive and speed with fewer worries.

Robustness is a key concept in AI and intelligent systems in general because of the learning capabilities that intelligent systems may develop over time. So, to build an AI system that is safe, secure, and trustworthy, engineers must work on the robustness principle to minimize variability and reduce potential risks and harms that may be associated with unknowns of the real world. This is why a human-in-the-loop approach is a vital approach in building AI systems as it allows engineers to check and identify the variability in performance between the behavior of the model in the lab and its behavior in the real world over time.

If you want to build a trustworthy system, start by asking your team the following questions. Have we identified the group of stakeholders that the AI systems may affect? And have we categorized them? Have we established a system by which we classify AI systems by degree of risk and level of impact on users' health, safety, and well-being? Have we identified the vulnerability of AI systems, and how these vulnerabilities are dealt with? Is our AI ecosystem transparent to stakeholders, and can we open our algorithms, data models, features, and training processes to external parties? These questions help ensure that robustness is not just a technical aspiration but a practical reality that underpins every AI system the organization deploys.

Chapter 5.15: Conducting the Context Analysis – A Five-Step Process for ISO Certification

The standard requires that you analyze the context of the organization, but it does not prescribe a specific methodology. While most organizations do conduct this type of analysis frequently, for ISO/IEC 42001, it is recommended that you conduct a new analysis of the context of your organization. Given the fluidity of the regulatory environment in which AI is evolving and the speed at which the technology is moving, it is recommended that you start from scratch. Avoid simply updating or refreshing old and existing documentation of organizational context analysis. There are many methodologies available for organizational analysis. You can use any of the known methodologies, such as SWOT analysis (Strengths, Weaknesses, Opportunities, and Threats), the PESTEL methodology (Political, Economic, Social, Technology, Environment, Legal), Scenario Planning, Stakeholders Analysis, Porter's Five Forces, or simply the Industry Analysis methodology. As long as you are asking the right questions, documenting the insights, aligning those insights with your AI strategy, and showing auditors how your organization meets the standard's controls, your analysis will support the certification process. Remember, the goal is to learn from the analysis about the trends impacting the development of AI technology, and how those trends affect the building of your AIMS, so you can document and justify your choices to auditors. You can also use the organizational analysis process for benchmarking purposes to learn about best practices that help you design a long-term sustainable and viable AI management system in an environment that is characterized by policy uncertainty and legal ambiguity.

The first step is to mobilize the team and clarify the mission. Put together a team of members who are well aware of the organizational strategy and have a deep knowledge of the industry, its directions, and the global trends affecting technology. In putting together the team, it is critical that you diversify your team to include many functions, expertise, and backgrounds. Go beyond the obvious functions, such as IT and strategy, to include legal affairs, compliance, HR, customer service, marketing, sales, purchasing, quality, and teams that work on technology or interact with technology vendors. The second step is to set the roadmap. Clarify the purpose and mission of the team and make sure that the team members are aware and have read the requirements listed in Section 4, Context of the Organization, of the Standard. ISO/IEC 42001 lists four reasons for which an organization analyzes its context. These reasons are: to identify the issues the organization faces concerning building and developing an AIMS, so the organization can achieve its strategy; to spell out the role of the organization concerning AIMS; to spell out how your AIMS affects and interacts with your stakeholders, locally and globally; and to identify the national and international legal framework, policies, and overall values and norms that may affect your AIMS.

The third step is to conduct discovery sessions. Once you put the team together, and you understand the purpose of the context analysis, it is important that you organize workshops and discovery sessions to discuss the blueprint of the analysis. The goal is to have all team members in charge of the AIMS contribute to the conversation. This is not only an analysis of the context for an ISO audit, but rather a learning opportunity for managers and employees to get acquainted and updated on a technology that is moving fast. The fourth step is to start with the external environment. Ask questions to generate insights and make sure you record those insights in a document. This is the document that you will show to the auditors. What economic impact is AI expected to have on our industry, and how should we position ourselves to take advantage of it? How does AI strategy fit into our overall business strategy, and how do we create coherence and alignment between the two beyond the automation imperative? What are the external factors that will enable us to achieve our organizational goals? What are the regulatory environment and government policies that we need to address? How does AI development affect regulation about employment, discrimination, customer protection, health and safety, and other matters? How has AI impacted public opinion and what are the new emerging customer attitudes that may affect our market position or AI products? If we are using third-party AI products, have we audited those products to make sure they comply with laws and regulations and subscribe to our ethical AI framework, and the generally agreed-upon principles recognized by ISO standards and other international organizations such as OECD and UNESCO?

The fifth step is to conduct an internal analysis. Once you finish the analysis of the external environment, you turn to the analysis of the internal context. In the internal context, you can also take advantage of some of the known analytical frameworks, such as the Balanced Scorecard, the Value Chain Analysis, and the Resource View methodology. The goal is to ask questions that generate more insights about how AI impacts the overall strategy and how the organization intends to create a coherent AI management system that is aligned with the overall strategy and in coherence with other management systems. Here are some of the questions that you can use to capture insights from the team. To what extent will AI affect our digital transformation strategy, and what are the risks of this technology—risks that relate to the technology itself, but also to the speed at which we are adopting this technology? How has the introduction of AI tools and AI products, including third-party products, affected our mission, vision, and values? How has AI affected our talent management strategy and how to future-proof our talent strategy? What are the employees' upskilling initiatives that we need to launch to adjust our human capital to this technology? To what extent do we need to rethink and reengineer our processes to accommodate the standard requirements?

Chapter 5.16: Chapter Summary & Key Takeaways

Context analysis is the foundational act of sense-making that determines whether an AI management system will be relevant, effective, and sustainable.
ISO/IEC 42001 Clause 4 requires organizations to analyze and evaluate internal and external factors that may affect AIMS performance.
Context analysis supports four key pillars: understanding the bigger picture, identifying key stakeholders, learning about links and interconnectedness, and improving agility.
Competitive landscape analysis using frameworks like Porter's Five Forces helps organizations understand how AI is reshaping their industry and where threats and opportunities lie.
Stakeholder analysis is a requirement of the standard and involves identifying providers, developers, users, producers, partners, AI subjects, and authorities, and understanding their needs and concerns.
The legal context is complex and evolving, with key frameworks including the GDPR, the EU AI Act, and various US federal, state, and local regulations.
The GDPR establishes seven principles for data processing that directly affect AI management systems, including lawfulness, fairness, transparency, purpose limitation, data minimization, accuracy, storage limitation, integrity, and confidentiality.
The EU AI Act is the first comprehensive AI legislation, adopting a risk-based approach with four categories: unacceptable, high, limited, and low risk.
The US AI regulatory landscape is a patchwork of federal agency guidance, state laws, and local ordinances, with notable examples including NYC Local Law 144 and the proposed Algorithmic Accountability Act.
The Executive Order on Safe, Secure, and Trustworthy AI (2023) mandates safety testing, privacy protection, and algorithmic discrimination prevention, while recognizing challenges of velocity, scope, authority, and method in AI regulation.
Ethical AI frameworks from ISO, OECD, UNESCO, and leading companies converge on five core principles: do no harm, fairness and non-discrimination, human oversight, explainability, and robustness.
The do no harm principle distinguishes between individual, collective, and social harm, and requires organizations to define, assess, and mitigate harm throughout the AI project life cycle.
Fairness and non-discrimination require attention to algorithmic bias, diverse teams, responsible AI principles in design, and governance and leadership engagement.
Human oversight and respect for human autonomy ensure that AI remains an enabler that empowers humans rather than a controller that manipulates them, with human-in-the-loop safeguards for high-risk decisions.
Explainability requires providing meaningful, accurate, and bounded explanations of AI decisions, distinguishing between ex ante and ex post explanations, and adhering to principles of explanation, meaningfulness, accuracy, and knowledge limit.
Robustness ensures consistent performance across environments and requires human-in-the-loop approaches to monitor variability between lab and real-world behavior.
Conducting context analysis follows a five-step process: mobilize the team, set the roadmap, conduct discovery sessions, analyze the external environment, and analyze the internal context.
Context analysis should be conducted from scratch, documented rigorously, and used not just for certification but for genuine learning and strategic alignment.

Chapter Notes

Mark this chapter as completed

Chapter 6 of 11

Chapter 6: The Tone at the Top – How Leadership Determines the Fate of Your AI Management System

Chapter 6.1: Introduction – Why Leadership Is the Sine Qua Non of AI Governance

The engagement of leadership in building the Artificial Intelligence Management System (AIMS) is not merely important; it is the single most critical determinant of whether that system will succeed or fail. An explicit commitment from leadership that defines the North Star, sets the tone, and creates the incentives that engage people is of paramount importance. In an environment characterized by fast-moving technology, regulatory uncertainty, shifting public sentiment, and competitive disruption, the role of leadership in guiding the teams in charge of deploying and implementing AI management systems is absolutely vital. Without strong, visible, and sustained leadership engagement, even the most technically sound AIMS will struggle to gain traction, secure resources, and deliver on its promise.

According to the ISO Standard 42001:2023, there are three main actions that leaders need to take to build a viable AI management system. First, they must set the vision and the priorities and demonstrate genuine engagement in building the AI management system. Second, they must develop an AI policy that defines the AI intent, the rules and procedures governing the AI management system, and other related processes. This policy should be an auditable document that can withstand the scrutiny of certification bodies. Third, they must define the roles, responsibilities, and authorities of the different actors engaged in the making of the AI management system. These three actions are not optional extras; they are the foundational requirements without which the entire governance structure lacks legitimacy and direction. In the following sections, we will explore each of these requirements in depth, examining how they should be put into practice and what best practices leading organizations have adopted.

Chapter 6.2: Setting the Vision – Defining Why AI Matters to Your Organization

AI is a transformative technology. Its impact on the economy and wider society is just beginning to be felt, but its trajectory is unmistakable. The emergence of generative AI and Large Language Models has ushered in a new era of disruptions that will be felt by all sectors of the economy. As Bill Gates has observed, the age of AI has begun, and it is creating a new reality that we have not seen before. More importantly, the environment surrounding AI evolution is constantly shifting. The technology itself is moving fast, social interest in the technology is growing, and calls to regulate this new technology are intensifying. AI has suddenly become a topic of national conversation, and demands to limit its impact have grown louder with each passing month.

In this shifting environment, a vision of what AI means for the organization is vital. The vision guides employees and sets goals for the organization so the entire organization is aligned with the same purpose. A vision inspires people and rallies them behind the AIMS while sharing with society and the general public the dream that the organization wants to achieve. Especially true during uncertainty, a vision provides a line of sight on where the organization wants to be. But how do you set an AI vision? And how do you create an engaged leadership? What are the steps that you need to take to build the leadership capability required for such transformative technology? The ISO Standard 42001:2023 lists a set of actions that leadership should engage in, and below we will discuss the most critical ones that have a direct impact on the viability of the AI management system.

To help your company develop an AI vision, and to test whether your leadership truly understands that vision, ask the following five questions. Have we clearly articulated our AI vision, and have we defined our aspiration for what AI will do for us? What do we want to achieve in the AI strategy and how should we challenge the status quo? To what extent has the AI vision been communicated to all stakeholders internally and externally? To what extent and in what way can employees align their behavior with the AI vision? In what way does the AI vision help structure organizational behavior? These questions are not merely diagnostic; they are generative. Answering them forces leadership to clarify its thinking, articulate its intentions, and commit to a direction. They also expose gaps between rhetoric and reality, ensuring that the vision is not just a glossy statement on a website but a living guide for decision-making at every level of the organization.

Chapter 6.3: Setting the Vision, Defining Priorities, and Establishing Strategic Direction

The role of leadership in defining the priorities is important as it provides the organization with clarity on where the organization wants to go. This AI vision should be in coherence with the organization's vision, mission, and values, and should send a clear message to all management levels about the importance of the AI management system and how this system helps the organization achieve its business goals. A vision that is disconnected from the broader organizational purpose will feel artificial and will fail to inspire genuine commitment. A vision that is integrated, by contrast, becomes a powerful force for alignment and motivation.

Defining the vision and how this vision breaks down into strategic goals and strategic KPIs helps you design a viable AI strategy. Organizations may approach AI strategy in different ways, depending on the economic sector and the industry in which they operate. But generally speaking, companies integrate AI to automate processes and decision-making and use it as a platform to link key functions. A critical element in the setting of an AI vision and strategy is to embrace a holistic view of AI strategy by integrating AI into the digital transformation strategy. Decoupling the AI strategy from the overall digital transformation strategy the company is going through weakens the AI strategy and creates more dissonance in the AI management system. A truly successful AI strategy is going to be linked and aligned with the operating model, the data infrastructure, the business model, and the overall organizational plan.

Furthermore, leadership needs to engage to make sure that AI strategy and the building of the overall AI management system should not be approached as a purely IT issue. The nature of a general-purpose technology makes AI an enabler that cuts across many functions, teams, and capabilities. Teams need to be aware of this imperative, and it is the leadership role that sets this tone. When AI is siloed within the IT department, it fails to realize its transformative potential. When it is embraced as an enterprise-wide capability, it becomes a source of competitive advantage that touches every function and every business unit. Leadership must therefore communicate clearly and repeatedly that AI is everyone's business, not just the business of technologists.

To illustrate what a compelling AI vision looks like, consider how leading technology companies have crafted their AI vision and mission statements. Microsoft's vision is "To empower every person and every organization on the planet to achieve more through artificial intelligence." IBM's vision is "To create a future where every decision, every business process, and every experience will be augmented with artificial intelligence." Google's vision is "To organize the world's information and make it universally accessible and useful with the help of artificial intelligence." Amazon's vision is "To use artificial intelligence to make every customer interaction more natural and intuitive." Facebook's vision is "To build technologies that help people connect with friends and family, find communities, and grow businesses using artificial intelligence." Tesla's vision is "To create a world where cars can drive themselves safely and efficiently with the help of artificial intelligence." NVIDIA's vision is "To advance artificial intelligence and create intelligent machines that can perceive, reason, and learn." Salesforce's vision is "To deliver the world's most intelligent customer relationship management using artificial intelligence." Intel's vision is "To develop artificial intelligence technologies that can transform industries, solve global challenges, and improve people's lives." And Siemens' vision is "To use artificial intelligence to transform industries and help solve some of the world's biggest challenges." Notice that each of these visions is clear, concise, to the point, and carries a powerful message that engages employees and makes customers believe or curious about their products. The best visions are memorable, aspirational, and grounded in a clear understanding of what the organization does and who it serves.

Chapter 6.4: Leading with Responsible AI Principles – The Business Case for Ethics

The transformational nature of AI requires a transformational leadership style. Leaders should engage the entire organization through a wider transformation strategy that helps employees understand the need to change, the scope of change, and how the results of the change will impact them. In the change plan, leadership should communicate key factors required for the successful implementation of an AI management system. The emphasis should be on how responsible AI principles should help the organization achieve its business goal, act responsibly, and achieve its compliance and certification requirements.

Responsible AI is a framework that helps you design and implement AIMS by the organization's ethical framework and the emerging AI ethical principles we discussed in the previous chapter. In simple words, a responsible AI approach means that your AI strategy and the overall AI management system should be designed with good intentions, and in compliance with laws and regulations while subscribing to ethical AI principles such as do no harm, fairness and discrimination, human oversight and respect for human autonomy, and explainability. Adopting RAI principles has many benefits to the organization as there is a real business case to be made for the organization that adopts RAI principles. Many reasons explain that. Organizations that create a better trustworthy relationship with the customer and stakeholders benefit their shareholders and employees. Organizations that use RAI throughout the product development cycle gain better product quality which in turn improves their customer retention and engagement. Organizations that develop a data management process tend to explore better-quality data that is reliable and that itself leads to better decisions. Organizations that use RAI principles have better chances of attracting talent. The value system and the ethics of companies are becoming a deciding factor for many hard-to-find talents. As regulation is looming and the call for an aggressive regulatory framework intensifies, organizations that adopt the RAI principles will have a first-mover advantage. These organizations will be ahead of the curve in terms of compliance and risk management.

The Responsible AI Institute (RAI Institute), a nonprofit organization and a member of the World Economic Forum Global AI Action Alliance, is among the first industry coalitions that have developed a comprehensive certification program to address responsible AI. The RAI Institute framework is made of six building blocks: systems and operations, explainability, interpretability, accountability, consumer protection, bias and fairness, and robustness. This framework not only builds on the many laws and regulations published but also takes into consideration best-emerging practices suggested by organizations such as the ISO, IEEE, UNESCO, OECD, and standards bodies such as ANSI, BSI, and SCC, as well as many organizations in the private sector such as IBM, Deloitte, EY, Microsoft, BCG, PwC, and McKinsey.

In July 2023, the White House announced that it had secured a voluntary commitment from leading AI companies to develop a framework that would help them manage AI risks and accelerate the path toward Responsible Artificial Intelligence. These companies—Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI—voluntarily committed to the White House proposal, which is built around three pillars: safety, security, and trust. The commitment includes several specific measures. Sandboxing: The companies commit to testing their systems internally before releasing them to the public, with testing carried out by third-party independent experts. Best practices: The companies commit to sharing information and knowledge across the industry and with different stakeholders, such as academia and the government, about AI best risk management practices. Cybersecurity: Companies commit to invest in cybersecurity to protect models and proprietary data that serve to build these models. Incident reporting: The companies commit to working with third-party experts to discover and report vulnerabilities in AI systems after they are released to the public. Watermarking system: The companies commit to developing mechanisms that help users know when they are interacting with AI-generated content. And prioritizing societal risk: The companies commit to prioritize research about the most critical harms AI systems pose to society, including bias, discrimination, and protection of privacy. These commitments, while voluntary, represent an important step in establishing norms of responsible behavior for the AI industry.

Chapter 6.5: Setting the Tone and Using Proactive Communication

When embarking on a new technology, communication is critical. This is especially true for a nascent technology that is also controversial. The setting of the tone is the essence of leadership commitment. The leadership role in framing the issue of why the organization needs to invest in an AI management system, how it should build a long-term vision for the AIMS, how it intends to engage with different stakeholders, how it navigates the known and unknown risks surrounding the development of AI technology, and how it applies RAI principles creates a change momentum that rallies employees around AI.

Leaders should explain the risks associated with nascent technology investments, emphasize responsible AI principles, and choose the right language and vocabulary to set the tone. They need to make compliance with ethical AI a priority and demonstrate how they intend to operationalize it. Always explain the context of the technology so people can see where you stand on difficult decisions. Tell the story of the moral and ethical dilemma leaders go through when deciding about risky technology investment. Finally, be humble and acknowledge the uncertainties of the technology so you can build trust and get the engagement you need from your employees.

Proactive communication means anticipating questions, concerns, and resistance before they crystallize into opposition. It means engaging with employees, customers, regulators, and other stakeholders early and often, rather than waiting for a crisis to force a response. It means being transparent about what you know and what you don't know, about what you can promise and what you cannot. In an environment of rapid change and deep uncertainty, this kind of honest, humble, and consistent communication is what builds the trust that is essential for successful AI governance.

Chapter 6.6: The AI Policy – Its Purpose, Characteristics, and Components

In addition to the AI vision that should provide the organization with clarity on where the organization stands concerning AI, Standard 42001:2023 also requires that you craft an AI policy. This is an official company document that captures the commitment and engagement of leadership to the AI management system. If the AI vision helps organizations and employees understand the purpose of AI, the AI policy serves as a reference document that defines procedures and guides behavior so employees stay compliant. According to the ISO/IEC 42001, the goal of an AI policy is to define the AI management system, identify the roles and responsibilities of the different people engaged with the AIMS, and show how the AI management system integrates into and interacts with other management systems.

Depending on the sector and the industry in which you operate, and the legal jurisdictions in which you are located, your AI policy should include several key elements. The first element is a statement on the scope of the policy, its purpose, and what the policy intends to achieve. This is generally accomplished by a small paragraph that is based on the strategy of the organization and other policies, like the data policy, the privacy policy, the HR policy, finance and accounting policy, purchasing policies, and other policy documents that may exist in the organization. The second element is guidelines on the use of AI in the organization. This is usually achieved by defining what is an acceptable use of AI applications, and what is not. Here you define the boundaries of the AI applications. Refer to this paragraph for laws and regulations, standard requirements, and compliance. Explicitly define what is prohibited and what is not, and address the gray areas. The third element is showing how the AI management system integrates with other management systems. This is a crucial element of the AI policy. This paragraph helps employees identify connections, interactions, and the links the AIMS has with other management systems. For most organizations, this would be a challenge. Adopting a holistic perspective, for a nascent technology that is cutting across all business functions, is a big hurdle. The risk of not identifying these connections will create misalignment and dissonance in the overall management system.

The fourth element is defining the roles and responsibilities. In this section, you need to identify the roles and responsibilities of different actors engaged with the AIMS. Keep in mind that while ISO/IEC 42001 does not define these roles, there is another ISO standard, ISO/IEC 22989:2022 Artificial Intelligence concepts and terminology, that provides definitions of different roles. Depending on the context of your organization, I suggest that you define roles and responsibilities by type of role: technical roles and business roles. The fifth element is a statement on how you manage data and privacy. Typically, organizations should already have an existing data and privacy policy that you can link to the AI policy. But if the organization does not have these policies, create new ones. In these documents you typically define the process by which you manage, store, and secure data, and how you handle, share, and protect personal information. The sixth element is explaining how you intend to make your AI system comply with regulations. This is a paragraph that defines compliance and how the organization conforms to laws, regulations, and standards. It may be good to list the set of ISO and industry standards that you comply with and outline ways of addressing noncompliance issues. The seventh element is a statement on how AI systems link with your talent management approaches. A good policy should include a paragraph on how the organization intends to train and develop employees in charge of AI systems. This should also link to the HR and training management system.

An AI policy should also include a paragraph on how the organization addresses the improvement of the AI management system. This is a control of ISO/IEC 42001. Finally, the policy should align with other company policies and should be reviewed to ensure it takes into consideration the change in the technology and the regulatory environment. The review and the alignment with other policies are a requirement and part of the controls listed in Annex A.

Chapter 6.7: Technical Roles in the AI Ecosystem – From Data Scientists to AI Designers

To implement an AI management system effectively, organizations must define and staff a range of technical roles, each with distinct responsibilities and required competencies. The first technical role is the data scientist. This is an engineer who processes data. This is a critical role to define because data scientists have a deep impact on data collecting, data sampling, and model training. Typically, they are not domain experts, so they should be trained in domain-specific knowledge. A data scientist who understands the business context in which their models will be deployed is far more likely to build models that are useful, fair, and aligned with organizational goals. Without this domain knowledge, data scientists may optimize for the wrong metrics or fail to account for important contextual factors.

The second technical role is the machine learning engineer. These are engineers in charge of AI model implementation, including the production and testing of the model's validity. Machine learning engineers bridge the gap between data science and software engineering. They take the models developed by data scientists and turn them into production-ready systems that can be deployed at scale. They are responsible for ensuring that models are robust, efficient, and reliable, and that they integrate properly with existing systems and infrastructure. The third technical role is the domain expert. These are experts who have specific knowledge about a particular field. These experts need to work hand in hand with data scientists and ML engineers to guide them into the particularity of each field such as marketing, customer service, HR, and procurement. Domain experts ensure that AI systems are designed with a deep understanding of the real-world problems they are meant to solve and the constraints within which they must operate.

The fourth technical role is the AI designer. These are employees who serve as interfaces between data scientists, ML engineers, and users. These are the translators of AI systems into user-friendly applications that impact the final evaluation of the product. AI designers are responsible for the user experience of AI systems, ensuring that they are intuitive, accessible, and aligned with user needs and expectations. They play a critical role in ensuring that AI systems are not just technically sound but also usable and valuable in practice. Together, these technical roles form the backbone of any AI development effort, and defining them clearly within the AI policy ensures that everyone understands their responsibilities and how they contribute to the overall AIMS.

Chapter 6.8: Business Roles in the AI Ecosystem – Leadership, Compliance, and Ethics

In addition to technical roles, an AI management system requires clearly defined business roles. The first of these is executive leadership. Senior leadership is in charge of the overall strategy and the direction of the organization. They set the vision, allocate resources, and hold ultimate accountability for the performance of the AIMS. Without engaged executive leadership, the AIMS will lack the authority and resources it needs to be effective. The second business role is the project manager. These are managers in charge of leading and implementing different AI projects. Project managers coordinate across teams, manage timelines and budgets, and ensure that AI projects deliver on their intended outcomes. They are the operational backbone of AI implementation.

The third business role is the compliance manager. These are experts with legal and standardization backgrounds that guide the organization on compliance issues. Compliance managers ensure that AI systems comply with applicable laws, regulations, and standards, and that the organization is prepared for audits and certification. They play a critical role in identifying and mitigating legal and regulatory risks. The fourth business role is the AI strategist. These are people with technical and strategy backgrounds. They have a good understanding of AI trends and how these trends affect the industry and the company. AI strategists help the organization navigate the rapidly evolving AI landscape, identifying opportunities and threats and ensuring that AI strategy is aligned with business strategy.

The fifth business role is the ethics committee. This is a new and important role that is directly linked to the development of AI. This committee oversees defining policies, developing frameworks, reviewing risky applications, and providing recommendations on how to manage and mitigate risks. The ethics committee serves as a check on the power of technical teams, ensuring that ethical considerations are given appropriate weight in decisions about AI design, deployment, and use. Ideally, the ethics committee should be diverse in its composition, with members from different functions, backgrounds, and perspectives, and should have the authority to veto applications that pose unacceptable risks. Together, these business roles ensure that AI is governed not just as a technical endeavor but as a strategic, ethical, and organizational one.

Chapter 6.9: How to Create an AI Policy – A Step-by-Step Process

Following are the steps that you need to take to create an AI policy. The first step is to form the team. As we argued earlier, teams in AI should be as diversified as possible. Put together a cross-functional and multidisciplinary team that has a range of expertise, such as policymaking, strategy, data science, marketing, compliance, legal, and HR. Depending on the sector in which you operate, your policymaking team should reflect those who will directly or indirectly interact with the AI management system. The second step is to engage with stakeholders. To ensure that the policy is as comprehensive as possible, you need to engage with your stakeholders such as employees, customers, partners, and vendors. The goal is to hear their perspective on how AI will impact them.

The third step is to conduct discovery sessions and workshop meetings with different stakeholders. To ensure that you are capturing the views and the perceptions of different segments of the stakeholders, conduct discovery sessions. Ensure that the discussion is open and covers a wide range of topics. The fourth step is to review the laws, regulations, and ethical framework. Your legal team should draft a manual of all the laws and regulations that apply to AI. This will guide your discussion and should help members of the team understand the risks and liabilities of AIMS. Take a look at the following ISO standards as they address, directly or indirectly, the issue of governance in AI: ISO 37000:2021 Governance of organizations – Guidance; ISO/IEC 38500:2015 Information technology – Governance of IT for the organization; ISO/IEC 38505-1:2017 Information technology – Governance of IT – Governance of data – Part 1: Application of ISO/IEC 38500 to the governance of data; ISO 31000:2018 Risk management – Guidelines; and ISO/IEC 23894:2023 Information technology – Artificial intelligence – Guidance on risk management.

Once the policy is drafted, it should be reviewed and approved by senior leadership and communicated widely throughout the organization. The policy should be a living document that is updated as laws, regulations, technologies, and organizational priorities evolve. It should be easily accessible to all employees and should be accompanied by training and awareness programs that help employees understand its implications for their daily work. A policy that sits on a shelf and is never referenced is worse than no policy at all, because it creates a false sense of compliance while leaving the organization exposed to risks.

Chapter 6.10: The AI Strategy – Developing Use Cases, Assessing Competition, and Reorganizing Internally

An essential step in the creation of a successful AIMS is the definition of a clear strategy on how the organization intends to use AI. While AI strategies may look different from one organization to another, depending on the industry and the economic sector where the organization operates, the process of formulating a strategy is relatively similar. The first step in developing an AI strategy is to develop AI use cases. Enhancing customer satisfaction is one of the most compelling use cases. Getting closer to your customers and knowing about their needs and wants is critical to building a long-term relationship with your customer base. The predictive power of intelligent systems, such as conversational AI, presents a strong use case. The power of AI and its ability to identify patterns, find connections between customer clicks and behaviors, and make recommendations is impressive. Response time to customer inquiries is also improved when you introduce AI. A survey by Salesforce Research found that 51 percent of companies choosing AI do so to personalize the customer experience. Seventy-eight percent of customers, in a survey conducted by Oracle, are more likely to do business with a company that personalizes their experience.

Other compelling use cases include agile and data-driven decision-making. AI applications allow decisions to be driven by what data suggests. AI provides better and faster decisions. It helps you reduce the time to make complex decisions, which adds more agility and nimbleness to your decision-making process. In a data-driven environment, response time to market changes is faster and more accurate. Creating efficiencies is another use case. The implementation of AI and the automation of different processes provide a real cost-saving opportunity. Automation relieves employees from mundane tasks, reduces errors, and allows talent to focus on more business-critical issues. Improving productivity is yet another use case. Faster processing with analytics and improvement in the accuracy of the data helps the organization improve its productivity by speeding up processes and saving wasted time. A study by Accenture found that AI can increase productivity by 40 percent and reduce staffing costs by 70 percent. In the marketing function, a study by IDC found that companies using AI increase the efficiency of their marketing processes by 30 percent.

The second step in the design of an AI strategy is to analyze your competitive landscape. While competitive analysis is a complex task, as it requires information that may not be available to you, it is a critical step in building your AI strategy. The good news about building a strategy for nascent technology is that most of the competition is at the same level as yourself, outside of the course of digital natives. So, conducting this analysis may be easier as you and your competitor are at about the same level. A framework suggested by the World Economic Forum provides a good starting point for how you should analyze the competitive landscape. The framework, made of four quadrants, asks two essential questions. Will the strategic impact of AI improve operational efficiencies or create a new business model? Here, the issue is to distinguish between a technology that merely improves efficiencies from a technology that radically transforms the business model. The second question of the WEF model asks about the competition. Who is deploying AI? Is it a new player from an adjacent industry or a competitor? An extreme scenario, in the WEF model, is a scenario in which you are facing a new entrant competitor introducing a completely new business model that represents more risks and requires swift actions.

The third step in designing an AI strategy is to reorganize internally. Designing an AI strategy requires the building of many capabilities that may or may not exist in your organization. A common mistake in formulating an AI strategy is to be tempted to fast-track the AI strategy and thus build it on top of existing capabilities that may not be suited for a powerful and transformational technology such as AI. Three critical capabilities need to be built or radically improved to create viable AI technology. The first is to build and update the current technology infrastructure to empower the AI management system. This may include new software, new cloud technology, and other tools and platforms that accommodate AIMS. You cannot build powerful technology such as AI on an existing old infrastructure. Leading organizations tend to invest in new infrastructure with end-to-end platforms that link different business functions and applications. The second is to design a data strategy. A good AI strategy requires a good data strategy. There is no AI strategy without a data strategy that defines the process by which you manage, store, and process data. The backbone of an AI strategy is a data strategy that is comprehensive, exploitable, and provides a high level of confidence in its accuracy. The third is to develop a talent strategy. According to McKinsey's 2022 AI Global Survey, leading companies in AI strategy are the ones that have invested in building a comprehensive talent strategy. Without a vision of how to upskill, reskill, and improve the core of AI talent competencies, such as data scientists and data engineers, AI strategy will stumble. More importantly, a change in the way teams communicate and work is also a critical component. Because AI cuts across many functions and teams, the silo culture needs to be replaced by a culture of communication and collaboration that breaks down barriers between people working on similar and different issues.

To assess your AI strategy, ask the following five questions. Have we defined our AI strategy and how it fits into our overall corporate strategy? How often and to what extent does our AI strategy get revisited and updated to reflect the external environment, market evolution, and new regulations? Have we spelled out structures and governance processes to support our AI strategy's activities? Have we defined who owns different AI activities and programs and how these programs and activities support the overall AI corporate strategy? Have we established metrics to measure our AI activities and programs? These questions help ensure that AI strategy is not just a document but a living framework that guides decision-making and resource allocation across the organization.

Chapter 6.11: AI Oversight – The Critical Role of the Board of Directors

As a governing body, the Board of Directors has a direct responsibility for how AI systems are used and the risks they produce for the organization. In addition to the ISO/IEC 42001, which addresses management and leadership issues, there is another ISO Standard, ISO/IEC 38507:2022 Governance Implications of the Use of Artificial Intelligence by Organizations, that directly addresses the issues of governance and oversight by Board of Directors. Clause 6.2 (Governance oversight of AI) of this Standard puts the liability of misuse of AI systems on the shoulders of different governing bodies, including the Board of Directors.

A look at how the Board of Directors deals with AI reveals that most Boards have not yet set the process by which they define their engagement with the oversight of the AIMS. A survey conducted in 2022 by the Institute of Directors (IOD), a UK-based organization for directors, shows that 80 percent of members surveyed did not have a system in place to audit AI. More importantly, the survey reveals that the reason for this lack of oversight is that board members did not know the type of questions to ask to exercise their oversight of AIMS. This is a revealing case. It shows that most Boards of Directors are not ready yet for AIMS oversight. It also speaks volumes about the nature of AI systems. An emerging and powerful technology that is moving faster than our governance system can adjust. Governing bodies, especially the ones that are not involved in the day-to-day operations, tend to be less engaged in general, and in the case of AI, they seem to be completely removed from the process. Lack of technology knowledge, the shifting of the regulatory framework, and the absence of consensus around best practices and frameworks are all reasons that explain this situation.

The Institute of Directors provides twelve principles that should guide the involvement of Board of Director members in AI systems oversight. According to IOD, Board members should do the following. Engage in and with the regulatory framework, as well as monitor how laws and regulations evolve in your country and other parts of the world. Board members should also establish an ethical framework for the use of AI that is auditable and measurable. This framework should ideally be integrated into the ISO auditable standards, such as ISO/IEC 42001 and 9001:2015. Conduct an impact assessment on how AI systems will affect the stakeholders, including the impact of AI applications on employees and their jobs. Align AI strategy with the business goals of the organization. Work to harmonize AI objectives with the vision, mission, and value of the organization. Establish an Ethics Committee that is diverse and with veto power to decide about AI applications and their use. Document and detect bias in data sources. Remove bias when bias is identified. Build employee AI capacity and empower employees to learn how to use AI and how to report harmful outcomes of AI systems. Use a privacy-by-design approach to protect the privacy and the data of AI systems. Train data scientists and engineers producing ML models on ethical AI principles. Provide a secure environment to enhance the safety and security of AIMSs to identify vulnerabilities and prevent cybersecurity attacks. Before full implementation, the Ethics Committee should provide recommendations on how the system impacts stakeholders. The ethics committee should approve and must have the final say on AI implementation. The Board of Directors is directly liable for AI applications. Finally, review and monitor the AI systems regulations to ensure that the AI systems are working as intended.

This set of principles provides a strong framework for the Board of Directors to engage with AI systems. They also comply with the spirit of ISO/IEC 42001 and subscribe to the responsible AI principles we discussed earlier. It is a good first step for a critical governing body, such as the Board of Directors, which tends to face criticism when things go wrong. In the age of AI, board-level oversight is not optional; it is an essential component of responsible governance. Boards that fail to engage with AI risk exposing their organizations to legal, reputational, and financial harms that could have been prevented with proper oversight.

Chapter 6.12: How to Gauge Leadership Engagement – Five Critical Questions

How do you know that leadership is truly engaged in supporting and promoting the AI management system? If you want to check whether your top management is truly engaged in supporting and developing the AIMS, ask the following five questions. First, what actions or steps has our top leadership been involved with to define and develop a common language and framework around AI policy and AI strategy? Second, have we defined the level of leadership that owns AI strategy? Third, to what extent is our top leadership engaged in leading our AI ethics? Fourth, what actions or steps has our top leadership taken to drive engagement around AI policy? Fifth, how and to what extent is our top leadership engaged in creating and fostering a risk management-based culture?

These questions are diagnostic tools that can reveal whether leadership engagement is genuine or merely rhetorical. If the answers reveal gaps—for example, if no one can articulate who owns AI strategy, or if leadership has not engaged with AI ethics—then it is a sign that the organization needs to invest more in building leadership capability and commitment. Leadership engagement is not a one-time event; it is an ongoing practice that must be sustained over time. It requires regular communication, visible participation in AI governance activities, and a willingness to make difficult decisions when ethical or risk considerations conflict with short-term business pressures. Organizations that get this right will be well-positioned to build AIMS that are effective, ethical, and sustainable. Those that do not will find themselves constantly reacting to crises rather than proactively shaping their AI future.

Chapter 6.13: Chapter Summary & Key Takeaways

Leadership engagement is the single most critical determinant of whether an AI management system will succeed or fail.
ISO/IEC 42001 requires leaders to set the vision, develop an AI policy, and define roles, responsibilities, and authorities.
A clear AI vision should be aligned with the organization's overall vision, mission, and values, and should be communicated widely.
AI strategy should be integrated with digital transformation strategy and should not be treated as a purely IT issue.
Responsible AI principles—do no harm, fairness, human oversight, explainability, and robustness—have a strong business case, including improved trust, quality, talent attraction, and regulatory readiness.
Proactive communication is essential for setting the tone, building trust, and engaging stakeholders during AI transformation.
An AI policy should include scope and purpose, guidelines for use, integration with other management systems, roles and responsibilities, data and privacy, compliance, talent management, monitoring and improvement, and review and alignment.
Technical roles in AI include data scientists, machine learning engineers, domain experts, and AI designers.
Business roles in AI include executive leadership, project managers, compliance managers, AI strategists, and ethics committees.
Creating an AI policy involves forming a team, engaging stakeholders, conducting discovery sessions, reviewing laws and regulations, and drafting, reviewing, and communicating the policy.
Developing an AI strategy involves identifying use cases, analyzing the competitive landscape, and reorganizing internally to build technology infrastructure, data strategy, and talent strategy.
The Board of Directors has direct responsibility for AI oversight, yet many boards are unprepared; the IOD provides twelve principles to guide board engagement.
Leadership engagement can be gauged by asking five critical questions about AI policy, strategy ownership, ethics, engagement, and risk culture.
Leadership engagement is an ongoing practice, not a one-time event, and requires sustained communication, participation, and difficult decision-making.

Chapter Notes

Mark this chapter as completed

Chapter 7 of 11

Chapter 7: The Risk Blueprint – Planning Your AI Management System for Resilience, Responsibility, and Results

Chapter 7.1: Introduction – Why Planning Is the Most Critical Section of the Standard

The planning section of the ISO/IEC 42001 Standard is, without exaggeration, the most critical section of the entire standard. Along with Annex A, the planning section forms the core of the standard because it contains the critical requirement clauses that relate to risk and that must be met for an AIMS to be compliant with ISO requirements. This section deals with the risk management framework and addresses three key actions: the risk assessment process and how to conduct it, the risk treatment options and how to address risk when it is identified, and the process of conducting an impact assessment of the AIMS. Planning is where the abstract commitments of leadership and the contextual insights of Clause 4 are translated into concrete, actionable strategies for managing the risks and opportunities that AI presents.

To understand Clause 6, we need to go over some of the key terminologies it adopts, namely the concepts of AI risk assessment, AI risk treatment, and AI impact assessment. A key terminology that Section 6 uses is the terminology of risk. While the term "risk" in language tends to convey a negative connotation, the way it is used in ISO/IEC 42001 and other standards that address risk management is different. Risk in ISO documents is used in a broader way to refer to any deviation from the initial objective, regardless of the consequences of this deviation, positive or negative. If the deviation is negative, then it needs treatment according to the risk management process outlined in the standard. If the deviation is positive, then this would be considered as an opportunity that the organization might want to take advantage of. This dual framing of risk as both threat and opportunity is essential for a mature approach to AI governance.

The concept of risk assessment (Clause 6.1.2) refers to the process by which an organization evaluates the likelihood and the impact of something going wrong that may affect the performance of AIMS. The concept of risk treatment (Clause 6.1.3) simply refers to the actions the organization takes to address the risk. Treatment actions to address the risks can be chosen from the ISO Standard 42001:2023 or simply chosen by the organization itself. The concept of impact assessment (Clause 6.1.4) is the process by which organizations assess the possible impact of the AIMS on users. From a practical perspective, and without getting into a discussion of semantics, the term risk assessment is a broader concept, while impact assessment focuses on the consequences of harm that may occur because of a particular risk, like assessing the harm of a data breach on privacy. In impact assessment, we address real effects and a real impact that may be harmful, while in risk assessment, we address probabilities of a scenario that may or may not materialize. ISO/IEC 42001 requires both actions: a risk analysis, with its risk treatment options, and an impact assessment that identifies harm to a particular type of user (individual, groups of individuals, or society).

Chapter 7.2: The Concept of Risk – Beyond the Negative Connotation

One of the most important conceptual shifts that organizations must make when adopting ISO/IEC 42001 is to embrace a broader, more nuanced understanding of risk. In everyday language, risk is almost always associated with negative outcomes: the risk of failure, the risk of loss, the risk of harm. But in the ISO framework, risk is defined as the effect of uncertainty on objectives. This definition is deliberately neutral. It acknowledges that uncertainty can lead to outcomes that are worse than expected, but it also acknowledges that uncertainty can lead to outcomes that are better than expected. The former are threats; the latter are opportunities. Both are risks in the ISO sense, and both must be managed.

This distinction is particularly important in the context of AI, where the same technology that creates risks can also create enormous opportunities. A generative AI system that hallucinates facts is a risk. But the same underlying model that produces hallucinations can also produce valuable creative content, generate insights from data, and automate complex tasks. The organization that only sees the risks and not the opportunities will be paralyzed by fear. The organization that only sees the opportunities and not the risks will be blindsided by harms. The organization that sees both will be able to make informed decisions about how to deploy AI in ways that maximize benefits while minimizing harms.

For the AIMS, the practical implication is that risk management should not be a purely defensive exercise. It should be a strategic capability that helps the organization identify, evaluate, and respond to uncertainty in a way that supports its objectives. This requires a culture in which risk is discussed openly, in which bad news travels fast, and in which learning from near misses and failures is valued as much as celebrating successes. It also requires processes and tools that make risk assessment and treatment systematic, repeatable, and auditable. The planning section of ISO/IEC 42001 provides the framework for building this capability.

Chapter 7.3: The Stakeholder Analysis – A Prerequisite for Effective Risk Management

In planning the implementation of your AI management system, you need to conduct a stakeholder analysis. This is a requirement of Section 4.2 that we have explained in a previous chapter, where the standard uses the term "interested parties." The goal is to engage with the people and the parties that interact with your AIMS early on, so you can assess, identify, and mitigate risks. You can conduct stakeholder analysis using the following process. First, create a list of stakeholders that you think will have an interest in your AIMS or might be impacted by it. Typically, a list of stakeholders may include customers, employees, vendors, regulators, investors, and members of the wider society. Stakeholder groups may have a different level of interest and may be impacted differently, so it is critical that you define and quantify if possible the level of stakeholders' involvement in your AIMS, their interest, and how AI will impact them, so you can identify the impact.

Second, identify the concerns and needs of your stakeholders through surveys, focus groups, direct communication, or meetings. Documenting stakeholders' concerns will be of great help during the risk management phase. The concerns of a customer who is worried about privacy, the concerns of an employee who is worried about job displacement, the concerns of a regulator who is worried about compliance, and the concerns of an investor who is worried about returns are all different, and they all need to be understood and addressed. A stakeholder analysis that only identifies stakeholders without understanding their concerns is incomplete and will not provide the insights needed for effective risk management.

Third, communicate the documentation on risk to teams involved in the production of AI systems throughout the AI project lifecycle. Risk management is not a one-time activity that happens at the beginning of a project and then is forgotten. It is an ongoing process that must be integrated into every stage of the AI project lifecycle, from design through development, deployment, and monitoring. Teams need to be aware of the risks that have been identified, the treatments that have been chosen, and the indicators that would signal that a risk is materializing. This requires clear communication channels, regular reviews, and a culture in which raising concerns is encouraged rather than punished. When stakeholders are engaged and informed, they become partners in risk management rather than passive recipients of its outcomes.

Chapter 7.4: A Typology of Risks – Understanding the Many Faces of AI Risk

As an intelligent system, AI is different from other technologies. Because it is built on machine learning approaches, AI systems can learn and evolve by themselves. This is a serious risk as the outcome of the new self-learned capabilities may deviate from the initial intention, thus creating new risks for which the organization is not prepared. So, what are the risks that may emerge as a result of AI systems deployment? And how do they affect the organization? To understand this, we need to look at the typology of AI risks. The World Economic Forum and the Committee of Sponsoring Organizations provide a useful typology that helps you categorize risks and the consequences that pertain to each risk.

The first type of risk is performance risk. Performance risk relates to the way an AI system is supposed to work. This is the most common and general AI risk. When something goes wrong in the performance of an AI system, it may be the result of a flaw in the machine learning model, or in the assumptions that went into building that model. A performance risk may also be the result of a typical statistical error that pertains to data manipulation, such as model overfitting or underfitting, or misclassification of data points. Common approaches to mitigate performance risk are to ensure that the data is of good quality, that the robustness testing is monitored, and that the engineers involved in handling data are well-trained to mitigate this type of risk. Performance risk is not just a technical issue; it has business consequences, including customer dissatisfaction, operational disruption, and reputational damage.

The second type of risk is security risk. Security risks relate to malicious attacks. An AI system may be subject to attacks or hacking that may affect its performance. A security risk can be the result of many factors, such as data poisoning, adversarial attacks, and other cyberattacks that may impede the way a system is supposed to work. Security risks are serious, and their impact goes beyond the AI systems. Having a cybersecurity strategy, upstream leadership involvement, and continuous awareness campaigns that raise the level of employees' cybersecurity awareness is one of the most common mitigating strategies. In McKinsey's 2022 State of the AI Report, only 51 percent of respondents said that they have designed mitigation procedures for cybersecurity risk. This suggests that many organizations are underprepared for the security risks that AI systems introduce.

The third type of risk is enterprise risk. Enterprise risks are the result of misalignment between the AI systems and different parts of the organizations. This could be the result of an absence of alignments, or the lack of supporting capabilities that should enhance the performance of the AI systems, like the lack of a talent strategy or data strategy. An enterprise-level AI risk is more complex. It is not easily identified because it deals with the white space of the organizational chart, and the issue of coordination between and among dispersed units and functions. While this type of risk can be the result of many factors such as size, culture, burnout, and fast changes, it is also indicative of weak leadership oversight. Enterprise risk is particularly insidious because it may not manifest until the AI system is deployed at scale, at which point remediation is more difficult and more costly.

The fourth type of risk is reputational risk. When any of the previous risks happen, it creates an impact on the reputation and the brand name of the organization. This is perhaps the most complex of all the risks, because once an issue gets in the public sphere, it becomes outside of the control of the organization. When a backlash against a product of the company happens, repairing it takes time, as it takes time to change people's negative perceptions. PR campaigns and apologies do little to repair the reputational damage, and that is why senior leadership needs to get involved in this particular risk ahead of time. A good mitigation strategy is to make sure that AI governance mechanisms are in place, and that rules and procedures of the AI life cycle project are followed. Reputational risk is also closely linked to trust, and trust, once lost, is difficult to regain.

The fifth type of risk is legal and regulatory risk. Violation of laws and regulations by AI systems presents a high risk. This is particularly true during regulatory uncertainties when legal requirements shift. A good mitigation approach is for the legal team and the HR team to engage early on during the production of different AI applications and algorithms. For instance, conducting algorithm audits of HR automated decision systems can help the organization avoid discrimination and bias. Only 26 percent of participants in the 2022 McKinsey State of AI Report said that they have built mitigation strategies for legal and regulatory risk. This is a concerning statistic given the increasing number of AI regulations being enacted around the world and the severe penalties that can result from non-compliance.

The sixth type of risk is AI scalability risk. A risk that is inherent to the nature of an automated system relates to AI's ability to scale and amplify the decision-making processes. When decisions are made by humans, the traditional way, their consequences tend to be limited in scope and context. When decisions get automated, their impact gets amplified through speed and scale. This has the potential of impacting a larger number of the population and creating a larger risk that may span different legal jurisdictions. A biased human loan officer might reject a handful of applicants per day; a biased AI system might reject thousands or millions, affecting entire communities and creating systemic harms that are difficult to reverse.

The seventh type of risk is the black box risk. Machine learning models, and particularly deep learning neural networks, use huge amounts of data to produce outputs, without explaining how the model reached its conclusion. Because of the complexity of the data variables an ML model uses to make decisions, the risk of humans not being able to understand how the system reached its decision is high. The lack of explainability and interpretability presents a legal and ethical risk. When senior leadership cannot explain how a system reaches its conclusion, it creates accountability and a trustworthiness problem that may affect the entire governance model. The black box risk is particularly challenging because it strikes at the heart of democratic accountability. If decisions that affect people's lives are made by systems that no one can explain, then the legitimacy of those decisions is called into question.

Chapter 7.5: Linking ISO/IEC 42001 to Other ISO Standards on Governance and Risk

To better understand how to implement the planning section of the 42001, you need to read it in conjunction with two other ISO/IEC standards: ISO/IEC 38507:2022, Governance Implications of the Use of Artificial Intelligence by Organizations, and ISO/IEC 23894:2023, AI Guidance on Risk Management. The first standard, ISO 38507 governance, presents an overview of the governance process, frameworks, tools, and the general philosophy of how governing bodies should engage with AI systems. It is a guidance standard, not a requirement, that can be of great help to all types of organizations, regardless of their size and industry. It is also linked to other ISO standards, such as the Governance of Organization (ISO 37000), the Governance of IT (ISO 38500), and the Governance of Data (ISO 38505).

The second ISO standard you need to look at during your planning phase is ISO/IEC 23894, Artificial Intelligence, Guidance on Risk Management. This is a critical guidance standard that adds value to the planning process. It builds on ISO 31000 Risk management and provides frameworks, tools, and processes by which an organization should conduct a risk management analysis of the AIMS. It provides detailed guidance on how to identify, analyze, evaluate, treat, monitor, and review risks, and it aligns with the broader risk management principles that are applicable across all domains.

Furthermore, to better implement Section 6 of 42001, you need to read it in conjunction with Section 4, the Context of the Organization, of the ISO/IEC standard 42001. A critical dimension is to incorporate the results of the organizational analysis you conducted into the implementation process by answering the following two important questions. First, how did your understanding of the context help you make a better plan for the standard implementation? Second, what specific issues has the context analysis identified that you need to pay attention to and address during the planning of the implementation process? Section 6 is linked to Section 4 because the risks that matter to your organization are shaped by its specific context. A risk that is material for a healthcare organization may be immaterial for a manufacturing company. A risk that is material in the European Union may be immaterial in a jurisdiction with different laws and regulations. Context analysis ensures that risk management is tailored to the realities of your organization, rather than applied as a generic checklist.

Chapter 7.6: AI Risk Management Planning – Principles, Framework, and Process

Putting together a viable framework for AI risk management is a complex task. This is especially true for a transformative technology that is rapidly evolving. A survey conducted by Deloitte, The State of the AI, in 2022, found that 50 percent of respondents to the survey consider risk as a barrier to scaling AI projects, while only 30 percent of respondents said that they have aligned their AI risk management process with that of the overall organizational risk management framework. Another more revealing finding about the complexity of the risk management process and how organizations deal with AI risks comes from the 2022 McKinsey State of AI Report. In this survey, McKinsey did a comparison of risk mitigation strategies between the years 2019 and 2022 and found that little has changed since 2019 in the way organizations deal with AI risk management and mitigation strategies.

Your approach to risk management should be an anticipatory one and as comprehensive as possible. Fortunately, during the last two years, there has been an explosion in AI publications, research, and frameworks that guide AI risk management. Helpful frameworks include the US National Institute of Standards and Technology (NIST) Artificial Intelligence Risk Management Framework (AI RMF 1.0), the UK Information Commissioner's Office (ICO) Data Privacy Impact Assessment, the COSO Enterprise Risk Management (ERM) Framework, the Responsible AI Institute Framework, and many frameworks published by organizations in the private and technology sector such as Microsoft, BCG, Deloitte, and PwC. These frameworks provide a wealth of tools, templates, and best practices that can be adapted to the specific needs of your organization.

According to the McKinsey 2022 State of AI Report, interest in AI, as shown by the level of investment and adoption, has accelerated between 2019 and 2022 as more companies report that AI has become a priority for their strategy. However, when it comes to AI risk management, the survey depicts a different portrait. There has been no progress during the last five years in AI risk management strategy, and, in some cases, the interest even decreased. According to the McKinsey report, the number of companies who have considered AI risks in security in 2019 was 48 percent, while in 2022 that number climbed only moderately to 51 percent, a three percentage point increase. For AI risks in regulatory compliance, the progress during the last three years was even less, 35 percent in 2019 versus 36 percent in 2022, a one percentage point increase. In personal and individual privacy risk, the interest was 30 percent in 2019 versus 28 percent in 2022, a drop in interest by two percentage points. For organizational reputation, the interest in risk was 29 percent in 2019 versus 22 percent in 2022. These are alarming figures as they show a clear lack of progress in AI risk management strategies despite the accelerated interest in AI investment and adoption. While one would assume that leadership is aware of the implications of this gap on organizational performance, these figures point also to the complexity of the AI risk function in an environment that is characterized by ambiguity and uncertainty.

Chapter 7.7: AI Risk Management Foundations – Eight Principles for Effective Risk Governance

A common mistake in thinking about risk management is to assign a person or a department to the function of risk and ask them to come up with a framework to identify, analyze, categorize, and mitigate AI risk. Because AI risk is everywhere, and AI risk tends to permeate all facets of the organization, it is important that, when you address AIMS risks, you take a comprehensive approach. A very helpful risk management framework is the NIST framework. This framework, published in 2023, provides an excellent approach to risks in AI systems. It is also built around a set of principles that are like the principles of ISO/IEC 42001. The following eight principles provide a foundation for effective AI risk management.

The first principle is that AI risk should be integrated into the enterprise risk management system. AI risk should be part of the enterprise risk management strategy and should be approached as part of management activity. Infusing an AI risk management philosophy into the entire organization raises the profile of AI risk and helps managers and employees engage with the AI risk. De-clustering AI systems risks from your overall risk management practice may negatively affect teams' communication and their common understanding of the risk issues, which in turn may affect their engagement in the AI risk activity. Questions to ask during a risk assessment process that will allow you to integrate risk into the overall organization strategy include: To what extent is our AI risk management system integrated into the overall risk management of the organization? How does the AI risk management system link to other risks? Do we have a central authority that addresses AI risk and risk in general? Is there a risk officer in charge of the entire AI system risk across the enterprise?

The second principle is to embrace a holistic perspective. The second principle of AI risk management, according to the ISO AI risk management, is to structure the risk management activity and to adopt a holistic approach to its deployment. When AI risk activities are structured, they are more sustainable and lead to consistent and coherent outcomes. The comprehensiveness of AI risk activities helps you align risk tools and practices with the vision of the organization. A survey conducted by the Economist Impact in 2023 indicates that 77 percent of executives agree on the need to use a comprehensive approach to risk management, 46 percent report investing in an enterprise-wide risk management approach, and 36 percent report integrating risk management into strategy and decision-making processes. Questions to ask to create a holistic approach and structured methodology for your AI risk management system include: Is our AI risk management process organized, well structured, and comprehensive? Does our AI risk system cover all aspects of the AI Project life cycle? Does our AI risk management system consider the impact on all our stakeholders? Does our AI risk process account for all details, the factors, and the perspectives that need to be addressed, and how comprehensive our risk approach is?

The third principle is to customize your approach. Because organizational context differs from one to another, customization of the approach and the process of risk analysis to the specific context of the organization is of paramount importance. Applying a standardized version of risk analysis may not yield positive results. Questions to ask include: Does the AI risk management system address the specificity of our context? How granular should our analysis of the context be? The more detail and more granularity the better customization is. Does our AI risk management system take into consideration the type of demographics we serve and the type of data we use? Does the organizational analysis take into consideration the type of challenges specific to our sector? In what way does the AI risk management system take into account the value system, our specific culture, and the way we do business?

The fourth principle is to be inclusive of your stakeholders' views. AI risk management process must be inclusive of stakeholders' views and positions. When an AI risk analysis is inclusive, engagement from different actors internally and externally is achieved. More importantly, engaging inclusively creates more diversity in the way the system is designed, which in turn helps better identify bias and blind spots that may create hidden risks. All groups that may be affected, directly or indirectly, by the AI systems should be identified and their voice should be reflected in the risk analysis process. Inclusiveness also adds a better transparency layer to the AIMS. Questions to ask to make your AI risk management system more inclusive include: Have we engaged with different segments of society that may be impacted by the AI systems? Does our AI risk management framework cover a wider range of demographics, and have we engaged with them to gauge their perception and views about our AI systems? Does our risk management classify impacted populations by degree of level of impact and risk?

The fifth principle is to adopt an agile mindset. Because of the shifting economic and regulatory environment of AI, and the changing nature of the technology itself, a good AI risk management philosophy is the one that takes into consideration change. Approaching this system with an agile and nimble mindset, able to adapt and adjust when technology shifts, helps the AI risk management system to be more responsive. Questions to ask include: Is our AI risk management agile and able to adapt to shifting environments? Can our AI risk management system adapt quickly to emerging risks? Can our AI risk management anticipate changes that may affect its validity? Can AIMS adjust to the shifting social perception of the people we serve?

The sixth principle is to spell out your assumptions. Another implication of the changing nature of AI technology is the assumptions under which we build AI systems. When designing an AI system, always assume that the design is to the best of what is available in terms of information, data, and overall knowledge. Things may change and that may affect the effectiveness of the AI systems. Questions to ask include: Do we train our engineers to keep track of emerging trends in the way data and information are used in producing AI systems? Are we transparent with the stakeholders about the type of information and assumptions we use to build the AI systems? Do we share with our stakeholders the level of our confidence in our data and what we think about its quality? Does our risk management system address the issue of transparency of data?

The seventh principle is to pay attention to cognitive bias. The analysis of risk is conducted by humans. Perception, interpretation, differences in views, professional approaches, and cultural backgrounds all interfere in the design and deployment of AI systems. This is particularly true for global companies that operate across cultures and cross-legal jurisdictions. Questions to ask include: Does our AI risk management system address the issue of diversity in teams in charge of AI production? Do we train engineers in charge of AI systems on cultural sensitivity issues that surround AI systems, data sampling, and data collection? Does our risk management require documentation of these types of training?

The eighth principle is to learn and improve. The AIMS can improve only if we can capitalize on previous implementation lessons that document aspects of AIMS performance. Assessing the system and learning from the assessment helps improve your AI risk management process. Questions to ask include: Does our AI risk management system take into consideration the need to improve the process by which risk is analyzed? Do we keep track of errors, incidents, and issues that help us improve the system? Finally, it is important to mention that risk drivers that drive organizational interests in improving risk management approaches differ from one organization to another. In a survey conducted by the Economist Impact in 2023, it was found that risks that drive organizations' efforts to improve risk management are technological risk (43 percent), health risk (39 percent), financial risk (41 percent), human capital risk (39 percent), and operational risks (38 percent).

Chapter 7.8: The Planning of Data Management Risk – An Imperative for AI Management Systems

Data has always been a contentious research subject. Scientists, researchers, policymakers, and business leaders use data to understand complex phenomena, solve problems, and make decisions. A 2022 CIO survey conducted by GovLoop, a US-based research company, reveals that 86 percent of organizations have either adopted or are considering data-driven projects, while 93 percent of federal chief data officers say leadership relies on data-driven insights to make decisions. Data has become a critical component of the modern management function and the decision-making process. With AI, data has acquired a new status and a new function. Data is now used to teach machines how to make decisions and train algorithms on how to create decision models. Data is the engine that powers the working and thinking of machines.

However, issues of data quality, such as representativeness, bias, and sampling, are still a challenge for most companies. A survey on the State of AI Readiness conducted by McKinsey in 2023 reports that the majority of participants in the survey continue to have issues with their training data. Respondents to the survey identified the following problems with data: data noise (68 percent), biased data (50 percent), and domain gap, or the lack of representativeness of the data (48 percent). A similar study conducted by BCG found that 70 percent of companies struggle with the quality of data from sourcing it to validating it and sharing it. This is why ISO/IEC 42001 makes data quality a cornerstone requirement of AIMS.

A crucial requirement for the certification process and the functioning of AIMS is to ensure that the data is of good quality. ISO/IEC 42001 requires that organizations define the requirements of good data quality and ensure that their data management practices meet those requirements (Annex A 7.4). Good-quality data, according to a Digital Curation Centre report published in 2020, is data that is "fit for use." A fit-for-use dataset has the following attributes. Sensitivity: data that is anonymized can be exploited commercially and protects the privacy of the individual as well as the causal influences. Representative: data that is free from cognitive bias, such as selection bias, exclusion bias, reporting bias, and detection bias. Accuracy: data that is objective, precise, reliable, valid, legitimate, and labeled. In the Zeitgeist survey, 33 percent of the respondents cite data labeling as a major issue. Accessibility: data that is open, accessible, standardized to a certain degree, and secure. Coverage: data that consistently has geographical and temporal coverage. Figure 7.3 provides a summary of the data quality requirements.

But how does data get corrupted? And in what stage of the data collection process does data lose its quality? According to Karen Hao, an American journalist specializing in AI and contributing to The Atlantic, data quality gets impacted during three stages: the data collection stage, the preparation stage, and the stage in which you frame the problem. It is during these three stages that bias seeps into the data. In the data collection phase, it is important that you pay attention to the way you collect, classify, and label data as bias may seep into the data unintentionally. Bias in this phase can take many forms, like the bias that affects the sampling. If the data you collected is inherently biased toward a group of people, then your machine learning decision will also carry in them that same bias. In HR, for instance, we use previous hiring practices and promotion data to make predictions about future workforce planning. But if the data you used to train AI to make those predictions about workforce planning is not a true representation of the workforce, then most likely your prediction will be biased against the group that your data did not account for.

According to Karen Hao, data preparation is an important step in which subconscious bias can be introduced into the AI decision-making process. When we prepare data, we use attributes to segment the sample being studied so we can learn about patterns, correlations, and causation. Examples of attributes may be age, gender, level of education, driving habits, number of traffic accidents, source of income, nationality, zip codes, etc. But if you use an attribute that is in itself discriminatory, then, of course, your algorithm will also learn how to discriminate "subconsciously." A zip code, for instance, while on the surface may be a neutral attribute, in reality, it might be a discriminatory factor. The third way that might create bias in your AI decision-making, according to Karen Hao, is the way you frame the problem your AI system wants to solve. This is another way through which bias seeps into AI decision-making without intention. How you ask the question and what type of language you use to describe the issue is in itself indicative of the subconscious bias you carry. We tend to ask questions and frame problems through our own social and cultural lenses, while problems and issues to be solved tend to have their own cultural and social context. For instance, can a research consulting company based in London or Paris frame or define a problem for a community living in a city in Africa? As we say, machines don't discriminate. It is the dataset that discriminates. And datasets by themselves do not discriminate. It is the way we handle datasets that can create a bias in AI decision-making. This is why it is important to train data scientists, and generally speaking, those who handle data in the AI space, on some basic ethical research principles so you can teach engineers to teach machines how to make better human decisions.

Chapter 7.9: The Planning of Change – AI Management System Change Strategy

The last two requirements of the planning section (6.2 and 6.3) are devoted to the way you plan and implement your AIMS change strategy. A critical recommendation of ISO/IEC 42001 is to plan your AIMS objectives at different levels of the organization (enterprise, business unit, function, etc.) in a way that is aligned with the overall strategy, and consistent with the AI policy. Lower-level objectives, according to ISO/IEC 42001, need also to be monitored, updated, and communicated by organizational objectives so employees are aware of the linkage between these objectives and the overall business and AI objectives.

A good framework that helps you implement this section is the well-known Kotter model of change. As I have argued, AI, as a new technology with the power of a general-purpose technology, will impact the entire organization. Implementing an AIMS without paying attention to this particular dimension may affect the overall AI planning process and may jeopardize the effectiveness of the AIMS. When you adopt a consistent framework with a proven methodology of change, implementation becomes easier. The first step in Kotter's model is to create a sense of urgency. When implementing your AIMS, rallying those who will be in charge of the AI systems' implementation behind common objectives is a challenge. Because people are also doing other things, have their priorities, and are focused on their day-to-day KPIs, their dedication to the AIMS may not be fully secured. As argued by Kotter, injecting a sense of urgency motivates people to reset their priorities. The sense of urgency is a powerful marketing tool that helps you deal with complacency, especially with skeptics. Without a sense of urgency, AIMS defaults to a backseat position on people's busy schedules. So, make your AIMS change strategy a now-priority task.

The second step in Kotter's model is to build the guiding team. When choosing people who lead the AIMS implementation, use the following suggestions: Identify employees available to meet as frequently as possible. Include a diverse background (different functions and professional backgrounds as well as cultural background). Appoint a senior leader with enough credibility and clout to lead the change. Their role in setting the tone is of critical importance. The third step in Kotter's model is to get the right vision. In Kotter's model, getting the right vision means aligning people around the purpose of the AIMS. We have already explained in previous sections how you establish a vision according to ISO/IEC 42001. Here you are connecting people with that vision with emphasis on the why of the AIMS, and how it should be implemented.

The fourth step is to communicate for buy-in. The opportunities and threats that the new AI age brings should be enough of a motivator to engage people around the AI system vision. However, because people may have a different perception of the technology, you need to work on a communication plan that helps you secure the buy-in of your internal stakeholders. Getting the buy-in is your first cognitive hurdle to rally the organization. Work with the marketing team to design a communication plan over a long period to explain the AIMS, and why you need to invest in it. The fifth step in Kotter's model is to empower teams. There are many actions you can take to empower the team, like showing the opportunities the AIMS will bring to the organization, discussing the threats of not investing early on in an AIMS and how this will impact the organization, allowing for risk and failure during the implementation to enhance the psychological safety of teams in charge of the AIMS, focusing on collaboration and cross-functional teams with industry knowledge, and helping your team see the outcome of their work by creating short-term wins to encourage and motivate them.

The sixth and final step in Kotter's model is perseverance. Kotter calls this step the "don't let up and make it stick" step. In this step, your executive presence, constant engagement, and motivation are of critical importance as they help employees stay focused on the AIMS implementation. Change is not a one-time event; it is a continuous process that requires sustained attention and effort. Organizations that treat change as a project with a beginning and an end will find that the changes they implement are not sustained over time. Organizations that treat change as a continuous capability will be able to adapt and evolve as circumstances change, maintaining the momentum of transformation over the long term.

Chapter 7.10: Integrating Risk Management into the AI Project Life Cycle

Risk management is not something that happens once, at the beginning of an AI project, and then is forgotten. It is an ongoing process that must be integrated into every stage of the AI project life cycle. During the design phase, risk management involves identifying potential harms, assessing the likelihood and severity of those harms, and designing controls to prevent or mitigate them. During the development phase, risk management involves testing the model for bias, robustness, and security, and addressing any issues that are identified. During the deployment phase, risk management involves monitoring the model's performance in the real world, detecting and responding to incidents, and continuously updating the model and its controls as new risks emerge.

Integrating risk management into the AI project life cycle requires clear roles and responsibilities, documented processes, and a culture in which risk awareness is everyone's responsibility. It also requires tools and techniques that are appropriate for the specific risks that AI presents. Traditional risk management tools, such as risk registers and heat maps, are useful but may not be sufficient for the complex, emergent, and rapidly evolving risks that AI systems introduce. Organizations may need to adopt new tools and techniques, such as algorithmic impact assessments, bias audits, adversarial testing, and red teaming, to effectively manage AI risk.

The ISO/IEC 42001 standard provides a framework for integrating risk management into the AI project life cycle, but it does not prescribe specific tools or techniques. Organizations must choose the tools and techniques that are appropriate for their context, their risk appetite, and the specific AI applications they are deploying. The key is to be systematic, to document the process, and to continuously improve based on experience and new information. Risk management is not a destination; it is a journey. The organizations that succeed in managing AI risk are those that treat it as a core capability, not a compliance burden. They invest in the people, processes, and tools needed to identify, assess, and respond to risk effectively. They create a culture in which risk is discussed openly and honestly, and in which learning from failures is valued. And they recognize that risk management is not just about avoiding harm; it is about enabling the organization to take intelligent risks that create value.

Chapter 7.11: Chapter Summary & Key Takeaways

The planning section (Clause 6) is the most critical section of ISO/IEC 42001, dealing with risk assessment, risk treatment, and impact assessment.
Risk in ISO standards is defined broadly as any deviation from objectives, whether negative (threat) or positive (opportunity).
Risk assessment evaluates the likelihood and impact of something going wrong; risk treatment defines actions to address risk; impact assessment evaluates the consequences of harm on users.
Stakeholder analysis is a prerequisite for effective risk management, involving identification of stakeholders, understanding their concerns, and communicating risk information throughout the AI project lifecycle.
A typology of AI risks includes performance risk, security risk, enterprise risk, reputational risk, legal and regulatory risk, AI scalability risk, and black box risk.
ISO/IEC 42001 should be read in conjunction with ISO/IEC 38507 (governance) and ISO/IEC 23894 (risk management) for a comprehensive approach.
Risk management should be integrated into enterprise risk management, adopt a holistic perspective, be customized to context, be inclusive of stakeholders, adopt an agile mindset, spell out assumptions, address cognitive bias, and promote learning and improvement.
Data quality is a cornerstone requirement of AIMS, with good data being sensitive, representative, accurate, accessible, and covering appropriate geographical and temporal scope.
Bias can enter data during collection, preparation, and problem framing, requiring training for data scientists and ethical research principles.
The Kotter model of change—urgency, guiding team, vision, communication, empowerment, and perseverance—provides a framework for implementing AIMS change strategy.
Risk management must be integrated into every stage of the AI project life cycle, from design through development, deployment, and monitoring.
Organizations may need new tools and techniques for AI risk management, such as algorithmic impact assessments, bias audits, adversarial testing, and red teaming.
Risk management is not a compliance burden but a core capability that enables intelligent risk-taking and value creation.

Chapter Notes

Mark this chapter as completed

Chapter 8 of 11

Chapter 8: The Support Scaffold – Building the Infrastructure, Competence, and Culture That Make AI Governance Possible

Chapter 8.1: Introduction – Why Support Is the Silent Engine of AI Governance

Section 7 of ISO/IEC 42001 addresses the question of support. By support, we mean the infrastructure and all the actions that you must put in place to enhance the efficiency and effectiveness of your AI management system. Support is often the least glamorous part of governance. It lacks the strategic sweep of leadership, the intellectual challenge of risk planning, or the technical precision of operations. Yet without adequate support, even the most brilliant strategy and the most rigorous risk framework will fail. Support is the silent engine that powers the entire AIMS. It provides the physical, human, and informational resources that make everything else possible.

The standard mandates five types of support. The first is resources: the tangible and intangible resources that you need to put in place for the AIMS to function, including infrastructure, data, tools, computing power, networking, manpower, and talents (Clause 7.1). The second is competence: the framework that identifies the knowledge, skills, and attitude (KSA) required of employees involved in the AIMS (Clause 7.2). The third is awareness: the campaigns that you engage in to inform and raise awareness of your stakeholders about the role and the importance of the AIMS, as well as the risks associated with your AIMS (Clause 7.3). The fourth is communication: the communication strategy you intend to put into place to explain and communicate with stakeholders about matters that relate to AIMS, including public relations campaigns with external stakeholders to explain your AI product, AI strategy, and overall AI approach. Important also to add in the communication strategy is how you address scenarios of reputational risk (Clause 7.4). The fifth is documented information: the documentation, records, and results of studies that relate to the functioning of the AIMS (Clause 7.5).

Some of the documented information required is generic and can be found in other requirements standards such as ISO 9001:2015 Quality Management System, but some of these documents are specific to the AIMS. Furthermore, Annex A provides more details on the type of documented information needed, and Annex B provides helpful guidelines and explanations on how to prepare the documented information. It is important to mention that the standard provides flexibility in the way you prepare the documents. In other words, the standard does not prescribe a specific template beyond the generic requirements of any ISO document, such as identification, dates, author, and so on. You can prepare your documents in any way that fits your type of business, as long as it justifies and explains your rationale in documenting what is required. In the following sections, we will explore each of the support elements in depth, examining what they require and how leading organizations have implemented them.

Chapter 8.2: Tangible Resources – The AI Infrastructure That Powers Governance

As we move toward a more general intelligence system, relying on traditional IT infrastructure to manage AI systems will impact the effectiveness of your AIMS and may not allow your AI infrastructure to grow and develop. Relying on traditional IT infrastructure may even impede the growth and development of your AI systems. AI systems require more powerful computing machines, cloud and hybrid infrastructure, storage capacity, and large amounts of data that legacy IT systems may not be able to deliver. This infrastructure not only allows for the AI systems to be more effective in achieving their objectives, but it can also drive innovation as it helps talents using cloud and hybrid to make connections with other open-source systems, which leads to more creativity and more innovative products. Investing in a modern infrastructure that supports intelligent systems allows you also to gain independence from third-party providers and helps you set the long-term foundations of an emerging ecosystem that will free your AI from IT legacy issues.

In his book on managing large-scale machine learning projects, author Simon Thompson argues that, when it comes to managing large-scale ML projects, business leaders need to be aware of three types of requirements: functional requirements, nonfunctional requirements, and system requirements. While the functional requirements refer to the performance of the model itself, the nonfunctional requirements and the system requirements refer to the infrastructure needed to allow the ML projects and, more generally, the AI systems to achieve their performance goals. Author Terry Storrar, from Leaseweb, a global IT infrastructure company, identifies four critical infrastructure assets that enhance the effectiveness of AI systems. These assets are computing performance, storage capacity, networking infrastructure, and the security of the infrastructure. Together, these four assets form the tangible foundation of any viable AIMS.

Chapter 8.3: Computing Performance – The Hardware Heart of AI

The performance of your computing power and how much calculation it can perform raises essentially a hardware question. The ability of your AI systems to process large volumes of data, make complex calculations, and yet produce accurate predictions is intimately tied to the type of hardware used. The discussion of whether to move from CPU to GPU to upgrade your computing processing power is an important discussion to have. While not all companies can afford a GPU investment, especially if you are a medium or small enterprise, investing in a GPU should be considered as a long-term investment. As the proliferation of data accelerates, companies will rely more and more on data to make new products and services.

Computing performance is not just about speed; it is also about capacity and scalability. AI models, particularly large language models and deep learning networks, require enormous amounts of computation to train and to run inference. A model that takes weeks to train on a single GPU may take hours on a cluster of hundreds of GPUs. This difference in training time can be the difference between a project that is commercially viable and one that is not. Similarly, a model that can process thousands of requests per second in production can serve many more users than one that can only process hundreds. Computing performance thus has direct implications for the value that AI systems can deliver.

For organizations that cannot afford to build their own high-performance computing infrastructure, cloud computing offers a viable alternative. Cloud providers offer access to GPU instances on a pay-as-you-go basis, allowing organizations to scale their computing resources up or down as needed. This model reduces the upfront capital investment required and allows organizations to experiment with AI without committing to expensive hardware. However, cloud computing also introduces new risks, including vendor lock-in, data sovereignty concerns, and dependency on external providers for critical infrastructure. Organizations must weigh these trade-offs carefully and choose the computing model that best fits their needs, resources, and risk appetite.

Chapter 8.4: Storage Capacity – The Data Reservoir Behind Every AI System

As we collect more data, storage capacity becomes an issue. While some companies may not need a larger storage capacity, others may need to support their AI systems. As we know, ML algorithms make better decisions and provide more accurate predictions when they are exposed to larger sets of data. When an ML model has access to more and better data, its level of accuracy and overall sophistication improves tremendously, especially for complex ML algorithms that are deployed on real-world issues. So, the question of storage capacity is a critical question to ponder as it involves other issues, such as cost, security, scalability, and compatibility with existing infrastructure.

Furthermore, data annotation, the process of labeling and adding metadata to create a better context for the model, is intimately linked to storage capacity. According to the Zeitgeist AI readiness survey, a survey published in 2023 by Scale AI, organizations that invest in infrastructure create a better environment for data annotation, which itself leads to faster model production and more frequent reviews and updates of the model. Data annotation is labor-intensive and time-consuming, but it is also essential for building high-quality AI systems. Without adequate storage capacity, organizations cannot store the annotated datasets that are needed for training and evaluation.

Storage capacity also has implications for data governance. Organizations that store large amounts of data must ensure that the data is secure, that access is controlled, and that data retention policies are followed. They must also comply with data protection regulations such as the GDPR, which imposes strict requirements on how personal data is stored and processed. Inadequate storage infrastructure can lead to data breaches, compliance violations, and reputational damage. Investing in secure, scalable, and compliant storage infrastructure is thus not just a technical requirement but a governance imperative.

Chapter 8.5: Networking Infrastructure – The Connective Tissue of AI Systems

Communication between different deep-learning algorithms requires speed and reliability. AI systems may not deliver on their promise if the networking and communication infrastructure does not match the demand level. Of particular importance to the networking issue is the latency problem, and the time it takes for a system to respond to inputs, like in the case of speech recognition and self-driving cars. While investment in upgrading the networking infrastructure incurs more cost, in the long run, it dramatically improves AI system latency, a critical element for enhancing customer experience.

Networking infrastructure is not just about speed; it is also about reliability and resilience. AI systems that are deployed in mission-critical applications, such as healthcare, finance, and transportation, must be available when they are needed. Downtime can have serious consequences, from financial losses to safety risks. Organizations must therefore design their networking infrastructure with redundancy, failover, and disaster recovery capabilities. They must also monitor network performance continuously and have processes in place to respond to outages and degradation.

The networking requirements of AI systems also vary depending on where the AI is deployed. Cloud-based AI systems require high-bandwidth connections between the organization and the cloud provider. Edge AI systems, which process data locally on devices rather than sending it to the cloud, require low-latency connections between devices and local processing units. Hybrid AI systems, which combine cloud and edge processing, require seamless integration between the two environments. Each of these deployment models has different networking requirements, and organizations must choose the model that best fits their use case, their performance requirements, and their infrastructure capabilities.

Chapter 8.6: Security of the Infrastructure – Protecting the AI Supply Chain

The security of your AI system infrastructure is of paramount importance. Whether you decide to invest in CPU or GPU, or upgrade your networking communication infrastructure or not, securing the AI system infrastructure is a must. AI systems handle sensitive data that, if breached or poisoned, will have legal implications and disastrous consequences on the business performance and the reputation of the organization. Security is not just about protecting data at rest or in transit; it is also about protecting the integrity of the AI models themselves, the training pipelines, and the deployment environments.

AI systems introduce new attack surfaces that traditional security measures may not address. Adversarial attacks, for example, involve manipulating input data to cause a model to make incorrect predictions. Data poisoning attacks involve injecting malicious data into training sets to corrupt the model's behavior. Model extraction attacks involve querying a model repeatedly to reverse-engineer its parameters or training data. These attacks can be difficult to detect and may have serious consequences, particularly in high-stakes domains such as healthcare, finance, and national security. Organizations must implement security controls that specifically address these AI-specific threats, including adversarial training, data validation, model monitoring, and access controls.

Finally, to conclude the section on tangible resources, it is important to take a look at both Annex A and Annex B of the ISO/IEC 42001. These two annexes provide more details on the type of resources needed and how you should approach their implementation. Annex A lists the controls that organizations must implement to meet the standard's requirements, while Annex B provides guidance on how to implement those controls. Together, they provide a comprehensive framework for building the tangible infrastructure that supports an effective AIMS. Organizations that invest in this infrastructure will be better positioned to develop, deploy, and govern AI systems that are reliable, secure, and trustworthy.

Chapter 8.7: Intangible Resources – The AI Competence Model

The second important clause of Section 7 addresses the issue of the competence model. While the standard does not require any specific competence model, it does suggest that for an AIMS to run effectively, a competence model that defines the type of knowledge, skills, and attitude (KSA) needs to be in place. So, let us define what a competence model is, and then discuss how you should build a competence model that is focused on AIMS.

A competence model is a framework that helps an organization define the knowledge, skills, and attitudes required for employees to perform. It is an inventory of skills and behaviors that the job description identifies as a requirement to perform on the job. A competence model identifies individual capabilities and answers the following basic question: "What type of knowledge, skills, and attitudes are required so my employees achieve their goals?" To put in place a competence model that supports your AIMS, you need to work with your HR and Training and Development departments to identify the type of KSA needed for different employees involved with the AIMS. This is not an easy task to do as you need to pay attention to three challenges: the challenge of shifting job descriptions required in an emerging field, the issue of legacy facing the building of a new management system, and the question of interdisciplinarity needed to coordinate complex management systems such as AIMS.

First, for an emerging field such as AI, identifying the knowledge, skills, behaviors, and overall capabilities needed is not an easy thing to do as job requirements continue to shift. The knowledge complexity of the AI field and the interconnectedness of the AIMS with other knowledge domains make it difficult to come up with an agreement on what constitutes a competence model for an AIMS. The disruptions the workplace is going through today and the cultural change taking place in many industries require that you come up with an agile approach to defining the type of capabilities needed to run an AIMS. So, start by moving away from the traditional job description process to adopt a nimble approach that focuses on results and takes into consideration the general shifts affecting the workplace.

Second, putting together an AIMS for the first time in an organization has its challenges. Building an AIMS on top of a legacy management system is a daunting challenge. Start small and focus on the core people doing the actual AI job. Then, expand from there to include all types of talents needed for the AIMS to operate, and how these talents interface with the new AIMS. Third, adopting an interdisciplinary approach to mapping out the individuals' competencies needed is a must. Because AI cuts across many fields and specializations, a comprehensive approach and a systemic view to building the skills and individual capabilities needed make your AIMS more effective.

Chapter 8.8: The Three Competence Domains – Planning, Data, and Execution

A good starting point to build an AI-focused competence model is to start with a competence model that is comprehensive and takes into consideration the specificity of an AIMS. A newly published model by the Broadband Commission for Sustainable Development provides a framework that defines in detail the type of knowledge, skills, and attitudes needed for an AI system. Although written for public service employees in general, it is a good framework that can be applied to all types of organizations. It aims to help leadership think of an AI competence model more comprehensively and systematically when putting together frameworks for AI systems. The Broadband Commission model identifies three competence domains, fourteen skills, five attitudes, and four proficiency levels that measure the performance of each competence domain.

The first competence domain is digital planning and design. This knowledge domain identifies concepts, principles, and practices that relate to the planning and design functions of digital transformation. For instance, technical proficiency, when combined with a strategic and anticipatory thinking approach, provides talents with the ability to plan and anticipate events that may affect the performance of an AIMS. An outside-in approach combined with a risk-based perspective on technology progress, on the other hand, helps you identify not only the market opportunities of your AIMS but also the risks associated with different AI applications. So, building this competence domain to run an AIMS is critical. Furthermore, digital planning and design competence needs applied skills that allow employees to become more nimble. The Broadband Commission model identifies four sets of skills that, when combined with digital planning and design skills, will allow the AIMS to be more effective: system thinking, problem identification, strategic foresight, and agility. Employees working on an AI project who exhibit these skills will be able to achieve better results. For instance, agile employees adapt better and respond faster to change requirements while problem identification skills allow teams to agree on the definition of the problem, which should allow for faster execution. System thinking skills help employees think in a more comprehensive and integrative way in addressing AI project planning and execution.

The second competence domain in the Broadband Commission model addresses the issue of data usage and data governance. This competence domain is critical to the functioning of an AIMS because it helps employees understand the process by which they turn raw data into exploitable insights while understanding the risk that is inherently built into the data. Data management, a field that used to be a vertical and obscure specialization for IT professionals, has become a horizontal function that impacts every aspect of the AIMS. Knowing how to deal with it not only helps employees leverage it in their decision-making process but also makes them aware of its harm to the user and the company. So, this is why your competence model should focus on data governance issues, so employees involved with the AIMS know how to collect data, how to label it and sample it, and how to maintain it and share it while fully compliant with privacy and personal information requirements throughout the AI project life cycle. Skills that help you operationalize this competence domain, according to the Broadband Commission, are digital literacy, data-driven decision-making, open data, legal and ethical skills, and AI fundamentals. When the AI team is trained on these skills, its ability to take advantage of the many opportunities the AIMS offers is limitless. Your employees become better at assessing the true value of an AI project and can identify the vulnerability of data breaches in the system early on, which makes the AI project resilient to attacks and breaches.

The third competence domain is digital management and execution. This competence domain, according to the Broadband Commission model, refers to the ability of the AI team to plan and execute an AI project. Beyond the planning and the design, the issue of execution is of paramount importance to the success and failure of an AI project. In the details of the execution resides the difference between failure and success. So, a critical competence domain for an AIMS is to ensure that employees involved in AI projects are good at managing and executing projects. Attention to detail, being nimble in the execution, social perceptiveness, time management and meeting deadlines, and transparency and courage to raise issues that may impact execution are the competencies that are needed for an AIMS. This competence domain gets enhanced with the following skills according to the Broadband Commission model: people centricity, iteration, agile execution, and agile leadership. According to the model, these skills are critical to the AI project execution. Customer centricity, for instance, allows team members involved in managing AI projects to learn about project pain points. It helps them put customers' needs and wants at the core of the AI project. Iteration skills, on the other hand, allow for the team to integrate feedback from different stakeholders involved in the project into different stages of the AI project life cycle, which should impact the overall effectiveness of the AIMS.

Chapter 8.9: The Five Attitudes of AI Competence – Creativity, Adaptability, Experimentation, Curiosity, and Trust

In addition to the skills and knowledge domains required of the AIMS, the Broadband Commission model also defines the types of attitudes and behaviors that are suitable for an AI project. Attitudes and behaviors are key to the successful implementation of an AI project because they provide us with a window into the value system, the culture, and the inner thinking of people. Learning about how people perceive things and interpret the world around them, what motivates them, and what turns them off helps you design strategies that reinforce positive behavior and eliminate negative behavior. The model identifies five attitudes that are essential for AI competence.

The first attitude is creativity. The ability of people to generate ideas, think critically, and solve problems using new and recycled ideas is critical to the effectiveness of an AIMS. This attitude is important because it helps your AIMS become more innovative in addressing new challenges and pain points. The second attitude is adaptability. The ability of people to adjust to new and changing circumstances, pivot to more critical issues, and respond faster to new requirements is an attitude that has become critical. Training people interacting with the AIMS on acquiring this attitude makes your AIMS more agile and nimble, especially during uncertainty. The third attitude is experimentation. For an emerging technology, people with an experimental mindset add tremendous value to AIMS. As a nascent technology, the potential of AI is tremendous. People who try things out and experiment with failure bring additional and more strength to the performance of the AIMS.

The fourth attitude is curiosity. This attitude is linked to experimentation. Curiosity is what fuels the creativity and innovation of people involved with AIMS. The fifth attitude is trust. As an attitude, trust is critical to experimentation. Trusting your employees to bring new ideas, play, and experiment with those ideas helps your AIMS become more innovative. If you remove trust and replace it with compliance, people default to a traditional job description that may impede innovation and creativity. Finally, for the competence model to work, it has to be clear and communicated. As well described by a PwC study, a successful competence model has to have four Cs: clarity of what is required to succeed; consistency in how you measure the skills needed; connecting skills and competencies to the organizational strategy; and continuity in the approach and the message being taught.

Chapter 8.10: Awareness – Ensuring Everyone Understands the AI Policy and Its Implications

For a technology that will transform the organization, communication of its impact and awareness of its effects are key. Communicating the objectives of the AIMS and explaining to different stakeholders the scope and the impact of AI technology is a critical dimension of an effective AIMS. Leadership needs to understand this dimension as well as its implications on the overall organization's performance and liability. This is why ISO/IEC 42001 (Section 7.3) requires that organizations design an awareness campaign and communication strategies to accompany the AIMS implementation. Three requirements relate to the awareness clause. Organizations must inform and make employees aware of the AI policy. Organizations must show how employees contribute to the overall effectiveness of the AIMS. Leadership must make employees aware of the risk of not conforming to the AI Management system requirements.

The first requirement is that all employees need to be aware of the AI policy. This is a relatively strong requirement as the leadership is expected to engage in an awareness campaign to explain the AI policy of the organization. The standard does not specify what exactly needs to be communicated to employees about the AI policy, but for your awareness campaign to be engaging, you need to communicate at least the following elements. Governance and leadership: employees need to know who is in charge of AIMS. They need to be aware of different roles and responsibilities within the AIMS, as well as the governance mechanism by which leadership exercises oversight. Employees need also to know how to reach different leadership levels in charge of the AI governance when they need to report something about the AIMS. AI scope and objectives: an AIMS exists to achieve business goals and strategic objectives. Employees need to know the objectives of AIMS, its scope, and why the organization invests in it. When employees are aware of leadership thinking about the AIMS, their commitment and engagement increase. Being open about the goals and objectives also impacts positively organizational trust.

Use of responsible AI: this is another critical dimension of the AI Management system that needs to be well communicated to employees. As AI becomes more mainstream technology and as certain AI applications become more controversial, leadership needs to take a proactive approach to clarify how the organization intends to use AI in a responsible AI way. Responsible AI simply means that the organization subscribes to the laws and regulations that guide the use of AI, as well as the ethical principles of do no harm, fairness and nondiscrimination, privacy protection, and explainability. AI risks: another equally critical aspect of the AIMS that employees need to be aware of is the issue of risk. In many ways, this is a challenge for leadership. A good approach to deal with this challenge is to come up with a risk classification approach that identifies risks from high to low, and how the organization intends to deal with different levels of risks. Employees, especially those involved directly with the AIMS, need to be aware of the tools and the methodology the organization adopts to measure and assess risk, so these risks are integrated throughout the AI project life cycle. Data usage: finally, the issue of data. A critically important aspect of the AIMS that employees need to be aware of is the company's policy on data and how employees should use data. This is the responsibility of leadership and management at different levels to make sure that their employees understand the inherent risk to data and the company's policy on data, especially who and when to access data, how to process it, and with whom to share it.

Chapter 8.11: How Employees Contribute to a Better AI Management System

The second requirement of the awareness clause is for leadership to ensure that employees are aware of their contribution to an effective AIMS. For an AIMS to function effectively and efficiently, it requires many moving parts. Many of these moving parts that optimize the effectiveness of the AIMS are not centralized in one unit, like in the case of traditional IT. As technology seeps into all facets of the organization, the challenge is to help employees understand that their work, even if it does not appear to be tech-related work, does in many ways impact the overall AI ecosystem. For sales, customer service, marketing, and HR teams who may think of themselves as not traditionally IT teams, their impact and their contribution to the effectiveness of AIMS are critical, especially with upstream teams dealing with data.

Following are a few suggestions that help different teams interacting with the AIMS improve the performance of the system. Communication between different AI teams: an AIMS requires input from a variety of teams—engineers, sales, marketing, HR, and so on. Depending on the context of the AIMS, seamless communication between these different teams optimizes the effectiveness of the AIMS and creates a better integrated AIMS. The use of data: because data comes from everywhere, teams engaged in managing and processing data management need to be aware of their upstream role in the overall health of the data ecosystem. When upstream teams provide low-quality data, this may affect the overall AIMS. The leadership role in raising the awareness level of employees manipulating data is critical. The need to reskill and upskill: as a nascent technology, AI requirements keep changing. AI knowledge itself keeps shifting, so helping employees gain the proper knowledge and skills and helping them translate that knowledge into practical processes allows for the AIMS to be better maintained and up-to-date, which should help the overall performance of the AIMS.

The third requirement of the awareness section is to ensure that employees are aware of compliance risk. Compliance risks are a serious threat to the organization and the AIMS. As AI technology becomes a societal issue and more ubiquitous, pressure for accountability and transparency in how the organization complies with ethics, laws, regulations, and standards increases. The leadership role is to explain and show employees the compliance process and how the organization intends to subscribe to it. Here are a few steps that could help your employees engage with compliance. Encourage your teams involved with the AI project to engage with the legal department early on during the AI project life cycle to flush out compliance issues. Search AI legislation and regulations in your country and other countries where you operate. Explain to your employees how your AIMS should comply with these regulations and help them understand the dos and don'ts of your AI applications. As AI regulation and policies keep shifting, create a watch list of AI-related legislation to monitor new and changing requirements in the world that may affect your AIMS. Countries' policies also are shifting, so it is critical to keep an eye on new changes in policies and requirements for AI systems, especially in jurisdictions where you operate. Search for international standards that guide organizations on compliance issues (ISO/IEC standards, NIST framework, European standards, etc.) and get certified on the main ones. Of critical importance is the new set of standards that will be developed by the European Committee for Standardization (CEN) and the European Committee for Electrotechnical Standardization (CENELEC) to operationalize the new EU AI Act.

Chapter 8.12: Communication – Strategies for Engaging Stakeholders and Building Trust

Depending on the context and the economic sector in which you operate, Section 7.4, Communication, provides you with the flexibility to define the best ways to communicate about your AIMS. As long as your AI communication strategy identifies the target audience (who will benefit from this communication), the frequency of communication (how often and when you communicate), the medium of communication, and the objective of the communication (how and why you communicate), the standard provides you with enough flexibility to set the AI communication strategy that fits better your context.

The first best practice is to encourage face-to-face communication. While emails, bulletins, and other electronic media are a good way to explain a new policy and raise people's awareness of the policy, communicating a new and complex system such as AIMS requires direct communication. To employees, direct communication has an enormous benefit. It shows that leadership cares about AI technology and considers it as a high-priority business issue. It also helps employees understand leadership's inner thinking, so they can connect better on such strategic issues. To leadership, direct communication helps them set the tone and build the trust needed during large-scale transformations. Direct communication also helps leadership understand granular implementation challenges and how to address them.

The second best practice is to recognize that the medium is the message. People don't care about what you say but how you say it. When leadership engages personally in communicating about the AIMS, employees sense better the urgency and the interest leadership devotes to AI. Personal engagement of leadership in the AIMS sends a far more important message about the AIMS. It shows that leadership pays careful attention to the investment the organization makes in building the AIMS and is willing to take actions to improve, support, and reward those engaged in making the system a better one.

The third best practice is to create policy champions. A new change always requires a change champion. AIMS champions are crucial to the implementation of the new system. Champions are critical because they are employees who care about the success of the AIMS and are passionate about the new change the AIMS brings to the organization. When change champions are in a leadership position, they lead by example and become role models. Champions also help alleviate resistance and engage skeptics in a discussion about the benefits of AI investments. By identifying and empowering champions at all levels of the organization, leadership can create a network of advocates who help spread awareness, build enthusiasm, and overcome resistance to change.

Chapter 8.13: Documented Information – The Evidence Base for Certification and Improvement

Clause 7.5 on documented information of the AIMS is generic. You can find the same clause in many other standards, such as ISO 9001:2015 Quality Management System, ISO 45001:2018 Occupational Health and Safety, and ISO 14001:2015 Environmental Management System. In this section, we will first provide some general guidelines on how to handle AIMS documentation. Then, we will identify all the actions that must be documented. A simple quick keyword search of the standard will bring up all the required documentation you need. So, let us start with a few general remarks.

First, it is important to mention that ISO/IEC 42001 requires two types of documented information: the documented information required by the standard itself, and the documented information that you deem necessary for an effective functioning of an AIMS. Organizations differ by size, sector, activities, and complexity level, so it makes sense that documented information on the AIMS may differ from one organization to another. Second, when you create and update documentation, documents need to be identified by title, date, author, and reference number. These are general requirements of all ISO management systems. Furthermore, documents can be in any format that suits the purpose of the activity. Documents are also needed to show the authority that reviews and approves the documented information. Finally, documented information that relates to the AIMS must be protected and available. This is a requirement that is meant to create control over access to the documentation, and the maintenance of the documentation itself against damage. The same controls apply to the documented information that is external to the organization and deemed necessary for the effectiveness of the AIMS.

For certification purposes, ISO/IEC 42001 requires the following documents. First, the scope and the activities of the AIMS, its boundaries, and its applicability, in addition to the overall context of the organization (Clause 4.3). Second, the AI policy must be available and documented (Clause 5.5). Third, AI risk and AI opportunities: this is a critical document to have. There are many controls of the risk analysis, so providing a registrar of all the risks is critical (Clause 6.1). Fourth, how the organization conducts its risk assessment (Clause 6.1.2). Fifth, three types of documentation relate to the treatment of risk: justification of why an organization excluded certain types of control, justification of why the organization established its controls, and how and in what way the organization intends to treat risks. Sixth, the process and results of the impact assessment must be documented. The standard provides the option to the organization to share this documentation with interested parties as defined by the organization (Clause 6.1.4). Seventh, AI objectives at different levels of the organization and how the organization plans to achieve them (Clause 6.2). Eighth, the skills, knowledge, and competence of the people engaged in the AIMS (Clause 7.2). Ninth, documentation of how the planning and control process works. This type of documentation is required only to make sure that things go as planned. Tenth, documented information on the results of the three activities that relate to risk: risk assessment, risk treatment, and impact assessment (Clauses 8.2, 8.3, 8.4). Eleventh, documented information on the monitoring and measurement activities of the AIMS (Clause 9.1). Twelfth, documentation of the audit program as well as the audit results (Clause 9.2.2). Thirteenth, the results of the management review of the AIMS (Clause 9.3.3).

Chapter 8.14: Chapter Summary & Key Takeaways

Support (Clause 7) is the silent engine of AI governance, providing the resources, competence, awareness, communication, and documentation needed for an effective AIMS.
The five types of support mandated by ISO/IEC 42001 are resources, competence, awareness, communication, and documented information.
Tangible resources include computing performance (CPU/GPU), storage capacity, networking infrastructure, and security of the infrastructure.
AI systems require more powerful computing, greater storage, lower latency, and stronger security than traditional IT infrastructure can provide.
Cloud computing offers scalable access to high-performance infrastructure but introduces risks of vendor lock-in, data sovereignty, and external dependency.
Security of AI infrastructure must address AI-specific threats such as adversarial attacks, data poisoning, and model extraction.
Intangible resources include the AI competence model, which defines the knowledge, skills, and attitudes required for employees involved in the AIMS.
The Broadband Commission model identifies three competence domains: digital planning and design, data use and governance, and digital management and execution.
Fourteen skills and five attitudes (creativity, adaptability, experimentation, curiosity, and trust) are identified as essential for AI competence.
A successful competence model has four Cs: clarity, consistency, connection to strategy, and continuity.
Awareness requirements include making employees aware of the AI policy, governance and leadership, AI scope and objectives, responsible AI use, AI risks, and data usage.
Employees contribute to AIMS effectiveness through communication between teams, data quality, reskilling and upskilling, and compliance awareness.
Communication best practices include face-to-face communication, recognizing that the medium is the message, and creating policy champions.
Documented information required for certification includes scope, AI policy, risk and opportunity registers, risk assessment process, risk treatment justifications, impact assessment results, AI objectives, competence records, planning and control documentation, risk activity results, monitoring and measurement records, audit program and results, and management review results.
Documents must be identified by title, date, author, and reference number, and must be protected and available.

Chapter Notes

Mark this chapter as completed

Chapter 9 of 11

Chapter 9: From Blueprint to Reality – Mastering the Operational Control of the AI Project Life Cycle

Chapter 9.1: Introduction – Why Operational Control Is Where Strategy Meets Execution

Clause 8 of ISO/IEC 42001 is about operational control. Operational control helps you manage, administer, and monitor the different processes and procedures that allow your AI management system to function effectively and to meet ISO standard requirements. If Clause 6 (Planning) is about deciding what you will do and why, Clause 8 (Operation) is about actually doing it—and doing it in a way that is controlled, measurable, and auditable. Clause 8 is intimately linked to Clause 6, so it is important to read it in conjunction with the Clause 6 requirements, which focus primarily on risk assessment, risk treatment, and impact assessment. The plans you made in Clause 6 become the operational controls you implement in Clause 8.

In simple terms, Clause 8 operational control answers the following simple question: How do I make sure that the processes and procedures put in place function as they were intended to? How do I monitor and control these processes? ISO/IEC 42001 requires three critical steps for operational control. First, define the required controls needed for the functioning of the AIMS. Second, establish a system to monitor, control, and provide corrective actions when needed. Third, document the planning and control process to enhance confidence in the system. To discuss the required controls, we will follow an AI Project Life Cycle approach. We will explain each stage of the AI project life cycle and discuss the controls required by the standard. This approach ensures that controls are not abstract requirements but practical safeguards embedded in the actual work of designing, developing, and deploying AI systems.

Chapter 9.2: The AI Project Life Cycle – A Framework for Managing Complexity

The purpose of an AI project is to solve a problem. Whether it is about automating mundane processes and repetitive tasks or creating completely new business opportunities, the AI project's goal is to take advantage of machine learning technology to improve the organizational decision-making process and impact the bottom line. A challenge, however, that organizations face in managing AI projects is the complexity of the AI project. An AI project differs from a traditional IT management project in many ways. An AI project is more complex, is highly uncertain, has higher risks and costs, and has many moving parts that may or may not fully align. So, a critical step in managing an AI project is to have a clear understanding of the framework that you need to apply to manage the entire AI life cycle of the project.

There are many frameworks you can choose from to put together a process to manage, execute, and improve an AI project life cycle. ISO/IEC 5338:2023 Artificial intelligence: AI system life cycle processes is one of them. But a simple framework that we would like to suggest is the US GSA AI Guide for Government. Made of three basic steps—Design, Develop, and Deploy—this framework is quite simple to follow and implement. Each of these three phases contains process groupings that break the work down into manageable, auditable chunks. The Design phase includes problem identification, data collection, and data wrangling. The Develop phase includes model building and model evaluation. The Deploy phase includes production deployment and monitoring. For each of these steps, ISO/IEC 42001 requires specific controls that must be implemented and documented.

The value of following a structured life cycle framework is that it makes the work of AI development visible, repeatable, and governable. When AI projects are managed ad hoc, without a clear framework, it is difficult to know whether the right controls have been applied, whether risks have been adequately addressed, and whether the resulting system is safe, fair, and effective. A structured life cycle framework provides a common language and a shared set of expectations that enable teams across functions—data scientists, engineers, domain experts, compliance officers, and business leaders—to work together effectively. It also provides the scaffolding for the documentation that auditors will review during the certification process.

Chapter 9.3: The Design Phase – Process Grouping 1, Identifying the Problem and Framing the Question

The first phase of the AI project life cycle is the Design phase. Three steps in the design phase need to be followed: identify the problem the AI project needs to solve, gather the data needed for the AI project, and clean and prepare the data. The first step is to identify the problem that your AI project needs to solve. A thorough discussion must take place between the project owner and the AI team to understand the nature of the problem, the pain points, and the issue the machine learning project is trying to solve. In this stage, the focus is on understanding the business problem and the scope of the project.

During the process of problem definition, you will find yourself overwhelmed with ideas that may compete to solve the problem. A careful selection of the idea that fits better the definition of the problem is critical to developing a good definition of the problem. This is where leadership and domain expertise matter enormously. Data scientists may be tempted to pursue technically interesting problems that have little business value. Domain experts may push for solutions that are not technically feasible. The project owner must balance these perspectives and ensure that the problem selected is both important to the business and tractable with available technology and data.

You will not be able to define the goal and select the problem statement unless you have analyzed the context of the organization. Context analysis of the organization and the problem requires a good knowledge of the industry, the organization, the competition, and how the AI project is supposed to add value. Given the complexity of the context in which your organization operates, a thorough understanding of the context entails an understanding of the following: the mission of the organization, how the AI project adds value, the nature of the relationship the organization has with its wider network of stakeholders, the competitive landscape of the organization, and the market forces impacting the organization.

A literature review also helps you understand how other organizations solved the same problem or a problem similar to yours. The benefit of the literature review is that it provides you with a scientific perspective on how and why you need that particular technology to solve that particular problem. It also helps you understand how the research community feels about the type of machine learning technology you plan to apply. After you have defined the problem, analyzed the context, and reviewed the major research on the subject, it is time to frame the question. The framing of the question is critical because this is what will guide engineers and data scientists throughout the AI project life cycle. The framing of the question becomes a roadmap that helps the team prioritize issues and make decisions as they progress. In the framing, you can state the question, define the problem in clear and simple language, define the desired outcome, set initial KPIs and milestones, and identify the resources needed.

Chapter 9.4: ISO/IEC 42001 Requirements for Process Grouping 1 – Responsible AI and Trustworthy AI

In the design phase, you need to document how you intend to identify and develop responsible AI objectives that apply to your AIMS and how you plan to integrate them. This is a critical requirement because it spells out your philosophy about responsible AI and how you intend to implement it throughout the AIMS. Responsible AI is not something that can be bolted on at the end of a project; it must be designed in from the beginning. This means that the design phase must include explicit consideration of ethical principles such as fairness, non-discrimination, transparency, explainability, and human oversight. It also means that the design phase must include processes for identifying and mitigating potential harms to individuals, groups, and society.

You also need to explain and document how you intend to develop and design processes that enhance the trustworthiness of your AIMS. In other words, explain how and by what means you will trust your processes to develop an AI project that is trustworthy. Both of these requirements are listed in the Annex A of the standard (A.6.1.2 and A.6.1.3). Trustworthiness is not just about the technical performance of the model; it is about the entire system of processes, controls, and governance that surround the model. A model that performs well in the lab but is deployed without proper monitoring, without human oversight, and without clear accountability mechanisms is not trustworthy, no matter how accurate it may be. The design phase is where you establish the foundations for trustworthiness by defining the processes and controls that will govern the AI system throughout its life cycle.

To meet these requirements, organizations should consider developing a Responsible AI Design Checklist that is used for every AI project. This checklist should include questions such as: What are the potential harms of this AI system, and who might be affected? How will we ensure that the training data is representative and free from bias? How will we test the model for fairness and robustness? How will we explain the model's decisions to affected stakeholders? How will we monitor the model's performance over time and respond to incidents? What mechanisms will we put in place for affected individuals to challenge or appeal decisions made by the AI system? By answering these questions during the design phase, organizations can ensure that responsible AI is not an afterthought but a foundational element of their AI projects.

Chapter 9.5: The Design Phase – Process Grouping 2, Data Collection and Data Wrangling

The second process grouping within the design phase is a critical stage. This is where the technical work starts and where the issues that impact the performance of your model start to seep into the project. Investing more time and resources to get this step right will save many hurdles and will help your project avoid all future problems. As we say, it is the data that will make or break your AI project. The first set of questions that you need to discuss concerns data collection methods, data sources, and the size of the data. How do you collect data? There are many methods like surveys, experiments, focus groups, polling, and so on. Where does the data come from? Is it primarily owned by your organization or public? And how did you get it? Is it primary data, data that has been collected by you directly? Or secondary data, data that has been collected by someone else and made available to you? How much of it you have and how much of it you need to run your project?

Once you have collected the data, you move to the next step, which is data wrangling. Here you start looking at the specifics of the data quality. Data wrangling involves the process by which data scientists clean and transform the raw data to avoid issues such as missing records, extreme values and outliers, formatting issues, and, more importantly, bias. This is a relatively technical step that requires data scientists who understand well the issues that surround data collection and data preparation methods. Data wrangling involves many steps. Cleaning the raw data that may come in different formats. Cleaning the data to fix errors and remove inconsistencies and outliers. Data conversion: once you clean the data you are ready to transform it into a suitable model.

It is important to mention that for this stage, the ISO/IEC 42001 requires many controls. It is critical that you look at the standard and Annex A to make sure that the methodology for data collection and data wrangling meets the standard requirements. How do you intend to use the data to enhance your AIMS? (Standard Annex A.7.2). How do you get the data, and how do you select your datasets? (Standard Annex A.7.3). How do you control the quality of the data, and what processes are used to make sure that the data meets your quality assurance requirements? (Standard Annex A.7.4). Where does the data come from? (Standard Annex A.7.5). What is the process by which you prepare the data to build the model? (Standard Annex A.7.6). The identification of data sources is a requirement and is part of the standard controls. ISO/IEC 42001 requires that you identify the data sources of the AI project: what it is, how you got it, how you plan to curate it, and how large it is (see Annex A.4.3).

The reason the standard places such emphasis on data is that data quality is the foundation of AI quality. A model trained on biased, unrepresentative, or low-quality data will produce biased, unreliable, and harmful outputs, no matter how sophisticated the algorithm. By requiring organizations to document their data sources, their data collection methods, and their data quality controls, the standard ensures that data-related risks are identified and managed early in the project life cycle, when they are easiest and cheapest to address. Organizations that skimp on data quality will pay the price later in the form of poor model performance, compliance violations, reputational damage, and loss of trust.

Chapter 9.6: The Development Phase – Process Grouping 3, Building and Evaluating the Model

Once you have developed a certain level of confidence in your data quality and checked that it meets your quality standard requirements, as well as the ISO/IEC 42001:2023 requirements, it is time to move to the next step according to the US GSA Guide. In this stage, your goal is to build the model, test it, and experiment with it iteratively so you can fine-tune the model as you go. There are two steps in this model development phase: building the model and evaluating it.

In the model building step, the focus is to define the features of the model, identify the parameters that best represent the model, work with the domain experts to make sure that the model captures well the issues you are trying to solve, use benchmarks for evaluation of the initial model, and start your testing, training, and retraining based on the results you get so you can fine-tune the model. Keep in mind that no model achieves results the first time, so retraining the model is a must-step to improve the model. Model building is an iterative process. Data scientists and machine learning engineers will try many different architectures, hyperparameters, and feature sets before arriving at a model that meets the performance requirements. This process requires patience, experimentation, and a willingness to learn from failure.

The evaluation step is critical as it allows you to test and evaluate the best model that meets your requirements. Testing is done usually on new data and your goal is to monitor how and in what way the model behaves with new data. The test and evaluation also allow you to identify the risks and the issues that your model may raise. Evaluation should not be limited to technical performance metrics such as accuracy, precision, and recall. It should also include assessments of fairness, robustness, explainability, and security. A model that is 99 percent accurate but discriminates against a protected group is not a good model. A model that performs well on test data but fails catastrophically when deployed in the real world is not a good model. Evaluation must be comprehensive and must include both quantitative and qualitative methods.

The ISO/IEC 42001 Standard requires that the tooling resources (programming languages, libraries, software kits, etc.) of the AI project must be documented. So, make sure that all the tools you use to build and develop the model are included in the documented information (see Annex A.4.4). Documenting tooling resources is important for several reasons. It enables reproducibility, allowing other teams to understand how the model was built and to replicate the results if necessary. It supports auditability, allowing auditors to verify that appropriate tools and methods were used. And it facilitates knowledge transfer, ensuring that the organization's AI capabilities are not dependent on a single individual or team.

Chapter 9.7: The Deployment Phase – Process Grouping 4, Monitoring the Model and KPIs

Once your model achieves the acceptable performance level you set initially, you need to move to the deployment step. This step is also called the production step. In the deployment step, the model should be able to take new data that was not part of the initial training set. Decide whether you want to deploy the project on-premises, in the cloud, or as a combination of both. It is also important to start monitoring the issues of cost, scalability, security, and accessibility and whether you are aligned with the initial estimate of the project.

In this stage, careful monitoring of the model behavior is important as the model's behavior may change over time or drift from its initial performance goals. One of the best practices is to set up an alert mechanism to start tracking performance issues. It is also critical that you keep retraining the model to help the model improve over time. Model drift is a common phenomenon in deployed AI systems. As the world changes, the data that the model encounters in production may differ from the data it was trained on. A model that was accurate when it was deployed may become less accurate over time as the underlying patterns change. Monitoring for drift and retraining the model when drift is detected is essential for maintaining the model's performance and reliability.

In this stage, it is also critical that you pay attention to the initial KPIs you set and whether the model meets those KPIs. If the model is not meeting your initial KPIs, then you need to either take a second look at the model or reassess your KPIs. The standard ISO/IEC 42001 requires that systems and computing resources involved in the AI project should be part of the documented information, so make sure you are documenting these resources as you go so you don't have to redo it during the certification process. Finally, it is important to mention that Annex A of the standard lists many requirements that must be met during the management of the AI life cycle. We suggest that you check the list of controls in the annex to decide which ones apply to your organization, and which you can substitute with your own controls. Most of these requirements are documentation of the processes of the AI project life cycle, so make sure you keep all the records as you progress from one step to another.

Deployment is not the end of the AI project life cycle; it is the beginning of the operational phase, which may last for years. During this phase, the organization must continue to monitor the model, respond to incidents, update the model as needed, and ensure that it remains compliant with laws, regulations, and ethical standards. The operational phase requires clear roles and responsibilities, well-defined processes, and a culture of continuous improvement. It also requires ongoing engagement with stakeholders, including users, regulators, and affected communities, to ensure that the AI system continues to serve its intended purpose without causing harm.

Chapter 9.8: An Ethical Perspective on Data Management and the Data Scientist Occupation

The role of data scientists in ensuring that the AI system is free of bias and discrimination is crucial. As a new field, data science is at the intersection of many fields of study such as statistics, math, and computer science. It has a powerful impact on people's lives, so data science must have the ethical tools that allow practitioners to understand how their system may affect people's lives. The Institute and Faculties of Actuaries (IFoA) and the Royal Statistics Society (RSS) in the UK developed an excellent guide that can serve as a framework for data scientists and help you address ethical questions as you go through the Design-Develop-Deploy model of data management. The guide raises some interesting questions and challenges facing engineers, managers, and scientists working in the field of data.

To help data scientists tackle these issues and challenges, the guide provides a framework made of five principles. These principles should guide data scientists throughout the whole data design, development, and deployment model. The first principle is to seek to enhance the value of data science. Because data in today's world has a huge impact on people's lives, data scientists need to embrace an ethical framework and a mindset that helps them think of fairness and well-being as guiding principles when they use data. Data scientists, and those involved more generally in the process of data management, need to embrace a "public good" philosophy to the issues of economic empowerment, employment, education, environment, equity and inclusion, fairness, health, and hunger. To enhance the value of data for society, the guide provides the following steps for data scientists: ask how the model impacts stakeholders, in what ways, and how the issues of employment, education, and environment, for instance, can be impacted by the type of data you use to train your algorithm models; identify the stakeholders who will be impacted by the model, their social and economic status, and whether the data truly is representative of their concerns; ask if the algorithm and the data that fuels it can create potential harm; ask about the types of bias and error your data may have and how this bias may impact stakeholders.

The second principle is to avoid harm. The role of data in creating harm is real. Issues of privacy, personal information, damage to reputation, and long-term psychological effects on people and communities are real and have all been well documented. To avoid or minimize harm, the guide suggests the following: identify the sources of data—where does the data come from, how has it been collected, and how do you intend to use it? Have you obtained the legal consent to use data, how, and in what ways? For instance, do subjects know about how their data is being used? Have you identified some best practices to handle the data with care? How to secure, store, use, and share personal information throughout the data project life cycle? These questions are not merely technical; they are ethical. They force data scientists to confront the human consequences of their work and to take responsibility for ensuring that their work does not cause harm.

The third principle is applying and monitoring professional competence. The role of data scientists and professionals involved in the handling of data, in general, is critical. The competence of the team, its skills, and its commitment to the highest ethical standard are critical factors in producing good and fair algorithms. As a professional involved in data manipulation, the guide suggests that you invest time to understand the legal framework, the ethical principles as well as the complexity and the nuances that surround the issues your algorithm model is tackling; convey that understanding to decision-makers and ensure that those in charge of the final approval know the risks associated with different steps of the data manipulation; and apply the correct statistical method and the correct algorithm model to address the issues at stake, while still monitoring the model performance and results.

The fourth principle is to seek to preserve or increase trustworthiness. The issue of trustworthiness in data is a critical challenge. People will not engage with your AI applications if these applications are not deemed trustworthy. The public backlash, when it happens against a product or service, impacts directly the issue of trust. A good practice, the guide recommends, is to work with the communities that will be impacted by the data you use to create trust and enhance the trustworthiness of the system. The guide suggests the following: engage and communicate with ethical and regulatory bodies to explain how you collect the data, what your data model does, and how you come up with a model; practice simplicity in communication and explanation; use data models that are simpler for the public to understand and be proactive in explaining how they may impact them; practice transparency in building your model. People disengage with opaque systems and engage with transparent ones because they understand how they work and how they impact them.

The fifth principle is to maintain accountability and oversight. The fifth principle in the guide of the Royal Statistics Society and the Institute and Faculty of Actuaries is to ensure human oversight and accountability by showing who is in charge of the model, and how those in charge of the model can be held accountable if something goes wrong. Following are some of the suggestions the guide provides: establish a governance model to plan, decide, review, and approve different stages of the data collection steps; establish and activate the role of the Ethics Committee in checking different data management stages; and engage and communicate with the public about how your oversight and governance system works. Finally, the guide provides a checklist that you can use in the different steps of your data management process—design, development, and deployment. You can find more information about the guides and ways of applying them at www.rss.org.uk and www.actuaries.org.uk.

Chapter 9.9: Integrating Operational Controls into Everyday Work

Operational control is not just about meeting the requirements of the standard; it is about integrating responsible AI practices into the everyday work of designing, developing, and deploying AI systems. The controls required by ISO/IEC 42001 should not be seen as bureaucratic hurdles to be cleared, but as practical safeguards that help teams build better, safer, and more trustworthy AI systems. When teams internalize these controls—when they become habits rather than checkboxes—the quality of AI systems improves, the risk of harm decreases, and the organization's ability to innovate responsibly increases.

Integrating operational controls requires leadership commitment, clear roles and responsibilities, adequate resources, and a culture that values quality and ethics as much as speed and efficiency. It requires training and awareness programs that help employees understand why the controls matter and how to apply them in their daily work. It requires tools and templates that make it easy to document decisions, track risks, and verify compliance. And it requires regular review and improvement, so that the control system evolves as the technology, the regulatory environment, and the organization's needs change.

Organizations that successfully integrate operational controls into their AI work will be able to demonstrate to auditors, regulators, customers, and other stakeholders that they are serious about responsible AI. They will be able to innovate with confidence, knowing that their AI systems are built on a foundation of quality, safety, and ethics. And they will be able to respond more effectively to incidents and crises because they have the processes, documentation, and culture in place to detect problems early, respond quickly, and learn from experience. Operational control is not glamorous, but it is essential. It is where the promise of AI governance becomes a reality.

Chapter 9.10: Chapter Summary & Key Takeaways

Clause 8 (Operation) is about operational control—ensuring that the processes and procedures of the AIMS function as intended and are monitored, measured, and documented.
ISO/IEC 42001 requires three steps for operational control: define required controls, establish monitoring and corrective action systems, and document the planning and control process.
The AI Project Life Cycle framework used in this chapter is the US GSA Guide: Design, Develop, and Deploy.
The Design phase (Process Groupings 1 and 2) involves problem identification, context analysis, literature review, question framing, data collection, and data wrangling.
Responsible AI and trustworthy AI requirements must be documented in the design phase, showing how ethical principles are integrated into the AI project.
Data quality controls in the design phase include documenting data use, data source selection, quality control processes, data origin, and data preparation processes.
The Develop phase (Process Grouping 3) involves model building and model evaluation, with attention to features, parameters, domain expert input, benchmarks, and iterative testing and retraining.
Model evaluation should include not just technical performance metrics but also fairness, robustness, explainability, and security assessments.
Tooling resources must be documented in the development phase, supporting reproducibility, auditability, and knowledge transfer.
The Deploy phase (Process Grouping 4) involves production deployment, monitoring model behavior, monitoring KPIs, and retraining as needed.
Model drift is a common phenomenon in deployed AI systems, requiring ongoing monitoring and retraining to maintain performance and reliability.
Systems and computing resources involved in the AI project must be documented as part of the operational controls.
The Institute and Faculty of Actuaries and Royal Statistics Society provide five ethical principles for data science: enhance value, avoid harm, apply and monitor professional competence, preserve or increase trustworthiness, and maintain accountability and oversight.
Operational controls should be integrated into everyday work, not treated as bureaucratic hurdles, to build better, safer, and more trustworthy AI systems.
Successful integration of operational controls requires leadership commitment, clear roles, adequate resources, training, tools, and a culture of continuous improvement.

Chapter Notes

Mark this chapter as completed

Chapter 10 of 11

Chapter 10: The Mirror of Governance – Measuring, Auditing, and Evaluating What Your AI System Actually Does

Chapter 10.1: Introduction – Why Performance Evaluation Is the Conscience of the AIMS

The success of any management system depends a great deal on the evaluation process of its performance. Management systems, complicated as they are, as they tackle many interrelated parts, need to be assessed and evaluated frequently to identify their shortcomings, spell out their weaknesses, and design strategies to improve them. Performance evaluation is critical because it allows leadership to realign resources, objectives, and missions to achieve the goals. Without rigorous evaluation, an organization cannot know whether its AI management system is working as intended, whether it is delivering value, or whether it is creating risks that have not been identified and mitigated.

But for the AIMS, performance evaluation takes on a new meaning in the sense that not only does it provide an overall assessment of the system, but also it helps audit the algorithm and the data that fuels the AIMS. Algorithm audit, as a relatively new and emerging field, helps management address issues of bias and discrimination, as well as the question of whether the model is behaving according to the goals set by the organization. This is why when we address the question of AIMS evaluation performance, we need to address two essential, and separate, questions. First, whether, overall, the system works in a coherent way creating synergies that maximize the collective potential of the different parts involved in it. Second, whether the algorithm and the data that fuels it are compliant with the laws and regulations, as well as the ethical framework of the company.

These are two different, and separate, assessments that both need to take place in the organization. While the first type of assessment can be conducted by assessors familiar with ISO management system frameworks, like ISO 9001 lead auditors, the second type of assessment is relatively complex and requires a technical background. There is a new standard under development, as of the writing of this book, ISO/IEC CD 42006 Requirements for Bodies Providing Audit and Certification of Artificial Intelligence Management Systems, that will define the audit and certification process as well as the skill set required of auditors. This new underdevelopment standard adds specific conformity assessment requirements to bodies providing audit and certification (ISO/IEC 17021-1:2015 Conformity Assessment Requirements for Bodies Providing Audit and Certification of Management Systems). In this chapter, we will address both questions to show you the general process of AIMS evaluation, beginning with the standard requirements and then moving into the practical frameworks and tools that organizations can use to measure, audit, and evaluate their AI systems.

Chapter 10.2: The Three Requirements of AIMS Performance Evaluation

To evaluate the performance of your AIMS, ISO/IEC 42001 requires three important steps. The first step is to design a framework to assess, monitor, and measure the performance of the AIMS. This framework should define what will be measured, how it will be measured, who will conduct the measurement, and how the results will be used. It should be aligned with the organization's strategic objectives and with the requirements of the standard. The second step is to set up an audit program to assess whether the AIMS complies with your own organization's requirements, as well as with the requirements of the standard ISO/IEC 42001. The audit program should be systematic, documented, and conducted by competent auditors who have the skills and independence to provide an objective assessment. The third step is to review the evaluation and assessment output to decide about ways and means of improving the AIMS. This review should be conducted by management and should result in decisions about resource allocation, process improvement, and corrective action.

The purpose of an evaluation system is to generate enough data that provide evidence of whether the system works as it was intended. The objective is to make evidence-based decisions to improve or change the system. This is especially true for complex systems such as AIMS, where the evaluation requires a technical review, as well as a sociotechnical evaluation that addresses the quality of the governance model. So, when deciding about the evaluation, you need to think in a more comprehensive way to address three questions. What do I need to evaluate? This is the scope of the evaluation. How do I evaluate? This is the tools and methods of evaluation. Who will conduct the evaluation? This is the skills of people engaged in the evaluation.

The scope of the performance evaluation depends on the size of the organization and the specific sector in which you operate, as well as prior evaluation of the system and the AI maturity level of the organization. According to the standard ISO/IEC 42001, the organization decides what to evaluate, the whole system or only part of it. Because ISO/IEC 42001 is a new standard, it is recommended that you conduct a full evaluation. You may want to start with the technical side of the AIMS, like data, model training, and algorithm behavior, while another parallel team works on the governance issue, such as leadership, AI policy, AI strategy, context analysis, and risk management of the AIMS. Breaking down the assessment process in this way helps you not to overwhelm the assessment process while allowing the organization to digest the assessment results and make the necessary changes before certification.

Chapter 10.3: The Assessment Criteria, Metrics, and KPIs for AI Governance

Once you identify the scope of the evaluation, you need to address the question of assessment criteria, metrics, and KPIs. There are two steps to the evaluation and measurement of an AIMS: the evaluation and measurement of the AI governance system as a whole, and the evaluation and measurement of the machine learning system. These two steps are linked, and they need both to be measured and evaluated.

The first step is to evaluate the governance of the AIMS. Here you can use many criteria. Peter Soendergaard, from the Soendergaard Group, suggests five important KPIs. The first KPI is data: measuring the quality of data and the process by which it is handled is an important KPI to start with in the evaluation process. The second KPI is security: how is the data handled, and how is the AI system protected from cyberattacks, such as evasion attacks, poison attacks, and extraction attacks, as well as other forms of cyberattacks? The third KPI is cost value: according to Soendergaard, this is a critical measure of the AI governance system that helps you identify the real value of the investment and its return on investment. The fourth KPI is bias: identifying and measuring bias in the data and the training model is critical as it helps you address legal and ethical issues. The fifth KPI is accountability: this KPI helps you identify the people using the system and deciding about contribution to the system. A sixth KPI is audit: auditing the algorithm frequently to measure its effectiveness and efficiency, as well as identify ethical issues such as bias and discrimination.

To assess the overall governance of the AIMS, you can use a series of structured questions. How and to what extent is leadership engaged in the AIMS? Are all the documentations required by the standard in place? Examples of documentation could be AI policy, AI frameworks, AI goals and objectives. Are the relevant policies that relate, either directly or indirectly, to the AIMS in place? For instance, HR and talent policies, procurement policy, and other policies that support the overall AI governance system. Have we identified and defined the roles and responsibilities of different actors involved in AIMS? Are the governing practices that relate to AIMS well understood and well defined? Do people understand them in a way that allows them to accomplish their mission? Do governing bodies understand the risk of AI systems and are they aware of AI applications being developed? Do leaders understand the AI risk? Have they been briefed on the type of AI risks we may run into? Have they cleared the impact assessment study? These questions provide a comprehensive framework for evaluating the governance of the AIMS.

The second step in the evaluation of the ML program is the evaluation of the ML model itself. Here are some of the ML metrics you can use, as suggested by Jerald Murphy. Mean Squared Error (MSE): a technique that identifies outlier results. Mean Time to Repair (MTTR): the time it takes to fix things from first contact. This is a well-known KPI in IT. First Contact Resolution Rate (FCRR): a classical IT measurement technique that helps you identify the issues that get resolved by support service without further investigation. Error analysis: analyze the prediction errors in the ML model to investigate the root cause. Data and concept drifting: a common ML error, this KPI helps you keep the ML model on target by monitoring data distribution and changes in data input patterns. These metrics provide a technical complement to the governance KPIs, ensuring that both the sociotechnical and the algorithmic dimensions of the AIMS are evaluated.

Chapter 10.4: The Large Language Models Audit – A Three-Layer Approach

The democratization of foundation models through large language models and the many applications that have become a world sensation recently brought to the conversation table a critical question about the audit of these models. If your AI system is part of a large foundation model ecosystem, using applications produced by another provider, then the question of algorithm transparency and algorithm audit is a critical question to address. While ISO/IEC 42001 does not address the issue of large language models per se, it is critical for your AIMS compliance and long-term performance and credibility to set up a framework that helps you address the issue of LLM audit. If you are developing, or using applications built on the top of LLM, then you might be part of the larger ecosystem of LLM and therefore it is critical to address issues of compliance.

A good framework for LLM audit is the framework suggested by authors Mökander, Schuett, Kirk, and Floridi in 2023. This framework provides a three-layer approach to LLM audit. The first layer is the audit of the governance structure of the technology provider. According to the authors, the audit framework should focus first on the technology provider's governance framework. In this layer of audit, the assessment addresses issues of the process by which LLM gets produced, and the quality by which technology providers manage the production of LLM. Questions to ask include: What management systems are involved in the LLM production and deployment? What incentive procedures are in place? How does the technology provider account for issues of risk? What is the overall quality management process of LLM development? How does the LLM provider document the process of LLM capabilities?

The second pillar of the LLM auditing process, according to Mökander and colleagues, is the audit of the model itself. Here the audit looks at the following issues. The limitation of the model: what are some of the limitations that the model can and cannot do? This is a crucial question for developers of API, so they are aware of how the model functions. Information about the capacity of the model: what is the capability of the model? This is also a critical question for companies building applications on top of LLM as it helps them solve design issues. Performance and robustness of the model: how does the model behave under different prompts and circumstances? Security and vulnerability issues of the model: how does the model handle adversarial attacks and to what extent is the data used to train the model protected?

The third component of the audit framework is to audit the application built on top of the model itself. The audit of the application should focus on compliance with the regulatory environment and the ethical framework of the developer's company. Applications audit, according to Mökander and colleagues, should address two critical questions. Is the intended use of the application legal, ethical, and subscribing to the ethical principles of the large language model's provider? What is the impact of the application on users, communities, and society? Finally, it is important to mention that the most critical contribution of this framework is that it addresses the critical issue of how and in what way LLM should be audited. The framework also provides a comprehensive approach that links the entire supply chain of large language model design and deployment: technology providers and application developers.

Chapter 10.5: Setting Up an Internal Audit Program – Goals and Considerations

The second step in the performance evaluation of the AIMS required by ISO/IEC 42001 is to set up an internal audit program. The purpose of the audit program is to check if the AIMS complies with the requirements of the organization and the ISO/IEC standard 42001. For the AIMS audit to comply with the standard, it has to achieve three goals. First, achieve conformity of the AIMS with the organizational AI requirements. Second, meet the requirements set by the current standard ISO/IEC 42001. Third, ensure that the AIMS is effectively implemented and maintained.

For audit programs to work effectively they need to take into consideration the organization's context. Factors such as business environment, legal context, size, the economic sector, and previous audits all need to be accounted for. In large organizations with a history of IT legacy, for instance, the audit program may need to be more thorough. The goal is to identify and spell out the practices that may impede the effective functioning of the AIMS and may create more risks for the organization. An audit program that is not tailored to the organization's context may miss important risks or waste resources on issues that are not material.

There are many internal audit programs that you can use, in addition to the framework suggested by the ISO/IEC 42001, depending on the context of your organization and how it fits that specific framework. A recent framework that received a lot of attention lately is the AI Risk Management Framework published by the National Institute of Standards and Technology in 2023 (NIST AI RMF 1.0). This framework is a comprehensive one and addresses many relevant aspects, such as effectiveness, trustworthiness, and responsible AI criteria. The NIST AI RMF provides tools that can add practical value to the auditing process. The NIST RMF is made of four major steps: Govern, Map, Manage, and Measure.

In the Govern step, the audit needs to focus on how the audit program addresses the regulatory environment of the AIMS. This is a challenge for the auditors as the legal requirements and regulations are shifting constantly. In this step, auditors need to take into consideration specific new and emerging regulations that address AI specifically, but also existing laws and regulations that address health and safety, discrimination, and consumer protection. In the Mapping step, the goal of the auditor is to spell out and identify the links that connect to different parts of the AIMS. In other words, here you map out how and in what way your AIMS affects the different components that it interacts with internally and externally. This is a challenging step. As a general-purpose technology, AIMS interacts with many parts of the organization and is impacted by many parts of the organization. So, an effort must be made to identify how the AIMS gets affected by the organization context, the culture, and the vision of the leadership, and how, in turn, the AIMS impacts the context and the wider society.

According to the NIST RMF, the measurement is a critical component of the audit program. Measuring the risks surrounding the AIMS, like errors and adverse impacts, helps the audit program build consistency and credibility. In this step, a focus on not only the competence and skills but also the objectivity and the independence of the audit program is critical to the success of the overall audit program. In the Manage step, the organization needs to spell out how it intends to measure the risks identified. You can focus on the following actions: How do the metrics identified fit the overall context of the organization and the audit process? How does the organization intend to prioritize the risks and how it intends to treat them? How do you document the way risks have been identified and treated? Together, these four steps provide a rigorous and comprehensive approach to internal audit that complements the requirements of ISO/IEC 42001.

Chapter 10.6: Algorithmic Audit – Inspecting the Inner Workings of the Model

In recent years, a new concept emerged in the audit literature. The concept of "algorithmic audit" is a new concept that refers to the process of inspecting how algorithm models are built and trained and how they behave. Algorithms, which are a set of instructions built on a set of data, may contain errors that may affect their behavior and their integrity, and may make them behave in a biased and discriminatory way. So, checking, inspecting, and testing how the algorithm was built and the type of data it was trained on has emerged in recent years as a critical issue in the conversation about overall AI systems.

A study conducted by A. Hasan, S. Brown, J. Davidovic, B. Lange, and M. Regan in 2022 on algorithmic bias and risk assessment provides a good framework for how to audit algorithms. First, the authors differentiate between two related concepts: ethical risk assessment of AI systems in general, and a narrower more technical assessment that focuses on algorithmic bias assessment. While in the first type of assessment, the purpose is to look at the sociotechnical environment in which the AI is being deployed, in the algorithmic assessment, the goal is to inspect the inner functioning of the algorithm itself.

In the ethical assessment, you need to first identify key stakeholders and the potential harm that may impact them. In this step, the authors suggest that you start with broad research to identify the possible harm that the algorithm being studied may have on people. This is a generic identification that does not necessarily involve the algorithm being studied. The purpose of this is simply to be aware of the possible and probable harm that may occur. In the second step, according to Hasan and colleagues, you focus on two important elements: data collection methods, how data was processed, managed, and labeled; and the testing metrics that were defined before the building of the ML model. These two elements are critical because they determine whether the algorithm will produce fair and accurate results or whether it will perpetuate and amplify existing biases.

The importance of algorithmic audit cannot be overstated. In a world where AI systems are increasingly making decisions that affect people's lives—whether they get a loan, a job, or parole—the fairness and accuracy of those systems are matters of justice. Algorithmic audits provide a mechanism for holding AI systems accountable, for detecting bias before it causes harm, and for ensuring that the promises of AI—efficiency, objectivity, and scale—are realized without sacrificing equity, fairness, and human dignity. Organizations that invest in algorithmic audit capabilities will be better positioned to build AI systems that are not only effective but also ethical and trustworthy.

Chapter 10.7: Management Review – The Leadership Stamp of Legitimacy

Once the audit is completed, management needs to review the results of the audit and act upon it. The goal of management review is to provide the audit program with a stamp of legitimacy. When management approves and signs off on an audit program, the program becomes an official program of the organization and a roadmap for future improvement. Management review is also critical because it shows how leadership engages with the AIMS. The leadership tone, guidance, and influence are exerted through the review of the AI program audit. Without management review, the audit program may be seen as a technical exercise with no real consequences, and the findings may be ignored or downplayed.

For an AIMS review to comply with ISO/IEC 42001, the review must include several key elements. First, the results of the previous review and audit. This is an important input that allows management to compare previous and current audits to identify gaps and improvements. Second, changes that have taken place since the last audit cycle and how the organization responded and adjusted to them. Changes that relate not only to laws and regulations but also to the economic and business environment must be spelled out so management can comprehend the improvements that are recommended. In this context, a must-do action is to address the regulation changes and how the new public interest and scrutiny are addressed by the AIMS.

Third, how stakeholders' needs and expectations have changed since the last time the audit was conducted, and in what way these changes have impacted the effectiveness of the AIMS. A good practice here is to map out all AIMS stakeholders and show how AIMS has impacted them since the last time the audit was completed. Fourth, what opportunities for improvement can the organization address based on the measurement and the nonconformity issues the audit has identified? This would be your future improvement roadmap. Finally, the entire audit results must be documented. This is a requirement of ISO/IEC 42001:2023. Documentation ensures that the audit findings are preserved, that decisions are traceable, and that future audits can build on the work of previous ones. It also provides evidence to certification bodies that the organization is serious about performance evaluation and continual improvement.

Management review is not just a compliance requirement; it is an opportunity for leadership to demonstrate its commitment to responsible AI, to hold the organization accountable for its AI systems, and to set the direction for future improvement. When leadership engages substantively with audit findings—asking tough questions, challenging assumptions, and making concrete commitments to action—it sends a powerful message throughout the organization that AI governance matters. When leadership treats management review as a rubber-stamping exercise, it undermines the entire AIMS and signals that AI governance is not a priority. The choice is stark, and its consequences are significant.

Chapter 10.8: The Continuous Nature of Performance Evaluation

Performance evaluation is not a one-time event. It is a continuous process that should be embedded in the organization's management rhythm. The AI landscape is changing rapidly, and the risks and opportunities associated with AI systems are constantly evolving. An evaluation that was valid six months ago may be outdated today. New regulations may have been enacted, new technologies may have emerged, new stakeholders may have raised concerns, and new incidents may have occurred. Organizations must therefore establish a rhythm of regular evaluation that keeps pace with the pace of change.

A continuous approach to performance evaluation includes several key practices. Regular monitoring of AI system performance against established KPIs, with automated alerts for anomalies or degradation. Periodic internal audits, at least annually, to assess compliance with the standard and the organization's own requirements. Management reviews, conducted at planned intervals, to review audit findings and make decisions about improvement. And continuous feedback mechanisms, including channels for employees, customers, and other stakeholders to report concerns or incidents. Together, these practices create a system of ongoing vigilance that helps the organization detect problems early, respond quickly, and learn from experience.

The benefits of continuous performance evaluation are substantial. It enables the organization to identify and address risks before they cause harm. It supports compliance with evolving legal and regulatory requirements. It provides evidence to stakeholders that the organization is serious about responsible AI. And it creates a culture of learning and improvement that drives innovation and excellence. Organizations that treat performance evaluation as a continuous process, rather than a periodic compliance exercise, will be better positioned to build AI systems that are not only effective but also ethical, trustworthy, and sustainable over the long term.

Chapter 10.9: Chapter Summary & Key Takeaways

Performance evaluation is the conscience of the AIMS, providing evidence of whether the system is working as intended, delivering value, and managing risk.
ISO/IEC 42001 requires three steps for performance evaluation: design a framework to assess, monitor, and measure; set up an internal audit program; and review evaluation output to improve the AIMS.
AIMS performance evaluation has two dimensions: the governance (sociotechnical) assessment and the technical (algorithmic) audit.
Governance KPIs include data quality, security, cost value, bias, accountability, and audit frequency.
Machine learning metrics include Mean Squared Error, Mean Time to Repair, First Contact Resolution Rate, error analysis, and data and concept drifting.
The Large Language Model audit framework has three layers: governance structure of the technology provider, model audit, and application audit.
Internal audit programs should be tailored to the organization's context, considering business environment, legal context, size, sector, and previous audits.
The NIST AI Risk Management Framework provides four steps for audit: Govern, Map, Measure, and Manage.
Algorithmic audit is a new field that inspects how algorithms are built and trained, and whether they behave in a biased or discriminatory way.
Algorithmic bias assessment distinguishes between ethical risk assessment (sociotechnical) and technical assessment (algorithmic bias).
Management review provides the audit program with legitimacy and ensures that leadership engages substantively with audit findings.
Management review must include previous results, changes since last audit, stakeholder changes, improvement opportunities, and documentation.
Performance evaluation should be continuous, with regular monitoring, periodic audits, management reviews, and feedback mechanisms.
Continuous evaluation enables early detection of risks, compliance with evolving regulations, stakeholder confidence, and a culture of learning and improvement.

Chapter Notes

Mark this chapter as completed

Chapter 11 of 11

Chapter 11: The Never-Ending Journey – Embedding Continuous Improvement into the DNA of Your AI Management System

Chapter 11.1: Introduction – Why Improvement Is Not an Afterthought but a Way of Being

Section 10 of the standard ISO/IEC 42001 addresses the issues of improvement, deviation, nonconformity, and continual improvement of the AIMS. At the end of the performance evaluation, and as a result of the management review, you should have a relatively good idea about the weaknesses and the strengths of your AIMS. The question that then arises is what you do with that knowledge. Do you file the audit report away in a drawer, satisfied that you have met the compliance requirement? Or do you use it as fuel for a continuous process of learning, adaptation, and improvement? The answer to that question determines whether your AIMS is a static artifact that will gradually become obsolete, or a living system that grows stronger and more effective over time. In the age of AI, where the technology, the regulatory environment, and the expectations of society are all evolving at a rapid pace, the ability to improve continuously is not a nice-to-have; it is a survival requirement.

To help you understand how to build a continual improvement plan to improve your AIMS and meet the standard requirement, we will adopt a PDCA approach to answer three critical required questions. How do you identify and measure the improvement needed to enhance the strength of AIMS? This is the Plan and Do phase. How do you implement the improvement you identified to address the shortcomings of the AIMS? This is the Check phase. What lessons have you learned that may help you correct the deviations and the nonconformity of your AIMS? This is the Act phase. But before we get into the details of the PDCA and the continual improvement plan, let us go over a few terminologies used in Section 10 of the standard ISO/IEC 42001.

Chapter 11.2: Understanding Deviation and Nonconformity – The Two Faces of Underperformance

The improvement section of the standard uses two critical terminologies that every AI governance professional must understand: deviation and nonconformity. These terms are often used interchangeably in casual conversation, but in the context of ISO management systems, they have distinct meanings and require distinct responses.

Deviation is an undesirable result or a gap between the actual performance and the initial target performance of the system. For your AIMS, many deviations may happen during the AI project life cycle, so it is critical to have gating questions and KPIs at the end of each stage of the AI life cycle. A deviation is not necessarily a failure of the system; it is a signal that the system is not performing as expected and that something needs to be investigated. For example, if a model that was expected to achieve 95 percent accuracy on a test set achieves only 88 percent, that is a deviation. If a data collection process that was supposed to yield 10,000 annotated samples yields only 6,000, that is a deviation. Deviations can be caused by many factors, including data quality issues, model architecture choices, process inefficiencies, or external changes in the environment.

Nonconformity, on the other hand, is simply the nonfulfillment of a required action, such as not having a documented AI policy, or not being able to define and document the roles and responsibilities of different actors engaged with the AIMS. A nonconformity is a failure to meet a specific requirement of the standard, the organization's own policies, or applicable laws and regulations. Nonconformities are more serious than deviations because they represent a breach of an obligation, whether that obligation is internal or external. They must be addressed through corrective action, and they must be documented and reported as part of the audit process.

When there is a deviation or a nonconformity, there are three types of actions you need to take according to the standard. First, simply accept the consequences of the nonconformity and learn from it while at the same time taking action to correct it. Second, understand why the deviation happened in the first place by examining and analyzing the circumstances that produced it. Third, update or make changes to the AIMS based on your understanding of the shortcoming. Of these three suggestions, the second one is the most important recommendation as it helps you conduct a root-cause analysis to understand the source of the deviation, where it comes from during the AI project life cycle, and how and why the organization missed it in the first place. It also helps you identify future actions that you may need to take to correct the shortcomings.

Root-cause analysis is a disciplined process of asking "why" repeatedly until you arrive at the fundamental cause of a problem. If a model is producing biased outputs, you might ask why. Because it was trained on biased data. Why was it trained on biased data? Because the data collection process did not include sufficient representation from certain groups. Why did the data collection process not include sufficient representation? Because the requirements for data representativeness were not clearly defined or enforced. Why were the requirements not clearly defined? Because the organization lacked a formal data governance framework. By continuing this line of questioning, you arrive at a root cause that, if addressed, will prevent similar problems from occurring in the future. This is far more valuable than simply correcting the symptom of the biased model without addressing the underlying cause.

Chapter 11.3: The Corrective and Preventive Actions Framework – The Immune System of the Organization

Corrective and Preventive Actions (CAPA) is a framework that helps managers structure efforts to conduct a root-cause analysis of the deviation and take actions to understand what could trigger the deviation or nonconformity in the future. Professor Nathan Conover, from the consulting firm Pathways, defines CAPA as the "immune system of the organization." CAPA helps you identify the risk before it happens and disseminate a culture of prevention among team members. Just as the immune system protects the body from disease by identifying and neutralizing threats, CAPA protects the organization from operational failures by identifying and addressing their causes.

Corrective actions help you identify deviation and nonconformity in the AIMS in two key ways. First, when there is a deviation, CAPA allows you to initiate corrective actions, which should allow you to take immediate actions to eliminate the problem. For instance, if the data quality does not meet your definition and has been identified as a deviation, then you need to reconsider the source of your data. You might need to find new data sources, improve your data collection processes, or invest in better data annotation. The corrective action addresses the immediate problem and restores the system to a state of compliance or acceptable performance.

Second, corrective actions help you eliminate the cause of the deviation or the cause of the nonconformity permanently. In the root-cause analysis, you should have gathered data on what happened, how it happened, why it happened, and what should be done to avoid the problem in the future. Generally speaking, the root-cause analysis addresses three types of problems: human error, such as an employee not following the protocol; a technical issue, such as how the model is being trained; or a process error, such as how data is wrangled. Each of these types of problems requires a different type of corrective action. Human errors may require additional training or clearer procedures. Technical issues may require changes to the model architecture or training process. Process errors may require redesigning the workflow or adding checkpoints and controls.

Once you correct the immediate deviation, the preventive actions come into play. In preventive actions, you are putting in place a system that helps you predict when a problem is going to happen, and how you could prevent it. It is like putting in place a trigger and an alarm system that sends signals when a problem is about to occur. Preventive actions are proactive rather than reactive. They are based on the understanding that problems rarely occur in isolation and that the conditions that produced one problem are likely to produce others if left unaddressed. By identifying and addressing those conditions, organizations can prevent future problems from occurring, saving time, money, and reputation.

Of critical importance to the CAPA methodology and the improvement system in general is documentation. Documenting the source of the problem, how it was identified, and how it was solved is critical to preventing the same problem from occurring in the future. Documentation also provides evidence to auditors and regulators that the organization has a functioning improvement system and that it takes its obligations seriously. Without documentation, the organization may find itself repeating the same mistakes, addressing the same problems over and over again without ever learning from experience. Documentation transforms individual learning into organizational learning, ensuring that the knowledge gained from addressing one problem is available to prevent or solve similar problems in the future.

Chapter 11.4: Continual Improvement and the PDCA Approach – The Engine of Organizational Learning

One of the most well-known frameworks in the improvement projects field is the Deming Cycle of Improvement: Plan-Do-Check-Act, commonly known as PDCA. The PDCA is a highly interactive framework. It treats improvement, not as one event, but rather as a continuous effort that spans over the entire lifecycle of the AI project. PDCA is also a great tool because it helps link different teams engaged with the AI project in one framework. The PDCA framework is a flexible and simple framework that you can apply to many processes of the business.

The first step of the PDCA cycle is planning. The planning of the improvement project requires many actions, the chief of which is to define the business problem and to get an agreement from all team members engaged in the AI project on what constitutes a problem. This is not an easy task as a typical AI project involves many moving parts, with each part being a mini project by itself. So, getting people to agree on the improvement plan is critical. Following are some of the questions that you need to address in this step. First, define the project business goal. In this stage, you need to have both technical teams and business teams define the business problem. A problem described by a team of data scientists may be viewed differently by a team working on customer experience. The goal is to get an agreement from everybody on the pain points and the issues the AI project is expected to address. Second, clarify the objective of the AI improvement project. In this stage, it is critical that you get an agreement on the objectives of the improvement project you intend to implement. Listing all the AI project improvement requirements, goals, and objectives would help you create a roadmap for different teams involved in the project improvement. Third, create metrics and milestones. It is important that you define the metrics you will use to assess the AI project improvement and how these metrics will be met so everyone on the team understands the deliverables. Fourth, identify the type of resources available. Identify the resources needed for the AI project. Think of financial resources, like the budget allocated to the AI project, physical resources, like computing power and the networking infrastructure, and human resources, the engineers and data scientists that the project will need.

The second step of the PDCA model is the implementation, or Do phase. Here you are implementing the different actions you planned, such as data collection—how data is collected, processed, and managed overall; how the model is working versus how it is supposed to work; the risks you have identified and how far you were in your prediction of risks; and testing and experimenting to make sure that the ML project is achieving the results planned. The Do phase is where plans meet reality. It is where assumptions are tested, where unexpected challenges emerge, and where the team learns what works and what does not. It is essential that the Do phase is conducted in a spirit of experimentation and learning, rather than blame and punishment. If team members are afraid to report problems or failures, the organization will lose the opportunity to learn from them.

The third step of the PDCA model is the Check phase. In this step, the goal is to verify and check that your AIMS is working as it is intended to. This is the step in which you will be testing and monitoring the AIMS processes. You can proceed in two steps. First, check the governance of the AIMS, or the sociotechnical requirements, such as leadership, the policy, the risk management, and how the overall AIMS is performing. Second, focus on the behavior of the algorithm, and whether the technical side of the AIMS is producing the results hoped for. You need to make sure that your AI algorithm is meeting the testing and measurement requirements that you have identified at the beginning of the PDCA cycle.

Following are sample questions that you may ask when testing the sociotechnical dimension of your AIMS. Do we have an AI policy, and is it updated and documented? Are the measurements we set to monitor the alignment and the coherence of the AIMS working as they were intended to, and what KPIs have been used to measure that? Are the metrics we set to assess the impact reliable and yielding results we can rely on? Then you move to the testing of the algorithm behavior itself. This is the technical analysis of your AIMS. Ask the following questions. Is the trained AI model performance working according to the metrics we have identified? Can we validate the model's accuracy against other relevant metrics to make sure that the algorithm is working as it is supposed to? What issues has the testing identified that we can act upon?

The fourth and final step in the PDCA framework is the Act phase. The Act step refers to the feedback and improvement actions that you plan to take to correct nonconformities and improve the system. Depending on the outcome of the previous step, here you are working on improving the AIMS based on the results of the checking and monitoring you did previously. Here also you can proceed in two ways. Focus on the governance issues, then tackle the technical aspect of data and algorithms. The Act phase is where learning is institutionalized. It is where the insights gained from the Check phase are translated into changes in policies, processes, training, or technology. It is where the organization demonstrates that it is capable of learning from experience and improving its performance over time. Without the Act phase, the PDCA cycle is incomplete, and the organization will continue to repeat the same mistakes.

Chapter 11.5: Applying PDCA to the AI Project Life Cycle

The PDCA framework is not just an abstract management concept; it can be applied directly to the AI project life cycle. In the Plan phase, you collect the data, make sure the model is working, list the risks, and test and experiment. In the Do phase, you implement the data collection and model building processes, monitor how the model is working versus how it is supposed to work, and identify risks and how far you were in your prediction of them. In the Check phase, you monitor the AI ecosystem, check if the measurements are capturing the essence of the AI system, and verify if the trained AI model performance is working according to the metrics. In the Act phase, you work on addressing the issues identified in the previous step, updating the model, the data, the processes, or the governance framework as needed.

Applying PDCA to the AI project life cycle ensures that improvement is not a one-time event at the end of a project, but an ongoing process that is integrated into every stage of the project. It ensures that lessons learned during development are fed back into design, that insights from deployment are used to improve training, and that the entire system becomes more effective and more trustworthy over time. It also ensures that the organization is prepared to respond to changes in the technology, the regulatory environment, and the expectations of stakeholders, rather than being caught off guard by them.

One of the key benefits of applying PDCA to the AI project life cycle is that it creates a culture of continuous learning. When teams are regularly reflecting on what is working and what is not, when they are encouraged to experiment and learn from failure, and when their insights are systematically captured and acted upon, the organization becomes more agile, more innovative, and more resilient. This culture of learning is essential in the age of AI, where the pace of change is rapid and the complexity of the challenges is high. Organizations that can learn faster than their competitors will have a significant advantage, not just in terms of AI performance, but in terms of their ability to navigate the ethical, legal, and social dimensions of AI governance.

Chapter 11.6: Conclusion – The Age of AI Demands a New Paradigm of Leadership and Management

The age of AI has ushered in a new era that requires a paradigm shift in the way we lead and manage change. The acceleration of change requires that we learn, adapt, adjust, and develop new mental frameworks faster than we used to. As a learning tool, standards help spread the word and establish the foundation of good practices to mature and reproduce success at scale. This is what ISO/IEC 42001 will do for AI systems, and this is why organizations should adopt the 42001 framework as fast as possible, so they can build the foundations of a dynamically shifting technology.

In this book, we have emphasized four key ideas. First, the transformation we are going through, while it seems to be led by technology, is affecting all facets of the economy. Organizations that are still stuck in the old paradigm of managing technology-led change as an IT issue will struggle and eventually fall behind. The type of transformation we live in today, although it is called digital transformation, requires a comprehensive approach that moves many things at the same time—from setting up a business strategy to creating a data-quality framework to managing AI projects, talent mobility, and employee growth. The faster, holistic, and agile approach is what is needed for the age of AI.

Second, in discussing the different clauses of the standard, the goal was to show you how to implement the standard and put together the foundation of a viable AI management system that can respond to the changing environment in which it lives. While the standard clauses are straightforward and can be understood by most compliance officers and quality managers familiar with ISO standardization procedures, showing how to implement the clauses, how to create the documentation needed, and how to streamline the system's processes and procedures required by the standard is one of the added values of this book.

Third, a key idea of the standard is around AI risk. This is why you will find the terms that relate to risk, such as risk assessment, risk treatment, impact assessment, and overall risk management, cited frequently in the standard. We have devoted an entire chapter to these concepts, in addition to explaining them when addressing AI technology and generative AI. The goal was to help you understand the intricacies of risk and how it connects with other enterprise risks.

Fourth, and to give the reader a more comprehensive view of how this standard can promote an emerging technology and mature its practices, we went beyond ISO/IEC 42001. While the bulk of the book is about AI management systems, as explained by ISO/IEC 42001, we also addressed other standards that form the overall ecosystem of AI technology and that have been jointly developed by ISO and IEC.

The journey of implementing an AI management system is not easy. It requires commitment, resources, patience, and a willingness to confront difficult questions about the ethical, legal, and social implications of AI. But it is also a journey that is rich with opportunity. Organizations that embrace this journey, that invest in building the capabilities described in this book, and that commit to continuous improvement will be well positioned to harness the power of AI for good—to create value for their stakeholders, to contribute to the well-being of society, and to build a future in which AI serves humanity rather than the other way around. The age of AI is here. The question is not whether your organization will be transformed, but whether you will lead that transformation with wisdom, courage, and responsibility. The choices we make today will resonate for generations to come.

Chapter 11.7: Chapter Summary & Key Takeaways

Improvement (Clause 10) is not an afterthought but a way of being, essential for keeping the AIMS effective in a rapidly changing environment.
Deviation is a gap between actual and target performance; nonconformity is the nonfulfillment of a required action.
When deviations or nonconformities occur, organizations must accept consequences, conduct root-cause analysis, and update the AIMS accordingly.
Root-cause analysis involves asking "why" repeatedly until the fundamental cause is identified and addressed.
Corrective and Preventive Actions (CAPA) is the immune system of the organization, identifying risks before they happen and fostering a culture of prevention.
Corrective actions address immediate problems and eliminate their causes; preventive actions predict and prevent future problems.
Documentation is critical to CAPA and improvement, transforming individual learning into organizational learning.
The PDCA (Plan-Do-Check-Act) cycle is a continuous improvement framework applicable to the entire AI project life cycle.
Plan involves defining the problem, clarifying objectives, creating metrics, and identifying resources.
Do involves implementing planned actions, collecting data, and testing and experimenting.
Check involves verifying governance and algorithm behavior against metrics.
Act involves addressing issues identified and institutionalizing learning.
PDCA applied to the AI project life cycle ensures improvement is integrated into every stage, not treated as a one-time event.
The age of AI demands a paradigm shift in leadership and management, requiring comprehensive, holistic, and agile approaches.
ISO/IEC 42001 provides a framework for building viable AI management systems that can respond to a changing environment.
AI risk management—including risk assessment, risk treatment, and impact assessment—is a central concern of the standard.
The book addresses other standards in the AI ecosystem beyond ISO/IEC 42001, providing a comprehensive view of AI governance.
The journey of implementing an AI management system requires commitment, resources, patience, and willingness to confront difficult questions.
Organizations that embrace continuous improvement will be positioned to harness AI for good and build a future where AI serves humanity.

Chapter Notes

Mark this chapter as completed