Security or competitiveness? Governance or the technological race? The discussions sparked by Dario Amodei’s call to slow down the development of artificial intelligence, the reactions from Silicon Valley and the response of Donald Trump, who believes that slowing the AI race would mean favouring China, have brought back to the centre of the debate an issue that is set to shape the coming years: how to govern an increasingly powerful technology without giving up innovation.
This question also intersects with the recent appeal by Pope Leo XIV, who, in a message published on X, stressed the need to distinguish “human art from what machines produce” and to safeguard “our humanity” in the age of artificial intelligence.
Reflecting on these challenges is Francesca Rossi, IBM Fellow and Global Leader for Responsible AI and AI Governance. Rossi works at IBM’s T.J. Watson Research Lab in New York and has been a leading figure in the global AI debate for years. She was a member of the European Commission’s High-Level Expert Group on Artificial Intelligence, has led some of the field’s major scientific organisations and continues to work with institutions, governments and international bodies on defining rules and standards for the responsible development of technology.
In this conversation with Formiche.net, Rossi explains why security and competitiveness are not conflicting objectives, which AI risks are currently being underestimated and why, in her view, the real competitive advantage in the coming years will not derive from the most powerful models, but from the ability to build reliable and trustworthy systems.
Q: IBM has long maintained that the central issue is not stopping innovation, but governing it responsibly. What does “responsible AI” mean in practical terms today?
A: Safety and competitiveness are not opposing forces that need to be balanced: in the long run, competitiveness depends on safety. Powerful but unreliable or insecure AI will either not be adopted or will be adopted poorly; incidents, economic losses, and distrust ultimately slow innovation. The useful question, therefore, is not whether to move fast or slowly, but how to move quickly in the right direction. This means investing in safety research and system evaluationwith the same determination devoted to improving capabilities. Those that can provide systems trusted by citizens, businesses, and institutions will gain a genuine and lasting competitive advantage.
- It means moving from principles to practice. There is now broad consensus on the principles: fairness, transparency, explainability, robustness, privacy protection, and human oversight. The difficult part is translating them into day-to-day processes throughout a system’s lifecycle.
- This includes risk assessment before deployment, monitoring afterward, technical tools to measure bias and robustness, clear roles and responsibilities, and internal bodies capable of making difficult decisions. At IBM, we have been doing this for years through a dedicated board and governance shared across all business units. Today, with the arrival of AI agents, responsibility takes on a new form, because risk no longer depends solely on the application domain, but also on what we delegate to a system, under what conditions, and how quickly we can regain control if something goes wrong.
Q: Does the race between the United States and China risk making it more difficult to develop shared international rules and standards for artificial intelligence?
A: Dialogue is certainly complex, but it is not impossible. There are very concrete shared interests: no country wants systems that slip out of control, are used for large-scale cyberattacks, or destabilize the information environment. There is room for convergence on these issues, particularly at the technical level.
- I am thinking of international standards, evaluation and testing methods, the sharing of safety practices, and the work of organizations such as the OECD and the United Nations. Technical standards have often proved capable of bridgingpolitical divides because they address shared practical needs. The contribution of companies, the scientific community, and civil society is essential to keeping these channels open even when dialogue between governments becomes more difficult.
Q: Much of the debate focuses on the future risks of AI. What risks are already present today but tend to be underestimated by governments, companies, and citizens?
A: The first concerns discrimination: systems that learn from data can reproduce and amplify existing inequalities in areas such as employment, credit, and healthcare, often in ways that are difficult to detect. The second concernsinformation: the problem is not only disinformation itself, but the erosion of trust in our ability to establish what is true. The third—and perhaps most underestimated—concerns people: relying on machines excessively or uncritically can, over time, weaken our capacity for judgment, learning, and decision-making.
- The risk is that we end up with tools that think for us rather than help us think better. I would also add workforce preparedness: there is a great deal of discussion about how many jobs AI will transform, but far lessa bout how to train people and redesign roles.
Q: Autonomous drones, cyberattacks, disinformation: is artificial intelligence changing the very concept of security?
A: AI is a dual-use technology: the same capabilities that help defend an IT system or identify a disinformation campaign can be used to attack that system or generate disinformation. Security therefore becomes a matter of speed and adaptability. Moreover, AI agents—capable of acting autonomously, using tools, and interacting with other systems—create risks that traditional cybersecurity practices do not cover.
- This is because these systems may exhibit emergent behavior that was not anticipated by their designers and must be assessed atthe level of the entire system rather than its individual components. It is therefore essential to monitor an AI system while it is in use and to ensure that decisions with serious and irreversible consequences remain subject to genuine and meaningful human control.
Q: You have worked for years on AI ethics. How can we prevent the pursuit of maximum technological efficiency from pushing human responsibility into the background?
A: First, by recognizing that ethical principles alone are not enough. Decisionsabout how AI is developed and used result from the interaction of three levels: values, governance, and incentives. If incentives reward speed or cost reductionalone, even the best intentions may give way. We therefore need governance that makes responsibility an integral part of decision-making, rather than an after-the-fact check.
- This should be seen as an advantage, not as a burden that slows innovation, because the responsible development and use of AI create value for everyone, including the company investing in governance. Then thereis a design question. We can build systems that replace human judgment, or systems that strengthen it—systems that know when to pause and reflect, thatsignal their own uncertainty, and that ask us questions instead of always offeringready-made answers. In my research, inspired by Daniel Kahneman’s work on “fast” and “slow” thinking, we focus precisely on architectures capable of consciously alternating between these two modes of decision-making.
Q: Pope Leo often stresses the need to keep the human person at the center of technological development. Can this principle be translated into concrete criteria for designing and governing AI systems?
A: Yes, and to some extent this has already been done. The Ethics Guidelines for Trustworthy AI, developed by the European Commission’s High-Level Expert Group of which I was a member, place human autonomy and oversight first. The Rome Call for AI Ethics, which IBM has supported from the outset, also emerged from a dialogue among religious communities, institutions, and businesses on these issues. In addition, one of IBM’s principles for responsibleAI states that AI should augment human intelligence, not replace it.
- In practical terms, putting people at the center means several things: ensuring that significant decisions can be understood and challenged; providing for human intervention, especially in irreversible decisions or those with a major impact on people; and evaluating a system not only in terms of the accuracy of its results, but also its effect on the capabilities and autonomy of those who use it. It also means investing in AI literacy, because informed people are able to place the right level of trust in systems—neither too much nor too little.
Q: Looking ahead to the next five years, what will matter more: increasingly powerful models or increasingly reliable and trustworthy systems?
A: Reliability, without question. Model capabilities will continue to grow, but the realenabler will be trust: trust determines whether and how a technology is adoptedin the contexts that truly matter, such as healthcare, finance, public administration, and the workplace. Reliability and trust, however, are not the same thing. Reliability is a property of systems, built through research and engineering.
- Trust is a relationship, earned through governance—within organizations, in interactions between people and machines, and through rules and standards. I believe the coming years will reward not so much those with the largest and most powerful model, but those able to build an ecosystemaround AI that people can trust in an informed way.
