“Governance frameworks should address the prospect of AI systems that cannot be safely tested.“

Michael Cohen, a CHAI Research Fellow, led a distinguished group of researchers from renowned institutes including the University of California, Berkeley, Center for Human-Compatible Artificial Intelligence, the University of Toronto, Schwartz Reisman Institute for Technology and Society, Université de Montréal, Mila–Quebec AI Institute, Schwartz Reisman Institute for Technology and Society, and Vector Institute and others to reimagine this vision. Their mission is to lay the foundational principles that will ensure AI evolves in a manner that is safe, ethical, and aligned with human interests.

Central to their concerns are Long-Term Planning Agents (LTPAs), advanced AI systems capable of strategic thought and action over extended periods. The authors suggested that “Developers should
not be permitted to build sufficiently capable LTPAs, and the resources required to build them should be subject to stringent controls.”

The conundrum of testing AI systems for compliance and ethical behavior poses a unique dilemma. How do we ascertain whether an AI system might attempt to bypass human control without inadvertently providing it the opportunity to do so during testing?

This leads us to the pivotal question: How can we ensure the responsible development and deployment of such AI systems? The authors advocate for a proactive regulatory approach, establishing guidelines and safeguards before potential issues arise. They envision a role for governments in actively overseeing AI development, ensuring all AI systems operate within defined ethical boundaries.

As we stand on the cusp of an AI-driven era, the call to action is unequivocal: it is imperative for all stakeholders — developers, legislators, and the general public — to engage in a meaningful dialogue about the direction of AI development. Let’s ensure that this journey towards integrating AI into our society is marked by foresight, ethical considerations, and a commitment to enhancing human well-being.

We invite you to delve into this crucial conversation. Share your insights, disseminate knowledge, and collaboratively steer the course of AI development towards a future that benefits humanity.

Navigate here to access the original article on Science.

Navigate here to engage in an on-going conversation with Michael Cohen on Twitter (aka X).