A framework for Evaluating Rapidly Developing Digital and Related Technologies: AI, Large Language Models and Beyond

Ahead of the AI Safety Summit 2023, the International Science Council releases a discussion paper on a framework for evaluating AI and rapidly developing digital technologies. Scientific researchers and practitioners in AI are invited to respond to the ISC's discussion paper and proposed framework on AI. 

Rapidly emerging technologies present challenging issues when it comes to their governance and potential regulation. The policy and public debates on artificial intelligence (AI) and its use have brought these issues into acute focus. While broad principles for AI have been promulgated by UNESCO, OECD and others, and there are nascent discussions regarding global or jurisdictional regulation of the technology, there is an ontological gap between the development of high-level principles and their incorporation into regulatory, policy, governance and stewardship approaches. This is where the non-governmental scientific community could have a particular role.

It has been proposed by a number of academics and policy experts that the International Science Council (ISC) – with its pluralistic membership from the social and natural sciences – establish a process to produce and maintain an annotated framework/checklist of the risks, benefits, threats and opportunities associated with rapidly moving digital technologies, including – but not limited to – AI. The purpose of the checklist would be to inform all stakeholders – including governments, trade negotiators, regulators, civil society and industry – of potential future scenarios, and would frame how they might consider the opportunities, benefits, risks and other issues.

The outputs would not act as an assessment body, but as an adaptive and evolving analytical framework which could underpin any assessment and regulatory processes that might be developed by stakeholders, including governments and the multilateral system. Any analytical framework should ideally be developed independent of governmental and industry claims, given their understandable interests. It must also be maximally pluralistic in its perspectives, thus encompassing all aspects of the technology and its implications.

This discussion paper provides the outline of an initial framework to inform the multiple global and national discussions taking place related to AI.

Depending on the response to this discussion paper, an expert working group would be formed by the ISC to further develop or amend the above analytical framework by which stakeholders might comprehensively look at any significant developments either of platforms or of use dimensions. The working group would be disciplinarily, geographically and demographically diverse, with expertise
spanning from technology assessment to public policy, from human development to sociology and futures and technology studies.

To engage with this discussion paper, please visit council.science/publications/framework-digitaltechnologies/