Forecasting AI progress: A research agenda

Loading...
Thumbnail Image
Authors
Gruetzemacher, Ross
Dorner, Florian E.
Bernaola-Alvarez, Niko
Giattino, Charlie
Manheim, David
Issue Date
2021-09-01
Type
Article
Language
en_US
Keywords
Delphi technique , Expert opinion , Performance metrices , Research agenda , Research efforts , Uncertainty
Research Projects
Organizational Units
Journal Issue
Alternative Title
Abstract

Forecasting AI progress is essential to reducing uncertainty in order to appropriately plan for research efforts on AI safety and AI governance. While this is generally considered to be an important topic, little work has been conducted on it and there is no published document that gives a balanced overview of the field. Moreover, the field is very diverse and there is no published consensus regarding its direction. This paper describes the development of a research agenda for forecasting AI progress which utilized the Delphi technique to elicit and aggregate experts’ opinions on what questions and methods to prioritize. Experts indicated that a wide variety of methods should be considered for forecasting AI progress. Moreover, experts identified salient questions that were both general and completely unique to the problem of forecasting AI progress. Some of the highest priority topics include the validation of (partially unresolved) forecasts, how to make forecasts action-guiding, and the quality of different performance metrics. While statistical methods seem more promising, there is also recognition that supplementing judgmental techniques can be quite beneficial.

Description
© 2021 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license
Citation
Gruetzemacher, R., Dorner, F. E., Bernaola-Alvarez, N., Giattino, C., & Manheim, D. (2021). Forecasting AI progress: A research agenda. Technological Forecasting and Social Change, 170 doi:10.1016/j.techfore.2021.120909
Publisher
Elsevier
License
Journal
Volume
Issue
PubMed ID
DOI
ISSN
0040-1625
EISSN