When Do We Decide A Model Is Foundational?

This Is My Type of Twitter Drama

I originally noted this in the ADSA newsletter and I wanted to share. I highly recommend their newsletter.

Foundational Models: Is It Time?

Last month, 149 faculty, research scientists, postdocs, and students at Stanford University proposed that the field of artificial intelligence is sufficiently mature enough to identify some key foundational models (in this case, language models). They declare these models foundational, which means that everything else is model tuning.

Lengthy token abstract below:

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and…