A new report by Stanford University has revealed that the high cost of training Artificial Intelligence (AI) models is limiting the involvement of non-industry actors in the technology revolution.
According to the Artificial Intelligence Index Report 2024, while AI companies seldom reveal the expenses involved in training their models, it is widely believed that these costs run into millions of dollars and are rising.
The report cited openAI as an instance, where the company’s CEO, Sam Altman, said in 2023 lthat the training cost for GPT-4 was over $100 million.
This revelation comes as the Nigerian government launched its Large Language Model (LLM), which is expected to position the country as AI leader in Africa. According to the Minister of Communications, Innovation and Digital Economy, Dr. Bosun Tijani, the LLM will be trained in five low-resource languages and accented English to ensure stronger language representation in existing datasets for the development of AI solutions.
To illustrate the growing cost of training AI, the researchers at Stanford cited an example of the Transformer model, which introduced the architecture that underpins virtually every modern LLM, noting that it costs only around $900 to train. They, however, estimated that OpenAI’s GPT-4 costs $78 million while Google’s Gemini Ultra costs around $191 million.
The report further noted that another challenge that may be encountered in launching into AI is data. According to the report, LLMs have been trained on meaningful percentages of all the data that has ever existed on the internet.