Chinese scientists claimed to have developed the world’s first “brain-like” artificial intelligence large language model similar to ChatGPT, designed to consume less power and work without Nvidia chips.
The AI model, named SpikingBrain1.0, mimics the way the human brain fires only the nerve cells it needs, according to researchers from the Chinese Academy of Sciences.
Large language models widely in use, like ChatGPT and Meta’s Llama, are dependent on the scaling law, which implies that a reasoning AI system’s performance improves with more data and larger models.
They work by applying a technique called “attention”, in which the AI model looks at all the words in a sentence at once to figure out which ones are most important for predicting the next word.
For instance, if one were to provide ChatGPT a prompt: “The cat went under the bed after seeing the stranger because it was scared”, the AI can process the words at once to understand that the “it” refers to the cat.
In this way, AI models compare every word to every other word, and if the prompt is very long, like an entire book, it could consume considerably more energy and slow down.
These models suffer from several hurdles due to the large number of words they process, including “extremely high training costs, high energy consumption, and complex deployment pipelines”, scientists explain in a yet-to-be peer-reviewed study posted in arXiv.
-in-Shanghai.jpeg)
Most commonly used AI models are also built only to run on Nvidia GPUs, limiting who can develop them.
“In addition, building large models on non-Nvidia computing platforms poses major challenges,” scientists wrote.
To overcome these limitations, researchers claim to have developed a new model that “draws inspiration from brain mechanisms”.
Instead of looking at the entire text, scientists claim their new model only looks closely at nearby words, like how the human brain focuses on recent context in conversations.
Such a setup can balance efficiency and accuracy, researchers say.
By using this method of attention, SpikingBrain can be 25 to 100 times faster than normal AI models, they say.
The new AI model also runs on China’s homegrown MetaX chip platform instead of using Nvidia chips, according to the study.
In recent years, since the first Donald Trump presidency, the US has ramped up its technology export restrictions to China, cutting off the Asian giant’s access to tools used to make chips for computer servers, artificial intelligence, and other advanced applications.
One company whose Chinese exports are particularly affected is the California chipmaker Nvidia, which is a world leader in designing specialised chips for AI technology.
Following these disruptions, China began rapidly developing its own homegrown AI ecosystem.
Instead of relying on an entire network like OpenAI’s ChatGPT, the new AI model selectively responds to input, reducing its power consumption, scientists say.
“This enables continual pre-training with less than 2 per cent of the data while achieving performance comparable to mainstream open-source models,” they wrote.
They claim SpikingBrain “achieves more than 100 times” the speed of traditional models in some cases.
“Overall, this work demonstrates the potential of brain-inspired mechanisms to drive the next generation of efficient and scalable large model design,” scientists wrote.
Source: https://www.independent.co.uk/