With the huge success of OpenAI’s ChatGPT text generator, a “generic” version was released as open source in late 2022. The new system can perform any of the tasks performed by the famous artificial intelligence chatbot.such as e-mail drafts, computer code suggestions, and academic texts.
Created by Philip Wang, the reverse engineer of Meta’s Make-A-Video rendering system, the new model is a combination of Google’s PaLM and Human Feedback Reinforcement Learning (RLHF).
But there is a problem: This combination of PaLM and RLHF is not pre-trained. This means that Wang’s AI users must compile the data collection themselves and use their personal hardware to train the model and further process requests. But we’re talking gigabytes of text here and hardware that can handle this task.
How to train artificial intelligence?
According to TechCrunchThe new text generator model works similarly to ChatGPT, but as it is a statistical tool that predicts words, it needs to be fed with lots of samples, such as posts from social networks, news texts, and various e-books.
To learn the probability of words forming, RLHF trains a language model that produces a wide variety of responses to a given human stimulus. These responses must then be sequenced by human volunteers. create a kind of “reward system” that will create a preference order.
Naturally, this is not an accessible process for most users. For example, Google’s PaLM natural language model required 540 billion language parameters. A 2020 study by AI21 Labs calculated the cost to train a model with only 1.5 billion parameters Up to US$ 1.6 million (R$ 8.8 million).
Source: Tec Mundo
