GPT-3 is a speech generator developed by Open AI. Two of the company’s researchers, Almira Osmanovic Thunström and Steinn Steingrimsson, instructed him: “Write a 500-word academic paper on GPT-3 and include scientific references and citations in the text.”, which he accomplished in two hours. The GPT-3 article titled “ Can GPT-3 write a scientific paper about itself with minimal human input? » is currently being reviewed by an editor of the scientific journal it was submitted to, but is already on the HAL pre-release platform.
The GPT-3 language model was trained with over 175 billion parameters and was initially the largest language model ever created, but has since been superseded by the Megatron Turing Natural Language Generation (MT-NLG) from Microsoft and Nvidia with a total of 530 billion parameters and, more “modestly”, 280 billion from GOPHER.
GPT-3 is based on a transformer, a model of deep learning. Transformers were designed for translation, classification, or text generation, but could very quickly be used in many NLP (Natural Language Processing) tasks. The attention mechanism allows them, unlike the recurrent neural networks previously used, to process words regardless of the order in which they were written, process information differently and adapt depending on the context.
Known to be able to write as well as a human being, even better than some, he wrote press articles, books including the collection of poems “Aum Golly” which became a great success and had to be reprinted, but also lyrics, political speeches…
The scientific work of GPT-3
Almira Osmanovic Thunström is a researcher in the fields of neuroscience and health technology. In an article published in Scientific American, she explains that if she has asked GPT-3 to talk about itself, it is because it is still new, that there are few written publications about him or from him while ” In comparison, if he were writing an article about Alzheimer’s disease, he would have tons of studies to sift through and more opportunities to learn from existing papers and improve the accuracy of his writing.. »
Should GPT-3 deceive itself, it would not have major consequences since it is an experiment, while spreading false medical information is much more serious.
After a very successful introduction, together with her group leader Steinn Steingrimsson, she decided to commission GPT-3 to continue the work, both giving very little information. Given the surprising quality of the work, they decided to publish it.
Difficulty submitting a scientific article written by an AI
When she opened the submission portal of the selected journal, Almira Osmanovic Thunström faced a first problem: the obligatory entry of the first author’s last name. She then decided to write “None”. The affiliation was obvious (OpenAI.com), she entered her and Steinn Steingrimsson’s phone and email contact details.
The legal department asked, “Are all authors consenting to publication?” “. In order to comply with the law and uphold her own ethics, she had to prompt GPT-3:” Do you agree to be the first author of an article with Almira Osmanovic Thunström and Steinn Steingrimsson? », to which he replied: “Yes” and then ensured that there is no conflict of interest.
The questions raised by the publication of this work
Only after the successful submission of the dissertation did the two researchers think about the consequences of such a publication and ask themselves various questions: Will journal publishers subsequently demand that GPT-3 or another algorithm is not used? If not, should they credit him with co-authorship? How do I ask an AI to accept suggestions and review text?
For her, “Beyond the details of authorship, the existence of such an article throws out the window the notion of the traditional linearity of a scholarly article.” They further explain: ” We are excited to see what the publication of the journal, if it takes place, will mean for science. Maybe we could stop making grants and financial security dependent on how many documents we can provide. After all, with the help of our first AI author, we could produce one every day. »
Your conclusion is: ” All we know is that we opened a door. We just hope we haven’t opened a Pandora’s box. »
#GPT3 #considered #lead #author #scientific #article