Language revolution in scientific research:

Scientific progress faces a major obstacle of excess information. The explosive growth of scientific materials and data has made it increasingly difficult to discover useful visions in the midst of a vast amount of information. Scientific knowledge is reached today through research engines, but is unable to regulate scientific knowledge alone.

In this article, we address Galactica: a large language model capable of storing, integrating and producing scientific knowledge. Galactica is trained in a vast scientific collection of papers, references, knowledge bases and many other sources.

Galactica outweighs current models in a range of scientific tasks. For example, in technical knowledge tests such as Latex equations, Galactica outweighs the latest GPT-3 issue by 68.2% against 49.0%. Galactica also offers good performance in the conclusion, with Chinchilla superior to the MMLU sports test of 41.3 per cent versus 35.7 per cent, and PaLM 540B in the MATH test of 20.4 per cent compared to 8.8 per cent. It is also achieving the best result so far on sub-functions such as PubMedQA and MedMCQA dev by 77.6% and 52.9% respectively. Although not trained in a public group, Galactica outnumbers BLOOM and OPT-175B in the BIG-bench test.

We believe that these findings illustrate the potential of language models as a new front for science. Therefore, we have made the model open to the scientific community.

But besides these advantages, there are challenges facing Galactica. The upgraded company removed the public illustrative offer because of the tendency of large language models such as Galactica to generate inaccurate and unreliable outputs, even with training in high-quality scientific and academic data. Public understanding of the strength and weakness of these language models is essential for their better development.

However, Galactica is an important step in scientific research, and we are on the threshold of a linguistic revolution that will help scientists to explore and derive knowledge in ways that have never been possible before.