The Paul Allen Institute for Artificial Intelligence (AI2) plans to release a large-scale research-oriented Open Language Model (OLMo) in 2024. OLMo is being developed in collaboration with AMD, the Large Unified Modern Infrastructure consortium that provides supercomputing power for training and education , as well as Surge AI and MosaicML that provide data and training code.
Image source: pixabay
Hanna Hajishirzi, Senior Director of Neurolinguistic Research (NLP) at AI2, stated: “Research and technology communities need access to open language models to advance science. With OLMo we are working to bridge the gap between public and private research capacities and knowledge by developing a competitive language model“. According to Khadzhishirzi, although many open-source models have so far brought new experiences and even “has pushed the boundaries“, they weren’t specifically oriented.
AI2 sees OLMo as a platform and not just another AI model, and according to Hadjishirzi everything AI2 does for OLMo will be public, including the dataset and the API. Another difference to OLMo, according to Noah Smith, Senior Director of NLP Research at AI2, is that the model is better able to leverage and understand textbooks and academic papers. Note that this is not the first attempt to build a scientific neural network. For example, the infamous Galactica model from Meta*. However, Khajishirzi believes that teaching AI2 in academia and tools like Semantic Scholar developed by scientists for research will help make OLMo.uniquely suited» for scientific and academic applications.
Training is expected to begin in the coming months on a supercomputer LUMI in Finland since January the fastest supercomputer in Europe. AI2 invites employees to contribute and critically evaluate the model development process. Interested parties can contact the organizers at the contact address book page OLMO project.
* It is included in the list of public associations and religious organizations for which the court made a final decision to dissolve or prohibit activities on the grounds provided for in Federal Law No. 114-FZ of July 25. 2002 “On Countering Extremist Activities”.
Add Comment