CoreWeave, a cloud provider built for GPU-accelerated workloads, today announced the release of the largest publicly available language model in partnership with EleutherAI, a grassroots collective of researchers working to open source AI research. The model – GPT-NeoX-20B – was trained by EleutherAI on CoreWeave’s state-of-the-art NVIDIA A100 training cluster and is set to provide businesses and researchers alike with access to build innovative products, applications, and advance scientific research.
Effective today, GPT-NeoX-20B is available on GooseAI, a fully managed inference service delivered by API, prior to a full open source release next week. With feature parity to other well known APIs, GooseAI delivers a plug-and-play solution for serving open source language models at over 70% cost savings by simply changing 2 lines of code. GooseAI is also being released today as a joint venture between CoreWeave and partner Anlatan – the creators of NovelAI.
At 20 billion parameters, GPT-NeoX-20B was trained on EleutherAI’s curated collection of datasets, The Pile. This was the same dataset used to train well-known models like Beijing Academy of Artificial Intelligence’s Wu Dao (1.75T parameters, multimodal), AI21’s Jurassic-1 (178B parameters), Anthropic’s language assistant (52B parameters), and Microsoft and NVIDIA’s Megatron-Turing NLG (340B parameters). GPT-NeoX-20B is a glimpse into the next generation of what open sourced AI systems could look like. EleutherAI hopes to remove the current barriers to research on the understanding and safety of such powerful models.
Founded in 2017, CoreWeave offers scalable, on-demand computing resources for Machine Learning and Artificial Intelligence use cases. Its infrastructure, scale across the broadest selection of NVIDIA GPUs, and DevOps expertise give clients the flexibility and freedom that they need to manage complex workloads. With the release of GooseAI, CoreWeave and Anlatan are delivering a massive step forward for visionary businesses who are building products on top of large language models, while making it even easier to deploy NLP services on top of CoreWeave Cloud.
