DeepSeek wants to increase the openness and accessibility of one of the fundamental elements of its AI models for other developers.
The open-source community will receive technical insights regarding the Chinese AI startup’s internal inference engine. One of the numerous steps in creating a large language model (LLM) is inferencing. New data that demonstrates the patterns the model has discovered depending on its parameters is produced by the trained AI model.
The training and deployment of DeepSeek’s AI models has been accelerated, according to the company’s internal inference engine and training framework. The startup’s inference engine is a modified version of vLLM, an open-source library for LLM inferencing created by academics at UC Berkeley in the United States, even though its training framework is based on the PyTorch platform.
We wish to contribute as much as we can to the community, especially in light of the increasing need for implementing models like DeepSeek-V3 and DeepSeek-R1. A note from a DeepSeek researcher on Hugging Face, an online repository for open-source AI models, reads, “We are extremely appreciative of the open-source ecosystem, without which our progress toward AGI [artificial general intelligence] would not be possible.”
The business is not, however, making its underlying inference engine completely accessible and open-source. Rather, DeepSeek stated that it will share with current open-source projects the design enhancements it made to the vLLM inference engine and information regarding its implementation. Additionally, it pledged to extract beneficial elements and make them available to the open-source community as stand-alone, reusable libraries.
DeepSeek noted several obstacles to making its inference engine completely open-source, including a highly customized codebase, limited maintenance bandwidth, and infrastructure limitations. As part of its “open-source week” campaign, DeepSeek released code repositories and other parts of its AI models as open-source in February of this year.
Beyond its affordability and computational effectiveness, DeepSeek’s innovation was praised by tech CEOs and AI researchers for being open-source. Its models, however, do not meet the Open Source Initiative’s (OSI) generally recognized definition of an open-source AI system. The training methodology, training code, and data required to train its flagship R1 model have not been made available under the permissive MIT license.
