Deepseek Coder: What is it and how it works

How a language model is created Deepseek? How are you doing against competition?

Well, let’s start with the definition Deepseek Coder: DeepSeek-Coder-V2-V2 is an open source code of the mixture (MOE), which reaches productivity comparable to GPT4-Turbo yield in specific code tasks.

In particular, Depseek-Coder-V2 previously studies from the intermediate control point of DeepSeek-V2 with 6 billion additional tokens. Due to this continuous previous training, Depseek-Coder-V2 is largely improved by the coding and mathematical reasoning of Deepseek-V2, while maintaining comparable performance in the general tasks of the language.

Deepseek Coder includes a series of models of code language, trained from scratch with 87 % code and 13 % of the natural language in English and Chinese, and each model is pre -prepared in 2T tokens. We provide several sizes of the code model, from versions 1b to 33b.

“Each model is previously trained in the code case at the repository level, using the size of the 16K window and the additional task of filling in the spaces, which leads to fundamental models (depeek-cell-bas). We will also configure the basic model with 2 billion data tokens of instructions on instructions to get adjusted models of instructions called Depseek-Coder-In-INSTRUCT, ”they say in DeepseekField

• Previously trained 2 billion Tokens in more than 80 programming languages.
• Different sizes of the model (1.3bIN 5.7bIN 6.7b And 33b) to satisfy various requirements.
• Window size 16kwhich allows the completion and filling out Project levelField
• The last performance Generation between open source models.
• Open and free source for research and commercial useField

On its website, Github Depseek claims that “If you want to use DeepSeek-Coder-V2 in BF16 format for the output, 80 GB*8 is required.”

DEPSeek Coder performance

In standard reference assessments, and, according to them, Depseek-Coder-V2 reaches higher performance compared to closed code models, such as GPT4-TURBO, CLAUDE 3 OPUS and Gemini 1.5 Pro in comparative coding and mathematics tests:

“Deepseek-Coder-V2 demonstrates significant achievements in various aspects of the tasks related to the code, as well as in general reasoning and capabilities. In addition, Depseek-Coder-V2 expands its compatibility with programming languages ​​from 86 to 338, while expanding the length of the context from 16K to 128K, ”says the Chinese company.

Here is the code in Github of Deepseek

Here is the code in Github of Deepseek