ML//model//open source model//DeepSeek

DeepSeek is a Chinese AI lab whose models V1 through V3 pushed open-weight models to frontier performance at a fraction of the cost.

DeepSeek is a Chinese AI lab whose models V1 through V3 pushed open-weight models to frontier performance at a fraction of the cost.

V3 uses a Mixture of Experts architecture with 671B parameters in total and about 37B active per token.

R1 (January 2025) matched o1-level reasoning using reinforcement learning with GRPO, with no supervised fine-tuning for the reasoning phase.

Its efficiency came from architecture as much as from hardware: the mixture of experts cuts the compute per token, and multi-head latent attention compresses what the KV cache must hold, which cuts the memory of serving long conversations.

The V3 report put its final training run at about 2.8 million H800 GPU-hours, roughly 5.6 million dollars at an assumed rental price, and said the figure left out earlier research and experiments. Read as the cost of building the model it misled many; it is the compute of one run (training cost).

The lab published full training recipes and open-sourced the weights, which forced a collapse of prices across the industry.