A GPU SIMT compiled within the weights of a language model

qwenGPU is a technical demonstration where Vixhal (@TheVixhal) inserted a SIMT (Single Instruction Multiple Thread) graphics processor directly into the weights of the Qwen3.5-2B model. Unlike traditional approaches where the GPU is an external hardware component, here, the first 45 layers of the model are compiled into CUDA code (NVIDIA PTX), executable during forward passes without requiring additional training. According to MarkTechPost, these layers use existing Qwen3.5-2B architectures: Gated DeltaNet networks to simulate GPU memory and registers, and SwiGLU MLPs for computations. The result is a model capable of generating functional CUDA code, as demonstrated by the exact reproduction of 49 CUDA kernels on an NVIDIA Tesla T4 during testing.

qwenGPU compiles a SIMT GPU directly into the weights of Qwen3.5-2B, executing real CUDA during forward passes without additional training.

Technical operation: real-time GPU emulation

The project relies on a hybrid architecture where the original Qwen3.5-2B model (24 layers) is preceded by 45 compiled layers, totaling 69 layers. These additional layers simulate a GPU with precise hardware features: warps with 4 lanes, a 4-stage pipeline, 15 registers per lane, and 126 words of global memory, all managed via DeltaNet recurrent states. As Vixhal explains on X, the model writes each step of CUDA launch, including registers, branches, and memory writes, token by token. In advanced mode, qwenGPU generates correct CUDA kernels on the first attempt in 4 out of 5 cases, a performance highlighted by daily.dev.

Compatibility and limitations: an academic demonstration for now

qwenGPU is designed as a proof of concept, not as an operational runtime. It works across various environments: Apple silicon (with a speed of 10 to 15 tokens/second on MPS), NVIDIA GPUs, or CPUs, provided 16 GB of free memory is available. The textual divergence from the original Qwen3.5-2B model is minimal (7e-4 in KL divergence), according to MarkTechPost. However, the project has limitations: no quantized version is available, and the 2B model remains unreliable for critical applications. To use it, a local build is required via the command just build, with dependencies such as Python 3.11+, uv, and clang for the NVPTX backend.

What this changes here: toward more integrated AI-hardware architectures in West Africa

For Beninese or West African businesses and administrations using critical infrastructures (data centers, banking or healthcare systems), qwenGPU opens up opportunities for hardware optimization of language models. Such integration could reduce dependence on expensive external GPUs by embedding part of the acceleration logic directly into the model, provided sufficient resources are available (minimum 16 GB of RAM). Local stakeholders will, however, need to assess the project’s maturity, which remains experimental, and its alignment with their performance and reliability requirements. Public institutions may be interested in such architectures for applications like automatic translation or local data analysis, where latency and energy costs are key criteria.