BF16 matrix multiply on an RTX 3090 Ti going from 88 to 157 TFLOPS is an eye-catching number. But what someone running a local LLM feels is prompt-processing time and tokens per second, and those have different bottlenecks.

The gpu-unlock repository was created Oct 18 and is changing fast (a "Release 2.0" commit landed Oct 27). The Oct 25 GeekNews attention was only the discovery path. This article relies on the README and commit history; I did not install it or run benchmarks. The calculations below are my own rough model.

What it does

Per the README, gpu-unlock is a Linux kernel module that lifts a half-rate cap on FP16/BF16/TF32 tensor math with FP32 accumulation on GeForce cards. It is not a patched driver or VBIOS flash; it runs at boot before the NVIDIA driver loads. Outputs are said to be bit-identical before and after, with only speed changing.

The briefing described it as covering some RTX 30 cards, but the current README lists Ampere (RTX 30) as stable (100+ user reports), Ada (RTX 40) and Turing (RTX 20) as experimental, and Blackwell (RTX 50) as unsupported. Support widened within days, so recheck the latest README before installing.

The conditions behind 157

The briefing carried "87.50→157.41 TFLOPS, 450 W, 8192³ GEMM, FP32 accumulation, median of five runs." The current README table I read says 88 → 157 TFLOPS (+80%) with conditions limited to dense matrix multiply, FP32 accumulation and stock power limits. It gives no GEMM size or repeat count and points to user compatibility-report issues for full details. The two-decimal values and the 8192³ condition appear in secondary coverage, but I could not find them in the current README. I treat only the rounded values as primary and the finer conditions as unverified.

I checked the arithmetic: 157/88 is +78.4%, while 87.50→157.41 is +79.9%. The README's +80% appears to come from unrounded values.

The README also lists per-card BF16 numbers: 3060 12 GB 27→51, 3090 75→125 (+65%), 4090 165→329 (+100%, peak MMA rate). For the 4090 the README itself says power runs out before tensor cores, so large GEMMs gain much less.

Why prefill speeds up and decode doesn't

LLM inference has two phases. Prefill processes the whole prompt at once and is compute-heavy; decode generates one token at a time, reading all weights from memory per token, so it is memory-bandwidth-bound. The README's workload table matches this: prefill +25–55%, time to first token 25–30% lower, decode unchanged, INT8/FP8/INT4 models unchanged, and llama.cpp/Ollama with quantized GGUF 0–15%. Those figures are aggregated user reports that I did not reproduce.

Below is the output of my script (gpu_prefill_decode.py) for this bottleneck difference. Assumptions: an 8B dense model, BF16 weights, 50% achieved efficiency; 1008 GB/s bandwidth is NVIDIA's published spec. It is a model, not a measurement.

before: prefill 4096 tok ~1.49s, decode upper bound ~63 tok/s
after: prefill 4096 tok ~0.83s, decode upper bound ~63 tok/s
GEMM share 90% -> overall 1.65x
GEMM share 50% -> overall 1.28x
GEMM share 20% -> overall 1.10x

Compute rises 1.78× and the decode ceiling stays the same. If GEMM is 20% of total time, the overall gain is 1.10×. A chatbot with short prompts and long answers will feel little; RAG that prefills long documents every time, BF16 LoRA training and image generation, where GEMM share is large, benefit more.

Before installing

  • Boot configuration changes: you add gpu_unlock.targets=all to the kernel command line and put the module in the initramfs. Procedures differ by bootloader and mistakes can cause boot trouble. To revert, remove the option from the command line.
  • Power-state dependence: if the GPU is fully powered off the setting resets. Effects may be unreliable on laptops; resume from suspend-to-RAM is re-applied, hibernation is not covered.
  • Heat and power: the README says damage is "very unlikely" but the card may run warmer under more work, with normal throttling intact. On a 24/7 server, monitor temperature and power.
  • Policy and warranty: vendor warranty, cloud or corporate policy and suitability for production servers are not reviewed here. Unverified.
  • Provenance: the author says they do not maintain a Windows build; a third-party port exists and the README urges caution with its installers. Do not trust downloads from outside the repository.

Conclusion for developers

The most useful first step is measuring what share of your workload's time is BF16 matrix multiply. If profiling shows heavy prefill and you have a dedicated development machine where boot settings are reversible, it is worth a trial. If you care about decode speed or run INT4/GGUF models, don't expect gains; for production machines or a single-purpose desktop, weigh the risk against the benefit. If you install it, record prefill time and decode tokens/second separately, before and after, with the same model and prompt length.