I Built a C++ Backend So My GPU Would Stop Eating Air

GPUs waste immense power processing padded zeros, a common but inefficient method for handling variable-length text in LLM batches. A new C++ backend, WarpGroup-Backend, cleverly solves this by treating it as a high-stakes game of Tetris, packing different sequences together to fill VRAM without any padding. This packing logic runs in a separate C++ thread to bypass Python's performance bottlenecks, ensuring the GPU is fed a continuous stream of useful data. By eliminating this wasteful "pretend work," the method dramatically accelerates LLM inference by up to 5.89x and prevents out-of-memory crashes.