Breaking the Hardware Barrier: Software FP8 for Older GPUs

An open-source library called Feather provides a software-based method to achieve FP8-like performance on older GPUs that lack native hardware support. The technique involves packing multiple lower-precision numbers, such as four FP8s, into a single FP32 container using bitwise operations. This approach reduces the memory footprint and alleviates the data transfer bottleneck common in memory-bound deep learning operations. By using custom Triton kernels to unpack the data on-the-fly, the library achieves significant performance gains, with benchmarks showing up to a 3.3x speedup for tasks like matrix-vector multiplication.