Transformers.js v4 Preview: Now Available on NPM!

The preview for Transformers.js v4 is now available on NPM, featuring a completely rewritten WebGPU Runtime in C++ for improved performance and cross-environment compatibility. This update leverages specialized ONNX Runtime operators to achieve significant speedups and enables models to run with hardware acceleration in browsers and server-side JavaScript environments. The repository has been restructured into a monorepo, the build system was migrated to esbuild for faster builds, and support for new models like GPT-OSS and FalconH1 has been added. Additionally, the tokenization logic has been extracted into a new standalone, lightweight library called @huggingface/tokenizers.