0
Why your fine-tuning job takes longer than the FLOPs say
https://huggingface.co/blog/Dharma-AI/predicting-fine-tuning-time(huggingface.co)Standard FLOP-based calculations for estimating model fine-tuning time are highly inaccurate because they only consider compute and ignore significant overheads like environment setup, data loading, and checkpointing. An analysis of 272 production supervised fine-tuning jobs on NVIDIA H200 GPUs demonstrated that these traditional estimates perform poorly. A CatBoost regression model, using only configuration data available before a job is dispatched, can predict the total duration with much higher accuracy. This machine learning approach achieves an R² of 0.98 by learning the nonlinear interactions between compute, configuration-specific signals, and system overheads, making it a superior tool for scheduling jobs on shared GPU clusters.
0 points•by ogg•2 hours ago
Comments (0)
No comments yet. Be the first to comment!
Have an account? Log in to join the discussion.