Is it worth moving from CPUs to GPUs for training small-to-medium tabular datasets?

All answers to this question.

It is absolutely worth it once you cross the million-row threshold. While it’s true that Deep Learning (CNNs/RNNs) saw the first major benefits, modern implementations of XGBoost, LightGBM, and CatBoost have highly optimized GPU histograms. In my tests, training a 5-million-row dataset on an NVIDIA T4 or V100 was about 10x to 15x faster than a high-end 8-core CPU. The key is to ensure you set the tree_method to gpu_hist in XGBoost. Just keep in mind that moving data between the CPU and GPU memory has overhead, so for very small datasets (under 100k rows), the CPU might actually still be faster.

   Answered 2025-07-22 by Laura Jenkins

Have you considered using a library like RAPIDS cuML? It allows you to run your entire data science pipeline, including Pandas-like operations (cuDF), directly on the GPU. What is your current training time per fold during cross-validation?

   Answered 2025-07-25 by Brian O'Connor

• Brian, right now a 5-fold CV is taking me nearly 45 minutes, which makes hyperparameter tuning a total nightmare. I hadn't heard of RAPIDS cuML, but if I can keep the data on the GPU for preprocessing too, that would solve the bottleneck I was worried about. Does cuML support all the standard Scikit-Learn metrics and transformations, or would I have to rewrite a lot of my existing pipeline code to make it compatible?

     Commented 2025-07-27 by Mark Sullivan

If you're on a budget, try using Google Colab's free GPU tier first. It’s a great way to benchmark your specific code before committing to a paid AWS or GCP instance

   Answered 2025-07-29 by Rachel Adams

• Great tip, Rachel! I used Colab for my last project and the speedup for CatBoost was incredible. It’s the best way to "try before you buy" in the cloud ML space.

     Commented 2025-07-31 by Mark Sullivan