Looking for advice on how to use pre-trained AI models with Hugging Face for image recognition?

All answers to this question.

For retail applications where you need to distinguish between very similar products, Vision Transformers (ViT) are generally superior due to their global attention mechanism. You can load a pre-trained ViT model using the ViTImageProcessor and ViTForImageClassification. However, "real-time" is a relative term. If you are running this on edge devices like cameras, you might find ViT a bit heavy. In those cases, I’d suggest looking at MobileViT or a fine-tuned ResNet, which are significantly faster for inference on hardware with limited computational power.

   Answered 2025-04-15 by Deborah Lewis

What specific hardware are you targeting for the "real-time" aspect? Are we talking about a powerful GPU-backed server or an ARM-based mobile processor? This really changes the recommendation for the model architecture.

   Answered 2025-05-20 by Matthew Walker

• Matthew, we are targeting NVIDIA Jetson modules at the edge. So, while we have some GPU power, we still need to be very mindful of the power consumption and thermal limits. That's why I was hesitant about the full-sized ViT models.

     Commented 2025-05-22 by Joshua Taylor

Hugging Face also has amazing support for YOLO models now via the ultralytics integration. It might be better for object detection than simple classification.

   Answered 2025-06-10 by Brian Hall

• Brian is spot on. For shelf monitoring, detecting multiple items (YOLO) is much more useful than just classifying one image. The image-segmentation pipeline is also worth a look for precise masking.

     Commented 2025-06-18 by Deborah Lewis