What is transfer learning in computer vision?

Explain how to use pretrained models like ResNet or VGG for new computer vision tasks.

Transfer Learning in Computer Vision

Transfer learning is a machine learning technique where a model developed for one task is reused as the starting point for a model on a second task. In computer vision, this typically involves using pre-trained models on large datasets like ImageNet.

Key Concepts:

1. Pre-trained Models:

   • Models like ResNet, VGG, Inception trained on massive datasets
   • Learn general visual features (edges, textures, patterns)
   • Save significant computation and data requirements

2. Transfer Learning Approaches:

   Feature Extraction:

   • Freeze pre-trained layers
   • Remove final classification layer
   • Add new layers for target task
   • Only train new layers
   • Best when:
     • Limited target dataset
     • Similar domains

   Fine-tuning:

   • Start with pre-trained weights
   • Retrain entire network or specific layers
   • Use smaller learning rate
   • Best when:
     • Larger target dataset
     • More different domains

3. Common Pre-trained Models:

   ResNet:

   • Various depths (18, 50, 101, 152 layers)
   • Excellent feature hierarchies
   • Good balance of performance and size

   VGG:

   • Simple architecture
   • Strong feature representations
   • Larger memory footprint

   EfficientNet:

   • State-of-the-art performance
   • Optimized architecture scaling
   • Good for mobile/edge devices

4. Implementation Steps:

   a. Load Pre-trained Model:

   model = torchvision.models.resnet50(pretrained=True)
   

   b. Modify Architecture:

   # Remove final layer
   num_features = model.fc.in_features
   model.fc = nn.Linear(num_features, num_classes)
   

   c. Configure Training:

   # Feature extraction
   for param in model.parameters():
       param.requires_grad = False
   model.fc.requires_grad = True
   

5. Best Practices:

   • Data Preprocessing:

     • Match preprocessing of original training
     • Use same image size, normalization

   • Learning Rate:

     • Smaller for fine-tuning (1e-4 to 1e-5)
     • Larger for new layers (1e-2 to 1e-3)

   • Layer Selection:

     • Earlier layers: generic features
     • Later layers: task-specific features

6. Advantages:

   • Reduced training time
   • Less data required
   • Better generalization
   • Lower computational resources

7. Limitations:

   • Domain shift can impact performance
   • May need architecture modifications
   • Memory/compute requirements of large models

8. When to Use:

   • Limited labeled data
   • Similar domain to pre-trained task
   • Time/resource constraints
   • Need for robust features

Transfer learning has become a fundamental technique in modern computer vision, enabling rapid development of new applications without massive datasets or computational resources.