Hand-Drawn Image Classification Using a Convolutional Neural Network
Date:
Author: Dylan Jacobs
Project: Github
Interactive project demonstration: Click here to try out the model yourself!!
Slide 1
Slide 2
Project Scope
● GOAL: Classify a hand-drawn sketch into 1 of 250 categories
● Dataset: TU-Berlin Hand Sketch
○ 250 image classes, 20,000 images
⤷ 80 images per class
● Difficulties:
○ Drawing variation
○ Sparse line drawings
○ Visually similar classes (mug vs cup, cat vs tiger)
● 72/18/10 Train/Validation/Test split
● Parameters:
○ Image size: 128x128
○ Batch size: 150
○ Learning Rate: 0.001 (scheduled)
○ Epochs: 130
Slide 3
Model Architecture
● Input: grayscale image resized to 128x128
● 5 convolutional blocks, increasing filters 32 → 64 → 128 → 256 → 512
● Each block: Conv2D + BatchNorm + ReLU + MaxPool
● Fully connected layers: 1024 → 512 → 256 → 250
● 50% Dropout to reduce overfitting
Architecture Reasoning
Increasing filters with depth lets the network move from simple low-level features to more complex object structure
● Batch normalization stabilizes training and helps the model converge.
● Grayscale input matches the dataset, since shape matters more than color for hand-drawn sketches
● Max pooling reduces spatial size and computation while preserving the most important features
Slide 4
Challenge: Limited Training Data
~14,000 training samples
Solution: use torch.RandomTransforms!
Slide 5
Training
● Optimizer: Adam
○ Initial LR=0.001
○ weight_decay=1e-4
● Loss: CrossEntropyLoss with label smoothing 0.1
● Scheduler: ReduceLROnPlateau (factor=0.1)
● Best-checkpoint saving: only saves weights when validation loss improves
● Epochs: 130
● Batch size: 150
● Trained on Colab T4 GPU
● Lack of data → use torch.RandomTransforms!
Slide 6
Challenge: Overfitting on first attempts!
Solutions:
● Add label_smoothing: replace hard 0/1 classification labels with “softened” probabilities (e.g. 0.05, 0.95)
● Increase Dropout probabilities to 50%
● Add weight_decay to the optimizer
○ Penalize large weights to reduce reliance on single weights
● Aggressively apply torch.RandomTransforms → more training data augmentation
Slide 7
Post-Training Weights
First 8 filters from each of the 5 convolution layers
Slide 8
Results
● Test accuracy: ~68%
● Main problem: lack of data!!
○ 14,000 training images for 250 classes ⇒ < 60 images per class
● Also, lack of GPU Compute Units
Slide 9
More Results
Slide 10
Conclusion ● The final model achieves 68% accuracy on a 250-class problem with a relatively small custom CNN
● 10,679,994 trainable parameters
● The biggest lessons:
○ Diagnose bottlenecks before iterating/changing design
○ Train acc > val acc means you can likely push the model harder
○ Size of data set is extremely important, especially with lots of classes
○ GPU Compute Units are important!!