Multi-label prediction with Banknotes

This notebook explains how to train a Deep Learning AI neural net to identify banknotes, using transfer learning, data augmentation and other state-of-the-art techniques. It uses multiple classes, so it separates a 10$ bill into 10 and $ classes. This is done so there is higher chances to identify all currencies of the same type across denominations (e.g. all dollars), and also learn about numbers across currencies (all 5s).

The model runs on a serverless backend and a static front-end available on iris.brunosan.eu. For documentation how to deploy the model, check the github repository.

In [1]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
In [2]:
import fastai
from fastai.vision import *
In [3]:
#For CPU only 
fastai.torch_core.defaults.device = 'cpu'
defaults.device= 'cpu'

Multiclassification

In [3]:
path = Path('.') #'/home/jupyter/.fastai/data/banknotes/')
path_imgs=path/'imgs'
path_imgs.mkdir(parents=True, exist_ok=True)
path_imgs
Out[3]:
PosixPath('imgs')
In [16]:
src = (ImageList.from_folder(path_imgs,recurse=True)
               .split_by_rand_pct(valid_pct=.2))

src
Out[16]:
ItemLists;

Train: ImageList (679 items)
Image (3, 500, 359),Image (3, 201, 500),Image (3, 225, 500),Image (3, 219, 500),Image (3, 281, 500)
Path: imgs;

Valid: ImageList (169 items)
Image (3, 215, 500),Image (3, 375, 500),Image (3, 640, 480),Image (3, 262, 500),Image (3, 480, 640)
Path: imgs;

Test: None
In [22]:
src.train.items[0]
Out[22]:
PosixPath('imgs/usd/50/IMG_20190730_232610.jpg')
In [7]:
#single class
func=lambda i: str(i.parent.relative_to(path_imgs) )
#multi class
func=lambda i: (i.parent.relative_to(path_imgs).parts )
func(src.train.items[0])
Out[7]:
('euro', '5')
In [8]:
ll = src.label_from_func(func); ll
#ll = src.label_from_folder(); ll
Out[8]:
LabelLists;

Train: LabelList (211 items)
x: ImageList
Image (3, 428, 500),Image (3, 255, 500),Image (3, 255, 500),Image (3, 500, 305),Image (3, 500, 435)
y: MultiCategoryList
euro;5,euro;5,euro;5,euro;5,euro;5
Path: /home/jupyter/.fastai/data/banknotes/imgs;

Valid: LabelList (52 items)
x: ImageList
Image (3, 278, 500),Image (3, 267, 500),Image (3, 217, 500),Image (3, 400, 500),Image (3, 333, 500)
y: MultiCategoryList
euro;200,euro;200,usd;100,usd;5,usd;10
Path: /home/jupyter/.fastai/data/banknotes/imgs;

Test: None
In [9]:
tfms = get_transforms(do_flip=True,flip_vert=True, 
                      max_rotate=90, 
                      max_zoom=1.5, 
                      max_lighting=0.5, 
                      max_warp=0.5)
In [10]:
#so its reproducible
#np.random.seed(42)
In [11]:
def get_data(size,bs):
    size=int(size)
    bs=int(bs)
    data = (ll.transform(tfms, size=size)
        .databunch(bs=bs) #for CPU only add ,num_workers=0
        .normalize(imagenet_stats))
    return data
size,bs=256/2,20
data=get_data(size,bs)
In [12]:
data.classes
Out[12]:
['1', '10', '100', '20', '200', '5', '50', '500', 'euro', 'usd']
In [13]:
data.show_batch(rows=4, figsize=(12,9))
In [14]:
arch = models.resnet50
In [15]:
acc_02 = partial(accuracy_thresh, thresh=0.2)
f_score = partial(fbeta, thresh=0.2)
#multiclass
learn = cnn_learner(data, arch, metrics=[acc_02, f_score])
#single class
#learn = cnn_learner(data, arch, metrics=[accuracy])

We use the LR Finder to pick a good learning rate.

In [16]:
learn.lr_find()
learn.recorder.plot()

LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.

Then we can fit the head of our network.

In [17]:
lr = 1e-2
In [18]:
learn.fit_one_cycle(10, slice(lr),callbacks=ShowGraph(learn))
epoch train_loss valid_loss accuracy_thresh fbeta time
0 0.773087 0.656208 0.271154 0.579044 00:02
1 0.676508 0.453052 0.526923 0.646792 00:02
2 0.553611 0.254357 0.807692 0.741027 00:02
3 0.457661 0.225245 0.884615 0.779478 00:02
4 0.396879 0.240159 0.855769 0.754953 00:02
5 0.356476 0.201313 0.886538 0.794969 00:02
6 0.319123 0.169044 0.909615 0.843629 00:02
7 0.291177 0.159696 0.909615 0.843046 00:02
8 0.267934 0.166017 0.896154 0.820270 00:02
9 0.246331 0.164613 0.903846 0.831099 00:02
In [19]:
learn.fit_one_cycle(5, slice(lr),callbacks=ShowGraph(learn))
epoch train_loss valid_loss accuracy_thresh fbeta time
0 0.168200 0.162801 0.896154 0.835130 00:02
1 0.173512 0.145427 0.915385 0.859460 00:02
2 0.172781 0.154199 0.911538 0.843629 00:02
3 0.170041 0.141247 0.932692 0.878885 00:02
4 0.165794 0.144410 0.928846 0.863151 00:02
In [20]:
learn.show_results(rows=3)