Faster way to use faster RCNN : using detectron2
I would like to introduce the detectron2 in this essay and show the thing PCB error detection demo. You can get the demo in colab, it can execute immediately. This essay is main to provide another example for detectron2.
(update colab in 2020/05/27. It can execute now. If you get some problem in colab, you may change the environment following the tutorial of detectron2 in colab)
Don’t forget to star my github if you like the demo.
Detectron2 was developed by facebookresearch. It combine the Detectron and maskrcnn-benchmark. You can feel that is quit easy to use after the experiment in the past. Moreover, it has a lots of pretrained model which can accelerate researches without building model of faster-RCNN ourselves. It saves tons of time.
You can also browse the faster-RCNN tutorial on pytorch official website. It seems easy to use. However, many parts of it have to make by yourself. It’s quit hard to do when you have to code training engine. I gave it up immediately as I realized and turn to detectron2.
Environment
Linux or macOSPython ≥ 3.6PyTorch ≥ 1.3torchvision that matches the PyTorch installation. You can install them together at pytorch.org to make sure of this.OpenCV, optional, needed by demo and visualizationpycocotools: pip install cython; pip install ‘git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'gcc & g++ ≥ 4.9
Check the environment and then
git clone https://github.com/facebookresearch/detectron2.git
cd detectron2 && pip install -e .You can also get PCB data I use in here. Following the format of dataset, we can easily use it. It is a dict with path of the data, width, height, information of bounding box. You will need to add segmentation if you are using mask-rcnn. Here is my example:
{‘file_name’: ‘../DeepPCB/PCBData/group20085/20085/20085000_test.jpg’,
‘image_id’: 0,
‘height’: 640,
‘width’: 640,
‘annotations’: [
{‘bbox’: [409.0, 394.0, 435.0, 422.0], ‘bbox_mode’: <BoxMode.XYXY_ABS: 0>, ‘category_id’: 2, ‘iscrowd’: 0},
{‘bbox’: [275.0, 383.0, 319.0, 417.0], ‘bbox_mode’: <BoxMode.XYXY_ABS: 0>, ‘category_id’: 2, ‘iscrowd’: 0},
{‘bbox’: [8.0, 163.0, 36.0, 191.0], ‘bbox_mode’:
<BoxMode.XYXY_ABS: 0>, ‘category_id’: 3, ‘iscrowd’: 0},
{‘bbox’: [244.0, 151.0, 270.0, 182.0], ‘bbox_mode’: <BoxMode.XYXY_ABS: 0>, ‘category_id’: 4, ‘iscrowd’: 0},
{‘bbox’: [338.0, 519.0, 364.0, 543.0], ‘bbox_mode’: <BoxMode.XYXY_ABS: 0>, ‘category_id’: 5, ‘iscrowd’: 0},
{‘bbox’: [476.0, 460.0, 502.0, 481.0], ‘bbox_mode’: <BoxMode.XYXY_ABS: 0>, ‘category_id’: 3, ‘iscrowd’: 0}
]}Code for build the custom data set:
We can get pretrain model in model zoo of detectron2 for training.
I believe that using model pretrain by coco dataset to detect PCB error will not get a good performance. However, I get 64 in AP value with only 3000 epochs. We could get 75 in AP value if we train for 30000 epochs. It’s better than I expect.
Here is the results of prediction.
tensor([0, 4, 3, 2, 1, 5], device='cuda:0')
Boxes(tensor([[289.1584, 326.0847, 325.2099, 358.8866],
[233.4901, 118.8735, 262.8138, 149.5809],
[106.2966, 181.0337, 140.5350, 211.7205],
[259.9618, 240.3079, 288.6585, 271.2735],
[500.9987, 334.7750, 527.8199, 374.0154],
[347.2850, 146.7405, 395.6854, 180.7809]], device='cuda:0'))
Although, its resolution isn’t quit well, you can find that model recognize the error on PCB and get right classification. We can’t know how well this model is, so we use the evaluation from coco to figure it out. In detectron2, it is quit easy to use evaluator to get AP, AP50, AP75.
| AP | AP50 | AP75 | APs | APm | APl |
|:------:|:------:|:------:|:------:|:------:|:------:|
| 64.318 | 93.136 | 79.455 | 62.998 | 65.486 | 70.000 |
[01/16 06:41:56 d2.evaluation.coco_evaluation]: Per-category bbox AP:
| category | AP | category | AP | category | AP |
|:-----------|:-------|:-----------|:-------|:-----------|:-------|
| open | 59.408 | short | 50.631 | mousebite | 63.177 |
| spur | 65.522 | copper | 77.720 | pin-hole | 69.452 |
OrderedDict([('bbox',
{'AP': 64.31825163056139,
'AP-copper': 77.72030131795276,
'AP-mousebite': 63.177111687191676,
'AP-open': 59.407758192556805,
'AP-pin-hole': 69.45151146413728,
'AP-short': 50.63100208959249,
'AP-spur': 65.52182503193735,
'AP50': 93.13612566723208,
'AP75': 79.45463612126157,
'APl': 69.99999999999999,
'APm': 65.48557070350444,
'APs': 62.997574669449286})])Its results is better than the paper I read A VGG-16 Based Faster RCNN Model for PCB Error Inspection in Industrial AOI Applications. It should not being compared with AP values only because we use different data set and different kind of backbone model, but we get 75 in AP values in only 30000 training epochs. You can realize how powerful resnet is.
Detectron2 can also customize the backbone model by your own. If you find that isn’t VGG16 in model zoo, you can create one by yourself. I didn’t do it, so I wouldn’t show it to mislead you guys.
Troubleshooting
The most wired bug I faced is using evaluator and get -1 in AP value. I had checked prediction image which had well performance in many image. Got right prediction, but wrong AP value. In the help of my friend, I realize that original point is at top left while I set it in bottom left. I set wrong bounding box for sure. However, it could train anyway and predicted well. Review the dataset you create if you get -1 in AP value.
Moreover, remember to delete the file used to evaluate. I save it at path “./output” in the demo. We have to delete it since file can’t be overwrited. In my demo, the file used to evaluate is called PCB_test_coco_format.json.
Other application
Detectron2 is a powerful tool. In addition to reduce time of PCB error detection, it provide other application such as keypoint, DensePose, TridentNet, etc. It seems that would have more application with convenience.
Reference
A VGG-16 Based Faster RCNN Model for PCB Error Inspection in Industrial AOI Applications
