Unsupervised pre-training for fully convolutional neural networks

Unsupervised pre-Training of neural networks has been shown to act as a regularization technique, improving performance and reducing model variance. Recently, fully convolutional networks (FCNs) have shown state-of-The-Art results on various semantic segmentation tasks. Unfortunately, there is no efficient approach available for FCNs to benefit from unsupervised pre-Training. Given the unique property of FCNs to output segmentation maps, we explore a novel variation of unsupervised pre-Training specifically designed for FCNs. We extend an existing FCN, called U-net, to facilitate end-To-end unsupervised pre-Training and apply it on the ISBI 2012 EM segmentation challenge data set. We performed a battery of significance tests for both equality of means and equality of variance, and show that our results are consistent with previous work on unsupervised pre-Training obtained from much smaller networks. We conclude that end-To-end unsupervised pre-Training for FCNs adds robustness to random initialization, thus reducing model variance.