Unsupervised pre-training for fully convolutional neural networks

Stiaan Wiehman, Steve Kroon, Hendrik De Villiers

Research output: Chapter in Book/Report/Conference proceedingConference paperAcademicpeer-review

5 Citations (Scopus)

Abstract

Unsupervised pre-Training of neural networks has been shown to act as a regularization technique, improving performance and reducing model variance. Recently, fully convolutional networks (FCNs) have shown state-of-The-Art results on various semantic segmentation tasks. Unfortunately, there is no efficient approach available for FCNs to benefit from unsupervised pre-Training. Given the unique property of FCNs to output segmentation maps, we explore a novel variation of unsupervised pre-Training specifically designed for FCNs. We extend an existing FCN, called U-net, to facilitate end-To-end unsupervised pre-Training and apply it on the ISBI 2012 EM segmentation challenge data set. We performed a battery of significance tests for both equality of means and equality of variance, and show that our results are consistent with previous work on unsupervised pre-Training obtained from much smaller networks. We conclude that end-To-end unsupervised pre-Training for FCNs adds robustness to random initialization, thus reducing model variance.

Original languageEnglish
Title of host publication2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference, PRASA-RobMech 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Print)9781509033355
DOIs
Publication statusPublished - 2017
Event2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference, PRASA-RobMech 2016 - Stellenbosch, South Africa
Duration: 30 Nov 20162 Dec 2016

Conference

Conference2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference, PRASA-RobMech 2016
CountrySouth Africa
CityStellenbosch
Period30/11/162/12/16

Fingerprint Dive into the research topics of 'Unsupervised pre-training for fully convolutional neural networks'. Together they form a unique fingerprint.

  • Cite this

    Wiehman, S., Kroon, S., & De Villiers, H. (2017). Unsupervised pre-training for fully convolutional neural networks. In 2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference, PRASA-RobMech 2016 [7813160] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/RoboMech.2016.7813160