Analysis and optimisation of network parameters in deep learning based neural networks

Deep learning neural networks, or, more precisely, Convolutional Neural Networks (CNNs), have demonstrated great potential in terms of their treatment of complex image classification problems over the past few years. CNNs have a large number of parameters and the system accuracy depends directly on the selection of these parameters. Despite CNNs’ recent successes, the diverse numbers of parameters and the impact and values of those parameters in deep learning are still unknown. Due to the unknown impacts and values of these deep learning parameters, many trial-and-error or ad hoc methods are used to configure deep learning systems, which increase training time, reduce performance and result in lower classification accuracies, especially when applied to complex real-world applications. In practice, optimal parameter selection remains the biggest obstacle when designing a real-world application using CNNs. This research has investigated the impact of deep learning parameters on classification accuracy and their applicability in real-world applications, such as scene parsing. In particular, this research has focused on the impact of changing the input window size, the filter size, the number of layers and the number of filters. A methodology based on repeated measures and an evolutionary algorithm to obtain the optimal values for the network parameters has been developed; additionally, a large number of experiments using the developed methodology and benchmark datasets were conducted. Finally, a comparative analysis of the results was presented and discussed. The original contributions of this thesis are Grid and Particle Swarm Optimisation (PSO) based methods for optimising and determining the impact of network parameters in deep learning-based real-world applications. The experimental analysis has shown that the PSO-based method has achieved 81.47% and 88.3% accuracies for CIFAR-10 and RSVD datasets respectively, which are better than the accuracies obtained using grid based parameter optimisation and other existing approaches.