Epilepsy is a highly prevalent disorder that can affect a person's quality of life. People with epilepsy are commonly affected by reoccurring seizures that potentially cause injury or death. Neurologists frequently use Electroencephalography (EEG) recordings to diagnose people with epilepsy. However, this can be both laborious and error-prone, as this can rely on competency and insight from undertrained neurologists. Machine learning-based methods have been recently proposed for seizure detection so that neurologists can make a quick and correct diagnosis. However, these methods often require features of the EEG signal to be extracted from data before the data can be used. Furthermore, the choice of features often requires domain knowledge about the data, and depending on the expert knowledge of the user, the selection of which features to extract can have a dramatic impact on the classifier's performance. In this paper, we propose a novel method that can autonomously extract features from deep within a convolutional neural network (CNN) and generate easy to understand rules/explanations used for the classification of seizures from EEG signals. The aim of creating rules/explanations is to explain the internal logic of our method to give the neurologist insight into the decision-making process. Thus, distilling trust. We evaluate our method against other classifiers in terms of accuracy, sensitivity, and specificity, and achieve an accuracy of 98.65%, sensitivity of 96.29%, specificity of 99.25%.