A Comparative Study of Classical Machine Learning and Deep Learning: Knn, Svm, and CNN

This research presents the comparative analysis of K-Nearest Neighbor (KNN), Support Vector Machine (SVM) and Convolutional Neural Network (CNN).We have focused on the demystification of the inner workings of KNN and SVM and CNN algorithms and have discussed alternatives to improve their performance based on different scenarios.Moreover, we have researched how and why KNN work better than SVM and CNN for specific problem and vice versa.Additionally, we have also investigated that why State of The Art (SOTA) KNN are less accurate than SVM in a certain problem space and vice versa.Consequently, we have studied the anatomy of KNN, SVM and CNN in detail.We have also carried out in depth analysis of what are the problem areas where SVM are established as gold standard.Furthermore, we have analyzed datasets relating to the problem areas where KNN works better and is preferred over SVM and CNN given that increasing the amount of data.The primary objective of our research is to investigate and analyze the performance of KNN, SVM and CNN.For this purpose, we have analyzed them by training and then evaluating the result of both algorithms deep learning and machine learning with different datasets i.e.Social Network Advertisement, Credit Card Fraud Detection.The later dataset contains number of transactions and goal is to classify whether the transactions are legitimate or fraudulent and another dataset Heart Cleveland Dataset from UCI Repository.We have compared and analyze two models i.e.KNN and SVM to address these problems.The classifier's output for data sets in various dimensional spaces varied.KNN often outperformed SVM in datasets with low dimensions, whereas SVM outperformed in both datasets with large dimensions and the hyper-plane.We tried various experiments to improve the performance on the dataset of Social Network Advertisement, Credit Card Fraud Detection, and Heart Cleveland Dataset and finally got the best accuracy of 93.0% by the first approach.Similarly, in the second approach SVM worked better as compared to KNN.For the third one also SVM worked well as compared to KNN because of high dimension space and high margin space.