Hyper-parameter tuning for support vector machine using an improved cat swarm optimization algorithm

Authors

  • Silifat Adaramaja Abdulraheem
    Department of Computer Science, Faculty of Physical Science, Ahmadu Bello University Zaria, Nigeria
  • Salisu Aliyu
    Department of Computer Science, Faculty of Physical Science, Ahmadu Bello University Zaria, Nigeria
  • Fatima Binta Abdullahi
    Department of Computer Science, Faculty of Physical Science, Ahmadu Bello University Zaria, Nigeria

Keywords:

Cat Swarm Optimization, Support Vector Machine, Opposition-based learning

Abstract

Support vector machine (SVM) is a supervised machine learning algorithm for classification and regression problems. SVM performs better when combined with other classifiers or optimized with an optimization algorithm. The SVM parameters such as kernel and penalty have good performance on the classification accuracy. Recently, a lot of evolutionary optimization algorithms were used for optimizing the SVM. In this paper, an Improved Cat Swarm Optimization (ICSO) was proposed for optimizing the parameters of SVM with the aim of enhancing its performance. CSOs have the problem of a low convergence rate and are easily trapped in local optima. To address this problem, a new parameter was added to the velocity for the tracing mode and the Opposition-Based Learning (OBL) technique was used to modify the CSO algorithm (ICSO-SVM). A new parameter was introduced to guide the cats’ positions to the local and global best positions in the velocity tracing mode of the CSO algorithm. The proposed algorithm was verified using 15 datasets from the University of California Irvine (UCI) data repository and also six different performance metrics were used. The experimental results clearly indicate that the proposed method performs better than the other state-of-the-art methods.

Dimensions

J.T.Senders,M.M.Zaki,A.V.Karhade,B.Chang, W.B.Gormley, M. L. Broekman, T. R. Smith & O. Arnaout “An introduction and overview of machine learning in neurosurgical care”, Acta Neurochirurgica 160 (2018) 29. https://link.springer.com/article/10.1007/s00701-017-3385-8 DOI: https://doi.org/10.1007/s00701-017-3385-8

C.L.Udeze, I.E.Eteng & A. E. Ibor, “ApplicationofMachineLearning and Resampling Techniques to Credit Card Fraud Detection”, Journal of the Nigerian Society of Physical Sciences (JNSPS) 4 (2022) 1. https://doi.org/10.46481/jnsps.2021.769 DOI: https://doi.org/10.46481/jnsps.2022.769

D. Opeoluwa, E. Gbenga, N. Juliana, A. U. Terang & S. A. Akinwumi, “COVID-19 Risk Factors, Economic Factors, and Epidemiological Factors nexus on Economic Impact: Machine Learning and Structural Equation Modelling Approaches”, Journal of the Nigerian Society of Physical Sciences (JNSPS) 3 (2021) 395. https://doi.org/10.46481/jnsps.2021.173 DOI: https://doi.org/10.46481/jnsps.2021.173

U. B. Mir, S. Sharma, A. K. Kar & M. P. Gupta, “Critical success factors for integrating artificial intelligence and robotics”, Digital Policy Regulation and Governance 22 (2020) 307. https://doi.org/10.1108/ DPRG- 03- 2020- 0032 DOI: https://doi.org/10.1108/DPRG-03-2020-0032

M. A. Goralski & T. K. Tan, “Artificial intelligence and sustainable development”, International Journal of Management Education 18 (2020) 1. https://doi.org/10.1016/j.ijme.2019.100330 DOI: https://doi.org/10.1016/j.ijme.2019.100330

N. Malik, S. N. Tripathi, A. K. Kar & S. Gupta, “Impact of artificial intelligence on employees working in industry 4.0 led organizations”, International Journal of Manpower 2011 (2021). https://doi.org/10.1108/ijm-03-2021-0173 DOI: https://doi.org/10.1108/IJM-03-2021-0173

S. M. C. Loureiro, J. Guerreiro & I. Tussyadiah, “Artificial intelligence in business: State of the art and future research agenda”, Journal Business Research 129 (2021) 911. https://doi.org/10.1016/j.jbusres.2020.11.001 DOI: https://doi.org/10.1016/j.jbusres.2020.11.001

B. E. Boser, V. N. Vapnik & I. M. Guyon, “A Training Algorithm Mar- gin for Optimal Classifiers”, Proceedings of the fifth Annual Workshop on Computational Learning Theory (1992) 144. https://doi.org/10.1145/ 130385.130401 DOI: https://doi.org/10.1145/130385.130401

D. S. Kim & J. S. Park, “Network-based intrusion detection with support vector machines”, International Conference on Information Networking (2003) 747. https://link.springer.com/chapter/10.1007/978-3-540-45235-573 DOI: https://doi.org/10.1007/978-3-540-45235-5_73

A. C. Enache & V. V. Patriciu, “Intrusionsdetectionbasedonsupportvec- tor machine optimized with swarm intelligence”, SACI 2014 - 9th IEEE International Symposium on Applied Computational Intelligence and Informatics (2014) 153. https://ieeexplore.ieee.org/document/6840052 DOI: https://doi.org/10.1109/SACI.2014.6840052

Y. Li, L. Guo, Z. H. Tian & T. B. Lu, “A lightweight web server anomaly detection method based on transductive scheme and genetic algorithms”, Computer Communications 31 (2008) 4018. https://doi.org/10.1016/j.comcom.2008.08.009 DOI: https://doi.org/10.1016/j.comcom.2008.08.009

K. C. Lin, Y. H. Huang, J. C. Hung & Y. T. Lin, “Feature Selection and Parameter Optimization of Support Vector Machines Based on Modified Cat Swarm Optimization”, International Journal of Distributed Sensor Networks 2015, (2015) 9. https://doi.org/10.1155/2015/365869 DOI: https://doi.org/10.1155/2015/365869

S. C. Chu & P. W. Tsai, “Computational intelligence based on the behaviour of cats”, International Journal of Innovative, Computing Information and Control 3 (2007) 163. https://api.semnticscholar.org/CorpusID: 58516637

M. Gomathy, “Optimal feature selection for speech emotion recognition using enhanced cat swarm optimization algorithm”, International Journal of Speech Technology 24 (2021) 155. https://doi.org/10.1007/s10772-020-09776-x DOI: https://doi.org/10.1007/s10772-020-09776-x

M. Orouskhani, M. Mansouri & M. Teshnehlab, “Average-inertia weighted cat swarm optimization”, Proceedings of Second International Conference on Advances in Swarm Intelligence (2011) 321. https://doi.org/10.1007/978-3-642-21515-5 38 DOI: https://doi.org/10.1007/978-3-642-21515-5_38

S. Mahdavi, S. Rahnamayan & K. Deb, “Opposition based learning: A literature review”, Swarm Evolutionary Computation 39 (2018) 1. https://doi.org/10.1016/j.swevo.2017.09.010 DOI: https://doi.org/10.1016/j.swevo.2017.09.010

P. W. Tsai, J. S. Pan, S. M. Chen, B. Y. Liao & S. P. Hao, “Parallel cat swarm optimization”, International Conference on Machine Learning and Cybernetics. ICMLC 6 (2008) 3328. https://doi.org/10.1109/ICMLC.2008.4620980 DOI: https://doi.org/10.1109/ICMLC.2008.4620980

K. C. Lin & H. Y. Chien, “CSO-based feature selection and parameter optimization for support vector machine”, 2009 Joint Conferences on Pervasive Computing JCPC 2009 (2009) 783. https://doi.org/10.1109/JCPC. 2009.5420080

B. Santosa & M. K. Ningrum, “Cat swarm optimization for clustering”, SoCPaR 2009 - Soft Computing and Pattern Recognition (2009) 54. https://doi.org/10.1109/SoCPaR.2009.23 DOI: https://doi.org/10.1109/SoCPaR.2009.23

P. W. Tsai, J. S. Pan, S. M. Chen & B. Y. Liao, “Enhanced parallel cat swarm optimization based on the Taguchi method”, Expert Systems Applications 39 (2012) 6309. https://doi.org/10.1016/j.eswa.2011.11.117 DOI: https://doi.org/10.1016/j.eswa.2011.11.117

M. Orouskhani, Y. Orouskhani, M. Mansouri & M. Teshnehlab, “A Novel Cat Swarm Optimization Algorithm for Unconstrained Optimization Problems”, International Journal of Information Technology and Computer Science 5 (2013) 32. https://doi.org/10.5815/ijitcs.2013.11.04 DOI: https://doi.org/10.5815/ijitcs.2013.11.04

I. Hadi & M. Sabah, “Improvement Cat Swarm Optimization for Efficient Motion Estimation”, International Journal of Hybrid Information Technology 8 (2015) 279. https://dx.doi.org/10.14257/ijhit.2015.8.1.25 DOI: https://doi.org/10.14257/ijhit.2015.8.1.25

J. Manurung, H. Mawengkang & E. Zamzami, “Optimizing Support Vector Machine Parameters with Genetic Algorithm for Credit Risk Assessment”, Journal of Physics Conference Series 930 (2017) 8. https://doi.org/10.1088/1742-6596/930/1/012026 DOI: https://doi.org/10.1088/1742-6596/930/1/012026

X. Nie, W. Wang & H. Nie, “Chaos quantum-behaved cat swarm optimization algorithm and its application in the PV MPPT”, Computational Intelligence and Neuroscience 2017, (2017). https://doi.org/10. 1155/2017/1583847 DOI: https://doi.org/10.1155/2017/1583847

K. C. Lin, K. Y. Zhang, Y. H. Huang, J. C. Hung & N. Yen, “Feature selection based on an improved cat swarm optimization algorithm for big data classification”, Journal of Supercomputing 72 (2016) 3210. https://doi.org/10.1007/s11227-016-1631-0 DOI: https://doi.org/10.1007/s11227-016-1631-0

P. Mohapatra, S. Chakravarty & P. K. Dash, “Microarray medical data classification using kernel ridge regression and modified cat swarm optimization based gene selection system”, Swarm Evolutionary Computation 28 (2016) 144. https://doi.org/10.1016/j.swevo.2016.02.002 DOI: https://doi.org/10.1016/j.swevo.2016.02.002

M. Y. Cho & T. T. Hoang, “Feature Selection and Parameters Optimization of SVM Using Particle Swarm Optimization for Fault Classification in Power Distribution Systems”, Computational Intelligence and Neuroscience 2017 (2017). https://doi.org/10.1155/2017/4135465 DOI: https://doi.org/10.1155/2017/4135465

Y. Kumar & G. Sahoo, “An improved cat swarm optimization algorithm based on opposition-based learning and Cauchy operator for clustering”, Journal of Information Processing Systems 13 (2017) 1000. https://doi.org/10.3745/JIPS.02.0022 DOI: https://doi.org/10.3745/JIPS.02.0022

A. Tharwat, A. Ella & B. E. Elnaghi, “A BA-based algorithm for parameter optimization of Support Vector Machine”, Pattern Recognition Letters 93 (2017) 13. https://doi.org/10.1016/j.patrec.2016.10.007 DOI: https://doi.org/10.1016/j.patrec.2016.10.007

H. Siqueira, E. Figueiredo, M. Macedo, C. J. Santana, C. J. A. Bastos- Filho & A. A. Gokhale, “Boolean Binary Cat Swarm Optimization Algorithm”, 2018 IEEE Latin American Conference on Computational Intelligence LA-CCI (2018) 1. https://doi.org/10.1109/LA-CCI.2018.8625226 DOI: https://doi.org/10.1109/LA-CCI.2018.8625226

A. Tharwat, T. Gabel & A. E. Hassanien, “Parameter optimization of support vector machine using dragonfly algorithm”, Advances in Intelligent System and Computing 639 (2018) 309. https://doi.org/10.1007/978- 3- 319- 64861- 3 29 DOI: https://doi.org/10.1007/978-3-319-64861-3_29

L. Pappula & D. Ghosh, “Cat swarm optimization with normal mutation for fast convergence of multimodal functions”, Journal of Applied Soft Computing 66 (2018) 473. https://doi.org/10.1016/j.asoc.2018.02.012 DOI: https://doi.org/10.1016/j.asoc.2018.02.012

A. Tharwat, T. Gabel & T. Gabel, “Parameters optimization of support vector machines for imbalanced data using social ski driver algorithm”, Neural Computing and Applications 32 (2020) 6925. https://doi.org/10. 1007/s00521-019-04159-z DOI: https://doi.org/10.1007/s00521-019-04159-z

M. A. Nanda, K. B. Seminar, D. Nandika & A. Maddu, “A comparison study of kernel functions in the support vector machine and its application for termite detection”, Information 9 (2018). https://doi.org/10.3390/info9010005 DOI: https://doi.org/10.3390/info9010005

S. Idris, O. Oyefolahan Ishaq & N. Ndunagu Juliana, “Intrusion Detection System Based on Support Vector Machine Optimised with Cat Swarm Optimization Algorithm”, 2nd International Conference of the IEEE Nigeria Computer Chapter (2019) 1. https://doi.org/10.1109/ NigeriaComputConf45974.2019.8949676 DOI: https://doi.org/10.1109/NigeriaComputConf45974.2019.8949676

S. A. Mulay, P. R. Devale & G. V. Garje, “Intrusion Detection System Using Support Vector Machine and Decision Tree”, International Journal of Computer Applications 3 (2010) 40. https://doi.org/10.5120/758-993 DOI: https://doi.org/10.5120/758-993

B. Y. Sun, D. S. Huang, L. Guo & Z. Q. Zhao, “Support vector machine committee for classification”, International Symposium on Neural Network 3173 (2004) 648. https://doi.org/10.1007/978-3-540-28647-9 106 DOI: https://doi.org/10.1007/978-3-540-28647-9_106

J. H. Min & Y. C. Lee, “Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters”, Expert System Applications 28 (2005) 603. https://doi.org/10.1016/j.eswa.2004.12.008 DOI: https://doi.org/10.1016/j.eswa.2004.12.008

R. Amami, D. Ben Ayed & N. Ellouze, “Practical Selection of SVM Supervised Parameters with Different Feature Representations for Vowel Recognition”, Computer Science Bibliography 7 (2015) 418. https://doi.org/10.48550/arXiv.1507.06020

H. R. Tizhoosh, “Opposition-based learning: A new scheme for machine intelligence”, Proceedings of International Conference of Computational Intelligence Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06 1 (2005) 695. https://doi.org/10.1109/ CIMCA.2005.1631345

C. Huang & C. Wang, “A GA-based feature selection and parameters optimization for support vector machines”, Expert Systems with Applications 31 (2006) 231. https://doi.org/10.1016/j.eswa.2005.09.024 DOI: https://doi.org/10.1016/j.eswa.2005.09.024

Published

2023-09-08

How to Cite

Hyper-parameter tuning for support vector machine using an improved cat swarm optimization algorithm. (2023). Journal of the Nigerian Society of Physical Sciences, 5(4), 1007. https://doi.org/10.46481/jnsps.2023.1007

Issue

Volume 5, Issue 4, November 2023

Section

Original Research
Copyright & Licensing

Copyright (c) 2023 Silifat Adaramaja Abdulraheem, Salisu Aliyu, Fatima Binta Abdullahi

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Hyper-parameter tuning for support vector machine using an improved cat swarm optimization algorithm. (2023). Journal of the Nigerian Society of Physical Sciences, 5(4), 1007. https://doi.org/10.46481/jnsps.2023.1007