Hyper-parameter tuning for support vector machine using an improved cat swarm optimization algorithm

Authors

  • Silifat Adaramaja Abdulraheem
    Department of Computer Science, Faculty of Physical Science, Ahmadu Bello University Zaria, Nigeria
  • Salisu Aliyu
    Department of Computer Science, Faculty of Physical Science, Ahmadu Bello University Zaria, Nigeria
  • Fatima Binta Abdullahi
    Department of Computer Science, Faculty of Physical Science, Ahmadu Bello University Zaria, Nigeria

Keywords:

Cat Swarm Optimization, Support Vector Machine, Opposition-based learning

Abstract

Support vector machine (SVM) is a supervised machine learning algorithm for classification and regression problems. SVM performs better when combined with other classifiers or optimized with an optimization algorithm. The SVM parameters such as kernel and penalty have good performance on the classification accuracy. Recently, a lot of evolutionary optimization algorithms were used for optimizing the SVM. In this paper, an Improved Cat Swarm Optimization (ICSO) was proposed for optimizing the parameters of SVM with the aim of enhancing its performance. CSOs have the problem of a low convergence rate and are easily trapped in local optima. To address this problem, a new parameter was added to the velocity for the tracing mode and the Opposition-Based Learning (OBL) technique was used to modify the CSO algorithm (ICSO-SVM). A new parameter was introduced to guide the cats’ positions to the local and global best positions in the velocity tracing mode of the CSO algorithm. The proposed algorithm was verified using 15 datasets from the University of California Irvine (UCI) data repository and also six different performance metrics were used. The experimental results clearly indicate that the proposed method performs better than the other state-of-the-art methods.

Dimensions

J.T.Senders,M.M.Zaki,A.V.Karhade,B.Chang, W.B.Gormley, M. L. Broekman, T. R. Smith & O. Arnaout “An introduction and overview of machine learning in neurosurgical care”, Acta Neurochirurgica 160 (2018) 29. https://link.springer.com/article/10.1007/s00701-017-3385-8

C.L.Udeze, I.E.Eteng & A. E. Ibor, “ApplicationofMachineLearning and Resampling Techniques to Credit Card Fraud Detection”, Journal of the Nigerian Society of Physical Sciences (JNSPS) 4 (2022) 1. https://doi.org/10.46481/jnsps.2021.769

D. Opeoluwa, E. Gbenga, N. Juliana, A. U. Terang & S. A. Akinwumi, “COVID-19 Risk Factors, Economic Factors, and Epidemiological Factors nexus on Economic Impact: Machine Learning and Structural Equation Modelling Approaches”, Journal of the Nigerian Society of Physical Sciences (JNSPS) 3 (2021) 395. https://doi.org/10.46481/jnsps.2021.173

U. B. Mir, S. Sharma, A. K. Kar & M. P. Gupta, “Critical success factors for integrating artificial intelligence and robotics”, Digital Policy Regulation and Governance 22 (2020) 307. https://doi.org/10.1108/ DPRG- 03- 2020- 0032

M. A. Goralski & T. K. Tan, “Artificial intelligence and sustainable development”, International Journal of Management Education 18 (2020) 1. https://doi.org/10.1016/j.ijme.2019.100330

N. Malik, S. N. Tripathi, A. K. Kar & S. Gupta, “Impact of artificial intelligence on employees working in industry 4.0 led organizations”, International Journal of Manpower 2011 (2021). https://doi.org/10.1108/ijm-03-2021-0173

S. M. C. Loureiro, J. Guerreiro & I. Tussyadiah, “Artificial intelligence in business: State of the art and future research agenda”, Journal Business Research 129 (2021) 911. https://doi.org/10.1016/j.jbusres.2020.11.001

B. E. Boser, V. N. Vapnik & I. M. Guyon, “A Training Algorithm Mar- gin for Optimal Classifiers”, Proceedings of the fifth Annual Workshop on Computational Learning Theory (1992) 144. https://doi.org/10.1145/ 130385.130401

D. S. Kim & J. S. Park, “Network-based intrusion detection with support vector machines”, International Conference on Information Networking (2003) 747. https://link.springer.com/chapter/10.1007/978-3-540-45235-573

A. C. Enache & V. V. Patriciu, “Intrusionsdetectionbasedonsupportvec- tor machine optimized with swarm intelligence”, SACI 2014 - 9th IEEE International Symposium on Applied Computational Intelligence and Informatics (2014) 153. https://ieeexplore.ieee.org/document/6840052

Y. Li, L. Guo, Z. H. Tian & T. B. Lu, “A lightweight web server anomaly detection method based on transductive scheme and genetic algorithms”, Computer Communications 31 (2008) 4018. https://doi.org/10.1016/j.comcom.2008.08.009

K. C. Lin, Y. H. Huang, J. C. Hung & Y. T. Lin, “Feature Selection and Parameter Optimization of Support Vector Machines Based on Modified Cat Swarm Optimization”, International Journal of Distributed Sensor Networks 2015, (2015) 9. https://doi.org/10.1155/2015/365869

S. C. Chu & P. W. Tsai, “Computational intelligence based on the behaviour of cats”, International Journal of Innovative, Computing Information and Control 3 (2007) 163. https://api.semnticscholar.org/CorpusID: 58516637

M. Gomathy, “Optimal feature selection for speech emotion recognition using enhanced cat swarm optimization algorithm”, International Journal of Speech Technology 24 (2021) 155. https://doi.org/10.1007/s10772-020-09776-x

M. Orouskhani, M. Mansouri & M. Teshnehlab, “Average-inertia weighted cat swarm optimization”, Proceedings of Second International Conference on Advances in Swarm Intelligence (2011) 321. https://doi.org/10.1007/978-3-642-21515-5 38

S. Mahdavi, S. Rahnamayan & K. Deb, “Opposition based learning: A literature review”, Swarm Evolutionary Computation 39 (2018) 1. https://doi.org/10.1016/j.swevo.2017.09.010

P. W. Tsai, J. S. Pan, S. M. Chen, B. Y. Liao & S. P. Hao, “Parallel cat swarm optimization”, International Conference on Machine Learning and Cybernetics. ICMLC 6 (2008) 3328. https://doi.org/10.1109/ICMLC.2008.4620980

K. C. Lin & H. Y. Chien, “CSO-based feature selection and parameter optimization for support vector machine”, 2009 Joint Conferences on Pervasive Computing JCPC 2009 (2009) 783. https://doi.org/10.1109/JCPC. 2009.5420080

B. Santosa & M. K. Ningrum, “Cat swarm optimization for clustering”, SoCPaR 2009 - Soft Computing and Pattern Recognition (2009) 54. https://doi.org/10.1109/SoCPaR.2009.23

P. W. Tsai, J. S. Pan, S. M. Chen & B. Y. Liao, “Enhanced parallel cat swarm optimization based on the Taguchi method”, Expert Systems Applications 39 (2012) 6309. https://doi.org/10.1016/j.eswa.2011.11.117

M. Orouskhani, Y. Orouskhani, M. Mansouri & M. Teshnehlab, “A Novel Cat Swarm Optimization Algorithm for Unconstrained Optimization Problems”, International Journal of Information Technology and Computer Science 5 (2013) 32. https://doi.org/10.5815/ijitcs.2013.11.04

I. Hadi & M. Sabah, “Improvement Cat Swarm Optimization for Efficient Motion Estimation”, International Journal of Hybrid Information Technology 8 (2015) 279. https://dx.doi.org/10.14257/ijhit.2015.8.1.25

J. Manurung, H. Mawengkang & E. Zamzami, “Optimizing Support Vector Machine Parameters with Genetic Algorithm for Credit Risk Assessment”, Journal of Physics Conference Series 930 (2017) 8. https://doi.org/10.1088/1742-6596/930/1/012026

X. Nie, W. Wang & H. Nie, “Chaos quantum-behaved cat swarm optimization algorithm and its application in the PV MPPT”, Computational Intelligence and Neuroscience 2017, (2017). https://doi.org/10. 1155/2017/1583847

K. C. Lin, K. Y. Zhang, Y. H. Huang, J. C. Hung & N. Yen, “Feature selection based on an improved cat swarm optimization algorithm for big data classification”, Journal of Supercomputing 72 (2016) 3210. https://doi.org/10.1007/s11227-016-1631-0

P. Mohapatra, S. Chakravarty & P. K. Dash, “Microarray medical data classification using kernel ridge regression and modified cat swarm optimization based gene selection system”, Swarm Evolutionary Computation 28 (2016) 144. https://doi.org/10.1016/j.swevo.2016.02.002

M. Y. Cho & T. T. Hoang, “Feature Selection and Parameters Optimization of SVM Using Particle Swarm Optimization for Fault Classification in Power Distribution Systems”, Computational Intelligence and Neuroscience 2017 (2017). https://doi.org/10.1155/2017/4135465

Y. Kumar & G. Sahoo, “An improved cat swarm optimization algorithm based on opposition-based learning and Cauchy operator for clustering”, Journal of Information Processing Systems 13 (2017) 1000. https://doi.org/10.3745/JIPS.02.0022

A. Tharwat, A. Ella & B. E. Elnaghi, “A BA-based algorithm for parameter optimization of Support Vector Machine”, Pattern Recognition Letters 93 (2017) 13. https://doi.org/10.1016/j.patrec.2016.10.007

H. Siqueira, E. Figueiredo, M. Macedo, C. J. Santana, C. J. A. Bastos- Filho & A. A. Gokhale, “Boolean Binary Cat Swarm Optimization Algorithm”, 2018 IEEE Latin American Conference on Computational Intelligence LA-CCI (2018) 1. https://doi.org/10.1109/LA-CCI.2018.8625226

A. Tharwat, T. Gabel & A. E. Hassanien, “Parameter optimization of support vector machine using dragonfly algorithm”, Advances in Intelligent System and Computing 639 (2018) 309. https://doi.org/10.1007/978- 3- 319- 64861- 3 29

L. Pappula & D. Ghosh, “Cat swarm optimization with normal mutation for fast convergence of multimodal functions”, Journal of Applied Soft Computing 66 (2018) 473. https://doi.org/10.1016/j.asoc.2018.02.012

A. Tharwat, T. Gabel & T. Gabel, “Parameters optimization of support vector machines for imbalanced data using social ski driver algorithm”, Neural Computing and Applications 32 (2020) 6925. https://doi.org/10. 1007/s00521-019-04159-z

M. A. Nanda, K. B. Seminar, D. Nandika & A. Maddu, “A comparison study of kernel functions in the support vector machine and its application for termite detection”, Information 9 (2018). https://doi.org/10.3390/info9010005

S. Idris, O. Oyefolahan Ishaq & N. Ndunagu Juliana, “Intrusion Detection System Based on Support Vector Machine Optimised with Cat Swarm Optimization Algorithm”, 2nd International Conference of the IEEE Nigeria Computer Chapter (2019) 1. https://doi.org/10.1109/ NigeriaComputConf45974.2019.8949676

S. A. Mulay, P. R. Devale & G. V. Garje, “Intrusion Detection System Using Support Vector Machine and Decision Tree”, International Journal of Computer Applications 3 (2010) 40. https://doi.org/10.5120/758-993

B. Y. Sun, D. S. Huang, L. Guo & Z. Q. Zhao, “Support vector machine committee for classification”, International Symposium on Neural Network 3173 (2004) 648. https://doi.org/10.1007/978-3-540-28647-9 106

J. H. Min & Y. C. Lee, “Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters”, Expert System Applications 28 (2005) 603. https://doi.org/10.1016/j.eswa.2004.12.008

R. Amami, D. Ben Ayed & N. Ellouze, “Practical Selection of SVM Supervised Parameters with Different Feature Representations for Vowel Recognition”, Computer Science Bibliography 7 (2015) 418. https://doi.org/10.48550/arXiv.1507.06020

H. R. Tizhoosh, “Opposition-based learning: A new scheme for machine intelligence”, Proceedings of International Conference of Computational Intelligence Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06 1 (2005) 695. https://doi.org/10.1109/ CIMCA.2005.1631345

C. Huang & C. Wang, “A GA-based feature selection and parameters optimization for support vector machines”, Expert Systems with Applications 31 (2006) 231. https://doi.org/10.1016/j.eswa.2005.09.024

Published

2023-09-08

How to Cite

Hyper-parameter tuning for support vector machine using an improved cat swarm optimization algorithm. (2023). Journal of the Nigerian Society of Physical Sciences, 5(4), 1007. https://doi.org/10.46481/jnsps.2023.1007

Issue

Volume 5, Issue 4, November 2023

Section

Original Research
Copyright & Licensing

Copyright (c) 2023 Silifat Adaramaja Abdulraheem, Salisu Aliyu, Fatima Binta Abdullahi

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Hyper-parameter tuning for support vector machine using an improved cat swarm optimization algorithm. (2023). Journal of the Nigerian Society of Physical Sciences, 5(4), 1007. https://doi.org/10.46481/jnsps.2023.1007

Similar Articles

1-10 of 222

You may also start an advanced similarity search for this article.