Novel way to predict stock movements using multiple models and comprehensive analysis: leveraging voting meta-ensemble techniques

Akila Dabara Kayit; Mohd Ismail Tahir

doi:10.46481/jnsps.2024.2039

Authors

Akila Dabara Kayit
[email protected]

School of Mathematical Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang 04-653 2624 Penang, Malaysia | Universal Basic Education Commission, (UBEC) Gwani Street, Off IBB Way, Wuse Zone 4, Abuja, Nigeria
Mohd Tahir Ismail
School of Mathematical Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang 04-653 2624 Penang, Malaysia

Keywords:

Stock prediction, Machine learning, Voting, Meta-ensemble, Predictive modeling

Abstract

The research introduces a method for anticipating stock market patterns by combining machine learning techniques with analysis methods. Multiple machine learning algorithms were integrated to address the limitations of stock market forecasting models. Using web scraping techniques, data were gathered from the S&P500 index over seven years, from September 5, 2016, to August 5, 2023. Companies like Microsoft Corporation (MSFT), Amazon.com Inc. (AMZN), JPMorgan Chase & Co (JPM), and Tesla, Inc. (TSLA) were selected based on their inclusion in the S&P 500 index. LR, RF, SVC, ADAB, and XGBC algorithms were applied as models by utilising optimisation using grid search and single algorithm approaches. Voting methods were employed to combine predictions from these models. The study employed rigorous statistical analyses, including the Kruskal-Wallis test to assess overall differences, followed by Pairwise Dunn’s Test with Bonferroni Correction for detailed algorithm comparisons. Additionally, Bootstrapping was utilised to calculate Confidence Intervals (CI) for robust estimation of algorithm performance. The methodology covered data collection, preprocessing, model training, and performance assessment. The outcomes indicate that the proposed approach accurately forecasts stock trends precisely and dependably. This study contributes to refining stock market prediction methodologies by introducing a strategy that enhances prediction accuracy while offering investors and financial professionals insights. Furthermore, assessing algorithm performance across metrics and companies highlights the versatility and effectiveness of machine-learning approaches in the fields.

Dimensions

REFERENCES

J. Singh & A. Chhabra, “Indian stock markets data analysis and prediction using machine learning”, International Journal for Research in Applied Science & Engineering Technology (IJRASET) 8 (2020) 30996. www.ijraset.com.

A. Timmermann, “Elusive return predictability”, International Journal of Forecasting 24 (2008) 1. https://doi.org/10.1016/j.ijforecast.2007.07.008.

M. Yousefi, M. Fathi & F. Fogliatto, “Patient visit forecasting in an emergency department using a deep neural network approach”, Kybernetes 49 (2019) 2335. https://doi.org/10.1108/k-10-2018-0520.

V. Moscato, A. Picariello & G. Sperlı̀, “A benchmark of machine learning approaches for credit score prediction”, Expert Systems with Applications 165 (2021) 113986. https://doi.org/10.1016/j.eswa.2020.113986.

X. Zhang, Y. Huang, K. Xu & L. Xing, “Novel modelling strategies for high-frequency stock trading data”, Financial Innovation 9 (2023) 39. https://doi.org/10.1186/s40854-022-00431-9.

S. Singh, T. K. Madan, J. Kumar & A. K. Singh, “Stock market forecasting using machine learning: today and tomorrow”, in 2019 2nd International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), 2019. https://doi.org/10.1109/icicict46008.2019.8993160.

X. Y. Liu, J. Wu & Z. H. Zhou, “Exploratory undersampling for class-imbalance learning”, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39 (2009) 539. http://dx.doi.org/10.1109/TSMCB.2008.2007853.

R. Polikar, “Ensemble-based systems in decision making”, IEEE Circuits and Systems Magazine 12 (2012) 21. https://doi.org/10.1109/MCAS.2006.1688199.

J. Benhabib, X. Liu & P. Wang, “Financial markets, the real economy, and self-fulfilling uncertainties”, The Journal of Finance 74 (2019) 1503. http://www.jstor.org/stable/45097592.

M. M. Hasan, J. Popp & J. Oláh, “Current landscape and influence of big data on finance”, Journal of Big Data 7 (2020) 1. https://doi.org/10.1186/s40537-020-00291-z.

A. Halevy, P. Norvig & F. Pereira, “The unreasonable effectiveness of data”, IEEE Intelligent Systems 24 (2018) 8. http://www.computer.org/portal/cms_docs_intelligent/intelligent/homepage/2009/x2exp.pd.

S. Lessmann, C. Thies & D. Unger, “Machine learning for financial market prediction: review and outlook”, Expert Systems with Applications 114 (2018) 1. https://doi.org/10.1016/j.eswa.2018.07.038.

G. Sermpinis & R. Leote de Carvalho, “Algorithmic trading and market dynamics”, Computational Economics 53 (2019) 1415. https://doi.org/10.1007/s10614-018-9853-y.

I. K. Nti, A. F. Adekoya & B. A. Weyori, “A comprehensive evaluation of ensemble learning for stock-market prediction”, Journal of Big Data 7 (2020) 1. https://doi.org/10.1186/s40537-020-00299-5.

A. H. Bukhari, S. N. Rana, M. A. Bashir, A. M. Alqarni, F. S. Almehmadi & A. Masood, “Fractional neuro-sequential ARFIMA-LSTM for financial market forecasting”, IEEE Access 8 (2020) 1. https://doi.org/10.1109/ACCESS.2020.2985763.

M. R. Toochaei & F. Moeini, “Evaluating the performance of ensemble classifiers in stock returns prediction using compelling features”, Expert Systems with Applications 213 (2023) 119186. https://doi.org/10.1016/j.eswa.2022.119186.

S. Deng, Y. Zhu, Y. Yu, & X. Huang, “An integrated approach of ensemble learning methods for stock index prediction using investor sentiments”, Expert Systems with Applications 238 (2024) 121710. https://doi.org/10.1016/j.eswa.2023.121710.

S. Ardabili, A. Mosavi, & A. Várkonyi-Kóczy, “Advances in machine learning modelling reviewing hybrid and ensemble methods”, Preprints, 2020. Online. https://doi.org/10.20944/preprints201908.0203.v1.

E. Ampomah, Z. Qin, G. Nyame, & F. Botchey, “Stock market decision support modelling with tree-based Adaboost ensemble machine learning models”, Informatica 44 (2021) 4. https://doi.org/10.31449/inf.v44i4.3159.

C. Manjunath, B. Marimuthu, & B. Ghosh, “Analysis of nifty 50 index stock market trends using hybrid machine learning model in quantum finance”, International Journal of Electrical and Computer Engineering (IJECE) 13 (2023) 3. https://doi.org/10.11591/ijece.v13i3.pp3549-3560.

K. K. Yun, S. W. Yoon, & D. Won, “Prediction of stock price direction using a hybrid GA-XGBoost algorithm with a three-stage feature engineering process”, Expert Systems with Applications 186 (2021) 115716. https://doi.org/10.1016/j.eswa.2021.115716.

W. Wang, & Y. Wu, “Risk Analysis of the Chinese Financial Market with the Application of a Novel Hybrid Volatility Prediction Model”, Mathematics 11 (2023) 18. https://doi.org/10.3390/math11183937.

H. Song, & H. Choi, “Forecasting Stock Market Indices Using the Recurrent Neural Network Based Hybrid Models: CNN-LSTM, GRU-CNN, and Ensemble Models”, Applied Sciences 13 (2023) 7. https://doi.org/10.3390/app13074644.

P. R. Mohapatra, A. K. Parida, S. K. Swain, & S. S. Basa, “Gradient Boosting and LSTM Based Hybrid Ensemble Learning for Two-Step Prediction of Stock Market”, Journal of Advances in Information Technology 14 (2023) 6. https://www.jait.us/uploadfile/2023/JAIT-V14N6-1254.pdf

S. Ren, X. Wang, X. Zhou, & Y. Zhou, “A novel hybrid model for stock price forecasting integrating Encoder Forest and Informer”, Expert Systems with Applications 234 (2023) 121080. https://doi.org/10.1016/j.eswa.2023.121080.

S. Pyo, J. Lee, M.-H. Cha, & H. Jang, “Predictability of machine learning techniques to forecast the trends of market index prices: Hypothesis testing for the Korean stock markets”, PLoS ONE 12 (2017) e0188107. https://doi.org/10.1371/journal.pone.0188107.

Y. Yu, “A study of stock market predictability based on financial time series models”, Mobile Information Systems 2022 (2022) 8077277. https://doi.org/10.1155/2022/8077277.

I. Javid, R. Ghazali, I. Syed, M. Zulqarnain, & N. Husaini, “Study on the Pakistan stock market using a new stock crisis prediction method”, PLoS ONE 17 (2022) 10. https://doi.org/10.1371/journal.pone.0275022.

A. Bagga, & H. Patel, “Stock market forecasting using ensemble learning and statistical indicators”, Journal of Engineering Research 11 (2022) 384. https://doi.org/10.36909/jer.16629.

N. Rouf, M. B. Malik, T. Arif, S. Sharma, S. Singh, S. Aich, & H.-C. Kim, “Stock Market Prediction Using Machine Learning Techniques: A Decade Survey on Methodologies, Recent Developments, and Future Directions”, Electronics 10 (2021) 21. https://doi.org/10.3390/electronics10212717.

Y. Li, & Y. Pan, “A novel ensemble deep learning model for stock prediction based on stock prices and news”, International Journal of Data Science and Analytics 13 (2020) 139. https://doi.org/10.1007/s41060-021-00279-9.

F. Yang, Z. Chen, J. Li, & L. Tang, “A novel hybrid stock selection method with stock prediction”, Applied Soft Computing 80 (2019) 820. https://doi.org/10.1016/j.asoc.2019.03.028.

Z. Yu, H. Du, Y. Liu, W. Sang, F. Zhuang, J. Liu, & G. Kou, “Stock movement prediction based on bi-typed hybrid-relational market knowledge graph via dual attention networks”, ArXiv, 2022. Online. https://doi.org/10.48550/arxiv.2201.04965.

S. Bhanja, & A. Das, “A black swan event-based hybrid model for Indian stock markets? trends prediction”, Innovations in Systems and Software Engineering 20 (2022) 121. https://doi.org/10.1007/s11334-021-00428-0.

B. Alhnaity, & M. Abbod, “A new hybrid financial time series prediction model”, Engineering Applications of Artificial Intelligence 95 (2020) 103873. https://doi.org/10.1016/j.engappai.2020.103873.

M. Sedighi, H. Jahangirnia, M. Gharakhani, & S. F. Fard, “A novel hybrid model for stock price forecasting based on metaheuristics and support vector machine”, Data 4 (2019) 2. https://doi.org/10.3390/data4020075.

M. Sharaf, E. Hemdan, A. El-Sayed, & N. El-Bahnasawy, “An efficient hybrid stock trend prediction system during COVID-19 pandemic based on stacked-LSTM and news sentiment analysis”, Multimedia Tools and Applications 82 (2022) 16. https://doi.org/10.1007/s11042-022-14216-w.

M. Jiang, L. Jia, Z. Chen, & others, “The two-stage machine learning ensemble models for stock price prediction by combining mode decomposition, extreme learning machine and improved harmony search algorithm”, Annals of Operations Research 309 (2022) 553. https://doi.org/10.1007/s10479-020-03690-w.

Y. Lin, Z. Lin, Y. Liao, Y. Li, J. Xu, & Y. Yan, “Forecasting the realised volatility of stock price index: A hybrid model integrating CEEMDAN and LSTM”, Expert Systems with Applications 206 (2022) 117736. https://doi.org/10.1016/j.eswa.2022.117736.

G. T. Ribeiro, A. A. P. Santos, V. C. Mariani, & L. S. Coelho, “Novel hybrid model based on echo state neural network applied to predict stock price return volatility”, Expert Systems with Applications 184 (2021) 115490. https://doi.org/10.1016/j.eswa.2021.115490.

Y. Lin, S. Liu, H. Yang, & H. Wu, “Stock trend prediction using candlestick charting and ensemble machine learning techniques with a novelty feature engineering scheme”, IEEE Access 9 (2021) 101433. https://doi.org/10.1109/ACCESS.2021.101433.

N. Vo, & R. ?lepaczuk, “Applying hybrid ARIMA-SGARCH in algorithmic investment strategies on S&P500 Index”, Entropy 24 (2022) 2. https://doi.org/10.3390/e24020158.

M. Rashidpoor Toochaei, & F. Moeini, “Evaluating the performance of ensemble classifiers in stock returns prediction using effective features”, Expert Systems with Applications 213 (2023) 119186. https://doi.org/10.1016/j.eswa.2022.119186.

E. S. Solano, & C. M. Affonso, “Solar Irradiation Forecasting Using Ensemble Voting Based on Machine Learning Algorithms”, Sustainability 15 (2023) 10. https://doi.org/10.3390/su15107943.

A. Almulihi, H. Saleh, A. Hussien, S. Mostafa, S. El-Sappagh, K. Alnowaiser, & H. Monsef, “Ensemble learning based on hybrid deep learning model for heart disease early prediction”, Diagnostics 12 (2022) 12. https://doi.org/10.3390/diagnostics12123215.

M. Hasnain, I. Ghani, S. Jeong, & A. Ali, “Ensemble learning models for classification and selection of web services: a review”, Computer Systems Science and Engineering 40 (2022) 327. https://doi.org/10.32604/csse.2022.018300.

F. Pedregosa, É. Cauvet, G. Varoquaux, C. Pallier, B. Thirion, & A. Gramfort, “Learning to rank from medical imaging data”, Machine Learning in Medical Imaging 7588 (2012) 234. https://doi.org/10.1007/978-3-642-35428-1_29.

T. Hasanin, T. M. Khoshgoftaar, & J. L. Leevy, “A comparison of performance metrics with severely imbalanced network security big data”, IEEE 20th International Conference on Information Reuse and Integration for Data Science (IRI), 2019. https://doi.org/10.1109/iri.2019.00026.

A. Sudjianto, J. Qiu, M. Li, & J. Chen, “Linear iterative feature embedding: an ensemble framework for an interpretable model”, Neural Computing and Applications 35 (2023) 13. https://doi.org/10.1007/s00521-023-08204-w.

J. Liu, “Enhancing credit risk prediction based on ensemble tree-based feature transformation and logistic regression”, Journal of Forecasting 43 (2023) 429. https://doi.org/10.1002/for.3040.

E. Ampomah, Z. Qin, & G. Nyame, “Evaluation of tree-based ensemble machine learning models in predicting stock price direction of movement”, Information 11 (2020) 6. https://doi.org/10.3390/info11060332.

O. Moses, “A survey of SVC device: its principle of operation, advantages, and limitations”, Current Journal of Applied Science and Technology 40 (2021) 21. https://doi.org/10.9734/cjast/2021/v40i4331615.

J. Barbosa, & J. Filho, “A new methodology for SVC modelling in the power flow problem based on sigmoid functions”. Preprint, 2023. https://doi.org/10.21203/rs.3.rs-2877428/v1.

C. Piao, N. Wang, & C. Yuan, “Rebalance weights Adaboost-svm model for imbalanced data”, Computational Intelligence and Neuroscience 2023 (2023) 4860536. https://doi.org/10.1155/2023/4860536.

N. Mandela, A. Shaker, & F. Etyang, “Comparison of ensemble models for the classification of malicious URLs”, International Journal for Research in Applied Science and Engineering Technology 11 (2023) 4. https://doi.org/10.22214/ijraset.2023.50088.

S. Sang, F. Qu, & P. Nie, “Ensembles of gradient boosting recurrent neural network for time series data prediction”, IEEE Access 2021 (2021) 1. https://doi.org/10.1109/access.2021.3082519.

A. Gavito, D. Klabjan, & J. Utke, “Gradient-boosted based structured and unstructured learning”, ArXiv, 2023. Online. https://doi.org/10.48550/arxiv.2302.14299.

W. Wanishsakpong, K. Sodrung, & A. Thongteeraparp, “A comparison of nonparametric statistics and bootstrap methods for testing two independent populations with unequal variance”, International Journal of Analysis and Applications 21 (2023) 36. https://doi.org/10.28924/2291-8639-21-2023-36.

E. Jordan, D. Kinnamon, G. Haas, M. Hofmeyer, E. Kransdorf, G. Ewald, & W. Burke, “Genetic architecture of dilated cardiomyopathy in individuals of African and European ancestry”, JAMA 330 (2023) 5. https://doi.org/10.1001/jama.2023.11970.

S. Mokhtar, Z. Yusof, & H. Sapiri, “Confidence intervals by bootstrapping approach: a significance review”, Malaysian Journal of Fundamental and Applied Sciences 19 (2023) 1. https://doi.org/10.11113/mjfas.v19n1.2660.

H. Htun, M. Biehl, & N. Petkov, “Survey of feature selection and extraction techniques for stock market prediction”, Financial Innovation 9 (2023) 26. https://jfin-swufe.springeropen.com/articles/10.1186/s40854-022-00441-7.

L. N. Mintarya, J. N. M. Halim, C. Angie, S. Achmad, & A. Kurniawan, “Machine learning approaches in stock market prediction: A systematic literature review”, Procedia Computer Science 216 (2023) 96. https://doi.org/10.1016/j.procs.2022.12.115

O. Papaspiliopoulos, “high-dimensional probability: an introduction with applications in data science”, Quantitative Finance 20 (2020) 1591. https://www.tandfonline.com/doi/abs/10.1080/14697688.2020.1813475.

O. Kallenberg, Foundations of Modern Probability, Springer Cham, Switzerland, 2021. https://doi.org/10.1007/978-3-030-61871-1.

H. Zhang, Q. Zou, Y. Ju, C. Song, & D. Chen, “Distance-based support vector machine to predict DNA N6-methyladenine modification”, Current Bioinformatics 17 (2022) 473. https://doi.org/10.2174/1574893617666220404145517

A. Roy, & S. Chakraborty, “Support vector machine in structural reliability analysis: A review”, Reliability Engineering & System Safety 233 (2023) 109126. https://doi.org/10.1016/j.ress.2023.109126

G. F. Fan, L. Z. Zhang, M. Yu, W. C. Hong, & S. Dong, “Applications of random forest in multivariable response surface for short-term load forecasting”, International Journal of Electrical Power & Energy Systems 139 (2022) 108073. https://doi.org/10.1016/j.ijepes.2022.108073.

M. Keith, “Random forest”, in Machine learning with regression in Python, Apress, 2020. http://dx.doi.org/10.1007/978-1-4842-6583-3_5.

M. Durairaj, & B. G. K. Mohan, “A convolutional neural network based approach to financial time series prediction”, Neural Computing & Applications 34 (2022) 13319. https://link.springer.com/article/10.1007/s00521-022-07143-2.

Y. Qin, D. Song, H. Chen, W. Cheng, G. Jiang, & G. Cottrell, “A dual-stage attention-based recurrent neural network for time series prediction”, ArXiv, abs/1704.02971, 2017. Online. https://doi.org/10.48550/arXiv.1704.02971.

O. Sagi, & L. Rokach, “Ensemble learning: A survey”, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8 (2018) e1249. https://doi.org/10.1002/widm.1249.

J. Kumar, A. K. Singh, A. Mohan, & R. Buyya, “Ensemble learning”, in Encyclopedia of Biometrics, Springer link, 2019. https://api.semanticscholar.org/CorpusID:9963037.

G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, & T.-Y. Liu, “LightGBM: A highly efficient gradient boosting decision tree”, in Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA, 2017, pp. 3146-3157. https://dl.acm.org/doi/10.5555/3294996.3295074.

M. Middlehurst, J. Large, M. Flynn, J. Lines, A. Bostrom, & A. Bagnall, “HIVE-COTE 2.0: a new meta ensemble for time series classification”, Machine Learning 110 (2021) 3211. https://link.springer.com/article/10.1007/s10994-021-06057-9.

Y. T. Cao, Y. Pruksachatkun, K.-W. Chang, R. Gupta, V. Kumar, J. Dhamala, & A. Galstyan, “On the intrinsic and extrinsic fairness evaluation metrics for contextualized language representations”, ArXiv, 2022. Online. https://doi.org/10.48550/arXiv.2203.13928.

D. Powers, “Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation”, ArXiv, 2011. Online. https://doi.org/10.48550/arXiv.2010.16061.

D. H. Chau, G. Tech, M. Roozbahani, J. Heer, J. T. Stasko, & C. Faloutsos, “Ensemble Methods”, in Machine learning with Spark™ and Python®, Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2018, pp. 449-523. https://www.cambridge.org/core/books/abs/predictive-statistics/ensemble-methods/07444D43CBFF927B8411D8DB857A9D48.

I. Naruei & F. Keynia, “A new optimization method based on COOT bird natural life model”, Expert Systems with Applications 183 (2021) 115352. https://doi.org/10.1016/j.eswa.2021.115352.

S. Faghani, B. Khosravi, K. Zhang, M. Moassefi, J. M. Jagtap, F. Nugen, S. Vahdati, S. P. Kuanar, S. M. Rassoulinejad-Mousavi, Y. Singh, D. V. Vera Garcia, P. Rouzrokh, & B. J. Erickson, “Mitigating Bias in Radiology Machine Learning: 3. Performance Metrics”, Radiol Artif Intell 4 (2022) e220061. https://doi.org/10.1148/ryai.220061.

M. Nabipour, P. Nayyeri, H. Jabani, S. S. Shahab, & A. H. Mosavi, “Predicting Stock Market Trends Using Machine Learning and Deep Learning Algorithms Via Continuous and Binary Data; a Comparative Analysis”, IEEE Access 8 (2020) 150199. https://api.semanticscholar.org/CorpusID:221283286.

C. H. Bhavani, et al., “Cervical Cancer Prediction using NGBFA Feature Selection Algorithm and Hybrid Ensemble Classifier”, International Journal on Recent and Innovation Trends in Computing and Communication 11 (2023) 390. https://doi.org/10.17762/ijritcc.v11i9.8820.

Y. Ge, Q. Wang, B. Zheng, X. Zhuang, Q. Li, C. Shen, & C. Wang, “Anti-Distillation Backdoor Attacks: Backdoors Can Really Survive in Knowledge Distillation”, Proceedings of the 29th ACM International Conference on Multimedia, 2021. https://api.semanticscholar.org/CorpusID:239012191.

I. H. Ji, J. H. Lee, M. J. Kang, W. J. Park, S. H. Jeon, & J. T. Seo, “Artificial Intelligence-Based Anomaly Detection Technology over Encrypted Traffic: A Systematic Literature Review”, Sensors 24 (2024) 898. https://doi.org/10.3390/s24030898.

C. Smith, F. Teng, E. Branch, S. Chu, & K. S. Joseph, “Maternal and Perinatal Morbidity and Mortality Associated With Anemia in Pregnancy”, Obstetrics & Gynecology 134 (2019) 1234. 10.1097/AOG.0000000000003557.

Y. Liu, S. Goudreau, T. Oiamo, D. Rainham, M. Hatzopoulou, & H. Chen, “Comparison of land use regression and random forests models on estimating noise levels in five Canadian cities”, Environmental Pollution 256 (2020) 113367. https://doi.org/10.1016/j.envpol.2019.113367.

H. Hewamalage, C. Bergmeir, & K. Bandara, “Recurrent neural networks for time series forecasting: Current status and future directions”, International Journal of Forecasting 37 (2021) 388. https://doi.org/10.1016/j.ijforecast.2020.06.008.

R. Amin, R. Yasmin, S. Ruhi, M. H. Rahman, & M. S. Reza, “Prediction of chronic liver disease patients using integrated projection-based statistical feature extraction with machine learning algorithms”, Informatics in Medicine Unlocked 36 (2023) 101155. https://doi.org/10.1016/j.imu.2022.101155.

I. Nitzke & J. Vrabec, “Numerical discrimination of thermodynamic Monte Carlo simulations in all eight statistical ensembles”, Journal of Chemical Theory and Computation 19 (2023) 3460. https://doi.org/10.1021/acs.jctc.3c00252.