Optimizing Initial Guesses for Nonlinear System Solvers Using Machine Learning: A Comparative Study of Classification Algorithms

International Journal of Computer Science and Engineering

© 2024 by SSRG - IJCSE Journal

Volume 11 Issue 12

Year of Publication : 2024

Authors : Iven Aabaah, Japheth Kodua Wiredu, Bakaweri Emmanuel Batowise

10.14445/23488387/IJCSE-V11I12P102

How to Cite?

Iven Aabaah, Japheth Kodua Wiredu, Bakaweri Emmanuel Batowise, "Optimizing Initial Guesses for Nonlinear System Solvers Using Machine Learning: A Comparative Study of Classification Algorithms," SSRG International Journal of Computer Science and Engineering , vol. 11,  no. 12, pp. 7-15, 2024. Crossref, https://doi.org/10.14445/23488387/IJCSE-V11I12P102

Abstract:

This paper focuses on the problem of improving initial guesses provided to solvers of nonlinear systems in terms of enhancing both convergence efficiency and reliability. A novel approach for constructing confidence models of initial guesses is proposed based on a Logistic Regression, Support Vector Machines (SVM), Random Forests, and K-Nearest Neighbors (KNN) classification schemes. Experimental evaluation across diverse nonlinear systems highlights Random Forests as the most effective model with an average accuracy of 81.69%, average precision – of 83.23%, average recall – of 82.16%, average F1 score of 82.69% and the highest AUC score equal to 0.90. Backed up by broad evaluation metrics, the above research inquiries mark the ideal potential of machine learning to revolutionize data processing by increasing solver adaptability, enhancing convergence patterns and economizing computations in scientific and engineering modalities.

Keywords:

Machine Learning algorithms, Nonlinear system solvers, Data pre-processing, Model Evaluation, Predictive modeling.

References:

[1] Richard H. Enns, and George C. McGuire, Nonlinear Physics with Mathematica for Scientists and Engineers, Springer Science & Business Media, pp. 1-695, 2012.
[Google Scholar] [Publisher Link]
[2] Aarav Chhabra, “Applied Mathematics: Building Theory and Practice,” International Journal for Research Publication and Seminar, vol. 15, no. 2, pp. 137-149, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Nasima Sawlat, Yalda Qani, and Naqibullah Sadeqi, “Numerical and Symbolic Analysis for Mathematical Problem-Solving with Maple,” Journal of Natural Science Review, vol. 2, no. 3, pp. 29-46, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Dipjyoti Nath et al., “Application of Machine Learning and Deep Learning in Finite Element Analysis: A Comprehensive Review,” Archives of Computational Methods in Engineering, vol. 31, pp. 2945-2984, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Zeynab Kaseb et al., “Power Flow Analysis using Quantum and Digital Annealers: A Discrete Combinatorial Optimization Approach,” Scientific Reports, vol. 14, no. 1, pp. 1-14, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Sergio Decherchi, and Andrea Cavalli, “Thermodynamics and Kinetics of Drug-Target Binding by Molecular Simulation,” Chemical Reviews, vol. 120, no. 23, pp. 12788-12833, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[7] S. Passone, P.W.H. Chung, and V. Nassehi, “Incorporating Domain-Specific Knowledge into a Genetic Algorithm to Implement Case Based Reasoning Adaptation,” Knowledge-Based Systems, vol. 19, no. 3, pp. 192-201, 2006.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Ali Al-Aradi et al., “Solving Nonlinear and High-Dimensional Partial Differential Equations Via Deep Learning,” arXiv, pp. 1-76, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Hayssam Dahrouj et al., “An Overview of Machine Learning-Based Techniques for Solving Optimization Problems in Communications and Signal Processing,” IEEE Access, vol. 9, pp. 74908-74938, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Maryam M Najafabadi et al., “Deep Learning Applications and Challenges in Big Data Analytics,” Journal of Big Data, vol. 2, pp. 1-21, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[11] William Grant Hatcher, and Wei Yu, “A Survey of Deep Learning: Platforms, Applications and Emerging Research Trends,” IEEE Access, vol. 6, pp. 24411-24432, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Olexandr Isayev et al., “Universal Fragment Descriptors for Predicting Properties of Inorganic Crystals,” Nature Communications, vol. 8, no. 1, pp. 1-12, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Jonathan Calver, Jienan Yao, and Wayne Enright, “Using Shooting Approaches to Generate Initial Guesses for ODE Parameter Estimation,” Recent Developments in Mathematical, Statistical and Computational Sciences, Waterloo, Canada, pp. 267-276, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Federico Toso, and Christie Maddock, “Initial Guess Generation Strategies for Spaceplane Trajectory Optimization,” Transactions of the Japan Society for Aeronautical and Space Sciences, pp. 1-8, 2017.
[Google Scholar] [Publisher Link]
[15] Haksun Li, “Heuristics,” Numerical Methods Using Java, pp. 625-654, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Ilias Theodorakos, and Michael Skipper Andersen, “Influence of the Initial Guess on the Estimation of Knee Ligament Parameters via Optimization Procedures,” Bioengineering, vol. 11, no. 12, pp. 1-12, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Manouchehr Sanei et al., “An Innovative Scheme to Make an Initial Guess for Iterative Optimization Methods to Calibrate Material Parameters of Strain-Hardening Elastoplastic Models,” Rock Mechanics and Rock Engineering, vol. 55, pp. 399-421, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Shitharth Selvarajan, “A Comprehensive Study on Modern Optimization Techniques for Engineering Applications,” Artificial Intelligence Review, vol. 57, no. 8, pp. 1-52, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Hajime Igarashi, Topology Optimization and AI-Based Design of Power Electronic and Electrical Devices, Academic Press, pp. 127-179, 2024.
[Google Scholar] [Publisher Link]
[20] Ramamani Tripathy et al., “Early Diabetes Prediction Using Supervised Machine Learning Techniques,” International Conference on Advances in Computing Research on Science Engineering and Technology, Indore, India, pp. 1-4, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Xiang Huang, “Predictive Models: Regression, Decision Trees, and Clustering,” Applied and Computational Engineering, vol. 79, pp. 124-133, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[22] K. Kalaiselvi et al., “Innovations in Natural Language Processing through Enhanced Linguistic Model Accuracy and Efficiency Using Advanced Reinforcement Learning Techniques,” Second International Conference on Advances in Information Technology, Chikkamagaluru, Karnataka, India, pp. 1-5, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Benjamin Ribba, “Reinforcement Learning as an Innovative Model-Based Approach: Examples from Precision Dosing, Digital Health and Computational Psychiatry,” Frontiers in Pharmacology, vol. 13, pp. 1-7, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Ilknur Gur Nalcaci et al., “Enhancing Sales Prediction Accuracy Through Incorporation of Expert Knowledge as Constraints Into the Model,” IEEE 18th International Conference on Application of Information and Communication Technologies, Turin, Italy, pp. 1-5, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[25] A. Sheik Abdullah et al., “Enhancing Predictive Analytics Through Hybrid Machine Learning Models: A Comparative Analysis Of SVM NB And Lasso-Ridge Techniques,” Second International Conference on Networks, Multimedia and Information Technology, Bengaluru, India, pp. 1-10, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Huaxiu Yao, “Towards Reliable Learning in the Wild: Generalization and Adaptation,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 20, pp. 22683-22683, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[27] Federico Matteucci et al., “Generalizability of Experimental Studies,” arXiv, pp. 1-16, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[28] Watchara Wongsa, Pikul Puphasuk, and Jeerayut Wetweerapong, “An Adaptive Differential Evolution Algorithm with a Bound Adjustment Strategy for Solving Nonlinear Parameter Identification Problems,” IT, Automation, Measurements in Economy and Environmental Protection, vol. 14, no. 2, pp. 119-126, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[29] Ziyi Huang, Henry Lam, and Haofeng Zhang, “Efficient Uncertainty Quantification and Reduction for Over-Parameterized Neural Networks,” Advances in Neural Information Processing Systems, vol. 36, 2023.
[Google Scholar] [Publisher Link]
[30] Alexey Orlov et al., “From High Dimensions to Human Comprehension: Exploring Dimensionality Reduction for Chemical Space Visualization,” Theoretical and Computational Chemistry, pp. 1-43, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[31] Jules S. Damji, and Michael Galarnyk, Considerations for Deploying Machine Learning Models in Production, Anyscale, 2021. [Online]. Available: https://www.anyscale.com/blog/considerations-for-deploying-machine-learning-models-in-production
[32] Isaac Azure et al., “AI-Enhanced Performance Evaluation of Python, MATLAB, and Scilab for Solving Nonlinear Systems of Equations: A Comparative Study Using the Broyden Method,” American Journal of Computational Mathematics, vol. 13, no. 4, pp. 644-677, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[33] Charu C. Aggarwal, Neural Networks and Deep Learning, Cham: Springer, 2023.
[CrossRef] [Google Scholar] [Publisher Link]