Perovskite tetragonality modeling for functional properties enhancement using Newtonian search based support vector regression computational method

Tetragonality occurs as a result of stretching the crystal structural lattice of perovskite along one of its lattice vectors such that the three axes are mutually perpendicular with two of the axes having equal lengths. This tetragonality distortion easily triggers functional properties such as pyroelectricity, ferroelectricity, capacitance, and piezoelectricity among others, while synthesizing functional ceramics for a particular application. This work addresses and circumvents the challenges of experimental stress involved in functional ceramics synthesis by developing a Newtonian search-based support vector regression (GSB-SVR) model for perovskite tetragonality prediction using dopants concentration and ionic radii as the model predictors. The performance of the proposed GSB-SVR model is compared with the existing Kelvin & Rick model and better performance of 35.82% improvement based on mean absolute percentage error (MAPE) and 36.44% improvement based on mean absolute error (MAE) is obtained. The influence of lanthanides and zirconium incorporation on functional ceramics on the material tetragonality is also modeled by the developed GSB-SVR model. The metal in the lanthanide series considered includes lanthanum (La), praseodymium (Pr), Neodymium (Nd), and samarium (Sm). The obtained variation in their tetragonality follows the same trend as their variation in atomic numbers. Maximum distortion occurs between concentrations of 0.05 and 0.1, and each of the examined tetragonality distortions has a parabolic tetragonality distortion variation. Titanium and zirconium dopants were incorporated into the crystal lattice structure of Pb0.9Ba0.1(ZrxTi1−x)O3 and Pb0.9Ba0.1(Zrx−1Tix)O3. The tetragonality distortion in Pb0.9Ba0.1(ZrxTi1−x)O3 was observed to be minimum while Pb0.9Ba0.1(Zrx−1Tix)O3 perovskite show maximum tetragonality distortion. The observed tetragonality distortion can be utilized to enhance the functional properties of perovskite. The precision of the developed model, its easily fetched predictors, and its pre-laboratory ability to effectively and efficiently model the perovskite tetragonality are of high importance in tailoring and enhancing functional properties of materials for desired applications. DOI:10.46481/jnsps.2022.248


Introduction
The science of materials is basically centered on dealing with relationships between structure and property, with materials' composition being an important processing parameter [1]. Perovskites belong to a large structural family of com-20 pounds with crystal structures that are similar to calcium titanate (CaTiO 3 ). They are chemically represented as ABX 3 , where A and B are cations with A having a larger atomic radius than B and X is an anion which can be an oxide or a halogen. The presence of (XB2A4) coordination, cuboctahedral coordination (AX12), and octahedral (BX6) coordination shells is defined by assuming that the number of bonds between cation-anion and anion-cation are equal in any particular structure of a given perovskite. [2]. Perovskite compounds are extensively applied in modern devices because their variability in composition and structure produces several important properties such as ferroelectricity, superconductivity, piezoelectrics, pyroelectrics, and spin-dependent transport [3,4,5,6]. These important functional properties of perovskite can be easily induced by tetragonality distortion. Tetragonality occurs as a result of stretching a cubic lattice along one of its lattice vectors such that the three axes are mutually perpendicular and two of the axes have equal lengths. The tetragonal unit cell is characterized by a four-fold symmetry axis around which the atoms align with their initial positions by rotating the unit cell at an angle of 90 • . Designing materials with special properties such as ferroelectrics, piezoelectrics, high-temperature superconductors, high-K capacitors, and pyroelectrics for many applications premises on the precise measurement of tetragonality distortion [6,7]. This work aims to address the challenges associated with experimental stress in functional ceramics synthesis through the development of a Newtonian search-based intelligent model that allows for the prediction of tetragonality in perovskites using dopants concentrations and ionic radii of the perovskite constituents as the model predictors. Aside from the superiority of the proposed hybrid gravitational searched-based support vector regression (GSB-SVR) model as compared with the existing model in the literature [1], the developed GSB-SVR model can be implemented using easily fetched descriptors and foster pre-laboratory modeling. Support vector regression (SVR) is a mathematical learning theory derived from Vapnik-Chervonenskis theory of statistical learning [8]. SVR makes use of a collection of training data that includes predictor variables and their respective experimental targets to adequately construct a predictive model. With the aid of only predictors, the strength of SVR extends to prediction of future data. Because this model does implement input data probabilistic distribution for obtaining exact data, it is referred to as a nonparametric process. Using kernel function, the input data is reorganized and mapped into feature space where high precision modeling and simulation are performed. Support vector regression (SVR) gains wider applicability as a data mining algorithm to solve both linear and nonlinear problems [9]. Hyper-parameters of the SVR algorithm play crucial roles in actualizing a robust model as they are optimized using optimization algorithm that is based on Newtonian mechanics.
Rashedi, Nezamabadi-pour, and Saryazdi [10] introduced gravitational search algorithm based on gravity in 2009. It is an algorithm that is inspired by Newton's second law of motion as well as the law of gravity. It relies on the universal assumption of recognizing three kinds of mass namely: active/ passive gravitational mass and inertial mass [11]. In GSA, every agent is considered to be an object and the mass of each agent determines the individual performance of that agent. Each object possesses four pieces of information namely: passive/active gravitational mass, position, and inertial mass; all masses obey Newton's second law and the law of gravity [12]. Gravitational force compels the objects to attract one another and also makes every object with a lighter mass to be attracted in the direction of objects with greater masses [10] which represents a more promising solution [13]. The performance of every object within the algorithm is measured based on its mass. Better solutions are represented by heavier masses and they move more slowly than lighter masses [14].

Gravitational Search Algorithm
Gravitational Search Algorithm (GSA) is a type of swarm algorithm created from Newton's second law of motion and the law of gravity [10]. The basic mathematical principle of the gravitational principle is: F is force of gravitation, G is constant of gravitational, the masses of the objects are denoted by M 1 and M 2 , and R is the separation between objects. In this algorithm, objects are considered as agents and every object's performance is evaluated from their individual masses. The gravitational force makes objects to attract one another, forcing all objects to gravitate in the direction of objects with higher masses. This is because objects with heavy masses have better fitness values and provide better solutions to problems and their movement is not as fast as the objects with lighter masses [15]. Each object in the GSA is identified by its position, its inertial mass (Mii), active gravitational mass (Mai), and passive gravitational mass (Mpi). GSA search process starts by creating a population of N individuals at random in the search space. We begin the algorithm by defining the position of the jth agent as: The space dimension of the problem is n and x d j illustrates agent's jth position with dth dimension. Force acting on mass j from mass k at a given time t is defined, according to Newton's gravitational theory, as: where mass of agent j is M j , M k is mass of agent k, G(t) is constant of gravitation at time t, is a small constant and the Euclidian distance R jk (t) between j and k agents is defined as: In a d dimension, the sum of forces acting on j agent is assumed to a weighted sum of the dth component of forces of other agents calculated at random.
The total force is given as: rand k is a random number in the interval [0, 1]. According to Newton's law of motion, the acceleration (a d j (t)) of the agent j at time t in dth direction is defined by: where M is the inertial mass of the jth agent. The sum of an agent's current velocity and current acceleration determines its next velocity, given, respectively, as: where v d t (t) and x d j (t) represents the agent's velocity and position respectively. rand is a uniform random variable in the interval [0, 1] and is used to give a randomized feature to the search.
By reducing the randomly initialized constant of gravitation G with time, the search accuracy is controlled. This is to say that G is a function of time t and the initial value G is given as: Gravitational and inertial masses are computed using fitness evaluation. The efficiency of an agent in relation to the solution it represents is dependent on the heaviness of the mass of the agent. In other words, heavier masses will not move as fast as the lighter masses. The masses are updated using and with the following assumptions: where fit j (t) is the fitness value of agent j at time t, worst(t) and best(t) are defined as d in and respectively, for a minimization problem addressed in this contribution.

Support Vector Regression
In 1995,Vapnik and co-workers proposed the support vector machine (SVM) [8] which is a tool obtained from statistical learning theory for carrying out classification and regression tasks. SVM was initially developed for solving classification problems but it later evolved into solving regression problems [16]. Several implementations of SVM have been achieved in different areas of research after it was proposed [17,18]. SVM is therefore a universal term that can be subdivided into support vector classification (SVC) and support vector regression (SVR) [19]. SVC employs only one slack variable while SVR uses two slack variables. Both SVC and SVR employ very similar algorithms, the difference is the number of slack variables and types of variables they predict. In general, linear regression with SVR is defined as where w ∈ K and b ∈ R . SVR algorithm's main purpose is to find w and b in a way that is not exceeded in every training dataset. To achieve this, vector w must be minimal and equation (15) must be flat. Minimization of the Euclidean norm w 2 by means of a transformation to a convex optimization problem is needed to flatten equation (15) as shown in equation (16) below.
Constraints that may prevent the possibility of the convex optimized problem in equation (16) are added by the introduction of slack variables (ξ j and ξ * j ). New optimization problem is presented as where C is regularization or penalty factor.
Performance generalization of SVR model is determined by regularization factor C, the epsilon parameter , and the kernel option, which are carefully selected user-defined parameters. The regularization factor balances the difference between complexity of model and error tolerance in training data, with greater errors being equivalent to lower values of C and vice versa. The number of support vectors in the insensitive zones is regulated by the epsilon parameter, while input data transformation to a feature space of higher dimension is governed by the kernel option [20]. This study used gravitational search algorithm to choose the parameters discussed above.

Methodology and the employed computational strategies
This section contains the proposed hybrid model's computational approach as well as an overview of the dataset used.

Description of the dataset
Using experimental ionic radii data extracted from the literature [1], the proposed hybrid gravitational searched support vector regression (GSB-SVR) model was developed.

Computational details of the proposed hybrid model
This research paper develops a robust model with optimal functionality resulting from the combination of GSA and SVR algorithms. In this hybrid model, all the computational tasks were implemented on the MATLAB programming environment. Before starting modeling and simulation, existing data was randomized in order to facilitate equal distribution of data points and a more effective computation when the data is split into 80% and 20% training and testing set respectively. With the help of the developed real SVR algorithm codes, support vectors were created with existing training data points, and the generalization ability of the generated support vectors is corroborated by a testing set of data. GSA is aimed at finding the best hyper parameter values of SVR for excellent generalization of the developed model when applied to a collection of data not used in the training process.
The penalty or regularization factor (C), epsilon ( ), and the kernel option (σ)) of the optimum kernel functions are the SVR hyper-parameters optimized by GSA in each population. The search for optimal values of hyper-parameters starts with initialization of GSA agents which requires putting N agents in a search space, with each agent encoding SVR hyper-parameters. Following the initialization of the number of agents, every agent in the population was used to train SVR algorithm with the training set of data, and their fitness was evaluated with the testing set of data. Each agent's mass was calculated using equation (11). Equations (5) and (6) were used to compute the total gravitational force and acceleration of every agent. The computation of position and velocity of each agent was carried out and repeated until maximum iteration is reached using equations (7) and (8). As a result, hyper parameters of SVR were optimized with GSA and the GSA-SVR model was developed. The optimal hyper-parameter value obtained from an agent with the highest fitness value at maximum iteration is then used to train the GSA-SVR model. The following are the procedures for the developed hybrid GSB-SVR model.
Step I: Randomization of data and partitioning: For even distribution and more efficient computation during modelling, the available data is randomized. The data is partitioned into two sets, 80% for training and 20% for testing.
Step II: Choose a kernel function from the listed options.
Step III: GSA agent initialization: Fill a search space with N agents while each agent encodes SVR hyper-parameters. (penalty factor, epsilon and kernel option). Compute each agent's fitness in the following way: (a) Train SVR algorithm with each agent and a chosen kernel function. The number of SVR trained algorithm is equal to the initial number of agents. Using the training dataset, determine the performance measuring parameters (mean absolute percentage error (MAPE) and mean absolute error (MAE)) for each trained algorithm. (b) Compute MAPE and MAE using the testing set of data to determine each trained algorithm's generalization capacity. (c) Compute the fitness of MAPE and MAE for each agent, choose and save the trained algorithm with the minimum value.
Step IV: Use equation (11) to estimate the mass of every agent with the assumptions contained in equation (12).
Step V: Calculate the overall force of gravity and acceleration for each agent using equation (5) and equation (5).
Step VI: Determine the velocity and position of the agent. That is Step VII: The velocity and position of the agents are updated until maximum iteration of 100 is attained.
Step VIII: Repeat Step III-Step VII with a different kernel function.
Step IX: Save the optimum kernel function and hyperparameters for future implementation.

Results and discussion
This section presents the outcomes of the developed GSB-SVR model with inclusion of results of the implemented optimization algorithm. Comparison of estimates of the developed GSB-SVR with the existing model is also presented.

Optimization of SVR hyper-parameters
Optimization of SVR hyper-parameters is presented in Figure 1. The effect of the number of agents in enhancing exploration and exploitation capacities of Newtonian optimization algorithm is presented in the figure. Local convergence was attained when the number of the agent was set at ten. The local solution can be attributed to weak exploitation due to limited number of agents exploiting the search space. Exploitation capacity becomes enhanced as the number of agents was increased to thirty while the search space is well explored at this value. Increase in the number of agents above thirty results into high level of complexity within the search space since larger number of agents are exploring the search space and each agent experience strong gravitational pull which further weakens exploration and exploitation capacities of the algorithm. Optimum hyper-parameters were attained when the number of agents exploring the space was set at thirty as presented in Figure 1. Other investigated factors influencing the optimization strength of the implemented population-based algorithm include the initial values of the gravitational constant, maximum number of iteration and parameter alpha that influences the gravitational pull between the agents.   The significance of lanthanides inclusion on Pb 1−3x L 2x (Ti)O 3 perovskite is presented in Figure 4. The metal in the lanthanide series considered include lanthanum (La), praseodymium (Pr), Neodymium (Nd) and samarium (Sm). The obtained variation in the tetragonality of the investigated perovskite follows similar trend of the variation in atomic number as we move from lanthanum to samarium. The tetragonality distortion variation of each of the investigated tetragonality distortion shows a parabolic trend with maximum distortion between the concentrations of 0.05 to 0.1. This observed behavior of the tetragonality distortion coupled with the precision of the developed GSB-SVR model strongly indicates the tendency of the developed model in enhancing functional properties of perovskite as can be easily induced by the tetragonality distortion. Incorporation of titanium and zirconium dopants into crystal lattice structure of Pb 0.9 Ba 0.1 (Zr x Ti 1−x )O 3 and Pb 0.9 Ba 0.1 (Zr x−1 Ti x )O 3 perovskites, respectively using the developed GSB-SVR model is presented in Figure 5. The tetragonality distortion in Pb 0.9 Ba 0.1 (Zr x Ti 1−x )O 3 was observed to be minimum when Pb 0.9 Ba 0.1 (Zr x−1 Ti x )O 3 perovskite suffers maximum tetragonality distortion. Tetragonality in other perovskite compounds can be investigated using the developed model purposely to reveal the functional properties needed for a particular application.

Conclusion
This work presents hybrid gravitational search algorithm and support vector regression for modeling the tetragonality distortion of perovskite compounds using the ionic radii and the concentration of dopants as descriptors to the model. The performance of developed GSB-SVR model is compared with the existing Kelvin & Rick model, the developed model is found to perform better than the existing model in terms of the measured absolute percentage error (MAPE) as well as mean absolute error (MAE). The developed model was employed to establish the influence of lanthanides on tetragonality distortion of Pb 1−3x L 2x (Ti)O 3 perovskite and the functional material shows maximum tetragonality for lanthanum concentration between 0.7 and 0.8. The significance of titanium and zirconium dopants on perovskite were also investigated for functional properties induction. The precision and accuracy demonstrated by developed model, easy accessibility of its descriptors as well as pre-laboratory ability to effectively and efficiently model the perovskite tetragonality are of high importance in tailoring and enhancing functional properties of materials for desired applications.