Optimizing Products with Precision: A Review of Response Surface methodology

Introduction

Experimental design holds paramount significance in research dimensions of science and industry. Experimentation includes application of treatments applied to units or subjects under study and is then, a part of scientific method based on the measurement of one or more responses. It is necessary to well observe the process and the operations of the system. For this reason, to obtain reliable results, an experimenter must plan and design experiments and analyse the results. One of the most commonly used experimental designs for optimization is the response surface methodology (RSM) (Aydar, 2018) [3] . RSM is a mathematical and statistical method for optimizing the response of interest which is dependent on various variables (Montgomery DC, 2002) [21] . The parameters that affect the process are called independent variables, while the responses are called dependent variables (Koç B, Kaymak-Ertekin F, 2009) [15] . The response surface designs developed for factorial experiments with equi-spaced doses with a flexibility in choosing the levels of each of the factors can be used by agricultural and horticultural research, food technology, etc. where the existing designs cannot be used (Wu and Ding, 1991) [30] . These designs are capable to obtain design points with fewer runs if levels for each of the factors are large (Aggrawal and Bansal., 1998) [1] . Polynomial which adequately represent true dose response relation is called Response surface and the special class of designed experiments for fitting response surfaces is called response surface designs (Parsad et al., 2004).

When there is more than one response then it is important to find the compromise optimum that does not optimize only one response (Oehlert 2000) [24] . When there are constraints on the design data, then the experimental design has to meet requirements of the constraints. The second goal is to understand how the response changes in a given direction by adjusting the design variables. In the probabilistic analysis, an explicit or implicit functional relationship between input parameters and output response is required, which is normally difficult to establish except for simple cases and even the established functional relationship is sometimes too complicated to perform the conventional probabilistic analysis through integration or through first or second order derivatives (Morshedi and Akbarian, 2014) [18] . RSM is useful in three different techniques or (i) statistical experimental design, in particular two level factorial or fractional factorial design, (ii) regression modeling techniques, and (iii) optimization methods.

Statistical experimental design

Data for statistical studies are obtained by conducting either experiments or surveys. Experimental design is the branch of statistics that deals with the design and analysis of experiments. The methods of experimental design are widely used in the fields of agriculture, medicine, biology, marketing research, and industrial production. In an experimental study, variables of interest are identified. One or more of these variables, referred to as the factors of the study, are controlled so that data may be obtained about how the factors influence another variable referred to as the response variable, or simply the response. Also, these factors under consideration are not qualitative instead it is quantitative in nature similar to regression analysis. Usually, RSM uses either first order or second order designs for generation of design points. For linear response surface designs 2 n factorial designs are sufficient but a nonlinear response surface designs require 3 n factorial designs. The treatment combinations will be difficult to handle if the number of factors are more than 3 especially in case of a 3 n experiment. There comes the importance of a CCD with lesser design points as compared to a full or fractional factorial design.

Factorial design

A factorial experiment is an experimental strategy in which design variables are varied together, instead of one at a time. Here combinations of two or more levels of more than one factor are the treatments. It provides an opportunity to study not only individual effects but also their interactions (Das and Giri, 1979) [20] . In a 2 k factorial design, each control variable is measured at two levels, which can be coded to take the values, -1, 1, that correspond to the so-called low and high levels, respectively, of each variable. This design consists of all possible combinations of such levels of the k factors. Thus, each row of the design matrix D consists of all 1s, all -1s, or a combination of 1s and -1s and represents a particular treatment combination. In this case, the number, n, of experimental runs is equal to 2 k provided that no single design point is replicated more than once (Khuri and Mukhopadhyay, 2010) [13] . The 3 k factorial design consists of all the combinations of the levels of the k control variables which have three levels each. If the levels are equally spaced, then they can be coded so that they correspond to -1, 0, 1.

The number of experimental runs for this design is 3 k , which can be very large for a large k. Fractions of a 3k design can be considered to reduce the cost of running such an experiment (Montgomery., 2005) [16] . Also, CCD with fewer design points can also be used as an alternative for this design.

Central composite design

CCD are special type of factorial designs in which first-order (2 N ) designs augmented by adding centre and axial points for the estimation of curvature in a second-order model (Montogomery, 1997). CCD presents an alternative to 3 N designs commonly used for construction of second-order models because the number of experimental points are reduced (15 in the case of CCD compared to 27 for a 3 3 fullfactorial) (Box and Wilson., 1951) [4] . This design consists of the following three portions: A complete (or a fraction of) 2 k factorial design whose factors levels are coded as -1, 1. This is called the factorial portion, an axial portion consisting of 2k points arranged so that two points are chosen on the axis of each control variable at a distance of "a" from the design centre (chosen as the point at the origin of the coordinates system) and n c center points (Box and Wilson., 1951) [4] . Thus, the total number of design points in a CCD is the sum of number of axial points, number of central points and number of factorial points. Thus, this design is developed in a manner consistent with the sequential nature of a response surface investigation in starting with a first-order design, to fit a first-degree model, followed by the addition of design points to fit the larger second-degree model. The first-order design serves in a preliminary phase to get initial information about the response system and to assess the importance of the factors in a given experiment. The additional experimental runs are chosen for the purpose of getting more information that can lead to the determination of optimum operating conditions on the control variables using the second-degree model.

Regression modelling techniques

In the probabilistic analysis, an explicit or implicit functional relationship between input parameters and output response is required, which is normally difficult to establish except for simple cases and even the established functional relationship is sometimes too complicated to perform the conventional probabilistic analysis through integration or through first or second order derivatives. In such circumstances, we can use the concept of response surface methodology to establish an approximate explicit functional relationship between input variables (x 1 , x 2 , x 3 …) and output response (y) through regression analysis for the range of expected variation in the input parameters.

The above relationship can be simple linear or factorial model, or more complex quadratic or cubic model. 'e' represents other sources of uncertainty not accounted for in 'f'', such as measurement error on the output response, other sources of variation inherent in the process or the system, effect of other variables and so on (Myers and Montgomery, 2002).

Optimization

One of the main objectives of RSM is the determination of the optimum settings of the control variables that result in a maximum (or a minimum) response over a certain region of interest, R. This requires having a 'good' fitting model that provides an adequate representation of the mean response because such a model is to be utilized to determine the value of the optimum. Optimization techniques used in RSM depend on the nature of the fitted model. For first-degree models, contour plots are sufficient to determine the optimum value (Myers and Khuri, 1979) [20] . The first-degree model is usually used at the preliminary stage of a response surface investigation. Second-degree models are used after a series of experiments have been sequentially carried out leading up to a region that is believed to contain the location of the optimum response (Andre and Khuri 2010) [13] .

Steps involved in RSM 1. Model fiiting

The relationship between a set of independent variables and the response y is determined by a mathematical model called regression model. The relationship between the response variable y and independent variables is usually unknown. Mainly two models are used in response surface design, a first-order model and a second-order model (Hill and Hunder., 1970) [11] . In most RSM problems, the true response function f is unknown. In order to develop a proper approximation for f, the experimenter usually starts with a low-order polynomial. After various screening experiments the factors which is having a true relationship with response variables were selected. The regression coefficient are determined using principle of least square and linear model is fitted. (Box and Draper, 1987) [5] . In most of the response surface methods the design points (natural variables) generated after conducting the experiment are converted to coded variables as a mean for standardization. Thus, this processed data will be dimension less with mean zero and having constant variance. Beside this the coded data carry all the property of an orthogonal design. Such as minimum variance of the regression coefficient, regression coefficients are uncorrelated thus the problem of multicollinearity can be eliminated, levels of the corresponding variables are linearly independent etc. (Box and Wilson, 1951) [4] .

First-order response surface model

In general, the low-order polynomial model is used to describe the response surface f. A polynomial model is usually a sufficient approximation in a small region of the response surface. If the response can be defined by a linear function of independent variables, then the approximating function is a first-order model (Khuri and Cornel., 1996) [14] . A first-order model with n independent variables can be expressed as :

The response y is a function of the design variables x 1 , x 2 …, xn, denoted as f, plus the experimental error. A first-order model is a multiple-regression model and 𝛽 are regression coefficients.

Designs for Fitting the First-Order Model

First-order model is used to describe the flat surfaces that may or may not be tilted. This model is not suitable for analysing maximum, minimum, and ridge lines. The first-order model approximation of the function f is reasonable when f is not too curved in that region and the region is not too big. First-order model is assumed to be an adequate approximation of true surface in a small region of the x's (Montgomery 2005) [16] . Usually, a factorial design is sufficient for fitting a first order model.

Orthogonal First-Order Design

The experimenter needs to design a model to be efficient. For that reason, we have to take estimation of variances into consideration. The orthogonal first-order designs minimize the variance of the regression coefficients. A first-order design is orthogonal if the off-diagonal elements of the (X´X) matrix are all zero (Montgomery., 2005) [16] . Consequently, the cross-products of the columns of the X matrix sum to zero, the inverse matrix of (X´X) can be obtained easily, and all of the regression coefficients are uncorrelated. When the columns of the X matrix are mutually orthogonal then the levels of the corresponding variables are linearly independent.

Second-Order Response Surface

When there is a curvature in the response surface the firstorder model is insufficient. A second-order model is useful in approximating a portion of the true response surface with parabolic curvature. The second-order model includes all the terms in the first-order model, plus all quadratic terms and all cross-product terms (Hill and Hunder., 1970) [11] .

The second-order model is flexible, because it can take a variety of functional forms and approximates the response surface locally. Therefore, this model is usually a good estimation of the true response surface.

Designs for Fitting Second-Order Model

There are many designs available for fitting a second-order model. The most popular one is the central composite design (CCD). (Box and Wilson, 1951) [4] . It consists of factorial points (from a 2 q design and 2 q-k fractional factorial design), central points, and axial points. When a first-order model shows evidence of lack of fit, axial points can be added to the quadratic terms with more center points to develop CCD.

Orthogonal Central Composite Design

Occasionally, a central composite design may contain only one observation at each of the n f factorial points and 2q axial points, and with n c observations at the center. This design is known as Khuri and Cornell orthogonal if

Where "n" is the total number of observations and 𝑛 = 𝑛 𝑓 + 2𝑞 + 𝑛 𝑐 (Dean 1999) [8] . Orthogonal central composite design with only one observation is achieved by appropriate choice of a and 𝑛 𝑐 .

𝑎 = [

The statistic F is compared to the critical Fa, q, N-q-1, if observed F-value is greater than the critical F, then H 0 will be rejected. Equivalently, H 0 is rejected when P-value for the statistic F is less than significant level. How well the estimated model fits the data can be measured by the value of R 2 . (Sivakumar and Srivastavav, 2007). The R 2 lies in the interval [0, 1]. When R 2 is closer to the 1, the better the estimation of regression equation fits the sample data. In general, the R 2 measures percentage of the variation of y around y bar that is explained by the regression equation. However, adding a variable to the model always increased R 2 , regardless of whether or not that variable statistically significant. Thus, some experimenter rather using adjusted-R 2 . When variables are added to the model, the adjusted-R 2 will not necessarily increase. In actual fact, if unnecessary variables are added, the value of adjusted -R 2 will vary. Also, the contrast between the mean of the center points and the mean of the factorial points provides a test for the lack of fit in a 2 q design. The lack of fit of a first order model occurs when the model does not adequately represent the mean response as a function of the factor level (Dean., 1999) [8] . The residual sum of squares SSE can be partitioned into two components,

SSE = SSPE + SSLOF

where SSPE is the sum of squares due to the pure error and SSLOF is sum of squares due to the lack of fit. If lack of fit is found to be significant, we will reject null hypothesis and go for higher order model.

Optimization of fitted model

One of the main objectives of RSM is the determination of the optimum settings of the control variables that result in a maximum (or a minimum) response over a certain region of interest, R. (Plackett and Burman., 1946) [26] . This requires having a 'good' fitting model that provi des an adequate representation of the mean response because such a model is to be utilized to determine the value of the optimum. Optimization techniques used in RSM depend on the nature of the fitted model. For first-degree models, the method of steepest ascent (or descent) and contour plots are viable technique for sequentially moving toward the optimum response. In case of second order mode contour plot is not used for estimation of optimum point, since it will represent only an optimum area. So, location of optimum point become difficult. There another technique is used for locating the optimum point. The stationary point (maximum or minimum point) of a second order equation is the point where the first derivative of the function is zero, necessary condition for optimality. By equating the first derivative to zero we will reach to the turning points. Sufficient condition is that at least one of the even derivatives will be non-zero. By putting the value of turning points to the non-zero even derivative we can determine that whether it is a maximum or minimum point. If the second derivative is greater than zero then it will be a minimum point and if it is less than zero then that point will be a maximum point. (Hoerl., 1959; Draper., 1963) [12,9] .

Case study

For many years, the pharmaceutical and food industries have taken a keen interest in vegetable lecithin's (Pardun and fat, 1989) [25] . Optimization of extraction parameters by conventional methods is arduous and time consuming. Recently, interest in statistical experimental designs has been observed, especially in optimizing parameters in chemical processes and in the pharmaceutical industry (Cochran, and Cox., 1957) [7] . An experiment was conducted to determine optimal conditions for fractionation of rapeseed lecithin with alcohols (Sosada., 1993) [29] v. Deoiled rapeseed lecithin was fractionated with ethanol, and optimum conditions have been determined to improve purified lecithin yield and phosphatidylcholine (PC) enrichment. The effect of extraction time, solvent volume, ethanol concentration and temperature on the yield and the PC enrichment have been described in the form of regression equations. A full factorial experiment method and a second order orthogonal design were used in the study. The authors Dean and Voss give a data set for fractionation experiment that is conducted by M. Sosada (1993) [29] in their case study sets. In order to demonstrate when first-order model is adequate to the given data versus when it is not. For PC contentment in lecithin a first order model was found to be sufficient and a full factorial 2 4 experiment was performed for design point generation whereas yield of lecithin a second order model was found to be effective and used a central composite design for design point generation. The regression equations were calculated for the maximum value of the response functions optimized by an electronic data processing method, and the results (yield and PC enrichment calculated from regression equations) were compared with those obtained in control experiments. A comparison of the yield and PC enrichment values calculated and from experiments (for optimal parameters) showed differences below 10%. Taking the complexity of the natural lecithin used in the experiments into consideration, the resulting differences are small. The optimum value corresponds to four factors can determined using response surface methodology (Nuran, 2007). By drawing contour plot the optimum levels of factors for PC was determined and values concentrate near to one whereas for yield linear programming method for optimization was performed and optimum values where determined. (Nicholson and Pullen., 1969; Harrington., 1965) [23,10] . The use of calculated optimal parameters in fractionation of deoiled rapeseed lecithin leads to an 81-96% increment in yield and a 58% increase in PC enrichment.

Conclusion

A first-order model uses low-order polynomial terms to describe some part of the response surface. This model is appropriate for describing a flat surface with or without tilted surfaces. Usually a first-order model fits the data by least squares. Once the estimated equation is obtained, an experimenter can examine the normal plot, the main effects, the contour plot, and ANOVA statistics to determine adequacy of the fitted model. Lack of fit of the first-order model happens when the response surface is not a plane. If there is a significant lack of fit of the first-order model, then a more highly structured model, such as second-order model, may be studied in order to locate the optimum. There are many designs available to conduct a second-order design.

The central composite design is one of the most popular ones.

An experimenter can start with 2 q factorial point, and then add center and axial points to get central composite design.

Adding the axial points will allow quadratic terms to be included into the model. Second-order model describes quadratic surfaces, and this kind of surface can take many shapes. Therefore, response surface can represent minimum, ridge or saddle point. Contour plot is a helpful visualization of the surface when the factors are no more than three. When there are more than three design variables, it is almost impossible to visualize the surface. For that reason, in order to locate the optimum value, one can find the stationary point. Once the stationary point is located, either an experimenter can draw a conclusion about the result or continue in further studying of the surface.

References

1. Aggarwal ML, Bansal A. Robust response surface design for quantitative and qualitative factors. Commun Stat - Theory Methods. 1998;27(1):89-106.
2. Kh Andre, Siuli M. Response surface methodology. In: Computational Statistics. 2nd ed. New York: John Wiley & Sons; 2010. p. 128-149.
3. Aydar AY. Utilization of Response Surface Methodology in Optimization of Extraction of Plant Materials. Stat Appr with Emphasis on Design of Experiments Appl to Chem Processes. 2018.
4. Box GE, Wilson KB. On the experimental attainment of optimum conditions. J R Stat Soc Ser B. 1951;13(1):145.
5. Box GE, Draper NR. Empirical model-building and response surfaces. Wiley Series in Probability and Mathematical Statistics. New York: John Wiley & Sons; 1987.
6. Bradley N. The response surface methodology. (M.A.) thesis. Indiana University South Bend; 2007.
7. Cochran WG, Cox GM. Experimental Designs. 2nd ed. London: John Wiley & Sons; 1957.
8. Dean A, Voss D. Design and Analysis of Experiments. New York: Springer; 1999.
9. Draper NR. Ridge analysis of response surfaces. Technometrics. 1963;5:469-479.
10. Harrington EC Jr. The Desirability Function. Ind Qual Control. 1965;21(10):494-498.
11. Hill WJ, Hunter WG. A review of response surface methodology: A literature survey. Technometrics. 1970;8:571-590.
12. Hoerl AE. Optimum solutions of many variable equations. Chem Eng Prog. 1959;55:69-78.
13. Khuri AI, Mukhopadhyay S. Response surface methodology. Wiley Interdiscip Rev Comput Stat. 2010;2(2):128-149.
14. Khuri AI, Cornell JA. Response Surfaces. 2nd ed. New York: Marcel Dekker; 1996.
15. Koc B, Kaymak-Ertekin F. Response surface methodology and food processing applications. Gıda. 2009;7:1-8.
16. Montgomery DC. Design and Analysis of Experiments: Response surface method and designs. New Jersey: John Wiley & Sons; 2005.
17. Montgomery DC. Response Surface Methods and Other Approaches to Process Optimization. In: Montgomery DC, editor. Design and Analysis of Experiments. New York: John Wiley & Sons; 1997. p. 427-510.
18. Morshedi A, Akbarian M. Indian Journal of Fundamental and Applied Life Sciences. 2014;4(S4):2434-2439.
19. Myers RH, Montgomery DC. Response Surface Methodology: Process and product optimization using designed experiment. New York: Wiley-Interscience Publication; 2002.
20. Myers RH, Khuri AI. A new procedure for steepest ascent. Commun Stat -Theory Methods. 1979;8:13591376.
21. Myers RH, Montgomery DC. Response Surface Methodology. New York: John Wiley & Sons; 2002.
22. Myers RH, Montgomery DC. Response Surface Methodology: Process and product optimization using designed experiment. New York: John Wiley & Sons; 1995.
23. Nicholson TAJ, Pullen RD. Statistical and Optimization Techniques in the Design of Rubber Compounds. Comput Aided Design. 1969;1(1):39-47.
24. Oehlert GW. Design and analysis of experiments: Response surface design. New York: W.H. Freeman and Company; 2000.
25. Pardun H. Fat ScL Technol. 1989;9:45.
26. Plackett RL, Burman JP. The design of optimum multifactorial experiment. Biometrika. 1946;33:305323.
27. Prasad R, Srivastava R, Batra PK. Designs for fitting response surfaces in agricultural experiments. IASRI Publication; 2004.
28. Sivakumar Babu GL, Srivastava A. Reliability analysis of allowable pressure on shallow foundation using response surface method. Comput Geotechnics. 2007;34:187-194.
29. Sosada M. Optimal conditions for fractionation of rapeseed lecithin with alcohols. J Am Oil Chem Soc. 1993;70(4):405-410. doi:10.1007/bf02552715.
30. Wu CFJ, Ding Y. Construction of response surface designs for qualitative and quantitative factors. IIQP Research Report 91-03, University of Waterloo, Waterloo, Canada; 1991.