Comparison of linear discriminant analysis and logistic regression for classification diabetes data in Kirkuk Governate

Introduction

This study's main goal was to compare and evaluate the efficacy of LR and LDA, particularly in relation to the assumption of normalcy. Both LDA and LR are well-known statistical techniques that analyse data with categorical variables that are directly related to the result and create linear classification models. They have distinct core values, however. LDA usually assumes that the unknown variance-covariance matrix is the same for each category of the variable's grouping and employs predictors that are normally distributed. In general, LDA works well if these theories are true. However, if the data contain inconsistent covariance structures or do not adhere to the multivariate normality criterion, the effectiveness of LDA may be compromised. However, since LR does not rely on certain assumptions about the distribution of the data, it is regarded as a potent substitute for LDA. When handling data that is normally not dispersed in a traditional manner, LR is very helpful. Nonetheless, the computing time is often longer. Using actual datasets, the research seeks to provide useful insights into the efficacy of different approaches, particularly with reference to the normalcy assumption. It will also compare the time required to compute LR and LDA.

Mythology 2.1 Linear Discriminant Analysis

To identify the characteristics that distinguish one category from another and to develop classification models that forecast whether incoming data will fall into one of the categories, linear discriminant analysis may be used. It is assumed by linear discriminant analysis that the variables used to characterise each group have a comparable covariance matrix and are normally distributed. Two classes are distinguished via the simplest linear discriminant analysis. A linear function that goes through each class' centroid may be used to distinguish between the two classes. A more thorough discussion of linear discrimination analysis may be found in Kachigan (1991). The distribution of X|Y is assumed to be a multivariate normal distribution in the conventional linear discriminant analysis paradigm, with a shared covariance matrix Σ and a mean vector μy. We may show that we allocate x to group 1 in the following way by using algebraic equations.:

The values of the coefficients of α and β are

The likelihood of belonging to group 1 is higher than the likelihood of belonging to group 0. In actuality, the variables π1, π0, μ1, μ0, and Σ are unknown.

Logistic Regression

Finding the most accurate and useful model for the connection between a collection of independent factors (predictors or variables that have an impact on) and a dependent variable (end) is the goal of logistic regression (LR). This approach is regarded as reliable, adaptable, easy to use, and yields useful data. LR does not need the dependent variables' distribution to be normal, in contrast to Linear Discriminant Analysis (LDA). The opposite of what is often assumed is that both LDA and LR may be used when there are more than two classes. To be clear, we concentrate on the case when there are only two potential values for the outcome variable (Y). The following is a mathematical explanation of the LR model: Yi, a Bernoulli random variable, is unaffected by other variables. To find the model's parameters, the maximum likelihood technique is used.

Comparison Criteria

Two measures are used in this study to evaluate the efficacy of LR and LDA. The criteria that were used to evaluate the two approaches are described. To evaluate the efficacy of the classification model, the first statistic is the percentage of correct classification, which is computed using the formula below:

The percentage of accurate classification The classification error is extremely inaccurate and lacks any significant statistical significance, despite being the most simple and most commonly used metric for comparing methods.

The second-best metric, the B-index, is a supplementary metric that calculates the degree to which the predictions are correct. This is beneficial because it demonstrates the value of the model in recognizing different groups and having accurate predictions regarding them. The B-index is the average of the squared differences between the estimated and actual values. A B-index of 1 (between 0 and 1) is indicative of accurate predictions. The B-index formula is as follows:

In this case, Yi is the actual membership number (1 or 0), Pi is the chance of being placed in group I, and n is the size of the two populations. A B-index of 1 indicates a flawless forecast, and the value falls between 0 and 1 when two populations of equivalent size are subjected to random forecasts.

Application 3.1 Data Descriptions

Logistic regression and linear discriminant analysis were evaluated using the statistical software R, this was done in order to demonstrate the utilization of diabetes information for a case study. The data was obtained at Azadi Hospital in the Kirkuk Province. The population was selected from those who had been tested for diabetes according to the recommendations of the World Health Organization (WHO), and were at least 21 years old. The job of classification was to determine if the patient had diabetes or not. The World Health Organization's guidelines define 68 of the 152 individuals aged 21 years or greater with 5 predictors that were examined for diabetes (grade 1), while all other diseases were considered 0. Table 1 describes the procedure. In order to have the most effective model based on DA, we must analyze the data. Since a significant amount of 0.000 was observed in this instance, a discriminant analysis can be conducted using a progression of steps that will lead to the conclusion that the correlation function is effective in categorizing individuals as belonging to a particular population. Wilk's lambda F is documented in Table (3). It's obvious that the stepwise strategy in this example is to decrease the value of Wilk's lambda by each iteration. Additionally, it's vital to recognize that tolerance is the portion of the variance that is not explained by the other variables in the equation. The log-likelihood of the basic model (-2) with only the intercept is 682.430, according to the data in Table 4; this number drops to 491.783 when the individual variable is included. The study's chi-square probability is 0.00, which is not as significant as 0.05. There is no evidence to support the premise that the model with the independent variable is identical to the model without it. The evidence supports the relationship between the independent and dependent variables. Based on the aforementioned results, Table (6) demonstrates the estimated logistic regression parameters for the final model's parameters that were obtained using the maximum likelihood method. The probability of most variables is greater than or equal to 1. This research discusses the classification results of the discriminant analysis model and the logistic regression model (Table 6). Additionally, the study showed that logistic regression had a higher accurate classification rate than discrimination analysis. Furthermore, the B value of LDA was only 0.83, whereas the B value of LR was 0.92. In other words, LR produced a flawless classification and was more successful in this inquiry than LDA. This research discusses the classification results of the discriminant analysis model and the logistic regression model (Table 6). Additionally, the study showed that logistic regression had a higher accurate classification rate than discrimination analysis. Furthermore, the B value of LDA was only 0.83, whereas the B value of LR was 0.92. In other words, LR produced a flawless classification and was more successful in this inquiry than LDA.

Results

Conclusion

In this work, we compared the overall accuracy of two classification techniques, LR and LDA. Compared to linear discriminant analysis, which was more successful at categorising items as either C or D, logistic regression was more successful at classifying things as either A or B. This is because both approach is impacted by a uniform covariance matrix, and the data on diabetes do not conform to the premise of a multivariate normal distribution. Furthermore, the performance of both LDA and MLR is significantly impacted by the difference between the average values of the groups, and the classification accuracy rises as the value of grows. In relation to the number of classes, we found that as the number of classes rises, the classification accuracy of LDA and LR dramatically declines. The outcomes of both approaches are similarly impacted by the difference number of categories, with LR having a little edge due to its smaller category count. If the variables are distributed in several dimensions or in a regular manner throughout the group, the LR model seems to be suitable for the classification problem.

The logistic regression approach outperformed the discriminant analysis technique when our diabetes database was compared using multinomial logistic regression and linear discrimination. Given the prevalence of fabricated real data, it is important to take into account both individual and group features that might impair the LDA method's efficacy. The logistic regression model was much more successful in distinguishing between patients with and without diabetes, according to the ROC curve and AUC. Therefore, our findings show that in order to diagnose chronic illnesses like diabetes early, it is important to explore several methods for classifying support systems. To evaluate and enhance the effectiveness of these techniques in actual clinical practice, further investigation and clinical trials are required.

References

1. Olufemi I, Obunadike C, Adefabi A, Abimbola D. Application of logistic regression model in prediction of early diabetes across United States. ResearchGate. 2023 Jan.
2. Pohar M, Blas M, Turk S. Comparison of logistic regression and linear discriminant analysis: A simulation study. Metodoloski Zvezki. 2004;1(1):143.
3. Subba B, Biswas S, Karmakar S. Intrusion detection systems using linear discriminant analysis and logistic regression. In: 2015 Annual IEEE India Conference (INDICON). IEEE. 2015 Dec:1-6.
4. Zhao H, Lai Z, Leung H, Zhang X. Linear discriminant analysis. In: Feature Learning and Understanding: Algorithms and Applications. 2020:71-85.