Discriminant analysis for heart disease attributes

Introduction

Heart disease remains a significant contributor to global morbidity and mortality rates, posing substantial challenges to public health systems and individual well-being. Understanding the multifaceted nature of heart disease is paramount, necessitating a comprehensive grasp of the intricate interplay between various physiological factors and lifestyle choices in its onset and progression. Efforts in research aimed at unravelling these complexities are crucial for advancing medical knowledge and refining clinical interventions to alleviate the burden of heart disease. In assessing Heart disease health, researchers consider a range of factors, including Age, chest pain type, resting blood pressure, serum cholesterol levels, fasting blood sugar levels, resting electrocardiographic results, maximum heart rate achieved, ST depression induced by exercise relative to rest, the slope of the peak exercise ST segment, and the number of significant vessels colored by fluoroscopy. Acknowledging the significance of these factors, researchers and healthcare practitioners strive to elucidate their correlations with demographic variables to enhance risk prediction models and tailor interventions effectively. Discriminant analysis, a robust multivariate technique, offers a systematic approach to exploring the relationships between heart disease parameters and outcome characteristics. By identifying linear combinations of these parameters, discriminant analysis facilitates classifying individuals into distinct outcome groups based on observed characteristics. This analytical approach sheds light on the factors influencing heart disease biology and contributes to developing predictive models for disease risk stratification.

The main objectives of this work are as follows:

1. To identify linear combinations of heart disease parameters through discriminant analysis. 2. Determine the effectiveness of these combinations in discriminating between outcome groups. 3. Ascertain the relative importance of different heart disease parameters in distinguishing between disease and non-disease categories. 4. Assess the statistical significance of discriminant functions to evaluate differences among outcome groups based on heart disease parameters.

Review of literature

Durrant and Kaban (2010) [7] utilized a novel approach incorporating random projections with Fisher's Linear Discriminant (FLD) classifier for classification tasks. Unlike previous methods focusing solely on preserving pairwise distances under projection, the authors' approach emphasizes leveraging the inherent class structure within the data. This unique methodology does not impose sparsity or lowdimensional constraints on the dataset, allowing for more flexible and accurate classification. Through their research, the authors derive a tight upper bound on the estimated misclassification error over random projections, demonstrating improved generalization with larger training datasets. Moreover, they highlight that covariance misspecification errors are not exacerbated in the lowdimensional space, providing further evidence of the efficacy of their approach. Overall, the authors' development and utilization of this innovative technique offer significant advancements in classification analysis. Ramayah et al. (2010) [20] present a step-by-step example, making it easier for readers to comprehend the intricacies of discriminant analysis. They carefully explain the necessary assumptions and procedures involved in discriminant analysis, including data preparation, model estimation, and interpretation of results. Nainggolan et al. (2018) [16] used discriminant analysis and classified Hypertension women aged 27 to 54 years living in the village in the central district of Bogor. The results of the multivariate discriminant analysis showed that the level of Vo2 max is the only distinction maker in the incidence of hypertension. ALKubaisi et al. (2019) [1] used discriminant analysis with three criteria to test the developed model, producing excellent projecting precision. The discriminant function has properly assessed and classified about 67% of the cases in the analysis. Also, the study included two discriminant tasks: The first was explained by 77%, and the second was presented by 23% of the Variance. Dibal and Abraham (2020) [5] applied Fisher's linear Discriminant Analysis (FLDF) to health data on diabetic patients from the University of Port Harcourt Teaching Hospital, Rivers, Nigeria. He created a predictive discriminant model that classifies patients into one of two groups (Diabetic and Non-Diabetic). Fisher's linear discriminant function correctly classifies 65.4% of the total observation. Garate-Escamila et al. (2020) [9] proposed a dimensionality reduction method and identified pertinent features related to heart disease by applying feature selection techniques. We evaluated six ML classifiers for validation using data from the UCI Machine Learning Repository's Heart Disease dataset, consisting of 74 features and a label. Among these, the Chisquare and principal component analysis (CHI-PCA) coupled with random forests (RF) yielded the highest accuracy rates, achieving 98.7% for Cleveland, 99.0% for Hungarian, and 99.4% for Cleveland-Hungarian (CH) datasets. The ChiSqSelector technique derived features of anatomical and physiological significance, including cholesterol levels, maximum heart rate, chest pain indicators, features associated with ST depression, and heart vessel characteristics. Experimental findings underscored that combining chi-square with PCA demonstrates superior performance across most classifiers. Conversely, employing PCA directly from raw data yielded lower results, necessitating increased dimensionality for performance enhancement.

Ricciardi et al. (2020) [21] presented a comprehensive analysis of data mining techniques applied to a population of 10,265 individuals assessed for myocardial ischemia by the Department of Advanced Biomedical Sciences. With 22 features extracted, linear discriminant analysis (LDA) is utilized twice, employing the Knime analytics platform and R statistical programming language to classify patients into normal or pathological categories. The Knime analysis solely focuses on classification, while the R-based method incorporates principal component analysis (PCA) for feature creation before classification. Results indicate classification accuracies of 84.5% and 86.0%, respectively, with high specificity (>97%) and sensitivity (62-66%). This practical implementation demonstrates the utility of traditional data mining techniques in aiding clinical decision-making, leveraging PCA for feature reduction. Ndako et al. (2020) [17] investigated if haematological measurements could differentiate between typhoid-positive and negative paediatric patients. Using Fisher's Linear Discriminant Method, 200 patients were analyzed. A discriminant score threshold of 0.0067 was established, with patients above classified as unfavourable and below as positive. Classification efficacy was assessed using retribution estimate and leaving-one-out approaches, indicating a 75.8% and 74.7% prevalence for typhoid-positive patients, respectively. These findings suggest a high prevalence of typhoid fever among paediatric patients, emphasizing the need for improved point-of-care diagnostics with robust positive predictive value. Liberia et al. (2020) explored Discriminant Function Analysis (DFA) to evaluate the effectiveness of Indigenous health-andwellness programs, particularly in the Eeyou Istchee territory, Canada. By analyzing various health parameters, DFA models were developed to discriminate between individuals with and without Type 2 Diabetes Mellitus (T2DM). The models exhibited high specificity (~97%) in classifying non-T2DM individuals. This research underscores the potential of DFA in point-of-contact evaluations for monitoring and assessing health interventions in rural and remote Indigenous communities, providing valuable insights for T2DM management and prevention strategies among the James Bay Cree population. Ding et al. (2023) [6] introduced the Sparse Variables Selection Exponential Local Fisher Discriminant Analysis (SELFDA) model to address shortcomings in fault classification using Local Fisher Discriminant Analysis (LFDA). By automatically identifying key faulty variables through the minor absolute shrinkage and selection operator, SELFDA enhances fault diagnosis performance and model interpretability. It overcomes the Small Sample Size (SSS) problem by employing a matrix exponential strategy, ensuring full-rank within-class scatter matrices. This approach, tested on the Tennessee Eastman process and a real-world diesel working process, outperforms existing methods, demonstrating its effectiveness in practical industrial applications. Rahamneh et al. (2023) [19] utilized discriminant analysis to distinguish between two types of Bowel and Esophageal cancer in Jordan, identifying significant variables such as sex, weight, and Platelets Count P.C. The correct classification rates for the first and second groups were 62.8% and 77% respectively, with misclassification rates of 37.2% and 23%. The proper classification ratio was 71.6%, with a false classification ratio of 28.4%. The method effectively identified vital independent variables for diagnosing both cancer types, with correct classification probabilities of 66.4% and 77.6% for the first and second groups, respectively. Henry et al. (2023) [11] investigated the spectral differences of tobacco leaves under macronutrient deficiencies. They employed information entropy and spectral derivatives methods to identify the most effective wavelengths for discrimination. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) algorithms were utilized to reduce data dimensionality and classify the symptoms. The study's findings revealed that the overall accuracy for classifying young, intermediate, and mature plants was 92%, 82%, and 75%, respectively. The results also indicated that nitrogen, sulfur, and magnesium deficiencies significantly impacted the classification accuracy. In contrast, deficiencies in phosphorus and potassium had minimal effect on the classification outcomes.

Materials and Methods Material

The dataset under scrutiny comprises 303 meticulously gathered patient data records, incorporating 14 distinct features alongside the target variable, shedding light on the dynamics of heart disease. These features encapsulate vital health indicators and clinical observations, comprehensively portraying factors influencing heart disease health. The dataset, meticulously assembled and accessible for analysis, offers valuable insights into the intricate dynamics of heart diseases, facilitating advancements in medical research and clinical practice. The dataset is available on Kaggle (https://www.kaggle.com/code/desalegngeb/heart-disease-predictions), allowing researchers, clinicians, and enthusiasts to explore and analyze the data. Each entry in the dataset reflects meticulous data collection processes, ensuring accuracy and reliability for subsequent analysis and interpretation. This dataset is a valuable resource for exploring the multifaceted aspects of heart disease and advancing our understanding of its underlying mechanisms.

Methods

Box-Cox method

The Box-Cox method is used in statistics and econometrics to transform non-normal data into approximately normal data. It is named after statisticians George Box and Sir David Cox and was introduced in 1964. Let

Such that for unknown  ()

is the  transformed data, X is the design matrix (possible covariates of interest),  is the set of parameters associated with the  transformed data, and [24] .

Multivariate analysis of variance

A manova technique (Johnson & Wichern, (1998) [13] is employed to test the significance of variation among all the five parameters considered simultaneously. The MANOVA model for comparing the population means vectors is as follows:

Where, ij E is a vector of random error distributed as (0, )

Here, the parameter vector  is the overall mean and i V represents the model's status in (4); each component of the observation vector ij Y satisfies the univariate model, and the variance-covariance matrix  is the same for all populations.

Variance Inflation Factor

The variance inflation factor is used to measure how much the Variance of the estimated regression coefficient is inflated if the independent variables are correlated. VIF is calculated as

Where the tolerance is simply the inverse of the VIF; the lower the tolerance, the more likely the multicollinearity among the variables. The value of VIF=1 indicates that the independent variables are not correlated. If the value of VIF is 1 < VIF < 5, it specifies that the variables are moderately correlated. If the VIF value is above 5, there will be multicollinearity among the predictors in the regression model (Goldstein, (1993) [10] and Shrestha, (2020) [23] . Another one is the scatterplot graphical method that signifies the linear relationship between pairs of independent variables. It is essential to look for scatterplots that indicate a linear relationship between pairs of independent variables. The correlation coefficient is calculated using the formula:

Where r is the correlation coefficient, n is the number of observations, X represents the first variable in the context, and Y is the second variable in the context. If the correlation coefficient value is higher with the pairwise variables, it indicates the possibility of collinearity (Young, (2018) [25] .

Box's-M test

The Box's-M-test for homogeneity of covariance matrices, introduced in 1949, examines the covariance matrices derived from multivariate normal data considering one or more classification factors. This test assesses the similarity between the separate covariance matrices by comparing the product of their log determinants to the log determinant of the combined covariance matrix, similar to a likelihood ratio test. The test statistic employs a chi-square approximation.

Wilk's lambda

In discriminant analysis, Wilk's lambda is utilized to assess the contribution of each level of an independent variable to the model. This scale ranges from 0 to 1, where a value of 0 indicates complete discrimination, while a value of 1 signifies no discrimination. To test the impact of each independent variable, it is successively included and excluded from the model, generating a Λ statistic. The significance of the change in Λ is evaluated using an F-test; if the computed F-value exceeds the critical value, the variable is retained in the model (Onwukwe, (2014)) [18] . Thus, a non-significant Wilks' lambda value is always preferred.

B is the between-groups matrix, and W is the within-group matrix. The Eigenvalue can be explained as the ratio of the between-groups sum of squares to the within-group sum of squares (McGarigal et al., (2000)) [15] .

Linear Discriminant Analysis

Linear Discriminant Analysis (LDA) is an extension of discriminant analysis; it shares ideas and techniques with multiple analyses of Variance (MANOVA). LDA aims to classify cases into three or more categories using continuous or dummy categorical variables as predictors (Cramer, (2003) [4] , Jang et al., (2015)) [12] . The term DA (Fisher, (1936)) [8] refers to numerous types of analyses. DA is the most popular statistical technique to classify individuals or observations into non-overlapping groups based on scores derived from a suitable "statistical decision function" constructed from one or more continuous predictor variables.

While investigating the differences between the groups or categories, the necessary step is to identify the attributes with the most contributions to maximum reparability between known groups or categories to classify a given observation into one of the groups. For that purpose, DA successively identifies the linear combination of attributes known as canonical discriminant functions (equations) that contribute maximally to group separation. Predictive DA addresses the question of how to assign new cases to groups. The form of the Equation or function is:

Where D is an independent variable and i D is the value of discriminant score from the i th category (i=1, 2…n), j  is the discriminant coefficient of j th attributes (j=0,1,2,…,k), and ik X is the k th independent variable of i th category. This function is similar to a regression equation or function. There ' s  are unstandardized discriminant coefficients analogous to the ones in the regression equation. These ' s  maximize the distance between the means of the dependent variable, and the standardized discriminant coefficients can also be used, like beta weight in regression.

Results and Discussion

The descriptive statistics in

Conclusion

In this study, the Heart disease parameter dataset analysis reveals valuable insights into population characteristics. The dataset represents a diverse population and includes comprehensive information on heart disease parameters. Descriptive statistics highlight variability and distribution characteristics, while normality tests indicate the effectiveness of transformation methods. Also, this study highlights significant correlations among key heart disease parameters such as Age, blood pressure, cholesterol levels, and ST depression. Discriminant analysis effectively identifies combinations of these parameters that discriminate between outcome groups, enhancing our understanding of heart disease biology. Multivariate analysis reveals variations in heart disease parameters across demographics, emphasizing the importance of individual characteristics in risk assessment and intervention strategies. Normalizing data using the Box-Cox method ensures the validity of our analyses. The confusion matrix reflects the classification accuracy of a heart disease prediction model, achieving 72.9% overall accuracy in distinguishing between individuals with and without heart disease. These findings contribute to developing more accurate predictive models and targeted interventions to alleviate the burden of heart disease on public health and individual well-being.

References

1. AlKubaisi M, Aziz WA, George S, Al-Tarawneh K. Multivariate discriminant analysis managing staff appraisal case study. Acad Strategic Manag J. 2019;18(5):1-12.
2. Box GE. A general distribution theory for a class of likelihood criteria. Biometrika. 1949;36(3/4):317-346.
3. Box GE, Cox DR. An analysis of transformations. J R Stat Soc Ser B Stat Methodol. 1964;26(2):211-243.
4. Dibal PN, Abraham AC. On applying linear discriminant function to evaluate data on diabetic patients at the University of Port Harcourt Teaching Hospital, Rivers, Nigeria. Am J Theor Appl Stat. 2020;9(3):53-56.
5. Ding Z, Xu Y, Zhong K. Exponential Local Fisher Discriminant Analysis with Sparse Variables Selection: A Novel Fault Diagnosis Scheme for Industry Application. Machines. 2023;11(12):1066.
6. Fisher RA. The use of multiple measurements in taxonomic problems. Ann Eugen. 1936;7(2):179-188.
7. Escamila GAK, Hassani ELAH, Andrès E. Classification models for heart disease prediction using feature selection and PCA. Informatics in Medicine Unlocked. 2020;19:100330.
8. Goldstein R. Conditioning Diagnostics: Collinearity and Weak data in regression. J Oper Res Soc. 1993;35(1):8586.
9. Henry J, Veazie P, Furman M, Vann M, Whipker B. Spectral discrimination of micronutrient deficiencies in greenhouse-grown flue-cured tobacco. Plants. 2023;12(2):280.
10. Jang DH, Cook ACM, Kim Y. Graphical methods for the sensitivity analysis in discriminant analysis. Commun Stat Appl Methods. 2015;22(5):475-485.
11. Johnson RA, Wichern DW. Applied Multivariate Statistical Analysis. Biometrics. 1998;54(3):1203.
12. Liberda EN, Zuk AM, Martin ID, Tsuji LJ. Fisher's linear discriminant function analysis and its potential utility as a tool for the assessment of health-and-wellness programs in indigenous communities. Int J Environ Res Public Health. 2020;17(21):7894.
13. Nainggolan O, Tjandrarini DH, Indrawati L. Discriminant analysis to predict hypertension in women aged 25-54 years. Glob J Health Sci. 2018;10(10):93.
14. Ndako JA, Olisa JA, Ifeanyichukwu IC, Okolie CE, Ojo SK, Jegede SL. Predictive evaluation of pediatric patients based on their typhoid fever status using linear discriminant model. Med Hypotheses. 2020;144:110264.
15. Onwukwe CE, Ogbonna EN, Ayeni A. Application of Wilks' lambda and hotelling's T2 with manova on drug addiction and drug abuse data. Adv Life Sci Technol. 2014;24:53-59.
16. Rahamneh AAA, Jresat SS, Zubaidi F, Al-Hawary SIS. Using the linear discriminant analysis method to classify types of bowels and esophageal cancer in Jordan. Inf Sci Lett. 2023;12(3):1299-1305.
17. Ramayah T, Ahmad NH, Halim HA, Chiun MSRMZ. Discriminant analysis: An illustrated example. Afr J Bus Manag. 2010;4(9).
18. Ricciardi C, Valente AS, Edmund K, Cantoni V, Green R, Fiorillo A, et al. Linear discriminant analysis and principal component analysis to predict coronary artery disease. Health Informatics Journal. 2020;26(3):21812192.
19. Shrestha N. Detecting multicollinearity in regression analysis. Am J Appl Math Stat. 2020;8(2):39-42. DOI: 10.12691/jams-8-2-1
20. Vélez JI, Correa JC, Ramos MF. A new approach to the Box-Cox transformation. Front Appl Math Stat. 2015;1:12. DOI: 10.3389/fams.2015.00012