Data analysis: Imperatives for optimal decision making

The quality of any decision making process largely depends on the data system. In this wise, the triad of experimental design, analysis and decision making is vital for planning. The challenges of data collection, use and mis-use of statistical analyses, software interpretations and opportunities for optimal decision making as well as suggestions for new thinking are highlighted.

Illustrations explain the paper; they do not show study data.

AI-generated illustration

↗ Enlarge

The Data Analysis ChainClose ×

AI-generated conceptual illustration (A conceptual data analysis workflow represented by symbolic objects). It explains the idea and does not show study data.

Open full-size image ↗

Introduction

Data analysis has always dominated discourse among researchers, statisticians and all users of data in their day to day activities. In this wise, the refrain has always been that serious research requires serious statistics. However, the triad of experimental design, analysis and decision making which is a chain has not been much emphasised. Data analysis poses general challenges in the data system (i.e. all men and materials involved at all stages of experimental design, data collection and analysis). As noted by Afonja (1985) [1] , the statistician faces the challenge of providing a realistic solution to these problems by devising optimal methods of data collection and analysis compatible with the environment so that at the end, he not only minimises any uncertainty and hence falsehood but assesses the exact degree of such uncertainties. The interdisciplinary nature of statistics and its near universal application calls for utmost interaction between the statistician (as the consultant) and the experimenter/researcher (as the client) in all stages of experimentation and analysis. In all these, data analysis seems to play a central role for optimal decision making.

Data Analysis as Triad -What precedes data analysis is design of experiments and what succeeds it is optimal

decision. In this chain, the design of experiment embodies as follows: -Recognition and objective of the problem which involves stating and developing clear experimental objectives. -Choice of factors (treatments) which involves randomizing the factors and levels of factors, replication, responses, etc. -Choice of experimental design and experimentation which involves data collection and collation.

Data analysis embodies as follows

-Data cleaning, screening for missing observations, outliers, etc.

-Appropriate statistical methods for numerical accuracy which include graphical, exploratory methods, etc. -Analytical software-SPSS, SAS, STATA, R, GENSTAT, STATISTICA, etc.

Optimal decision embodies as follows -Inferences, recommendations, presentations. -Implementation of policy direction

This can be summarised in Nduka (1997) [7] Wrong design of experiments + right analysis  biased result Right design of experiments + wrong analysis  biased result Right design of experiment + right analysis  unbiased result

Challenges to Statisticians (Data Analysts)

We are living in the e-age where technology has made it easier to collect large data. This data explosion has thrown up its own challenges/implications in analysing and making sense of all the data collected. The following are also challenges: -Interdisciplinary nature of statistics calls for utmost interaction between statisticians and experimenters (researchers), Nduka (1997) [7] , Asiribo (1991) [4] . -Statistician-researcher-misconception. Some statisticians are more interested in the secondary aspect of research activity ie data analysis thereby being perceived as data "crunchers", also insufficient exposure to the rudiments of other disciplines. Again, some researchers are not involving the statistician all through the stages of experiments and often come with finished experimental data whose result of analysis may not conform to their expectation and may express disappointment with the results.

-Lack of basic knowledge of statistical principles and methods. -Dearth of statistician, lack of awareness and sufficient recognition of the role/importance of statistics, etc. -Student population explosion versus the shortage of personnel, poor training, retraining availability, computing software, etc. -Mis-use and abuse of statistical methods and software in data analysis.

Statistical Packages in Data Analysis

The existence and availability of an array of software packages has tremendously enhanced data analysis and ease computational difficulties in complex experiments and results. [11] . SIMULA (general package for simulation studies) SOLAS (general package for missing values in an experiment), etc.

University Data System

University ranks among the generators of large data, but ranks low in the use of data in its operations unlike others; internet providers, mobile communication networks, banks, airlines, etc, who make use of theirs in the competitive environment.

As the conception or misconception of its existence as a social institution, it is yet to tap into the use of data for its maximal operations. The Academic Planning Unit has the core mandate to collect, collate, analyse all data generated for the development of the university data bank for better service delivery, NUC (1996) [9] . The core mission is to develop a well-coordinated data system to guide policies for better decision making.

The primary generators of data in Nduka (2015) [8] . -Alternative/new approach to statistics training of less theory, more practice which by focusing on impact will flourish and transform statistics as a discipline to realise the vision of big data and produce the ''golden era of statistics'' (Rudin, et al, 2014) [10] . -Develop statistical games eg survey game to show the benefits of stratification, agricultural practices and yields, basic experimental designs, games theory, etc. -Start off as a general practice statistician and then specialty area of interest eg survey statistician, medical statistician, biometricians, etc. -Statistics to reformulate how research is to be done, team work (interdisciplinary eg statistics and computer science need each other) to tackle big problem (ie''all data revolution''). -Evolve new software packages to tackle emerging problems of ''big data'' and other complex problems in MDAs, industries, etc, because statistical analysis and data mining will be the hottest skills in future.

Conclusion

Data analysis will continue to play central role in our data system. In this wise, statisticians as main practitioners of data analysis need to appreciate their importance in statistical awareness, research, development, consultancy and guidance for better decision making. Major planning and decision making in government, academia, industry and general public in future will be data driven, hence the need to position ourselves for the challenges, opportunities, and benefit therein.

Training, retraining and interdisciplinary exposure hold the key to yawning gaps to meet the surging demands ahead. The 'large data' revolution calls for innovation, creativity, new skills, development of methodologies and software packages to analyse such data for meaningful impact in the society. Science of the future is data science and such future belongs to the statistician who as the major data analyst will be highly employable. The challenges, opportunities of 'big data' need to be appraised in ethical issues (personal privacy and information security), growing heterogeneity of data sources, undesigned or undersigned data in all ramifications.

1. Afonja B. Facts, Figures and Falsehood Syndrome in the Society. Inaugural lecture, university of Ibadan, 1985.
2. American Statistical Association More students earning statistics degrees not enough to meet surging demand in science. Daily, 2015, 1-5.
3. American Statistical Association: Success, Opportunities and Challenges for Statistics and Biostatistics in the Data Science: A report of July 2016 NSF-sponsored workshop for chairs of Departments of Biostatistics and Statistics, 2016, 1-15.
4. Asiribo OE. Statistical Consulting in Agriculture. Journ. Nig. Stat. Asso. 1991; 6:1-7. Https://bi-survey.com/big data.
5. Kettenring JR, Koehler KJ, Mckenzie JD. (Jr). Challenges and Opportunities for Statistics in the Next 25 years. The American Statistician. 2015; 69(2):86-9.
6. Lohr S. Hottest New Jobs: Statistics and Mathematics in Introduction to Data Analysis and Decision Making. Published: Cangage learning, 2010, 1-3.
7. Nduka EC. Statistics in Agricultural Research: the Nigerian Environment. Proc 12 th Ann. Conf. Farm Mgt Assoc. Nig, 1997, 129-133.
8. Rudin C, Dunson D, Irizarry JIH, Laber E, Leek J, Mc Cormick T et al. Discovery with Data: Leveraging Statistics with Computer Science To Transform Science and Society, Alexandria, VA: ASA, 2014, 87.
9. Stern R. Allen E, Barahona C, Dale I, Grayer C, Leidi S. SSC Statistical Resources Handbook, Statistical Services Center, University of Reading, U.K, 2005.