Bayesian approach to zero-inflated dynamic panel ordered probit model with an application in drug abuse

Introduction

Some categorical datasets are characterized by zero inflation due to non-participation decision and zero consumption of substances such as drug. For instance, the first zero may be recorded from a respondent who has inelastic demand for cigarettes while the second zero may be recorded from a respondent who is currently not smoking due to low income but are likely to become smoker in case their income is raised. Overlooking the sources of these zeros may leads to model misspecification and concealing differential effects explanatory variables have on the two zeros as reported by Gurmu and Dagne (2009) [7] . Harris and Zhao (2007) [8] proposed the Zero Inflated Ordered Probit (ZIOP) model that permit the splitting of the observed non-users into non-participants and zero consumption potential users. The study revealed that the use of OP model would confuse the effects of some significant covariates that have contrasting impacts on the two zeros. The ZIOP Model allowed for the recognition of covariates that are significant in each source of zero and are key to policy making. Gurmu and Dagne (2009) [7] extended ZIOP model to Bayesian approach and applied the model in modelling tobacco consumption. Oh et al. (2012) [13] proposed a Bayesian analysis of sequential ZIOP model and applied it to alcohol consumption in Korea. Their analysis showed that about 30% of non-drinkers are genuine non-drinkers. A study on the marginal effect of each covariate showed that certain covariates have effects on the genuine non-drinkers and potential drinkers in converse directions, which may not be easily revealed by an OP model.

According to Yong-Woo (2016) [18] , analysing the dynamics of individuals' health status involves the recognition of true state dependence in the persistence of the health status. The persistence arise from the tendency of the past health status being a significant determinant of the current health status. Therefore, the observed heterogeneity, unobserved heterogeneity and true state dependence should be cautiously structured in order to properly recognize how health in the subsequent period is accounted for by current health. According to Heckman (1981c) [9] , the poor treatment of unobserved individual effects gives rise to a conditional association between past and current experiences, which is referred to as spurious state dependence. To separate these dependences is significant in policy-making. For instance, if true state dependence is present in health evolution, then short-term health interventions tend to improve individual health in the long-term. However, if unobserved individual effect is related over time and not carefully controlled for, then the previous health status may seem to be a determinant of current health, solely because it is a proxy for such temporally persistent unmeasured variables. In this case, short-term health interventions have no effect on the longer-term health status. Contoyannis et al. (2004) [4] proposed the Dynamic panel ordered probit (DPOP) model based on Wooldridge (2005) [17] approach for initial observation and used it to study self-assessed health based on British Household Panel Survey for the 1991-1998 period. Wooldridge (2005) [17] suggested an unobserved individual effects conditional on the initial values and exogenous variables. Specifying the distribution of unobserved individual effects on these variables can lead to very tractable functional forms and consistent estimators for dynamic panel data with random effects. They found a substantial positive state dependence and unobserved individual effects. Including state dependence considerably lowers the impact of observed individual heterogeneity. Conditioning on the first period health outcomes and within-individual averages of the exogenous variables lowers the impact of heterogeneities and state dependence. In their model, unobservable individual effect accounts for around 30% of the unexplained variation in health. Yong-Woo (2016) [18] also proposed DPOP model based on Rabe-Hesketh and Skrondal (2013) [15] approach for initial observation and used it to study self-assessed health based on Korea Labour and Income Panel Study for the 2003-2012 period. Their results revealed that the health dynamics in the country are characterized by significant positive state dependence and 20% unobserved individual effect. The effect of many covariates vanishes when the state dependence and unobserved individual effects are controlled for. They also found a significant positive association between the initial period health observation and unobserved latent health. Therefore, this indicated that it is essential to control for self-assessed heath at the beginning of observations. The Semi-parametric Zero inflated dynamic panel probit model with selectivity proposed by Christelis and Galdeano (2009) [3] based on Akay (2009) [1] approach for initial conditions, non-parametric distribution for unobserved individual effects and known cut points may results in biased and inconsistent estimates. The constrained model proposed by Akay (2009) [1] and used by Christelis and Galdeano (2009) [3] to model the initial condition was shown by Rabe-Hesketh and Skrondal (2013) [15] to be severely biased. It implicitly sets the coefficients of the initial covariates equal to the coefficients for the subsequent periods, which is at odds with the form of the correct distribution. The main intention is that the conditional distribution of the unobserved effect, given the covariates at all periods (including the first observation), relies more directly on the initial-period explanatory variables than on the covariates at the other periods-in some cases it depends only on the initial-period covariates and the first response variable. The coefficients of the initial-period covariates should therefore not be constrained to be equal to the coefficients at the other periods. They showed that the bias for the constrained model practically vanishes when the initial period covariates are included as additional covariate or by using Wooldridge's original auxiliary model. The model cannot be applied in data with unknown cut points or without heaping. They ignored the association between the unobserved individual effects that is used to determine whether the factors affecting the unobserved individual effects in participation decision are the same as the one affecting the unobserved individual effects at consumption levels. The model could not allow estimation of the inter-unit correlation coefficients that is used to determine the latent error variance associated with the unobserved heterogeneity. They used simulated maximum likelihood approach proposed by Lee and Oguzoglu (2007) [11] and Kano (2008) [10] where the individual's effects are integrated out by computing the double integral by simulation. Mulkay (2015) [12] pointed that this procedure could be time-consuming and imprecise even though we use modern simulator like Geweke Hajivassiliou-Keane or Halton simulators, because we need to compute R cumulative density function with a large value of R in order to obtain sufficient precision in the log-likelihood function. This paper proposed Zero inflated dynamic panel ordered probit models with independent error terms (ZIDPOPI) and Zero inflated dynamic panel ordered probit models with correlated error terms (ZIDPOPC) that incorporates the state dependence, unobserved heterogeneities based on parametric distribution with selectivity, initial conditions based on Rabe-Hesketh and Skrondal (2013) [15] approach and unknown cut points. Since the solutions are not of closed form, Bayesian approach was used to estimate the parameters of the models since it allows sampling from the conditional distributions.

Zero Inflation Dynamic Panel Ordered Probit Model

Suppose b it y represents binary specification for either non-participation or participation for i th respondent at time y  denotes the inclination for participation (smoking, change, symptoms) and is given in vector form by,

 are vectors of unknown coefficients.  is the time invariant individual-specific fixed effect. To capture the state dependence,

where k  are cut points of ordinal response. In order to ensure that the cumulative distribution function for 0 it y is appropriately defined, we expect that 1 k kk    . We specify that

  for identification purpose and to avoid the handling of boundary parameters.

u is standard Gaussian random variable, the probabilities are given by

On the other hand, b it u and o it u could be assumed to be correlated since they correspond to the same individual. Thus, it y is given by,

The corresponding probabilities are given by

where (f, g; )   denotes the cumulative standardized bivariate normal distribution with correlation coefficient  between the two bivariate random elements.

Therefore, the chance of a zero observation has been ''inflated'' as it is a blend of the probability of ''non-participation'' from the split probit model and the probability of ''zero consumption'' from the OP process. This study assumed a balanced panel model where information about a respondents and required variables are reported at each wave. Rabe-Hesketh and Skrondal (2013) [15] specifies the individual time invariant error term 1i

The individual effects are considered, in every period, to be linear in the strictly exogenous covariates and the initial conditions. With Substituting equation ( 8) into equation ( 1) leads to a final underlying latent variable specification:

Substituting equation ( 9) into equation ( 2) leads to a final underlying latent variable specification:

ii were considered to be uncorrelated with each other, and independently and identically distributed over time and across individuals. They follows a bivariate normal distribution with mean zero and covariance matrices below.

Where 22 1 eb eo   for identification purpose. The entries of the covariance matrices are also estimated. All time dummies are ignored from i x and i z to evade the perfect collinearity. The estimates of 1 b h and 1 o h are significant as they reveal the association between the individual effect and initial observation.

Bayesian Approach to Zero inflated dynamic ordered probit model

Conditioned on the unobserved individual effects  , the responses  

h h y h x h x p y others y x w h h y h x h x y z v h h y

where  is the standard normal distribution function.

be a vector of unknown parameters. The prior distribution of the random effects  is assumed to have a hierarchical structure. Then, the prior distributions are specified as follows:

We have adopted a Bayesian hierarchical model approach for estimation and inference in the ZIDPOP models with exogenous variable selection. Therefore, we have to choose a prior for each parameter in the model. The prior specifications for most of the parameters are straightforward in the sense that we set them relatively non-informative and conjugate. The main motivation for employing the non-informative priors is that these priors would not dictate the inference of the parameters involved in the model. The Metropolis-Hastings algorithm was used due to the inability to sample from the full conditional distributions. The joint posterior distribution of the parameters conditional on the data was obtained by combining the likelihood function and the specified prior distributions via Bayes' theorem, as:    which are selected to make the distribution proper, that is, integrates to 1 but diffuse with large variances. The prior distributions are given by,

Then the full conditional posterior distribution required to implement the MCMC algorithm is given by,

The proposal distribution for each of the variables of interest given above was normal distribution that is symmetrical. The probability of acceptance is given by,

 then accept the proposal:

The prior for variance -covariance matrix

We assume non-informative normal priors for individual random effect. The prior for the individual random effect in binary and ordinal part was

The only unknown parameter in  is  , the correlation between

Full conditional posterior distribution required to implement the MCMC algorithm is given by,

others y x w h h y h x h x y x w h h y h x h x y

Finally, in choosing prior distributions for the threshold parameters, 's  , caution is needed because of the order restriction on them. Sampling from the complete conditional distribution for 's  results in slow mixing. Instead, we use Cowles' (1996) [5] algorithm to simulate ' s given by

. The first term denotes the contribution from the likelihood function. The second term accounts for the difference in the normalization of the proposal densities on the truncated normal intervals from which candidate points are drawn. When implementing this algorithm, the acceptance rate for the cut points should be monitored. If this rate falls too much below 25% or above 50%, MH  should be lowered or raised, respectively. The parameter were estimated using Single Component Metropolis-Hastings algorithm.

Model Selection

The model selection procedure was performed using Deviance Information Criterion (DIC) proposed by Spiegelhalter et al. ( 2002) [15] . It is given by

  is the posterior mean of the D(θ), and

 is the difference in the posterior mean of the deviance and the deviance evaluated at the posterior mean of the parameters. The model with a smaller DIC should be preferred to models with larger DIC. Models are penalized both by the value of D , which favours a better fit, but also by the effective number of parameters D p . The advantage of DIC compared to other criteria such as Akaike Information Criterion is that the DIC is simply obtained from the MCMC samples. DIC is estimated from posterior samples:

. L in the number of the iterations.

Average Partial Effects

Average partial effects (APEs) are used to indicate the size of between ordinal observation and the covariates. The average partial effects are computed by scaling the coefficient vector. The partial effects are averaged over the population distribution of heterogeneity and computed using the population averaged parameters a  . In the random effects specifications these are given by

. The sign of the APE has a clear qualitative interpretation, with a positive sign inferring a positive association with ordered response and vice versa. In the zero inflated dynamic ordered probit model it is possible to compute APEs for each of the two categories of participation and categories of consumption. We obtain the average partial effects of covariate

 

on various probabilities assuming the regression errors follow the bivariate normal distribution.

If  is a binary regressors, the marginal effect of  on probability, say P, is the difference in the probability evaluated at 1 and 0, conditional on observable values of covariates:     10 ii PP     . For continuous explanatory variables, the marginal effect is given by the partial derivative of the probability of interest with respect to i  , Wooldridge (2002) [17] shows that computing the partial effect at the observed values of the regressors for each observation and averaging the estimates over the observations provides a consistent estimate of the APEs. The average partial effects on the probability of participation was given by;

 is the probability density function of the standard normal distribution and i   was the coefficient in the inflation part associated with variable i  .

In terms of the zeros category, the effect on the probability of non-participation (zero inflation) was

represents the marginal effect on the probability of zero consumption.

The average partial effects on the probability of observing zero inflation was obtained as a sum of the marginal effects in ( 28) and (29); that is,

The average partial effects for the remaining choices are:

The asymptotic standard errors of the ,, k i t APE are computed using the Delta method as the square roots of the diagonal elements of

Results and Discussions 3.1 Data Simulation

This section describe the data generating process that was used to simulate the data. In order to generate the initial values problem, we operated the scheme for twenty-five periods before data were observed. The number of individuals was (n = 750), the initial period (s = -25) and the number of periods for the current simulation, so that each individual was observed from -25 to 36. We generated the right hand side variables by drawing values for the covariates, the individual fixed effects and the idiosyncratic errors for both binary and ordinal models. The initial outcome was assumed to follow a Bernoulli trial with probability 0.5 for Dynamic panel binary probit model. The initial outcome was assumed to follow a binomial trial with probability 0.5 for Dynamic panel ordered probit model. The following parameters were used for simulation. The parameters for binary model were fixed at 1 0.

To evaluate the accuracy of the estimates, which capture both bias and variability, Root Mean Square Error (RMSE) were calculated for each parameter. The RMSE was given by

. RMSE is always positive. A value of zero, though never achieved in practice, would indicate a perfect fit. In practice, a parameter with a lower value of RMSE is more accurate than one with a higher value.

Results for Bayesian Inference in Simulated Data

The MCMC simulation was run for 40,000 iterations. The first 10,000 samples were discarded as the burn-in period. The thinning interval was six. The acceptance rates were maintained between 20% and 30% except for the cut points. The acceptance rates were maintained between 25% and 50% for the cut points as proposed by Cowles (1996) [5] . There is a moderate correlation between the error terms in ZIDPOPC model of 0.431 implying that the factors affecting the participation decision are the same as the one affecting the consumption levels. There is a moderate correlation between the unobserved individual effects in ZIDPOPC model of 0.435 implying that the factors affecting the unobserved individual effects in participation decision are the same as the one affecting the unobserved individual effects at consumption levels. The variance of the individual effects in participation decision was 0.938. This indicated that 46.80% of the latent error variance is associated with unobserved individual effects, as evaluated by the intra unit correlation coefficient in participation decision. The variance of the individual effects for the decision on consumption levels was 0.924. This indicated that 46.06% of the latent error variance is attributable to unobserved individual effects, as evaluated by the intra-unit correlation coefficient at consumption levels. The Deviance Information Criteria (DIC) was given as a measure of goodness-of-fit. The model with the smallest DIC was considered to provide a better fit than the rest. The ZIDPOPC model clearly fitted the better than ZIDPOPI and DPOP models. The ZIDPOPI model clearly fitted the better than the DPOP model.

Assessing the Accuracy of the Estimators in the Models

The accuracy of the three models was compared using RMSE.

Assessing the Consistency of the Estimators in the Models

The Table 3 and Table 4 gives the results from the models when n=350 and T=10 and n=750 and T=10 respectively. The study compares estimates in Table 3 when n=350 and T=10 and Table 4 when n=750 and T=10. The results from these two tables indicated that as n increases from 350 to 750, the estimates tend to the true values. For instance, the true value of 𝜙 1 was 0.5, when n=350, 𝜙 1 was 0.532 for ZIDPOPC model and when n=750, 𝜙 1 was 0.517. This showed that as n was increased, the estimates tend to the true value. This indicated that ZIDPOPC model's estimators were consistent.

Application of the Models to Real Life Data

The main dataset used in this thesis was the National Longitudinal Survey of Youth 1997 (NLSY97). We eliminated the respondents with missing data for both responses and covariates. We constructed smoking intensities from three questions from the NLSY97. The zero consumption was created from two questions: "Have you ever smoked a cigarette?" and "During the last 30 days, how many days did you smoke a cigarette?" individuals who continuously answered "No" to the first question have perfectly inelastic demand for cigarettes, and for these individuals zero demands for cigarettes are optimal choices. Individuals who answer "None" to the second question belong to one of these two categories: those whose optimal choice of zero consumption are defined by corner solutions and infrequent smokers. A positive consumption was created from the question "When you smoked during the last 30 days, how many cigarettes did you usually smoke each day?" Let 𝑦 𝑖𝑡 be a measures of tobacco consumptions, by an individual 𝑖 at time 𝑡. 𝑦 𝑖𝑡 is number of sticks of cigarettes smoked on days that individual𝑖 smokes, including zero consumption. For example, in the drinking sample, 𝑖 = 1, 2, . . . , 2500 and 𝑡 = 0, 1, 2, . . . , 8. Describing the frequency distribution of the crude 𝑦 𝑖𝑡 measurements does not actually aid in condensing the smoking intensities for empirical analysis as some respondents reported to have smoked 99 sticks of cigarette on the days they smoked. Transforming the crude number of cigarettes smoked to 0-3 ordinal one-unit interval is popular in smoking literature. Thus, the study constructed four ordinal outcomes (0-3) of smoking intensities from their respective observed quantities𝑦 𝑖𝑡 .

This can be summarized as follows; The ordinal values 𝑦 𝑖𝑡 = 0, 1, 2, 3, stands for zero, low, moderate and high levels of smoking intensities.

Results for Bayesian Inference in Smoking Data

The MCMC simulation was run for 21,000 iterations. The first 3,000 samples were discarded as the burn-in period. The thinning interval was equal to four. The convergences were analyzed using trace and density plots. The acceptance rates were maintained between 20% and 30%. The acceptance rates were maintained between 25% and 50% for the cut points as proposed by Cowles (1996) [5] . Table 5 shows estimates standard errors and DIC values for the three models. The variance of individual effects for participation was 0.986. This indicated that 49.30% of the latent error variance is associated to individual effect, as evaluated by the intra-unit correlation coefficient in smoking decision. The variance of individual effects for consumption levels was 0.976. This indicated that 48.79% of the latent error variance is associated to unobserved individual effect, as evaluated by the intra-unit correlation coefficient at consumption levels. There was a weak correlation coefficient of 0.116 between the error terms. This implied that the variables affecting the participation are different from the one affecting consumption levels. There was a moderate correlation coefficient of 0.468 between the individual effects. This implied that the variables affecting the individual effects at participation were the same as the one affecting individual effects at consumption levels.

The DIC was given as a measure of goodness-of-fit. The model with the smallest DIC was considered to fit the data better than the rest. The ZIDPOPI model clearly fitted the data better than the ZIDPOPC and DPOP model. This is attributed to the very weak correlation between the error terms in ZIDPOPC model. The ZIDPOPC model clearly fitted the data better than the DPOP model.

References

1. Akay A. Finite-sample comparison of alternative methods for estimating dynamic panel data models. Journal of Applied Econometrics. 2012;27(7):1189-204.
2. Çağlayan-Akay E, Van MH. Determinants of the Levels of Development Based on the Human Development Index: Bayesian Ordered Probit Model. International Journal of Economics and Financial Issues. 2017;7(5):425-431.
3. Christelis D, Galdeano SA. Smoking persistence across countries: An analysis using semi-parametric dynamic panel data models with selectivity. Journal of Applied Econometrics. 2009;15:334-360.
4. Contoyannis P, Jones A, Rice R. The dynamics of health in the British household panel study. Journal of Applied Econometrics. 2004;19(4):473-503.
5. Cowles MK. Accelerating Monte Carlo Markov chain convergence for cumulative-link generalized linear models. Statistics and Computing. 1996;6(2):10-111.
6. Gurmu S, Dagne AG. Bayesian Approach to Zero-Inflated Bivariate Ordered Probit Regression Model, with an application to tobacco use. Journal of Probability and Statistics. 2012;5:1-26.
7. Gurmu S, Dagne GA. Bayesian Approach to Zero-Inflated Ordered Probit Models, with an Application. In Seminar on Bayesian Inference in Econometrics and Statistics. St. Louis: Washington University in St. Luis, 2009.
8. Harris MN, Zhao X. A Zero-inflated Ordered Probit Model, with an Application to Modelling Tobacco Consumption. Journal of Econometrics. 2007;141(2):1073-1099.
9. Kano S. Like husband, like wife: A bivariate dynamic probit analysis of spousal obesities. Osaka Prefecture University, 2008.
10. Lee WS, Oguzoglu U. Well-being and ill-being: A bivariate panel data analysis, IZA Discussion Paper Series No. 3108. 2007, 30.
11. Mulkay B. Bivariate probit estimation for panel data: a two-step Gauss Hermite quadrature approach with an application to product and process innovations for France. University's de Montpellier, Montpellier, 2015.
12. Oh M, Oh HT, Park S. Bayesian Analysis of Korean Alcohol Consumption Data Using a Zero-Inflated Ordered Probit Model. Korean Journal of Applied Statistics. 2012;25(2):363-376.
13. Spiegelhalter DJ, Best N, Carline B, Van DL. Bayesian method of model complexity and fit. Journal of the Royal Statistical Society. 2002;64:583-639.
14. Rabe-Hesketh S, Skrondal A. Avoiding biased versions of Wooldridge's simple solution to the initial conditions problem. Economics Letters. 2013;120(2):346-349.
15. Wooldridge JM. Simple solutions to the initial conditions problem in dynamic, nonlinear panel data models with unobserved heterogeneity. Journal of Applied Econometrics. 2005;20(1):39-54.
16. Wooldridge JM. Econometric Analysis of Cross Section and Panel Data. MIT Press: Cambridge, MA. 2002;108(2):245-54.
17. Yong-Woo L. State dependence, unobserved heterogeneity, and health dynamics in Korea. Hitotsubashi Journal of Economics. 2016;57(2):195-221.