| Sign In to gain access to subscriptions and/or personal tools. |
Addressing Selection Bias in Dental Health Services Research1 Department of Pediatric Dentistry, School of Dentistry, CB 7450 Brauer Hall, and 2 Department of Health Policy and Administration, School of Public Health, University of North Carolina, Chapel Hill, NC 27599-7450, USA; Correspondence: * corresponding author, jessica_lee{at}dentistry.unc.edu
When randomization is not possible, researchers must control for non-random assignment to experimental groups. One technique for statistical adjustment for non-random assignment is through the use of a two-stage analytical technique. The purpose of this study was to demonstrate the use of this technique to control for selection bias in examining the effects of the The Supplemental Program for Women, Infants, and Children’s (WIC) on dental visits. From 5 data sources, an analysis file was constructed for 49,512 children ages 1–5 years. The two-stage technique was used to control for selection bias in WIC participation, the potentially endogenous variable. Specification tests showed that WIC participation was not random and that selection bias was present. The effects of the WIC on dental use differed by 36% after adjustment for selection bias by means of the two-stage technique. This technique can be used to control for potential selection bias in dental research when randomization is not possible.
Key Words: dental use • selection bias • endogeneity • WIC • non-randomization
Randomization remains the gold standard by which investigators control for selection bias. When a randomized controlled trial is not possible, researchers must control for non-random assignment to experimental groups, or estimates of the treatment or program effects can be biased. Propensity scores, a method used in some studies, can control only for selection on observable characteristics, and therefore cannot be used to control for selection on unobserved variables. Instead, investigators can use a two-stage technique developed by economists. This method relies on the availability of additional variables called ‘instrumental variables’, which induce variation in the main explanatory variable, but have no direct effect on the main outcome. Although this technique has become common in health services research as a method to deal with selection bias, also known as endogeneity (Rivers and Vuong, 1988; McClellan et al., 1994; Terza, 2005), it is rarely used in dental research. We demonstrate how this technique can be applied in dental research as an option when randomization is not feasible.
Issue of Endogeneity
Many public health programs are designed to help increase access to health services. Because of practical and ethical concerns, evaluation studies of their effectiveness are usually unable to rely on randomization; instead, many rely on observational or quasi-experimental designs. The possibility therefore exists that any observed effects of the intervention on dental use are due to selection bias, and not to the program or intervention itself. Under these conditions, a standard, single-stage analysis of the effect of the program on use of services will overestimate the program’s positive effects, and a case of endogeneity may arise. To address this design problem, we implemented a system of simultaneous equations that explicitly model participation in The Supplemental Program for Women, Infants, and Children’s (WIC), and the error correlation structure, in the empirical analysis for this study. Using the Anderson and Aday (1974) conceptual framework (Fig.
The Supplemental Program for Women, Infants, and Children (WIC) The WIC is administered by the Food and Nutrition Services of the US Department of Agriculture and serves over 7.4 million individuals (US General Accounting Office, 1992; North Carolina Food and Nutrition Services, 1999). The WIC is often the first point of entry into the health care system for many poor women and children and can improve the linkage between clients and health care providers, including dentists, through referrals and networking (Rush et al., 1988; Jones et al., 2000). Although previous studies of the WIC and health care utilization have noted a positive effect of WIC programs on the use of health services, most did not control for the non-random nature of WIC participation (Besharov and Germanis, 2001; Buescher et al., 2003). This paper presents the two-stage technique of controlling for selection bias during examination of the role of child WIC participation in the probability of having a Medicaid-reimbursed dental visit. We focused on the methodological issues arising from this analytical approach. Substantive findings for the effect of WIC on dental services use have been reported elsewhere (Lee et al., 2004).
Data Sources and Study Cohort This study used a longitudinal cohort design that examined children born in 1992 and followed them for five years until their fifth birthday in 1997. We used the following linked North Carolina (NC) administrative datasets for our investigation: composite birth records, Medicaid eligibility enrollment files, Medicaid dental claims, the WIC files, and the Area Resource File. The linkage process for these files has been reported previously, and a matching rate of 98.5% was established (Buescher et al., 2003). All children born in NC in 1992, and who were enrolled in the Medicaid program, were eligible for inclusion in the study. Children were excluded if they had more than one Medicaid ID in their records or if they had recorded periods of Medicaid enrollment indicated prior to the date of birth. A sample size of 49,795 was established. A Medicaid enrollment history was created for each child in which enrollment status was indicated for each month of life from birth to age five years (months 01–60). A dental visit was defined as having one or more dental claims filed through Medicaid.
Correcting for the Endogeneity of WIC— The Two-stage Method
First-stage OLS Regression (Table 2
The second stage used the residuals from the first-stage regression model to control for selection bias.
Second-stage Logit Equation (Table 3
WIC Participation was measured as the number of months in the WIC. Dental Visit was measured as the child having a Medicaid-reimbursed dental visit, and Control Variables included Medicaid enrollment, maternal age, maternal education, race, and dentist-per-population ratio. Instrumental variables included the number of full-time WIC clinics, multiple sites, and hours open per month.
Descriptive Statistics Of the 81,518 live births in North Carolina in 1992, 53,591 were enrolled in Medicaid, and 49,795 met the study inclusion criteria at birth. Our cohort was reduced to 21,277 at one year of age, because the eligibility for Medicaid changes from 185% of the Federal poverty level during the first year of life to 133% of the Federal poverty level thereafter. The average number of months per year enrolled in Medicaid was 7.6. More than 50% of the cohort was on the WIC at any time during the study period. The average length of child WIC participation was 4.4 mos per year. The average maternal age was 21 yrs, with an average educational level of 11th grade. Forty-eight percent of the population was non-white (Table 1  ).
Specification Testing Results Four specification tests supported the choice of the two-stage method as the preferred model for our study (Table 1  ). The first test applied was the test for endogeneity, to determine if participation in the WIC was endogenous in relation to the use of dental services. The error term from the first stage should not be a significant predictor in the second-stage model. The Hausman specification test, with the t statistic used on the coefficient of the predicted error, when included in the single equation, showed that the null hypothesis of ‘no endogeneity present’ was rejected (p < 0.01). These results indicated that WIC participation was endogenous (Table 3, column 2). Next, we examined the strength of the instrumental variables and for sufficient explanatory power in the first stage. The first-stage OLS regression (Table 2, column 1) indicated that the t test and F test for our proposed instrumental variables were significant (p < 0.05), and that we had strong instrumental variables, as long as they could be excluded from our second-stage dental use model. The two-stage method is preferred over a single-equation logit, if the R-squared in the first equation is greater than 0.3, or correlation between two error terms is large and the R-squared of the first equation is greater than 0.1, or if the percentage of explanatory variables in the first-stage regression that are instruments is greater than 0.25 (Bollen et al., 1995). Nearly one-third of the variables in our first equation were instruments, and the R-squared was 0.20. The next test examined whether the WIC instrumental variables belonged in the second-stage equation. The test for overidentifying restrictions failed to reject the null hypothesis that the overidentifying restrictions were valid at the 0.05 level, and indicated that the instrumental variables could be excluded from the main equation of WIC and dental use. The likelihood ratio test was constructed under the null that the overidentifying restrictions would be examined for validity. The results (Table 2, column 2) indicated that the instrumental variables could be excluded from the main equation. The collective results of our specification testing indicated that WIC participation was endogenous and that we had good instrumental variables. Accordingly, the first-stage residual values were added to our second-stage random-effects logit model for dental visits.
Effects of the WIC on Oral Health Services Use
Bias introduced by non-random design allocation can lead to either an over- or underestimation of treatment effects and can give misleading results (Deeks et al., 2003). Ioannidis and colleagues reviewed results from randomized control trials (RCT) and non-randomized studies (Ioannidis et al., 2001). In their findings across 45 topic areas and incorporating 240 RCTs and 168 non-randomized studies, they noted larger treatment effects more often in the non-randomized studies. The results from our single-equation method resulted in a predicted probability of 33% compared with the two-stage methods results, with predicted probability of 21%. The results of our study indicate that there would be an overestimate of results by 36% if selection bias were not controlled for in this evaluation of this public health program. Although our predicted probability results seem low, the literature reports that children under five yrs of age have difficulty accessing care, with reported use rates in the single digits (Edelstein et al., 2000; Mayer et al., 2000). Our study is the first to examine a public health program (WIC) and dental health services utilization using the two-stage statistical modeling approach. A strong criticism of previous WIC child health studies has been their inability to control for the potential selection bias of enrollment in the WIC program (Besharov and Germanis, 2001). We conducted extensive tests for these sources of bias in the relationship between WIC participation and the use of oral health services, and found that selection bias did exist. Random assignment of families to WIC participation would be a stronger design and would help overcome any selection bias. However, the implementation of this strategy in a community setting would be difficult, and such a design is not ethically defensible. Our study demonstrates the feasibility of using the two-stage analysis to control for selection bias when examining the effects of a public health program on use of dental services. Our results should also be considered in light of two major limitations. First, this study used claims data and can capture only the dental visits that were reimbursed by the Medicaid program. It has been well-documented that Medicaid children have disproportionately more dental disease than other children, and that they also have the most unmet dental needs (Davidoff et al., 2000; Newacheck et al., 2000). Therefore, the likelihood of Medicaid children getting care outside the program is low. Second, we recognize that the two-stage procedure can be done only if good instrumental variables exist, and their availability may be limited for some studies.
This research was conducted with the support of NIDCR Grant 1K22DE14743, AAPDF’s Omnii Fellowship, MCH Grant 6T83-MC-00015-11, AHRQ Grants T32-HS-00032 and 1-RO3-HS11607-01, and the USDA’s Food and Nutrition Services special projects grant (# GMD-OAE-97.017). A preliminary report was presented at the 80th General Session of the International Association for Dental Research Meeting, 2002, in San Diego, CA, USA. Received for publication April 14, 2004. Revision received May 9, 2005. Accepted for publication June 22, 2005.
Journal of Dental Research, Vol. 84, No. 10,
942-946 (2005)
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
|
 |
|
Sign In to gain access to subscriptions and/or personal tools.
TOP
ABSTRACT
INTRODUCTION
