Logistic Regression

Built using Zelig version 5.1.0.90000

Logistic Regression for Dichotomous Dependent Variables with logit.

Logistic regression specifies a dichotomous dependent variable as a function of a set of explanatory variables.

Syntax

With reference classes:

z5 <- zlogit$new()
z5$zelig(Y ~ X1 + X ~ X, weights = w, data = mydata)
z5$setx()
z5$sim()

With the Zelig 4 compatibility wrappers:

z.out <- zelig(Y ~ X1 + X2, model = "logit", weights = w, data = mydata)
x.out <- setx(z.out)
s.out <- sim(z.out, x = x.out, x1 = NULL)

Examples

Basic example

Load Zelig and attach the sample turnout dataset:

library(Zelig)

## Loading required package: survival

data(turnout)

Estimating parameter values for the logistic regression:

z.out1 <- zelig(vote ~ age + race, model = "logit", data = turnout,
                cite = FALSE)

Summarize estimated paramters:

summary(z.out1)

## Model: 
## 
## Call:
## z5$zelig(formula = vote ~ age + race, data = turnout)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.9268  -1.2962   0.7072   0.7766   1.0723  
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.038365   0.176920   0.217 0.828325
## age         0.011263   0.003053   3.689 0.000225
## racewhite   0.645551   0.134482   4.800 1.58e-06
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 2266.7  on 1999  degrees of freedom
## Residual deviance: 2228.8  on 1997  degrees of freedom
## AIC: 2234.8
## 
## Number of Fisher Scoring iterations: 4
## 
## Next step: Use 'setx' method

Set values for the explanatory variables:

x.out1 <- setx(z.out1, age = 36, race = "white")

Simulate quantities of interest from the posterior distribution:

s.out1 <- sim(z.out1, x = x.out1)
summary(s.out1)

## 
##  sim x :
##  -----
## ev
##           mean         sd       50%     2.5%     97.5%
## [1,] 0.7481377 0.01176792 0.7482215 0.723684 0.7707342
## pv
##         0    1
## [1,] 0.25 0.75

Show the results graphically:

plot(s.out1)

First differences

Estimating the risk difference (and risk ratio) between low education (25th percentile) and high education (75th percentile) while all the other variables held at their default values.

z.out2 <- zelig(vote ~ race + educate, model = "logit", data = turnout,
                cite = FALSE)
x.high <- setx(z.out2, educate = quantile(turnout$educate, prob = 0.75))
x.low <- setx(z.out2, educate = quantile(turnout$educate, prob = 0.25))
s.out2 <- sim(z.out2, x = x.high, x1 = x.low)
summary(s.out2)

## 
##  sim x :
##  -----
## ev
##           mean         sd       50%      2.5%     97.5%
## [1,] 0.8232843 0.01008723 0.8240251 0.8028227 0.8415789
## pv
##          0     1
## [1,] 0.182 0.818
## 
##  sim x1 :
##  -----
## ev
##           mean         sd       50%   2.5%     97.5%
## [1,] 0.7098712 0.01293696 0.7098027 0.6843 0.7353405
## pv
##          0     1
## [1,] 0.257 0.743
## fd
##            mean         sd        50%       2.5%       97.5%
## [1,] -0.1134131 0.01194812 -0.1131622 -0.1360887 -0.08955555

plot(s.out2)

Model Definition

Let \(Y_i\) be the binary dependent variable for observation \(i\) which takes the value of either 0 or 1.

The stochastic component is given by

\[\begin{aligned} Y_i &\sim& \textrm{Bernoulli}(y_i \mid \pi_i) \\ &=& \pi_i^{y_i} (1-\pi_i)^{1-y_i}\end{aligned} \]

where \(\pi_i=\Pr(Y_i=1)\).

The systematic component is given by:

\[\pi_i \; = \; \frac{1}{1 + \exp(-x_i \beta)}.\]

where \(x_i\) is the vector of \(k\) explanatory variables for observation \(i\) and \(\beta\) is the vector of coefficients.

Quantities of Interest

The expected values for the logit model are simulations of the predicted probability of a success:

\[ E(Y) = \pi_i= \frac{1}{1 + \exp(-x_i \beta)}, \]

given draws of \(\beta\) from its sampling distribution.

The predicted values are draws from the Binomial distribution with mean equal to the simulated expected value \(\pi_i\).
The first difference for the logit model is defined as

\[\textrm{FD} = \Pr(Y = 1 \mid x_1) - \Pr(Y = 1 \mid x).\]

The risk ratio is defined as

\[\textrm{RR} = \Pr(Y = 1 \mid x_1) \ / \ \Pr(Y = 1 \mid x).\]

In conditional prediction models, the average expected treatment effect (att.ev) for the treatment group is

\[ \frac{1}{\sum_{i=1}^n t_i}\sum_{i:t_i=1}^n \left\{ Y_i(t_i=1) - E[Y_i(t_i=0)] \right\}, \]

where \(t_i\) is a binary explanatory variable defining the treatment (\(t_i=1\)) and control (\(t_i=0\)) groups. Variation in the simulations are due to uncertainty in simulating \(E[Y_i(t_i=0)]\), the counterfactual expected value of \(Y_i\) for observations in the treatment group, under the assumption that everything stays the same except that the treatment indicator is switched to \(t_i=0\).

In conditional prediction models, the average predicted treatment effect (att.pr) for the treatment group is

\[ \frac{1}{\sum_{i=1}^n t_i}\sum_{i:t_i=1}^n \left\{ Y_i(t_i=1) - \widehat{Y_i(t_i=0)}\right\}, \]

where \(t_i\) is a binary explanatory variable defining the treatment (\(t_i=1\)) and control (\(t_i=0\)) groups. Variation in the simulations are due to uncertainty in simulating \(\widehat{Y_i(t_i=0)}\), the counterfactual predicted value of \(Y_i\) for observations in the treatment group, under the assumption that everything stays the same except that the treatment indicator is switched to \(t_i=0\).

Output values

The Zelig object stores fields containing everything needed to rerun the Zelig output, and all the results and simulations as they are generated. In addition to the summary functions demonstrated above, use standard R utility functions such as coef, vcov, predict to extract model estimates and zelig_qi_to_df to extract simulations.

2017-05-04