------------------------------------------------------------------------------------------------------
       log:  c:\Imbook\bwebpage\Section2\mma07p4boot.txt
  log type:  text
 opened on:  18 May 2005, 21:36:29

. 
. ********** OVERVIEW OF MMA07BOOT4.DO **********
. 
. * STATA Program 
. * copyright C 2005 by A. Colin Cameron and Pravin K. Trivedi 
. * used for "Microeconometrics: Methods and Applications" 
. * by A. Colin Cameron and Pravin K. Trivedi (2005)
. * Cambridge University Press 
. 
. * Chapter 7.8 pages 254-256
. * Bootstrap applied to probit model
. * Provides
. *  (1) Bootstrap confidence intervals
. *  (2) Bootstrap hypothesis test without refinement
. *  (3) Bootstrap hypothesis test with refinement: percentile-t method
. 
. * Note corrections to book
. *  - sample size is N=40  not N=30
. *  - use 999 bootstrap replications not 1000
. *  - for asymptotic refinement p.256 the critical region 
. *      is (-1.89, 1.80) not (-2.62, 1.83)
. 
. * For more detail on bootstrap see 
. * Chapter 11: Bootstrap Methods pages 355-383
. * and program mma11p1boot.do
. 
. ********** SETUP **********
. 
. set more off

. version 8

.   
. ********** GENERATE DATA **********
. 
. * DGP is Probit: Pr[y=1] = PHI(a + bx)
. * where  x is N[0,1]
. * and  a = 0  and b = 1
. 
. * Change the following for different sample size N
. global numobs "40"

. 
. * Probit example with slope coefficient equal to 1
. set seed 10105

. set obs $numobs
obs was 0, now 40

. gen x = invnorm(uniform())

. gen y = 0

. replace y = 1 if 0+1.0*x+invnorm(uniform()) > 0
(19 real changes made)

. save xyforsim, replace
file xyforsim.dta saved

. summarize 

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
           x |        40   -.0359197    .9203391  -2.210579    1.45199
           y |        40        .475    .5057363          0          1

. probit y x

Iteration 0:   log likelihood = -27.675866
Iteration 1:   log likelihood = -22.927488
Iteration 2:   log likelihood = -22.735204
Iteration 3:   log likelihood = -22.733966
Iteration 4:   log likelihood = -22.733966

Probit estimates                                  Number of obs   =         40
                                                  LR chi2(1)      =       9.88
                                                  Prob > chi2     =     0.0017
Log likelihood = -22.733966                       Pseudo R2       =     0.1786

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           x |   .8168831   .2942893     2.78   0.006     .2400867    1.393679
       _cons |  -.0725436   .2162576    -0.34   0.737    -.4964006    .3513135
------------------------------------------------------------------------------

. save mma07p4boot, replace
file mma07p4boot.dta saved

. 
. * Write data to a text (ascii) file so can use with programs other than Stata
. outfile y x using mma07p4boot.asc, replace

. 
. ********** (1) BOOTSTRAP CONFIDENCE INTERVALS ********** 
. 
. * Stata produces four bootstrap 100*(1-alpha) confidence intervals
. * (1)-(2) have no asymptotic refinement
. * (3)-(4) have asymptotic refinement
. 
. * (1) Regular asymptotic normal: bhat +/- t(S-1)_alpha/2*se(bhat)
. *     except instead of using the initial se(bhat) 
. *     we use the standard deviation of bhat from the bootstrap reps
. *     and use t(S-1) rather than z for critical value
. *     where S = number of bootstrap reps
. 
. * (2) Percentile method: which orders the bhat(s) from simulations and
. *     goes from alpha/2 lowest bhat(s) to the alpha/2 highest bhat(s) 
. *     where (s) denotes the s-th bootstrap sample
. 
. * (3) Bootstrap-corrected. Same as (4) with a=0
. 
. * (4) Bootstrap-corrected and accelerated. 
. *     This works with the pivotal Wald statistic.
. *     See the manual [R]bootstrap or a textbook.
. *     e.g. Efron and Tibsharani (1993, pp.184-188) with a=0
. *     This orders the bhats from simulations and
. *     goes from p1 to the p2 highest
. *     where p1 and p2 are bias-correction adjustments to alpha/2 and 1-alpha/2
. *     Let p1 = Phi(2z0 - z_alpha/2)
. *         p2 = Phi(2z0 + z_alpha/2)
. *         z0 measures the median bias in bhat with
. *         z0 = Phi-inv(fraction of the bhat(s) < bhat)
. *     And if z0=0 then p1 = alpha/2 and no correction 
. 
. * Change the following for different number of simulations S
. * From page 399, for testing better to use 999 than 1000
. global breps "999"   /* The number of bootstrap reps used below */ 

. 
. * (1A) Simplest bootstrap is of all the estimated coefficients
. set seed 10105

. bootstrap "probit y x" _b, reps($breps) bca

command:      probit y x
statistics:   b_x        = _b[x]
              b_cons     = _b[_cons]

Bootstrap statistics                              Number of obs    =        40
                                                  Replications     =       999

------------------------------------------------------------------------------
Variable     |  Reps  Observed      Bias  Std. Err. [95% Conf. Interval]
-------------+----------------------------------------------------------------
         b_x |   999  .8168831  .1017329  .3763803   .0782956   1.555471   (N)
             |                                       .3495505   1.878616   (P)
             |                                       .2808956   1.600026  (BC)
             |                                       .1552112   1.480223 (BCa)
      b_cons |   999 -.0725436 -.0176301  .2448404  -.5530047   .4079175   (N)
             |                                       -.596443   .4247662   (P)
             |                                      -.5528302   .4381396  (BC)
             |                                      -.5205303   .4445401 (BCa)
------------------------------------------------------------------------------
Note:  N   = normal
       P   = percentile
       BC  = bias-corrected
       BCa = bias-corrected and accelerated

. 
. * (1B) This bootstrap is of MLE of b2 and the associated standard error
. *     and additionally gives the bias-accelerated method of Efron
. set seed 10105

. bootstrap "probit y x" _b[x] _se[x], reps($breps) bca

command:      probit y x
statistics:   _bs_1      = _b[x]
              _bs_2      = _se[x]

Bootstrap statistics                              Number of obs    =        40
                                                  Replications     =       999

------------------------------------------------------------------------------
Variable     |  Reps  Observed      Bias  Std. Err. [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _bs_1 |   999  .8168831  .1017329  .3763803   .0782956   1.555471   (N)
             |                                       .3495505   1.878616   (P)
             |                                       .2808956   1.600026  (BC)
             |                                       .1552112   1.480223 (BCa)
       _bs_2 |   999  .2942893  .0422005  .0932673   .1112667   .4773118   (N)
             |                                       .2323841   .5831083   (P)
             |                                       .2214397   .4475662  (BC)
             |                                       .2162534   .4143377 (BCa)
------------------------------------------------------------------------------
Note:  N   = normal
       P   = percentile
       BC  = bias-corrected
       BCa = bias-corrected and accelerated

. 
. * (1C) This bootstrap repeats (2) 
. *     but will permit bootstrapping if Stata commands are more than one line
. use mma07p4boot, clear

. program define commandtobootstrap, rclass
  1.   version 8.0
  2.   quietly probit y x
  3.   return scalar b2hat=_b[x]
  4.   return scalar seb2hat=_se[x]
  5. end

. set seed 10105

. bootstrap "commandtobootstrap" r(b2hat) r(seb2hat), reps($breps)

command:      commandtobootstrap
statistics:   _bs_1      = r(b2hat)
              _bs_2      = r(seb2hat)

Bootstrap statistics                              Number of obs    =        40
                                                  Replications     =       999

------------------------------------------------------------------------------
Variable     |  Reps  Observed      Bias  Std. Err. [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _bs_1 |   999  .8168831  .1017329  .3763803   .0782956   1.555471   (N)
             |                                       .3495505   1.878616   (P)
             |                                       .2808956   1.600026  (BC)
       _bs_2 |   999  .2942893  .0422005  .0932673   .1112667   .4773118   (N)
             |                                       .2323841   .5831083   (P)
             |                                       .2214397   .4475662  (BC)
------------------------------------------------------------------------------
Note:  N   = normal
       P   = percentile
       BC  = bias-corrected

. 
. ********** (2) BOOTSTRAP HYPOTHESIS TESTS - NO REFINEMENT p.255 ********** 
. 
. * We want to test H0: b2 = 1 against Ha: b2 not equal 1
. 
. * For a simple test such as this we can just use
. * the bootstrap confidence intervals from (1) 
. * and reject if bhat2 is not in the confidence interval
. 
. * Here we instead present a common method without refinement
. * essentially (1) above, performing the usual Wald test, 
. * except the standard error is estimated by bootstrap.
. * This is useful when hard to obtain standard error by other means.
. * Here  W = (b2hat - b2_0) / seb2hat_boot  where b2_0 = 1
. * and reject at level .05 if |W| > z_.025 = 1.96
. 
. use mma07p4boot, clear

. * Save the estimate 
. quietly probit y x

. scalar b2est = _b[x]

. * Obtain the bootstrap standard error
. set seed 10105

. bootstrap "probit y x" _b, reps($breps) bca

command:      probit y x
statistics:   b_x        = _b[x]
              b_cons     = _b[_cons]

Bootstrap statistics                              Number of obs    =        40
                                                  Replications     =       999

------------------------------------------------------------------------------
Variable     |  Reps  Observed      Bias  Std. Err. [95% Conf. Interval]
-------------+----------------------------------------------------------------
         b_x |   999  .8168831  .1017329  .3763803   .0782956   1.555471   (N)
             |                                       .3495505   1.878616   (P)
             |                                       .2808956   1.600026  (BC)
             |                                       .1552112   1.480223 (BCa)
      b_cons |   999 -.0725436 -.0176301  .2448404  -.5530047   .4079175   (N)
             |                                       -.596443   .4247662   (P)
             |                                      -.5528302   .4381396  (BC)
             |                                      -.5205303   .4445401 (BCa)
------------------------------------------------------------------------------
Note:  N   = normal
       P   = percentile
       BC  = bias-corrected
       BCa = bias-corrected and accelerated

. matrix sebboot = e(se)

. scalar seb2boot = sebboot[1,1]   /* x is first then constant */

. * Calculate the test statistic
. scalar Wald = (b2est - 1)/seb2boot

. 
. * DISPLAY RESULTS at bottom p.255
. * Note: Text had typo: 
. * (1-0.817)/0.376 = -0.487 should be (0.817-1)/0.376 = -0.487
. 
. di "Probit slope estimate is:        "   b2est
Probit slope estimate is:        .8168831

. di "Bootstrap standard estimate is:  "   seb2boot 
Bootstrap standard estimate is:  .37638029

. di "Wald statistic (no refinement) is: " Wald
Wald statistic (no refinement) is: -.48652096

. di "Reject at level .05 if |Wald| > 1.96"
Reject at level .05 if |Wald| > 1.96

. 
. ********** (3) BOOTSTRAP HYPOTHESIS TESTS - PERCENTILE-T p.256 ********** 
. 
. * Stata does not give this. For methods see
. * e.g. Efron and Tibsharani (1993, pp.160-162)  
. * e.g. Cameron and Trivedi (2005) Chapter 11.2.6-11.2.7
. * For sample s compute t-test(s) = (bhat(s)-bhat) / se(s)
. * where bhat is initial estimate 
. * and bhat(s) and se(s) are for sth round.
. * Order the t-test(s) statistics and choose the alpha/2 percentiles
. * which give the critical values for the t-test
. 
. * Implementation requires saving the results from each bootstrap replication 
. * in order to obtain ccritical values from percentiles of bootstrap distribution
. 
. * (3A) Here bootstrap computes (b(s) - bhat) / se(s)  s = 1,...,S
.  
. use mma07p4boot, clear

. * Save the estimate and the Wald test statistic 
. quietly probit y x

. scalar b2est = _b[x]

. scalar Wald = (_b[x] - 1)/_se[x]

. * Then bootstrap calculates (b(s) - bhat) / se(s)
. set seed 10105

. bootstrap "probit y x" ((_b[x]-b2est)/_se[x]), reps($breps) /*
>    */ level(95) saving(mma07p4bootreps) replace

command:      probit y x
statistic:    _bs_1      = (_b[x]-b2est)/_se[x]

Bootstrap statistics                              Number of obs    =        40
                                                  Replications     =       999

------------------------------------------------------------------------------
Variable     |  Reps  Observed      Bias  Std. Err. [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _bs_1 |   999         0  .1003619  .9350234  -1.834837   1.834837   (N)
             |                                      -1.890602   1.801358   (P)
             |                                      -2.101316   1.565618  (BC)
------------------------------------------------------------------------------
Note:  N   = normal
       P   = percentile
       BC  = bias-corrected

. * Then get data sets with result from each bootstrap
. use mma07p4bootreps, clear
(bootstrap: probit y x)

. sum                 /* Here just _bs_1 */

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
       _bs_1 |       999    .1003619    .9350234  -3.032139   2.572848

. gen b2test = _bs_1  /* _bs_1 is the bootstrap result of interest */

. sum b2test, detail  /* Gives percentiles but not 2.5% and 97.5%  */

                           b2test
-------------------------------------------------------------
      Percentiles      Smallest
 1%    -2.188575      -3.032139
 5%    -1.540843      -2.605178
10%    -1.137846      -2.599248       Obs                 999
25%    -.4995352      -2.566578       Sum of Wgt.         999

50%     .1238111                      Mean           .1003619
                        Largest       Std. Dev.      .9350234
75%     .7789762        2.22565
90%     1.338348       2.359132       Variance       .8742688
95%     1.560646       2.377491       Skewness      -.2505319
99%     2.014282       2.572848       Kurtosis       2.853737

. _pctile b2test, p(2.5,97.5)

. 
. * DISPLAY RESULTS on p.256
. 
. * Note: Error on p.256  Here get (-1.89, 1.80) not (-2.62, 1.83)
. di "Lower 2.5 and upper 2.5 percentile of coeff b for z: " r(r1) "  and  " r(r2) 
Lower 2.5 and upper 2.5 percentile of coeff b for z: -1.8906019  and  1.8013585

. di "Reject H0 if Wald = " Wald " lies outside " r(r1) " ," r(r2) ")"
Reject H0 if Wald = -.62223436 lies outside -1.8906019 ,1.8013585)

. 
. * (3B) Equivalently bootstrap calculates b(s) and se(s)   s = 1,...,S
. *      and then later calculate (b(s) - bhat) / se(s)
. 
. use mma07p4boot, clear

. * Save the estimate and the Wald test statistic 
. quietly probit y x

. scalar b2est = _b[x]

. scalar Wald = (_b[x] - 1)/_se[x]

. * Then bootstrap calculates b(s) and se(s)
. set seed 10105

. bootstrap "probit y x" _b[x] _se[x], reps($breps) /*
>    */ level(95) saving(mma07p4bootreps) replace

command:      probit y x
statistics:   _bs_1      = _b[x]
              _bs_2      = _se[x]

Bootstrap statistics                              Number of obs    =        40
                                                  Replications     =       999

------------------------------------------------------------------------------
Variable     |  Reps  Observed      Bias  Std. Err. [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _bs_1 |   999  .8168831  .1017329  .3763803   .0782956   1.555471   (N)
             |                                       .3495505   1.878616   (P)
             |                                       .2808956   1.600026  (BC)
       _bs_2 |   999  .2942893  .0422005  .0932673   .1112667   .4773118   (N)
             |                                       .2323841   .5831083   (P)
             |                                       .2214397   .4475662  (BC)
------------------------------------------------------------------------------
Note:  N   = normal
       P   = percentile
       BC  = bias-corrected

. * Then get data sets with result from each bootstrap
. use mma07p4bootreps, clear
(bootstrap: probit y x)

. sum                 /* Here _bs_1 and _bs_2 */

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
       _bs_1 |       999     .918616    .3763803   .0030288   3.806198
       _bs_2 |       999    .3364898    .0932673   .2162534    1.34312

. gen b2test = (_bs_1 - b2est)/_bs_2

. _pctile b2test, p(2.5,97.5)

. 
. * DISPLAY RESULTS on p.256
. * Note: Error on p.256  Here get (-1.89, 1.80) not (-2.62, 1.83)
. di "Lower 2.5 and upper 2.5 percentile of coeff b for z: " r(r1) "  and  " r(r2) 
Lower 2.5 and upper 2.5 percentile of coeff b for z: -1.8906019  and  1.8013583

. di "Reject H0 if Wald = " Wald " lies outside " r(r1) " ," r(r2) ")"
Reject H0 if Wald = -.62223436 lies outside -1.8906019 ,1.8013583)

. 
. ********** CLOSE OUTPUT
. log close
       log:  c:\Imbook\bwebpage\Section2\mma07p4boot.txt
  log type:  text
 closed on:  18 May 2005, 21:36:36
----------------------------------------------------------------------------------------------------
