1. Comparing Procedures for Mediation Analysis

Baron & Kenny (1986) Criteria

The four criteria for mediation are:

  1. the IV significantly affects the (candidate) mediator,
  2. the (candidate) mediator significantly affects the DV,
  3. the IV significantly affects the DV, and
  4. when the (candidate) mediator is controlled for, the IV does not significantly affect the DV (Baron & Kenny, 1986).

Partial mediation exists when criteria a, b, and c are met, but criterion d is not met. Complete mediation exists when all four criteria are met.

Criteria a and b seem essential to the concept of mediation.

Criterion c is theoretically essential, but in practice may lead to an incorrect conclusion. That is, there may be variables that are so weakly related that they fail to pass a significance test but which nevertheless meet criteria a and b by a hair because they are each related to a third variable more strongly.

Criterion d involves reasoning from a null result, which is almost always problematic because null results can be found for any number of reasons. And as Baron & Kenny (1986) note, controlling for the mediator, which is hypothesized to be correlated to both the IV and DV, leads to multicollinearity, diminishing the power of the analysis. Thus, I believe a null result could be found because of multicollinearity rather than complete mediation.

MacKinnon et al. (2007) Recommendations

I’m not sure I understand exactly what is different in MacKinnon et al. (2007) vs. Baron & Kenny (1986), but it seems like MacKinnon et al. (2007) introduce a way to test for the size and significance of the mediated effect. Specifically, MacKinnon et al. (2007) propose using \(\hat{c} - \hat{c}^{\prime}\) or \(\hat{a}\hat{b}\) for the mediated effect. (These are equivalent when using normal theory OLS.) Then the standard error can be found using \(\sigma_{\hat{a}\hat{b}} = \sqrt{\sigma_{\hat{a}}^2\hat{b}^2 + \sigma_{\hat{b}}^2\hat{a}^2}\) and a test-statistic can be created in the form of \(\frac{\hat{a}\hat{b}}{\sigma_{\hat{a}\hat{b}}}\) using the normal distribution as a reference.

This handles my final criticism (of criterion d) by eliminating the need to test for the null. And it introduces a way to measure the size of the mediated effect.

2. Conducting Mediation Analysis

Let’s move onto the practical analysis. I will initialize my working environment by loading some R packages.

# Projects
library(here)

# R Markdown
library(knitr)

# Statistics
library(psych)

# Tidyverse
library(readxl)
library(tidyverse)
library(broom)
library(glue)

Then I will load the data.

tbl.2 <- read_excel(here('data', 'MEDIATE.xls'))

tbl.2[1:10, ] %>%
  kable
PAYO SECO ATHO PREO PAYH SECH ATHH PREH JOBSAT TOI IDENT OCB
4.000000 3.666667 4.000000 3.000000 3.000000 3.000000 4.000000 2.666667 3.666667 3.00 4.166667 4.000000
4.666667 4.333333 4.666667 4.666667 5.000000 5.000000 4.333333 2.333333 3.333333 3.50 2.166667 2.500000
2.333333 2.333333 5.000000 2.333333 3.333333 3.000000 5.000000 2.000000 3.000000 3.25 1.833333 2.833333
4.000000 3.000000 4.666667 4.000000 5.000000 3.666667 5.000000 3.666667 3.666667 2.50 2.166667 3.583333
4.000000 3.000000 3.666667 4.000000 3.000000 3.000000 4.000000 3.000000 4.000000 2.25 3.666667 3.583333
3.333333 2.666667 5.000000 5.000000 4.000000 5.000000 5.000000 3.000000 4.000000 3.75 4.333333 4.416667
3.000000 2.333333 2.000000 2.000000 3.000000 3.333333 3.000000 2.000000 3.000000 3.50 3.666667 4.000000
3.333333 3.000000 3.666667 2.333333 3.000000 3.000000 3.000000 2.000000 3.666667 2.50 2.333333 3.916667
2.666667 3.000000 2.000000 4.000000 3.000000 2.666667 1.666667 3.666667 5.000000 1.25 3.833333 4.750000
3.000000 2.666667 2.000000 3.000000 3.000000 3.000000 2.000000 3.000000 3.000000 2.75 3.000000 3.750000 
describe(tbl.2)[c('n', 'mean', 'sd', 'min', 'max')] %>%
  kable
n mean sd min max
PAYO 969 3.479188 0.8725597 1 5
SECO 967 3.119269 1.0922852 1 5
ATHO 968 3.553891 0.9893694 1 5
PREO 967 3.298518 0.8768126 1 5
PAYH 966 3.362836 0.7806071 1 5
SECH 965 3.114680 1.0594249 1 5
ATHH 966 3.310041 0.9728338 1 5
PREH 965 2.922798 0.8580354 1 5
JOBSAT 967 3.685626 0.9173250 1 5
TOI 966 2.411146 0.8842549 1 5
IDENT 965 3.526079 0.8160464 1 5
OCB 959 3.651405 0.6445873 1 5

There are some missing data, but for this assignment, I will proceed without using an imputation procedure. Otherwise the data look usable.

A. Baron & Kenny (1986) Procedure

Using the data, I will use the Baron & Kenny (1986) procedure to evaluate whether rewards mediate the relationship between organizational values and employee behavior.

I will test each of the four content dimensions with each of the four outcomes. I will proceed by testing each criterion in turn. After testing each criterion, I will eliminate non-significant relationships from future tests because each criterion is necessary for mediation. The only exception to this rule is criterion d, which I will test last to distinguish the partial mediation models from the complete mediation models.

Criterion A

Recall that criterion a is that there is a significant relationship between the independent variables and the candidate mediator in the expected direction. In the present context, we are looking for positive, significant effects of values on rewards.

chr.2.iv <- c("PAY", "SEC", "ATH", "PRE")
chr.2.iv %>%
  map_df(~ lm(glue("{.}H ~ {.}O"), tbl.2) %>% tidy) %>%
  kable
term estimate std.error statistic p.value
(Intercept) 1.4873632 0.0823564 18.06009 0
PAYO 0.5391495 0.0229579 23.48424 0
(Intercept) 0.9410113 0.0717254 13.11964 0
SECO 0.6967837 0.0217069 32.09970 0
(Intercept) 0.7533919 0.0794090 9.48749 0
ATHO 0.7201975 0.0215376 33.43901 0
(Intercept) 1.1464040 0.0898951 12.75268 0
PREO 0.5388207 0.0263326 20.46211 0

It appears all four independent variables (for organizational values) have significant positive effects on their respective mediators.

Criterion B

Criterion b is that there is a significant relationship between the candidate mediator and the dependent variable in the expected direction. Here, we are looking for rewards to positively affect job satisfaction, organizational identification, and organizational citizenship behaviors and negatively affect turnover intentions.

fun.2.a <- function(dv, iv, suf, suf2 = NULL) {
  print(dv)
  iv %>%
    map_df(~ lm(glue("{dv} ~ {.}{suf}{ifelse(is.null(suf2), '', paste(' + ', ., suf2, sep = ''))}"),
                tbl.2) %>%
             tidy) %>%
    kable %>%
    print
}

chr.2.dv <- c('JOBSAT', 'TOI', 'IDENT', 'OCB')

chr.2.dv %>%
  walk(~ fun.2.a(., chr.2.iv, 'H'))

[1] “JOBSAT”

term estimate std.error statistic p.value
(Intercept) 2.5954950 0.1257717 20.636550 0
PAYH 0.3241483 0.0364269 8.898596 0
(Intercept) 2.7002714 0.0856431 31.529353 0
SECH 0.3163068 0.0260408 12.146589 0
(Intercept) 3.0045220 0.1023781 29.347314 0
ATHH 0.2058327 0.0296800 6.935075 0
(Intercept) 2.2309634 0.0930280 23.981635 0
PREH 0.4975125 0.0305399 16.290557 0

[1] “TOI”

term estimate std.error statistic p.value
(Intercept) 3.2445178 0.1230631 26.364659 0e+00
PAYH -0.2481772 0.0356333 -6.964747 0e+00
(Intercept) 3.1154147 0.0851593 36.583363 0e+00
SECH -0.2266389 0.0258824 -8.756470 0e+00
(Intercept) 2.8940514 0.0998545 28.982698 0e+00
ATHH -0.1463549 0.0289440 -5.056486 5e-07
(Intercept) 3.4451830 0.0954226 36.104468 0e+00
PREH -0.3536700 0.0313419 -11.284247 0e+00

[1] “IDENT”

term estimate std.error statistic p.value
(Intercept) 2.8239893 0.1140757 24.755388 0.0e+00
PAYH 0.2092331 0.0330238 6.335823 0.0e+00
(Intercept) 3.0093396 0.0798324 37.695698 0.0e+00
SECH 0.1663845 0.0242582 6.858896 0.0e+00
(Intercept) 3.1105443 0.0921612 33.751119 0.0e+00
ATHH 0.1261616 0.0267165 4.722236 2.7e-06
(Intercept) 2.4610106 0.0863291 28.507298 0.0e+00
PREH 0.3649441 0.0283508 12.872422 0.0e+00

[1] “OCB”

term estimate std.error statistic p.value
(Intercept) 3.2371142 0.0911613 35.509760 0.0000000
PAYH 0.1229580 0.0263898 4.659304 0.0000036
(Intercept) 3.3667841 0.0639840 52.619127 0.0000000
SECH 0.0908128 0.0194409 4.671214 0.0000034
(Intercept) 3.4431488 0.0735301 46.826413 0.0000000
ATHH 0.0627761 0.0213195 2.944537 0.0033128
(Intercept) 2.9694521 0.0703553 42.206502 0.0000000
PREH 0.2329716 0.0231052 10.083065 0.0000000

As the output shows, all models meet criterion b.

Criterion C

Criterion c is simply that there is a significant main effect of the IV on the DV in the expected direction. As before, we are looking for positive effects of all IVs on all DVs except turnover intentions, which we hypothesize will be negative.

chr.2.dv %>%
  walk(~ fun.2.a(., chr.2.iv, 'O'))

[1] “JOBSAT”

term estimate std.error statistic p.value
(Intercept) 2.7061282 0.1168402 23.160933 0
PAYO 0.2818529 0.0325746 8.652531 0
(Intercept) 2.7124858 0.0830670 32.654185 0
SECO 0.3122772 0.0251451 12.419002 0
(Intercept) 2.9892436 0.1076437 27.769793 0
ATHO 0.1961530 0.0291915 6.719525 0
(Intercept) 2.6970154 0.1102465 24.463506 0
PREO 0.2997832 0.0322940 9.282932 0

[1] “TOI”

term estimate std.error statistic p.value
(Intercept) 3.1115239 0.1146658 27.135592 0.0e+00
PAYO -0.2013188 0.0319695 -6.297208 0.0e+00
(Intercept) 3.0713968 0.0832359 36.899925 0.0e+00
SECO -0.2120716 0.0251858 -8.420284 0.0e+00
(Intercept) 2.8686975 0.1051265 27.288057 0.0e+00
ATHO -0.1289089 0.0285137 -4.520948 6.9e-06
(Intercept) 3.0962375 0.1085271 28.529629 0.0e+00
PREO -0.2075148 0.0318033 -6.524954 0.0e+00

[1] “IDENT”

term estimate std.error statistic p.value
(Intercept) 2.7086580 0.1043581 25.955428 0.00e+00
PAYO 0.2354118 0.0290745 8.096843 0.00e+00
(Intercept) 2.8966031 0.0763968 37.915262 0.00e+00
SECO 0.2026995 0.0231109 8.770741 0.00e+00
(Intercept) 3.1396325 0.0968434 32.419680 0.00e+00
ATHO 0.1097024 0.0262662 4.176561 3.23e-05
(Intercept) 2.6819242 0.0979234 27.387973 0.00e+00
PREO 0.2570374 0.0286783 8.962768 0.00e+00

[1] “OCB”

term estimate std.error statistic p.value
(Intercept) 3.3078357 0.0852728 38.7912164 0.0000000
PAYO 0.0987469 0.0237386 4.1597684 0.0000347
(Intercept) 3.3705035 0.0621479 54.2335506 0.0000000
SECO 0.0901647 0.0187926 4.7978843 0.0000019
(Intercept) 3.6528900 0.0774721 47.1510570 0.0000000
ATHO -0.0000309 0.0210221 -0.0014684 0.9988287
(Intercept) 3.2663724 0.0800166 40.8211641 0.0000000
PREO 0.1171604 0.0234219 5.0021774 0.0000007

All effects are significant except the effect of ATHO on OCB. Thus, I will drop the authority-OCB model from the analysis of criterion d. All other models I can now state have either partial or complete mediation.

Criterion D

Criterion d distinguishes partial from complete mediation. When controlling for the mediator, if the IV has no effect on the DV, the relationship between the IV and the DV is completely mediated. Otherwise it is only partially mediated.

chr.2.dv %>%
  walk(~ fun.2.a(., chr.2.iv, 'O', 'H'))

[1] “JOBSAT”

term estimate std.error statistic p.value
(Intercept) 2.3935226 0.1339386 17.870304 0.0000000
PAYO 0.1698465 0.0404863 4.195160 0.0000298
PAYH 0.2088233 0.0453104 4.608725 0.0000046
(Intercept) 2.5471183 0.0892821 28.528893 0.0000000
SECO 0.1882211 0.0358486 5.250443 0.0000002
SECH 0.1773645 0.0369925 4.794607 0.0000019
(Intercept) 2.8907128 0.1122542 25.751497 0.0000000
ATHO 0.1075518 0.0428278 2.511262 0.0121933
ATHH 0.1249953 0.0435813 2.868096 0.0042199
(Intercept) 2.1574706 0.1105005 19.524527 0.0000000
PREO 0.0456474 0.0358679 1.272653 0.2034498
PREH 0.4711470 0.0366688 12.848717 0.0000000

[1] “TOI”

term estimate std.error statistic p.value
(Intercept) 3.3741516 0.1317400 25.6122020 0.0000000
PAYO -0.1075094 0.0398535 -2.6976179 0.0071063
PAYH -0.1755270 0.0445733 -3.9379389 0.0000881
(Intercept) 3.2050053 0.0896630 35.7450010 0.0000000
SECO -0.1099474 0.0359775 -3.0560010 0.0023053
SECH -0.1454999 0.0371172 -3.9200143 0.0000949
(Intercept) 2.9526711 0.1097149 26.9122131 0.0000000
ATHO -0.0551781 0.0418793 -1.3175518 0.1879687
ATHH -0.1049563 0.0426218 -2.4625024 0.0139722
(Intercept) 3.4921214 0.1130388 30.8931125 0.0000000
PREO -0.0281848 0.0367022 -0.7679317 0.4427172
PREH -0.3376877 0.0375833 -8.9850477 0.0000000

[1] “IDENT”

term estimate std.error statistic p.value
(Intercept) 2.5948858 0.1205212 21.530537 0.0000000
PAYO 0.1942109 0.0363464 5.343336 0.0000001
PAYH 0.0770467 0.0406740 1.894253 0.0584925
(Intercept) 2.8633875 0.0830828 34.464264 0.0000000
SECO 0.1780698 0.0333300 5.342636 0.0000001
SECH 0.0354455 0.0343937 1.030580 0.3029985
(Intercept) 3.0692872 0.1012111 30.325601 0.0000000
ATHO 0.0445753 0.0385894 1.155118 0.2483303
ATHH 0.0912037 0.0392653 2.322760 0.0204008
(Intercept) 2.3198848 0.1019202 22.761778 0.0000000
PREO 0.0922785 0.0330677 2.790592 0.0053659
PREH 0.3096322 0.0338587 9.144846 0.0000000

[1] “OCB”

term estimate std.error statistic p.value
(Intercept) 3.1761108 0.0980700 32.3861730 0.0000000
PAYO 0.0512402 0.0296983 1.7253583 0.0847871
PAYH 0.0881844 0.0330773 2.6660123 0.0078056
(Intercept) 3.3213300 0.0674370 49.2508866 0.0000000
SECO 0.0533578 0.0270190 1.9748221 0.0485772
SECH 0.0524248 0.0278641 1.8814494 0.0602161
(Intercept) 3.5530185 0.0803631 44.2120826 0.0000000
ATHO -0.0950036 0.0307066 -3.0939118 0.0020330
ATHH 0.1318170 0.0312107 4.2234529 0.0000264
(Intercept) 2.9906864 0.0835611 35.7904089 0.0000000
PREO -0.0113376 0.0271028 -0.4183188 0.6758085
PREH 0.2390168 0.0277010 8.6284580 0.0000000

The results show that the effects of the prestige value on job satisfaction, turnover intentions, and OCBs are completely mediated by rewards. The effects of the authority value on turnover intentions and identification are completely mediated by rewards. And finally, the effect of the pay value on OCBs is completely mediated by rewards.

The other nine effects are only partially mediated by rewards.

B. MacKinnon et al. (2007) Procedure

THe MacKinnon et al. (2007) procedure swaps out criterion d for an estimate of the mediated effect, along with standard error estimates and a significance test.

I will write a function to calculate \(\hat{a}\hat{b}\), \(\sigma_{\hat{a}\hat{b}}\), and \(p\).

fun.2.b <- function(dv, iv) {
  mod.a <- lm(glue('{iv}H ~ {iv}O'), tbl.2)
  mod.b <- lm(glue('{dv} ~ {iv}H'), tbl.2)
  a <- coef(mod.a)[2]
  b <- coef(mod.b)[2]
  ab <- a * b
  sa <- mod.a %>%
    tidy %>%
    filter(term != '(Intercept)') %>%
    pull(std.error)
  sb <- mod.b %>%
    tidy %>%
    filter(term != '(Intercept)') %>%
    pull(std.error)
  sab <- sqrt(((sa^2) * (b^2)) + ((sb^2) * (a^2)))
  z <- ab / sab
  p <- (1 - pnorm(z)) * 2
  tibble(value = iv, outcome = dv, estimate = ab, std.error = sab, test.stat = z, p.val = p)
}

tbl.2.c <- chr.2.dv %>%
  map_df(~ map(chr.2.iv, ~ fun.2.b(.y, .x), .y = .x))

tbl.2.c %>%
  kable
value outcome estimate std.error test.stat p.val
PAY JOBSAT 0.1747644 0.0210022 8.321248 0.0000000
SEC JOBSAT 0.2203974 0.0194004 11.360450 0.0000000
ATH JOBSAT 0.1482402 0.0218303 6.790572 0.0000000
PRE JOBSAT 0.2680700 0.0210337 12.744795 0.0000000
PAY TOI -0.1338046 0.0200388 -6.677287 2.0000000
SEC TOI -0.1579183 0.0186934 -8.447792 2.0000000
ATH TOI -0.1054044 0.0210824 -4.999648 1.9999994
PRE TOI -0.1905647 0.0192854 -9.881296 2.0000000
PAY IDENT 0.1128079 0.0184414 6.117110 0.0000000
SEC IDENT 0.1159340 0.0172843 6.707483 0.0000000
ATH IDENT 0.0908612 0.0194321 4.675841 0.0000029
PRE IDENT 0.1966394 0.0180474 10.895738 0.0000000
PAY OCB 0.0662928 0.0145054 4.570223 0.0000049
SEC OCB 0.0632769 0.0136888 4.622526 0.0000038
ATH OCB 0.0452112 0.0154137 2.933187 0.0033550
PRE OCB 0.1255299 0.0138790 9.044581 0.0000000

The p-values for TOI are incorrect because the function I wrote assumes a positive effect. Here are the correct p-values for TOI.

tbl.2.c %>%
  filter(outcome == "TOI") %>%
  pull(test.stat) %>%
  map(~ pnorm(.) * 2)
## $PAYO
## [1] 2.434066e-11
## 
## $SECO
## [1] 2.968647e-17
## 
## $ATHO
## [1] 5.743509e-07
## 
## $PREO
## [1] 5.017975e-23

I had hoped to find some non-significant mediated effects, which would have made a contrast with the Baron & Kenny (1986) procedure straightforward. Unfortunately, there appears to be a significant mediated effect for all models, including even the authority-OCB model, which does not have a significant main effect.

However, the MacKinnon et al. (2007) results do show the relative effect sizes of the different models. Prestige had the largest mediated effect of any predictor for each of the outcome variables. Authority had the smallest mediated effects.

C. Conclusion

Both analyses suggest that rewards do seem to mediate the effects of values on all four outcomes. In one case, with regards to authority, values did not appear to affect the outcome of OCB and therefore mediation is ruled out. Finally, the strength of mediation varied. Rewards may have mattered most for prestige and least for authority.

References

Baron, R. M., & Kenny, D. A. (1986). The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations. Journal of Personality and Social Psychology, 51(6), 10.

MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation Analysis. Annual Review of Psychology, 58(1), 593–614. https://doi.org/10.1146/annurev.psych.58.110405.085542