Discrimination tests

In discrimination experiments, candidates are sent on the same test (e.g. job, house rental) and one examines whether they receive the same outcome. The number of non negative or explicitly positive answers are examined in details, looking for outcome differences. In what follows, we consider a test about the effect of gender and origin on the recruitment of software developers (inter1 data set). The candidates can have a French, Moroccan, Senegalese or Vietnamese origin, suggested by their first and last names.

library(callback)
m <- inter1
table(m$origin, m$lastn)
#>    
#>     Bertrand Diallo Diouf Kaidi Moreau Pham Tran Zalegh
#>   F      310      0     0     0    310    0    0      0
#>   M        0      0     0   310      0    0    0    310
#>   S        0    310   310     0      0    0    0      0
#>   V        0      0     0     0      0  310  310      0
table(m$origin, m$firstn)
#>    
#>     Abdallah Amadou Anthony Fatou Jamila Minh Trang Sophie Tien Hiep
#>   F        0      0     310     0      0          0    310         0
#>   M      310      0       0     0    310          0      0         0
#>   S        0    310       0   310      0          0      0         0
#>   V        0      0       0     0      0        310      0       310

The contents of the data set is:

str(m)
#> 'data.frame':    2480 obs. of  11 variables:
#>  $ offer    : Factor w/ 310 levels "1","2","3","4",..: 1 1 1 1 1 1 1 1 2 2 ...
#>  $ firstn   : Factor w/ 8 levels "Abdallah","Amadou",..: 7 4 5 6 1 2 3 8 1 2 ...
#>  $ lastn    : Factor w/ 8 levels "Bertrand","Diallo",..: 5 3 4 7 8 2 1 6 8 2 ...
#>  $ origin   : Factor w/ 4 levels "F","M","S","V": 1 3 2 4 2 3 1 4 2 3 ...
#>  $ sentorder: int  3 7 6 2 1 5 4 8 8 4 ...
#>  $ gender   : Factor w/ 2 levels "Man","Woman": 2 2 2 2 1 1 1 1 1 1 ...
#>  $ callback : logi  TRUE TRUE TRUE TRUE FALSE FALSE ...
#>  $ paris    : Factor w/ 2 levels "No","Yes": 1 1 1 1 1 1 1 1 1 1 ...
#>  $ cont     : Factor w/ 2 levels "LTC","STC": 1 1 1 1 1 1 1 1 1 1 ...
#>  $ ansorder : int  1 2 3 4 9 9 9 9 9 9 ...
#>  $ date     : Factor w/ 3 levels "April 2009","February 2009",..: 2 2 2 2 2 2 2 2 2 2 ...

The offer variable is very important. It indicates the job offer identification. It is important because, in order to test discrimination, the workers must candidate on the same job offer. This is the cluster parameter of the callback() function. With cluster = "offer" we are sure that all the computations will be paired, which means that we will always compare the candidates on the very same job offer. This is essential to produce meaningful results since otherwise the difference of answers could come from the differences of recruiters and not from the differences in gender or origin.

The second important variables are the ones that define the candidates. Here, there are two variables : the suggested origin (F for French, M for Moroccan, V for Vietnamese and S for Senegalese) and the gender. Combined together, they give the candidate variable that we use in the analysis. The origin and gender variables are factors and the reference levels of these factors implicitly define the reference candidate. By convention, the reference candidate is the one that is the less susceptible to be discriminated against. Here the French origin man should be taken because his French origin and gender should not be discrimination sources in the French labor market. In practice, we will check that this candidate really had the highest callback rate. We can find the reference levels of our factors by looking at the first level given by the levels() function.

levels(m$origin)
#> [1] "F" "M" "S" "V"
levels(m$gender)
#> [1] "Man"   "Woman"

By default, the levels are ordered after their alphabetical ordering. It is pure chance that we find the French man as a reference. It can be changed with the relevel function. For instance, if one wants to take the woman as a reference, enter:

m$gender2 <- relevel(m$gender, ref = "Woman")
levels(m$gender2)
#> [1] "Woman" "Man"

and the new factor gender2 has “Woman” as the reference. The last element we need is, obviously, the outcome of the job hiring application. It is given by the callback variable. It is a Boolean variable, TRUE when the recruiter gives a non negative callback (in this data set), and FALSE otherwise.

We can know launch the callback() function, which prepares the data for statistical analysis. Here we need to choose the comp parameter. Indeed, we realize that there are $n=8$ candidates so that $n(n-1)/2=8\times 7/2=28$ comparisons are possible. This is a large number and this is why callback() performs the statistical analysis according to the reference candidate only by default with comp = "ref". This reduces our analysis to $n-1=7$ comparisons. One can get the 28 comparisons by setting comp = "all" instead.

dtest <- callback(
  data = m,
  cluster = "offer",
  candid = c("origin", "gender"),
  callback = "callback"
)

The dtest object contains the formatted data needed for the callback analysis. Using print() gives the mains characteristics of the experiment :

print(dtest)
#> 
#>  Structure of the experiment 
#>  ---------------------------
#>  
#>  Candidates defined by: origin gender 
#>  Callback variable: callback 
#>  
#>  Number of tests for each candidate:
#> 
#>   F.Man F.Woman   M.Man M.Woman   S.Man S.Woman   V.Man V.Woman 
#>     310     310     310     310     310     310     310     310 
#> 
#>  
#>  Number of tests for each pair of candidates:
#> 
#>  F.Man.vs.F.Woman F.Man.vs.M.Man F.Man.vs.M.Woman F.Man.vs.S.Man
#>               310            310              310            310
#>  F.Man.vs.S.Woman F.Man.vs.V.Man F.Man.vs.V.Woman
#>               310            310              310
#> 
#>  
#>  Number of tests with all the candidates: 310

We find that the experiment is standard since all the candidates have been sent to all the tests. When a candidate of the same type is send several times to a test, the most favorable answer is kept (the “max” rule). The reader is informed that there are other ways to deal with this issue.

Global statistics

We can take a look at the global callback rates of the candidates, by entering :

print(stat_raw(dtest))
#> 
#>  Proportions: raw callback rates 
#>  Confidence intervals: Student at 95 %
#>  
#>         tests callback inf_p_callback p_callback sup_p_callback
#> F.Man     310       86     0.22730239 0.27741935      0.3275363
#> F.Woman   310       70     0.17900426 0.22580645      0.2726086
#> M.Man     310       65     0.16411033 0.20967742      0.2552445
#> M.Woman   310       32     0.06916861 0.10322581      0.1372830
#> S.Man     310       43     0.10001944 0.13870968      0.1773999
#> S.Woman   310       26     0.05284271 0.08387097      0.1148992
#> V.Man     310       38     0.08587036 0.12258065      0.1592909
#> V.Woman   310       62     0.15522525 0.20000000      0.2447748

and get a graphical representation with :

plot(stat_raw(dtest))

It is possible to change the definition of the confidence intervals, the confidence level and the colors in the plot. If you prefer the Clopper-Pearson definition, a 90% confidence interval, a “steelblue3” bar and a black confidence interval enter :

g <- stat_raw(dtest, level = 0.9,method="cp")
print(g)
#> 
#>  Proportions: raw callback rates 
#>  Confidence intervals: Clopper-Pearson at 90 %
#>  
#>         tests callback inf_p_callback p_callback sup_p_callback
#> F.Man     310       86     0.23570047 0.27741935      0.3223433
#> F.Woman   310       70     0.18721293 0.22580645      0.2683608
#> M.Man     310       65     0.17222662 0.20967742      0.2513276
#> M.Woman   310       32     0.07612170 0.10322581      0.1361644
#> S.Man     310       43     0.10749071 0.13870968      0.1752003
#> S.Woman   310       26     0.05943210 0.08387097      0.1144672
#> V.Man     310       38     0.09312657 0.12258065      0.1575588
#> V.Woman   310       62     0.16327759 0.20000000      0.2410655
plot(g, col = c("steelblue3","black"))

When all the candidates are sent to all the tests, the previous figures may be used to measure discrimination. However, when there is a rotation of the candidates so that only a part of them is sent on each test, it could not be the case. For this reason, we prefer to use matched statistics, which only compare candidates that have been sent to the same tests.

Since we do pairwise comparisons, we will consider two candidates $1$ and $2$ that are send on the same test. There are four possible outcomes: no callback (denoted $0$ for both candidates), one of the two candidates is called back (denoted $1$ for the candidate called back, $0$ for the other), or both candidates are called back (denoted $1$ for both candidates). We count the corresponding cases and use the following notations:

$n_{00}$ : no candidate is called back
$n_{10}$ : candidate $1$ is called back only
$n_{01}$ : candidate $2$ is called back only
$n_{11}$ : both candidates are called back

In order to get the result of the discrimination tests, we will use the stat_count function. It can be saved into an object for further exports, or printed. The following instruction:

sp <- stat_paired(dtest)

does not produce any printed output, but saves an object with class stat_count into s. We can get the statistics with:

print(sp)
#> 
#>  Callback counts:
#>  ----------------
#>                  tests callback disc callback1 Neither Only 1 Only 2 Both
#> F.Man vs F.Woman   310      113   70        86      70    197     43   27
#> F.Man vs M.Man     310      106   61        86      65    204     41   20
#> F.Man vs M.Woman   310       96   74        86      32    214     64   10
#> F.Man vs S.Man     310      100   71        86      43    210     57   14
#> F.Man vs S.Woman   310       97   82        86      26    213     71   11
#> F.Man vs V.Man     310       97   70        86      38    213     59   11
#> F.Man vs V.Woman   310      111   74        86      62    199     49   25
#>                  Difference calldif
#> F.Man vs F.Woman         43      16
#> F.Man vs M.Man           45      21
#> F.Man vs M.Woman         22      54
#> F.Man vs S.Man           29      43
#> F.Man vs S.Woman         15      60
#> F.Man vs V.Man           27      48
#> F.Man vs V.Woman         37      24

The callback counts describe the results of the paired experiments. The first column defines the comparison under the form “candidate 1 vs candidate 2”. Here “F.Man vs F.Woman” means that we compare French origin men (“F.Man”) with the French origin woman (“F.Woman”). Out of 310 tests, 113 got at least one callback. The French origin men got 86 callbacks and the French origin women 70. The difference, called net discrimination, equals 86-70=16 callbacks. We can go further in the details thanks to the next columns. Out of 310 tests, neither candidate was called back in $n_{00}=197$ of the job offers, $n_{10}=43$ called only men, $n_{01}=27$ called only women and $n_{11}=43$ called both. Discrimination only occurs when a single candidate is called back. The net discrimination is thus $n_{10}-n_{01}=43-27=16$ (the “Difference” column). The corresponding line percentages are available with .

sp$props
#>                  p_callback   p_cand1    p_cand2     p_c00     p_c10      p_c01
#> F.Man vs F.Woman  0.3645161 0.2774194 0.22580645 0.6354839 0.1387097 0.08709677
#> F.Man vs M.Man    0.3419355 0.2774194 0.20967742 0.6580645 0.1322581 0.06451613
#> F.Man vs M.Woman  0.3096774 0.2774194 0.10322581 0.6903226 0.2064516 0.03225806
#> F.Man vs S.Man    0.3225806 0.2774194 0.13870968 0.6774194 0.1838710 0.04516129
#> F.Man vs S.Woman  0.3129032 0.2774194 0.08387097 0.6870968 0.2290323 0.03548387
#> F.Man vs V.Man    0.3129032 0.2774194 0.12258065 0.6870968 0.1903226 0.03548387
#> F.Man vs V.Woman  0.3580645 0.2774194 0.20000000 0.6419355 0.1580645 0.08064516
#>                       p_c11 p_cand_dif
#> F.Man vs F.Woman 0.13870968 0.05161290
#> F.Man vs M.Man   0.14516129 0.06774194
#> F.Man vs M.Woman 0.07096774 0.17419355
#> F.Man vs S.Man   0.09354839 0.13870968
#> F.Man vs S.Woman 0.04838710 0.19354839
#> F.Man vs V.Man   0.08709677 0.15483871
#> F.Man vs V.Woman 0.11935484 0.07741935

We can save the output or print it, like in the previous example. Printing is the default.

Matched statistics

callback rates

In fact, there are three ways that can be used to compute proportions in discrimination studies. First, you can divide the number of callbacks by the number of tests. We call it “matched callback rates” given by the function stat_mcr(). Second, you can restrict your analysis to the tests which got at least one callback. We call it “total callback shares”, given by the function stat_tcs(). Last you can divide by the number of tests where only one candidate has been called back. We call it “exclusive callback shares”, given by the function stat_ecs().

The callback rate of candidates $1$ and $2$ , denoted respectively $p_1$ and $p_2$ , are obtained by dividing the number of callbacks of each candidate by the total number of discrimination tests $n$ :

$\begin{align*} p_1 &= \frac{n_{10}+n_{11}}{n}\\ p_2 &= \frac{n_{01}+n_{11}}{n}\\ \text{with } n &= n_{00}+n_{10}+n_{01}+n_{11} \end{align*}$

The absence of discrimination is measured by: $p_1=p_2\Leftrightarrow n_{10}=n_{01}$

The stat_mcr() function provides the proportions, the confidence intervals and the equality tests. By default, the level is 95% and can be changed with the leveloption. The Student definition is obtained with:

mcr <- stat_mcr(dtest)
print(mcr)
#> 
#>  Proportions: matched callback rates 
#>  Confidence intervals: Student at 95 %
#>  
#>                  tests inf_p_callback p_callback sup_p_callback inf_p_cand1
#> F.Man vs F.Woman   310      0.3106416  0.3645161      0.4183907   0.2273024
#> F.Man vs M.Man     310      0.2888373  0.3419355      0.3950337   0.2273024
#> F.Man vs M.Woman   310      0.2579222  0.3096774      0.3614326   0.2273024
#> F.Man vs S.Man     310      0.2702542  0.3225806      0.3749071   0.2273024
#> F.Man vs S.Woman   310      0.2610009  0.3129032      0.3648056   0.2273024
#> F.Man vs V.Man     310      0.2610009  0.3129032      0.3648056   0.2273024
#> F.Man vs V.Woman   310      0.3043985  0.3580645      0.4117306   0.2273024
#>                    p_cand1 sup_p_cand1 inf_p_cand2    p_cand2 sup_p_cand2
#> F.Man vs F.Woman 0.2774194   0.3275363  0.17900426 0.22580645   0.2726086
#> F.Man vs M.Man   0.2774194   0.3275363  0.16411033 0.20967742   0.2552445
#> F.Man vs M.Woman 0.2774194   0.3275363  0.06916861 0.10322581   0.1372830
#> F.Man vs S.Man   0.2774194   0.3275363  0.10001944 0.13870968   0.1773999
#> F.Man vs S.Woman 0.2774194   0.3275363  0.05284271 0.08387097   0.1148992
#> F.Man vs V.Man   0.2774194   0.3275363  0.08587036 0.12258065   0.1592909
#> F.Man vs V.Woman 0.2774194   0.3275363  0.15522525 0.20000000   0.2447748
#>                  inf_cand_dif p_cand_dif sup_cand_dif
#> F.Man vs F.Woman -0.001263807 0.05161290    0.1044896
#> F.Man vs M.Man    0.018669997 0.06774194    0.1168139
#> F.Man vs M.Woman  0.123097544 0.17419355    0.2252896
#> F.Man vs S.Man    0.087439176 0.13870968    0.1899802
#> F.Man vs S.Woman  0.140210120 0.19354839    0.2468867
#> F.Man vs V.Man    0.104550333 0.15483871    0.2051271
#> F.Man vs V.Woman  0.023420286 0.07741935    0.1314184
#> 
#>  Student test 
#>                  statistic       p_stat c_stat
#> F.Man vs F.Woman  1.920642 5.569699e-02    .  
#> F.Man vs M.Man    2.716294 6.973512e-03    ** 
#> F.Man vs M.Woman  6.708070 9.404184e-11    ***
#> F.Man vs S.Man    5.323431 1.961932e-07    ***
#> F.Man vs S.Woman  7.140081 6.704916e-12    ***
#> F.Man vs V.Man    6.058490 3.990671e-09    ***
#> F.Man vs V.Woman  2.821082 5.095962e-03    ** 
#> 
#>  Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.10 ' ' 1

and a corresponding plot with:

plot(mcr)

This represents the difference of proportions and their confidence intervals. Another plot is available, with the confidence intervals of the callback rate of the two candidates. However, the reader is informed that these confidence intervals with level $1-\alpha$ can be misleading because their crossing does not guarantee the equality of the callback rates at the $\alpha$ level. To get it anyway, enter:

plot(mcr, dif = FALSE)

The difference analysis is not available with the Clopper-Pearson intervals.

Total callback shares

With the total callback shares approach, we restrict the analysis to the tests with at least one callback. The total callback shares of the candidates $1$ and $2$ , denoted respectively $s_1$ and $s_2$ are defined by:

$\begin{align*} s_1 &= \frac{n_{10}}{n_c}\\ s_2 &= \frac{n_{01}}{n_c}\\ n_c &= n_{10}+n_{01}+n_{11} \end{align*}$ and the equal treatment test is: $s_1=s_2\Leftrightarrow n_{10}=n_{01}$ The total callback rates and the callback shares are related by the relationship: $p_i=s_i\times \pi_c$

where $\pi_c=n_c/n$ is the overall response rate to the study.

This is equivalent to the previous approach, with a different normalization. Here as well, three tests are available: the Fisher independence test, the chi-squared tests and the asymptotic Student test. For the confidence intervals enter:

tcs <- stat_tcs(dtest)
print(tcs)
#> 
#>  Proportions: total callback shares 
#>  Confidence intervals: Student at 95 %
#>  
#>                  callback inf_p_cand1   p_cand1 sup_p_cand1 inf_p_cand2
#> F.Man vs F.Woman      113   0.2896314 0.3805310   0.4714305  0.15910006
#> F.Man vs M.Man        106   0.2925535 0.3867925   0.4810314  0.11297051
#> F.Man vs M.Woman       96   0.5706498 0.6666667   0.7626835  0.04194644
#> F.Man vs S.Man        100   0.4712713 0.5700000   0.6687287  0.07080339
#> F.Man vs S.Woman       97   0.6422230 0.7319588   0.8216945  0.04916358
#> F.Man vs V.Man         97   0.5093540 0.6082474   0.7071409  0.04916358
#> F.Man vs V.Woman      111   0.3476147 0.4414414   0.5352682  0.14629340
#>                    p_cand2 sup_p_cand2 inf_p_equal   p_equal sup_p_equal
#> F.Man vs F.Woman 0.2389381   0.3187761  0.28963141 0.3805310   0.4714305
#> F.Man vs M.Man   0.1886792   0.2643880  0.32888536 0.4245283   0.5201712
#> F.Man vs M.Woman 0.1041667   0.1663869  0.14355961 0.2291667   0.3147737
#> F.Man vs S.Man   0.1400000   0.2091966  0.19951018 0.2900000   0.3804898
#> F.Man vs S.Woman 0.1134021   0.1776405  0.08139008 0.1546392   0.2278883
#> F.Man vs V.Man   0.1134021   0.1776405  0.18755161 0.2783505   0.3691494
#> F.Man vs V.Woman 0.2252252   0.3041570  0.24425961 0.3333333   0.4224071
#>                  inf_cand_dif p_cand_dif sup_cand_dif
#> F.Man vs F.Woman -0.003358593  0.1415929    0.2865444
#> F.Man vs M.Man    0.056416334  0.1981132    0.3398101
#> F.Man vs M.Woman  0.425195198  0.5625000    0.6998048
#> F.Man vs S.Man    0.285491775  0.4300000    0.5745082
#> F.Man vs S.Woman  0.480742510  0.6185567    0.7563709
#> F.Man vs V.Man    0.354957715  0.4948454    0.6347330
#> F.Man vs V.Woman  0.067443695  0.2162162    0.3649887
#> 
#>  Student test 
#>                  statistic       p_stat c_stat
#> F.Man vs F.Woman  1.935463 5.545457e-02    .  
#> F.Man vs M.Man    2.772269 6.587279e-03    ** 
#> F.Man vs M.Woman  8.133027 1.577637e-12    ***
#> F.Man vs S.Man    5.904254 4.993566e-08    ***
#> F.Man vs S.Woman  8.909281 3.274494e-14    ***
#> F.Man vs V.Man    7.021780 3.130531e-10    ***
#> F.Man vs V.Woman  2.880168 4.778501e-03    ** 
#> 
#>  Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.10 ' ' 1

with the graphical representation of the confidence intervals :

plot(tcs)

or with the representation of the callback percentages :

plot(tcs, dif = FALSE)

For the Fisher test, enter:

print(stat_tcs(dtest,method = "cp"))
#> 
#>  Proportions: total callback shares 
#>  Confidence intervals: Clopper-Pearson at 95 %
#>  
#>                  callback inf_p_cand1   p_cand1 sup_p_cand1 inf_p_cand2
#> F.Man vs F.Woman      113   0.2908336 0.3805310   0.4766535   0.1637459
#> F.Man vs M.Man        106   0.2937587 0.3867925   0.4863220   0.1192271
#> F.Man vs M.Woman       96   0.5631170 0.6666667   0.7596193   0.0510942
#> F.Man vs S.Man        100   0.4671337 0.5700000   0.6686090   0.0787054
#> F.Man vs S.Woman       97   0.6324160 0.7319588   0.8168495   0.0579887
#> F.Man vs V.Man         97   0.5038751 0.6082474   0.7058239   0.0579887
#> F.Man vs V.Woman      111   0.3472714 0.4414414   0.5388144   0.1513684
#>                    p_cand2 sup_p_cand2 inf_p_equal   p_equal sup_p_equal
#> F.Man vs F.Woman 0.2389381   0.3282806  0.29083362 0.3805310   0.4766535
#> F.Man vs M.Man   0.1886792   0.2762471  0.32906132 0.4245283   0.5243221
#> F.Man vs M.Woman 0.1041667   0.1832332  0.14953397 0.2291667   0.3261494
#> F.Man vs S.Man   0.1400000   0.2237280  0.20357419 0.2900000   0.3892660
#> F.Man vs S.Woman 0.1134021   0.1938539  0.08921326 0.1546392   0.2421973
#> F.Man vs V.Man   0.1134021   0.1938539  0.19211280 0.2783505   0.3785783
#> F.Man vs V.Woman 0.2252252   0.3142694  0.24670696 0.3333333   0.4291030
#>                  p_cand_dif
#> F.Man vs F.Woman  0.1415929
#> F.Man vs M.Man    0.1981132
#> F.Man vs M.Woman  0.5625000
#> F.Man vs S.Man    0.4300000
#> F.Man vs S.Woman  0.6185567
#> F.Man vs V.Man    0.4948454
#> F.Man vs V.Woman  0.2162162
#> 
#>  Fisher exact test 
#>                     statistic       p_stat c_stat
#> F.Man vs F.Woman 3.055425e-02 3.055425e-02    *  
#> F.Man vs M.Man   2.263738e-03 2.263738e-03    ** 
#> F.Man vs M.Woman 3.098685e-16 3.098685e-16    ***
#> F.Man vs S.Man   2.047536e-10 2.047536e-10    ***
#> F.Man vs S.Woman 4.126946e-19 4.126946e-19    ***
#> F.Man vs V.Man   3.888710e-13 3.888710e-13    ***
#> F.Man vs V.Woman 9.909588e-04 9.909588e-04    ***
#> 
#>  Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.10 ' ' 1

in this case, notice that the statistic equals the p-value of the test.

exclusive callback shares

This third approach considers discrimination cases only. They are defined by: $\begin{align*} e_1&=\frac{n_{10}}{n_d}\\ e_2&=\frac{n_{01}}{n_d}\\ n_d&=n_{10}+n_{01} \end{align*}$ and the equal treatment case is obtained when: $e_1=e_2\Leftrightarrow n_{10}=n_{01}$ which is equivalent to the two other approaches. We also have:

$p_i=e_i\times \pi_d$ where $\pi_d=n_d/n$ is the sample discrimination proportion. The function to use is now stat_ecs. In order to get the difference test and graphic, use:

ecs <- stat_ecs(dtest)
print(ecs)
#> 
#>  Proportions: exclusive callback shares 
#>  Confidence intervals: Student at 95 %
#>  
#>                  disc inf_p_cand1   p_cand1 sup_p_cand1 inf_p_cand2   p_cand2
#> F.Man vs F.Woman   70   0.4973830 0.6142857   0.7311884  0.26881159 0.3857143
#> F.Man vs M.Man     61   0.5509050 0.6721311   0.7933572  0.20664275 0.3278689
#> F.Man vs M.Woman   74   0.7851199 0.8648649   0.9446099  0.05539013 0.1351351
#> F.Man vs S.Man     71   0.7079719 0.8028169   0.8976619  0.10233810 0.1971831
#> F.Man vs S.Woman   82   0.7905088 0.8658537   0.9411985  0.05880146 0.1341463
#> F.Man vs V.Man     70   0.7554532 0.8428571   0.9302610  0.06973896 0.1571429
#> F.Man vs V.Woman   74   0.5518352 0.6621622   0.7724891  0.22751090 0.3378378
#>                  sup_p_cand2 inf_cand_dif p_cand_dif sup_cand_dif
#> F.Man vs F.Woman   0.5026170 -0.005233971  0.2285714    0.4623768
#> F.Man vs M.Man     0.4490950  0.101810098  0.3442623    0.5867145
#> F.Man vs M.Woman   0.2148801  0.570239712  0.7297297    0.8892197
#> F.Man vs S.Man     0.2920281  0.415943810  0.6056338    0.7953238
#> F.Man vs S.Woman   0.2094912  0.581017549  0.7317073    0.8823971
#> F.Man vs V.Man     0.2445468  0.510906498  0.6857143    0.8605221
#> F.Man vs V.Woman   0.4481648  0.103670447  0.3243243    0.5449782
#> 
#>  Student test 
#>                  statistic       p_stat c_stat
#> F.Man vs F.Woman  1.950287 5.520547e-02    .  
#> F.Man vs M.Man    2.840259 6.146665e-03    ** 
#> F.Man vs M.Woman  9.118748 1.115644e-13    ***
#> F.Man vs S.Man    6.367750 1.737316e-08    ***
#> F.Man vs S.Woman  9.661360 3.930644e-15    ***
#> F.Man vs V.Man    7.825524 4.167739e-11    ***
#> F.Man vs V.Woman  2.929373 4.529480e-03    ** 
#> 
#>  Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.10 ' ' 1
plot(ecs)

For the callback shares, enter :

plot(ecs, dif = FALSE)

For the Pearson test with the Wison intervals, enter:

ecs <- stat_ecs(dtest, method = "wilson")
print(ecs)
#> 
#>  Proportions: exclusive callback shares 
#>  Confidence intervals: Wilson at 95 %
#>  
#>                  disc inf_p_cand1   p_cand1 sup_p_cand1 inf_p_cand2   p_cand2
#> F.Man vs F.Woman   70   0.4900359 0.6142857   0.7259203  0.27407969 0.3857143
#> F.Man vs M.Man     61   0.5388397 0.6721311   0.7836560  0.21634405 0.3278689
#> F.Man vs M.Woman   74   0.7609303 0.8648649   0.9298227  0.07017734 0.1351351
#> F.Man vs S.Man     71   0.6880072 0.8028169   0.8842697  0.11573025 0.1971831
#> F.Man vs S.Woman   82   0.7684569 0.8658537   0.9279255  0.07207450 0.1341463
#> F.Man vs V.Man     70   0.7319501 0.8428571   0.9152454  0.08475465 0.1571429
#> F.Man vs V.Woman   74   0.5418852 0.6621622   0.7654580  0.23454199 0.3378378
#>                  sup_p_cand2 inf_cand_dif p_cand_dif sup_cand_dif
#> F.Man vs F.Woman   0.5099641   0.05302368  0.2285714    0.4041192
#> F.Man vs M.Man     0.4611603   0.16126875  0.3442623    0.5272558
#> F.Man vs M.Woman   0.2390697   0.60606089  0.7297297    0.8533986
#> F.Man vs S.Man     0.3119928   0.46066817  0.6056338    0.7505994
#> F.Man vs S.Woman   0.2315431   0.61519214  0.7317073    0.8482225
#> F.Man vs V.Man     0.2680499   0.55085882  0.6857143    0.8205698
#> F.Man vs V.Woman   0.4581148   0.15841131  0.3243243    0.4902373
#> 
#>  Pearson test 
#>                  statistic       p_stat c_stat
#> F.Man vs F.Woman  6.428571 1.122989e-02    *  
#> F.Man vs M.Man   13.114754 2.929793e-04    ***
#> F.Man vs M.Woman 75.918919 2.955547e-18    ***
#> F.Man vs S.Man   49.690141 1.800484e-12    ***
#> F.Man vs S.Woman 84.902439 3.134562e-20    ***
#> F.Man vs V.Man   63.114286 1.950543e-15    ***
#> F.Man vs V.Woman 14.297297 1.560888e-04    ***
#> 
#>  Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.10 ' ' 1
plot(ecs)

An R package for discrimination measurement

Emmanuel Duguet

2025-03-03