Package to assess the calibration of probabilistic classifiers using confidence bands for monotonic functions. Besides testing the classical goodness-of-fit null hypothesis of perfect calibration, the confidence bands calculated within that package facilitate inverted goodness-of-fit tests whose rejection allows for a sought-after conclusion of a sufficiently well-calibrated model. The package creates flexible graphical tools to perform these tests.  For construction details see also Dimitriadis, Dümbgen, Henzi, Puke, Ziegel (2022) <arXiv:2203.04065>.

Plots calibration curves and computes statistics for assessing calibration performance. See De Cock Campo (2023) <doi:10.48550/arXiv.2309.08559> and Van Calster et al. (2016) <doi:10.1016/j.jclinepi.2015.12.005>.

Performs Bayesian calibration of computer models as per
 Kennedy and O'Hagan 2001.  The package includes routines to find the
 hyperparameters and parameters; see the help page for stage1() for a
 worked example using the toy dataset.  A tutorial is provided in the
 calex.Rnw vignette; and a suite of especially simple one dimensional
 examples appears in inst/doc/one.dim/.

Transforms your uncalibrated Machine Learning scores to well-calibrated prediction estimates that can be interpreted as probability estimates. The implemented BBQ (Bayes Binning in Quantiles) model is taken from Naeini (2015, ISBN:0-262-51129-0). Please cite this paper: Schwarz J and Heider D, Bioinformatics 2019, 35(14):2458-2465.

Assesses the quality of estimates made by complex sample designs,
  following the methodology developed by the National Institute of Statistics Chile (2020, <https://www.ine.cl/docs/default-source/institucionalidad/buenas-pr%C3%A1cticas/clasificaciones-y-estandares/est%C3%A1ndar-evaluaci%C3%B3n-de-calidad-de-estimaciones-publicaci%C3%B3n-27022020.pdf>)
  and by Economic Commission  for Latin America and
  Caribbean (2020, <https://repositorio.cepal.org/bitstream/handle/11362/45681/1/S2000293_es.pdf>).

In discrimination experiments candidates are sent on the same test (e.g. job, house rental) and one examines whether they receive the same outcome. The number of non negative answers are first examined in details looking for outcome differences. Then various answering rates and their exacts confidence intervals are computed. Last, exact and asymptotic discrimination tests are performed (Student, binomial).