calibrate {CORElearn}R Documentation

Calibration of probabilities according to the given prior.

Description

Given probability scores predictedProb as provided for example by a call to predict.CoreModel and using one of available methods given by methods the function calibrates predicted probabilities so that they match the actual probabilities of a binary class 1 provided by correctClass.

Usage

calibrate(correctClass, predictedProb, class1=1, method = c("isoReg","binIsoReg","binning","chiMerge"), weight=NULL,noBins=10)

Arguments

correctClass A vector of correct class labels for a binary classification problem.
predictedProb A vector of predicted class 1 probability scores of the same length as correctClass.
class1 A class value (factor) or an index of the class value to be taken as a class to be calibrated.
method One of isoReg, binIsoReg, binning, or chiMerge. See details below.
weight If specified, should be of the same length as correctClass and gives weights for all the instances, otherwise a default weight of 1 for each instance is assumed.
noBins If model="binning" or model="binIsoReg" specifies desired number of bins i.e., calibration bands.

Details

Depending on the specified method one of the following calibration methods is executed.

Value

A function returns a list with two vector components of the same length:

interval The boundaries of the intervals. Lower boundary 0 is not explicitly included but should be taken into account.
calProb The calibrated probabilities for each corresponding interval.

Author(s)

Marko Robnik-Sikonja, Petr Savicky

References

I. Kononenko, M. Kukar: Machine Learning and Data Mining: Introduction to Principles and Algorithms. Horwood, 2007

A. Niculescu-Mizil, R. Caruana: Predicting Good Probabilities With Supervised Learning. Proceedings of the 22nd International Conference on Machine Learning (ICML'05), 2005

See Also

CORElearn, predict.CoreModel.

Examples

# generate data
train <-classDataGen(noInst=200)
cal <-classDataGen(noInst=200)

# build random forests model with certain parameters
modelRF <- CoreModel(class~., train, model="rf",   selectionEstimator="MDL",minNodeWeight=5,rfNoTrees=100)
# prediction 
pred <- predict.CoreModel(modelRF, cal, rfPredictClass=FALSE)

# calibrate for a chosen class1 and method
class1<-1
calibrate(cal$class, pred$prob[,class1], class1=1, method="binning", noBins=5)

[Package CORElearn version 0.9.22 Index]