calibrate {CORElearn} | R Documentation |
Given probability scores predictedProb
as provided for example by a call to predict.CoreModel
and using one of available methods given by methods
the function calibrates predicted probabilities so that they
match the actual probabilities of a binary class 1 provided by correctClass
.
calibrate(correctClass, predictedProb, class1=1, method = c("isoReg","binIsoReg","binning","chiMerge"), weight=NULL,noBins=10)
correctClass |
A vector of correct class labels for a binary classification problem. |
predictedProb |
A vector of predicted class 1 probability scores of the same length as correctClass . |
class1 |
A class value (factor) or an index of the class value to be taken as a class to be calibrated. |
method |
One of isoReg , binIsoReg , binning , or chiMerge . See details below. |
weight |
If specified, should be of the same length as correctClass and gives weights for all the instances,
otherwise a default weight of 1 for each instance is assumed. |
noBins |
If model="binning" or model="binIsoReg" specifies desired number of bins i.e., calibration bands. |
Depending on the specified method
one of the following calibration methods is executed.
"isoReg"
isotonic regression calibration based on pair-adjacent violators (PAV) algorithm.
"binning"
calibration into a pre-specified number of bands given by noBins
parameter.
"binIsoReg"
first binning method is executed, following by a isotonic regression calibration.
"chiMerge"
first intervals are merged by a chi-squared statistics criterion, following by the isotonic regression calibration.
A function returns a list with two vector components of the same length:
interval |
The boundaries of the intervals. Lower boundary 0 is not explicitly included but should be taken into account. |
calProb |
The calibrated probabilities for each corresponding interval. |
Marko Robnik-Sikonja, Petr Savicky
I. Kononenko, M. Kukar: Machine Learning and Data Mining: Introduction to Principles and Algorithms. Horwood, 2007
A. Niculescu-Mizil, R. Caruana: Predicting Good Probabilities With Supervised Learning. Proceedings of the 22nd International Conference on Machine Learning (ICML'05), 2005
# generate data train <-classDataGen(noInst=200) cal <-classDataGen(noInst=200) # build random forests model with certain parameters modelRF <- CoreModel(class~., train, model="rf", selectionEstimator="MDL",minNodeWeight=5,rfNoTrees=100) # prediction pred <- predict.CoreModel(modelRF, cal, rfPredictClass=FALSE) # calibrate for a chosen class1 and method class1<-1 calibrate(cal$class, pred$prob[,class1], class1=1, method="binning", noBins=5)