modelEval {CORElearn} | R Documentation |
Using predictions of given model produced by predict.CoreModel
and correct labels,
computes some statistics evaluating the quality of the model.
modelEval(model, correctClass, predictedClass, predictedProb=NULL, costMatrix=NULL, priorClProb = NULL, avgTrainPrediction = NULL)
model |
The model structure as returned by CoreModel , or NULL if some other predictions are evaluated. |
correctClass |
A vector of correct class labels for classification problem and function values for regression problem. |
predictedClass |
A vector of predicted class labels for classification problem and function values for regression problem. |
predictedProb |
An optional matrix of predicted class probabilities for classification. |
costMatrix |
Optional cost matrix can provide nonuniform costs for classification problems. |
priorClProb |
If model=NULL a vector of prior class probabilities shell be provided in case of classification. |
avgTrainPrediction |
If model=NULL mean of prediction values on training set shell be provided in case of regression. |
The function uses the model
structure as returned by CoreModel
,
predictedClass
and (optionally) predictedProb
returned by
predict.CoreModel
. Predicted values are compared with true values
and some statistics are computed measuring the quality of predictions.
Optional cost matrix can provide nonuniform costs for classification problems. For regression
problem this parameter is ignored. The costs can be different from the ones used for building the model
in CoreModel
and predicting with the model in predict.CoreModel
.
If no costs are supplied uniform costs are assumed where necessary.
If a non CORElearn model is evaluated, one should set model=NULL
, and a vector of prior of class
probabilities priorClProb
shell be provided in case of classification,
and in case of regression avgTrainPrediction
shell be mean of prediction values on training set.
For classification problem function returns list with the components
accuracy |
classification accuracy |
averageCost |
average classification cost |
informationScore |
statistics measuring information in the predicted probabilities |
AUC |
Area under the ROC curve |
predictionMatrix |
matrix of miss-classifications |
sensitivity |
sensitivity for two class problems |
specificity |
specificity for two class problems |
brierScore |
Brier score of predicted probabilities |
kappa |
Kappa statistics measuring randomness of the predictions |
MSE |
Mean Squared Error |
RMSE |
Relative Mean Squared Error |
MAE |
Mean Absolute Error |
RMAE |
Relative Mean Absolute Error |
Marko Robnik-Sikonja, Petr Savicky
Igor Kononenko, Matjaz Kukar: Machine Learning and Data Mining: Introduction to Principles and Algorithms. Horwood, 2007
CORElearn
,
CoreModel
,
predict.CoreModel
.
# use iris data # build random forests model with certain parameters model <- CoreModel(Species ~ ., iris, model="rf", selectionEstimator="MDL",minNodeWeight=5,rfNoTrees=100) # prediction with node distribution pred <- predict.CoreModel(model, iris, rfPredictClass=FALSE) # Model evaluation mEval <- modelEval(model, iris[["Species"]], pred$class, pred$prob) print(mEval)