By default, Weka treats all types of classification errors equally. In many practical cases though, not all errors are equal. If the costs are known, it is possible to use this information to build a cost sensitive model or to alter model predictions to minimize expected costs. The way to do this in Weka is to create cost files. Cost files are small text files that define a cost matrix. An example of this is shown below:

% Rows Columns

2 2

% Matrix elements

0 2

1 0

Like we described in the ARFF file format, comment lines begin with ‘%’ and are ignored. The Second line defines the size of the cost matrix. The size of the cost matrix depends on the number of levels in the class variable. The cost matrix must be a square matrix with zeros on the diagonal and all other elements attach a cost to a particular kind of misclassification error. The default is to treat all errors equally so all non-diagonal elements in the matrix are set to 1.

In our example the class variable weight has two levels (low and normal weight) so the size of the cost matrix is 2x2. It also defines that misclassifying low birth-weight instance is to be twice as expensive as misclassifying a normal birth-weight instance. From here, the user can choose how this cost information is used in one of two ways.

The first option is to alter predictions to take in the account the cost information. This means being surer in cases where misclassification is expensive and less so when it is not as expensive. This is done by changing the threshold levels. It is important to note that this option does not influence the tree building process; the model is exactly the same as before. What is different here is the classification of the leaf nodes.

If for example it is determined, that to misclassify a low birth weight case as normal weight is more expensive the class designation rule can be changed. By default, the simple majority class in the leaf is the class that is assigned to it. In our birth weight example, this corresponds with a threshold of 51%. However, this prediction has a probability of success of only 51%. If it is expensive to misclassify something as low birth weight, the minimum percentage required for a leaf to be designated as low birth weight can be raised to say 70%. The idea here is to minimize the total expected costs by making cheap mistakes more often and expensive mistakes less often.

The other option is to re-weight the data according to the cost matrix so that the generated model is inherently less likely to make expensive mistakes. This is done by resampling the data so that a particular class is over represented. Therefore, if sensitivity to low birth weight cases is required we can resample the data so that there are a lot more low birth weight cases in the resample data.

The generated tree will then be more sensitive to this class as opposed to the other classes. Weka allows any classification algorithm to be made cost sensitive by using the meta-classification algorithm CostSensitiveClassifier and setting its base classifier to be the desired algorithm.

The default base classifier is set to the simple rule-based algorithm ZeroR. To change this open the options window for the meta-classifier and then in that window click on the choose button. From the list then choose the desired algorithm. To set the options for the base classifier simply click on its name and this will open the options window for the algorithm. The CostSensitiveClassifier works by re-weighting the data using the information in the cost matrix. The data is then passed to the base classifier to generate the new model.