Advantages of the Experimenter Interface:

The main advantage of using the Experimenter interface is that it allows the user to compare multiple learning schemes across one or more datasets. In simple mode, a user can setup simple experiments very quickly and easily. The built-in analyzer is also easy to use and allows the user to perform various kinds of tests easily.

Though we did not look at the advanced mode, it gives the user the ability to run experiments that are more complex and even to break the experiment across multiple computers. In this mode, the user has a finer control of how each run of the experiment will be conducted.

Disadvantages of the Experimenter Interface:

The Experimenter’s main weakness is the lack of preprocessing facilities as none of Weka’s Filters can be accessed in this interface. Another disadvantage is the lack of any graphical visualization tools. It would have been nice, for example, to display multiple ROC curves on the same graph. Another problem is that when more than 1 of the same learning scheme is being tested but with different options, it is not easy to tell what resultset belongs to what learning scheme. It would be nice if a user would be able to label a given scheme or some way of easily identifying a particular result set as the names given automatically by the system can be a bit confusing. Another disadvantage is that it is not possible to select the class variable as the Experimenter will always take the last attribute in the data set to be the class variable.