Quickstart example

Quick example of use of the library to optimize a decision tree classifier. Firstly, we import the necessary libraries to get data and plot the results.

from mloptimizer.core import Optimizer
from mloptimizer.hyperparams import HyperparameterSpace
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris

Load the iris dataset to obtain a vector of features X and a vector of labels y. Another dataset or a custom one can be used

X, y = load_iris(return_X_y=True)

Split the dataset into training and test sets

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

Define the HyperparameterSpace, you can use the default hyperparameters for the machine learning model that you want to optimize. In this case we use the default hyperparameters for a DecisionTreeClassifier. Another dataset or a custom one can be used

hyperparam_space = HyperparameterSpace.get_default_hyperparameter_space(DecisionTreeClassifier)

The TreeOptimizer class is the main wrapper for the optimization of a Decision Tree Classifier. The first argument is the vector of features, the second is the vector of labels and the third (if provided) is the name of the folder where the results of mloptimizer Optimizers are saved. The default value for this folder is “Optimizer”

opt = Optimizer(estimator_class=DecisionTreeClassifier, features=X_train, labels=y_train,
                hyperparam_space=hyperparam_space, folder="Optimizer")

To optimizer the classifier we need to call the optimize_clf method. The first argument is the number of generations and the second is the number of individuals in each generation. The method returns the best classifier with the best hyperparameters found.

clf = opt.optimize_clf(10, 10)

print(clf)

Genetic execution:   0%|          | 0/11 [00:00<?, ?it/s]/home/docs/checkouts/readthedocs.org/user_builds/mloptimizer/envs/dev/lib/python3.10/site-packages/deap/creator.py:185: RuntimeWarning:

A class named 'FitnessMax' has already been created and it will be overwritten. Consider deleting previous creation of that class or rename it.

/home/docs/checkouts/readthedocs.org/user_builds/mloptimizer/envs/dev/lib/python3.10/site-packages/deap/creator.py:185: RuntimeWarning:

A class named 'Individual' has already been created and it will be overwritten. Consider deleting previous creation of that class or rename it.


Genetic execution:  55%|█████▍    | 6/11 [00:00<00:00, 53.52it/s]
Genetic execution: 100%|██████████| 11/11 [00:00<00:00, 46.16it/s]
DecisionTreeClassifier(ccp_alpha=0.00256, max_depth=19,
                       min_impurity_decrease=0.063, min_samples_leaf=20,
                       min_samples_split=38, random_state=402428)

Train the classifier with the best hyperparameters found Show the classification report and the confusion matrix

from sklearn.metrics import classification_report, confusion_matrix, \
    ConfusionMatrixDisplay
import matplotlib.pyplot as plt

clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
cm = confusion_matrix(y_test, y_pred)
print(classification_report(y_test, y_pred))
disp = ConfusionMatrixDisplay.from_predictions(
    y_test, y_pred, display_labels=clf.classes_,
    cmap=plt.cm.Blues
)
disp.plot()
plt.show()

del opt

• 
• 

              precision    recall  f1-score   support

           0       1.00      1.00      1.00        11
           1       0.92      1.00      0.96        11
           2       1.00      0.88      0.93         8

    accuracy                           0.97        30
   macro avg       0.97      0.96      0.96        30
weighted avg       0.97      0.97      0.97        30