Ctrl+K

mloptimizer.evaluation

mloptimizer.evaluation#

Submodules#

Package Contents#

Classes#

Evaluator

Evaluator class to evaluate the performance of a classifier

Functions#

kfold_stratified_score(features, labels, clf, metrics)

Computes KFold cross validation score using n_splits folds.

temporal_kfold_score(features, labels, clf, metrics[, ...])

Computes KFold cross validation score using n_splits folds.

train_score(features, labels, clf, metrics)

Trains the classifier with the features and labels.

train_test_score(features, labels, clf, metrics[, ...])

Trains the classifier with the train set features and labels,

kfold_score(features, labels, clf, metrics[, ...])

Evaluates the classifier using K-Fold cross-validation.
mloptimizer.evaluation.kfold_stratified_score(features, labels, clf, metrics, n_splits=4, random_state=None)[source]#

Computes KFold cross validation score using n_splits folds. It uses the features and labels to train the k-folds. Uses a stratified KFold split. The score_function is the one used to score each k-fold.

Parameters:

  • features (list) – List of features

  • labels (list) – List of labels

  • clf (object) – classifier with methods fit, predict and score

  • n_splits (int) – number of splits

  • metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

  • random_state (int) – random state for the stratified kfold

Returns:

average_metrics – mean score among k-folds test splits

Return type:

dict

mloptimizer.evaluation.temporal_kfold_score(features, labels, clf, metrics, n_splits=4)[source]#

Computes KFold cross validation score using n_splits folds. It uses the features and labels to train the k-folds. Uses a temporal KFold split. The score_function is the one used to score each k-fold.

Parameters:

  • features (list) – List of features

  • labels (list) – List of labels

  • clf (object) – classifier with methods fit, predict and score

  • n_splits (int) – number of splits

  • metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

Returns:

average_metrics – mean score among k-folds test splits

Return type:

dict

mloptimizer.evaluation.train_score(features, labels, clf, metrics)[source]#

Trains the classifier with the features and labels.

Parameters:

  • features (list) – List of features

  • labels (list) – List of labels

  • clf (object) – classifier with methods fit, predict and score

  • metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

Returns:

metrics_output – dictionary with the metrics over the train set

Return type:

dict

mloptimizer.evaluation.train_test_score(features, labels, clf, metrics, test_size=0.2, random_state=None)[source]#

Trains the classifier with the train set features and labels, then uses the test features and labels to create score.

Parameters:

  • features (list) – List of features

  • labels (list) – List of labels

  • clf (object) – Classifier with methods fit, predict, and score

  • metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

  • test_size (float, optional) – Proportion of the dataset to include in the test split

  • random_state (int, optional) – Controls the shuffling applied to the data before applying the split

Returns:

metrics_output – dictionary with the metrics over the test set

Return type:

dict

mloptimizer.evaluation.kfold_score(features, labels, clf, metrics, n_splits=5, random_state=None)[source]#

Evaluates the classifier using K-Fold cross-validation.

Parameters:

  • features (array-like) – Array of features

  • labels (array-like) – Array of labels

  • clf (object) – Classifier with methods fit and predict

  • metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

  • n_splits (int, optional) – Number of folds. Must be at least 2

  • random_state (int, optional) – Controls the randomness of the fold assignment

Returns:

average_metrics – mean score among k-folds test splits

Return type:

dict

class mloptimizer.evaluation.Evaluator(features: numpy.array, labels: numpy.array, eval_function, fitness_score='accuracy', metrics=None, tracker: mloptimizer.aux.Tracker = None, individual_utils=None)[source]#

Evaluator class to evaluate the performance of a classifier

Parameters:

  • features (array-like) – The features to use to evaluate the classifier

  • labels (array-like) – The labels to use to evaluate the classifier

  • eval_function (function) – The evaluation function to use to evaluate the performance of the classifier

  • fitness_score (str) – The fitness score to use to evaluate the performance of the classifier

  • metrics (dict) – The metrics to use to evaluate the performance of the classifier Dictionary of the form {“metric_name”: metric_function}

  • tracker (Tracker) – The tracker to use to log the evaluations

  • individual_utils (IndividualUtils) – The individual utils to use to get the classifier from the individual

evaluate(clf, features, labels)[source]#

Evaluate the performance of a classifier

Parameters:

  • clf (object) – The classifier to evaluate

  • features (array-like) – The features to use to evaluate the classifier

  • labels (array-like) – The labels to use to evaluate the classifier

Returns:

metrics – Dictionary of the form {“metric_name”: metric_value}

Return type:

dict

evaluate_individual(individual)[source]#
Contents