mloptimizer.evaluation.model_evaluation

Module Contents

Functions

score_metrics(labels, predictions, metrics)
train_score(features, labels, clf, metrics)	Trains the classifier with the features and labels.
train_test_score(features, labels, clf, metrics[, ...])	Trains the classifier with the train set features and labels,
kfold_score(features, labels, clf, metrics[, ...])	Evaluates the classifier using K-Fold cross-validation.
kfold_stratified_score(features, labels, clf, metrics)	Computes KFold cross validation score using n_splits folds.
temporal_kfold_score(features, labels, clf, metrics[, ...])	Computes KFold cross validation score using n_splits folds.

mloptimizer.evaluation.model_evaluation.score_metrics(labels, predictions, metrics)

mloptimizer.evaluation.model_evaluation.train_score(features, labels, clf, metrics)

Trains the classifier with the features and labels.

Parameters:

• features (list) – List of features

• labels (list) – List of labels

• clf (object) – classifier with methods fit, predict and score

• metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

Returns:

metrics_output – dictionary with the metrics over the train set

Return type:

dict

mloptimizer.evaluation.model_evaluation.train_test_score(features, labels, clf, metrics, test_size=0.2, random_state=None)

Trains the classifier with the train set features and labels, then uses the test features and labels to create score.

Parameters:

• features (list) – List of features

• labels (list) – List of labels

• clf (object) – Classifier with methods fit, predict, and score

• metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

• test_size (float, optional) – Proportion of the dataset to include in the test split

• random_state (int, optional) – Controls the shuffling applied to the data before applying the split

Returns:

metrics_output – dictionary with the metrics over the test set

Return type:

dict

mloptimizer.evaluation.model_evaluation.kfold_score(features, labels, clf, metrics, n_splits=5, random_state=None)

Evaluates the classifier using K-Fold cross-validation.

Parameters:

• features (array-like) – Array of features

• labels (array-like) – Array of labels

• clf (object) – Classifier with methods fit and predict

• metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

• n_splits (int, optional) – Number of folds. Must be at least 2

• random_state (int, optional) – Controls the randomness of the fold assignment

Returns:

average_metrics – mean score among k-folds test splits

Return type:

dict

mloptimizer.evaluation.model_evaluation.kfold_stratified_score(features, labels, clf, metrics, n_splits=4, random_state=None)

Computes KFold cross validation score using n_splits folds. It uses the features and labels to train the k-folds. Uses a stratified KFold split. The score_function is the one used to score each k-fold.

Parameters:

• features (list) – List of features

• labels (list) – List of labels

• clf (object) – classifier with methods fit, predict and score

• n_splits (int) – number of splits

• metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

• random_state (int) – random state for the stratified kfold

Returns:

average_metrics – mean score among k-folds test splits

Return type:

dict

mloptimizer.evaluation.model_evaluation.temporal_kfold_score(features, labels, clf, metrics, n_splits=4)

Computes KFold cross validation score using n_splits folds. It uses the features and labels to train the k-folds. Uses a temporal KFold split. The score_function is the one used to score each k-fold.

Parameters:

• features (list) – List of features

• labels (list) – List of labels

• clf (object) – classifier with methods fit, predict and score

• n_splits (int) – number of splits

• metrics (dict) – dictionary with metrics to be used keys are the name of the metric and values are the metric function

Returns:

average_metrics – mean score among k-folds test splits

Return type:

dict