.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/regression_example.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_regression_example.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_regression_example.py:


Regression Example
==================
This example shows how to optimize a regression model using the mloptimizer library.

.. GENERATED FROM PYTHON SOURCE LINES 8-11

Load the diabetes dataset
-------------------------
The diabetes dataset is used to demonstrate the optimization of a regression model.

.. GENERATED FROM PYTHON SOURCE LINES 11-15

.. code-block:: default

    from sklearn.datasets import load_diabetes

    X, y = load_diabetes(return_X_y=True)


.. GENERATED FROM PYTHON SOURCE LINES 16-17

The dataset has 10 features, but we will use only one for this plot (the Body Mass Index (BMI)).

.. GENERATED FROM PYTHON SOURCE LINES 17-24

.. code-block:: default

    import matplotlib.pyplot as plt
    plt.scatter(X[:,2], y)
    plt.xlabel("Body Mass Index")
    plt.ylabel("Disease Progression")
    plt.title("Diabetes Dataset")
    plt.show()


.. GENERATED FROM PYTHON SOURCE LINES 25-29

Default hyperparameter space
----------------------------
Define the HyperparameterSpace, you can use the default hyperparameters for the machine learning model
that you want to optimize. In this case we use the default hyperparameters for a RandomForestRegressor.

.. GENERATED FROM PYTHON SOURCE LINES 29-34

.. code-block:: default

    from mloptimizer.domain.hyperspace import HyperparameterSpace
    from sklearn.ensemble import RandomForestRegressor

    hyperparam_space = HyperparameterSpace.get_default_hyperparameter_space(RandomForestRegressor)


.. GENERATED FROM PYTHON SOURCE LINES 35-38

We use the GeneticSearch class to optimize a regression model.
Configure the genetic algorithm with default parameters shown explicitly.
Note: Values reduced for faster documentation builds. For production, use larger values.

.. GENERATED FROM PYTHON SOURCE LINES 38-57

.. code-block:: default

    from mloptimizer.interfaces import GeneticSearch

    genetic_params = {
        'generations': 8,
        'population_size': 15,
        'n_elites': 2,
        'seed': 42
    }

    mlopt = GeneticSearch(
        estimator_class=RandomForestRegressor,
        hyperparam_space=hyperparam_space,
        cv=5,
        scoring='rmse',  # Root Mean Squared Error (available: 'rmse', 'mse', 'mae')
        **genetic_params
    )

    mlopt.fit(X, y)
    clf = mlopt.best_estimator_

.. GENERATED FROM PYTHON SOURCE LINES 58-62

The best estimator with optimized hyperparameters is stored in ``best_estimator_``.
This estimator is trained with the best hyperparameters found during optimization.
As the literature suggests, you can retrain with all the data or use cross-validation
estimators for making predictions with better generalization.

.. GENERATED FROM PYTHON SOURCE LINES 62-64

.. code-block:: default

    print(clf)


.. GENERATED FROM PYTHON SOURCE LINES 65-69

Custom hyperparameter space
---------------------------
The hyperparameter space can be defined by the user.
In this example, we define a new hyperparameter space for the RandomForestRegressor model.

.. GENERATED FROM PYTHON SOURCE LINES 69-80

.. code-block:: default

    from mloptimizer.domain.hyperspace import Hyperparam
    fixed_hyperparams = {'n_estimators': 100}
    evolvable_hyperparams = {'max_depth': Hyperparam(name='max_depth', min_value=1,
                                                     max_value=10, hyperparam_type='int'),
                             'min_samples_split': Hyperparam(name='min_samples_split', min_value=2,
                                                             max_value=10, hyperparam_type='int'),
                             'min_samples_leaf': Hyperparam(name='min_samples_leaf', min_value=1,
                                                            max_value=10, hyperparam_type='int')}
    custom_hyperparam_space = HyperparameterSpace(fixed_hyperparams, evolvable_hyperparams)



.. GENERATED FROM PYTHON SOURCE LINES 81-84

Furthermore, it is possible to set the metrics used to evaluate the model.
We configure the genetic algorithm for this custom hyperparameter space.
Note: Values reduced for faster documentation builds.

.. GENERATED FROM PYTHON SOURCE LINES 84-103

.. code-block:: default

    genetic_params_custom = {
        'generations': 8,
        'population_size': 15,
        'n_elites': 2,
        'cxpb': 0.5,
        'mutpb': 0.8,
        'seed': 42
    }

    mlopt = GeneticSearch(
        estimator_class=RandomForestRegressor,
        hyperparam_space=custom_hyperparam_space,
        cv=5,
        scoring='rmse',  # Root Mean Squared Error
        **genetic_params_custom
    )

    mlopt.fit(X, y)

    print(mlopt.best_estimator_)

.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.000 seconds)


.. _sphx_glr_download_auto_examples_regression_example.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example




    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: regression_example.py <regression_example.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: regression_example.ipynb <regression_example.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_