.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/regression_example.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_regression_example.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_regression_example.py:


Regression Example
==================
This example shows how to optimize a regression model using the mloptimizer library.

.. GENERATED FROM PYTHON SOURCE LINES 8-11

Load the diabetes dataset
-------------------------
The diabetes dataset is used to demonstrate the optimization of a regression model.

.. GENERATED FROM PYTHON SOURCE LINES 11-15

.. code-block:: default

    from sklearn.datasets import load_diabetes

    X, y = load_diabetes(return_X_y=True)


.. GENERATED FROM PYTHON SOURCE LINES 16-17

The dataset has 10 features, but we will use only one for this plot (the Body Mass Index (BMI)).

.. GENERATED FROM PYTHON SOURCE LINES 17-24

.. code-block:: default

    import matplotlib.pyplot as plt
    plt.scatter(X[:,2], y)
    plt.xlabel("Body Mass Index")
    plt.ylabel("Disease Progression")
    plt.title("Diabetes Dataset")
    plt.show()


.. GENERATED FROM PYTHON SOURCE LINES 25-29

Default hyperparameter space
----------------------------
Define the HyperparameterSpace, you can use the default hyperparameters for the machine learning model
that you want to optimize. In this case we use the default hyperparameters for a RandomForestRegressor.

.. GENERATED FROM PYTHON SOURCE LINES 29-34

.. code-block:: default

    from mloptimizer.domain.hyperspace import HyperparameterSpace
    from sklearn.ensemble import RandomForestRegressor

    hyperparam_space = HyperparameterSpace.get_default_hyperparameter_space(RandomForestRegressor)


.. GENERATED FROM PYTHON SOURCE LINES 35-36

We use the Optimizer class to optimize a regression model.

.. GENERATED FROM PYTHON SOURCE LINES 36-42

.. code-block:: default

    from mloptimizer.interfaces import GeneticSearch
    mlopt = GeneticSearch(estimator_class=RandomForestRegressor,
                          hyperparam_space=hyperparam_space)

    mlopt.fit(X, y)
    clf = opt.best_estimator_

.. GENERATED FROM PYTHON SOURCE LINES 43-47

The best individual is returned by the optimize_clf method.
However, this individual is not the trained model, but the hyperparameters used to train the best model.
As the literature suggests, we can use the best hyperparameters to train the model again with all the data
to obtain the best model or use a cross-validation estimator to make new predictions.

.. GENERATED FROM PYTHON SOURCE LINES 47-49

.. code-block:: default

    print(clf)


.. GENERATED FROM PYTHON SOURCE LINES 50-54

Custom hyperparameter space
---------------------------
The hyperparameter space can be defined by the user.
In this example, we define a new hyperparameter space for the RandomForestRegressor model.

.. GENERATED FROM PYTHON SOURCE LINES 54-65

.. code-block:: default

    from mloptimizer.domain.hyperspace import Hyperparam
    fixed_hyperparams = {'n_estimators': 100}
    evolvable_hyperparams = {'max_depth': Hyperparam(name='max_depth', min_value=1,
                                                     max_value=10, hyperparam_type='int'),
                             'min_samples_split': Hyperparam(name='min_samples_split', min_value=2,
                                                             max_value=10, hyperparam_type='int'),
                             'min_samples_leaf': Hyperparam(name='min_samples_leaf', min_value=1,
                                                            max_value=10, hyperparam_type='int')}
    custom_hyperparam_space = HyperparameterSpace(fixed_hyperparams, evolvable_hyperparams)



.. GENERATED FROM PYTHON SOURCE LINES 66-67

Furthermore, it is possible to set the metrics used to evaluate the model, and use one of them as the fitness score.

.. GENERATED FROM PYTHON SOURCE LINES 67-79

.. code-block:: default

    from sklearn.metrics import mean_squared_error, root_mean_squared_error
    #regression_metrics = {
    #        "mse": mean_squared_error,
    #        "rmse": root_mean_squared_error
    #}

    mlopt = GeneticSearch(estimator_class=RandomForestRegressor,
                          hyperparam_space=custom_hyperparam_space,
                          scoring='rmse')

    mlopt.fit(X, y)

    print(mlopt.best_estimator_)

.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.000 seconds)


.. _sphx_glr_download_auto_examples_regression_example.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example




    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: regression_example.py <regression_example.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: regression_example.ipynb <regression_example.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_