.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/plot_lightgbm_regressor.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_plot_lightgbm_regressor.py: LightGBM Regressor Optimization ================================ Hyperparameter optimization for LightGBM regressor using genetic algorithms. .. GENERATED FROM PYTHON SOURCE LINES 6-21 .. code-block:: default from sklearn.datasets import load_diabetes from sklearn.model_selection import train_test_split from sklearn.metrics import mean_squared_error, r2_score import numpy as np import plotly from mloptimizer.interfaces import HyperparameterSpaceBuilder, GeneticSearch from mloptimizer.application.reporting.plots import plotly_search_space, plotly_logbook # Import LightGBM try: from lightgbm import LGBMRegressor except ImportError: raise ImportError("Please install lightgbm: pip install lightgbm") .. GENERATED FROM PYTHON SOURCE LINES 22-23 Load and prepare the dataset .. GENERATED FROM PYTHON SOURCE LINES 23-29 .. code-block:: default print("Loading Diabetes dataset...") data = load_diabetes() X, y = data.data, data.target print(f"Dataset shape: {X.shape}") .. rst-class:: sphx-glr-script-out .. code-block:: none Loading Diabetes dataset... Dataset shape: (442, 10) .. GENERATED FROM PYTHON SOURCE LINES 30-31 Split the data .. GENERATED FROM PYTHON SOURCE LINES 31-35 .. code-block:: default X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42 ) .. GENERATED FROM PYTHON SOURCE LINES 36-37 Define the LightGBM hyperparameter space .. GENERATED FROM PYTHON SOURCE LINES 37-41 .. code-block:: default hyperparam_space = HyperparameterSpaceBuilder.get_default_space( estimator_class=LGBMRegressor ) .. GENERATED FROM PYTHON SOURCE LINES 42-43 Configure and run the genetic optimization .. GENERATED FROM PYTHON SOURCE LINES 43-62 .. code-block:: default genetic_params = { 'generations': 5, 'population_size': 8, 'n_elites': 2, 'seed': 42, 'use_mlflow': False, 'use_parallel': False } opt = GeneticSearch( estimator_class=LGBMRegressor, hyperparam_space=hyperparam_space, cv=3, **genetic_params ) print("Starting LightGBM Regressor optimization...") opt.fit(X_train, y_train) .. rst-class:: sphx-glr-script-out .. code-block:: none Starting LightGBM Regressor optimization... Genetic execution: 0%| | 0/6 [00:00 
GeneticSearch(cv=KFold(n_splits=3, random_state=42, shuffle=True),
                  estimator_class=<class 'lightgbm.sklearn.LGBMRegressor'>,
                  generations=5,
                  hyperparam_space=HyperparameterSpace(fixed_hyperparams={}, evolvable_hyperparams={'learning_rate': Hyperparam('learning_rate', 1, 100, 'float', 1000), 'max_depth': Hyperparam('max_depth', 2, 15, 'int'), 'n_estimators': Hyperparam...nt'), 'num_leaves': Hyperparam('num_leaves', 20, 150, 'int'), 'subsample': Hyperparam('subsample', 600, 1000, 'float', 1000), 'colsample_bytree': Hyperparam('colsample_bytree', 600, 1000, 'float', 1000), 'reg_alpha': Hyperparam('reg_alpha', 0, 10, 'float', 10), 'reg_lambda': Hyperparam('reg_lambda', 0, 10, 'float', 10)}),
                  n_elites=2, population_size=8, seed=42, use_parallel=False)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 63-64 Evaluate the optimized model .. GENERATED FROM PYTHON SOURCE LINES 64-74 .. code-block:: default best_reg = opt.best_estimator_ y_pred = best_reg.predict(X_test) test_mse = mean_squared_error(y_test, y_pred) test_r2 = r2_score(y_test, y_pred) print(f"\nOptimization completed!") print(f"Best parameters: {opt.best_params_}") print(f"Test MSE: {test_mse:.4f}") print(f"Test R2: {test_r2:.4f}") .. rst-class:: sphx-glr-script-out .. code-block:: none Optimization completed! Best parameters: {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 0.613, 'importance_type': 'split', 'learning_rate': 0.082, 'max_depth': 3, 'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0, 'n_estimators': 50, 'n_jobs': None, 'num_leaves': 90, 'objective': None, 'random_state': 42, 'reg_alpha': 0.8, 'reg_lambda': 0.3, 'subsample': 0.643, 'subsample_for_bin': 200000, 'subsample_freq': 0} Test MSE: 2716.6917 Test R2: 0.4872 .. GENERATED FROM PYTHON SOURCE LINES 75-76 Visualize the search space .. GENERATED FROM PYTHON SOURCE LINES 76-88 .. code-block:: default population_df = opt.populations_ top_params = ['learning_rate', 'max_depth', 'n_estimators', 'num_leaves', 'fitness'] df_filtered = population_df[top_params] g_search_space = plotly_search_space(df_filtered, top_params) g_search_space.update_layout( title="LightGBM Regressor Hyperparameter Search Space", autosize=True, width=None, height=650 ) plotly.io.show(g_search_space, config={'responsive': True}) .. raw:: html :file: images/sphx_glr_plot_lightgbm_regressor_001.html .. GENERATED FROM PYTHON SOURCE LINES 89-90 Visualize the optimization evolution .. GENERATED FROM PYTHON SOURCE LINES 90-99 .. code-block:: default g_logbook = plotly_logbook(opt.logbook_, population_df) g_logbook.update_layout( title="LightGBM Regressor Optimization Evolution", autosize=True, width=None, height=500 ) plotly.io.show(g_logbook, config={'responsive': True}) .. raw:: html :file: images/sphx_glr_plot_lightgbm_regressor_002.html .. GENERATED FROM PYTHON SOURCE LINES 100-101 Analyze optimization results .. GENERATED FROM PYTHON SOURCE LINES 101-106 .. code-block:: default print("\n=== Optimization Performance ===") print(f"Unique evaluations performed: {opt.n_trials_}") print(f"Total individuals in population history: {len(population_df)}") print(f"Optimization time: {opt.optimization_time_:.4f} seconds") print(f"Time per evaluation: {opt.optimization_time_ / opt.n_trials_:.4f} seconds") .. rst-class:: sphx-glr-script-out .. code-block:: none === Optimization Performance === Unique evaluations performed: 28 Total individuals in population history: 48 Optimization time: 1.5901 seconds Time per evaluation: 0.0568 seconds .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 3.185 seconds) .. _sphx_glr_download_auto_examples_plot_lightgbm_regressor.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_lightgbm_regressor.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_lightgbm_regressor.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_