.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/plot_mlp_neural_network.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_plot_mlp_neural_network.py: Neural Network (MLP) Optimization ================================== Hyperparameter optimization for Multi-Layer Perceptron neural networks. .. GENERATED FROM PYTHON SOURCE LINES 6-17 .. code-block:: default from sklearn.datasets import load_breast_cancer from sklearn.neural_network import MLPClassifier from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score, f1_score from sklearn.preprocessing import StandardScaler import numpy as np import plotly from mloptimizer.interfaces import HyperparameterSpaceBuilder, GeneticSearch from mloptimizer.application.reporting.plots import plotly_search_space, plotly_logbook .. GENERATED FROM PYTHON SOURCE LINES 18-19 Load and prepare the dataset .. GENERATED FROM PYTHON SOURCE LINES 19-29 .. code-block:: default print("Loading Breast Cancer dataset...") data = load_breast_cancer() X, y = data.data, data.target # Scale features for neural networks scaler = StandardScaler() X = scaler.fit_transform(X) print(f"Dataset shape: {X.shape}") .. rst-class:: sphx-glr-script-out .. code-block:: none Loading Breast Cancer dataset... Dataset shape: (569, 30) .. GENERATED FROM PYTHON SOURCE LINES 30-31 Split the data .. GENERATED FROM PYTHON SOURCE LINES 31-35 .. code-block:: default X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, stratify=y ) .. GENERATED FROM PYTHON SOURCE LINES 36-37 Define the hyperparameter space .. GENERATED FROM PYTHON SOURCE LINES 37-41 .. code-block:: default hyperparam_space = HyperparameterSpaceBuilder.get_default_space( estimator_class=MLPClassifier ) .. GENERATED FROM PYTHON SOURCE LINES 42-43 Configure and run the genetic optimization .. GENERATED FROM PYTHON SOURCE LINES 43-63 .. code-block:: default genetic_params = { 'generations': 5, 'population_size': 8, 'n_elites': 2, 'seed': 42, 'use_mlflow': False, 'use_parallel': False } opt = GeneticSearch( estimator_class=MLPClassifier, hyperparam_space=hyperparam_space, cv=3, scoring='accuracy', **genetic_params ) print("Starting MLPClassifier optimization...") opt.fit(X_train, y_train) .. rst-class:: sphx-glr-script-out .. code-block:: none /home/docs/checkouts/readthedocs.org/user_builds/mloptimizer/checkouts/master/examples/plot_mlp_neural_network.py:52: UserWarning: Expected mutations per offspring is very low (0.48). With mutpb=0.8, indpb=0.2, and 3 hyperparameters, the population will converge prematurely. Recommended: mutpb >= 0.8, indpb >= 0.2 (gives ~0.5 mutations/offspring). opt = GeneticSearch( /home/docs/checkouts/readthedocs.org/user_builds/mloptimizer/checkouts/master/examples/plot_mlp_neural_network.py:52: UserWarning: Some hyperparameters have very small integer ranges (< 10 distinct values): 'activation' (3 values: 0 to 2). Small ranges limit search granularity. Consider increasing the range or scale for float types. opt = GeneticSearch( Starting MLPClassifier optimization... Genetic execution: 0%| | 0/6 [00:00 
GeneticSearch(cv=StratifiedKFold(n_splits=3, random_state=42, shuffle=True),
                  estimator_class=<class 'sklearn.neural_network._multilayer_perceptron.MLPClassifier'>,
                  generations=5,
                  hyperparam_space=HyperparameterSpace(fixed_hyperparams={'max_iter': 500, 'hidden_layer_sizes': [100]}, evolvable_hyperparams={'alpha': Hyperparam('alpha', 1, 1000, 'float', 10000), 'learning_rate_init': Hyperparam('learning_rate_init', 1, 100, 'float', 10000), 'activation': Hyperparam('activation', 0, 2, 'list', ['relu', 'tanh', 'logistic'])}),
                  n_elites=2, population_size=8, scoring='accuracy', seed=42,
                  use_parallel=False)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 64-65 Evaluate the optimized model .. GENERATED FROM PYTHON SOURCE LINES 65-75 .. code-block:: default best_clf = opt.best_estimator_ y_pred = best_clf.predict(X_test) test_accuracy = accuracy_score(y_test, y_pred) test_f1 = f1_score(y_test, y_pred, average='binary') print(f"\nOptimization completed!") print(f"Best parameters: {opt.best_params_}") print(f"Test accuracy: {test_accuracy:.4f}") print(f"Test F1: {test_f1:.4f}") .. rst-class:: sphx-glr-script-out .. code-block:: none Optimization completed! Best parameters: {'activation': 'relu', 'alpha': 0.076, 'batch_size': 'auto', 'beta_1': 0.9, 'beta_2': 0.999, 'early_stopping': False, 'epsilon': 1e-08, 'hidden_layer_sizes': [100], 'learning_rate': 'constant', 'learning_rate_init': 0.0036, 'max_fun': 15000, 'max_iter': 500, 'momentum': 0.9, 'n_iter_no_change': 10, 'nesterovs_momentum': True, 'power_t': 0.5, 'random_state': 42, 'shuffle': True, 'solver': 'adam', 'tol': 0.0001, 'validation_fraction': 0.1, 'verbose': False, 'warm_start': False} Test accuracy: 0.9561 Test F1: 0.9645 .. GENERATED FROM PYTHON SOURCE LINES 76-77 Visualize the search space .. GENERATED FROM PYTHON SOURCE LINES 77-89 .. code-block:: default population_df = opt.populations_ top_params = ['alpha', 'learning_rate_init', 'activation', 'fitness'] df_filtered = population_df[top_params] g_search_space = plotly_search_space(df_filtered, top_params) g_search_space.update_layout( title="MLPClassifier Hyperparameter Search Space", autosize=True, width=None, height=650 ) plotly.io.show(g_search_space, config={'responsive': True}) .. raw:: html :file: images/sphx_glr_plot_mlp_neural_network_001.html .. GENERATED FROM PYTHON SOURCE LINES 90-91 Visualize the optimization evolution .. GENERATED FROM PYTHON SOURCE LINES 91-100 .. code-block:: default g_logbook = plotly_logbook(opt.logbook_, population_df) g_logbook.update_layout( title="MLPClassifier Optimization Evolution", autosize=True, width=None, height=500 ) plotly.io.show(g_logbook, config={'responsive': True}) .. raw:: html :file: images/sphx_glr_plot_mlp_neural_network_002.html .. GENERATED FROM PYTHON SOURCE LINES 101-102 Analyze optimization performance .. GENERATED FROM PYTHON SOURCE LINES 102-107 .. code-block:: default print("\n=== Optimization Performance ===") print(f"Unique evaluations performed: {opt.n_trials_}") print(f"Total individuals in population history: {len(opt.populations_)}") print(f"Optimization time: {opt.optimization_time_:.4f} seconds") print(f"Time per evaluation: {opt.optimization_time_ / opt.n_trials_:.4f} seconds") .. rst-class:: sphx-glr-script-out .. code-block:: none === Optimization Performance === Unique evaluations performed: 33 Total individuals in population history: 48 Optimization time: 9.0605 seconds Time per evaluation: 0.2746 seconds .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 10.158 seconds) .. _sphx_glr_download_auto_examples_plot_mlp_neural_network.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_mlp_neural_network.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_mlp_neural_network.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_