ExPSO-DL: An Exponential Particle Swarm Optimization Package for Deep Learning Model Optimization

FEATURE	EXPSO	XPSO	PPSO	FMPSO	TAPSO
Exploration-Exploitation Balance	Dynamic via exponential control	Fixed coefficient decay	Phasor-based control	Fuzzy logic-based mutation	Time-adaptive parameter tuning
Velocity Control	Adaptive exponential velocity bounds	❌	Scalar phasor-based updates	Mutation-induced adjustments	Time-based inertia weight updates
Leaping Strategy	Exponential leaping function	❌	❌	❌	❌
Personal/Global Best/Worst Usage	Full use of pbest, gbest, pworst	with forgetting mechanism	✓	✓	✓
Adaptivity	High (dynamic γ and w updates)	Medium (decay memory)	Medium (phasor response)	Medium (fuzzy decisions)	High (time-adaptive parameters)
DL/ML Framework Integration	Supports PyTorch/TensorFlow APIs	❌	❌	❌	❌
Parallel Evaluation Support	Parallel training support	❌	❌	❌	❌
Code Extensibility	Modular Python classes	❌	❌	❌	❌

Table 2

Unit Tests for ExPSO Library Functions.

TEST NAME	PURPOSE	STATUS
test_initialize_particles()	Verifies correct generation of initial positions and velocities	Passed
test_velocity_update()	Ensures accurate computation of velocity updates per ExPSO equations	Passed
test_position_update()	Checks boundary handling and correct position updates	Passed
test_fitness_evaluation()	Confirms objective function evaluation returns valid and expected outputs	Passed
test_global_best_selection()	Validates that global best particle is correctly identified and tracked	Passed
test_integration()	Tests compatibility with PyTorch models and parameter space setup	Passed
test_termination_criteria()	Checks early stopping and max iteration limits work as intended	Passed
test_invalid_input_handling()	Ensures graceful handling of invalid configuration or input parameters	Passed
test_initialize_particles()	Verifies correct generation of initial positions and velocities	Passed

Illustrative example of ExPSO with Rosenbrock function.

Output result of ExPSO after optimizing for the Rosenbrock function.

Illustrative example of ExPSO with MLP model.

An illustrative example of train and evaluating the MLP model with ExPSO parameters.

Table 3

Comparison of optimization algorithms on unimodal benchmark functions.

METHOD	RESULT	F₁	F₂	F₃	F₄	F₅	F₆
ExPSO	Avg.	0.00e+00	0.00e+00	0.00e+00	0.00e+00	0.00e+00	0.00e+00
ExPSO	S.D.	0.00e+00	0.00e+00	0.00e+00	0.00e+00	3.17e+09	0.00e+00
XPSO	Avg.	2.64e–05	0.00e+00	8.33e+00	1.81e–01	9.25e+00	0.00e+00
XPSO	S.D.	1.24e–04	5.65e–10	3.31e–04	2.05e–01	9.16e+00	0.00e+00
PPSO	Avg.	0.00e+00	0.00e+00	0.00e+00	7.99e–05	0.00e+00	0.00e+00
PPSO	S.D.	5.54e–10	1.70e–09	1.25e–10	1.52e–04	1.18e–09	0.00e+00
FMPSO	Avg.	0.00e+00	0.00e+00	0.00e+00	0.00e+00	7.97e–01	0.00e+00
FMPSO	S.D.	2.43e–09	6.30e–10	4.55e–10	1.50e–10	1.62e+00	0.00e+00

Table 4

Comparison of optimization algorithms on multimodal benchmark functions.

METHOD	RESULT	F₇	F₈	F₉	F₁₀	F₁₁	F₁₂
ExPSO	Avg.	–1.25e+04	0.00e+00	0.00e+00	0.00e+00	0.00e+00	0.00e+00
ExPSO	S.D.	1.91e–01	0.00e+00	0.00e+00	0.00e+00	2.73e–09	3.21e–09
XPSO	Avg.	–1.08e+04	3.98e+00	0.00e+00	3.53e–02	3.44e–08	0.00e+00
XPSO	S.D.	3.19e+02	6.12e+02	6.12e–10	3.41e–02	1.36e–07	7.51e–10
PPSO	Avg.	–1.19e+04	0.00e+00	0.00e+00	0.00e+00	0.00e+00	0.00e+00
PPSO	S.D.	7.26e+02	1.96e–10	2.80e–10	2.24e–10	3.07e–09	1.27e–10
FMPSO	Avg.	–1.10e+04	2.28e+01	0.00e+00	5.34e–02	1.38e–02	2.20e–03
FMPSO	S.D.	4.22e+02	1.64e+01	3.66e–10	4.58e–02	5.92e–02	4.47e–03

Table 5

Comparison of optimum results and statistical results for the PVD problem. Values represent the objective function f(x), with Best, Worst, Mean, and Standard Deviation (S.D.) across 30 independent runs. Bold values indicate the best performance for each metric.

F	EXPSO	PPSO	FMPSO	MSPSO	XPSO	TAPSO	EO	DSOS	HHO	IRGA	EBCMAR	IMODE
f(x)	6059	6216	6771	6090	44232	6424	6059	6059	6064	6118	6123	6156
Best	6059	6216	6771	6090	44232	6424	6059	6059	6064	6118	6123	6156
Worst	7332	13846	27782	7461	99145	6424	7544	6820	7544	7544	7473	20218
Mean	6197	6473	17380	6547	70587	6424	6641	6095	6684	6863	6979	9355
S.D.	350	1852	10507	791	27522	0.000	566	148	425	372	399	3079

Table 6

Comparison of optimum results and statistical results for the CSD problem.

F	EXPSO	PPSO	FMPSO	MSPSO	XPSO	TAPSO	EO	DSOS	HHO	IRGA	EBCMAR	IMODE
f(x)	0.012665	0.012669	0.012719	0.012719	0.97149	0.012793	0.012250	0.012666	0.012665	0.012719	0.012666	0.012665
Best	0.012665	0.012669	0.012719	0.012719	0.97149	0.012793	0.012250	0.012666	0.012665	0.012719	0.012666	0.012665
Worst	0.015234	0.017773	5.4901	0.20984	2.0172	0.012793	0.013318	0.012686	0.017275	0.016659	0.012971	0.013004
Mean	0.012857	0.013277	2.3035	0.094071	1.3531	0.012793	0.012866	0.012673	0.136438	0.013684	0.012718	0.012692
S.D.	0.000523	0.001271	2.8465	0.10297	0.5773	0.000000	0.000173	0.000004	0.001001	0.000913	0.000006	0.000007

Table 7

Comparison of optimum results and statistical results for the WBD problem.

F	EXPSO	PPSO	FMPSO	MSPSO	XPSO	TAPSO	EO	DSOS	HHO	IRGA	EBCMAR	IMODE
f(x)	1.690	1.6956	1.69746	1.70960	1.891	1.6954	1.6952	1.6952	1.7119	1.7212	1.6952	1.6953
Best	1.695	1.695	1.697	1.709	1.861	1.695	1.695	1.695	1.711	1.721	1.695	1.695
Worst	1.841	1.793	2.004	1.955	24.15	1.695	1.697	1.695	2.242	3.235	1.695	2.123
Mean	1.707	1.743	1.798	1.816	9.458	1.695	1.695	1.695	1.816	2.201	1.695	1.830
S.D.	0.029	0.040	0.178	0.126	6.68	0.000	0.000	0.00	0.107	0.364	0.000	0.117

Table 8

Comparison of optimal results and statistical results for the SRD problem.

F	EXPSO	PPSO	FMPSO	MSPSO	XPSO	TAPSO	EO	DSOS	HHO	IRGA	EBOWITH CMAR	IMODE
f(x)	2994	2994	3048	2994	3060	2994	2994	2994	2997	2994	2639	2994
Best	2994	2994	3048	2994	3060	2994	2994	2994	2997	2994	2639	2994
Worst	3033	3.055	3285	3058	5383	2994	2994	2994	3803	2994	2992	2994
Mean	2999	3019	3156	3020	4153	2994	2994	2994	3114	2994	2848	2994
S.D.	9.18	3.246	119.73	33.59	676.1	0.00	0.00	0.00	180.9	0.00	59.768	0.00

Accuracy Comparison of ExPSO Library with FST-PSO, Pyswarms, QPSO, and FastPSO Libraries for Deep and Machine Learning Models.

References

Authors

Metrics

Articles in this issue

DOI: https://doi.org/10.5334/jors.521 | Journal eISSN: 2049-9647

Journal RSS Feed

Language: English

Submitted on: Jun 18, 2024

Accepted on: Sep 27, 2025

Published on: Nov 11, 2025

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Publication frequency: 1 issue per year

Keywords:

Particle Swarm Optimization,

Python,

Machine Learning,

Deep Learning,

Optimization Techniques,

Heuristic Algorithms

© 2025 Insaf Kraidia, Khelil Kassoul, Naoufel Cheikhrouhou, Saima Hassan, Samir Brahim Belhaouari, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 13 (2025): Issue 1

ExPSO-DL: An Exponential Particle Swarm Optimization Package for Deep Learning Model Optimization

Figures & Tables

Table 1

Figure 1

Algorithm 1

Algorithm 2

Algorithm 3

Table 2

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Table 3

Table 4

Table 5

Table 6

Table 7

Table 8

Figure 7

Paradigm

My account