Opfunu: An Open-source Python Library for Optimization Benchmark Functions

Introduction

In the realm of numerical optimization, benchmark functions play a pivotal role in evaluating and comparing the performance of optimization algorithms [1]. These functions serve as standardized test cases, allowing researchers and practitioners to gauge the efficacy and robustness of different optimization techniques across a variety of problem domains. By subjecting algorithms to a diverse set of benchmark functions, researchers can assess their ability to handle various challenges such as multimodality, nonlinearity, and high dimensionality, thereby gaining insights into strengths and limitations of their algorithms. One notable initiative in the realm of benchmark functions is the Congress on Evolutionary Computation (CEC) competition [2]. The CEC competition, held annually since 2005, has been instrumental in driving advancements in optimization research by providing a platform for the evaluation and comparison of optimization algorithms on standardized benchmark functions. These benchmark functions, meticulously curated by the competition organizers, represent real-world optimization problems and pose significant challenges to algorithm developers. However, despite the importance of benchmark functions in driving innovation and progress in optimization research, there has been a notable gap in readily accessible and diverse benchmark function libraries. This gap underscores the critical need for a comprehensive suite of benchmark functions readily available to researchers and practitioners alike.

To address this gap, we introduce opfunu, an open-source Python library meticulously designed to provide a vast array of benchmark functions for numerical optimization tasks. Opfunu offers a rich collection of benchmark functions, including those featured in the CEC competition and over 200 traditional functions characterized by varying complexities and dimensions. Opfunu is built on top of Numpy [3] library, a powerful numerical computing library for Python, ensuring ease of use, computational efficiency, and seamless integration with existing Python-based optimization frameworks. Its modular structure enhances flexibility, allowing users to easily incorporate custom benchmark functions or extend the library’s functionality as needed.

Opfunu has been utilized in various research studies across the optimization community, facilitating rigorous experimentation, algorithmic development, and comparative analysis. For instance, [4] utilized CEC2014 benchmark functions to evaluate and compare the Hybridization of Galactic Swarm and Evolution Whale Optimization model. In study [5] proposed the nQSV-net model for workload modeling and demonstrated the effective optimization capabilities of the Improved Queuing search algorithm on the CEC2014 benchmark function set. Additionally [6], selected 20 benchmark functions from CEC2014 and CEC2015 to assess the Improved Sea Lion Optimization algorithm against other techniques. Moreover [7], proposed a specialized library for metaheuristic algorithms and utilized functions from CEC2017 to evaluate the performance of these algorithms. Notably, recent metaheuristic algorithms, such as the Q-learning based Vegetation Evolution algorithm [8], have incorporated Opfunu library functions for testing on the CEC2020 function set, as well as engineering problems and WSN coverage optimization problems. Similarly [9], proposed the strengthened RIME (rime-ice) algorithm and utilized functions from the Opfunu library’s CEC2020 set. Furthermore [10], proposed the evolutionary multi-mode slime mold optimization algorithm for addressing continuous optimization problems and validated its performance using the CEC2013 function set from Opfunu. These studies collectively highlight the utility and versatility of Opfunu in advancing optimization research and fostering algorithmic innovation.

In comparison with existing software packages, modules, and libraries offering similar functionalities, Opfunu stands out due to its comprehensive collection of benchmark functions. Additionally, Opfunu provides extensive documentation, user guides, examples, test cases, and notably, it is easy to install and use. For example [11], is a package that only includes the CEC2017 benchmark functions, is not packaged as a library, and lacks documentation, making it challenging for users to understand and utilize effectively. Similarly, [12], [13], and [14] are three modules containing sets of benchmark functions for multi-dimensional problems, but they are not packaged as libraries and lack user guides and examples. [15] includes unimodal and multimodal benchmark functions but lacks a documentation website and is not packaged as a library. [16] provides basic benchmark functions in the Julia programming language but lacks examples and a documentation website. Moreover [17], offers a library providing standard functions for single and multi-objective optimization problems. However, despite having documentation and examples, it is written in the R programming language and only offers standard functions without the inclusion of CEC function sets. Opfunu, on the other hand, not only provides a comprehensive range of benchmark functions but also offers clear documentation, ease of installation, and usage, making it a standout choice for researchers and practitioners in the field of optimization.

Implementation and architecture

Opfunu is implemented in Python, leveraging the powerful numerical computing capabilities of the NumPy library. The software follows a modular architecture, designed to provide flexibility, extensibility, and ease of maintenance. Figure 1 provides a visual representation of Opfunu’s modular architecture and the relationships between its components. Within this architecture, several key components are outlined, including the `benchmark` module and three primary packages: “name_based,” “cec_based,” and “utils.”

Figure 1

• benchmark module: Contains a class named `Benchmark` that all functions in the library must inherit from. The aim is to utilize object-oriented programming inheritance to reuse code and optimize the library structure.

• name_based package: Contains all traditional benchmark functions (excluding CEC functions). It is divided into 26 modules corresponding to the alphabet. For instance, functions with names starting with the letter “a” are placed in the “a_func” module, such as “Ackley01”, “Adjiman”, “Alpine01”, etc. Similarly, all traditional functions are distinguished based on their names.

• cec_based package: Contains all CEC functions spanning from 2005 to 2022. It includes a `cec` module that contains the `CecBenchmark` class. This class inherits from the `Benchmark` class and serves as the parent class for all CEC benchmark functions. The rationale behind this class is to accommodate the unique characteristics of CEC benchmark functions, such as external data loading requirements and the need for bias value settings. The package is further divided into submodules based on the year CEC was organized, such as cec2014, cec2015, etc. The classes defining CEC functions are separated by the year of the CEC competition.

• utils package: Contains helper functions used throughout the library components to promote code reuse. It has two main modules: operator and visualize. The operator module contains functions for performing common mathematical operations and frequently reused functions, while the visualize module provides functions for visualizing benchmark functions in 2D or 3D.

Code template

Users can access more indepth examples either in the Github repository or in the documentation. In this section, we provide code templates for integrating benchmark functions into some popular libraries such as ScikitOpt [18], Mealpy [19], and Opytimizer [20].

In the example below, we use the Differential Evolution (DE) algorithm in the ScikitOpt library to solve the F10 function in the CEC 2015 competition. As can be seen, the first 2 lines define the F10 function from the `opfunu` library, while the remaining lines define the DE algorithm from the ScikitOpt library. Metaheuristic algorithms typically require a fitness (objective) function and need to know the lower bounds and upper bounds for the decision variables of the problem. Therefore, the function classes in Opfunu provide these attributes, and users only need to call them for use. For instance: f10.evaluate is the fitness function, f10.lb and f10.ub are the lower bounds and upper bounds for the decision variables.

Similarly to the example above, the following example utilizes the Genetic Algorithm (GA) to solve the F3 problem in the CEC 2017 competition. As can be seen, the first 2 lines still define the F3 problem from the opfunu library, while the remaining lines define the GA algorithm from the Mealpy library. Here, f3.evaluate represents the fitness function, and f3.lb and f3.ub denote the lower bounds and upper bounds of the decision variables.

Similarly to the examples above, in the following example, we utilize the Artificial Bee Colony (ABC) algorithm from the Opytimizer library to solve the F5 problem in the CEC 2022 competition. As can be observed, the first 2 lines define the F5 function from the opfunu library. The remaining lines define the ABC algorithm from the Opytimizer library. Here, f5.ndim indicates the number of decision variables, f5.evaluate represents the fitness value, and f5.lb and f5.ub denote the lower and upper bounds of the problem.

Through the examples above, it is evident that integrating the opfunu library into other optimization algorithm libraries is straightforward.

Quality control

Opfunu undergoes rigorous testing to ensure its reliability, functionality, and performance across various scenarios. We utilize pytest library for both functional and component testing to verify the integrity of the software. These tests involve running the library with various benchmark functions and comparing the obtained results against known optimal solutions or established benchmarks.

Additionally, Opfunu is tested in multiple environments to ensure compatibility and reliability across different platforms. These environments include:

• Local development environments, such as Windows, macOS, and Linux, using popular Python distributions like pip and Pypi.

• Continuous integration (CI) pipelines, where automated tests are run on cloud-based platforms like GitHub Actions to validate code changes and ensure consistency across different code branches. Opfunu’s testing primarily occurs in different Python environments like Python 3.7, 3.8, 3.9, 3.10, and 3.11.

To assist users in quickly assessing the software’s correctness, Opfunu provides the following resources:

• Documentation: Opfunu’s documentation includes detailed instructions on installing the software, using its features, and interpreting the output.

• Example Usage: Opfunu includes example scripts demonstrating how to use its functionalities with sample input data. These examples showcase how to import the library, select functions, and execute them with sample data.

• Test Script: A well-structured test script is included within the codebase. This script demonstrates how to run functional tests, allowing users to verify the library’s integrity on their machines.

• Sample Code Snippets: The GitHub repository may also include sample code snippets that users can directly employ to test Opfunu’s functionalities with various benchmark functions.

These testing and support mechanisms ensure Opfunu’s reliability, functionality, and ease of use for researchers and practitioners in the optimization community.

Operating system

This package can be run on any operating system where python can be run (GNU/Linux, Mac OSX, Windows).

Programming language

python 3.7+

Additional system requirements

None

Dependencies

numpy >= 1.16.5, matplotlib >= 3.3.0, Pillow >= 9.1.0, requests >= 2.27.0

List of contributors

Nguyen Van Thieu

Software location

Archive

• Name: Zenodo

• Persistent identifier: https://zenodo.org/doi/10.5281/zenodo.3620960

• Licence: Creative Commons Attribution 4.0 International

• Publisher: Nguyen Van Thieu

• Version published: 1.0.2

• Date published: 22/03/24

Code repository

• Name: Github

• Identifier: https://github.com/thieu1995/opfunu

• Licence: GNU General Public License (GPL) V3 license

• Date published: 06/12/2019

Language

English