Perprof-py: A Python Package for Performance Profile of Mathematical Optimization Software

Abel Soares Siqueira; Raniere Gaia Costa da Silva; Luiz-Rafael Santos

doi:10.5334/jors.81

Perprof-py: A Python Package for Performance Profile of Mathematical Optimization Software

Journal of Open Research Software

Volume 4 (2016): Issue 1

By: Abel Soares Siqueira, Raniere Gaia Costa da Silva and Luiz-Rafael Santos

Open Access

|Apr 2016

Abstract

A very important area of research in the field of Mathematical Optimization is the benchmarking of optimization packages to compare solvers. During benchmarking, one usually collects a large amount of information like CPU time, number of functions evaluations, number of iterations, and much more. This information, if presented as tables, can be difficult to analyze and compare due to large amount of data. Therefore tools to better process and understand optimization benchmark data have been developed. One of the most widespread tools is the Performance Profile graphics proposed by Dolan and Moré [2]. In this context, this paper describes perprof-py, a free/open source software that creates Performance Profile graphics. This software produces graphics in PDF using LaTeX with PGF/TikZ [22] and PGFPLOTS [4] packages, in PNG using matplotlib [9], and in HTML using Bokeh [1]. Perprof-py can also be easily extended to be used with other plot libraries. It is implemented in Python 3 with support for internationalization, and is under the General Public License Version 3 (GPLv3).

References

Bokeh Available at http://bokeh.pydata.org/ (visited on 05/30/2015)
Search in Google Scholar Back to article
Dolan E D Moré J J ‘Benchmarking optimization software with performance profiles’ Mathematical Programming 2002 91 2 201 213 10.1007/s101070100263
Open DOI Search in Google Scholar Back to article
Dolan E D Moré J J Munson T S ‘Optimality Measures for Performance Profiles’ SIAM Journal on Optimization 2006 Jan 16 3 891 909 10.1137/040608015
Open DOI Search in Google Scholar Back to article
Feuersänger C Manual for Package PGFPLOTS 2013 Version 1.9
Search in Google Scholar Back to article
Higham N J ‘The scaling and squaring method for the matrix exponential revisited’ SIAM Journal on Matrix Analysis and Applications 2005 26 4 1179 1193 10.1137/04061101X
Open DOI Search in Google Scholar Back to article
Higham N J ‘The scaling and squaring method for the matrix exponential revisited’ SIAM Review 2009 51 4 747 764 10.1137/090768539
Open DOI Search in Google Scholar Back to article
Higham N J Lin L ‘A Schur-Padé algorithm for fractional powers of a matrix’ SIAM Journal on Matrix Analysis and Applications 2011 32 3 1056 1078 10.1137/10081232X
Open DOI Search in Google Scholar Back to article
Higham N J Lin L ‘An improved Schur-Padé algorithm for fractional powers of a matrix and their Frechet derivatives’ SIAM Journal on Matrix Analysis and Applications 2013 34 3 1341 1360 10.1137/130906118
Open DOI Search in Google Scholar Back to article
Hunter J D ‘Matplotlib: A 2D graphics environment’ Computing In Science & Engineering 2007 9 3 90 95 10.1109/MCSE.2007.55
Open DOI Search in Google Scholar Back to article
Mittelmann H D ‘Benchmarking interior point LP/QP solvers’ Optimization Methods and Software 1999 11 1–4 655 670 10.1080/10556789908805767
Open DOI Search in Google Scholar Back to article
Mittelmann H Benchmark for optimization software url: http://plato.la.asu.edu/bench.html (visited on 08/21/2015)
Search in Google Scholar Back to article
Al-Mohy A H Higham N J ‘A new scaling and squaring algorithm for the matrix exponential’ SIAM Journal on Matrix Analysis and Applications 2009 31 3 970 989 10.1137/09074721X
Open DOI Search in Google Scholar Back to article
Al-Mohy A H Higham N J ‘Computing the action of the matrix exponential, with an application to exponential integrators’ SIAM Journal on Scientifc Computing 2011 33 2 488 511 10.1137/100788860
Open DOI Search in Google Scholar Back to article
Al-Mohy A H Higham N J ‘Improved inverse scaling and squaring algorithms for the matrix logarithm’ SIAM Journal on Scientifc Computing 2012 34 4 C153 C169 10.1137/110852553
Open DOI Search in Google Scholar Back to article
Moré J J Wild S M ‘Benchmarking Derivate-Free Optimization Algorithms’ SIAM Journal on Optimization 2009 20 172 191 10.1137/080724083
Open DOI Search in Google Scholar Back to article
Moré J J Wild S M ‘Benchmarking Derivative-Free Optimization Algorithms’ SIAM Journal on Optimization 2009 20 1 172 191 10.1137/080724083.
Open DOI Search in Google Scholar Back to article
Moré J Bondarenko A Bortz D Dolan E Merritt M Munson T COPS: Large-Scale Optimization Problems Available at http://www.mcs.anl.gov/~more/cops/ (visited on 08/21/2015)
Search in Google Scholar Back to article
Orban D Friedlander M NLPy Available at https://github.com/dpo/nlpy (visited on 05/30/2015)
Search in Google Scholar Back to article
PyYAML Available at http://pyyaml.org/ (visited on 08/25/2015)
Search in Google Scholar Back to article
Relton S perfprof Available at https://github.com/sdrelton/perfprof (visited on 08/26/2015)
Search in Google Scholar Back to article
Siqueira A S da Silva R G C Santos L-R Perprof-py Available at https://github.com/ufpr-opt/perprof-py/ (visited on 05/30/2015)
Search in Google Scholar Back to article
Tantau T The TikZ and PGF Packages 2012 Manual for version 2.10-cvs
Search in Google Scholar Back to article
Travis CI Available at http://travis-ci.org (visited on 05/30/2015)
Search in Google Scholar Back to article
YAML Ain’t Markup Language Available at http://yaml.org/ (visited on 08/25/2015)
Search in Google Scholar Back to article
Zhang W PerfPlot.jl Available at https://github.com/weijianzhang/PerfPlot.jl (visited on 08/26/2015)
Search in Google Scholar Back to article

Articles in this issue

DOI: https://doi.org/10.5334/jors.81 | Journal eISSN: 2049-9647

Journal RSS Feed

Language: English

Published on: Apr 22, 2016

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Publication frequency: 1 issue per year

Keywords:

software benchmarking,

mathematical optimization,

performance profile,

Python 3

© 2016 Abel Soares Siqueira, Raniere Gaia Costa da Silva, Luiz-Rafael Santos, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 4 (2016): Issue 1