Adaptive Operator Portfolio Search with Fold Based Landscape Probing
Downloads
Abstract
Adaptive operator selection allocates search effort among candidate mechanisms whose usefulness changes with the current population. We study AOP-KFP, a portfolio of nine operators with periodic probing of held-out parents, exponentially smoothed improvement credits, and a softmax distribution mixed with uniform exploration. The folds organize local comparisons within a population snapshot; they do not establish statistical generalization to unseen problems. We give an explicit accounting of probe evaluations and a conditional hitting-probability result for elitist search, while distinguishing these statements from finite-budget performance. The supplied CEC 2017 table contains 25 functions at dimension 30 with 51 reported runs per algorithm. Its displayed means give AOP-KFP 17 row minima, jSO three, EBOwithCMA four, and L-SHADE one. Additional supplied rank, significance, ablation and sensitivity summaries require reconciliation with their underlying observations: several corrected-p-value entries are arithmetically inconsistent, and the reported Sobol first-order indices sum to more than one. The study therefore identifies a testable operator-allocation mechanism and descriptive benchmark evidence, with evaluation accounting and raw-output reconciliation required before confirmatory superiority or reproducibility claims can be made.
Keywords
Adaptive operator selection Population search Held-out parent probing Function-evaluation budget Benchmark comparisonReferences (27)
- [1] Fialho, Á., Da Costa, L., Schoenauer, M., & Sebag, M. (2010). Analyzing bandit-based adaptive operator selection mechanisms. Annals of mathematics and artificial intelligence, 60(1), 25–64. https://doi.org/10.1007/s10472-010-9213-y
- [2] Thierens, D. (2005). An adaptive pursuit strategy for allocating operator probabilities. Proceedings of the 7th annual conference on genetic and evolutionary computation (pp. 1539–1546). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/1068009.1068251
- [3] Veček, N., Črepinšek, M., & Mernik, M. (2017). On the influence of the number of algorithms, problems, and independent runs in the comparison of evolutionary algorithms. Applied soft computing, 54, 23–45. https://doi.org/10.1016/j.asoc.2017.01.011
- [4] Vargha, A., & Delaney, H. D. (2000). A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of educational and behavioral statistics, 25(2), 101–132. https://doi.org/10.3102/10769986025002101
- [5] Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The journal of machine learning research, 7, 1–30. https://doi.org/10.5555/1248547.1248548
- [6] Garcia, S., & Herrera, F. (2008). An extension on" statistical comparisons of classifiers over multiple data sets" for all pairwise comparisons. Journal of machine learning research, 9(12), 2677–2694. https://www.jmlr.org/papers/volume9/garcia08a/garcia08a.pdf
- [7] Dokeroglu, T., Canturk, D., & Kucukyilmaz, T. (2024). A survey on pioneering metaheuristic algorithms between 2019 and 2024. https://doi.org/10.48550/arXiv.2501.14769
- [8] Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011). Sequential model-based optimization for general algorithm configuration. Learning and intelligent optimization (pp. 507–523). Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-25566-3_40
- [9] López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016). The irace package: Iterated racing for automatic algorithm configuration. Operations research perspectives, 3, 43–58. https://doi.org/10.1016/j.orp.2016.09.002
- [10] Tanabe, R., & Fukunaga, A. S. (2014). Improving the search performance of shade using linear population size reduction. 2014 IEEE congress on evolutionary computation (CEC) (pp. 1658–1665). IEEE. https://doi.org/10.1109/CEC.2014.6900380
- [11] Biedrzycki, R., Arabas, J., & Warchulski, E. (2022). A version of nl-shade-rsp algorithm with midpoint for cec 2022 single objective bound constrained problems. 2022 IEEE congress on evolutionary computation (CEC) (pp. 1–8). IEEE. https://doi.org/10.1109/CEC55065.2022.9870220
- [12] Hansen, N., Auger, A., Ros, R., Mersmann, O., Tušar, T., & Brockhoff, D. (2021). COCO: a platform for comparing continuous optimizers in a black-box setting. Optimization methods and software, 36(1), 114–144. https://doi.org/10.1080/10556788.2020.1808977
- [13] Wu, G., Mallipeddi, R., & Suganthan, P. N. (2017). Problem definitions and evaluation criteria for the CEC 2017 competition on constrained real-parameter optimization. National university of defense technology, Changsha, hunan, pr China and Kyungpook national university, Daegu, South Korea and Nanyang technological university, Singapore, technical report, 9(10), 2017. https://github.com/P-N-Suganthan/CEC2017-BoundContrained
- [14] Kumar, A., Price, K. V, Mohamed, A. W., Hadi, A. A., & Suganthan, P. N. (2021). Problem definitions and evaluation criteria for the CEC 2022 special session and competition on single objective bound constrained numerical optimization. IEEE Congress On Evolutionary Computation: Tech. Rep, Italy: Centro Congressi Padova. https://github.com/P-N-Suganthan/2022-SO-BO
- [15] Biedrzycki, R. (2024). Revisiting CEC 2022 ranking: A new ranking method and influence of parameter tuning. Swarm and evolutionary computation, 89, 101623. https://doi.org/10.1016/j.swevo.2024.101623
- [16] Biedrzycki, R. (2026). Analysis and simplification of the winner of the CEC 2022 optimization competition on single objective bound constrained search. Evolutionary computation, 34(2), 195–212. https://doi.org/10.1162/evco.a.27
- [17] Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American statistical association, 32(200), 675–701. https://doi.org/10.1080/01621459.1937.10503522
- [18] Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Routledge. https://www.routledge.com/Statistical-Power-Analysis-for-the-Behavioral-Sciences/Cohen/p/book/9780203771587
- [19] Cliff, N. (1993). Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological bulletin, 114(3), 494–509. https://doi.org/10.1037/0033-2909.114.3.494
- [20] Sawilowsky, S. S. (2009). New effect size rules of thumb. Journal of modern applied statistical methods, 8(2), 26. https://doi.org/10.22237/jmasm/1257035100
- [21] Franceschi, L., Donini, M., Perrone, V., Klein, A., Archambeau, C., Seeger, M., … & Frasconi, P. (2025). Hyperparameter optimization in machine learning. Foundations and trends in machine learning, 18(6), 975–1109. https://doi.org/10.1561/2200000088
- [22] Morales-Hernández, A., Van Nieuwenhuyse, I., & Rojas Gonzalez, S. (2023). A survey on multi-objective hyperparameter optimization algorithms for machine learning. Artificial intelligence review, 56(8), 8043–8093. https://doi.org/10.1007/s10462-022-10359-2
- [23] Feurer, M., & Hutter, F. (2019). Hyperparameter optimization. In Automated machine learning: Methods, systems, challenges (pp. 3–33). Springer International Publishing, Cham. https://doi.org/10.1007/978-3-030-05318-5_1
- [24] Bergstra, J., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. Advances in neural information processing systems (pp. 1–9). Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2011/file/86e8f7ab32cfd12577bc2619bc635690-Paper.pdf
- [25] Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics bulletin, 1(6), 80–83. https://doi.org/10.2307/3001968
- [26] Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, 65–70. https://doi.org/10.2307/4615733
- [27] Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society: Series b (methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
