Commit 31c9fa5a authored by Johann Dreo's avatar Johann Dreo
Browse files

fix latex in LESSON

parent 045d4b9a
Metaheuristics (IA-308) Metaheuristics (IA-308)
======================= =======================
Compile as PDF: `pandoc -f markdown --toc -o LESSON.pdf LESSON.md`.
Introduction Introduction
------------ ------------
Metaheuristics are mathematical optimization algorithms solving `$\argmin_{x \in X} f(x)$` (or argmax). Metaheuristics are mathematical optimization algorithms solving $argmin_{x \in X} f(x)$ (or argmax).
Synonyms: Synonyms:
- search heuristics, - search heuristics,
- evolutionary algorithms, - evolutionary algorithms,
- stochastic local search. - stochastic local search.
...@@ -53,11 +56,13 @@ Way to model a solution: encoding. ...@@ -53,11 +56,13 @@ Way to model a solution: encoding.
### Main models ### Main models
Encoding: Encoding:
- continuous (s. numeric), - continuous (s. numeric),
- discrete metric (integers), - discrete metric (integers),
- combinatorial (graph, permutation). - combinatorial (graph, permutation).
Fitness: Fitness:
- mono-objective, - mono-objective,
- multi-modal, - multi-modal,
- multi-objectives. - multi-objectives.
...@@ -77,11 +82,13 @@ Performance evaluation ...@@ -77,11 +82,13 @@ Performance evaluation
### What is performance ### What is performance
Main performances axis: Main performances axis:
- time, - time,
- quality, - quality,
- probability. - probability.
Additional performance axis: Additional performance axis:
- robustness, - robustness,
- stability. - stability.
...@@ -95,6 +102,7 @@ Proof-reality gap is huge, thus empirical performance evaluation is gold standar ...@@ -95,6 +102,7 @@ Proof-reality gap is huge, thus empirical performance evaluation is gold standar
Empirical evaluation = scientific method. Empirical evaluation = scientific method.
Basic rules of thumb: Basic rules of thumb:
- randomized algorithms => repetition of runs, - randomized algorithms => repetition of runs,
- sensitivity to parameters => design of experiments, - sensitivity to parameters => design of experiments,
- use statistical tools, - use statistical tools,
...@@ -103,11 +111,13 @@ Basic rules of thumb: ...@@ -103,11 +111,13 @@ Basic rules of thumb:
### Useful statistical tools ### Useful statistical tools
Statistical tests. Statistical tests:
- classical null hypothesis: test equality of distributions. - classical null hypothesis: test equality of distributions.
- beware of p-value. - beware of p-value.
How many runs? How many runs?
- not always "as many as possible", - not always "as many as possible",
- maybe "as many as needed", - maybe "as many as needed",
- generally: 15 (min for non-parametric tests) -- 20 (min for parametric-gaussian tests). - generally: 15 (min for non-parametric tests) -- 20 (min for parametric-gaussian tests).
...@@ -118,12 +128,14 @@ Use robust estimators: median instead of mean, Inter Quartile Range instead of s ...@@ -118,12 +128,14 @@ Use robust estimators: median instead of mean, Inter Quartile Range instead of s
### Expected Empirical Cumulative Distribution Functions ### Expected Empirical Cumulative Distribution Functions
On Run Time: ERT-ECDF. On Run Time: ERT-ECDF.
```
$ERTECDF(\{X_0,\dots,X_i,\dots,X_r\}, \delta, f, t) := \#\{x_t \in X_t | f(x_t^*)>=\delta \}$ $$ERTECDF(\{X_0,\dots,X_i,\dots,X_r\}, \delta, f, t) := \#\{x_t \in X_t | f(x_t^*)>=\delta \}$$
$\delta \in [0, max_{x \in \mathcal{X}}(f(x))]$
$X_i := \{\{ x_0^0, \dots, x_i^j, \dots, x_p^u | p\in[1,\infty[ \} | u \in [0,\infty[ \} \in \mathcal{X}$ $$\delta \in \left[0, \max_{x \in \mathcal{X}}(f(x))\right]$$
```
with $p$ the sample size, $r$ the number of runs, $u$ the nubmer of iterations, $t$ the number of calls to the objective $$X_i := \left\{\left\{ x_0^0, \dots, x_i^j, \dots, x_p^u | p\in[1,\infty[ \right\} | u \in [0,\infty[ \right\} \in \mathcal{X}$$
with $p$ the sample size, $r$ the number of runs, $u$ the number of iterations, $t$ the number of calls to the objective
function. function.
The number of calls to the objective function is a good estimator of time because it dominates all other times. The number of calls to the objective function is a good estimator of time because it dominates all other times.
...@@ -136,6 +148,7 @@ The dual of the ERT-ECDF can be easily computed for quality (EQT-ECDF). ...@@ -136,6 +148,7 @@ The dual of the ERT-ECDF can be easily computed for quality (EQT-ECDF).
### Other tools ### Other tools
Convergence curves: do not forget the golden rule and show distributions: Convergence curves: do not forget the golden rule and show distributions:
- quantile boxes, - quantile boxes,
- violin plots, - violin plots,
- histograms. - histograms.
...@@ -146,11 +159,13 @@ Algorithm Design ...@@ -146,11 +159,13 @@ Algorithm Design
### Neighborhood ### Neighborhood
Convergence definition(s). Convergence definition(s):
- strong, - strong,
- weak. - weak.
Neighborhood: subset of solutions atteinable after an atomic transformation: Neighborhood: subset of solutions atteinable after an atomic transformation:
- ergodicity, - ergodicity,
- quasi-ergodicity. - quasi-ergodicity.
...@@ -158,17 +173,20 @@ Neighborhood: subset of solutions atteinable after an atomic transformation: ...@@ -158,17 +173,20 @@ Neighborhood: subset of solutions atteinable after an atomic transformation:
### Structure of problem/algorithms ### Structure of problem/algorithms
Structure of problems to exploit: Structure of problems to exploit:
- locality (basin of attraction), - locality (basin of attraction),
- separability, - separability,
- gradient, - gradient,
- funnels. - funnels.
Structure with which to capture those structures: Structure with which to capture those structures:
- implicit, - implicit,
- explicit, - explicit,
- direct. - direct.
Silver rule: choose the algorithmic template that adhere the most to the problem model. Silver rule: choose the algorithmic template that adhere the most to the problem model.
- taking constraints into account, - taking constraints into account,
- iterate between problem/algorithm models. - iterate between problem/algorithm models.
...@@ -188,12 +206,19 @@ Most generic way of thinking about algorithms: grammar-based algorithm selection ...@@ -188,12 +206,19 @@ Most generic way of thinking about algorithms: grammar-based algorithm selection
Example: modular CMA-ES. Example: modular CMA-ES.
Parameter setting tools: Parameter setting tools:
- ParamILS, - ParamILS,
- SPO, - SPO,
- i-race. - i-race.
Design tools: Design tools:
- ParadisEO.
- ParadisEO,
- jMetal,
- Jenetics,
- ECJ,
- DEAP,
- HeuristicLab.
### Landscape-aware algorithms ### Landscape-aware algorithms
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment