Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
Sielskyi Yevhenii
sho
Commits
31c9fa5a
Commit
31c9fa5a
authored
Jan 06, 2020
by
Johann Dreo
Browse files
fix latex in LESSON
parent
045d4b9a
Changes
1
Show whitespace changes
Inline
Side-by-side
LESSON.md
View file @
31c9fa5a
Metaheuristics (IA-308)
=======================
Compile as PDF:
`pandoc -f markdown --toc -o LESSON.pdf LESSON.md`
.
Introduction
------------
Metaheuristics are mathematical optimization algorithms solving
`$\
argmin_{x \in X} f(x)$
`
(or argmax).
Metaheuristics are mathematical optimization algorithms solving
$
argmin_{x
\i
n X} f(x)$ (or argmax).
Synonyms:
-
search heuristics,
-
evolutionary algorithms,
-
stochastic local search.
...
...
@@ -53,11 +56,13 @@ Way to model a solution: encoding.
### Main models
Encoding:
-
continuous (s. numeric),
-
discrete metric (integers),
-
combinatorial (graph, permutation).
Fitness:
-
mono-objective,
-
multi-modal,
-
multi-objectives.
...
...
@@ -77,11 +82,13 @@ Performance evaluation
### What is performance
Main performances axis:
-
time,
-
quality,
-
probability.
Additional performance axis:
-
robustness,
-
stability.
...
...
@@ -95,6 +102,7 @@ Proof-reality gap is huge, thus empirical performance evaluation is gold standar
Empirical evaluation = scientific method.
Basic rules of thumb:
-
randomized algorithms => repetition of runs,
-
sensitivity to parameters => design of experiments,
-
use statistical tools,
...
...
@@ -103,11 +111,13 @@ Basic rules of thumb:
### Useful statistical tools
Statistical tests.
Statistical tests:
-
classical null hypothesis: test equality of distributions.
-
beware of p-value.
How many runs?
-
not always "as many as possible",
-
maybe "as many as needed",
-
generally: 15 (min for non-parametric tests) -- 20 (min for parametric-gaussian tests).
...
...
@@ -118,12 +128,14 @@ Use robust estimators: median instead of mean, Inter Quartile Range instead of s
### Expected Empirical Cumulative Distribution Functions
On Run Time: ERT-ECDF.
```
$ERTECDF(\{X_0,\dots,X_i,\dots,X_r\}, \delta, f, t) := \#\{x_t \in X_t | f(x_t^*)>=\delta \}$
$\delta \in [0, max_{x \in \mathcal{X}}(f(x))]$
$X_i := \{\{ x_0^0, \dots, x_i^j, \dots, x_p^u | p\in[1,\infty[ \} | u \in [0,\infty[ \} \in \mathcal{X}$
```
with $p$ the sample size, $r$ the number of runs, $u$ the nubmer of iterations, $t$ the number of calls to the objective
$$ERTECDF(
\{
X_0,
\d
ots,X_i,
\d
ots,X_r
\}
,
\d
elta, f, t) :=
\#\{
x_t
\i
n X_t | f(x_t^
*
)>=
\d
elta
\}
$$
$$
\d
elta
\i
n
\l
eft[0,
\m
ax_{x
\i
n
\m
athcal{X}}(f(x))
\r
ight]$$
$$X_i :=
\l
eft
\{\l
eft
\{
x_0^0,
\d
ots, x_i^j,
\d
ots, x_p^u | p
\i
n[1,
\i
nfty[
\r
ight
\}
| u
\i
n [0,
\i
nfty[
\r
ight
\}
\i
n
\m
athcal{X}$$
with $p$ the sample size, $r$ the number of runs, $u$ the number of iterations, $t$ the number of calls to the objective
function.
The number of calls to the objective function is a good estimator of time because it dominates all other times.
...
...
@@ -136,6 +148,7 @@ The dual of the ERT-ECDF can be easily computed for quality (EQT-ECDF).
### Other tools
Convergence curves: do not forget the golden rule and show distributions:
-
quantile boxes,
-
violin plots,
-
histograms.
...
...
@@ -146,11 +159,13 @@ Algorithm Design
### Neighborhood
Convergence definition(s).
Convergence definition(s):
-
strong,
-
weak.
Neighborhood: subset of solutions atteinable after an atomic transformation:
-
ergodicity,
-
quasi-ergodicity.
...
...
@@ -158,17 +173,20 @@ Neighborhood: subset of solutions atteinable after an atomic transformation:
### Structure of problem/algorithms
Structure of problems to exploit:
-
locality (basin of attraction),
-
separability,
-
gradient,
-
funnels.
Structure with which to capture those structures:
-
implicit,
-
explicit,
-
direct.
Silver rule: choose the algorithmic template that adhere the most to the problem model.
-
taking constraints into account,
-
iterate between problem/algorithm models.
...
...
@@ -188,12 +206,19 @@ Most generic way of thinking about algorithms: grammar-based algorithm selection
Example: modular CMA-ES.
Parameter setting tools:
-
ParamILS,
-
SPO,
-
i-race.
Design tools:
-
ParadisEO.
-
ParadisEO,
-
jMetal,
-
Jenetics,
-
ECJ,
-
DEAP,
-
HeuristicLab.
### Landscape-aware algorithms
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment