Fitness Functions

fitness(f::AbstractFitness, c::AbstractCandidate)

Compute the fitness metric f for candidate c.

AccuracyFitness <: AbstractFitness
AccuracyFitness(dataset)

Measure fitness as the accuracy on a dataset.

TrainThenFitness{I,L,O,F} <: AbstractFitness
TrainThenFitness(;dataiter, defaultloss, defaultopt, fitstrat, invalidfitness=0.0)

Measure fitness using fitstrat after training the model using dataiter.

Loss function and optimiser may be provided by the candidate if lossfun(c; defaultloss) and opt(c; defaultopt) are implemented, otherwise defaultloss and defaultopt will be used.

The data used for training is the result of itergeneration(dataiter, gen) where gen is the generation number. This defaults to returning dataiter but allows for more complex iterators such as StatefulGenerationIter.

If the model loss is ever NaN or Inf the training will be stopped and invalidfitness will be returned without calculating the fitness using fitstrat.

Tip: Use TimedIterator to stop training of models which take too long to train.

struct TrainAccuracyFitness <: AbstractFitness
TrainAccuracyFitness(;drop=0.5, kwargs...)

Measure fitness as the accuracy on the training data set. Beware of overfitting!

Parameter drop determines the fraction of examples to drop for fitness measurement. This mitigates the penalty for newly mutated candidates as the first part of the training examples are not used for fitness.

Other keyword arguments are passed to TrainThenFitness constructor. Note that fitstrat should generally be left to default value.

Advantage vs AccuracyFitness is that one does not have to run through another data set. Disadvantage is that evolution will likely favour candidates which overfit.

SizeFitness <: AbstractFitness
SizeFitness()

Measure fitness as the total number of parameters in the function to be evaluated.

TimeFitness{T} <: AbstractFitness

Measure fitness as time to evaluate a function.

Time for first nskip evaluations will be discarded.

LogFitness{F, MF} <: AbstractFitness
LogFitness(fitnesstrategy::AbstractFitness) = LogFitness(;fitnesstrategy)
LogFitness(;currgen=0, candcnt=0, fitnesstrategy, msgfun=default_fitnessmsgfun)

Logs the fitness of fitnessstrategy along with some candiate information.

GpuFitness{F} <: AbstractFitness
GpuFitness(fitnesstrategy)

Move candidates to gpu before calculating their fitness according to fitnesstrategy.

Copies parameters back to the given candidate after fitness have been computed to ensure that updated parameters from training are used. Assumes canidates have parameters on cpu for that step.

Note that if no gpu is available this should be a noop.

MapFitness <: AbstractFitness
MapFitness(mapping::Function, base::AbstractFitness)

Maps fitness x from base to mapping(x).

EwmaFitness(base)
EwmaFitness(α, base)

Computes the exponentially weighted moving average of the fitness of base. Assumes that candidates previous fitness metric is available through fitness(cand).

Main purpose is to mitigate the effects of fitness noise.

See https://github.com/DrChainsaw/NaiveGAExperiments/blob/master/fitnessnoise/experiments.ipynb for some hints as to why this might be needed.

AggFitness <: AbstractFitness
AggFitness(aggfun::Function, fitnesses::AbstractFitness...)

Aggreagate fitness value from all fitnesses using aggfun