Mutation Operations

NoutMutation <:AbstractMutation{AbstractVertex} NoutMutation(l1::Real,l2::Real, rng::AbstractRNG) NoutMutation(limit, rng::AbstractRNG=rng_default) NoutMutation(l1,l2)

Mutate the out size of a vertex or vector of vertices.

Size is changed by x * nout(v) rounded away from from zero where x is drawn from U(minrel, maxrel) where minrel and maxrel are l1 and l2 if l1 < l2 and l2 and l1 otherwise.

AddVertexMutation <:AbstractMutation{AbstractVertex} AddVertexMutation(s::AbstractArchSpace, outselect::Function, WeightInit::AbstractWeightInit, rng::AbstractRNG) AddVertexMutation(s, outselect::Function=identity) AddVertexMutation(s, rng::AbstractRNG) AddVertexMutation(s, wi::AbstractWeightInit)

Insert a vertex from the wrapped AbstractArchSpace s after a given vertex v.

The function outselect takes an AbstractVector{AbstractVertex} representing the output of v and returns an AbstractVector{AbstractVertex} which shall be reconnected to the vertex v' returned by s. Defaults to identity meaning all outputs of v are reconnected to v'.

RemoveVertexMutation <:AbstractMutation{AbstractVertex} RemoveVertexMutation(s::RemoveStrategy) RemoveVertexMutation()

Remove the given vertex v using the configured RemoveStrategy.

Default size align strategy is IncreaseSmaller -> DecreaseBigger -> AlignSizeBoth -> FailAlignSizeWarn -> FailAlignSizeRevert.

Default reconnect strategy is ConnectAll.

Note: High likelyhood of large accuracy degradation after applying this mutation.

AddEdgeMutation <: AbstractMutation{AbstractVertex} AddEdgeMutation(p; rng=rngdefault, mergefun=defaultmergefun(rng=rng), filtfun=noshapechange, utilityfun=defaultneuronselect) AddEdgeMutation(p::Probability; rng=rngdefault, mergefun=defaultmergefun(rng=rng), filtfun=noshapechange, utilityfun=defaultneuronselect)

Add an edge from a vertex vi to another vertex vo randomly selected from vs = filtfun(vi).

Higher values of p will give more preference to earlier vertices of vs.

If vo is not capable of having multiple inputs (determined by singleinput(v) == true), vm = mergefun(voi) where voi is a randomly selected input to vo will be used instead of vo and vo will be added as the output of vm.

When selecting neurons/outputs after any eventual size change the output of utilityfun(v) will be used to determine the utlity of each output in vertex v. Note that length(utilityfun(v)) == nout(v) must hold.

Note: High likelyhood of large accuracy degradation after applying this mutation.

RemoveEdgeMutation <: AbstractMutation{AbstractVertex} RemoveEdgeMutation(;utilityfun=defaultneuronselect, rng=rngdefault)

Remove an edge from a vertex vi to another vertex vo randomly selected from outputs(vi).

Vertex vi must have more than one output and vertex vo must have more than one output for the edge to be removed. Otherwise no change is made.

If there are multiple edges between vi and vo no change will be made due to NaiveNASlib not being able to revert a failed operation in this case..

When selecting neurons/outputs after any eventual size change the output of utilityfun(v) will be used to determine the utlity of each output in vertex v. Note that length(utilityfun(v)) == nout(v) must hold.

Note: High likelyhood of large accuracy degradation after applying this mutation.

KernelSizeMutation{N} <: AbstractMutation{AbstractVertex} KernelSizeMutation(Δsizespace::AbstractParSpace{N, Int}; maxsize, pad, rng) KernelSizeMutation2D(absΔ::Integer;maxsize, pad, rng) KernelSizeMutation(absΔ::Integer...;maxsize, pad, rng)

Mutate the size of filter kernels of convolutional layers.

Note: High likelyhood of large accuracy degradation after applying this mutation.

KernelSizeMutation2D is a convenience constructor for KernelSizeMutation(absΔ, absΔ;...).

ActivationFunctionMutation{T,R} <: AbstractMutation{AbstractVertex} where {T <: AbstractParSpace{1}, R <: AbstractRNG} ActivationFunctionMutation(actspace::AbstractParSpace{1}, rng::AbstractRNG) ActivationFunctionMutation(acts...;rng=rngdefault) ActivationFunctionMutation(acts::AbstractVector;rng=rngdefault)

Mutate the activation function of layers which have an activation function.

Note: High likelyhood of large accuracy degradation after applying this mutation.

struct OptimiserMutation{F} <: AbstractMutation{Optimisers.AbstractRule}
OptimiserMutation(optfun)
OptimiserMutation(os::Union{Tuple, <:AbstractArray})

Mutatates optimisers not wrapped in ShieldedOpt through optfun.

Invoked recursively for Optimisers.OptimiserChains.

LearningRateMutation(rng=rng_default)

Return an OptimiserMutation which mutates the learning rate of optimisers.

AddOptimiserMutation{F} <: AbstractMutation{Optimisers.AbstractRule}

Adds optimiser generated by optgen(os) to the set of optimisers where os is the existing set.

An attempt to merge optimisers of the same type is made using mergeopts.

TrainBatchSizeMutation{R<:Real, Q, RNG<:AbstractRNG} 
TrainBatchSizeMutation(l1, l2, quantizeto, rng) 
TrainBatchSizeMutation(l1, l2, rng::AbstractRNG) 
TrainBatchSizeMutation(l1, l2, quantizeto)
TrainBatchSizeMutation(l1, l2)

Mutate the batch size used for training.

Maximum possible relative change is determined by the numbers l1 and l2.

Behaviour depends on quantizeto (default Int) in the following way.

If quantizeto is a DataType (e.g Int) then the largest possible increase is maxrel * batchsize and the largest possible decrease is minrel * batchsize where minrel and maxrel are l1 and l2 if l1 < l2 and l2 and l1 otherwise. Note that if (minrel, maxrel) is not symetric around 0 the mutation will be biased.

More precisely, the new size is round(quantizeto, (x+1) * batchsize) where x is drawn from U(minrel, maxrel).

If quantizeto is a an array or tuple of values then the new size is drawn from quantizeto with elements closer to the current batch size being more likely.

More precisely, the new size is quantizeto[i] where i = j + round(Int, x * length(quantizeto)) where x is drawn from U(minrel, maxrel) and j is the index for which quantizeto[j] is the closest to the current batch size.

Use the function mutate_batchsize to get a feeling for how different values of l1, l2 and quantizeto affect the new batch sizes. Note that setting l1 == l2 means that x in the descriptions above will always equal to l1 (and l2) can also be useful in this context.

Examples

julia> using NaiveGAflux

julia> m = TrainBatchSizeMutation(-0.1, 0.1);

julia> m(BatchSizeIteratorMap(16, 32, identity))
BatchSizeIteratorMap{typeof(identity)}(NaiveGAflux.TrainBatchSize(15), NaiveGAflux.ValidationBatchSize(32), identity)

julia> m = TrainBatchSizeMutation(-0.1, 0.1, ntuple(i -> 2^i, 10)); # Quantize to powers of 2

julia> m(BatchSizeIteratorMap(16, 32, identity))
BatchSizeIteratorMap{typeof(identity)}(NaiveGAflux.TrainBatchSize(32), NaiveGAflux.ValidationBatchSize(32), identity)

julia> NaiveGAflux.mutate_batchsize(Int, 16, -0.3, 0.3)
14

julia> NaiveGAflux.mutate_batchsize(Int, 16, -0.3, 0.3)
19

julia> NaiveGAflux.mutate_batchsize(ntuple(i -> 2^i, 10),  16, -0.3, 0.3)
64

julia> NaiveGAflux.mutate_batchsize(ntuple(i -> 2^i, 10),  16, -0.3, 0.3)
8

VertexMutation <: DecoratingMutation{CompGraph} VertexMutation(m::AbstractMutation{AbstractVertex}, s::AbstractVertexSelection) VertexMutation(m::AbstractMutation{AbstractVertex})

Applies a wrapped AbstractMutation{AbstractVertex} to each selected vertex in a CompGraph.

Vertices to select is determined by the configured AbstractVertexSelection.

MutationProbability{T} <: DecoratingMutation{T}
MutationProbability(m::AbstractMutation{T}, p::Probability)
MutationProbability(m::AbstractMutation{T}, p::Number)

Applies m with probability p.

WeightedMutationProbability{T,F} <: DecoratingMutation{T}
WeightedMutationProbability(m::AbstractMutation::T, pfun::F)

Applies m to an entity e with a probability pfun(e).

HighUtilityMutationProbability(m::AbstractMutation{T}, pbase::Real, rng=rng_default; spread=0.5)

Return a WeightedMutationProbability which applies m to vertices with an (approximately) average probability of pbase and where high neuronutility compared to other vertices in same graph means higher probability.

Parameter spread can be used to control how much the difference in probability is between high and low utlity. High spread means high difference while low spread means low difference.

LowUtilityMutationProbability(m::AbstractMutation{T}, pbase::Real, rng=rng_default; spread=2)

Return a WeightedMutationProbability which applies m to vertices with an (approximately) average probability of pbase and where low neuronutility compared to other vertices in same graph means higher probability.

Parameter spread can be used to control how much the difference in probability is between high and low utlity. High spread means high difference while low spread means low difference.

MutationChain{T} <: DecoratingMutation{T}
MutationChain(m::AbstractMutation{T}...)

Chains multiple AbstractMutation{T}s after each other.

Input entities will be mutated by the first AbstractMutation{T} in the chain and the output will be fed into the next AbstractMutation{T} in the chain and so on. The output from the last AbstractMutation{T} is returned.

LogMutation{T} < :DecoratingMutation{T}
LogMutation(strfun, m::AbstractMutation{T}; level = Logging.Info, nextlogfun=e -> PrefixLogger("   "))
LogMutation(strfun, level::LogLevel, nextlogfun, m::AbstractMutation{T})

Logs all mutation operations.

Argument strfun maps the mutated entity to the logged string.

Calling nextlogfun(e) where e is the entity to mutate produces an AbstractLogger which will be used when applying m(e).

By default, this is used to add a level of indentation to subsequent logging calls which makes logs of hierarchical mutations (e.g. mutate a CompGraph by applying mutations to some of its vertices) easier to read. Set nextlogfun = e -> current_logger() to remove this behaviour.

MutationFilter{T} <: DecoratingMutation{T}
MutationFilter(predicate, m)

Applies mutation m only for entities e for which predicate(e) returns true.