Crossover Operations

VertexCrossover{S, PF, CF}
VertexCrossover(crossover ;selection=FilterMutationAllowed(), pairgen=default_pairgen)
VertexCrossover(crossover, deviation::Number; selection=FilterMutationAllowed())

Applies crossover to each pair of selected vertices from two CompGraphs.

Vertices to select from the first graph is determined by selection (default FilterMutationAllowed) while pairgen (default default_pairgen) determines how to pair the selected vertices with vertices in the second graph.

The default pairing function will try to pair vertices which have similar relative topologial order within their graphs. For instance, if the first graph has 5 vertices and the second has 10, it will pair vertex 2 from the first graph with vertex 4 from the second (assuming they are of compatible type). The parameter deviation can be used to inject noise in this process so that the pairing will randomly deviate where the magnitude of deviation sets how much and how often.

See also crossover.

CrossoverSwap{F1, F2, F3, S} <: AbstractCrossover{<:AbstractVertex}
CrossoverSwap(pairgen, mergefun, strategy, selection)
CrossoverSwap(;pairgen=default_inputs_pairgen, mergefun=default_mergefun, strategy=default_crossoverswap_strategy, selection=FilterMutationAllowed())
CrossoverSwap(deviation::Number; mergefun=default_mergefun, strategy=default_crossoverswap_strategy, selection=FilterMutationAllowed())

Swap out a part of one graph with a part of another graph, making sure that the graphs do not become connected in the process.

More concretely, swaps a set of consecutive vertices vs1 set of consecutive vertices vs2 returning the swapped v1 and v2 respectively if successful or v1 and v2 if not.

The last vertex in vs1 is v1 and the last vertex of vs2 is v2. The other members of vs1 and vs2 are determined by pairgen (default default_inputs_pairgen) and selection (default FilterMutationAllowed).

If a vertex v is not capable of having multiple inputs (determined by singleinput(v) == true), vm = mergefun(vi) where vi is the input to v will be used instead of v and v will be added as the output of vm if necessary.

See also crossoverswap.

Note: High likelyhood of large accuracy degradation after applying this mutation.

OptimiserCrossover{C} <: AbstractCrossover{Optimisers.AbstractRule}
OptimiserCrossover()
OptimiserCrossover(crossover)

Apply crossover between optimisers.

Type of crossover is determined by crossover (default optimiserswap) which when given a a tuple of two optimisers will return the result of the crossover operation as a tuple of optimisers.

Designed to be composable with most utility AbstractMutations as well as with itself. For instance, the following seemingly odd construct will swap components of a Optimisers.OptimiserChain with a probability of 0.2 per component:

OptimiserCrossover(MutationProbability(OptimiserCrossover(), 0.2))

Compare with the following which either swaps all components or none:

MutationProbability(OptimiserCrossover(), 0.2)

LearningRateCrossover()

Return an OptimiserCrossover which will swap learning rates between optimisers but not change anything else.

Does not do anything if any of the optimisers don't have a learning rate (e.g. WeightDecay).

IteratorMapCrossover{C} <: AbstractCrossover{AbstractIteratorMap}
IteratorMapCrossover() 
IteratorMapCrossover(crossover)

Apply crossover between AbstractIteratorMaps.

Type of crossover is determined by crossover (default iteratormapswap) which when given a a tuple of two AbstractIteratorMaps will return the result of the crossover operation as a tuple of AbstractIteratorMaps.

Designed to be composable with most utility AbstractMutations as well as with itself. For instance, the following seemingly odd construct will swap components of two IteratorMaps with a probability of 0.2 per component:

IteratorMapCrossover(MutationProbability(IteratorMapCrossover(), 0.2))

Compare with the following which either swaps all components or none:

MutationProbability(IteratorMapCrossover(), 0.2)

crossover(g1::CompGraph, g2::CompGraph; selection=FilterMutationAllowed(), pairgen=default_pairgen, crossoverfun=crossoverswap)

Perform crossover between g1 and g2 and return the two children g1' and g2'.

How the crossover is performed depends on pairgen and crossoverfun as well as selection.

selection is used to filter out at which vertices the crossoverpoints may be.

pairgen return indices of the potential crossover points to use out of the allowed crossover points. New crossover points will be drawn from pairgen until it returns nothing.

crossoverfun return the result of the crossover which may be same as inputs depending on implementation.

crossoverswap((v1,v2)::Tuple; pairgen=default_inputs_pairgen, selection=FilterMutationAllowed(), kwargs...)

Perform crossoverswap! with v1 and v2 as output crossover points.

Inputs are selected from the feasible set (as determined by separablefrom and selection) through the supplied pairgen function.

Inputs v1 and v2 along with their entire graph are copied before operation is performed and originals are returned if operation is not successful.

Additional keyword arguments will be passed on to crossoverswap!.

crossoverswap!(v1::AbstractVertex, v2::AbstractVertex) = crossoverswap!(v1,v1,v2,v2)
crossoverswap!(vin1::AbstractVertex, vout1::AbstractVertex, vin2::AbstractVertex, vout2::AbstractVertex)

Swap vertices vin1 to vout1 with vin2 and vout2 so that vin1 to vin2 is placed in the same position of the graph as vin2 to vout2 and vice versa.

Vertices must come from different graphs.

This operation can fail, leaving one or both graphs in a corrupted state where evaluating them results in an error (typically a DimensionMismatch error).

Return a tuple (success1, success2) where success1 is true if vin2 and vout2 was successfully swapped in to the graph which previously contained vin1 and vout1 and vice versa for success2.

default_pairgen(vs1, vs2, deviation = 0.0; rng=rng_default, compatiblefun = sameactdims, ind1 = rand(rng, eachindex(vs1)))

Return integers ind1 and ind2 so that vs1[ind1] and vs2[ind2] are a suitable pair for crossover.

Input function compatiblefun(v1,v2) shall return true if v1 and v2 can be swapped and is used to determine the set of vertices to select from given.

From the set of compatible vertices in vs2, the one which has the smallest relative topologial distance from vs1[ind2] is selected. The parameter devation can be used to randomly deviate from this where larger magnitude means more deviation.

default_inputs_pairgen(vs1, vs2, args...;kwargs...)

Same as default_pairgen except it also ensures that shape changes of feature maps are consistent between the pairs.

Feature map here refers to the shape of inputs to convolutional-type layers (Conv, Pooling) in dimensions other than the batch dimension or the channel dimension.

For example, for 2D convolutions, the arrays may have the shape WxHxCxB where C and B are the channel and batch dimensions respectively. The shape of the feature map in this case is then WxH.

More concretely, if x is the shape of a feature maps input to vin1 and f1(x) is the shape of the feature maps output from vout1 and f2 describes the same relation between vin2 and vout2 then only vertices vin2' for which f1(x) == f2(x) ∀ x for a selected vertex vin1 may be returned.

This function assumes that the last vertex in vs1 and vs2 are vout1 and vout2 respectively.

This prevents issues where graphs become inconsistent due to

1. Feature maps become zero sized
2. Feature maps of different sizes are merged (e.g. concatenated or element wise added)

Note that this is more strict than needed as a change in the feature maps size does not necessarily result in 1) or 2).