Iterator Maps

AbstractIteratorMap

Abstract type for mapping training and validation dataset iterators using maptrain(im, iter) and mapvalidation(im, iter) respectively where im is the struct extending AbstractIteratorMap.

Main reason for existence is to enable dispatch to AbstractMutation{AbstractIteratorMap} and AbstractCrossover{AbstractIteratorMap} so that strategies for data augmentation and batch size selection can be evolved.

ShieldedIteratorMap{T}
ShieldedIteratorMap(map)

Shields map from mutation and crossover.

BatchSizeIteratorMap{F} <: AbstractIteratorMap 
BatchSizeIteratorMap(limitfun, trainbatchsize, validationbatchsize, model)

AbstractIteratorMap which sets the batch size of training and validation iterators to trainbatchsize and validationbatchsize respectively. limitfun is used to try to ensure that batch sizes are small enough so that training and validating model does not risk an out of memory error. Use batchsizeselection to create an appropriate limitfun.

Examples

julia> using NaiveGAflux

julia> import NaiveGAflux: maptrain, mapvalidation # needed for examples only

julia> bsim = BatchSizeIteratorMap(4, 8, batchsizeselection((32,32,3)));

julia> collect(maptrain(bsim, (1:20,)))
5-element Vector{Vector{Int64}}:
 [1, 2, 3, 4]
 [5, 6, 7, 8]
 [9, 10, 11, 12]
 [13, 14, 15, 16]
 [17, 18, 19, 20]

julia> collect(mapvalidation(bsim, (1:20,)))
3-element Vector{Vector{Int64}}:
 [1, 2, 3, 4, 5, 6, 7, 8]
 [9, 10, 11, 12, 13, 14, 15, 16]
 [17, 18, 19, 20]

julia> map(x -> Pair(x...), maptrain(bsim, ((1:20, 21:40),))) # Pair to make results a bit easier on the eyes
5-element Vector{Pair{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}, SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}}:
     [1, 2, 3, 4] => [21, 22, 23, 24]
     [5, 6, 7, 8] => [25, 26, 27, 28]
  [9, 10, 11, 12] => [29, 30, 31, 32]
 [13, 14, 15, 16] => [33, 34, 35, 36]
 [17, 18, 19, 20] => [37, 38, 39, 40]

julia> map(x -> Pair(x...), maptrain(bsim, BatchIterator((1:20, 21:40),12))) # Pair to make results a bit easier on the eyes
5-element Vector{Pair{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}, SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}}:
     [1, 2, 3, 4] => [21, 22, 23, 24]
     [5, 6, 7, 8] => [25, 26, 27, 28]
  [9, 10, 11, 12] => [29, 30, 31, 32]
 [13, 14, 15, 16] => [33, 34, 35, 36]
 [17, 18, 19, 20] => [37, 38, 39, 40]

IteratorMaps{T} <: AbstractIteratorMap 
IteratorMaps(maps...)
IteratorMaps(maps::Tuple)

Aggregates multiple AbstractIteratorMaps. maptrain and mapvalidation are applied sequentially starting with the first element of maps.

maptrain(im::AbstractIteratorMap, iter)

Return an iterator (default iter) suitable for training.

mapvalidation(im::AbstractIteratorMap, iter)

Return an iterator (default iter) suitable for validation.

limit_maxbatchsize(im::AbstractIteratorMap, args...; kwargs...)

Return an AbstractIteratorMap which is capable of limiting the batch size if applicable to the type of im (e.g. if im is a BatchSizeIteratorMap), otherwise return im.