Batch Size Utilities

batchsizeselection(inshape_nobatch::Tuple; maxmemutil=0.7, uppersize=nothing, alternatives=nothing, batchsizefun=limit_maxbatchsize)

Return a batch size selection callable which may be used to select an appropriate batch size when given a model and a suggested batch size.

inshape_nobatch is the size of the input without the batch dimension (e.g. 3 values for images) to be assumed. See BatchSizeSelectionWithDefaultInShape

batchsizefun is a function with the following signature:

batchsizefun(batchsize, model; inshape_nobatch, availablebytes)

It returns the largest batch size not larger than batchsize which can be used for model without using more than availablebytes bytes of memory. The type of batchsize may be used to e.g. determine if one shall account for backwards pass (if typeof(batchsize) === TrainBatchSize) or not (if typeof(batchsize) == ValidationBatchSize).

maxmemutil is the maximum memory utilization which typically need to be < 1 to account for inaccuracies in the estimation. See BatchSizeSelectionScaled

If uppersize is not nothing the maximum possible batchsize smaller or equal to uppersize will be used. See BatchSizeSelectionMaxSize

If alternatives is not nothing, the returned batchsize will be quantized to the closest matching size in alternatives which is not bigger than the unquantized batch size. See BatchSizeSelectionFromAlternatives.

Examples

julia> using NaiveGAflux, Flux

julia> import NaiveGAflux: TrainBatchSize, ValidationBatchSize # Needed only for examples


julia> graph =  let 
                    v0 = conv2dinputvertex("v0", 3);
                    v1 = fluxvertex("v1", Conv((3,3), nout(v0) => 8), v0);
                    CompGraph(v0, v1);
                end;

julia> bs = batchsizeselection((32,32,3));

julia> bs(TrainBatchSize(128), graph; availablebytes = 10_000_000) # availablebytes supplied for doctest reasons
84

julia> bs(ValidationBatchSize(128), graph; availablebytes = 10_000_000)
128

julia> bs = batchsizeselection((32,32,3); maxmemutil=0.1);

julia> bs(TrainBatchSize(128), graph; availablebytes = 10_000_000)
12

julia> bs(ValidationBatchSize(128), graph; availablebytes = 10_000_000)
24

julia> bs = batchsizeselection((32,32,3); uppersize=1024);

julia> bs(TrainBatchSize(128), graph; availablebytes = 10_000_000)
84

julia> bs(ValidationBatchSize(128), graph; availablebytes = 10_000_000)
170

julia> bs = batchsizeselection((32,32,3); uppersize=1024, alternatives = 2 .^ (0:10));

julia> bs(TrainBatchSize(128), graph; availablebytes = 10_000_000)
64

julia> bs(ValidationBatchSize(128), graph; availablebytes = 10_000_000)
128

BatchSizeSelectionWithDefaultInShape{T, F}
BatchSizeSelectionWithDefaultInShape(default_inshape)
BatchSizeSelectionWithDefaultInShape(default_inshape, batchsizefun)

Batch size selection with a default assumed inshape used for estimating valid batch sizes.

batchsizefun is a function with the following signature:

batchsizefun(batchsize, model; inshape_nobatch, availablebytes)

It returns the largest batch size not larger than batchsize which can be used for model without using more than availablebytes bytes of memory. The type of batchsize may be used to e.g. determine if one shall account for backwards pass (if typeof(batchsize) === TrainBatchSize) or not (if typeof(batchsize) == ValidationBatchSize).

Returns the result of batchsizefun with default value of inshape_nobatch = default_inshape when called as a function with valid inputs to batchsizefun.

Composable with other batch size selection types which may be used as batchsizefun. See examples.

Examples

julia> using NaiveGAflux, Flux

julia> import NaiveGAflux: TrainBatchSize, ValidationBatchSize # Needed only for examples


julia> graph =  let 
                    v0 = conv2dinputvertex("v0", 3);
                    v1 = fluxvertex("v1", Conv((3,3), nout(v0) => 8), v0);
                    CompGraph(v0, v1);
                end;

julia> bs = BatchSizeSelectionWithDefaultInShape((32,32,3));

julia> bs(TrainBatchSize(512), graph; availablebytes = 10_000_000) # availablebytes supplied for doctest reasons
120

julia> bs(TrainBatchSize(512), graph; availablebytes = 1000_000_000)
512

julia> sbs = BatchSizeSelectionWithDefaultInShape((32,32,3), BatchSizeSelectionScaled(0.5));

julia> sbs(TrainBatchSize(512), graph; availablebytes = 10_000_000)
60

julia> sbs(TrainBatchSize(512), graph; availablebytes = 1000_000_000)
512

julia> bs(ValidationBatchSize(512), graph; availablebytes=10_000_000)
243

BatchSizeSelectionScaled{F}
BatchSizeSelectionScaled(scale)
BatchSizeSelectionScaled(scale, batchsizefun)

Batch size selection with a margin applied when estimating valid batch sizes.

batchsizefun is a function with the following signature:

batchsizefun(batchsize, model; inshape_nobatch, availablebytes)

It returns the largest batch size not larger than batchsize which can be used for model without using more than availablebytes bytes of memory. The type of batchsize may be used to e.g. determine if one shall account for backwards pass (if typeof(batchsize) === TrainBatchSize) or not (if typeof(batchsize) == ValidationBatchSize).

Returns the result of batchsizefun with default value of availablebytes = floor(scale * availablebytes) when called as a function with valid inputs to batchsizefun.

Composable with other batch size selection types which may be used as batchsizefun. See examples.

Examples

julia> using NaiveGAflux, Flux

julia> import NaiveGAflux: TrainBatchSize, ValidationBatchSize # Needed only for examples


julia> graph =  let 
                    v0 = conv2dinputvertex("v0", 3);
                    v1 = fluxvertex("v1", Conv((3,3), nout(v0) => 8), v0);
                    CompGraph(v0, v1);
                end;

julia> bs = BatchSizeSelectionScaled(0.5);

julia> bs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 10_000_000) # availablebytes supplied for doctest reasons
60

julia> bs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 1000_000_000)
512

julia> sbs = BatchSizeSelectionScaled(0.5, BatchSizeSelectionWithDefaultInShape((32,32,3)));

julia> sbs(TrainBatchSize(512), graph; availablebytes = 10_000_000)
60

julia> sbs(TrainBatchSize(512), graph; availablebytes = 1000_000_000)
512

julia> bs(ValidationBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes=10_000_000)
121

BatchSizeSelectionFromAlternatives{T, F}
BatchSizeSelectionFromAlternatives(alts)
BatchSizeSelectionFromAlternatives(alts, batchsizefun)

Batch size selection from a set of available alternatives. Useful for iterators which need to be pre-loaded with batch size, for example the iterators in this package.

batchsizefun is a function with the following signature:

batchsizefun(batchsize, model; inshape_nobatch, availablebytes)

It returns the largest batch size not larger than batchsize which can be used for model without using more than availablebytes bytes of memory. The type of batchsize may be used to e.g. determine if one shall account for backwards pass (if typeof(batchsize) === TrainBatchSize) or not (if typeof(batchsize) == ValidationBatchSize).

Returns the largest number in alts smaller than the result of batchsizefun when called as a function with valid inputs to batchsizefun.

Composable with other batch size selection types which may be used as batchsizefun. See examples.

Examples

julia> using NaiveGAflux, Flux

julia> import NaiveGAflux: TrainBatchSize, ValidationBatchSize # Needed only for examples


julia> graph =  let 
                    v0 = conv2dinputvertex("v0", 3);
                    v1 = fluxvertex("v1", Conv((3,3), nout(v0) => 8), v0);
                    CompGraph(v0, v1);
                end;

julia> bs = BatchSizeSelectionFromAlternatives(2 .^ (0:10));

julia> bs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 10_000_000) # availablebytes supplied for doctest reasons
64

julia> bs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 1000_000_000)
512

julia> sbs = BatchSizeSelectionFromAlternatives(2 .^ (0:10), BatchSizeSelectionScaled(0.5));

julia> sbs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 10_000_000)
32

julia> sbs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 1000_000_000)
512

julia> bs(ValidationBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes=10_000_000)
128

BatchSizeSelectionMaxSize{F}
BatchSizeSelectionMaxSize(uppersize) 
BatchSizeSelectionMaxSize(uppersize, batchsizefun)

Batch size selection which always try to select uppersize. Basically the strategy to select the largest batchsize which fits in memory.

batchsizefun is a function with the following signature:

batchsizefun(batchsize, model; inshape_nobatch, availablebytes)

It returns the largest batch size not larger than batchsize which can be used for model without using more than availablebytes bytes of memory. The type of batchsize may be used to e.g. determine if one shall account for backwards pass (if typeof(batchsize) === TrainBatchSize) or not (if typeof(batchsize) == ValidationBatchSize).

Returns the result of batchsizefun but with the batchsize as uppersize of the same type as batchsize (i.e. to differentiate between train size and validation size).

Composable with other batch size selection types which may be used as batchsizefun. See examples.

Examples

julia> using NaiveGAflux, Flux

julia> import NaiveGAflux: TrainBatchSize, ValidationBatchSize # Needed only for examples


julia> graph =  let 
                    v0 = conv2dinputvertex("v0", 3);
                    v1 = fluxvertex("v1", Conv((3,3), nout(v0) => 8), v0);
                    CompGraph(v0, v1);
                end;

julia> bs = BatchSizeSelectionMaxSize(1024);

julia> bs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 10_000_000) # availablebytes supplied for doctest reasons
120

julia> bs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 1000_000_000)
1024

julia> sbs = BatchSizeSelectionMaxSize(1024, BatchSizeSelectionScaled(0.5));

julia> sbs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 10_000_000)
60

julia> sbs(TrainBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes = 1000_000_000)
1024

julia> bs(ValidationBatchSize(512), graph; inshape_nobatch=(32,32,3), availablebytes=10_000_000)
243