Search Spaces

SingletonParSpace{N, T} <:AbstractParSpace{N, T}
SingletonParSpace(p::T...)
Singleton2DParSpace(p::T)

Singleton search space. Has exactly one value per dimension.

Singleton2DParSpace is a convenience constructor for a 2D SingletonParSpace of [p, p].

ParSpace{N, T} <:AbstractParSpace{N, T}
ParSpace(p::AbstractVector{T}...)
ParSpace1D(p...)
ParSpace2D(p::AbstractVector)

Search space for parameters.

Return independent uniform random values for all N dimensions from the search space when invoked.

ParSpace1D is a convenience constructor for an 1D ParSpace of p. ParSpace2D is a convenience constructor for a 2D ParSpace of [p, p]`.

CoupledParSpace{N, T} <:AbstractParSpace{N, T}
CoupledParSpace(p::AbstractParSpace{1, T}, N) 
CoupledParSpace(p::AbstractVector{T}, N)

Search space for parameters.

Return the same uniformly sampled value for all N dimensions from the search space when invoked.

BaseLayerSpace
BaseLayerSpace(outsizes, activationfunctions)

Search space for basic attributes common to all layers.

outsizes is the output sizes (number of neurons). activationfunctions is the activation functions.

NamedLayerSpace <:AbstractLayerSpace
NamedLayerSpace(name::String, s::AbstractLayerSpace)

Adds a name to an AbstractLayerSpace.

LoggingLayerSpace <: AbstractLayerSpace
LoggingLayerSpace(s::AbstractLayerSpace; level=Logging.Debug, nextlogfun=() -> PrefixLogger("   "))
LoggingLayerSpace(msgfun, s::AbstractLayerSpace; level = Logging.Debug, nextlogfun = () -> PrefixLogger("   "))
LoggingLayerSpace(level::LogLevel, msgfun, nextlogfun, s::AbstractLayerSpace)

Logs msgfun(layer) at loglevel level after creating a layer from s.

Calling nextlogfun() produces an AbstractLogger which will be used when creating layer from s.

By default, this is used to add a level of indentation to subsequent logging calls which makes logs of hierarchical archspaces easier to read. Set nextlogfun = () -> current_logger() to remove this behaviour.

DenseSpace <:AbstractLayerSpace
DenseSpace(base::BaseLayerSpace)
DenseSpace(outsizes, activations)

Search space of Dense layers.

ConvSpace{N} <:AbstractLayerSpace

ConvSpace{N}(;outsizes, kernelsizes, activations=identity, strides=1, dilations=1, paddings=SamePad(), convfuns=Conv)
ConvSpace(convfun::AbstractParSpace, base::BaseLayerSpace, ks::AbstractParSpace, stride::AbstractParSpace, dilation::AbstractParSpace, pad)

Search space of ND convolutional layers.

Constructor with keyword arguments takes scalars, vectors or AbstractParSpaces as inputs.

BatchNormSpace <:AbstractLayerSpace
BatchNormSpace(activationfunctions...)
BatchNormSpace(activationfunctions::AbstractVector)

Search space of BatchNorm layers.

PoolSpace{N} <:AbstractLayerSpace
PoolSpace{N}(;windowsizes, strides=1, paddings=SamePad(), poolfun=[MaxPool, MeanPool])

Search space of ND pooling layers.

Constructor with keyword arguments takes scalars/tuples, vectors or AbstractParSpaces as inputs.

LayerVertexConf
LayerVertexConf(layerfun, traitfun)
LayerVertexConf(;layerfun=ActivationContribution ∘ LazyMutable, traitfun=validated() ∘ default_logging())

Generic configuration template for computation graph vertices with Flux layers as their computation.

Both layerfun and traitfun are forwarded to NaiveNASflux.fluxvertex.

Intention is to make it easy to add logging, validation and pruning metrics in an uniform way.

Shielded(base=LayerVertexConf(); allowed = tuple())

Create a LayerVertexConf which is shielded from mutation.

Keyword allowed can be used to supply a tuple (or array) of AbstractMutation types to allow.

ConcConf
ConcConf(layerfun, traitfun)

Generic configuration template for concatenation of vertex outputs.

Both layerfun and traitfun are forwarded to NaiveNASflux.concat.

NoOpArchSpace <: AbstractArchSpace

Returns input vertex without any modification.

LoggingArchSpace <: AbstractArchSpace
LoggingArchSpace(s::AbstractArchSpace; level=Logging.Debug, nextlogfun=in -> PrefixLogger("   "))
LoggingArchSpace(msgfun, s::AbstractArchSpace; level=Logging.Debug, nextlogfun=in -> PrefixLogger("   "))
LoggingArchSpace(msgfun::Function, level::LogLevel, nextlogfun, s::AbstractArchSpace)

Logs msgfun(vertex) at loglevel level after creating a vertex from s.

Calling nextlogfun(in) where in is the input vertex produces an AbstractLogger which will be used when creating vertex from s.

By default, this is used to add a level of indentation to subsequent logging calls which makes logs of hierarchical archspaces easier to read. Set nextlogfun = e -> current_logger() to remove this behaviour.

VertexSpace <:AbstractArchSpace
VertexSpace([conf::LayerVertexConf], lspace::AbstractLayerSpace)

Search space of one AbstractVertex with compuation drawn from lspace.

conf is used to attach other metadata to the vertex. See LayerVertexConf.

ArchSpace <:AbstractArchSpace
ArchSpace(ss::AbstractLayerSpace...; conf=LayerVertexConf()) 
ArchSpace(ss::AbstractArchSpace...)

Search space of AbstractArchSpaces.

Draws one vertex from one of ss (uniformly selected) when invoked.

ConditionalArchSpace{P, S1, S2} <: AbstractArchSpace
ConditionalArchSpace(predicate, iftrue, iffalse=NoOpArchSpace())
ConditionalArchSpace(;predicate, iftrue, iffalse=NoOpArchSpace())

Use iftrue if predicate(invertex) returns true, iffalse otherwise.

RepeatArchSpace <:AbstractArchSpace
RepeatArchSpace(s::AbstractArchSpace, r::Integer) 
RepeatArchSpace(s::AbstractArchSpace, r::AbstractVector{<:Integer})

Search space of repetitions of another AbstractArchSpace where number of repetitions is uniformly drawn from r.

Output of each generated candidate is input to next and the last output is returned.

ArchSpaceChain <:AbstractArchSpace
ArchSpaceChain(s::AbstractArchSpace...)

Chains multiple AbstractArchSpaces after each other.

Input vertex will be used to generate an output vertex from the first AbstractArchSpace in the chain which is then used to generate a next output vertex from the next AbstractArchSpace in the chain and so on. The output from the last AbstractArchSpace is returned.

ForkArchSpace <:AbstractArchSpace
ForkArchSpace(s::AbstractArchSpace, r::Integer; conf=ConcConf())
ForkArchSpace(s::AbstractArchSpace, r::AbstractVector{<:Integer}; conf=ConcConf())

Search space of parallel paths from another AbstractArchSpace where number of paths is uniformly drawn from r.

Input vertex is input to a number of paths drawn from an AbstractParSpace. Concatenation of paths is output.

ResidualArchSpace <:AbstractArchSpace
ResidualArchSpace(s::AbstractArchSpace, [conf::VertexConf])

Turns the wrapped AbstractArchSpace into a residual.

Return x = y + in where y is drawn from the wrapped AbstractArchSpace when invoked with in as input vertex.

conf is used to decorate trait and wrap the computation (e.g in an ActivationContribution).

FunctionSpace <: AbstractArchSpace
FunctionSpace(funs...; namesuff::String, conf=LayerVertexConf(ActivationContribution, validated() ∘ default_logging()))

Return a SizeInvariant vertex representing fun(x) when invoked with in as input vertex where x is output of in where fun is uniformly selected from funs.

GlobalPoolSpace(Ts...)
GlobalPoolSpace(conf::LayerVertexConf, Ts...)

Short for FunctionSpace with global average or global max pooling.

Also adds a MutationShield to prevent the vertex from being removed by default.

DefaultWeightInit <: AbstractWeightInit

Use the layer default weight initialization.

IdentityWeightInit <: AbstractWeightInit
IdentityWeightInit()

Initialize weights with an identity mapping.

ZeroWeightInit <: AbstractWeightInit
ZeroWeightInit()

Initialize weights as zeros.

Main use case is to let residual layers be and identity mapping