Vertex Mutation

Δnin!([utilityfun], v, Δ...)
Δnin!([utilityfun], v1 => Δ1, v2 => Δ2,...)
Δnin!([utilityfun], Δs::AbstractDict)

Change input size of all provided vertices with the associated Δ and make the appropriate changes to other vertices so that the graph is aligned w.r.t activations. Return true if successful (false if not successful).

For Δs provided as integers it must be possible to change the size by exactly Δ or else the attempt will be considered failed. A failed attempt will be retried immediately in relaxed form where the wanted size changes are moved to the objective. The relaxation means that input size might not change by exactly Δ. Use relaxed(Δ) to indicate that a size change is relaxed in the initial attempt.

For vertices with more than one input, the size change must be expressed as a tuple with one element per input. Use missing to indicate that no special treatment is needed for an input. Both missing and relaxed can be mixed freely inside and outside the tuples (see examples).

Note that the above constrain makes Δnin! much more cumbersome to use compared to Δnout! and in most cases there are no direct advantages of using Δnin! over Δnout! as they both boil down to the same thing.

Argument utilityfun provides a vector utility = utilityfun(vx) for any vertex vx in the same graph as v where utility[i] > utility[j] indicates that output neuron index i shall be preferred over j for vertex vx. It may also provide a scalar which will be used as utility of all neurons of vx. If not provided, defaultutility(vx) will be used.

Note that Δnin!([utilityfun], args...) is equivalent to Δsize!([utilityfun], ΔNin(args...)).

Examples

julia> using NaiveNASlib, NaiveNASlib.Extend, NaiveNASlib.Advanced

julia> module TestNNLib
           using NaiveNASlib, NaiveNASlib.Extend
           mutable struct Affine{T} W::Matrix{T} end
           NaiveNASlib.nout(a::Affine) = size(a.W, 1)
           NaiveNASlib.nin(a::Affine) = [size(a.W, 2)]
           NaiveNASlib.Δsize!(a::Affine, ins::AbstractVector, outs::AbstractVector) = a.W = parselect(a.W, 2=>ins[1], 1=>outs)
           affine(in, outsize) = absorbvertex(Affine(ones(outsize, nout(in))), in)
           export affine
       end;

julia> using .TestNNLib;

julia> iv = affine(inputvertex("in", 3), 6);

julia> v1 = affine(iv, 4);

julia> v2 = affine(iv, 5);

julia> Δnin!(v1, 3);

julia> Δnin!(v1 => -3, v2 => -2);

julia> Δnin!(Dict(v1 => 3, v2 => 2));

julia> v3 = conc(affine(iv, 4), affine(iv, 5), affine(iv, 6); dims=2);

julia> v4 = conc(affine(iv, 7), affine(iv, 8); dims=2);

julia> Δnin!(v3, 3, missing, 2);

julia> Δnin!(v3 => (3, missing, 2), v4 => (-2, 0));

julia> Δnin!(v3, relaxed(3), missing, 2);

julia> Δnin!(v3 => (relaxed(3), missing, 2), v4 => relaxed(-2, 0)) do v
           randn(nout(v))
       end;

Δnout!([utilityfun], v, Δ...)
Δnout!([utilityfun], v1 => Δ1, v2 => Δ2,...)
Δnout!([utilityfun], Δs::AbstractDict)

Change output size of all provided vertices with the associated Δ and make the appropriate changes to other vertices so that the graph is aligned w.r.t activations. Return true if successful (false if not successful).

For Δs provided as integers it must be possible to change the size by exactly Δ or else the attempt will be considered failed. A failed attempt will be retried immediately in relaxed form where the wanted size changes are moved to the objective. The relaxation means that output size might not change by exactly Δ. Use relaxed(Δ) to indicate that a size change is relaxed in the initial attempt.

Argument utilityfun provides a vector utility = utilityfun(vx) for any vertex vx in the same graph as v where utility[i] > utility[j] indicates that output neuron index i shall be preferred over j for vertex vx. It may also provide a scalar which will be used as utility of all neurons of vx. If not provided, defaultutility(vx) will be used.

Note that Δnout!([utilityfun], args...) is equivalent to Δsize!([utilityfun], ΔNout(args...)).

Examples

julia> using NaiveNASlib, NaiveNASlib.Extend, NaiveNASlib.Advanced

julia> module TestNNLib
           using NaiveNASlib, NaiveNASlib.Extend
           mutable struct Affine{T} W::Matrix{T} end
           NaiveNASlib.nout(a::Affine) = size(a.W, 1)
           NaiveNASlib.nin(a::Affine) = [size(a.W, 2)]
           NaiveNASlib.Δsize!(a::Affine, ins::AbstractVector, outs::AbstractVector) = a.W = parselect(a.W, 2=>ins[1], 1=>outs)
           affine(in, outsize) = absorbvertex(Affine(ones(outsize, nout(in))), in)
           export affine
       end;

julia> using .TestNNLib;

julia> iv = affine(inputvertex("in", 3), 6);

julia> v1 = affine(iv, 4);

julia> v2 = affine(iv, 5);

julia> Δnout!(v1, 3);

julia> Δnout!(v1 => -3, v2 => -2);

julia> Δnout!(Dict(v1 => 3, v2 => 2));

julia> Δnout!(v1, relaxed(2));

julia> Δnout!(v1 => relaxed(2), v2 => -1) do v
           randn(nout(v))
       end;

Δsize!(s::AbstractΔSizeStrategy) 
Δsize!(utilityfun, s::AbstractΔSizeStrategy)

Change size of (potentially) all vertices which s has a chance to impact the size of.

Argument utilityfun provides a vector utility = utilityfun(vx) for any vertex vx in the same graph as v where utility[i] > utility[j] indicates that output neuron index i shall be preferred over j for vertex vx. It may also provide a scalar which will be used as utility of all neurons of vx. If not provided, defaultutility(vx) will be used.

Δsize!([utilityfun], [s::AbstractΔSizeStrategy], g::CompGraph)
Δsize!([utilityfun], [s::AbstractΔSizeStrategy], v::AbstractVertex)

Change size of (potentially) all vertices of graph g (or graph to which v is connected) according to the provided AbstractΔSizeStrategy (default UpperInsertBound).

Return true of operation was successful, false otherwise.

Argument utilityfun provides a vector utility = utilityfun(vx) for any vertex vx in the same graph as v where utility[i] > utility[j] indicates that output neuron index i shall be preferred over j for vertex vx. It may also provide a scalar which will be used as utility of all neurons of vx. If not provided, defaultutility(vx) will be used.

Δsize!([utilityfun], [s::AbstractΔSizeStrategy], vs::AbstractVector{<:AbstractVertex})

Change size of (potentially) all vertices in vs according to the provided AbstractΔSizeStrategy (default UpperInsertBound).

Argument utilityfun provides a vector utility = utilityfun(vx) for any vertex vx in the same graph as v where utility[i] > utility[j] indicates that output neuron index i shall be preferred over j for vertex vx. It may also provide a scalar which will be used as utility of all neurons of vx. If not provided, defaultutility(vx) will be used.

relaxed(Δ)

Return Δ => Relaxed() which indicates that Δ shall be relaxed when changing size.

See Δnout! and Δnin!.

insert!(vin::AbstractVertex, factory::Function, outselect::Function=identity)

Replace vin as input to all outputs of vin with vertex produced by factory.

The function factory takes vin as input.

Example:

Before:

    vin
   / | \
  /  |  \
 v₁ ... vₙ

After:

    vin
     |
   vnew₁
     ⋮
   vnewₖ
   / | \
  /  |  \
 v₁ ... vₙ

Note that the connection vin -> vnew₁ as well as all connections vnewₚ -> vnewₚ₊₁ is done by factory in the above example.

The function outselect can be used to select a subset of outputs to replace (default all).

remove!(v::MutationVertex, strategy=RemoveStrategy())

Removes v from the graph by removing it from its inputs and outputs.

It is possible to supply a strategy for how to 1) reconnect the inputs and outputs of v and 2) align the input and output sizes of the inputs and outputs of v.

Default strategy is to first set nin==nout for v and then connect all its inputs to all its outputs.

create_edge!(from::AbstractVertex, to::AbstractVertex; [pos], [strategy])

Create and edge from from to to at index pos in inputs(to).

Sizes will be adjusted based on given strategy.

remove_edge!(from::AbstractVertex, to::AbstractVertex; [nr], [strategy])

Remove edge from from to to.

If there are multiple edges from from to to then nr can be used to distinguish which one shall be removed.

Sizes will be adjusted based on given strategy.