Get rid of Activations enum

[ileasile]

Activations enum duplicates Activation interface and makes adding new activation function harder for library contributors and impossible for library users.

Get rid of enum and add convenient "provider" service which stores activation functions objects.

[zaleslaw]

A good idea, I will think about it. Thanks

[knok16]

What do you think about the next course of action:

1. Replace usages of Activations enum with Activation interface
2. Move all implementations of Activation interface (SigmoidActivation, LishtActivation, etc) into separate files and move documentation from Activations enum into implementation classes
3. Delete Activations enum or maybe transform it into the registry of activation functions like:

object Activations {
    val Linear = LinearActivation()
    val Sigmoid = SigmoidActivation()
    ...

4. Remove some of the Activation layers, for example org.jetbrains.kotlinx.dl.api.core.layer.activation.ELU since right now there are 2 ways to create ELU activation layer: org.jetbrains.kotlinx.dl.api.core.layer.activation.ELU() and ActivationLayer(EluActivation()) - and it means 2x more code to support

[zaleslaw]

I agree with your argument about the importance of customization and it's a part of Keras's spirit too.
I suggest planning these API changes in the next release, 0.4.

We should raise a discussion and ask the community about a few of the proposed API changes.

[knok16]

Agree, such changes can break API, plus it can differ from Keras API, which is one more thing to consider before implementing such changes (since it can complicate adoption)

[zaleslaw]

I suppose we could go far from the basic Keras API in a few moments to be a more Kotlin framework adopting the best Kotlin features