Create Jupyter notebooks examples

[avan1235]

We can show the process of working with KotlinDL and the following steps using the jupyter notebooks examples.
Such example would include step by step description to show the process of network building and its visualization (when some visualization functions are published)
In my opinion it's worth to publish in form of noteboo at least the example of:

• conv2d image classification with images and activations visualisations (depends on Visualisations of model with letsplot #70)
• conv1d audio classification with sound and activations visualisations (depends on Add Conv3D with introducing AbstractConv #104 and Visualisations of model with letsplot #70)
• transfer learning step by step guide in notebook

I can of course work on it if it would be suitable and when the requested changes would be merged

The problem with introducing notebooks from my prespective is using the code from KotlinDL that haven't been published yet. I think a good solution would be to publish the SNAPSHOT version of the library after every merge of commit to merge master - it's api could change but after the final release of KotlinDL someone would have to check all notebooks to use the official jar with dependencies that would work.

I checked and see no possibility to actually include in notebooks classes from the local repository (not published jar file) as Kotlin jupyter supports the @file:DependsOn but not @file:Import annotation which allows including dependencies from another files (and even if it would I think we would have to include to many files manually instead of including single jar with e.g. SNAPSHOT version of library)

[zaleslaw]

You described why I don't use notebooks like examples/integration-tests, it doesn't work well in IDEA, and so on.

It's a good idea to publish sometimes (every 2 weeks, for example, the developer version of the library), but I'm not sure about the SNAPSHOT version after each commit (but accept you)

The proposed functionality is covered in our tutorials (but they are not runnable, of course), maybe better to start by converting all these tutorials to the notebooks? or have their equivalents?
In this way, we need to update them one per release (after the API is frozen and we in the phase of documentation writing)

What is the notebook? Is it a code or just text? If it's code, we need to have an instrument to refactor it together during development (but it doesn't work this way, as I understand). If it's a text, we could wait before API is frozen (I never try to update tutorials after each commit).

So, I suggest that the visualization API will be merged to release the dev version like 0.3-dev-1, and you could use it to write the first version of notebooks.

Also, we need to understand where we should keep these notebooks (with Kotlin or Python code)? Could it be mixed with code?
To answer this question I suggest researching different projects with the same problem..

[avan1235]

You described why I don't use notebooks like examples/integration-tests, it doesn't work well in IDEA, and so on.

It's a good idea to publish sometimes (every 2 weeks, for example, the developer version of the library), but I'm not sure about the SNAPSHOT version after each commit (but accept you)

The proposed functionality is covered in our tutorials (but they are not runnable, of course), maybe better to start by converting all these tutorials to the notebooks? or have their equivalents?
In this way, we need to update them one per release (after the API is frozen and we in the phase of documentation writing)

That sounds like a great idea to work with tutorials. I mean the notebooks in known .ipynb format so it is readable from GH page as well as markdown documents (let's see at some random example here). I think of it like of the runnable extension of Markdown that can have its execution results also included (which I would prefer to included to increase the readability)

What is the notebook? Is it a code or just text? If it's code, we need to have an instrument to refactor it together during development (but it doesn't work this way, as I understand). If it's a text, we could wait before API is frozen (I never try to update tutorials after each commit).

So yes, it's code interleaved with text but would need manual refactoring process if there is no eligible plugin for IDEA. But the important thing is that the notebooks would rely on the api from jar file (not from local files) so as far as I understand they could be broken iff someone changes the library jar file but not update the code (which would be just stupid).

So, I suggest that the visualization API will be merged to release the dev version like 0.3-dev-1, and you could use it to write the first version of notebooks.

That would be satisfying solution

Also, we need to understand where we should keep these notebooks (with Kotlin or Python code)? Could it be mixed with code?

I worked with .ipynb notebooks which are markdown with kotlin/python code (depending on kernel use but I think single kernel is possible for file) but I don't see the need to mix the python with kotlin (the exmples with exporting / importing models can be separated and linked)

To answer this question I suggest researching different projects with the same problem..

I just saw the pytorch approach and they use some other alternative callled sphinx but I am not sure if it would be a good approach in Kotlin now

[zaleslaw]

I created an issue #119 related to first step - tutorial conversion to the notebooks

[avan1235]

Great, so we can leave this topic for now util some visualization functions would be available (as official or dev release) and now will focus on the tutorials in notebook formats to see if they can be a better approach than markdown files