[Feature Request] Expand current Dataset API

[jbaron]

Currently the dataset api allows only for simple X -> Y data sets. So one input and one output. There are many models that require more input and/or output data to be fed into the training loop.

Since other API's in KotlinDL rely on the dataset API, would it make sense to already come up with a more generic API?

Typically in the Python world the output it is either arbitrary length Tuple (so perhaps a List in Kotlin) or a Dict (which is nicer IMHO since the key gives some indication what the individual tensors are used for).

[zaleslaw]

@jbaron Thanks for the feedback. I totally agree with you. The dataset API is minimal. You shared a few tips and ideas; it could be the initial point for the development. If you have any ideas, please share them on this issue. Maybe you could share any links or notebooks with your cases to learn them in detail.

[jbaron]

@zaleslaw I'll give it some more thoughts and see if I can come up with some examples. Perhaps better to not to just copy directly copy Python API's (and as a result use Map everywhere ;). Also it is key for many training examples that the data set can keep up with the training loop, so that often implies some sort of parallel behavior.