embeddings RoadMap ?

[iwaitu]

Is there a plan on the RoadMap to support text embeddings? Similar to Ollama, where in Python I can do:

import ollama
model_id = "llama"
result = ollama.embeddings(model=model_id, prompt="test embedding")

It would be great if GenAI could add an embeddings object.

[kunal-vaishnavi]

The embeddings (a.k.a. last hidden states) are now accessible via ONNX Runtime GenAI. You can create the ONNX model to output the hidden states using ONNX Runtime GenAI's model builder with --extra_options include_hidden_states=true.