OpenVINO Execution Provider

[bopeng1234]

Hi there,

I am wondering if the onnxruntime-genai support OpenVINO as an Execution Provider.

as I found in example, only provide the execution sample on CPU/CUDA/DML, no OpenVINO.

To my understanding, the OpenVINO EP now, can only run one time inference, for example tradition image classification, but can't run text generation (multi round inference and kv caching)?

Best Regards!

[RyanUnderhill]

Yes, it doesn't currently support it but we can add basic support for it (where onnxruntime uses it). I think GenAI doing CPU scoring is no problem for OpenVINO as it's CPU based.

[bopeng1234]

thanks, do we have any branch we are working on, to add openvino ep to onnxruntime-genai