Using multiple gpu for single prompt inferencing

[zorolevid]

I have 4 gpu core of 32 gb each but the prompt length is big so it's needing 40 gb of memory due to which I am getting out of memory error is there any way using which I can use 2 cores of gpu for inferencing. I am using qween 72B chat model.