[HOW-TO] high resolution streaming with raspberry pi 4B + arducam 64MP (imx477)

[noemiLuna]

Hi,
I'm working on a raspberry pi 4B and I'm trying to perform a video streaming with a resolution equal to 3840x2160
My goal is basically to reach the same performance of the following libcamera command:

libcamera-vid --codec mjpeg --flush 1 -n -t 0 -k --inline --listen --quality 85 --width 3840 --height 2160 --framerate 20

but I need to do it through picamera2.
To reach this goal, ideally I need to have just the "lores" stream but I'm not able to find a way to deactivate the "main" stream.
Indeed, keeping active both streams is not possible if we need an high resolution because of memory issues due to the fact that if "lores" stream resolution is set to 3840x2160, "main" stream resolution must be higher or the following error will be raised:

" File "/usr/local/lib/python3.11/dist-packages/picamera2/picamera2.py", line 830, in check_camera_config
raise RuntimeError("lores stream dimensions may not exceed main stream")
RuntimeError: lores stream dimensions may not exceed main stream "

I already set cma memory to 512 in the config.txt but memory is still not enough.

I hope someone have some suggestion to deactivate "main" stream and keep just "lores" which seams to be faster.
(to delete main stream, It seams not enough to define just "lores" stream in the configuration like in the following command: create_video_configuration(lores={"size": (3840, 2160), "format": "YUV420"}, encode="lores"))

I also tried to define main stream with same configuration of the example above:

however, the use of "yuv420" format and MJPEGencoder on "main" stream raises the following error:

File "/usr/local/lib/python3.11/dist-packages/picamera2/encoders/v4l2_encoder.py", line 149, in _start
fcntl.ioctl(self.vd, VIDIOC_STREAMON, typev)
ProcessLookupError: [Errno 3] No such process

If you have any suggestion let me know.

thank you in advance for your help.

[davidplowman]

Hi, I think I've already answered this question on the forum. My advice was to avoid the "lores" image because you have to have a "main" image, so you may as well just use that. Also reduce the buffer_count, maybe try buffer_count=3.

Although "format": "YUV420" should in principle work with the MJPEGEncoder, that's a hardware block on the Pi 4, so you might find there's a hard limit to the size it can handle (I can't remember off the top of my head, I'm afraid).

If it still isn't working, please post a short but self-contained code snippet - configuring and starting the recording should be less than a dozen lines, I would think.

[noemiLuna]

Hi, thanks for reply! I still haven't solved the problem. I wasn't able to make main stream work with MJPEG encoder.

here is a code snipped:

from picamera2 import Picamera2
from picamera2.encoders import MJPEGEncoder
from picamera2.outputs import FileOutput
from threading import Condition
import io

class StreamingOutput(io.BufferedIOBase):
def init(self):
self.frame = None
self.condition = Condition()

camera = Picamera2()
encoder = MJPEGEncoder()
encoder.format = "YUV420"
config = camera.create_video_configuration(main={"size": (3840, 2160), "format": "YUV420"}, encode="main",buffer_count=3)
camera.configure(config)

streamOut = StreamingOutput()
encoder.output = FileOutput(streamOut)
encoder.size = config["main"]["size"]

camera.start()
camera.start_encoder(encoder)

[davidplowman]

Hi, you don't say in what way it doesn't work. I would start with the absolute simplest example possible, such as

import time
from picamera2 import Picamera2
from picamera2.encoders import MJPEGEncoder
from picamera2.outputs import FileOutput

camera = Picamera2()
config = camera.create_video_configuration({'size': (3840, 2160), 'format': 'YUV420'}, buffer_count=3)
camera.configure(config)
encoder = MJPEGEncoder()
output = FileOutput("test.mjpg")

camera.start_recording(encoder, output)
time.sleep(5)
camera.stop_recording()

Does that recording anything? If not, in what way does it fail?

[noemiLuna]

Just tried and this is the output.

[1:17:07.323424285] [1312] INFO Camera camera_manager.cpp:325 libcamera v0.3.2+124-e0a2304e-dirty (2024-12-09T06:48:46GMT)
[1:17:07.340101982] [1318] WARN CameraSensorProperties camera_sensor_properties.cpp:305 No static properties available for 'arducam_64mp'
[1:17:07.340156036] [1318] WARN CameraSensorProperties camera_sensor_properties.cpp:307 Please consider updating the camera sensor properties database
[1:17:07.403854929] [1318] WARN RPiSdn sdn.cpp:40 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[1:17:07.405499151] [1318] INFO RPI vc4.cpp:455 Registered camera /base/soc/i2c0mux/i2c@1/arducam_64mp@1a to Unicam device /dev/media2 and ISP device /dev/media0
[1:17:07.405619093] [1318] INFO RPI pipeline_base.cpp:1145 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/arducam_64mp.yaml'
[1:17:07.407544235] [1312] INFO Camera camera_manager.cpp:325 libcamera v0.3.2+124-e0a2304e-dirty (2024-12-09T06:48:46GMT)
[1:17:07.424347225] [1321] WARN CameraSensorProperties camera_sensor_properties.cpp:305 No static properties available for 'arducam_64mp'
[1:17:07.424398391] [1321] WARN CameraSensorProperties camera_sensor_properties.cpp:307 Please consider updating the camera sensor properties database
[1:17:07.522682187] [1321] WARN RPiSdn sdn.cpp:40 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[1:17:07.524171246] [1321] INFO RPI vc4.cpp:455 Registered camera /base/soc/i2c0mux/i2c@1/arducam_64mp@1a to Unicam device /dev/media2 and ISP device /dev/media0
[1:17:07.524253041] [1321] INFO RPI pipeline_base.cpp:1145 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/arducam_64mp.yaml'
[1:17:07.530744079] [1312] INFO Camera camera.cpp:1197 configuring streams: (0) 3840x2160-YUV420 (1) 3840x2160-SRGGB10_CSI2P
[1:17:07.531202328] [1321] INFO RPI vc4.cpp:630 Sensor: /base/soc/i2c0mux/i2c@1/arducam_64mp@1a - Selected sensor format: 3840x2160-SRGGB10_1X10 - Selected unicam format: 3840x2160-pRAA
Traceback (most recent call last):
File "/opt/pathoxtrack/api/teststream.py", line 12, in
camera.start_recording(encoder, output)
File "/usr/local/lib/python3.11/dist-packages/picamera2/picamera2.py", line 1760, in start_recording
self.start_encoder(encoder, output, pts=pts, quality=quality, name=name)
File "/usr/local/lib/python3.11/dist-packages/picamera2/picamera2.py", line 1700, in start_encoder
_encoder.start(quality=quality)
File "/usr/local/lib/python3.11/dist-packages/picamera2/encoders/encoder.py", line 227, in start
self._start()
File "/usr/local/lib/python3.11/dist-packages/picamera2/encoders/v4l2_encoder.py", line 149, in _start
fcntl.ioctl(self.vd, VIDIOC_STREAMON, typev)

the file test.mjpeg is saved but is empty

[davidplowman]

I wonder if the image size is too large for the hardware MJPEG encoder on the Pi 4. Can you try size (1280, 720) instead?

[noemiLuna]

Unfortunately you are right! Up to 2312x1736 I'm not getting the error. A resolution of 3840x2160 seams to be too large.
But why? with libcamera-vid is possible to have a stream with that resolution....

[davidplowman]

rpicam-vid (aka. libcamera-vid) doesn't use the h/w MJPEG encoder, it does software encode. In theory, you can force the use of software encode in Python as well. Replace MJPEGEncoder by JpegEncoder. However, you might get other problems. Python doesn't like YUV420, so you'd probably have to change the format to "RGB888" (for example), which is less efficient and uses more memory for no benefit.

[noemiLuna]

yes I can confirm and it's the reason I was trying to use the MJPEGencoder with resolution of 3840x2160: using jpegencoder with RGB888 format reduces a lot the performance and is not possible to use YUV420 format.
So there isn't a solution for my problem, isn't it?

[davidplowman]

I suppose it all depends on exactly what you want to do. Is there something missing from rpicam-vid that you need?

Slightly longer term, I have submitted patches upstream to allow software JPEG encode of YUV420, which will allow the Python performance to improve to be much closer to rpicam-vid, but it may take some time to get that accepted.

[noemiLuna]

the only missed think from rpicam-vid is that is less easy to be integrated with my python workflow and alsp, when camera is closed, resources are not always correctly released resulting in errors when you try to start a new streaming or take a still acquisition.
Any suggestion about it?

I hope these updates will be accepted soon! thank you for your help...

[davidplowman]

the only missed think from rpicam-vid is that is less easy to be integrated with my python workflow and alsp, when camera is closed, resources are not always correctly released resulting in errors when you try to start a new streaming or take a still acquisition. Any suggestion about it?

I can't think of any reason for that. Obviously you have to run rpicam-vid as a separate process, so there might be timing issues when shutting it down, remembering that only one process can use the camera at a time. As always, if you can make a (short, simple!) test case then we can look at it.

I hope these updates will be accepted soon! thank you for your help...

Me too!!