6.3.2025

For Dioflex, we keep developing new applications to make their work easier, clearer, and faster. The latest project is an application for handling customer complaints. It's a relatively simple tool that scans a barcode to identify the returned product and prints a label with a QR code and issue description on a special printer.

Of course, it's a bit more complex than that—product information is stored in multiple different information systems, the returned product needs to be photographed (both an overview and a microscopic image), recorded, and a notification about the complaint must be sent to the customer. Thanks to our app, the time needed to log a complaint has been reduced from ten minutes to just a few seconds.

Due to the need for photographs, the entire application was designed for Raspberry Pi 5, which allows the connection of two HQ cameras. And the cameras turned out to be a tough nut to crack. Despite the widespread use of Raspberry Pi, working with cameras is far from easy, and the necessary libraries contain some frustrating bugs.

On Raspberry Pi, a camera is connected to a Qt + QML application using Kamera v Rpi se do Qt + QML aplikace připojuje pomocí GStreamer (GST). GST is a framework for processing video and audio, allowing different filters and image manipulators to be chained together to achieve the desired functionality step by step. One example of this is simply displaying the output from any camera on the screen:

gst-launch-1.0 v4l2src ! videoconvert ! autovideosink

This GST pipeline can be used on a regular Linux PC with a connected USB (or other) camera. However, Raspberry Pi is slightly different. The HQ camera is not connected via the standard Linux V4L2 system; instead, it uses the libcamera: library:

gst-launch-1.0 libcamerasrc ! videoconvert ! autovideosink

On a Raspberry Pi without a graphical interface, this command will display the camera output directly in the console (which is why it's more convenient to work with Raspberry Pi over a network). If we want to send the output to a Qt application, we need to change the output element in the pipeline:

libcamerasrc ! videoconvert ! qtvideosink

Such a GST pipeline cannot be run directly from the command line; instead, a simple QML application must be used. For example:

import QtQuick 2.7
import QtMultimedia 5.15

Item {
    width: 800;
    height: 600;

    MediaPlayer {
        id: player;
        autoLoad: true;
        autoPlay: true;
        source: "gst-pipeline: libcamerasrc ! videoconvert ! qtvideosink"
        }

    VideoOutput {
        source: player;
        anchors.fill: parent;
        }

    }

One issue is resolution. The libcamerasrc element automatically selects the lowest available resolution. Fortunately, GST takes this into account and allows you to increase the resolution manually. To list the available resolutions and connected cameras, you can use:

root@raspberrypi:~# libcamera-vid --list-cameras 
Available cameras
-----------------
0 : imx477 [4056x3040 12-bit RGGB] (/base/soc/i2c0mux/i2c@1/imx477@1a)
    Modes: 'SRGGB10_CSI2P' : 1332x990 [120.05 fps - (696, 528)/2664x1980 crop]
           'SRGGB12_CSI2P' : 2028x1080 [50.03 fps - (0, 440)/4056x2160 crop]
                             2028x1520 [40.01 fps - (0, 0)/4056x3040 crop]
                             4056x3040 [10.00 fps - (0, 0)/4056x3040 crop]

I requested a higher resolution in the GST pipeline (only showing the source parameter from the QML application):

gst-pipeline: libcamerasrc ! video/x-raw,width=4056,height=3040 ! videoconvert ! qtvideosink

However, when I tried to set a higher resolution, instead of a proper image, I got diagonal stripes with corrupted color information. On the left: the expected image, on the right: the broken image.

The diagonal stripes suggest that somewhere along the GST pipeline, the line length was misinterpreted during processing. Behind this simple sentence lies a week of debugging, hundreds of megabytes of hexadecimal dumps, and a lot of frustration and lost nerves.

I won’t bore you with technical details—here is the final working solution for both Raspberry Pi 4 and Raspberry Pi 5:

// Rpi 4:
gst-pipeline: libcamerasrc ! 
    video/x-raw,width=4056,height=3040,format=NV12 !
    rawvideoparse format=nv12 width=4056 height=3040 plane-strides=<4064,4064> plane-offsets=<0,12354560> frame-size=37063680 !
    qtvideosink sync=false

// Rpi 5:
gst-pipeline: libcamerasrc ! 
    video/x-raw,width=4056,height=3040,format=NV12 !
    rawvideoparse format=nv12 width=4056 height=3040 plane-strides=<4096,4096> plane-offsets=<0,12451840> frame-size=37355520 !
    qtvideosink sync=false

It can also be useful to distinguish cameras by name when multiple cameras are connected to the Raspberry Pi. Camera identification can be obtained using: libcamera-vid --list-cameras (as described earlier). To select a specific HQ camera from multiple connected cameras, you can use the following approach:

gst-pipeline: libcamerasrc camera-name=/base/soc/i2c0mux/i2c@1/imx477@1a ! ...

On my Raspberry Pi 5, the cameras are named as follows: /base/axi/pcie@120000/rp1/i2c@80000/imx477@1a and /base/axi/pcie@120000/rp1/i2c@88000/imx477@1a. The only difference is a single character, and even realizing that the names were actually different took an enormous amount of time!

I didn’t investigate the exact cause of these issues. The problem could be in GStreamer, but it could just as well be in libcamera.

If you try to run a GST pipeline at higher resolution directly on the Raspberry Pi console, you’ll run into another issue: The image will have completely incorrect colors. This does not affect the Qt application in any way, but it can mislead you when debugging the GST pipeline from the command line.

The Raspberry Pi is a low-cost computer capable of handling a vast amount of work for very little money. Its low price, high adaptability, and various communication options make it ideal for use in environments where a regular PC would not be suitable. Our software, carefully designed to meet the highly specific requirements of our customers, is running on hundreds of installations: Dioflex, Náš nový vrátný, Vacushape, Robe.