v4l2 capture deserializer parallel camera data

[dskim]

Hi, I'm developing the parallel camera on a custom board:

TinyRex Max module (iMX6Q) with max9272(deserializer) + max9271(serializer) + ISP + AR0143AT(image sensor)

The deserializer(max9272) parallel output are connected with TinyRex.

Yocto linux was installed, referring to http://wiki.voipac.com/xwiki/bin/vie...+tinyrex/yocto


The deserializer is controlled by I2C on #0 bus at 0x48.

# i2cdetect -y 0
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: 40 -- -- -- -- -- -- -- 48 -- -- -- -- -- -- --
50: UU -- -- -- -- -- -- 57 -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --

# i2cdump -y 0 0x48
No size specified (using byte-data access)
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f    0123456789abcdef
00: 80 90 1f 00 87 29 02 00 00 00 00 00 00 b6 7a 00    ???.?)?......?z.
10: 00 00 00 00 0a 22 30 54 30 c8 0f 5f ff 30 0a 02    ....?"0T0??_.0??
20: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX    XXXXXXXXXXXXXXXX
...<skip>...
f0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX    XXXXXXXXXXXXXXXX

The driver for the deserializer is not included in the BSP, kernel sources.

For the deserializer, i wrote a device tree but it's not accurate.

&i2c1 {
   /* Deserializer MAX9272*/
   max9272: max9272@48 {
      status = "okay";
      compatible = "maxim,max9272";
      reg = <0x48>;
      clocks = <&clks IMX6QDL_CLK_DUMMY>;
      clock-names = "csi_mclk";
      pwn-gpios = <&gpio3 10 GPIO_ACTIVE_HIGH>;
      led-gpios = <&gpio2 22 GPIO_ACTIVE_HIGH>;
      csi_id = <0>;
      pinctrl-names = "default";
      pinctrl-0 = <&pinctrl_ipu1_csi0>;
      mclk = <37000000>;
      mclk_source = <0>;
   };
};

I want to capture or preview video with my custom camera, but failed to open /dev/video0, /dev/video1 files.

# v4l2-ctl --all
ERROR: v4l2 capture: slave not found!
Failed to open /dev/video0: Resource temporarily unavailable

# gst-launch-1.0 imxv4l2src device=/dev/video0 ! imxv4l2sink
ERROR: v4l2 capture: slave not found!
ERROR: v4l2 capture: slave not found!
Can't get caps from capture device, use the default setting.
Perhaps haven't capture device.
====== IMXV4L2SRC: 4.1.6 build on Sep 18 2019 17:40:41. ======
====== IMXV4L2SINK: 4.1.6 build on Sep 18 2019 17:40:41. ======
WARNING: erroneous pipeline: could not link imxv4l2src0 to imxv4l2sink0

The error message "ERROR: v4l2 capture: slave not found!" came from mxc_v4l2_capture module, (drivers/media/platform/mxc/capture/mxc_v4l2_capture.c)

below condition returns true so the error occurs.

cam->sensor == NULL

cam->sensor have to have driver_data, but it seems sensor is NULL.


1. Do i need a driver for the deserializer?
If then, Please give me a link for reference.
If not, how can i capture video with deserializer parallel camera input?

2. How do i create a device tree for the deserializer? There are many samples for the image sensor, but not samples for the deserializer.

Thanks. Regards,
Das Kim

[marek]

Hi Mr. Das Kim,

Robert ask me if i can provide more detailed feedback on this topic.
I don't know the answer for your question at all but I will try to type down some comments.

I2cdetect do not displays UU but rather 48. As you correctly pointed out there is no driver controlling this chip.
I think at least some dummy driver is necessary so it can respond for upper layer requests.
For example it can report in which format data are provided and report and accept resolution and frame-rate settings from application.

~/workdir/voipac/imx6/linux/linux-fslc-4.1-rex$ grep -r max9272 ./

The 'max9272' string from device tree not point to any existing driver so i think
all device tree note settings are ignored because there is no driver that will work with them.

Here is for example settings for adv7610 that is also connected to parallel camera interface.

/ {
    v4l2_cap_0 {
        compatible = "fsl,imx6q-v4l2-capture";
        ipu_id = <0>;
        csi_id = <0>;
        mclk_source = <0>;
        mipi_camera = <0>;
        default_input = <1>;
        status = "okay";
    };
};

&i2c3 {
    adv7610: adv7610@4c {
        compatible = "adv,adv7610";
        reg = <0x4c>;
        pinctrl-names = "default";
        pinctrl-0 = <&pinctrl_ipu1_csi0>;
        csi_id = <0>;
        reset-gpio = <&gpio1 7 GPIO_ACTIVE_HIGH>;
    };
};

Driver

~/workdir/voipac/imx6/linux/linux-fslc-4.1-rex/drivers/media/platform/mxc/capture/adv7610.c

1. Do i need a driver for the deserializer?
I think you do. I would try to emulate some existing driver interface. The examples before are for 'V4L2_IF_TYPE_BT1120_PROGRESSIVE_SDR' input.
Your input signal is probably different. Feel free to use other kernels to evaluate CSI input. Kernel provided by voipac have modification to
support RPI MINI camera v1.3 and ADV7610. The changes might affect (should not) other sensors.

2. How do i create a device tree for the deserializer? There are many samples for the image sensor, but not samples for the deserializer.
I do not think there are many differences. The problematic part is usually the driver or "how to configure i.mx6" register so it accepts input signal.
Per my understanding you have to provide to i.MX6 CSI compatible signal (it does not matter if it comes from image sensor or deserializer)
and i2c interface is just for initialization and configuration.

Here are some output from ADV7610 testing (It is connected to CSI0 parallel interface):
http://wiki.voipac.com/xwiki/bin/vie...2Fmax2Fultra29

Generally you can use information from:
Reference manual (includes CSI description)
https://www.nxp.com/products/process...umentation_Tab

Google search for NXP support forum
i.mx6 csi parallel site:community.nxp.com

[dskim]

Thanks for your help, Marek.

As you said, I wrote the deserializer(max9272) driver based on the max9286_mipi driver (from NXP community), then the error "

ERROR: v4l2 capture: slave not found!

" disappeared.

When i captured using the Gstreamer,

# gst-launch-1.0 v4l2src device="/dev/video0" ! 'video/x-raw,format=(string)UYVY,width=640,height=480,framerate=30/1' ! videoconvert ! autovideosink
Setting pipeline to PAUSED ...
display(/dev/fb0) resolution is (1920x1080).
====== OVERLAYSINK: 4.1.6 build on Sep 18 2019 17:40:44. ======
ipu_csi_window_size_crop: Error left=0 top=59565955
display(/dev/fb0) resolution is (1920x1080).
display(/dev/fb0) resolution is (1920x1080).
mxc_v4l2_output v4l2_out: Bypass IC.
mxc_v4l2_output v4l2_out: Bypass IC.
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: v4l2 capture: width or height too small.
ERROR: v4l2 capture: unsupported ioctrl!
ERROR: v4l2 capture: unsupported ioctrl!
UYVY

Then the output results were as followsphoto #1)
If I wave my hand in front of the lens, the green area and white area moves. That means output parallel data flow well.

Why does it happen? Any idea or suggestions will welcome.

Thanks.
Das Kim

[marek]

I'm not an expert in this area all so I can't really advice you. I saw your post on NXP forum while I was trying to google for some information about AR0143AT(image sensor). They are working with i.MX and cameras on daily basis and its very good that you are in touch with them.

I don't know what output is AR0143AT providing. 'AR0143AT Overview' mentions parameter color 'RGB'. Lines below from your test run + picture are interesting. There is a string UYVY which is something that max9286_mipi driver sets (max9286_data[0].pix.pixelformat = V4L2_PIX_FMT_UYVY). But AR0143AT do not mentions UYVY at all, so most likely it do not provide support for this format. And also 'ERROR: v4l2 capture: width or height too small.' maybe means that resolutions is not set correctly. 'unsupported ioctrl!' may happen because same feature or function call is not implemented in driver or upper layers. It should not be critical.

ERROR: v4l2 capture: width or height too small.
ERROR: v4l2 capture: unsupported ioctrl!
ERROR: v4l2 capture: unsupported ioctrl!
UYVY

The attached picture (GREEN+PINK) might be a result of invalid input format (in reality it is RGB but the driver is reporting UYVY). But correcting input format is just the begging. The bits per pixel, resolutions and data rate mode must be set correctly as well. I don't know details about it.