Why is the video frame not smooth when using Raspberry Pi OpenCV?

[droppgs]

hi!

I am writing because there is something I do not understand in the process of using raspberry.

Before using Webos, I am trying to stream video using a webcam camera on a Raspberry Pi.

I tried to test it first using the Raspbian operating system, but the video frame was too slow to find the problem.

The first problem I thought was that I was using an SD card with a slow reading speed. So I replaced it with an SD card with a very fast reading speed, but the video frame is too slow.

I thought it was a problem with the webcam camera, so I connected the webcam to my laptop and tested it. The video output from the webcam is very smooth.

Is it slow to output video using a webcam on a Raspberry Pi?

I am asking if there is any way to solve it. help..

And the resolution of the webcam is 1980 x 1020.

The code was tested with the OpenCV example.

'''python code

import cv2

webcam = cv2.VideoCapture(0) # Score 0 is a laptop webcam, and the webcam number can be selected by looking at the connection number.

if not webcam.isOpened():
print("Could not open webcam")
exit()

while webcam.isOpened():
status, frame = webcam.read()

if status:
    cv2.imshow("test", frame)

if cv2.waitKey(1) & 0xFF == ord('q'):
    break

webcam.release()
cv2.destroyAllWindows

'''

[ChangyongAhn]

Long time no see ~

I don't know what's your whole Environment,

but i'm sure that RPI4 Board has enough performence for video streaming.

And! about 6 month ago, i had some project using webcam.

if you are just testing streaming, with "VNC", it might be Wi-Fi network issue or just VNC has low FPS.

give me an hour. ill check.

If you want Faster Storage, use USB booting with USB 3.0 supported.
(Micro SD Card Slot has limited speed) and you can use sata to usb adaptor.

like this.

[ChangyongAhn]

i tested following codes.

i used display using HDMI port.

import cv2

webcam = cv2.VideoCapture(0) 
webcam.set(cv2.CAP_PROP_FRAME_WIDTH,1920)
webcam.set(cv2.CAP_PROP_FRAME_HEIGHT,1080)
if not webcam.isOpened():
    print("Could not open webcam")
    exit()

while webcam.isOpened():
    status, frame = webcam.read()
    if status:
        cv2.imshow("test", frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

webcam.release()
cv2.destroyAllWindows

and 1920*1080 had low FPS.

1280*720 was okay... (Maybe FPS had doubled)

RPI4 's video performance is horrible.

[droppgs]

Sorry for checking late!

As a result, it was a problem with the webcam codec.

If you change the codec to a supported webcam, it works well.

[droppgs]

There are several ways to smooth the frame...

1. GPU usage
2. Lightening the model
3. Reduce image size
4. Stop Streaming

I've tried 2 and 3 here. But still opencv with webcam is slow. No 4 times. The reason is real-time streaming.

Then I have to use 'no.1' to solve the problem.

When I researched the same data as number 1...There was a Jetson Nano™ developer kit [B01 version], did you have any experience with it?

[ChangyongAhn]

I don't have, because webOS doesn't support Jetson Nano now.

but i know a guy (my friend) who had project developing autonomous car with dual Jetson Nano boards. And he made device driver for IMU sensor himself.

I helped his project just a little,

IF YOU ARE PLANNING REAL TIME AI Model processing with NANO board, and expect that board can solve the problem,

I don't recommend for several reason.

1. Jetson Nano™ board has GPU cores based on Maxwell architecture, which is used in GTX 7XX & 9XXs..

2. Plus, Jetson Nano designed to use low power. (~20W).

How about streaming with websocket and using GPU server?

Or, using high performance mobile device could be a solution. ( Smart Phones)

[ChangyongAhn]

I've used well-tuned sample TFLite models with raspberrypi 4.

GitHub -...

But, when i use webcam, the maximum FPS was about ~10. (maybe input frame was 720p).

If you can make code using multi-threading, how about just processing a few frames per a sec, not whole frames? You can still display all frames.

Or, if you can, try make algorithm to cropping sub frames images for processing. because objects have a tendency to move continuously.

it might be a hard task. Cuz that idea occurred to me just now.
(i dont know if it works well or not.)

• check these articles.

https://pytorch.org/tutorials/intermediate/realtime_rpi.html

https://pyimagesearch.com/2017/10/09/optimizing-opencv-on-the-raspberry-pi/

https://da2so.tistory.com/10?category=1044817

[droppgs]

oh my god! Thank you so mach.

It's an excuse, but the assignment was until yesterday,

So I'm only replying now.

Even if it's 10 fps, I'll give it a try. I'll try again tomorrow.

진짜... 감사합니다 ㅠㅠ

수요일에 늦은 종강을 했지만 과제 마감이 어제까지 였습니다ㅠㅠ

그래서 메일을 지금 확인 했네여. 10프레임이어도 한번 해봐야 겠네여 ㅎㅎ 주신 자료 참고해서 내일 한번 해볼게여!

다시 한번 감사합니다!

[ChangyongAhn]

I found a project that using AI features.
check this video!

ROS2 on webOS: Web-app enabled robots (with narrative)

[droppgs]

Oh! Thanks for finding the material.

I am trying to proceed with Jetson nano 4GB now, but a problem occurred while installing OpenCV.

In the process of installing OpenCV, a repetitive error occurs in the process of specifying the CMAKE option and executing make -j4.

I went through various solutions as follows, but Opencv does not install properly.

1. Install Ubuntu18.04, Jetpack4.6.1, OpenCV4.5.0 - Failed

2. Jetpack 4.5.1 installed and challenged - failed

3. In the process of cmake, it is not properly installed even if you follow the instructions on the 'Q-Engineering' site.

I don't know what's wrong.

진짜 매번 챙겨주셔서 감사합니다..ㅎㅎ 혹시 제슨나노 빌드에러가 발생해서 일주일째 에러 못잡고 있는데 OpenCV를 설치하는 과정에서 CMAKE 옵션을 지정하고 make -j4를 하는 과정에서 반복적인 에러가 발생합니다.

다음과 같이 여러가지 솔루션을 진행했는데 Opencv가 제대로 설치가 안됩니다.

1. Ubuntu18.04, Jetpack4.6.1, OpenCV4.5.0 설치 -실패 ㅠㅠ

2. Jetpack4.5.1설치해서 도전 -실패 ㅠㅠ

3. cmake하는 과정에서 'Q-엔지니어링' 사이트의 명령어를 따라해도 제대로 설치가 안됩니다..

무엇이 문제인지 모르겠습니다. 젯슨나노 사용하기 어렵네요..ㅎㅎ ㅠㅠ

[ChangyongAhn]

I want to help you.. but ... I don't know either...

[droppgs]

I solved that problem.

Thank you for answering each question.

Have a good weekend!