Raspberry Pi camera board – Gstreamer

So, my Raspberry Pi camera board has arrived and I have started playing with it.

Raspberry Pi camera board

My first impressions were tiny, super cheap camera. Its low light capabilities are not great but I can live with that. The important bit is the quality, full 1080p at 25 frames per second (UK). That on its own is pritty awesome.

I was though only really interested in getting the pi to work remotely for my robot to replace the 3-4 fps standard def USB webcam I currently use. Currently though with the camera, there is no V4L driver so we have to make do with their provided applications then pipe it out to streaming applications. Their recommended method is slow and very laggy (netcat and mplayer with wifi provides 3-6 seconds of lag). So I had to find another option. I happened to be sitting on the #raspberrypi IRC on freenode and heard a user having success with gstreamer. So this is how to set up gstreamer to stream HD video with less than 0.5 seconds of lag.

First we need to add a repositary with gstreamer1.0

 

sudo nano /etc/apt/sources.list

 

and add to the end

 

deb http://vontaene.de/raspbian-updates/ . main

 

Then do an sudo apt-get update

next grab gstreamer

 

sudo apt-get install gstreamer1.0

 

 

On your recieving end you will also need gstreamer. Because I mainly use a mac, I decided to get it working on the mac so with help from arcanescu on IRC, we figured out how to get it working on mac os (10.8)

The simplest way is with brew, a package manager like apt-get, but for mac os. To install it run this in terminal on your mac simply run

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

It will grab and install brew. Now update it with

 

brew update 

 

Now we need to grab gstreamer

 

 brew install gstreamer gst-libav gst-plugins-ugly gst-plugins-base gst-plugins-bad gst-plugins-good 

 

Once that installs you should be good to go. Enter

 

 raspivid -t 999999 -h 720 -w 1080 -fps 25 -hf -b 2000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse !  rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=YOUR-PI-IP-ADDRESS port=5000 

 

on your raspberry pi and enter on your mac

 

 gst-launch-1.0 -v tcpclientsrc host=YOUR-PI-IP-ADDRESS port=5000  ! gdpdepay !  rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false 

 

The picture quality is good but I see a number of flickers which isnt a problem for me, if it is for you, maybe try adjusting the resolution

 

So thanks again to arcanescu, I take no credit for this as it was him that came up with this, if you ever see him in IRC, give him a virtual pat on the back

His blog garagedeveloper.wordpress.com

51 thoughts on “Raspberry Pi camera board – Gstreamer”

  1. I tried to use sudo apt-get install gstreamer1.0 but that does not exist . Did you add another repo somewhere?

    Thanks for a nice article

  2. Hi

    On the receiving end is there a gstreamer switch that will allow me to record the video stream? I can record on the sending end by piping raspvid through tee then to gstreamer but it would be very useful for my application to record the video at the receiver.

    Thanks in advance and thanks for the great instructions.

  3. Thanks for the excellent post. I got it to work as described and now I want to learn more about gstreamer…can you recommend any good tutorials for building gstreamer pipelines?
    Many of the search results on-line point to resources for building an SDK.
    Like David M. above, I’m looking to ‘tee’ the video to the screen and a filesink.

    1. I am no expert on Gstreamer so the wrong person to ask. arcanescu knows what he is doing so you would be best to ask him via his blog or pop on the raspberrypi IRC channel (freenode) and hit him up there.

  4. Pingback: Anonymous
  5. Thanks for this helpful article! I’m a bit of a n00b, so when you say ‘add … to the end’, do you mean when you go into the nano editor or before you do? the whole thing didnt work for me because I cant get past this basic hurdle!;)

  6. Hi there,

    I’m a newbie so bear with me… I did exactly as described. The streamer seems to run on the Pi but I get a message on my MAC: WARNING: erroneous pipeline: no element “rtph264depay”

    What am I doing wrong?

    Michael

  7. Hi, really nice tutorial thank you!

    I managed to compile both on my mac and rasp pi, but when i start gstreamer on rasp pi camera led is on but it stucked on |Pipeline is prerolling”

    1. Not original but true: Thanks
      I managed to connect my laptop with Debian 7.1.0 to the Raspberry Pi/Raspbian using the Netcat connection. It woks fine but with the known delay.
      I am trying to reproduce your experiment and seems everything goes well on the Raspberry side. However on my laptop it says:

      Estableciendo el conducto a PAUSA …
      No protocol specified
      No protocol specified
      No protocol specified
      libEGL warning: DRI2: xcb_connect failed
      No protocol specified
      libEGL warning: DRI2: xcb_connect failed
      libEGL warning: GLX: failed to load GLX
      No protocol specified
      No protocol specified
      ERROR: El conducto no quiere pausarse.
      ERROR: del elemento /GstXvImageSink:autovideosink0-actual-sink-xvimage: Could not initialise Xv output
      Información adicional de depuración:
      xvimagesink.c(1291): gst_xvimagesink_xcontext_get (): /GstXvImageSink:autovideosink0-actual-sink-xvimage:
      Could not open display
      Estableciendo el conducto a NULL …
      Liberando la tubería…

      Half in Spanish Half in English. The problems in Spanish are about the pipeline trying to go to PAUSE but not managed. So at the end it set it to NULL and release the pipe.

      Please do you have any idea what could happen?
      Thanks again / Borja

  8. I would like to have several raspberry pis with each pi streaming video from its pi camera (just like in the above blog post) over wifi. Then a different pi receiving and displaying one of the streams and being able to switch between streams. The display pi could ssh into the camera pi that it wants and execute the above streamer (that’s one simple way to switch). But what would be the corresponding decoder and display that matches the above encoder? Just doing the above mac command on the display pi doesn’t seem to work. I have looked around for a “camera on one pi, display on another pi” example, but all I have found is pi-to-home-computer streaming. Side note: I will probably be writing a touch panel interface to select the stream.

    I am hoping that the one-pi-to-another-pi setup is a fairly easy question…

  9. hi, i have an error saying

    WARNING: erroneous pipeline: no element “avdec_h264”
    on my rip

    and
    ERROR: pipeline doesn’t want to preroll.
    Setting pipeline to NULL …
    Freeing pipeline …

    what is the issue? thanks

  10. Hi, regarding the flicker issue, I had that for a while, but as is common with the Pi it turned out to be a power problem. Not only do you need a good supply, but the USB cable is very often too thin in cheap cables; 28 gauge will cause flicker, and ethernet might stop working. 22 gauge should work, or just try different cheap cables and power supplies until it goes away.

  11. What kind of a stream does this produce ? I tried viewing it with VLC on a PC as rtsp and http, but VLC could not display it.

    It’s it’s some kind of a special stream, how can I produce one that VLC understands ?

  12. Hey,
    Thanks for the tutorial! Very helpful but I have a question. For some reason it runs SUPER slow the third time that I’ve run it. Does gstreamer record and save the files or does it just stream them?

  13. Thanks for guide! With your help I was able to get stream working and delay did not raise even that I ran camera full hd / full fps / 5.5Mbit/s.

    I experienced that sending video over UDP did reduce delay even more. I was not expecting much difference between UDP / TCP on ethernet, but it really did seem reduce delay from around 500ms to somewhere around 100-200ms that was barely noticeable anymore.

    To setup sending stream over UDP:

    On OSX start first host to be ready to listed UDP packets:


    gst-launch-1.0 -v udpsrc address=HOST_IP_ADDR port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

    Then start sending on raspi:


    raspivid -t 0 -b 5500000 -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! udpsink port=5000 host=HOST_IP_ADDR

    One thing I couldn’t get to work was being able to start receiver, after stream sending has already started… maybe someone knows gstreamer magic how to do that 🙂

    1. Adding caps to reading side of stream helped to get started even in middle of UDP stream


      gst-launch-1.0 -v udpsrc address=HOST_IP_ADDR port=5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

      For those who are interested what those caps mean here is explication: http://www.iana.org/assignments/rtp-parameters/rtp-parameters.xhtml

      Also I removed one unnecessary filter pass from streaming side:

      raspivid -t 0 -b 5500000 -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink port=5000 host=HOST_IP_ADDR

      To store UDP stream to file one can replace stream reader with


      gst-launch-1.0 -v udpsrc address=HOST_IP_ADDR port=5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! h264parse ! matroskamux ! filesink location=capture.avi

  14. Hey, thx for this great tutorial. Now i have one problem. I want to implement the video-signal into the program live-view-rift on a windows-computer. Can u help me with the pipeline parameter. I didnt fount out which are the right ones. Only to type in the ip isnt working.

  15. i got this working in Windows so thanks for that.
    however i can’t seem to find a way to play it in fullscreen. any ideas?

  16. Hello
    I have a wolfson audio card for my rev b and I would like to stream just audio to my Pi 2 using this method.. would you be able to assist me?
    Thanks

  17. If like me you were getting the error:
    “libgstreamer1.0-0 is already the newest version.”

    it’s because you don’t need to edit the file /etc/apt/sources.list. The package is already available in the Raspbian repository.

  18. When I try to edit my source list files I get the following…
    deb http://mirrordirector.raspbian.org/raspbian/ Jessie main contrib non-free r$
    #Uncomment line below then ‘apt-get update’ to enable ‘apt-get source’#deb-src http://archive.raspbian.org/rasbian/jesse main contrib non-free rpi
    I am stuck now because I don’ know how to do what they are asking me to do let alone executing the fix. I am brand new to the raspberry pi programing. I just wanted to make my camera stream over the internet. Can you help? I have no clue on how to uncomment something, not to mention even where to find what I am trying to uncomment. Thanks.

    1. Some of the stuff is a little outdated for this, but I got it working earlier today. Being new to the programming is not an issue. Just find the individual steps, then search online for how to do them.

      For example, code for installing Brew no longer works unless you have Xcode, but Xcode doesn’t work for OSX unless it’s Xcode 6.4 and things like that.

      Break down the individual parts, search online how to do each of those, and you’ll fly through this.

      Main problem people get is they have different ports for Mac and RPi, but still expect them to work.

  19. Hello I have a problem in my mac the video is inversed, the picture and the text is inversed, I want to read text with the camera what can I do for this ?
    Thanks for reply

  20. I am having a problem on the Mac side:

    brew install gstreamer gst-libav gst-plugins-ugly gst-plugins-base gst-plugins-bad gst-plugins-good

    Error: You must `brew link libpng` before gstreamer can be installed
    Error: You must `brew link libpng xz` before gst-libav can be installed
    Error: You must `brew link libpng jpeg` before gst-plugins-ugly can be installed
    Error: You must `brew link libpng` before gst-plugins-base can be installed
    Error: You must `brew link libpng jpeg` before gst-plugins-bad can be installed
    Error: You must `brew link libpng jpeg` before gst-plugins-good can be installed

  21. I’m wondering how to show gst-launch’s video to the web page in our server instead of calling the gst from the client.

    or maybe we can show it in video player from the client?

Leave a Reply

Your email address will not be published. Required fields are marked *