Using the GStreamer SDK in openFrameworks under OSX 10.8 Mountain Lion

Using the GStreamer SDK in openFrameworks under OSX 10.8 Mountain Lion

It looks like developing streaming audio/video applications in OSX 10.8 Mountain Lion has just gotten easier. The GStreamer website has gotten a fresh new look at http://www.gstreamer.com and there is a new precompiled application framework for use in developing OSX applications. This framework is a universal binary version meaning it can be used under both the i386 and x86_64 architectures. Good news is that openFrameworks application using it can be compiled under Mountain Lion 10.8.

Using the framework is pretty straight forward, yet there are a couple of tricks to get it working 100% which I will go over. I will be using the videoPlayerExample to help me describe the process and to show how to use the GStreamer framework below.

Download and Install the GStreamer Framework

  1. Goto http://docs.gstreamer.com/display/GstSDK/Installing+on+Mac+OS+X to download the GStreamer files.
  2. There are three (3) files that need to be downloaded. Find them under the section Download and Install the SDK.
    • Runtime
    • Development files
    • Deployment Packages
  3. They will be .dmg files which once clicked on will mount as drives. Go into each drive that is mounted and click on/install the packages inside.
  4. To be safe, install all packages inside each mounted drive.
  5. This will create a framework in the /Library/Frameworks folder named GStreamer.framework
  6. This framework will be the one to include in your openFrameworks/application projects.

Download openFrameworks

  1. Goto http://www.openframeworks.cc/download/ and download the OSX openFrameworks package. The version at the time of this writing was 0073.

Modify the ofConstants.h file:

From the openFrameworks directory goto the directory /libs/openFrameworks/utils/ and open the file ofConstants.h for editing.

  1. Search for “OF_VIDEO_CAPTURE_GSTREAMER” without the quotes. It should be on or around line 202.
  2. Find the section with the following:
    #elif defined(TARGET_OSX)
    //on 10.6 and below we can use the old grabber
    #ifndef MAC_OS_X_VERSION_10_7
    #define OF_VIDEO_CAPTURE_QUICKTIME
    #else
    #define OF_VIDEO_CAPTURE_QTKIT
    #endif
  3. Modify this section as follows: (Add OF_VIDEO_CAPTURE_GSTREAMER)
    #elif defined(TARGET_OSX)
    //on 10.6 and below we can use the old grabber
    #ifndef MAC_OS_X_VERSION_10_7
    #define OF_VIDEO_CAPTURE_QUICKTIME
    #else
    //#define OF_VIDEO_CAPTURE_QTKIT
    #define OF_VIDEO_CAPTURE_GSTREAMER
    #endif
  1. Search for “OF_VIDEO_PLAYER_GSTREAMER” without the quotes. It should be on or around line 244.
  2. Find the section with the following:
    #elif defined(TARGET_OSX)
    //for 10.7 and 10.8 users we use QTKit for 10.6 users we use QuickTime
    #ifndef MAC_OS_X_VERSION_10_7
    #define OF_VIDEO_PLAYER_QUICKTIME
    #else
    #define OF_VIDEO_PLAYER_GSTREAMER
    #endif
  3. Modify this section as follows: (Add OF_VIDEO_PLAYER_GSTREAMER)
    #elif defined(TARGET_OSX)
    //for 10.7 and 10.8 users we use QTKit for 10.6 users we use QuickTime
    #ifndef MAC_OS_X_VERSION_10_7
    #define OF_VIDEO_PLAYER_QUICKTIME
    #else
    //#define OF_VIDEO_PLAYER_QTKIT
    #define OF_VIDEO_PLAYER_GSTREAMER
    #endif

This will enable the GStreamer headers in openFrameworks to work under OSX 10.8 Mountain Lion. If you want to use GStreamer under OSX 10.6 modify that section accordingly to enable GStreamer.

Modify the ofGstVideoGrabber.cpp file:

  1. From the openFrameworks directory goto the directory /libs/openFrameworks/video/ and open the file ofGstVideoGrabber.cpp for editing.
  2. Add #include <gst/gst.h> right above #ifdef TARGET_LINUX. This should be on or around line 17

Include the GStreamer framework in your XCode project:

Add the GStreamer.framework located at /Library/Frameworks to your project. The videoPlayerExample project is being used here.

 

Include the GStreamer Framework

Add the openFrameworks GStreamer header files to your XCode project:

By default for the OSX distribution version of openFrameworks GStreamer is not enabled. So the header files are not included in the example XCode project files. To fix this, add the openFrameworks GStreamer header files to the openFrameworks project within your XCode project.

 

Add the GStreamer header files to your project.

Update the project build setting:

There are two build settings that need to be modified in both the application project and the openFrameworks project. The Header Search Path and the Frameworks Search Path. Don’t know why but for some reason the user frameworks path /Library/Frameworks is not defined and causes problems when compiling with GStreamer. So it needs to be defined…

Frameworks Search Path: /Library/Frameworks

Header Search Path: /Library/Frameworks/GStreamer.framework/Headers

Application Project Build Settings:

 

Applications Project Build Settings

openFrameworks Project Build Settings:

 

openFrameworks Project Build Settings

Using GStreamer in openFrameworks:

After all of that you should be setup to now start using the GStreamer features. If you were following along using the videoPlayerExample, you should be able to compile the project and openFrameworks will now be using GStreamer to play the video fingers video file.

To go further I will demonstrate how to play an rtsp:// stream using this same videoPlayerExample project.

  1. Find an rtsp:// test stream at http://wiki.multimedia.cx/index.php?title=RTSP
  2. Modify the testApp.cpp file
    <code>void testApp::setup(){
    ofBackground(255,255,255);frameByframe = false;// Uncomment this to show movies with alpha channels
    // fingerMovie.setPixelFormat(OF_PIXELS_RGBA);
    fingerMovie.loadMovie("rtsp://media.lscube.org/tests/tc.mov");
    fingerMovie.play();
    }
    void testApp::draw(){
    ofSetHexColor(0xFFFFFF);

    fingerMovie.draw(20,20);
    }

This will play the rtsp:// stream that you specified in setup()….

Static OpenCV 2.4.3 Libraries with (FFMPEG, x264, and CUDA) on OSX 10.8

Static OpenCV 2.4.3 Libraries with (FFMPEG, x264, and CUDA) on OSX 10.8

Ok, so this guide will show you how to compile OpenCV as a x86_64 (64 bit) Static Library with FFMPEG (x264, Xvid, Ogg, Vorbis, Lame, and FAAC) and CUDA enabled on Apple OSX 10.8 Mountain Lion. This is very useful for using the latest OpenCV with openFrameworks.

1. First you want to download and decompress all of the dependent tools and libraries into a designated folder. Using CUDA is optional, so don’t install it if you won’t be using it.

Dependencies

Tools:

  1. CMAKE  – http://www.cmake.org/cmake/resources/software.html
  2. pkg-config- http://pkg-config.freedesktop.org/releases/
  3. Yasm- http://yasm.tortall.net/Download.html

Libraries:

  1. FAAC – http://www.audiocoding.com/downloads.html
  2. libogg- http://xiph.org/downloads/
  3. libvorbis- http://xiph.org/downloads/
  4. LAME- http://sourceforge.net/projects/lame/files/lame/
  5. XvidCore- http://www.xvid.org/Downloads.43.0.html
  6. x264- http://www.videolan.org/developers/x264.html
  7. CUDA- http://developer.nvidia.com/cuda-downloads

Core:

  1. FFMPEG- http://ffmpeg.org/download.html
  2. OpenCV- http://opencv.org/downloads.html

2. Install CMAKE and CUDA if you will be using it.

3. Now create a folder where you have downloaded all of the tools and libraries called “opt”. This is where all of your header files and compiled libraries will be copied to.

4. Configure and compile all of the downloaded tools, libraries, and core software. I will be using [DOWNLOAD_FOLDER] as the folder location where all of the downloaded files were saved. You will notice that –prefix=[DOWNLOAD_FOLDER]/opt is added to almost all of the configure commands, this is to tell each one where to install the libraries after compiling is finished.

Configuration and Compiling

FAAC:

cd faac-*/
./configure --enable-static --enable-shared CFLAGS="-arch x86_64" LDFLAGS="-arch x86_64" --prefix=[DOWNLOAD_FOLDER]/opt
make -j4 &amp;&amp; sudo make install

LAME:

cd lame-*/
./configure --disable-dependency-tracking --enable-static --enable-shared CFLAGS="-arch i386 -arch x86_64" LDFLAGS="-arch i386 -arch x86_64" --prefix=[DOWNLOAD_FOLDER]/opt
make -j4 &amp;&amp; sudo make install

LIBOGG:

cd libogg-*/
./configure --disable-dependency-tracking --enable-static --enable-shared CFLAGS="-arch i386 -arch x86_64" LDFLAGS="-arch i386 -arch x86_64" --prefix=[DOWNLOAD_FOLDER]/opt
make -j4 &amp;&amp; sudo make install

PKG-CONFIG:

cd pkg-config-*/
./configure
make -j4 &amp;&amp; sudo make install

LIBVORBIS:

cd libvorbis-*/
./configure --disable-dependency-tracking --enable-static --enable-shared --with-pic CFLAGS="-arch i386 -arch x86_64" LDFLAGS="-arch i386 -arch x86_64" --prefix=[DOWNLOAD_FOLDER]/opt --with-ogg-libraries=[DOWNLOAD_FOLDER]/opt/libogg.a/ --with-ogg-includes=[DOWNLOAD_FOLDER]/opt/include/ogg --build=x86_64

YASM:

cd yasm-*/
./configure &amp;&amp; make -j4 &amp;&amp; sudo make install

X264:

cd x264-*/
./configure --enable-static --enable-shared --enable-pic CFLAGS="-arch i386 -arch x86_64" LDFLAGS="-arch i386 -arch x86_64" --prefix=[DOWNLOAD_FOLDER]/opt
make -j4 &amp;&amp; sudo make install

XVID CORE:

cd xvidcore/build/generic
./configure --disable-assembly --enable-static --enable-shared --enable-pic CFLAGS="-arch i386 -arch x86_64" LDFLAGS="-arch i386 -arch x86_64" --prefix=[DOWNLOAD_FOLDER]/opt

FFMPEG:

cd ffmpeg-*/
./configure --disable-debug --disable-ffplay --disable-ffserver --enable-pthreads --enable-pic --enable-static --enable-libmp3lame --enable-libfaac --enable-libx264 --enable-libvorbis  --enable-nonfree --enable-gpl --enable-libxvid --arch=x86_64 --prefix=[DOWNLOAD_FOLDER]/opt --extra-ldflags="-L[DOWNLOAD_FOLDER]/opt/lib" --extra-cflags="-I[DOWNLOAD_FOLDER]/opt/include" --cc=clang
make -j4 &amp;&amp; sudo make install

OPENCV:

Patch CUDA DETECT CMAKE File

Edit File: OpenCV-*/cmake/OpenCVDetectCUDA.cmake

Comment out lines 6-9…

#if (NOT MSVC AND NOT CMAKE_COMPILER_IS_GNUCXX OR MINGW)
#  message(STATUS "CUDA compilation was disabled (due to unsuppoted host compiler).")
#  return()
#endif()

Patch MODULES CMAKE File

Edit File: OpenCV-*/modules/highgui/CMakeLists.txt

Add the following to the “CMakeLists.txt” file on line ~152, above “if(HAVE_FFMPEG)”…

if(APPLE)
list(APPEND HIGHGUI_LIBRARIES ${BZIP2_LIBRARIES} -L[DOWNLOAD_FOLDER]/opt/lib -lmp3lame -logg -lvorbis -lvorbisenc -lfaac -lx264 -lxvidcore -lbz2)
endif(APPLE)

Compile OpenCV

cd OpenCV-*/
mkdir release
cd release
cmake -DCMAKE_INSTALL_PREFIX=[DOWNLOAD_FOLDER]/opt -DFFMPEG_INCLUDE_DIR=[DOWNLOAD_FOLDER]/opt/include -DFFMPEG_LIB_DIR=[DOWNLOAD_FOLDER]/opt/lib -DBUILD_SHARED_LIBS=NO -DCMAKE_EXE_LINKER_FLAGS="-L[DOWNLOAD_FOLDER]/opt/lib -lmp3lame -lvorbis -lfaac -lx264 -lxvidcore"-DCMAKE_SHARED_LINKER_FLAGS="-L[DOWNLOAD_FOLDER]/opt/lib -lmp3lame -lvorbis -lfaac -lx264 -lxvidcore" ..
make -j4 &amp;&amp; sudo make install

5. Copy 3rd party libraries from OpenCV to your library folder.

release/3rdparty/lib  =>  [DOWNLOAD_FOLDER]/opt/lib

6. Compile a program with the OpenCV libraries. By this point you should have a whole bunch of files in the “opt” folder.

To Compile a Static Binary

WITHOUT CUDA:

g++ -o freak freak_demo.cpp -I../opt/include/ -L../opt/lib/ -lopencv_core -lopencv_highgui -lopencv_features2d -lopencv_imgproc -lopencv_legacy -lopencv_flann -lopencv_nonfree -lopencv_gpu ../opt/lib/libpng.a ../opt/lib/libjpeg.a ../opt/lib/libtiff.a ../opt/lib/libjasper.a -lzlib -lIlmImf -framework Cocoa
g++ -o retina retinaDemo.cpp -I../opt/include/ -L../opt/lib/ -lopencv_core -lopencv_highgui -lopencv_features2d -lopencv_imgproc -lopencv_legacy -lopencv_flann -lopencv_nonfree -lopencv_gpu ../opt/lib/libpng.a ../opt/lib/libjpeg.a ../opt/lib/libtiff.a ../opt/lib/libjasper.a -lzlib -lIlmImf -framework Cocoa ../opt/lib/libswscale.a ../opt/lib/libavformat.a ../opt/lib/libavcodec.a ../opt/lib/libx264.a ../opt/lib/libxvidcore.a ../opt/lib/libvorbis.a ../opt/lib/libvorbisenc.a ../opt/lib/libogg.a ../opt/lib/libmp3lame.a ../opt/lib/libfaac.a -framework QuartzCore -framework VideoDecodeAcceleration ../opt/lib/libIlmImf.a ../opt/lib/libavfilter.a ../opt/lib/libavutil.a ../opt/lib/libavfilter.a -framework QTKit -lopencv_contrib -lbz2

WITH CUDA:

LDFLAGS=-static g++ -o retina retinaDemo.cpp -I../opt/include/ -L../opt/lib/ -lopencv_core -lopencv_highgui -lopencv_features2d -lopencv_imgproc -lopencv_legacy -lopencv_flann -lopencv_nonfree -lopencv_gpu ../opt/lib/libpng.a ../opt/lib/libjpeg.a ../opt/lib/libtiff.a ../opt/lib/libjasper.a -lzlib -lIlmImf -framework Cocoa ../opt/lib/libswscale.a ../opt/lib/libavformat.a ../opt/lib/libavcodec.a ../opt/lib/libx264.a ../opt/lib/libxvidcore.a ../opt/lib/libvorbis.a ../opt/lib/libvorbisenc.a ../opt/lib/libogg.a ../opt/lib/libmp3lame.a ../opt/lib/libfaac.a -framework QuartzCore -framework VideoDecodeAcceleration ../opt/lib/libIlmImf.a ../opt/lib/libavfilter.a ../opt/lib/libavutil.a ../opt/lib/libavfilter.a -framework QTKit -lopencv_contrib -lbz2 -L/usr/local/cuda/lib -lcuda -lcublas -lcudart -lcufft -lcurand -lnpp

Install Script

The following install script is based on the script from Hunter Ford – http://hunterford.me/compiling-ffmpeg-on-mac-os-x/ and information from http://tech.enekochan.com/2012/07/27/install-opencv-2-4-2-with-ffmpeg-support-in-mac-os-x-10-8/. This will download and install all of the necessary libraries  to a folder “/opt” within the directory that you execute the script from.

InstallOpenCV_2.4.3

For reference there is also another script that looks useful at https://github.com/enekochan/installation-scripts/blob/master/installOpenCV-2.4.2-ffmpeg-OSX-ML.sh