This time OpenCV was ported to the Apple iPhone platform.
First of all we need to compile OpenCV library itself so that it can be used on the iPhone. There are two ways here:
1. Use OpenCV as a private framework.
2. Compile OpenCV as a static library.
First approach looks more comfortable for using, though I was not able to make it work properly on the iPhone (it works fine on the simulator, but not on the real hardware).
But anyway, let’s see how both approaches can be followed.
1. Private framework
Instructions on how to build universal OpenCV framework for simulator and iPhone (this will support i686 and ARM) can be found here.
To add this framework to your application do following:
1. Create new application in the Xcode.
2. Right-click framework group and select “Add -> Existing Frameworks”
3. Select OpenCV.framework folder you have created.
4. In the Xcode menu select “Project -> New Build Phase -> New Copy Files Build Phase”
5. In the opened window select “Frameworks” as destination and close the window.
6. Now expand group “Targets -> your_target” and drug OpenCV.framework from the Frameworks group to the “Copy Files” group under your target.
7. Add “#import <OpenCV/OpenCV.h>” anywhere in your code (not anywhere, but… well…. you know…)
8. You will probably have to change type of your sources – change extension of the source files where you use OpenCV APIs from “.m” to “.mm”
Now you should be able to use OpenCV routines in your application. But again, for me it was working perfectly on the simulator. But on the iPhone application was crashing right after start. I’ll investigate this and add an update later.
2. Static library
This approach is less convenient in using, but it works on both simulator and hardware. So, let’s start.
1. To create static library follow these instructions.
2. Now, when you have five *.a files, to make you life easier, put libraries in the separate folder. Then walk through the sources of OpenCV (folders cv, cvaux, cxvore etc.) and copy all the header files to the separate location. Thus, you will have folder (let’s call it “OpenCV.lib”) with all *.a files and subfolder (let say “hdrs”), which contains all header files.
3. Go ahead and create new application in the XCode.
4. Add all the OpenCV header files to you project – right click “Classes” group and select “Add -> Existing Files” and double-click “/…/OpenCV.lib/hdrs” folder you created on step 2.
5. Somewhere in your code include files cv.h, ml.h and highgui.h.
6. Now double click your target (under “Targets” group) and go to the “Build” tab.
7. In the “Linking” section find option “Other Linker Flags” and add paths to your OpenCV library. This field should look like this: “/…/OpenCV.lib/libcv.a /…/OpenCV.lib/libcvaux.a ” and so on.
8. Ok, you are now ready to go!
9. No, stop. Don’t forget to add libstdc++ library to you project. Other wise you’ll face compilation issues.
10. Well now you are ready.
Few useful notes.
1. OpenCV works with IplImage, while your application will require UIImage for displaying. To convert from IplImage to UIImage you can use following function (thanks to this guy for the function):
Then, use CGimage you got to create UIImage which can be displayed to the user.
2. If you will try to load image using cvLoadImage on the iPhone, process it with OpenCV (e.g. try to find faces there) and then display it, all you will see is a black rectangle with some color junk at the beginning (by the way face detection will not work on this picture – 0 object will be found). This is because cvLoadImage does not work properly on the iPhone hardware for some reasons (though, as usual, everything is fine on the simulator). To cure this try to open image using APIs from iPhone SDK and then convert it to the IplImage (boris, thanks again):
3. If you are going to change OpenCV itself, adding option “–enable-debug” might be useful for debugging. But note, that this will reduce performance (it might work up to 1.5 times slower). Also, it worth adding it anyway, since if you will enter some OpenCV API in the XCode debugger it might freeze or crash application. Also, if you are using OpenCV as a static library, make sure that all OpenCV headers, which are added to you project, are up-to-date. Otherwise, application might not work properly.
And now the sad part…
Performance of face detection on the iPhone is painful. Processing of VGA image (640×480) with three faces on it takes 6 to 20 seconds (depends on cvHaarDetectObjects parameters). 320×240 image is a bit faster, but still slow – 1-6 seconds.
Okay, that’s all, folks.