I originally had a simple goal: do a computer vision task solely from my browser. Having seen some very nice colour transfer effects before, I figured I could try a simple one. So I found a nice and popular algorithm: color Transfer by jrosebr1 and started analysing it.
I quickly realised that there might be a problem since it mostly relied on OpenCV which is a Python library.
The python-OpenCV library is quite impressive and allows plenty of advanced applications for minimal effort (such as image or video processing and object detection). The question was: can we do all that in the browser?
And then began my dreadful experience …
Don't expect OpenCV.js to doall the thingsyou can do with OpenCV in C / C++ / python:it is not as advanced. So web computer vision is still a pretty complex task.
- OpenCV.js does not implement the same functions as for the other languages
- There is no way to know in advance if something is going to work other than testing it and praying
The tutorials on the official website help a lot but:
- they don't cover everything
- they make us wonder if the only functions we can use are the ones that we see in the tutorial
- It feels like reverse engineering, debugging along the way and comparing if the values are the same. Spoiler alert: they are not!
- The library isn't lightweight either: approximately 7 Mo, so it will take a while to load for an end user (3 seconds at 20 MB/s)
.js file as the ones used in the python algorithm.
The expected outcome is to be able to pick one colorful picture and a black and white picture (from your browser), click a button and after a few seconds see the black and white picture colorised: computer vision!
Get OpenCV js:
Here you have a choice,
- you can be lazy (like me) and directly download the opencv.js file from this link
- or clone the repo and build it yourself: instructions here.
Set up a playground:
The first step was to set up a basic webpage to be able to pick our input images and display the output image from our wonderful color transfer algorithm.
This wasn't the main focus of this experiment so I basically took the one from the RisingStack article and customised it:
This section shows the breakdown of the computer vision algorithm I chose to translate.
1. The first step in jrosebr1's algorithm isconverting our image from the RGB space to the LAB space:
What's the LAB space you might ask? As Wikipedia puts it:
It expresses colour as three numerical values, L* for the lightness and a* and b* for the green–red and blue-yellow colour components.
I have —unsuccessfully— looked around for what this
astype('float32') does and if there is an equivalent in opencv.js but haven't found anything . Let's hope this doesn't break too much…
2. Next we want to get the image stats:
So far so good, but the tricky part is in the
Here as you can see I used
cv.meanStdDev. Why didn't I use
.std()? Simply because they don't exist in Opencv.js
So after some googling around, I found this post mentioning
cv.meanStdDev which seemed to do what I wanted.
I tested and printed the stats output for the source image and … I don't have the same values Not sure if it comes from the
But except for
bMean, the values are only off by a bit so I figured I could go on and see what the output images would look like.
3. And now we need to translate and scale all the values in the
Looks easy right?
⚠️⛔️ Well, that's because it does not work!
I looked a bit at what this LAB image was made of and this is the OpenCV matrix for the L part of the Lab plan of a picture of a sunset (the
b part are built the same with different values):
opencv.Mat object- Output of the L part of a LAB image
Without helper functions to do arithmetic operations, I don't think it's worth spending more time with this library.
My experience with computer vision in the browser using OpenCV.js has really not been great, but that doesn't mean you shouldn't use it!
There are some breath-taking demos of in browser computer vision done with OpenCV on their website such as:
So in my opinion, if what you want to do can be done in a couple openCV calls without additional manipulation, just go for it!
Look at the tutorials and hopefully, you'll find out how the functions you want to use work.
However, if you want to do more than what OpenCV offers out of the box, like manipulating pixels, I suggest you try and find another way of doing it (in a python backend for instance)
OpenCV is a powerful tool which does image manipulation and processing at a fast pace.How do I use OpenCV on my website? ›
Deploy Computer Vision Model on Webpage| OpenCV & Python ...Is OpenCV still relevant? ›
OpenCV is not only used by prominent research centres like Stanford, MIT, CMU, Cambridge, and INRIA but also by tech giants like Google, Yahoo, Microsoft, Intel, Sony, Honda, Toyota, among others. Since its alpha release in January 1999, OpenCV has been used in many applications, products, and research efforts.Can I use OpenCV with nodejs? ›
opencv4nodejs allows you to use the native OpenCV library in nodejs. Besides a synchronous API the package provides an asynchronous API, which allows you to build non-blocking and multithreaded computer vision tasks. opencv4nodejs supports OpenCV 3 and OpenCV 4.How do I use OpenCV in react JS? ›
- Open your project in Android Studio.
- Follow the tips in Android Studio for synchronizing your project.
- Download the newest version of OpenCV for Android. ...
- Import OpenCV to Android Studio – from File -> New -> Import Module, choose sdk/java folder in the unzipped OpenCV archive.
- Update build.
It has C++, Python, Java and MATLAB interfaces and supports Windows, Linux, Android and Mac OS. OpenCV leans mostly towards real-time vision applications and takes advantage of MMX and SSE instructions when available.Which is better OpenCV or Tensorflow? ›
To summarize: Tensorflow is better than OpenCV for some use cases and OpenCV is better than Tensorflow in some other use cases. Tensorflow's points of strength are in the training side. OpenCV's points of strength are in the deployment side, if you're deploying your models as part of a C++ application/API/SDK.Is OpenCV and CV2 same? ›
CV2 is OpenCV. OpenCV and PIL both have image processing tools such as: Image filters (blur, sharpen, etc.)Is OpenCV artificial intelligence? ›
OpenCV stands for Open Source Computer Vision. To put it simply, it is a library used for image processing. In fact, it is a huge open-source library used for computer vision applications, in areas powered by Artificial Intelligence or Machine Learning algorithms, and for completing tasks that need image processing.
Is OpenCV worth learning? ›
Yes! It is definitely worth it to start learning OpenCV through Python. Since Python saves you a lot of time on the declaration of variables etc, it is much easier to use it with a basic knowledge of Image Processing and Numpy.Is OpenCV used in industry? ›
OpenCV is extensively used in companies, research groups, and governmental bodies. Well-established companies like Google, Yahoo, Microsoft, Intel, IBM, Sony, Honda, Toyota employ this library. Moreover, significant startups like Applied Minds, VideoSurf, and Zeitera make extensive use of OpenCV.Is OpenCV deep learning? ›
Two weeks ago OpenCV 3.3 was officially released, bringing with it a highly improved deep learning ( dnn ) module. This module now supports a number of deep learning frameworks, including Caffe, TensorFlow, and Torch/PyTorch.Is OpenCV a good library? ›
OpenCV is the obvious answer since it's the most popular. PCL (Point Cloud Library) is also popular if you work with 3D data a lot. ROS is again used heavily in robotics, mainly dealing with robot navigation. MATLAB if you have the money is pretty good also for prototyping.What is OpenCV and why it is important? ›
OpenCV is the huge open-source library for the computer vision, machine learning, and image processing and now it plays a major role in real-time operation which is very important in today's systems. By using it, one can process images and videos to identify objects, faces, or even handwriting of a human.Is OpenCV good Reddit? ›
OpenCV is quite capable and very large, but the C-based API and years of baggage makes the modern versions and C++ quite clunky. The syntax can be a bit of a mess, and sometimes debugging errors can be arcane.