Ever wondered how netflix always comes up with a such an awesome recommendation field for us ? Well lets start with the basic in case if you don’t know what netflix is , According to their website
“Netflix is a streaming service that offers a wide variety of award-winning TV shows, movies, anime, documentaries and more — on thousands of internet-connected devices. You can watch as much as you want, whenever you want, without a single ad — all for one low monthly price”
So in short words netflix is an over-the-top media service where people can watch movies and other shows based on monthly pricing. The reason why i got hooked on to this topic is when i was browsing through netflix few days back i noticed something I’ve never noticed before my two profiles recommended same movies (which usually happens a lot) but the thing that amused me was both accounts have different thumbnails for same movies. Profile 1 had movie a with thumbnail representing romance while profile 2 had same movie representing action-thriller genre it was something similar to the picture given below
This thing was struck in my head ever since so i decided to dig up about it in internet to find the truth and turns out its “Wizardry” done by the application of artificial intelligence. Artificial intelligence is leverages computers and machines to mimic the problem-solving and decision-making capabilities of the human mind, Suppose your best friend wants you to recommend a movie for him the first thing we’ll recommend is movies that apt to his favorite genre or movies of his favorite actor/actress because that’s what our intelligence will make us do. The exact same thing is happening in netflix as well by using the AI technology it will keep track of types of movies/shows we’re watching or actors who are involved in that shows and from this dataset It’ll show us movies and shows which are related to these parameters. This is brief working of the recommendation system in netflix.
Now coming back to our main topic which is personalized thumbnail generation in netflix it also uses the similar strategy which recommendation system uses. Here in my case my profile 1 showed movie A with romantic thumbnail because most of my shows/movies which i watched recently from that profile are mostly romantic movies hence it literally wanted me to catch my attention to this movie by putting a romantic themed thumbnail now in the case of profile 2 which my friend uses he is mostly into action-thriller movies so since this recommended movie is blend of romantic action-thriller and he watches mostly action-thriller movies netflix showed action-thriller themed thumbnail for the second profile. It was really fascinating thing to be honest how they uses advance technologies for such a minor details below are the few examples of the personalized thumbnail generation of netflix using AI
Netflix uses computer vision to pick the correct image for the specifically themed thumbnail the the above image showing will smith showing using the help of computer vision it’s scanning various scenes from a movie to find a suitable image to create thumbnail for that movie. computer vision plays a huge role in netflix as well They’ve build a technology called “Archer” to make this work. According to netflix sources
Archer is an easy to use MapReduce style platform for media processing that uses containers so that users can bring their OS-level dependencies.
If you want to deep dive into usage of archer refer the link in end of this article for detailed info about that technology now simplifying things again below image shows examples of personalized artworks/thumbnails according to user interest.
In the first picture Profile 1 watches romantic movies frequently so the movie “Good will hunting” thumbnail was generated in romantic theme now coming to bottom side the profile 2 watches comedic movies frequently so his thumbnail has been generated featuring comedy actor Robin Williams indicating it could possibly be a comedy movie this is exactly what happened with my experience which i stated earlier
Other than the personalized thumbnails they’re also using AI for various other functions some of them are given below
Optimal Streaming Quality
When you sit at home, your broadband speed varies so and what they don’t want to do is they want don’t to lose quality. So, where there are no predicting speeds at different times and this will then automatically be used by their machine learning algorithms to determine what films people might want to watch and whether they want to cash them in a regional server to make it then faster to download to your house or not. Again, they’re monitoring the quality and what is impacted higher so they can automatically scale down or scale up the quality of a movie streaming to your house.
Pre Production
Surprised aren’t you ? till now we’ve only talked about how they uses AI to their client side easier so how about using AI in production side ? They use it for pre-production especially around finding a location to shoot a movie in, for example, so artificial intelligence tools will look at things like the actors their availability. Where they’ve located the camera teams their availability, and where they’re located, and all of this information is now used to identify the best places. And again, they will look at what sort of scenes are required, and then those AI scan now recommend areas, cities, and places where you might want to shoot a movie
Post Production
So the editing process this was done pretty manually in the past. Nowadays, the editing is still done manually, but then the quality checks are driven by AI so the AI will look at where do we need some human quality control when have and when in the past have there been mistakes? Maybe synching a subtitle to a certain scene? If this was a problem, AI would flag this up, so again, this has made this whole post-production process a lot more efficient and effective
The above are the few areas where netflix uses artificial intelligence to improve their product from minute details like thumbnail personalization which plays a huge role to pre and post production for a movie/show.
“75 percent of users select movies based on the company’s recommendations, and Netflix wants to make that number even higher.” — Netflix’s Senior Data Scientist, Mohammad Sabah
Netflix’s senior data scientist Mr. Mohammed Sabah stated the above in 2014 which was 7 years ago and comparing the AI techniques from 7 before and now we know how much that field of technology improved over time so the numbers are much higher than 75% now for sure
Today, Netflix is undoubtedly the epitome of content recommendation and personalized user experience and as shown above from 5 years from now on the number games will go even further and user experience will keep getting improved by usage of AI maybe who knows if they’ll generate an entire movie using AI without actually having any actors physically taking part in near future ? Only the time will tell !