We're experts in software development and share news and articles about IoT, web, mobile, VR, AR and game development.

How to Make Augmented Reality Face Morphing App?

Thanks to the advances in all fields of Artificial Intelligence, mobile vendors are getting really good at facial recognition technology and can now build a high-quality Augmented Reality app that accurately detects face parts, handles complex facial transformations and enhances photos with Instagram-like filters. Here’s how R-Style Lab developers do it.

Neural networks & what they have to do with Augmented Reality mobile app development

Face morphing apps like Face Swap Live, Snapchat lenses and MSQRD took the world by storm last year and continue to define mobile app development trends through 2017.

What do popular face editor applications have in common? 

Obviously, they analyze a photograph, identify control points – that is, eyes, nose, lips and other face parts – and swap a user’s face for a celebrity’s or cartoon hero’s head. Some face morphing applications simply enhance user picks by adding make-up, changing hair color or performing advanced facial transformation to replicate the results of a plastic surgery.

The technology is often referred to as “facial recognition”, “face morphing” or “Augmented Reality” and is powered by artificial neural networks. Artificial neurons take data – let’s say, an image displaying a smiling person – as an input – and send it to hidden neuron layers where the picture is compared to the images processed by the network in the past. Based on the data, neural networks identify facial landmarks and create a vertex mesh enabling software to transform face parts or the entire image along X, Y or Z axes. Complex facial recognition solutions also identify basic emotions (sadness, happiness, fear) and change photos accordingly.

How to Make Augmented Reality Face Morphing App?

There are several open source APIs facilitating face morphing processes. These include Google Cloud Vision, IBM Watson Visual Recognition, Face++, Intel OpenCV, etc., so you don’t even need to build software from scratch. However, the feature set of a face recognition framework (including the transformation logic) depends on the intended functionality of your application.

A MSQRD clone is one story; and what if you want to build an Augmented Reality app that will both perform facial transformation and enhance image quality? That’s exactly what the R-Style Lab team did.

Glamozis case: going beyond facial recognition

Glamozis is a cutting-edge iPhone app development project which may give you a hint of what your target audience wants.

Our customer – a well-known Russian plastic surgeon – wanted to create a selfie editor app enabling users to see what they would look like after a plastic surgery or simply improve the quality of pictures they post on social media.

Although it’s a custom neural networks library that helps the app recognize face parts and apply transformations correctly, we had to implement OpenGL shaders to achieve face morphing and image editing effects. The OpenGL library interacts with a smartphone’s graphics processing unit (GPU) and is therefore able to render images at a speed of 60 frames per second (that is, almost in real time).

How to Make Augmented Reality Face Morphing App? How to Make Augmented Reality Face Morphing App?

What makes Glamozis stand out from the competition?
  • Custom filters. The R-Style Lab team has designed five custom filters which do not distort image background and allow users to edit real-life photos (and not just staged ones). It doesn’t matter if you turn your head sideways or smile with your teeth: the smart app will identify face parts correctly anyway (even if you take photos from different angles)! In order to achieve the desired effect with shaders, we had to create filters in Photoshop first and make use of non-linear color gradients to balance skin tones;
  • Multiple facial transformation effects. Besides classic cosmetic surgery effects like nose job and lip augmentation, the app supports 12 advanced effects including “Sexy”, “Pretty”, “Kawaii” and “Mulan”. The filters completely change a person’s appearance and create cartoon-like or exaggerated looks which either cannot be achieved by means of modern plastic surgery techniques or require complex (and expensive!) surgical treatment.

By the way, Glamozis is already available on the App Store; want to give it a try?

How to Make Augmented Reality Face Morphing App?

How to make an Augmented Reality app that will delight users?

According to Pavel Shylenok, CTO at R-Style Lab, a successful face-morphing AR app should have several features including:

  • AI-based facial transformation framework (which identifies transformation points correctly and enables smooth image rendering);
  • At least 20 custom effects (the more the better; make sure the effects do not replicate those of popular face swap apps and…look great);
  • User-centered design (app users shouldn’t be racking their brain trying to figure out how to apply/undo filters or upload pics to Instagram);
  • Flexible monetization strategy (which takes into account both premium and non-paying users and provides multiple filter bundles including lifetime subscription);
  • Social sharing (you know what it is, right?).

Unless you want to pull a Snapchat, you don’t really need user profile/registration modules and storage space; Pavel argues that face swap apps should in fact be anonymous since users only care about the end result of the facial transformation process (and don’t need another social media app on their smartphones).

How to Make Augmented Reality Face Morphing App?

How much does it cost to build a face morphing mobile application?

Ok, the actual price of a face-morphing mobile app depends on several factors including its feature set, the median iOS/Android developer hourly rates in your country (or the country you’re going to outsource Augmented Reality app development to) and the size of the company providing mobile app development services. Therefore we shall measure the efforts in man-hours and leave the calculations to you:

  • The development of a fully-fledged facial transformation framework takes up to 3 thousand man-hours and is only billed according to the Time & Material (T&M) model;
  • It takes 40-50 man-hours to design and deploy a custom filter similar to those of Glamozis. A viable face swap app should offer at least 20 filters and effects, so add another 800 hours to the estimate;
  • UI, UX, backend part and features supporting app monetization usually take 400-500 man-hours to develop.

In the end, we’ve arrived at an impressive figure – almost 4.5 thousand man-hours! That’s a lot of work and lot of money – and it’s up to you to allocate your time, efforts and budget wisely.

The road to face morphing app success starts with a thorough market research which involves target audience and competition analysis, the choice of a monetization strategy (we recommend that you conduct A/B testing to see what works best) and MVP development. Provided you make timely changes to the scope, align your expectations with reality, pay attention to marketing and address a reliable AR vendor, you may get a call from Mr. Zuckerberg one day.

We don’t peddle trends. We streamline business.