Get a Free Quote

Intelligent Video Analysis System for Personalized Home Automation

The client, a UK-based smart home startup, envisioned an intelligent system that would rely on advanced video analysis to track and recognize users as they move around the house. It would also monitor user activity and daily habits to learn their preferences. Based on that information, the system would respond with personalized settings, sending commands to smart lights, home entertainment systems, climate control devices and other smart electronics.

The client, a UK-based smart home startup, envisioned an intelligent system that would rely on advanced video analysis to track and recognize users as they move around the house. It would also monitor user activity and daily habits to learn their preferences. Based on that information, the system would respond with personalized settings, sending commands to smart lights, home entertainment systems, climate control devices and other smart electronics.
Technologies & tools
C++, Lua, Objective-C, Bluetooth Low Energy, Modbus, Arduino, Raspberry Pi
Project team
2 iOS developers, 2 C++ developers, 1 PM, 1 system architect

Solution

To get funding for further development of the project, the client needed an MVP of the user recognition system, as well as tablet and mobile demo apps that would showcase smart home functionality. In line with the product design concept, the solution was planned to be marketed as a premium light fixture. It would take advantage of built-in professional video cameras to get a granular image and precision with video object detection from as far from an object as a 12-meter distance. The R-Style Lab team elaborated and designed the solution architecture, and employed its embedded system engineering skills to develop both hardware and software parts of the home automation MVP. To achieve the results that would meet the client's requirements and expectations, we created:

  • customized Arduino and drivers for camera mounts repositioning. A motor controller on camera mount chassis gets commands to enable 360-degree rotation and tilt of the professional HD Sony cameras within the light fixture and to ensure that a user is always in a camera focus.
  • image acquisition library. In order to achieve efficient image processing at the CPU capacity, the team selected a custom library over OpenCV, which would otherwise use up more resources and result in noticeably slower performance.
  • integration with proprietary image recognition library. The obtained image data is fed into the library, which returns an image recognition level-of-confidence score.
  • customized Raspberry Pi with motion detectors. To get accurate user presence detection, motion-triggered camera position management and eventually efficient image recognition, R-Style Lab evaluated several motion sensor options. Doppler radar and PIR sensors imposed some serious constraints for motion sensing and motion direction detection. Their performance was hindered either by unacceptably slow response rates or low noise reduction ability. The final choice to complete the task was made in favor of video camera sensors and Raspberry Pi.
  • Bluetooth Low Energy communication. Using BLE protocol we provided for communication in a low power mode for users' mobile phones and smart home devices that made part of the client's home automation solution.

Having successfully implemented the user recognition component of the system that underlay the client's idea of full-blown automation with no human assistance, they needed to demonstrate the potential of the solution-to-be. Our team created for this purpose some of the planned automation features.

Demo Smart Home Functionality:

  • integration with Milight and a TV set - interfaced via the controller API and a COM port, lights and TV program settings adjusted for each user that was tracked in the home automation solution as they entered a room;
  • voice control - allowed users to take incoming calls in a hands-free mode via in-home smart speakers;
  • call roaming via smart speakers - the challenge of this task was that Bluetooth did not support roaming. There was no way to transfer mobile-to-smart-speaker connection from one speaker to another as a user moved around the house. Eventually, audio packets were routed via PoE adapter or Ethernet connection, and audio streaming flowed from speaker to speaker as the video-enabled object recognition system followed a user from room to room.

The MVP has been successfully released. The client is planning to expand its functionality into a fully fledged smart home automation solution as he completes the next investment round.

Do You Have Your Own Project to Discuss?
We are Here to Help You!

Other Related Projects

Get a Free Quote

Initial response in 12 business hours



Name shouldn't be empty

Email shouldn't be empty


Field shouldn't be empty

By sending this message you agree to our Privacy Policy