Google Duplex: Understanding the Core Technology Behind Assistant's Phone Calls

 

Top Google executives, led by CEO Sundar Pichai, took the stage at the company’s annual developer conference to reveal new features for its products and services. Some that we found pretty interesting (not to mention useful) were the new Smart Compose feature in Gmail and the integration of Assistant in Maps.

But the feature that stole the show has got to be that demo powered by Google Duplex, the company’s new experimental technology which lets an AI-powered bot make phone calls on your behalf. Sounds crazy, right?

Well, Google just showed the world that such a technology exists and it will be available via Google Assistant in the near future. The demo - which showed a voice assistant call a hair saloon and book an appointment on behalf of a customer - was nothing short of amazing, and it just goes to show how far ahead Google is in terms of voice-based AI assistants and conversational, natural language processing.

At the core of it, Duplex is a recurrent neural network that can be trained for highly specialised tasks and it uses Google’s automatic speech recognition technology, so it can interface with the user. Thanks to WaveNet, an AI-based generative program that’s part of Google’s DeepMind division, Duplex can sound a lot closer to a human than ever before, complete with “umms" and “aahs”, as seen in the video above.

The cool thing about Duplex is that it enables the Assistant to relieve you of mundane tasks, like having to make a restaurant reservation or a hair salon appointment. The creepy part is the other party has no idea they are talking to a machine, something that can be abused in any number of different ways.

This feature is said to be rolling out in the coming weeks in the US as an experiment to “normal users”. Google will be focusing on a few specific scenarios initially, before expanding it further.

“It is something that we are really thoughtful about and thinking hard, about how we bring this to market,” said Lillian Rincon, Product Management Director for the Google Assistant during a media briefing at Google I/O in Mountain View, California.

So, how does this actually work? From the looks of it, it would be like giving any other command to the Assistant. For example, when booking an appointment at a salon, you will have to give the Assistant some key parameters to work with, like the name of the salon and the timings you wish to book. If the company or business has an online booking service, then it will book it through that, else it would make a phone call.