More and more people are using the phrase “Ok Google” to start a conversation to find information or perform actions using the Google Assistant. Launched last fall, the new assistant is already available on over 100 million devices and is rapidly evolving. Today at Google I/O, its creator took the stage to discuss how Google Assistant is expanding its capabilities and reach on smartphones and Google Home.

Google Home

Google Home, a standalone voice activated smart-appliance and speaker, is getting the most attention, with a number of significant improvements on the way. In a few months, Google Home will be able to make hands-free VOIP calls to any phone in the U.S. or Canada for free. There’s no additional setup, apps to install, or reliance on a landline or cell phone; just ask the Assistant to place the call. And with multi-user support (the Assistant recognizes individual voices), it’s smart enough to call your “mom” and not your roommate’s “mom.” Google Home will also provide proactive notifications in the near future. Initially, the Assistant will focus on travel, keeping track of upcoming flights and appointments and then alerting you to flight delays or traffic alerts. For the latter, it will even calculate a new travel time and suggest leaving earlier so you’re not late to your next appointment. Another new feature, this one rolling out today, is the ability to use Assistant on Google Home to create reminders and calendar appointments, which will then appear on your other Android devices.

Google Home is also getting some new media capabilities. In addition to Deezer, Pandora, Soundcloud, and others it will soon support free music streaming from Spotify, and with the addition of Bluetooth, it will be able to play music from a nearby Android or iOS device.

Moving from audio to video, Google Home is adding some new video streaming partners too, including CBS All Access, HBO NOW, and HGTV. Later this year, the Assistant on Google Home will gain the ability to send visual responses to your TV via Chromecast. For example, saying “Ok Google. What's on my calendar today?” will show your daily schedule on the TV. Of course it will show other common things too such as the weather forecast, and it will integrate with Google’s YouTube TV service. It was not clear during the onstage demo, but I assume Google Home will use visual responses when it senses a Chromecast device and switch back to audio responses when the TV is off.

A visual response from Google Home showing the weather forecast on the TV

The last bit of Google Home news is that the device will be available in Australia, Canada, France, Germany, and Japan later this year.

More people will soon be able to use Google Assistant on Android devices too. First, it’s learning to speak several new languages this summer, including Brazilian Portuguese, French, German, and Japanese, along with Italian, Korean, and Spanish by the end of the year. Google Assistant is also coming to the iPhone in the form of a new app that’s already available on Apple’s App Store. Now iPhone users can use its natural language interface to ask questions, schedule appointments, make phone calls, send texts and emails, play music, and more.

Ordering food using Actions and the new Transactions API

The previously announced Google Assistant SDK will allow third parties to integrate Assistant into even more devices, and by leveraging Actions on Google, developers can integrate the Assistant directly within their apps. There’s also a new Transactions API (currently a developer preview) that will allow developers to offer a “conversational ordering experience” that will help a customer add items to a shopping cart, collect address and billing information, make the purchase or reservation using your fingerprint or other authentication method, and even send order confirmation or shipping updates.

One of the priorities for Google Assistant is making it more conversational; however, there are times in public when you cannot actually speak to your phone, which is why, starting today, you can also type your queries and avoid annoyed or quizzical looks from nearby strangers.

Google Lens: Recognizing the name of a restaurant and providing relevant information

Perhaps the coolest new feature coming to Android devices is Google Lens, a new AR experience that uses computer vision to recognize objects in the environment, allowing you to have a conversation with Assistant about what you see. By tapping the Google Lens icon in the camera, it can perform visual language translation (similar to what Google Translate does); help you scan business cards and create new contacts, and even recognize a sign for an upcoming concert and offer you the ability to buy tickets, add the event to your calendar, or even sample some of the band’s songs.

Google Lens: Using computer vision to read an ID tag and automatically sign in to a Wi-Fi network

New products like Google Assistant and Google Lens make it clear that Google is no longer a search company; it’s an artificial intelligence company that’s focused on integrating speech recognition, language processing, contextual awareness, and computer vision with Google Search and data from other Google services such as Gmail to provide new experiences across more devices using a more natural interface.