Google AIY Voice Kit Hacking — Part 2

Welcome back to the last part of Google AIY Voice Kit Hacking tutorial. We focused on to assemble the hardware of Voice Kit in the first part. Here we would focus on to implement the software and configure it accordingly. If you have missed the first part of the tutorial, you can quickly visit it from clicking here: Google AIY Voice Kit Hacking — Part 1

To boot the kit, you need a SD card. Minimum memory should be 8 GB. I bought a 32 GB SanDisk Ultra Micro SD card.You should aim for better quality Micro SD card.

Also buy a Micro SD Card USB reader. Insert the micro SD card into the reader. It is suggested to format the SD card with SD Card Formatter (A freeware which you can download easily). But I observed that Windows default external drive format option works fine. Just remember to select FAT 32 option during formatting.

Download the AIY Voice Kit SDK image from the following link. For direct download, follow this link. Now write the image to SD card via card writing utility — Etcher.io which is a popular tool. It will take around 15–20 min to complete the image write processing.

Image result for sd card insert in google voice kit

Now comes the hack. Usually you need a HDMI monitor, a keyboard and a mouse to boot the OS within Voice Kit. Because Raspberry Pi does not have any visual interface. Since I did not have spare HDMI monitor, so I searched for a solution to do it via laptop and got one.

You need to provide an IP address to the voice kit so that you can connect to it anytime. During first connection establishment between your laptop and the voice kit, you need an ethernet cable.

Now follow the below steps to boot your raspberry pi 3 and connect it via your laptop:

  1. Connect your Micro USB charger to the voice kit and plug it on. A red light will start blinking on.
  2. Connect the ethernet cable between your laptop and voice kit. Check the voice kit green ethernet port light blinking, indicating it is connected fine.
  3. Go to “Run” option, type “cmd” and hit Enter. Once the command prompt is open, type “ipconfig” and hit Enter again.
  4. Note down the IPv4 Address related to “Ethernet adapter Ethernet” section

5. Now open the ayiproject SDK image files of micro SD card via Windows explorer. Find “cmdline.txt” file and add the following section:

ip=192.xxx.xxx.4::192.xxx.xxx.1

The second IP is the IP of your laptop (IPv4). Copy the same IP with change in last digit and put it in the first IP location which will be the ethernet IPv4 address of your voice kit. Keep the rest syntax (::) intact.

Also create a new file called “ssh” in the same folder without any extension or anything written within (0 KB). This will help to connect the voice kit via SSH terminal.

6. Now eject the SD card from your laptop and insert it within Voice Kit SD card slot.

It’s better to stick a piece of adhesive tape at the end of the SD card. It’ll give you the ease to take it out if necessary from the cardboard slot.
Now wait for sometime until the image is booting properly. It can take up to 10–15 min for the first time.

7. Grab a cup of coffee and let’s do few things before the booting is completed. Download “PUTTY” in your desktop from this link. It is a freeware software to connect your voice kit via SSH terminal.

8. Download “XMing” software — necessary to open GUI based Raspberry Pi 3 desktop from your laptop. Here’s the download link. Install it and start “XMing” software. It’ll start running in the background.

9. The green light near to SD card slot of the voice kit will stop blinking and turns into steady mode. This indicates that the OS boot has been completed.

10. Open the putty — put the voice kit IP “192.xxx.xxx.4” in Host Name section. Click on SSH menu and select X11 sub-menu. Select the checkbox for “Enable x11 forwarding”. Write “localhost:0” within X display location input. Now hit Open.

11. The login would require the default username and password (user: pi & password: raspberry). Once logged in, check the IP address of pi3

12. Now start the x-server session. Write “startlxde” and hit Enter. This will open the GUI screen of raspberry pi via XMing. Currently it is not connected to any internet. See the network icon at right corner of the desktop. Find your Wifi name there and add it. To get more details, see the video.

Note that it is currently not able to _5 GHz connection. So out of my two frequency Wifi bandwidth (I named them: Swapratim & Swapratim_5GHz), pi3 will be connected to Swapratim Wifi only.

##Sometimes, X server connection does not work with XMing. This troubled me for a while. So I came up with another way to open the GUI. Below are the steps:

  1. Download VNC-Viewer
  2. Install and start VNC-Viewer.
  3. Go to putty and execute the command — “sudo raspi-config”. This will open the Configuration tool wizard. Select Interfacing Options.

By default, the VNC view is disabled. So enable it.

4. Now open VNC-Viewer from Desktop and type the IP address of your voice kit. This will open the VNC-View Desktop

Open the WiFi icon and search for your WiFi to join with password. Now click “Check audio” and “Check WiFi”. They should work fine.

Connecting to Cloud

Open up a web browser by clicking on the globe icon in the menu bar at the top of the screen and then go to console.cloud.google.com. Even if you’ve never used Google Cloud Platform before, you can use your normal Google account to sign in.

The Google Cloud Platform “Getting Started” screen

The problem is that Google makes a lots of changes in the dashboard on a regular basis. As a result, you might not see the same dashboard as you are now looking in the screenshot.

Even it can look different from country to country. After some time from writing this article, the design can change again. So bottom line here is to look a bit carefully into the dashboard. You’ll eventually find all options and menus. It might need a little bit extra patience.

However this may, or may not, be a problem for you. Just bear in mind that when configuring the Google Assistant API you might see something that looks a little different.

Select or Create a Project

Clicking on the “Select a Project ▾” menu to the right of the Google Cloud platform logo didn’t give me a drop down menu as expected. Instead I was presented with a popup window, and hitting the + button let me create a new project. Now go to the Dashboard. Though the design often changes here too.

At this point, now I have a working project, I needed to enable the “Google Assistant API” which is the service that lies behind the Voice Kit.

Enable Google Assistant API

By default,Google Assistant API is been disabled. Click it to enable. Now you need the credentials. So click on the credentials menu from the left pane of the dashboard.

Create credentials for Google Assistant API

Click on Create Credentials drop down and select OAuth client ID option. It’ll take few steps to generate the credentials for the API. The credential generated page would look like this.

Click on the down arrow with the line underneath to download the credentials as a JSON file. Find the JSON file you just downloaded, it’ll be namedclient_secrets_XXXX.json, and rename it to assistant.json. Then move it to /home/pi/assistant.json.

You’ll now need to go to your Google account’s Activity Controls panel. This is where you can configure the information that Google stores about you, and you need to enable “Web & App Activity,” “Location History,” “Device Information,” and “Voice & Audio Activity.”

Note that under “Web & App Activity” you must also tick the additional box to “Include Chrome browsing history and activity from websites and apps that use Google services.”

If you’re doing this from a different browser make sure you’re logged into the same Google account as when you were configuring the application.

Now go back to the desktop, click on the “Start dev terminal” icon and type,

$ sudo systemctl stop voice-recognizer

this will manually bring the voice recognizer service to a halt. We’ll need to start it up from the command line to allow us to configure it.

Configuring the Device

In the dev terminal type,

src/main.py

to start the recognizer. Starting up for the first time, it should automatically open a browser window and let you choose an account to authorise. If you are logged to multiple Google accounts make sure you pick the same account as the one where you set up your Activity Controls.

If everything goes okay you should be presented with the text,

The authentication flow has completed, you may close this window.

in the browser window. The button should also stop pulsing, instead it will briefly blink every few seconds to show that the Voice Assistant is ready to use.

Hitting ^C in the dev terminal will stop the voice recognizer service.

Starting the Voice Service on Boot

As an alternative to running the application manually, you can also run it as a system service. You start the service by entering,

$ sudo systemctl start voice-recognizer

and you can stop the service by entering,

$ sudo systemctl stop voice-recognizer.

Starting the application as a service also lets you to have it start up when the Raspberry Pi boots. To do so you need to enable the voice recognizer service like so,

$ sudo systemctl enable voice-recognize

and next time you boot the Raspberry Pi the voice recognizer should start automatically in the background.

Enjoy the video of Google Voice Kit:

Advertisements

About Swapratim

Blog Writer in Medium for Chatbots Magazine - world's biggest chatbot magazine. Also he writes in Bot Publication and Chatbots Journal on regular basis (https://medium.com/@swapratimroy). Experienced in IT development & support pursuing different roles & responsibilities. He has worked on several innovative projects. Android apps developed by him – In Time SMS, AndroLogic which are freely available in Google Play Store. He has several in house tools built available for free usage on internet. Apart from this, he likes to share his learning with everyone which he had learned over years. Sole purpose of this blog is to share the author's knowledge in IT, passion for travel and recent trends with you. Hope you will enjoy his contents.
Aside | This entry was posted in Chatbot and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s