Building a Google Assistant Agent with API.AI and AWS Lambda

I wanted to learn about API.AI and deploying to Actions on Google. My kids love talking to our Google Home, sometimes more than they like talking to me about their day at school. I decided to build something that they would like and that would help them "remember" what their day at school was like, so I built an agent to tell us what is for lunch at school.

In this article, I will explain what I built and provide the code for this simple application. I'll walk through what I learned about API.AI and deploying the agent to Google. I'll assume that you have some basic understanding of AWS, specifically S3, Lambda and API Gateway, although I will walk through my Node Lambda code.

(By the way, if you want to talk to the agent yourself, just ask Google to "Talk to St. Joseph's Lunch Menu")

Lambda

To start with, I created a JSON file, saved in S3, that I need to open each time my service is called and read the contents. API.AI sends parameters in a certain format (more on that later) and I need to grab the date that the user is looking for from the JSON it sends so that I can look up the menu for the requested date at that point.

    var requestbody = JSON.parse(event.body);

    var lunchdate = requestbody.result.parameters.date;

    const bucket = 'mylunchmenu';
    const key = 'menu.txt';
    const params = {
        Bucket: bucket,
        Key: key,
    };
    
    s3.getObject(params, (err, data) => {
        if (err) {
            console.log(err);
        } else {
            
            var lookupJson = JSON.parse(data.Body);
            
            var arrayFound = lookupJson.days.filter(function(day) {
                return day.date == lunchdate;
            }); 

            var menu;
            
            if (arrayFound.length > 0)
            {
                menu = arrayFound[0].menu;
            }
            else
            {
                menu = "No menu found";
            }
        }
    });
    

 

Since I need to put an API Gateway in front of this, I need my Lambda reply to be in the correct format so that the response will be successful.
            
    var responseCode = 200;

    var responseBody = {
        speech: "On " + lunchdate + ", " + menu,
        displayText: "On " + lunchdate + ", " + menu,
        source: "source"
    };
        
    var response = {
        statusCode: responseCode,
        headers: {
        },
        body: JSON.stringify(responseBody)
    };
            
    console.log('Returned event:', JSON.stringify(responseBody));
    context.succeed(response);    

Note that this code has to be inside the S3 getObject call, otherwise your Lambda code won't return anything, because it will return before the S3 code actually reads the file.

Here's the full code.

API.AI

Getting started with API.AI, I created a new intent called "What are we having to eat tomorrow?" and added several different scenarios under 'User Says'

API.png

 

API.AI is smart enough to figure out that tomorrow is a date, so a @sys.date parameter is added to the intent. The key here is that users may not always ask for a date, so we need to add scenarios where they don't, and prompt them for the date.

 

API2.png

I added a couple of prompts, just to add some variety.

API4.png

 

With this done, I hooked up my agent to the API I created at Amazon. To do this, I enabled a Webhook and entered the URL from API Gateway.

API7.png

 

I still need responses in my intent, just in case the API is down, so I entered a couple of responses back on the intent page.

API3_0.png

 

With this done, I need to enable the webhook in 'Fulfillment' on my Intent (I left 'Use webhook for slot-filling' unchecked - we don't want to do this here. If a user doesn't include a date, I want the intent to prompt the user for it).

Finally, I went into Integrations and enabled the 'Actions on Google' integration, and went through the wizard.

API6.png

 

Back on my intent, there is a new section called 'Actions for Google'. 'End Conversation' needs to be enabled here so that the agent closes after a the user gets the menu.

API5.png

 

I won't go through the deployment process with Google in detail, but it is pretty straightforward. Once submitted, it took about a week to get approved, then sat in a 'Deploying' status for a couple days after that. The kids love it!