Smart Waste Management System-Part 2-Storing and Extracting data from the DynamoDB database and showing on the website.

AWS Services: AWS IoT Core, AWS DynmoDB, AWS API Gateway, AWS IAM, AWS Lambda(python)

Introduction

In the previous blog, we have learned about how to send sensor data to AWS IoT Core. So in this blog let’s move ahead and learn how to store data directly into the database and with help of Lambda will extract the data from the database after attaching the IAM Role with it and will further learn how to send data on frontEnd with the help of REST API KEY.

Implementation:

This project’s implementation is divided into three-phase:
1. Client-side which includes coding regarding NodeMCU ESP8266 so that it will send data to AWS IoT Core Console and registering your device with AWS.(This has been covered in the previous blog.)

2. AWS configuration includes storing data into the database, creating an API key, and configuring the Lambda function(python).

3. FrontEnd-side which includes creating a website or app and pass the created API KEY link in that to get the latest data for ease of access.

AWS Configuration

1.First of all, let’s see how to connect AWS IoT Core and DynamoDB so that we can store data in the database.

• Open AWS Management Console and navigate to the ‘AWS IoT Core’ page under the Services.

• Under the ‘Act’ section you’ll find ‘Rules’, navigate to the ‘Rules’ page.

• Choose on ‘create a rule’ and in the Name and Description text box enter the name that you want.

• In the ‘Rule query statement’ enter the below-given code:
  (Note: Here ‘outTopic’ is the topic that you are subscribed with)

SELECT *, timestamp() AS ts FROM 'outTopic'

• In ‘Set, one or more actions’ choose ‘Add Actions’.

• Select ‘Split message into multiple columns of a DynamoDB table (DynamoDBv2)’ and then click on ‘configure action’.

• After that select ‘Create new resource’, you’ll be redirected to the dynamoDB console.

• Create a new table and name it as per your wish.
  i. Primary key — mac_Id (String).
  ii. Click on ‘Add sort key’.
  iii.Sort key — ts (Number).

• Scroll down and uncheck ‘default settings’ and under the ‘Auto Scale’ uncheck ‘Read Capacity’ and ‘Write Capacity’ and change both the values to ‘1’.

• Scroll down and click on ‘Create’. This will take a few seconds to create and your dashboard will look like:

• Again Go back to the AWS IoT console and click on the ‘refresh’ button next to ‘create new resource’. After refreshing, you will be able to see the table that you have recently created, choose that table.

• Under the ‘Choose or create a role to grant AWS IoT access to perform this action.’ in that choose ‘create a new role’.

• Enter the name as per your wish and click on ‘Create role’.

• Click on ‘Add Action’. Your dashboard will look like:

• Scroll down to the bottom of the page and click on ‘Create rule’.

• Now just plug your NodeMCU into your PC and in your AWS IoT core console navigate to the ‘Test’ section, subscribe to the topic that you defined in your code in my case it’s ‘outTopic’.
• The output screen will look like:

• Redirect to the DynamoDB console and select the table in which data will be stored.
  Choose ‘Items’ and here you will find all your sensor data.

• Hurray! we have successfully connected AWS IoT Core with AWS DynamoDB.

2. First of all, we will create an AWS IAM role so that AWS Lambda can interact with the DynamoDB database.

• In the AWS management console, navigate to AWS IAM.

• Navigate to the ‘Roles’ pane and click on Create role.

• Choose ‘Lambda’ and then click on ‘Next: Permissions’. In the search box search for ‘dynamoDB’ and check the checkbox of “DynamoDB full access” then click on Next: Tags.
  (Note: You can choose as per your wish.)

• On the final screen give the role-name as per your wish and then click on create the role. On the role page, you will be able to see the role that you have created.

Pheww! Created lambda and DynamoDB role so that we can easily be able to fetch the data from DynamoDB using python.

3. Now, We will create a lambda function and will write some lines of code in that so that we can get some data related to our DynamoDB table.

• Open AWS Lambda console.

• Click on the Create function and enter the below-given information:
  Function Name: demo
  Runtime: Python 3.8
  Under the change default execution role choose ‘Use an existing role’ and in that select the role that you have created earlier.
  And finally, click on ‘Create’.

Finally, we have configured almost all our required things, last and final step is to create API and write code in lambda so we can get data on our HTML page.

4. Now, we will see how to create a ‘REST API’ using the AWS API GATEWAY.

• Open AWS Management Console and navigate to the ‘AWS API GATEWAY’ page.

• Scroll down and choose REST API without a Private tag as our API key will be triggered from HTML Page and it may not be within VPC.

• After clicking on Build, choose ‘new API’, give name and description as per your wish, and finally click on ‘Create’.

• Your dashboard will look as shown below:

• Navigate to ‘Actions > Create Resource’. Enter Resource Name: GET.
  (Note: Here we have given GET because our API key will request for data. If you are going to enter the user input into the database then you must enter PUT for ease of access)
• Make sure that you check the checkbox of ‘Enable API Gateway CORS’ and then click on Create Resource.

• Again click on ‘Actions > Create method’ and in that choose GET. A configuration page will be opened in that choose Lambda Function and in the lambda function write the name of the function that you have defined and then click on Save.

• If you see a popup menu about adding permissions to lambda function then in that click ok.

• After clicking OK your dashboard will show one diagram about how the process flows.

• Now again click on Deploy API from the Actions drop-down menu.

• Enter the below-given information:
  Deployment stage: [New Stage]
  Stage name: v1
  Stage description: v1
  After configuring this step click on Deploy.

• Expand the v1, you will be able to see the GET method and on clicking it you will find one link, kindly save that link.

• Copy the API URL and paste in the browser you will see the demo page as output.
  You can configure as per your own needs.

Congrats! You have successfully created a REST API KEY to get data.

5. Now, the last and final step add code to AWS Lambda and it will return some information related to our table, and the API key will pass it to the FrontEnd.

• Demo code that shows how many tables are there in the DynamoDB database & Demo code that shows the latest data can be found from this Github link.

Copy the code that you want to execute and paste it in the lambda console then simply deploy and test it.

Frontend side

1.Designing the prototype of a basic structure of the index portal.

2. This portal contains a total of 3 pages including the home index page.

• Home Page — ADMIN
• About Us
• Contact Us

3. Development of website with basic HTML, HTML 5, CSS, JavaScript & Bootstrap framework for better User Interface & User Experience.
4. Constructing JavaScript to fetch data from the Application Programming interface URL.

The HTML code where to replace your API key can be found from this Github link.

Conclusion:

In this blog you learned how to configure the following things:
1. Connect AWS IoT Core to DynamoDB
2. Created AWS IAM role for lambda and DynamoDB
3. Created Lambda function for python
4. Created REST API KEY
5. Inserted link in the HTML code to display data on Frontend.

Useful Links:

Github: https://github.com/dhruvshah1105/Smart-waste-management-system-on-aws-cloud.git

I hope, you all have liked this blog, and if have any doubt then kindly comment below.