Trigger AWS IoT Greengrass component deployments from AWS CodeCommit

As computing power extends into Internet of Thing (IoT) devices, software becomes more and more crucial to making decisions, processing data, and providing insight to end users at the device. Devices are the backbone of many businesses and applications. Regardless of the use case, customers benefit from quick, widespread deployment of software updates and improvements across device fleets. With AWS IoT Greengrass V2, AWS customers can easily build, deploy, and manage custom device software as AWS IoT Greengrass components. AWS IoT Greengrass handles activities such as tracking the component versions, managing the fleets of devices, and orchestrating component updates.

In this blog, we will demonstrate how to automate the deployment of component changes on AWS IoT Greengrass. This solution reduces the time it takes developers to deploy custom device software across a fleet from minutes to seconds, saving developers valuable time and improving agility.

Introduction

This solution uses a centralized code repository with Continuous Integration and Continuous Deployment (CI/CD) to assist in following DevOps best practices. For more information about CI/CD and DevOps on AWS, reference the Practicing Continuous Integration and Continuous Delivery on AWS whitepaper.

In this article, we will present a method for automating the deployment of custom AWS IoT Greengrass components. AWS IoT Greengrass includes AWS-provided components to add common functionality to your devices and also allows for custom components creation. Custom components may analyze data, display a front-end dashboard, or run an application with intermittent connectivity to the cloud. For demonstration purposes, we will be using three pre-built components. However, this solution is easily adaptable to components you may already have running in your environment.

This solution clones a Github repository to an AWS CodeCommit repository as an example implementation. Other implementations can be used as long as the code is deployed to CodeCommit. At the beginning of the steps below, we will outline how to clone a repository from Github into AWS CodeCommit. However, other code repositories can be imported into AWS CodeCommit if necessary for your use case. For more information on migrating code to an AWS CodeCommit repo, see the documentation here.

Prerequisites

For this walk through, you should have the following prerequisites:

An AWS account
A basic understanding of:

AWS CloudFormation
AWS IoT Core
AWS IoT Greengrass
AWS CodeBuild
AWS CodeCommit
AWS CodePipeline

Install AWS CLI

Make sure you have installed and setup AWS CLI with the necessary permissions:

AWS CLI Setup

Install CDK

Here is a guide to Get Started with AWS CDK:

Installing aws-cdk

Please verify the cdk package version installed is 2.x or higher.

Solution Overview

The solution utilizes an AWS CodeCommit repo to store the component code, a Lambda function to trigger the build, and AWS CodeBuild to orchestrate the deployment of the updated component. Finally, AWS IoT Greengrass pushes the updated component out to the devices.

A commit is made to the AWS CodeCommit repository. An Amazon CloudWatch Event event has been configured such that any time a commit is made to the configured repository, the event occurs.
An AWS Lambda function is triggered by the Amazon CloudWatch Event.  The AWS Lambda function first determines if the commit was made on a file that is part of the source code for the component. If that is the case, it saves the component name(s) as AWS CodeBuild Project Environment Variables and starts the AWS CodePipeline.
AWS CodeBuild job is triggered by the AWS CodePipeline. The AWS CodeBuild job runs a shell script which deploys the component to the devices using the AWS IoT Greengrass API.

AWS Cloud Development Kit (AWS CDK) is a framework for defining cloud infrastructure in code, and provisioning it via AWS CloudFormation. If you are new to the AWS CDK, follow the getting started guide. 

The CDK will deploy the following resources in the AWS Account:

AWS CodePipeline
AWS Lambda
CodeBuild
Amazon S3
[Optional] Amazon EC2
AWS CLI v2
AWS CDK v2

Deploy the solution

Create and clone code repository

Create CodeCommit Project

Go to AWS CodeCommit Console
Select Create Repository
Provide name for the repository. For example ggv2-cdk-blog-test
Choose Create

Clone the CodeCommit repository on your local machine, for example if the CodeCommit repository is named  ggv2-cdk-blog:

git clone codecommit::us-east-2://ggv2-cdk-blog
cd ggv2-cdk-blog

In order to deploy the cdk you will need to copy the cdk contents from github repository:

To easily copy the contents of this github project to your new project, copy export.zip to your CodeCommit project directory, and unzip

Note: The .gitignore file is part of export.zip, if you don’t find the file after unzip check your settings to view hidden files in the IDE

Source code updates

Please update following attributes in cdk.json file with appropriate values:

account

Account ID of your AWS account, for example: 1234567890

codecommit_repository_arn

For example: arn:aws:codecommit:us-east-1:111111111111:MyDemo*

region

For example: us-east-1

create_core_device

acceptable values are true or false
Even if you choose false make sure to provide a name for an existing core device in the option core_device_name and core_device_group_name

default_branch_name

 branch to track for the CodePipeline. For example: main

  core_device_name

 The name of your new/existing Greengrass core device.

 core_device_group_name

The name of your new/existing Greengrass core device group.

 project_prefix

 Any string value

Deploy CDK pipeline

Set up your virtualenv for Python. You may need to use python3 in replacement of python, depending on your local python configuration.

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txt

Bootstrap your account/region for CDK – replace the appropriate variables (i.e. ACCOUNT-ID, REGION, ADMIN-PROFILE) before executing.

Commands to run:

export CDK_NEW_BOOTSTRAP=1 
npx cdk bootstrap –cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess
aws://ACCOUNT-ID/REGION 
 Note: Verify you have the AdministratorAccess policy in your AWS account or you can customize the policy to be used by AWS CDK in order to create AWS resources
For example, like this:
export CDK_NEW_BOOTSTRAP=1                                    
npx cdk bootstrap –cloudformation-execution-policies arn:aws:iam::aws:policy/<CustomPolicy>
aws://123456789/us-east-2

Commit updates to the repository and deploy the CDK app. You may need to git push origin <main branch name> , instead of git push

git add –all
git commit -m “initial commit”
git push
cdk deploy

 Optional: To update export.zip in your own project, run the following:

git archive -o ./export.zip HEAD

The code repository for this blog has sample custom AWS IoT Greengrass components, that will display the message Hello World in the log file of the component. The next section will provide more information about building your own AWS IoT Greengrass components.

Adding your own AWS IoT Greengrass components

The code in this blog uses Greengrass Development Kit (gdk cli) in order to build and publish Greengrass components. For more information please check this documentation.

To add new components to the project, create a new component directory in the components directory. Make sure your components include the following:

gdk-config.json (GDK configuration file)
buildspec.yml (for CodeBuild)
requirements.txt (for Python dependencies; currently used by provided buildspec.yml examples)

Building AWS IoT Greengrass components

Here are 5 tips to build AWS IoT Greengrass v2 Components. For more information please refer below:

Develop AWS IoT Greengrass components
Use the AWS IoT Device SDK to communicate with the Greengrass nucleus, other components, and AWS IoT Core

Cleaning Up

CDK Cleanup

Run the following command from your terminal on the path where the code repository exists (Example: Users/johndoe/desktop/ggv2-cdk-blog ~ %)
cdk destroy

Conclusion

You now have setup a DevOps pipeline for multiple components in the same code repository. With these enhancements, when a developer pushes code updates to components in the repository, the AWS IoT Greengrass V2 components automatically deploy a new version with the updates. Expect to see developers be able to iterate faster, creating business value and innovation at a new pace.

Learn more about AWS IoT Greengrass through this hands on workshop.  For more information about public components for AWS IoT Greengrass, please visit https://github.com/aws-greengrass

About the Authors

Jon Slominski is a Sr. Solutions Architect with the Prototyping & Cloud Engineering (PACE) team at AWS. Building prototypes focused on IoT, AI/ML, and robotics, Jon helps customers innovate and envision the art of the possible. Outside of work, Jon enjoys spending time and traveling with his wife and daughters.

Joyson Neville Lewis is an IoT Data Architect at AWS Professional Services. He has worked as a Software/Data engineer before diving into the Conversational AI and Industrial IoT space where he works with companies to connect the dots between business and AI using Voice Assistant/Chatbot and IoT solutions.

Jack Tanny is an Associate Data/ML Engineer in AWS’s Professional Services team. He builds solutions that use data to solve problems and unlock business value for our customers. In his free time, you can usually find Jack in the mountains, biking, skiing, or camping.

Leave a Comment

Generated by Feedzy