As computing power extends into Internet of Thing (IoT) devices, software becomes more and more crucial to making decisions, processing data, and providing insight to end users at the device. Devices are the backbone of many businesses and applications. Regardless of the use case, customers benefit from quick, widespread deployment of software updates and improvements across device fleets. With AWS IoT Greengrass V2, AWS customers can easily build, deploy, and manage custom device software as AWS IoT Greengrass components. AWS IoT Greengrass handles activities such as tracking the component versions, managing the fleets of devices, and orchestrating component updates.
In this blog, we will demonstrate how to automate the deployment of component changes on AWS IoT Greengrass. This solution reduces the time it takes developers to deploy custom device software across a fleet from minutes to seconds, saving developers valuable time and improving agility.
Introduction
This solution uses a centralized code repository with Continuous Integration and Continuous Deployment (CI/CD) to assist in following DevOps best practices. For more information about CI/CD and DevOps on AWS, reference the Practicing Continuous Integration and Continuous Delivery on AWS whitepaper.
In this article, we will present a method for automating the deployment of custom AWS IoT Greengrass components. AWS IoT Greengrass includes AWS-provided components to add common functionality to your devices and also allows for custom components creation. Custom components may analyze data, display a front-end dashboard, or run an application with intermittent connectivity to the cloud. For demonstration purposes, we will be using three pre-built components. However, this solution is easily adaptable to components you may already have running in your environment.
This solution clones a Github repository to an AWS CodeCommit repository as an example implementation. Other implementations can be used as long as the code is deployed to CodeCommit. At the beginning of the steps below, we will outline how to clone a repository from Github into AWS CodeCommit. However, other code repositories can be imported into AWS CodeCommit if necessary for your use case. For more information on migrating code to an AWS CodeCommit repo, see the documentation here.
Prerequisites
For this walk through, you should have the following prerequisites:
An AWS account
A basic understanding of:
AWS CloudFormation
AWS IoT Core
AWS IoT Greengrass
AWS CodeBuild
AWS CodeCommit
AWS CodePipeline
Install AWS CLI
Make sure you have installed and setup AWS CLI with the necessary permissions:
Install CDK
Here is a guide to Get Started with AWS CDK:
Please verify the cdk package version installed is 2.x or higher.
Solution Overview
The solution utilizes an AWS CodeCommit repo to store the component code, a Lambda function to trigger the build, and AWS CodeBuild to orchestrate the deployment of the updated component. Finally, AWS IoT Greengrass pushes the updated component out to the devices.
A commit is made to the AWS CodeCommit repository. An Amazon CloudWatch Event event has been configured such that any time a commit is made to the configured repository, the event occurs.
An AWS Lambda function is triggered by the Amazon CloudWatch Event. The AWS Lambda function first determines if the commit was made on a file that is part of the source code for the component. If that is the case, it saves the component name(s) as AWS CodeBuild Project Environment Variables and starts the AWS CodePipeline.
AWS CodeBuild job is triggered by the AWS CodePipeline. The AWS CodeBuild job runs a shell script which deploys the component to the devices using the AWS IoT Greengrass API.
AWS Cloud Development Kit (AWS CDK) is a framework for defining cloud infrastructure in code, and provisioning it via AWS CloudFormation. If you are new to the AWS CDK, follow the getting started guide.
The CDK will deploy the following resources in the AWS Account:
AWS CodePipeline
AWS Lambda
CodeBuild
Amazon S3
[Optional] Amazon EC2
AWS CLI v2
AWS CDK v2
Deploy the solution
Create and clone code repository
Create CodeCommit Project
Go to AWS CodeCommit Console
Select Create Repository
Provide name for the repository. For example ggv2-cdk-blog-test
Choose Create
Clone the CodeCommit repository on your local machine, for example if the CodeCommit repository is named ggv2-cdk-blog:
git clone codecommit::us-east-2://ggv2-cdk-blog
cd ggv2-cdk-blog
In order to deploy the cdk you will need to copy the cdk contents from github repository:
To easily copy the contents of this github project to your new project, copy export.zip to your CodeCommit project directory, and unzip
Note: The .gitignore file is part of export.zip, if you don’t find the file after unzip check your settings to view hidden files in the IDE
Source code updates
Please update following attributes in cdk.json file with appropriate values:
account
Account ID of your AWS account, for example: 1234567890
codecommit_repository_arn
For example: arn:aws:codecommit:us-east-1:111111111111:MyDemo*
region
For example: us-east-1
create_core_device
acceptable values are true or false
Even if you choose false make sure to provide a name for an existing core device in the option core_device_name and core_device_group_name
default_branch_name
branch to track for the CodePipeline. For example: main
core_device_name
The name of your new/existing Greengrass core device.
core_device_group_name
The name of your new/existing Greengrass core device group.
project_prefix
Any string value
Deploy CDK pipeline
Set up your virtualenv for Python. You may need to use python3 in replacement of python, depending on your local python configuration.
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txt
Bootstrap your account/region for CDK – replace the appropriate variables (i.e. ACCOUNT-ID, REGION, ADMIN-PROFILE) before executing.
Commands to run:
export CDK_NEW_BOOTSTRAP=1
npx cdk bootstrap –cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess
aws://ACCOUNT-ID/REGION
Note: Verify you have the AdministratorAccess policy in your AWS account or you can customize the policy to be used by AWS CDK in order to create AWS resources
For example, like this:
export CDK_NEW_BOOTSTRAP=1
npx cdk bootstrap –cloudformation-execution-policies arn:aws:iam::aws:policy/<CustomPolicy>
aws://123456789/us-east-2
Commit updates to the repository and deploy the CDK app. You may need to git push origin <main branch name> , instead of git push
git add –all
git commit -m “initial commit”
git push
cdk deploy
Optional: To update export.zip in your own project, run the following:
git archive -o ./export.zip HEAD
The code repository for this blog has sample custom AWS IoT Greengrass components, that will display the message Hello World in the log file of the component. The next section will provide more information about building your own AWS IoT Greengrass components.
Adding your own AWS IoT Greengrass components
The code in this blog uses Greengrass Development Kit (gdk cli) in order to build and publish Greengrass components. For more information please check this documentation.
To add new components to the project, create a new component directory in the components directory. Make sure your components include the following:
gdk-config.json (GDK configuration file)
buildspec.yml (for CodeBuild)
requirements.txt (for Python dependencies; currently used by provided buildspec.yml examples)
Building AWS IoT Greengrass components
Here are 5 tips to build AWS IoT Greengrass v2 Components. For more information please refer below:
Develop AWS IoT Greengrass components
Use the AWS IoT Device SDK to communicate with the Greengrass nucleus, other components, and AWS IoT Core
Cleaning Up
CDK Cleanup
Run the following command from your terminal on the path where the code repository exists (Example: Users/johndoe/desktop/ggv2-cdk-blog ~ %)
cdk destroy
Conclusion
You now have setup a DevOps pipeline for multiple components in the same code repository. With these enhancements, when a developer pushes code updates to components in the repository, the AWS IoT Greengrass V2 components automatically deploy a new version with the updates. Expect to see developers be able to iterate faster, creating business value and innovation at a new pace.
Learn more about AWS IoT Greengrass through this hands on workshop. For more information about public components for AWS IoT Greengrass, please visit https://github.com/aws-greengrass
About the Authors
Jon Slominski is a Sr. Solutions Architect with the Prototyping & Cloud Engineering (PACE) team at AWS. Building prototypes focused on IoT, AI/ML, and robotics, Jon helps customers innovate and envision the art of the possible. Outside of work, Jon enjoys spending time and traveling with his wife and daughters.
Joyson Neville Lewis is an IoT Data Architect at AWS Professional Services. He has worked as a Software/Data engineer before diving into the Conversational AI and Industrial IoT space where he works with companies to connect the dots between business and AI using Voice Assistant/Chatbot and IoT solutions.
Jack Tanny is an Associate Data/ML Engineer in AWS’s Professional Services team. He builds solutions that use data to solve problems and unlock business value for our customers. In his free time, you can usually find Jack in the mountains, biking, skiing, or camping.