In a recent chat with our AWS Solutions Architect, he pointed me in the direction of some really cool open source DevOps tools from the guys over at Stelligent. They have a bunch of neat utilities and frameworks on their public GitHub, and they do a bunch of very interesting DevOps podcasts that are worth taking some time to listen to.
Anyways, one of the utilities I really like is cfn-nag which is designed to scan CloudFormation templates for known vulnerabilities and bad practices. This is something that fills a gap I have in my DevOps portfolio since I know most best practices and know what I should or shouldn’t do from an infrastructure stand point (most of the time), but my developers shouldn’t have to be AWS security experts. Since stacks are maintained by developers after I hand them off, I need a way to ensure they don’t accidentally introduce security leaks into the platform. As is one of my mantras I have to automate everything, I can’t expect developers to remember to scan their stacks before they get deployed.
There are a couple of ways that I could automate this. In a perfect world I’d deploy cfn-nag as a Lambda function and then call the Lambda function directly from CloudFormation, and Stelligent offers a serverless version for you in the serverless application repository here.
The more I thought about it the more I want this to be extensible to do more than just scan stacks. I’d like to be able to run all my security scanning in one place without needing to write a bunch of custom code, or trying to smash together multiple languages from third parties and then shoe horn this behemoth into one or more Lambda functions.
Enter AWS CodeBuild. I already use CodeBuild extensively in my deployment pipelines, so adding a security buildspec to my micro service repositories is dead simple. This also lets me use either an off the shelf CodeBuild docker image, or build and publish my own that has all the disparate libraries in it that I know I’ll eventually need. Adding additional scans will be as easy as adding a command to the buildspec, and I have the added bonus that I can do scanning differently based on the micro service needs and implementation.
So now that we have a way forward, let’s see how it’s done.
First step is going to be to add a new buildspec to my repository. I already have a set that are contained in a folder in the root of the project called deployment. For clarity, I’ll call it buildspec-security.yml to fit my current naming scheme. Since I know I’m going to start with a Ruby based docker image, all I need to do is install the cfn-nag gem, and run it.
version: 0.2
env:
variables:
placeholder: "value"
phases:
install:
commands:
- apt-get update
- gem install cfn-nag -v 0.3.53
pre_build:
commands:
- echo Scanning CloudFormation Stack using cfn_nag
- /usr/local/bundle/gems/cfn-nag-0.3.53/bin/cfn_nag_scan --input-path deployment/deploy.cf.json --output-format json
- echo Scan Complete
I ran into an issue while implementing this which requires me to lock to a version in the mean time, since when I install gems in Ubuntu in CodeBuild it wasn’t picking up the gem name in the bin folder. When I was calling cfn_nag_scan without the path Ubuntu couldn’t find it. I’m comfortable locking it to a version for now, and then I’ll come back and re-address this later and call this MVP (yay agile!).
This is a pretty simple command, I’m calling cfn_nag_scan, giving it my CloudFormation path, then setting my output format. Default output format is text, I prefer JSON so I can go parse the logs later a bit easier if I need to.
If this command fails, as it does when it finds a security audit failure, it will return a non-zero exit code and fail the CodeBuild project. This is important because it will fail the pipeline and keep the stack from being deployed.
Speaking of Pipelines, let’s set ours up. My pipelines pull source from GitHub using a Source activity, and then I’ll pass that as an input to my scan CodeBuild project.
Here’s the CloudFormation for the CodeBuild project. I’m relying on existing roles that I have, and I wont cover off what’s required for that here, but essentially that role can be assumed by CodeBuild, it has access to the Pipeline S3 bucket, and can log to CloudFormation.
"SecurityCodebuildProject": {
"Type": "AWS::CodeBuild::Project",
"DependsOn": "CodeBuildDevServiceRole",
"Properties": {
"Name": {
"Fn::Sub": "${ServiceName}-security"
},
"Description": {
"Fn::Sub": "Manages security scanning for builds in all clouds"
},
"ServiceRole": {
"Ref": "CodeBuildDevServiceRole"
},
"Artifacts": {
"Packaging": "NONE",
"Type": "CODEPIPELINE",
"Name": {
"Fn::Sub": "${ServiceName}-dev"
}
},
"TimeoutInMinutes": 60,
"Environment": {
"ComputeType": "BUILD_GENERAL1_SMALL",
"PrivilegedMode": false,
"Image": "aws/codebuild/ruby:2.5.1",
"Type": "LINUX_CONTAINER"
},
"Source": {
"BuildSpec": "deployment/buildspec-security.yml",
"Type": "CODEPIPELINE"
}
}
}
Two things to note here, the environment object covers off the Ruby CodeBuild container that I’m going to use, and my Source shows the buildspec coming from the deployment folder, and the code will come from CodePipeline.
Finally, I’ll add the CodeBuild project to my existing pipeline, adding it between my Source and Build stages.
{
"Name": "Secure",
"Actions": [
{
"InputArtifacts": [
{
"Name": {
"Fn::Sub": "${ServiceName}-public-src"
}
}
],
"Name": "StackScan",
"ActionTypeId": {
"Category": "Build",
"Owner": "AWS",
"Version": "1",
"Provider": "CodeBuild"
},
"Configuration": {
"ProjectName": {
"Ref": "SecurityCodebuildProject"
}
},
"RunOrder": 1
}
]
}
And that’s it! We’ll deploy this CloudFormation as an update to my CICD stack, commit the buildspec to my repository, and then let it run.
Here’s my results:
That’s it! It took me about an hour to figure out how to do this smoothly and another hour or so get it implemented and tested. This is nice and portable across all my micro services, and sets me up for success in the future to do more security work in that docker image.
Hopefully this gets you off in the right direction for automating your security processes in the cloud.
As always, if you have tweaks or suggestions on this work, I’d love to hear from you. Drop me an email!
Until next time,
James.