AWS Lambda and Secrets Manager Tutorial in Python

Daniel

2 years ago

Learn how to use a Lambda Function to access a secret stored in Secrets Manager with boto3 and Python.

Introduction

In this article, I’m going to show you how to access secrets stored in the Amazon Secrets Manager service from an AWS Lambda function. Secrets Manager is a managed AWS service that allows you to store hard-coded API keys, database usernames, passwords, or any secret string values instead of hard-coding them into your source code.

As a best practice, Secrets Manager should be your go-to way to handle these sensitive values being used in any application. To learn more about the other types of features Secrets Manager offers, check out this link over at AWS.

To get started, we first need to create our secret in Secrets Manager.

Prefer video content? Check out my video tutorial on this topic on YouTube.

Step 1 – Creating Our Secret

To create a secret, search for the Secrets Manager service in the AWS console. The home screen should look like this:

Before we begin, note that there is a 40-cent cost per month for each secret you store within Secrets Manager.

However, if this is your first time using Secrets Manager on your AWS account, you are automatically eligible for the free trial of the service. This allows you to create, retrieve, and rotate secrets for 30 days without being charged a penny.

Note that the 30 day period starts when you create your first secret on your account. There’s nowhere in the AWS console (that I know of at least) where you can see the expiry date so you may want to write it down.

Click “Store a new secret” and select the secret type that you want to store. If you’re using Amazon RDS, DocumentDB, or Redshift, I highly recommend selecting one of those options because you will also get automatic secret rotation as an added benefit. This adds an extra layer of security on top of your storage.

However, if you’re storing a secret value in plain text, like an API key for GitHub, you’ll want to use the “Other type of secret” option. In this demo, we will use the “Other type of secret”.

*Choosing a Secret Type. Some services like RDS, DocumentDB, and Redshift offer special features like automatic credential rotation.*

Key-Value Pairs

Once you’ve chosen the “Other type of secret” option, you’ll see options asking you to set key-value pairs.

You can store a JSON object with the key name and key password, or you can just put it in plain text for a single row. For this case, I’m going to use the dialogue box and call my key “secret” and value “password” as seen in the image below.

*Setting our key value pairs in our Secrets Manager secret.*

You’ll also notice you have the option of specifying an Encryption key. When you first create an AWS account, AWS automatically creates KMS Keys which will manage the key used to encrypt your data.

You can choose to use the AWS managed key or create your own which you can assign permissions to. For now, we’ll leave this as using the AWS managed KMS key.

Hit “Next” now, and let’s call the secret aws-managed-secret. I’d recommend putting this on your clipboard so you don’t forget it and then hit “Next.”

On the next page, you’ll be asked whether you’d like to enable key rotation as seen below. This is an optional feature that can allow you to automatically rotate the contents of your secret on certain intervals. You can choose to enable this but be warned you’ll need to implement a Lambda Function that handles the rotation. Tread carefully.

*Optional when creating our secret, we can configure automatic rotation.*

Proceed through the wizard and complete creating your secret. Note the ARN (Amazon Resource Name) of your secret on the summary page. Keep this value handy as we’ll need it when we implement the code for our Lambda function.

Step 2 – Accessing Our Secret From AWS Lambda With Python Boto3

Note that your Lambda function requires permissions to access secrets manager and to decrypt the value using a KMS key. You’ll need to apply the following permissions to your Lambda function.

secretsmanager:GetSecretValue
kms:Decrypt

In your Lambda function, we’ll need to add the following pieces of code. The complete code block is located at the bottom of this article.

First, we need to import some dependencies including json, boto3, and base64.

On line 6, we store the name of our secret in a variable called secret_name. Make sure to replace this with your secret name from Step 1.

We’ll also specifyour region in the region_name variable. Do note that Secrets Manager is region specific. Make sure to use the region value where your secret is currently located.

import json
import boto3
import base64

# Your secret's name and region
secret_name = "aws-managed-secret"
region_name = "us-east-1"

Our next step is create a boto3 session and establish our secrets manager client. We pass in the name of the service and the region for the client as seen below.

#Set up our Session and Client
session = boto3.session.Session()
client = session.client(
    service_name='secretsmanager',
    region_name=region_name
w)

We now define the function that acts as the entry point for our Lambda function called lambda_handler. In it, we call secrets manager using the get_secret_value method from our client. We pass in our secret_name under the SecretId field.

We store the response in a variable and print it out. Note that printing out your secret value is a really bad idea in any real code – but its useful for debugging.

def lambda_handler(event, context):

    # Calling SecretsManager
    get_secret_value_response = client.get_secret_value(
        SecretId=secret_name
    )

    #Raw Response
    print(get_secret_value_response)

We can extract the key/value from the secret by accessing the SecretString field from the response object as seen below.

    #Extracting the key/value from the secret
    secret = get_secret_value_response['SecretString']
    print(secret)

Calling print(secret) yields the key and value we stored in SecretsManager. Congratulations, you’re done!

The complete code snippet is available below.

import json
import boto3
import base64

# Your secret's name and region
secret_name = "aws-managed-secret"
region_name = "us-east-1"

#Set up our Session and Client
session = boto3.session.Session()
client = session.client(
    service_name='secretsmanager',
    region_name=region_name
)

def lambda_handler(event, context):

    # Calling SecretsManager
    get_secret_value_response = client.get_secret_value(
        SecretId=secret_name
    )
    
    #Raw Response
    print(get_secret_value_response)
   
    #Extracting the key/value from the secret
    secret = get_secret_value_response['SecretString']
    print(secret)