How To Insert Multiple DynamoDB Items at Once with Boto3

Looking for an example of how to Insert Multiple Items (also known as batch) in your DynamoDB Table using Python Boto3? This is the article for you.

Note that the full code is available at the bottom of this article.

Before you start, remember that you need proper IAM Permissions in order to interact with DynamoDB. For deleting, you require the dynamodb:BatchWriteItem permision (strange, I know). Make sure that the user or role you are using has this permission. If you don’t know how to create an IAM user/role with a policy, check out my YouTube tutorial here.

Check out other Python DynamoDB Based Operation Articles here:

1. How To Query DynamoDB with Boto3
2. How To Insert Into a DynamoDB Table with Boto3
3. How To Delete an Item in a DynamoDB Table with Boto3
4. How To Delete Multiple DynamoDB Items at Once with Boto3

Setup

First we need to import boto3, which is the Python SDK that allows us to interact with DynamoDB APIs.

import boto3

Next we need to get a reference to the DynamoDB resource using the below code snippet.

Note that we are using the DynamoDB resource and not the client object. The resource object offers higher level APIs which make it easier to access It is also the more modern way of interacting with Dynamo. However, resource may not always be completely up to date and you can alternatively use the client. The client offers more verbose lower level APIs. I personally suggest you use the resource object for all your interactions with Dynamo.

You can learn more about the DynamoDB Resource and Table objects in the boto3 documentation here.

dynamodb = boto3.resource('dynamodb')

Next up we need to get a reference to our DynamoDB table using the following lines.

We’re now ready to start putting our items in batch.

For this tutorial, we are goign to use the table’s batch_writer. The batch writer is a high level helper object that handles deleting items from DynamoDB in batch for us. The batch_writer documentation describes itself as:

“This method creates a context manager for writing objects to Amazon DynamoDB in batch. The batch writer will automatically handle buffering and sending items in batches. In addition, the batch writer will also automatically handle any unprocessed items and resend them as needed. All you need to do is call put_item for any items you want to add, and delete_item for any items you want to delete.”

We’re going to be using put_item for this demonstration to add items at once to Dynamo.

Now before we can go ahead and formulate our input objects, you need to know if your table structure uses either a) just a partition key or b) a partition key + a sort key. Reason being that in order to insert an item that has a partition key, all you need to do is pass the partition key value in the request. However, if your table uses a partition key + sort key combination, you MUST also provide the sort key value in addition to the partition key value. This is because when using a PK + SK combination, multiple records can share the same PK. Therefore, in order to uniquely identify the record you want to insert, you must supply the sort key value as well.

In my example below, the CustomerID is the partition key and the OrderID is the sort key on my table.

Forming our Input Object

We need to form our input objects as independent python dictionary objects. Mine look like the following:

item_1 = {"CustomerID":"c-1", "OrderID":"o-1"} 
item_2 = {"CustomerID":"c-1", "OrderID":"o-2"}

Add the items to a list so we can pass them to our batch_writer.

items_to_add = [item_1, item_2]

We’re ready now to perform the deletion.

Using Batch Writer To Insert Our Objects

We use the following code:

with table.batch_writer() as batch:
    for item in items_to_add:
        response = batch.put_item(Item={
            "CustomerID": item["CustomerID"],
            "OrderID": item["OrderID"]
        })

What’s happening here is that we’re using the batch object’s put_item method for every item in our list. Behind the scenes, the batch_writer combines multiple putitem requests into a single BatchWriteItem API call to make to AWS.

I verified this by looking at the CloudTrail logs after I ran the code above. The output from CloudTrail was:

{
   "Records":[
      {
         "eventTime":"2022-10-02T21:05:12Z",
         "eventSource":"dynamodb.amazonaws.com",
         "eventName":"BatchWriteItem",
         "awsRegion":"us-east-1",
         "userAgent":"Boto3/1.24.84 Python/3.10.2 Windows/10 Botocore/1.27.84 Resource",
         "requestParameters":{
            "requestItems":[
               {
                  "operation":"Put",
                  "tableName":"CustomerOrders",
                  "key":{
                     "OrderID":"o-1",
                     "CustomerID":"c-1"
                  },
                  "items":[
                     "OrderID",
                     "CustomerID"
                  ]
               },
               {
                  "operation":"Put",
                  "tableName":"CustomerOrders",
                  "key":{
                     "OrderID":"o-2",
                     "CustomerID":"c-2"
                  },
                  "items":[
                     "OrderID",
                     "CustomerID"
                  ]
               }
            ]
         }
      }
   ]
}

Notice that there is just a single API call with the eventName field set to BatchWriteItem.

You can go ahead and check your table now to confirm the items are inserted.

Mission accomplished! Here’s the entire code snippet for you to use.

import boto3

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('CustomerOrders')

item_1 = {"CustomerID":"c-1", "OrderID":"o-1"}
item_2 = {"CustomerID":"c-2", "OrderID":"o-2"}

items_to_delete = [item_1, item_2]

with table.batch_writer() as batch:
    for item in items_to_delete:
        response = batch.put_item(Item={
            "CustomerID": item["CustomerID"],
            "OrderID": item["OrderID"]
        })