Aws Dynamodb For Serverless Microservices

DynamoDB is the Serverless NoSQL Database offering by AWS. Being Serverless makes it easier to consider AWS DynamoDB for Serverless Microservices since it goes inline with the patterns and practices while designing serverless architectures in AWS.

Are you still confused what a Serverless NoSQL Database means? Let me give you a quick overview. We use the term Serverless when we don’t need to manage any servers (Software Updates, OS Patching, OS Security & etc.), rather someone else manages them for us and provides an abstracted view. When it comes to DynamoDB, AWS manages the underlying infrastructure, software and provides us an abstract view of Tables, Indexes (GSI, LSI), Throughput, Auto Scaling and Security Policies, which consists of high-level constructs for the NoSQL database. The below diagram shows AWS DynamoDB configuration patterns.

AWS DynamoDB configurations

This article provides an overview of the principles, patterns and best practices while using AWS DynamoDB for Serverless Microservices.

Principles in Using AWS DynamoDB

AWS DynamoDB is more suited for storing JSON documents and use as a storage for key-value pairs. Having multiple types of indexes as well as multiple types of query possibilities makes it convenient to be used for different types of storage and query requirements. However, it is important to understand that DynamoDB is a NoSQL database which is difficult to be compared with a Relational Database, side by side. This also makes it really difficult to for a person who is coming from a Relational Database background to design DynamoDB tables. Therefore it is important to understand several underlying principles in using DynamoDB. The following list contains 12 principals I follow when designing DynamoDB tables and queries.

  1. Use GUID’s or Unique Attributes, instead of incremental IDs.
  2. Don’t try to normalize your tables.
  3. Having duplicate attributes in multiple tables is fine as long as you have implemented ways to synchronize the changes.
  4. Keeping pre-computed data upon updates is efficient with DynamoDB if you need to query them often.
  5. Don’t try to keep many relationships across tables. This will end up needing to query multiple tables to retrieve required attributes.
  6. Embrace eventual consistency.
  7. Design your transactions work with conditional writes.
  8. Design your tables, attributes, and indexes thinking of the nature of queries.
  9. Use DynamoDB triggers and streams to propagate changes and design event-driven data flows.
  10. Think about item sizes and using indexes effectively when listing items to minimize throughput requirements.
  11. Think about the growth of attribute data, to design whether to store them as a nested object or use a different table for it.
  12. Avoid using DynamoDB Scan operation whenever possible.

Patterns for Serverless Microservices

AWS DynamoDB is used for Serverless Microservices with different configuration patterns for various use cases.

Direct Access from RESTful API

Direct Access from Restful API

The most common pattern for Serverless Microservices is to connect DynamoDB to an API Endpoint Code (Inside AWS Lambda) which is invoked through AWS API Gateway. It is also possible to directly connect DynamoDB to API Gateway if the Microservice offers support for direct DynamoDB queries.

Note: It is also possible to invoke AWS Lambda as a RESTful endpoint if the client has AWS IAM credentials or AWS STS temporary credentials.

Event-Driven Updates

Event Driven Updates

DynamoDB also can be updated, based on events other than Direct Access from RESTful API. For example, DynamoDB can be used to store metadata of files uploaded to Amazon S3. Using S3 Upload Trigger, Lambda function can be invoked upon file upload which is able to update the DynamoDB table. A similar approach can be used to perform DynamoDB updates in response to  Amazon SNS.

Data Synchronization Between Microservices

Data Syncronizations between microservices

If there are same attributes stored in multiple Microservices DynamoDB tables, you can use Amazon Simple Notification Service (SNS) Topics. Using Amazon SNS it is possible to inform attribute changes from one service to another without each of them knowing each other.

For example, let’s say Service #1 Company Profile Table and Service #2 Company Statistics Table shared company name attribute. If the company name is modified in Service #1 that change needs to be propagated to Service #2 Company Statistics Table. Knowing these requirements, it is possible for Service #1 to publish the attribute change using DynamoDB Streams and a Lambda function to the SNS topic. When the change happens the Lambda function in Service #2 subscribed to the topic will update the Company Statistics Table.

Conclusion

If you are new to AWS DynamoDB, it is important to understand its capabilities and limitations before moving into the database design.  It is equally important to have a proper mindset to design the data model using NoSQL principals and configuration patterns. This will include unlearning some of the concepts learned from Relational Database Design. In addition, using DynamoDB can be really challenging for some use cases. If you are struggling to think of how to update multiple tables concurrently, querying multiple tables or limitations of indexes for your use case, these can be hints to revisit the original decision to use DynamoDB in the first place. However, AWS DynamoDB is an integral part of the AWS Serverless Technology Stack which still remains as the leading Serverless NoSQL database in AWS.

—END —

Share your views with us and don’t forget to follow us on Twitter and Medium  @totalcloudio.

Editorial Note: This awesome guest blog post is written by Ashan Fernando, Amazon Certified Solutions Architect Professional

Aws Dynamodb For Serverless Microservices

Smart Scheduling at your fingertips

Go from simple to smart, real-time AWS resource scheduling to save cost and increase team productivity.

Learn More
More Posts

You Might Also Like

AWS Tips & Tricks
Componentized Cloud Management: The way ahead for Cloud Automation
When something gets complex, our primary approach is to break it down — even cloud management. If you’re a part of a growing company that uses the cloud, you can see your infrastructure becoming more…
May 29, 2020
Sayonee
AWS Tips & Tricks
Cost Optimization with AWS Serverless Resource Scheduling
You must be aware of EC2 scheduling and its benefits on cost optimization. However, scheduling doesn't need to stop at just EC2 or RDS. There are plenty of other AWS serverless resources that can be scheduled to save costs. While the traditional way might be tedious, Totalcloud provides a safe alternative.
May 28, 2020
Hrishikesh
AWS Use Case Files
Deploying an EKS Cluster With TotalCloud's Code-Free Workflows
Totalcoud workflows can be used for many creative applications. One such application was developed as part of a customer request. With simply 2 workflows, we removed the hassle of provisioning your EKS clusters on AWS. What normally takes grueling efforts of scripting or configuring have now been reduced to just a few clicks.
May 28, 2020
Hrishikesh
AWS Tips & Tricks
5 Best Practices for Tagging AWS Resources
Tagging AWS resources is a simple concept that can come with a bunch of different benefits when used appropriately. Here are the 5 best practices on how you can make the most out of your AWS EC2 tags. Also, learn the common mistakes you could make and how to avoid it.
May 12, 2020
Hrishikesh
AWS Tips & Tricks
Helpful Tips for EC2 Rightsizing
Optimize your cloud costs and boost performance with these tips for rightsizing. Here we go through all the different methods for rightsizing and the approach you need to follow to make sure you are constantly aware of the changing demands in your environment.
May 6, 2020
Hrishikesh
Cost Analyzer
Instance Comparison Chart
Use this new Instance Comparison chart provided by Totalcloud to find the right AWS EC2 Instance that matches your requirements and budget. With several options of sorting and filtering, you can narrow down your optimal machines easily.
April 30, 2020
Hrishikesh