Updating an old CDK Application and components

Hi to everyone,

In today’s episode of cleaning up old and forgotten code, I will be focusing on a recent issue that was raised with me when an application developed in 2020 using AWS CDK, Python and other bits and bobs was no longer functioning when deploying to a new customer.

To say I had to relearn AWS CDK in a hurry is an understatement 🙂 

For those who are familiar with CDK you might look at the fact it was originally on AWS CDK 1.46.0 version to know how old the code is. So what did I need to in order to make the application work?

1. Fix the AWS CDK

Technically AWS CDK 1.46 is not broken but it is so old that some of the more recent changes to AWS services are not there and some of the improvements in the way AWS API Gateway’s are deployed are missing.

I decided I had to bring the AWS CDK to the current version. So I updated the package.json for the application and replaced the first graphic with the second graphic

AWS CDK 1.x required that you define every component you might use in the package.json so you can see since the application touches a lot of AWS Services it has a lot of specific definitions.

If you also look at other areas you will notice that there are aged components everywhere (node JS versions, not supported, Typescript dependencies that are years out of date and so on).

So I decided to go through and update all of that. Doing it like that has risk but if you work through the testing appropriately you can mitigate issues before releasing back to production.

I went and sought out the latest version (I know it has had a minor update since I did this) and changed to AWS CDK 2.x (2.95.0). I also updated all other dependencies to supported versions based on the available NPM packages at the time.

It’s obviously not that easy to simply switch over from AWS CDK 1.x to 2.x. You also need to update the import statements. I tend to prefer single line import statements (not everyone does) but I used this sort of structure to import using the same descriptive function name so that I didnt have to go through every line of code.

It looks something like this (replacing each individual import with a single statement for each feature from the aws-cdk-lib module. 

Note: Construct is no longer part of the CDK Core module as of CDK 2.x (I assume most know this already but I want to ensure that I help those who dont). It requires to be installed separately (constructs) and then imported accordingly.

 Troubleshooting Deployment Failures

Surprisingly enough (at least for me) this change went smoothly and most of the Lambda, API Gateway, RDS, Deployment components were fine. I did have to update the Codebuild image and buildspec to use a supported version of Python and Node since the versions used in the original deployment were unsupported now. Additionally I had to update the Codebuild Image to a later version to support the updated Python and Node Deployments.

AWS Provide excellent logging capabilities and the way I was able to piece through those issues were to use the tail logging of Codebuild to see all the red (red = bad 🙂 ) and then once I could understand that it was failing due to unsupported components I fixed them one at a time.

A Key thing to make sure you note is that if you are deploying CDK via Codebuild (as I did) you need to ensure that the version you deploy MATCHES or is compatible with the code you are working with. Otherwise you will try to run CDK Synth or Deploy on a function it will FAIL.

API Gateway/Lambda issues

Again as I keep pointing out in my work, logging is gold, platinum or Netherite even (for the Minecraft aficionado out there like my kids). When done right logging can tell you EVERYTHING you need to know. Especially if all the users get in response to API Gateway request is “Internal Server Error”.

I looked at the logs and what I found was that the module used to support the backend Lambda connection to PostgreSQL was unsupported, using a depreciated component of Marshmallow-sqlalchemy so that when it was trying to connect to PostgreSQL it was failing.

As partof this I started trawling through the Lambda Code and found that at least 3-4 declarations were different now and code was

Optimising Lambda

AWS Lambda has this cool feature called Layers which can be used when you need to bundle in a module or component that is not part of the internally available capability. I found that 3-4 modules that we needed were being bundled in the zip file with the code and as such that became clunky.

I literally followed the AWS documentation here (Working with AWS Lambda Layers)  and created a layer which included some key module installs such as the customised version of psycopg2 (required because of some static libraries that are missing in the AMI that Lambda uses in the backend). This was sourced from aws-psycopg2 · PyPI

 doing this reduced the bloat in the Lambda package size and improved the way the functions worked.

RDS Changes

AWS RDS PostgreSQL is deployed for this application and operates in a secure manner with TLS encryption between application and DB. The initial deployment was using version 10.x which as you can imagine is quite old. I updated the AWS CDK now deploy version 15.4 of postgreSQL and I also noted that the version of CDK no longer let me deploy db.t2.small instances and instead I was required by the version licensing to deploy db.t3.small instead.

AWS document quite well what instances you can select for a particular version of a DB engine. This is here: DB instance classes – Amazon Relational Database Service

Basically I simply had to change the instance family from t2 to t3 with no other changes in my CDK.

This required 1 line of code update (I use Typescript for CDK) and had to change BURSTABLE2 to BURSTABLE3 (as that is how CDK defines the Tx instance families):

CDK is very well documented as well so I was able to find the exact code.

Concluding

A Key caveat and I made this note clear to the responsible team was that some of the components were left in an unsupported state for so long that upgrading the Stack using CDK Deploy would possibly be unsafe for the existing deployments. I have advised them of all of the components that are updates and they can and should deploy those manually. Some such as RDS will have been upgraded from the original version by AWS Mandate (I was around when we had to previously move everything beyond version 11.x and I think we moved to 12.x at the time).

However I was able to successfully deploy from codebuild this architecture and apply updates to Lambda engines and components through code successfully. I think it is a weekend well spent (but I am a big geek)