Picture this: You’re wrestling with complex configurations in your AWS CDK project, drowning in a sea of settings and validations. Enter Pydantic, your new best friend in the world of Python data validation. Let’s dive into how this powerful combo can revolutionize your cloud development workflow!

Pydantic has a few key features that make it a great choice for managing settings in AWS CDK:

1. Type validation: Pydantic provides runtime type checking for data structures, which can help catch errors early in the development process.
2. Settings management: Pydantic provides a way to define settings in a structured way, which can be easily managed and validated.
3. Data serialization: Pydantic can be used to serialize and deserialize data, which can be useful when working with AWS CDK constructs.

The AWS CDK comes with its own way of managing settings using the cdk.json file. This can be a great starting point, but as the complexity of the settings grows in real world production environments, it can become difficult to manage and you end up writing a lot of custom code to validate and glue everything together.

Let's see how we can define settings using Pydantic with a simple example:

from pydantic import (
    BaseModel,
    Field,
)

from pydantic_settings import BaseSettings, SettingsConfigDict


class DbSettings(BaseModel):
    host: str = "db.foo.example.com"
    port: int = 5432


class Settings(BaseSettings):
    namespace: str = Field(default="myapp", description="Namespace")
    environment: str = Field(default="dev", description="Environment")
    account: str = Field(..., description="Account Id")
    region: str = Field(..., description="AWS Region")
    # api_key: str = Field(..., description="API Key")
    db: DBSettings = DBSettings(host='dev.example.com', port=5432)

    model_config = SettingsConfigDict(env_prefix="MY_")


# We could load the settings from a file, but for simplicity we'll just define them here
def load_settings() -> dict:
    return 


settings = Settings(**load_settings())

print(settings.model_dump_json(indent=2))
""""

"""

This is just the tip of the iceberg. See the Pydantic documentation for more information on how to use all the features of Pydantic.

Lets take this a step further and see how we can use these settings with AWS CDK by loading a yaml configuration file and building a simple stack.

A full working example can be found here but let's walk through the steps to set it up.

• Setup the AWS CDK environment by following the official documentation.
• Make sure you have the latest python version installed on your machine. The latest and greatest way to manage python is to use uv but we'll stick with the basics for now.

Check you have the prerequisites installed:

python --version
pip --version
cdk --version

Create a new directory for the project:

mkdir myapp
cd myapp

Now, let's initialize a new CDK project:

cdk init app --language python

Activate the virtual environment:

source .venv/bin/activate

Next, we will install the required dependencies:

pip install pyyaml pydantic_settings -r requirements.txt

Now we'll add a dev.yaml file to the config dir of the project(replace the values with your own):

mkdir config

cat < config/dev.yaml
namespace: myapp
environment: dev
account: '111111111111'
region: us-west-2
EOF

We'll create a settings.py file under the myapp directory to define our settings:

from pydantic_settings import BaseSettings, SettingsConfigDict, PydanticBaseSettingsSource
from typing import Type, Tuple

from pydantic import Field, BaseModel

class DBSettings(BaseModel):
    host: str = Field(..., description="Database Host")
    port: int = Field(..., description="Database Port")

class Settings(BaseSettings):
    # this sets the order of precedence for settings sources
    # the first source in the return list has the highest precedence
    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: Type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> Tuple[PydanticBaseSettingsSource, ...]:
        return env_settings, init_settings, file_secret_settings

    namespace: str = Field(default="myapp", description="Namespace")
    environment: str = Field(default="dev", description="Environment")
    account: str = Field(..., description="Account Id")
    region: str = Field(..., description="AWS Region")
    # api_key: str = Field(..., description="API Key")
    db: DBSettings = DBSettings(host='dev.example.com', port=5432)

    model_config = SettingsConfigDict(env_prefix="MY_")

In our app.py file, we'll load the settings from the yaml file and create a simple stack:

#!/usr/bin/env python3
import os

import aws_cdk as cdk
import yaml

from myapp.myapp_stack import MyappStack
from myapp.settings import Settings

config_file = os.getenv('CONFIG_FILE', 'dev.yaml')

def load_config_file(config_file: str ) -> dict:

    with open(config_file, "r") as stream:
        yaml_data = yaml.load(stream, yaml.SafeLoader)
        return yaml_data


settings = Settings(**load_config_file(config_file))

print(f"Settings: ")

app = cdk.App()

stack_name = f"--stack"

MyappStack(app, stack_name,
    settings=settings,
    )

app.synth()

Finally, we'll create a stack in the `myapp_stack.py` file:

from aws_cdk import (
    Stack,
    CfnWaitConditionHandle,
    Tags
)
from constructs import Construct

from myapp.settings import Settings

class MyappStack(Stack):

    def __init__(self, scope: Construct, construct_id: str, settings: Settings,  **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # The code that defines your stack goes here

        # NullResource
        CfnWaitConditionHandle(self, "NullResource")

        # We can use the settings object to access the values
        # This is a basic example but we can use it to make decisions in our stack
        Tags.of(self).add("Namespace", settings.namespace)

Now that we have everything set up, we can run some command. Lets start by doing a `cdk ls` to list the stacks:

CONFIG_FILE=./config/dev.yaml cdk ls
Settings: namespace='myapp' environment='dev' account='123456789012' region='us-west-2' db=DBSettings(host='dev.example.com', port=5432)
myapp-dev-stack

Lets synthesize the stack:

CONFIG_FILE=./config/dev.yaml cdk synth
Settings: namespace='myapp' environment='dev' account='123456789012' region='us-west-2' db=DBSettings(host='dev.example.com', port=5432)
Resources:
  NullResource:
    Type: AWS::CloudFormation::WaitConditionHandle
    Metadata:
      aws:cdk:path: myapp-dev-stack/NullResource
...

And finally, deploy the stack:

CONFIG_FILE=./config/dev.yaml cdk deploy

Thats our deployment done for dev but we can also deploy for prod by creating a new config file, changing the `CONFIG_FILE` environment variable to `prod.yaml` and running the commands again. eg.

CONFIG_FILE=./config/prod.yaml cdk ls
Settings: namespace='myapp' environment='prod' account='222222222222' region='us-west-2' db=DBSettings(host='dev.example.com', port=5432)
myapp-prod-stack

Notice the different account and region values in the output. This makes it easy to manage different environments with different settings.

One final note, we can override the settings in the yaml file by setting environment variables with the `MY_` prefix. For example, to override the `namespace` value, we can set the `MY_NAMESPACE` environment variable.

MY_NAMESPACE=foo CONFIG_FILE=./config/dev.yaml cdk ls
foo-dev-stack

So there you have it, a simple way to manage settings in AWS CDK using Pydantic. This is just the beginning, and there are many more advanced features in Pydantic that can be used to make your life easier when working with AWS CDK.