Terraform Modules Best Practices: Separating Lambda, SSM, and Layers

Byjaganrajagopalme

Imagine you’re building a car. You don’t build the entire car as one solid piece. You build separate parts: the engine, the wheels, the seats, and then you put them together. Terraform modules are exactly like that.

Terraform Modules

What are these “Modules”?

  1. Lambda Function Module (lambda/): This is the Engine of your car.
    • It’s the actual code that does the work (like processing an image, saving data, or sending an email).
    • The lambda_function.py file is like the engine’s instruction manual.
  2. SSM Parameter Module (ssm-parameter/): This is the Secret Glove Compartment.
    • It’s a secure place to store important secrets and configuration details (like database passwords, API keys, and URLs).
    • Instead of writing the password directly in your engine’s manual (very unsafe!), the engine just knows how to open the glove compartment to get the secret when it needs it.
  3. Lambda Layer Module (lambda-layer/): This is the Toolkit in the Trunk.
    • It contains common tools (software libraries) that multiple engines (Lambda functions) might need.
    • Instead of putting a copy of the same toolkit in every single car, you just have one toolkit in the trunk that all engines can share. This saves space and makes updates easy.
terraform/
├── main.tf
├── variables.tf
├── outputs.tf
├── modules/
│   ├── lambda/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   └── lambda_function.py
│   ├── ssm-parameter/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   └── lambda-layer/
│       ├── main.tf
│       ├── variables.tf
│       ├── outputs.tf
│       └── requirements.txt
└── examples/
    └── complete/
        ├── main.tf
        ├── variables.tf
        └── outputs.tf

1. Root Module Files

terraform/main.tf

terraform {
  required_version = ">= 1.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

# Create SSM parameters
module "ssm_parameters" {
  source = "./modules/ssm-parameter"
  
  parameters = var.ssm_parameters
}

# Create Lambda layer
module "lambda_layer" {
  source = "./modules/lambda-layer"
  
  layer_name          = var.layer_name
  compatible_runtimes = var.compatible_runtimes
  description         = var.layer_description
}

# Create Lambda function
module "lambda_function" {
  source = "./modules/lambda"
  
  function_name          = var.function_name
  runtime                = var.runtime
  handler                = var.handler
  source_code_path       = var.source_code_path
  layers                 = [module.lambda_layer.layer_arn]
  environment_variables  = var.environment_variables
  ssm_parameter_arns     = module.ssm_parameters.parameter_arns
  
  depends_on = [
    module.ssm_parameters,
    module.lambda_layer
  ]
}

terraform/variables.tf

variable "aws_region" {
  description = "AWS region"
  type        = string
  default     = "us-east-1"
}

variable "function_name" {
  description = "Name of the Lambda function"
  type        = string
  default     = "my-lambda-function"
}

variable "runtime" {
  description = "Lambda runtime"
  type        = string
  default     = "python3.9"
}

variable "handler" {
  description = "Lambda handler"
  type        = string
  default     = "lambda_function.lambda_handler"
}

variable "source_code_path" {
  description = "Path to Lambda source code"
  type        = string
  default     = "./modules/lambda/lambda_function.py"
}

variable "layer_name" {
  description = "Name of the Lambda layer"
  type        = string
  default     = "my-lambda-layer"
}

variable "compatible_runtimes" {
  description = "List of compatible runtimes for the layer"
  type        = list(string)
  default     = ["python3.9"]
}

variable "layer_description" {
  description = "Description of the Lambda layer"
  type        = string
  default     = "Custom Lambda layer with dependencies"
}

variable "environment_variables" {
  description = "Environment variables for Lambda function"
  type        = map(string)
  default     = {}
}

variable "ssm_parameters" {
  description = "Map of SSM parameters to create"
  type = map(object({
    value       = string
    description = string
    type        = string
  }))
  default = {
    "database_url" = {
      value       = "postgresql://localhost:5432/mydb"
      description = "Database connection URL"
      type        = "SecureString"
    }
    "api_key" = {
      value       = "secret-api-key"
      description = "API key for external service"
      type        = "SecureString"
    }
  }
}

terraform/outputs.tf

output "lambda_function_arn" {
  description = "ARN of the Lambda function"
  value       = module.lambda_function.lambda_arn
}

output "lambda_function_name" {
  description = "Name of the Lambda function"
  value       = module.lambda_function.lambda_name
}

output "lambda_layer_arn" {
  description = "ARN of the Lambda layer"
  value       = module.lambda_layer.layer_arn
}

output "ssm_parameter_arns" {
  description = "ARNs of the SSM parameters"
  value       = module.ssm_parameters.parameter_arns
}

output "lambda_invoke_arn" {
  description = "Invoke ARN of the Lambda function"
  value       = module.lambda_function.lambda_invoke_arn
}

2. Lambda Module

modules/lambda/main.tf

resource "aws_lambda_function" "this" {
  filename      = var.source_code_path
  function_name = var.function_name
  role          = aws_iam_role.lambda_role.arn
  handler       = var.handler
  runtime       = var.runtime
  layers        = var.layers

  environment {
    variables = merge(var.environment_variables, var.ssm_parameter_arns)
  }

  source_code_hash = filebase64sha256(var.source_code_path)

  tags = var.tags
}

resource "aws_iam_role" "lambda_role" {
  name = "${var.function_name}-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "ssm_access" {
  name = "${var.function_name}-ssm-access"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ssm:GetParameter",
          "ssm:GetParameters"
        ]
        Resource = values(var.ssm_parameter_arns)
      }
    ]
  })
}

modules/lambda/variables.tf

variable "function_name" {
  description = "Name of the Lambda function"
  type        = string
}

variable "runtime" {
  description = "Lambda runtime"
  type        = string
}

variable "handler" {
  description = "Lambda handler"
  type        = string
}

variable "source_code_path" {
  description = "Path to Lambda source code"
  type        = string
}

variable "layers" {
  description = "List of Lambda layer ARNs"
  type        = list(string)
  default     = []
}

variable "environment_variables" {
  description = "Environment variables for Lambda function"
  type        = map(string)
  default     = {}
}

variable "ssm_parameter_arns" {
  description = "ARNs of SSM parameters for Lambda access"
  type        = map(string)
  default     = {}
}

variable "tags" {
  description = "Tags to apply to resources"
  type        = map(string)
  default     = {}
}

modules/lambda/outputs.tf

output "lambda_arn" {
  description = "ARN of the Lambda function"
  value       = aws_lambda_function.this.arn
}

output "lambda_name" {
  description = "Name of the Lambda function"
  value       = aws_lambda_function.this.function_name
}

output "lambda_invoke_arn" {
  description = "Invoke ARN of the Lambda function"
  value       = aws_lambda_function.this.invoke_arn
}

output "lambda_role_arn" {
  description = "ARN of the Lambda IAM role"
  value       = aws_iam_role.lambda_role.arn
}

modules/lambda/lambda_function.py

import json
import boto3
import os

def lambda_handler(event, context):
# Example of using SSM parameters
ssm_client = boto3.client(‘ssm’)

# Get database URL from SSM
database_url = ssm_client.get_parameter(
    Name='database_url',
    WithDecryption=True
)['Parameter']['Value']

# Get API key from SSM
api_key = ssm_client.get_parameter(
    Name='api_key',
    WithDecryption=True
)['Parameter']['Value']

return {
    'statusCode': 200,
    'body': json.dumps({
        'message': 'Lambda executed successfully',
        'database_url': database_url,
        'api_key': api_key
    })
}

3. SSM Parameter Module

modules/ssm-parameter/main.tf

resource "aws_ssm_parameter" "this" {
  for_each = var.parameters

  name        = each.key
  description = each.value.description
  type        = each.value.type
  value       = each.value.value

  tags = var.tags
}

modules/ssm-parameter/variables.tf

variable "parameters" {
  description = "Map of SSM parameters to create"
  type = map(object({
    value       = string
    description = string
    type        = string
  }))
  default = {}
}

variable "tags" {
  description = "Tags to apply to resources"
  type        = map(string)
  default     = {}
}

modules/ssm-parameter/outputs.tf

output "parameter_arns" {
  description = "ARNs of the created SSM parameters"
  value       = { for k, v in aws_ssm_parameter.this : k => v.arn }
}

output "parameter_names" {
  description = "Names of the created SSM parameters"
  value       = { for k, v in aws_ssm_parameter.this : k => v.name }
}

4. Lambda Layer Module

modules/lambda-layer/main.tf

resource "null_resource" "install_dependencies" {
  triggers = {
    requirements = filemd5("${path.module}/requirements.txt")
  }

  provisioner "local-exec" {
    command = "mkdir -p python && pip install -r requirements.txt -t python/"
  }
}

data "archive_file" "layer" {
  depends_on = [null_resource.install_dependencies]

  type        = "zip"
  source_dir  = "${path.module}/python"
  output_path = "${path.module}/layer.zip"
}

resource "aws_lambda_layer_version" "this" {
  layer_name          = var.layer_name
  description         = var.description
  filename            = data.archive_file.layer.output_path
  source_code_hash    = data.archive_file.layer.output_base64sha256
  compatible_runtimes = var.compatible_runtimes

  tags = var.tags
}

modules/lambda-layer/variables.tf

variable "layer_name" {
  description = "Name of the Lambda layer"
  type        = string
}

variable "compatible_runtimes" {
  description = "List of compatible runtimes"
  type        = list(string)
  default     = ["python3.9"]
}

variable "description" {
  description = "Description of the layer"
  type        = string
  default     = "Custom Lambda layer"
}

variable "tags" {
  description = "Tags to apply to resources"
  type        = map(string)
  default     = {}
}

modules/lambda-layer/outputs.tf

output "layer_arn" {
  description = "ARN of the Lambda layer"
  value       = aws_lambda_layer_version.this.arn
}

output "layer_version" {
  description = "Version of the Lambda layer"
  value       = aws_lambda_layer_version.this.version
}

modules/lambda-layer/requirements.txt

requests==2.31.0
boto3==1.28.0

5. Example Usage

examples/complete/main.tf

module "lambda_infrastructure" {
  source = "../.."

  function_name = "example-lambda-function"
  runtime       = "python3.9"
  handler       = "lambda_function.lambda_handler"

  ssm_parameters = {
    "database_url" = {
      value       = "postgresql://example.com:5432/mydb"
      description = "Production database URL"
      type        = "SecureString"
    }
    "api_key" = {
      value       = "prod-api-key-123"
      description = "Production API key"
      type        = "SecureString"
    }
  }

  environment_variables = {
    ENVIRONMENT = "production"
    LOG_LEVEL   = "INFO"
  }
}

Usage Instructions

  1. Initialize Terraform:
cd terraform
terraform init

2.Plan the deployment:

terraform plan

3.Apply the configuration:

terraform apply

4.Test the example:

cd examples/complete
terraform init
terraform plan
terraform apply

This modular structure provides:

  • Separation of concerns with dedicated modules for each AWS service
  • Reusability – each module can be used independently
  • Maintainability – easy to update individual components
  • Scalability – can easily add more parameters, layers, or functions
  • Security – proper IAM roles and SSM parameter types

The Lambda function can access SSM parameters securely through the provided IAM permissions, and the layer provides common dependencies that can be shared across multiple functions.

How Does It All Connect?

The main.tf file in the root directory is like the Chief Engineer. It does three things:

  1. It tells the Secret Glove Compartment team (module "ssm_parameters") what secrets to create.
  2. It tells the Toolkit team (module "lambda_layer") what tools to pack.
  3. Finally, it tells the Engine team (module "lambda_function"): “Here is your engine code. Use the toolkit from the trunk and remember, you have permission to access the secret glove compartment.”

The depends_on instruction is the Chief Engineer making sure the glove compartment and toolkit are built before the engine is installed, so everything works right away.

For the Business: The Value Proposition

This Terraform structure isn’t just technical jargon; it solves real business problems and provides tangible value.

Business Use Cases

  1. Secure Handling of Sensitive Data (SSM Module):
    • Scenario: Your application needs to connect to a paid database service (like AWS RDS) or an external API (like Twilio for SMS or SendGrid for email). These services require passwords and keys.
    • Problem: Hard-coding these secrets into application code is a major security risk. If the code is ever leaked, your company’s assets and customer data are exposed. This can lead to massive fines and loss of trust.
    • Solution: The SSM Parameter module securely stores these secrets in a dedicated, encrypted AWS service. Developers never see the actual production passwords. The Lambda function gets temporary permission to read them only when it runs. This is a fundamental security best practice.
  2. Code Reuse & Faster Development (Lambda Layer Module):
    • Scenario: Your company has five different Lambda functions that all need to connect to your database in the same way. Or they all need to use the same in-house library for logging or analytics.
    • Problem: Without layers, each team would copy-paste the same connection code or library into their own function. This creates:
      • Wasted Effort: Duplication of work.
      • Inconsistency: One team updates their connection logic, but the others don’t.
      • Bloat: Each function is larger than it needs to be, which can slightly increase running costs.
    • Solution: The Lambda Layer module packages this common code once. Every function can use the same centralized, company-approved version. Updates are made in one place and automatically available to all functions, ensuring consistency and saving developer time.
  3. Rapid & Reliable Deployment (Lambda Module):
    • Scenario: You need to deploy a new microservice or update an existing one to add a feature for a marketing campaign next week.
    • Problem: Manual deployment through the AWS console is slow, error-prone, and not repeatable. What works on a developer’s laptop might fail in production because a step was missed.
    • Solution: The Lambda module defines the function’s infrastructure as code (IaC). Deploying a new feature or a whole new environment (like staging or production) becomes as simple as running terraform apply. This is:
      • Fast: Deploy in minutes, not days.
      • Reliable: Every deployment is identical, eliminating “works on my machine” problems.
      • Auditable: You have a version-controlled history of every change made to your infrastructure.

Tangible Business Benefits

  • Improved Security & Compliance: Reduces the risk of data breaches. Helps meet compliance requirements (like SOC 2, ISO 27001) by enforcing secure practices.
  • Increased Developer Productivity: Developers spend less time on repetitive setup and configuration and more time building features that provide customer value.
  • Reduced Operational Risk: Standardized, repeatable deployments mean fewer production outages caused by configuration errors.
  • Cost Optimization: Efficient code packaging and reuse can lead to slightly lower AWS runtime costs. More importantly, it saves significant developer hours, which is a major cost saving.
  • Scalability: This modular approach allows the business to scale its number of applications and services without the infrastructure becoming a tangled, unmanageable mess. New teams can onboard quickly using the pre-approved, secure modules.

In short, this Terraform setup transforms your cloud infrastructure from a hand-built cottage into a modern, scalable, and secure factory—a critical foundation for any business running in the cloud.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *