Lambda Builder

A cross-platform Terraform module for building AWS Lambda deployment packages with dependency management, build validation, and intelligent change detection.

Features

• Cross-platform compatibility (Linux, macOS, Windows with WSL)

• Smart dependency management with pip

• Architecture-specific builds - Support for both x86_64 and arm64 (Graviton2) Lambda functions

• Configurable exclusions for unwanted files and folders

• Build validation and integrity checks

• Extra folder inclusion for shared resources

• Comprehensive logging and debugging output

• Hash-based change detection - only rebuilds when source code actually changes

• Persistent hash storage - saves hash in .base64sha256 file next to ZIP

Change Detection System

The module uses a sophisticated hash-based change detection system:

1. Source Hash Calculation: Calculates SHA256 hash of all files in the source directory

2. Hash Persistence: Saves the hash to {output_zip}.base64sha256 (same location as ZIP)

3. Change Detection: Compares current hash with stored hash

4. Smart Rebuilding: Only rebuilds when:

   • Source code hash has changed

   • Output ZIP file doesn’t exist

   • Hash file doesn’t exist

   • Build configuration has changed

This ensures fast, incremental builds and prevents unnecessary rebuilds when source code hasn’t changed.

Usage

module "lambda_builder" {
  source = "./modules/lambda/builder"

  # Required parameters
  source_dir = "./project/lambdas/my_function"
  output_zip = "./build/my_function.zip"
  project_dir = "./project"
  build_dir = "./build"
  source_hash = "manual_build_v1"  # Change this to trigger rebuild

  # Optional configuration
  python_version = "3"
  pip_version = "3"
  validate_build = true
  show_package_contents = true
  clean_build_dir = true

  # Install Python dependencies from requirements.txt
  with_requirements = true  # Set to false to skip pip install

  # Exclude unwanted files and folders
  exclude_folders = [
    "__pycache__",
    ".pytest_cache",
    "tests",
    "*.egg-info"
  ]
  exclude_files = [
    "*.pyc",
    "*.pyo",
    ".DS_Store",
    "*.log"
  ]

  # Include additional folders
  extra_folders = [
    "./shared_libs",
    "./config"
  ]
}

Variables

Variable	Type	Default	Description
source_dir	string	-	Source directory containing Lambda function code
output_zip	string	-	Output path for the deployment package ZIP
project_dir	string	-	Project root directory
build_dir	string	-	Temporary build directory
source_hash	string	-	Hash code of the source (used for change detection)
with_requirements	bool	false	Install Python dependencies from requirements.txt in source_dir
python_version	string	“3”	Python version for dependency installation (3.7-3.12)
pip_version	string	“3”	Pip version for dependency installation (3.7-3.12)
architecture	string	“x86_64”	Target Lambda architecture: x86_64 or arm64
validate_build	bool	true	Validate ZIP file integrity after creation
show_package_contents	bool	true	Show ZIP contents in build output
clean_build_dir	bool	true	Clean build directory before building
exclude_folders	list(string)	[see below]	Folders to exclude from the package
exclude_files	list(string)	[see below]	Files to exclude from the package
extra_folders	list(string)	[]	Additional folders to include in the package

Default Exclusions

Default exclude_folders:

[
  "*.dist-info",
  "__pycache__",
  "*.pyc",
  "*.pyo",
  ".git",
  ".gitignore",
  ".DS_Store",
  "*.swp",
  "*.tmp",
  "tests",
  "test",
  "testing",
  "docs",
  "documentation",
  "*.egg-info",
  ".venv",
]

Default exclude_files:

[
  "*.pyc",
  "*.pyo",
  "*.pyc~",
  "*.pyo~",
  "*.log",
  "*.tmp",
  "*.swp",
  ".DS_Store",
  "Thumbs.db",
  "*.md",
  "*.rst",
]

Outputs

Basic Outputs

Output	Description
zip_file	Path to the generated deployment package ZIP file
build_dir	Directory where the build process was executed
source_dir	Source directory that was packaged
project_dir	Project root directory
python_version	Python version used for dependency installation
pip_version	Pip version used for dependency installation

Change Detection Outputs

Output	Description
source_hash	The SHA256 hash of the source directory contents

Examples

Basic Usage

module "simple_lambda" {
  source = "./modules/lambda/builder"

  source_dir = "./project/lambdas/handler_one"
  output_zip = "./build/handler_one.zip"
  project_dir = "./project"
  build_dir = "./build"
  source_hash = "manual_build_v1"
}

# Check build status
output "build_status" {
  value = {
    zip_file = module.simple_lambda.zip_file
    source_hash = module.simple_lambda.source_hash
  }
}

Advanced Configuration with Dependencies

module "complex_lambda" {
  source = "./modules/lambda/builder"

  source_dir = "./project/lambdas/handler_two"
  output_zip = "./build/handler_two.zip"
  project_dir = "./project"
  build_dir = "./build"
  source_hash = "manual_build_v2"

  python_version = "3"
  pip_version = "3"

  # Enable Python dependency installation
  with_requirements = true  # Will install from {source_dir}/requirements.txt

  exclude_folders = [
    "__pycache__",
    ".pytest_cache",
    "tests",
    "*.egg-info",
    ".git"
  ]

  exclude_files = [
    "*.pyc",
    "*.pyo",
    ".DS_Store",
    "*.log",
    "*.tmp"
  ]

  extra_folders = [
    "./shared_libs",
    "./config/production"
  ]

  validate_build = true
  show_package_contents = true
}

# Monitor change detection
output "change_detection_info" {
  value = {
    zip_file = module.complex_lambda.zip_file
    source_hash = module.complex_lambda.source_hash
  }
}

Multiple Functions

module "handler_one" {
  source = "./modules/lambda/builder"

  source_dir = "./project/lambdas/handler_one"
  output_zip = "./build/handler_one.zip"
  project_dir = "./project"
  build_dir = "./build"
  source_hash = "manual_build_v1"
}

module "handler_two" {
  source = "./modules/lambda/builder"

  source_dir = "./project/lambdas/handler_two"
  output_zip = "./build/handler_two.zip"
  project_dir = "./project"
  build_dir = "./build"
  source_hash = "manual_build_v1"

  extra_folders = ["./shared_libs"]
}

module "handler_three" {
  source = "./modules/lambda/builder"

  source_dir = "./project/lambdas/handler_three"
  output_zip = "./build/handler_three.zip"
  project_dir = "./project"
  build_dir = "./build"
  source_hash = "manual_build_v1"

  python_version = "3"
  exclude_folders = ["tests", "__pycache__"]
}

ARM64 (Graviton2) Lambda

module "arm64_lambda" {
  source = "./modules/lambda/builder"

  source_dir = "./project/lambdas/arm64_handler"
  output_zip = "./build/arm64_handler.zip"
  project_dir = "./project"
  build_dir = "./build"
  source_hash = "manual_build_v1"

  # Target ARM64 architecture
  architecture = "arm64"

  # Install dependencies for ARM64
  with_requirements = true
  python_version = "3.11"
}

Benefits of ARM64:

• Up to 34% better price-performance than x86_64

• Lower latency for compute-intensive workloads

• Requires architecture-specific Python packages (numpy, pandas, pillow, etc.)

Change Detection Details

How It Works

1. Hash Calculation: The module calculates a SHA256 hash of all files in the source directory during plan phase

2. Trigger Detection: The hash is included in the resource triggers

3. Change Detection: Terraform detects when the source hash changes and triggers a rebuild

4. Automatic Rebuilding: The build script runs only when source code or configuration changes

Hash Calculation

The hash is calculated by:

• Finding all files in the source directory

• Computing SHA256 hash of each file

• Sorting the hashes

• Computing a final SHA256 hash of the sorted list

This ensures that any change to any file in the source directory will trigger a rebuild.

Debugging Change Detection

Use the outputs to debug change detection:

output "debug_info" {
  value = {
    zip_file = module.my_lambda.zip_file
    source_hash = module.my_lambda.source_hash
  }
}

File Structure

After building, you’ll have:

./build/
├── handler_one.zip              # Lambda deployment package
├── handler_one.zip.base64sha256 # Source code hash
└── ...

Best Practices

1. Use meaningful source_hash values: Change the source_hash when you want to force a rebuild

2. Keep build_dir separate: Use a dedicated build directory for each function

3. Match architecture to Lambda function: Set architecture to match your Lambda function’s architecture setting

4. Consider ARM64 for cost savings: ARM64 provides better price-performance for most workloads

5. Exclude unnecessary files: Use the default exclusions or customize as needed

6. Validate builds: Keep validate_build = true for production deployments

7. Monitor outputs: Use the outputs to track build status and hash values

8. Use consistent naming: Follow a consistent naming pattern for your ZIP files

9. Version your builds: Include version information in your source_hash values