=============================================================================== Lambda Builder =============================================================================== A cross-platform Terraform module for building AWS Lambda deployment packages with dependency management, build validation, and **intelligent change detection**. Features ------------------------------------------------------------------------------- - **Cross-platform compatibility** (Linux, macOS, Windows with WSL) - **Smart dependency management** with pip - **Architecture-specific builds** - Support for both x86_64 and arm64 (Graviton2) Lambda functions - **Configurable exclusions** for unwanted files and folders - **Build validation** and integrity checks - **Extra folder inclusion** for shared resources - **Comprehensive logging** and debugging output - **Hash-based change detection** - only rebuilds when source code actually changes - **Persistent hash storage** - saves hash in ``.base64sha256`` file next to ZIP Change Detection System ------------------------------------------------------------------------------- The module uses a sophisticated hash-based change detection system: 1. **Source Hash Calculation**: Calculates SHA256 hash of all files in the source directory 2. **Hash Persistence**: Saves the hash to ``{output_zip}.base64sha256`` (same location as ZIP) 3. **Change Detection**: Compares current hash with stored hash 4. **Smart Rebuilding**: Only rebuilds when: - Source code hash has changed - Output ZIP file doesn't exist - Hash file doesn't exist - Build configuration has changed This ensures **fast, incremental builds** and prevents unnecessary rebuilds when source code hasn't changed. Usage ------------------------------------------------------------------------------- .. code-block:: hcl module "lambda_builder" { source = "./modules/lambda/builder" # Required parameters source_dir = "./project/lambdas/my_function" output_zip = "./build/my_function.zip" project_dir = "./project" build_dir = "./build" source_hash = "manual_build_v1" # Change this to trigger rebuild # Optional configuration python_version = "3" pip_version = "3" validate_build = true show_package_contents = true clean_build_dir = true # Install Python dependencies from requirements.txt with_requirements = true # Set to false to skip pip install # Exclude unwanted files and folders exclude_folders = [ "__pycache__", ".pytest_cache", "tests", "*.egg-info" ] exclude_files = [ "*.pyc", "*.pyo", ".DS_Store", "*.log" ] # Include additional folders extra_folders = [ "./shared_libs", "./config" ] } Variables ------------------------------------------------------------------------------- .. list-table:: :header-rows: 1 :widths: 20 15 15 50 * - Variable - Type - Default - Description * - source_dir - string - \- - Source directory containing Lambda function code * - output_zip - string - \- - Output path for the deployment package ZIP * - project_dir - string - \- - Project root directory * - build_dir - string - \- - Temporary build directory * - source_hash - string - \- - Hash code of the source (used for change detection) * - with_requirements - bool - false - Install Python dependencies from ``requirements.txt`` in source_dir * - python_version - string - "3" - Python version for dependency installation (3.7-3.12) * - pip_version - string - "3" - Pip version for dependency installation (3.7-3.12) * - architecture - string - "x86_64" - Target Lambda architecture: ``x86_64`` or ``arm64`` * - validate_build - bool - true - Validate ZIP file integrity after creation * - show_package_contents - bool - true - Show ZIP contents in build output * - clean_build_dir - bool - true - Clean build directory before building * - exclude_folders - list(string) - [see below] - Folders to exclude from the package * - exclude_files - list(string) - [see below] - Files to exclude from the package * - extra_folders - list(string) - [] - Additional folders to include in the package Default Exclusions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Default exclude_folders:** .. code-block:: text [ "*.dist-info", "__pycache__", "*.pyc", "*.pyo", ".git", ".gitignore", ".DS_Store", "*.swp", "*.tmp", "tests", "test", "testing", "docs", "documentation", "*.egg-info", ".venv", ] **Default exclude_files:** .. code-block:: text [ "*.pyc", "*.pyo", "*.pyc~", "*.pyo~", "*.log", "*.tmp", "*.swp", ".DS_Store", "Thumbs.db", "*.md", "*.rst", ] Outputs ------------------------------------------------------------------------------- Basic Outputs ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 :widths: 30 70 * - Output - Description * - zip_file - Path to the generated deployment package ZIP file * - build_dir - Directory where the build process was executed * - source_dir - Source directory that was packaged * - project_dir - Project root directory * - python_version - Python version used for dependency installation * - pip_version - Pip version used for dependency installation Change Detection Outputs ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 :widths: 30 70 * - Output - Description * - source_hash - The SHA256 hash of the source directory contents Examples ------------------------------------------------------------------------------- Basic Usage ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: hcl module "simple_lambda" { source = "./modules/lambda/builder" source_dir = "./project/lambdas/handler_one" output_zip = "./build/handler_one.zip" project_dir = "./project" build_dir = "./build" source_hash = "manual_build_v1" } # Check build status output "build_status" { value = { zip_file = module.simple_lambda.zip_file source_hash = module.simple_lambda.source_hash } } Advanced Configuration with Dependencies ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: hcl module "complex_lambda" { source = "./modules/lambda/builder" source_dir = "./project/lambdas/handler_two" output_zip = "./build/handler_two.zip" project_dir = "./project" build_dir = "./build" source_hash = "manual_build_v2" python_version = "3" pip_version = "3" # Enable Python dependency installation with_requirements = true # Will install from {source_dir}/requirements.txt exclude_folders = [ "__pycache__", ".pytest_cache", "tests", "*.egg-info", ".git" ] exclude_files = [ "*.pyc", "*.pyo", ".DS_Store", "*.log", "*.tmp" ] extra_folders = [ "./shared_libs", "./config/production" ] validate_build = true show_package_contents = true } # Monitor change detection output "change_detection_info" { value = { zip_file = module.complex_lambda.zip_file source_hash = module.complex_lambda.source_hash } } Multiple Functions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: hcl module "handler_one" { source = "./modules/lambda/builder" source_dir = "./project/lambdas/handler_one" output_zip = "./build/handler_one.zip" project_dir = "./project" build_dir = "./build" source_hash = "manual_build_v1" } module "handler_two" { source = "./modules/lambda/builder" source_dir = "./project/lambdas/handler_two" output_zip = "./build/handler_two.zip" project_dir = "./project" build_dir = "./build" source_hash = "manual_build_v1" extra_folders = ["./shared_libs"] } module "handler_three" { source = "./modules/lambda/builder" source_dir = "./project/lambdas/handler_three" output_zip = "./build/handler_three.zip" project_dir = "./project" build_dir = "./build" source_hash = "manual_build_v1" python_version = "3" exclude_folders = ["tests", "__pycache__"] } ARM64 (Graviton2) Lambda ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: hcl module "arm64_lambda" { source = "./modules/lambda/builder" source_dir = "./project/lambdas/arm64_handler" output_zip = "./build/arm64_handler.zip" project_dir = "./project" build_dir = "./build" source_hash = "manual_build_v1" # Target ARM64 architecture architecture = "arm64" # Install dependencies for ARM64 with_requirements = true python_version = "3.11" } **Benefits of ARM64:** - Up to 34% better price-performance than x86_64 - Lower latency for compute-intensive workloads - Requires architecture-specific Python packages (numpy, pandas, pillow, etc.) Change Detection Details ------------------------------------------------------------------------------- How It Works ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1. **Hash Calculation**: The module calculates a SHA256 hash of all files in the source directory during plan phase 2. **Trigger Detection**: The hash is included in the resource triggers 3. **Change Detection**: Terraform detects when the source hash changes and triggers a rebuild 4. **Automatic Rebuilding**: The build script runs only when source code or configuration changes Hash Calculation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The hash is calculated by: - Finding all files in the source directory - Computing SHA256 hash of each file - Sorting the hashes - Computing a final SHA256 hash of the sorted list This ensures that any change to any file in the source directory will trigger a rebuild. Debugging Change Detection ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Use the outputs to debug change detection: .. code-block:: hcl output "debug_info" { value = { zip_file = module.my_lambda.zip_file source_hash = module.my_lambda.source_hash } } File Structure ------------------------------------------------------------------------------- After building, you'll have: .. code-block:: text ./build/ ├── handler_one.zip # Lambda deployment package ├── handler_one.zip.base64sha256 # Source code hash └── ... Best Practices ------------------------------------------------------------------------------- 1. **Use meaningful source_hash values**: Change the ``source_hash`` when you want to force a rebuild 2. **Keep build_dir separate**: Use a dedicated build directory for each function 3. **Match architecture to Lambda function**: Set ``architecture`` to match your Lambda function's architecture setting 4. **Consider ARM64 for cost savings**: ARM64 provides better price-performance for most workloads 5. **Exclude unnecessary files**: Use the default exclusions or customize as needed 6. **Validate builds**: Keep ``validate_build = true`` for production deployments 7. **Monitor outputs**: Use the outputs to track build status and hash values 8. **Use consistent naming**: Follow a consistent naming pattern for your ZIP files 9. **Version your builds**: Include version information in your source_hash values