4. Cloud Resume Challenge: Upload Site Files to S3

4. Cloud Resume Challenge: Upload Site Files to S3

In the previous module, you created your S3 bucket. Now it's time to upload your site files. In this module you will create the resource block for creating objects for your S3 Bucket. Let's get started!


Terraform S3 Bucket Object Reference

I have included a basic template from HTML5 UP! to utilize for the exercises.


Create a new branch in GitHub

  • Login to GitHub

  • Select the Issues tab > Select New Issue

  • Add the a title, e.g. Upload S3 bucket files

  • Select Submit new issue

  • On the right pane, under Development, Select Create a branch

  • Leave the defaults > Select Create branch

  • Open your IDE Terminal.

  • Input the following:

git fetch origin
git checkout YOUR_BRANCH_NAME

Modifying s3_object files

In this section you will be modifying the main, variables and outputs Terraform files located in ./infra/modules/aws/s3/s3_object/.

main.tf

  • To create an S3 Object, modify your ./infra/modules/aws/s3/s3_object/main.tf

    • Input the following:

        resource "aws_s3_object" "object" {
          bucket       = var.bucket_name
          key          = var.key
          source       = var.file_source
          content_type = var.content_type
          etag         = var.etag
        }
      

      The only required fields for creating an S3 Object are the bucket and key arguments. However, here you will define the source (where the file is located), content_type (what kind of file it is), and etag (triggers updates when the file is modified).

  • Save the file

variables.tf

  • Now we have to define the variable types for each of the arguments.

    • Input the following:

        variable "bucket_name" {
          description = "The name of the S3 bucket"
          type = string
        }
      
        variable "content_type" {
          description = "(Optional) Standard MIME type describing the format of the object data, e.g., application/octet-stream. All Valid MIME Types are valid for this input."
          type = string
          default = "text/html"
        }
      
        variable "etag" {
          description = "(Optional) Triggers updates when the value changes. The only meaningful value is filemd5(\"path/to/file\") (Terraform 0.11.12 or later) or $${md5(file(\"path/to/file\"))} (Terraform 0.11.11 or earlier). This attribute is not compatible with KMS encryption, kms_key_id or server_side_encryption = \"aws:kms\", also if an object is larger than 16 MB, the AWS Management Console will upload or copy that object as a Multipart Upload, and therefore the ETag will not be an MD5 digest (see source_hash instead)."
          type = string
          nullable = true
        }
      
        variable "file_source" {
          description = "(Optional, conflicts with content and content_base64) Path to a file that will be read and uploaded as raw bytes for the object content."
          type = string
          nullable = true
        }
      
        variable "key" {
          description = "(Required) Name of the object once it is in the bucket."
          type = string
          nullable = false
        }
      

Each of the values are strings. The nullable argument specifies is the value can be null. The default value for content_type will be text/html if there is no variable passed to the argument.

  • Save the file

outputs.tf

  • Here are some useful outputs if you decide you required validations for your files.

    • Input the following:

        output "etag" {
          description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
          value = aws_s3_object.object.etag
        }
      
        output "id" {
          description = "key of the resource supplied above"
          value = aws_s3_object.object.id
        }
      
        output "tags_all" {
          description = "Map of tags assigned to the resource, including those inherited from the provider default_tags configuration block."
          value = aws_s3_object.object.tags_all
        }
      
        output "version_id" {
          description = "Unique version ID value for the object, if bucket versioning is enabled."
          value = aws_s3_object.object.version_id
        }
      

  • Save the file


Modifying my_portfolio files

In this section, you will modify the terraform files within ./infra/modules/my_portfolio.

main.tf

  • Here you will define a locals block, and two separate module blocks

    • Input the following:

        locals {
          mime_types = jsondecode(file("${path.module}/mime.json"))
        }
      
        module "upload_assets" {
          source = "../aws/s3/s3_object"
      
          for_each     = fileset("${var.file_path}/asset_files", "**")
          bucket_name  = module.static_website.bucket_id
          key          = "/${each.key}"
          file_source  = "${var.file_path}/asset_files/${each.value}"
          content_type = lookup(local.mime_types, regex("\\.[^.]+$", each.value), "text/html")
          etag         = filemd5("${var.file_path}/asset_files/${each.key}")
        }
      
        module "upload_html" {
          source = "../aws/s3/s3_object"
      
          for_each     = fileset("${var.file_path}/html_files", "**")
          bucket_name  = module.static_website.bucket_id
          key          = "/${each.key}"
          file_source  = "${var.file_path}/html_files/${each.value}"
          etag         = filemd5("${var.file_path}/html_files/${each.key}")
        }
      

      The locals variable references the mime.json file within my_portfolio folder. This file contains key:value pairs for common Content-Types. The jsonencode function provides JSON syntax that can be used to pull the string values for each key.

      The for_each block will iterate through the given file paths, ${var.file_path}/asset_files and ${var.file_path}/html_files, using the fileset "**" pattern. It will set each file, including subfolders, placing them into the root of the S3 Bucket.

      The two module block approach is due to a couple of html files that do not have the .html extension. This is to have a cleaner appearing site.

      Example:
      example.com/generic.html vs. example.com/generic

      Notice that the module.upload_assets.content_type has a regex function. Since those HTML files do not have an extension, the regex function will throw an error that they do not match the given regex pattern. The regex pattern is essentially looking for the dot extension of the file and matching the key to it's value in the mime.json.

      The module.upload_html block will use the default value "text/html" provided in the s3_object/variables.tf.

  • Save the file

variables.tf

  • Update the ./infra/modules/my_portfolio/variables.tf to include the file_path variable.

    • Input the following:

        variable "file_path" {
          description = "Relative path for the public folder or any future folders containing S3 bucket files."
          type = string
        }
      

      As the description notes, this will be the relative path from ./infra/main.tf to the files you want. In the case of this module, we will target the public folder for var.file_path.

outputs.tf

  • Below are some examples of outputs for module.upload_html and module.upload_assets :

      output "index_html_etag" {
        description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
        value = module.upload_html["index.html"].etag
      }
    
      output "assets_css_main_etag" {
        description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
        value = module.upload_assets["assets/css/main.css"].etag
      }
    

    Each file of each module will have to be defined for their respective outputs. For brevity, I've only included two. Feel free to find the values you want and include them.

  • Save the file


Modifying ./infra/main.tf

main.tf

  • Modify your ./infra/main.tf

    • Input the following:

        file_path   = "../public"
      

variables.tf and outputs.tf

  • Since the file_path has been hard coded, you do not have to modify your variables.tf

  • Optionally, you can include the outputs.tf. Based off the example above:

output "index_html_etag" {
  description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
  value = module.my_static_website.index_html_etag
}

output "main_css_etag" {
  description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
  value = module.my_static_website.assets_css_main_etag
}

This will output those values to the Terraform Cloud outputs.


Pushing to GitHub

  • Ensure your files are saved.

  • In your IDE Terminal, type the following:

git add .

Add all files that were changed.

git commit -m "creating s3 objects from asset_files and html_files folders"

Commit the changes with a comment.

git push

Push to GitHub.


Create Pull Request

  • Login to GitHub.

  • You should see the push on your repository.

  • Select Compare and pull request.

  • Validate the changes that were made to be pushed to main

  • Select Create pull request.

Your Terraform Plan should run before you can merge to main.

If you are using the same site files from the original template, the plan to add 73 should match.

  • Select Merge pull request > Confirm merge.

  • Delete branch.

  • In your IDE Terminal, type the following:

git checkout main
git pull

Validation:

  • AWS Management Console:

    • In the Search bar, search for "s3" > Select "Buckets"

    • Select your Bucket, and you should see the files uploaded

  • You can validate the Content-Type by selecting any file. In this example, I have selected the generic file, which has no defined dot file extension. Scrolling to the Metadata section you will see the content type is text/html.


You have completed this module and uploaded your site files to your S3 bucket. In the next module you will configure your S3 Bucket to be a static hosting website, and tie in CloudFront for content delivery.