4. Cloud Resume Challenge: Upload Site Files to S3

In the previous module, you created your S3 bucket. Now it's time to upload your site files. In this module you will create the resource block for creating objects for your S3 Bucket. Let's get started!

Terraform S3 Bucket Object Reference

I have included a basic template from HTML5 UP! to utilize for the exercises.

Create a new branch in GitHub

Login to GitHub
Select the Issues tab > Select New Issue
Add the a title, e.g. Upload S3 bucket files
Select Submit new issue

On the right pane, under Development, Select Create a branch
Leave the defaults > Select Create branch

Open your IDE Terminal.
Input the following:

git fetch origin

git checkout YOUR_BRANCH_NAME

Modifying s3_object files

In this section you will be modifying the main, variables and outputs Terraform files located in ./infra/modules/aws/s3/s3_object/.

main.tf

To create an S3 Object, modify your ./infra/modules/aws/s3/s3_object/main.tf
- Input the following:
```
  resource "aws_s3_object" "object" {
    bucket       = var.bucket_name
    key          = var.key
    source       = var.file_source
    content_type = var.content_type
    etag         = var.etag
  }
```
  The only required fields for creating an S3 Object are the bucket and key arguments. However, here you will define the source (where the file is located), content_type (what kind of file it is), and etag (triggers updates when the file is modified).
Save the file

variables.tf

Now we have to define the variable types for each of the arguments.

Input the following:

  variable "bucket_name" {
    description = "The name of the S3 bucket"
    type = string
  }

  variable "content_type" {
    description = "(Optional) Standard MIME type describing the format of the object data, e.g., application/octet-stream. All Valid MIME Types are valid for this input."
    type = string
    default = "text/html"
  }

  variable "etag" {
    description = "(Optional) Triggers updates when the value changes. The only meaningful value is filemd5(\"path/to/file\") (Terraform 0.11.12 or later) or $${md5(file(\"path/to/file\"))} (Terraform 0.11.11 or earlier). This attribute is not compatible with KMS encryption, kms_key_id or server_side_encryption = \"aws:kms\", also if an object is larger than 16 MB, the AWS Management Console will upload or copy that object as a Multipart Upload, and therefore the ETag will not be an MD5 digest (see source_hash instead)."
    type = string
    nullable = true
  }

  variable "file_source" {
    description = "(Optional, conflicts with content and content_base64) Path to a file that will be read and uploaded as raw bytes for the object content."
    type = string
    nullable = true
  }

  variable "key" {
    description = "(Required) Name of the object once it is in the bucket."
    type = string
    nullable = false
  }

Each of the values are strings. The nullable argument specifies is the value can be null. The default value for content_type will be text/html if there is no variable passed to the argument.

Save the file

outputs.tf

Here are some useful outputs if you decide you required validations for your files.

Input the following:

  output "etag" {
    description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
    value = aws_s3_object.object.etag
  }

  output "id" {
    description = "key of the resource supplied above"
    value = aws_s3_object.object.id
  }

  output "tags_all" {
    description = "Map of tags assigned to the resource, including those inherited from the provider default_tags configuration block."
    value = aws_s3_object.object.tags_all
  }

  output "version_id" {
    description = "Unique version ID value for the object, if bucket versioning is enabled."
    value = aws_s3_object.object.version_id
  }

Save the file

Modifying my_portfolio files

In this section, you will modify the terraform files within ./infra/modules/my_portfolio.

main.tf

Here you will define a locals block, and two separate module blocks
- Input the following:
```
  locals {
    mime_types = jsondecode(file("${path.module}/mime.json"))
  }

  module "upload_assets" {
    source = "../aws/s3/s3_object"

    for_each     = fileset("${var.file_path}/asset_files", "**")
    bucket_name  = module.static_website.bucket_id
    key          = "/${each.key}"
    file_source  = "${var.file_path}/asset_files/${each.value}"
    content_type = lookup(local.mime_types, regex("\\.[^.]+$", each.value), "text/html")
    etag         = filemd5("${var.file_path}/asset_files/${each.key}")
  }

  module "upload_html" {
    source = "../aws/s3/s3_object"

    for_each     = fileset("${var.file_path}/html_files", "**")
    bucket_name  = module.static_website.bucket_id
    key          = "/${each.key}"
    file_source  = "${var.file_path}/html_files/${each.value}"
    etag         = filemd5("${var.file_path}/html_files/${each.key}")
  }
```
  The locals variable references the mime.json file within my_portfolio folder. This file contains key:value pairs for common Content-Types. The jsonencode function provides JSON syntax that can be used to pull the string values for each key.
  
  The for_each block will iterate through the given file paths, ${var.file_path}/asset_files and ${var.file_path}/html_files, using the fileset "**" pattern. It will set each file, including subfolders, placing them into the root of the S3 Bucket.
  
  The two module block approach is due to a couple of html files that do not have the .html extension. This is to have a cleaner appearing site.
  
  Example:
  https://example.com/generic.html vs. https://example.com/generic
  
  Notice that the module.upload_assets.content_type has a regex function. Since those HTML files do not have an extension, the regex function will throw an error that they do not match the given regex pattern. The regex pattern is essentially looking for the dot extension of the file and matching the key to it's value in the mime.json.
  
  The module.upload_html block will use the default value "text/html" provided in the s3_object/variables.tf.
Save the file

variables.tf

Update the ./infra/modules/my_portfolio/variables.tf to include the file_path variable.
- Input the following:
```
  variable "file_path" {
    description = "Relative path for the public folder or any future folders containing S3 bucket files."
    type = string
  }
```
  As the description notes, this will be the relative path from ./infra/main.tf to the files you want. In the case of this module, we will target the public folder for var.file_path.

outputs.tf

Below are some examples of outputs for module.upload_html and module.upload_assets :

  output "index_html_etag" {
    description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
    value = module.upload_html["index.html"].etag
  }

  output "assets_css_main_etag" {
    description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
    value = module.upload_assets["assets/css/main.css"].etag
  }

Each file of each module will have to be defined for their respective outputs. For brevity, I've only included two. Feel free to find the values you want and include them.

Save the file

Modifying ./infra/main.tf

main.tf

Modify your ./infra/main.tf
- Input the following:
```
  file_path   = "../public"
```

variables.tf and outputs.tf

Since the file_path has been hard coded, you do not have to modify your variables.tf
Optionally, you can include the outputs.tf. Based off the example above:

output "index_html_etag" {
  description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
  value = module.my_static_website.index_html_etag
}

output "main_css_etag" {
  description = "ETag generated for the object (an MD5 sum of the object content). For plaintext objects or objects encrypted with an AWS-managed key, the hash is an MD5 digest of the object data. For objects encrypted with a KMS key or objects created by either the Multipart Upload or Part Copy operation, the hash is not an MD5 digest, regardless of the method of encryption. More information on possible values can be found on Common Response Headers."
  value = module.my_static_website.assets_css_main_etag
}

This will output those values to the Terraform Cloud outputs.

Pushing to GitHub

Ensure your files are saved.
In your IDE Terminal, type the following:

git add .

Add all files that were changed.

git commit -m "creating s3 objects from asset_files and html_files folders"

Commit the changes with a comment.

git push

Push to GitHub.

Create Pull Request

Login to GitHub.
You should see the push on your repository.
Select Compare and pull request.
Validate the changes that were made to be pushed to main
Select Create pull request.

Your Terraform Plan should run before you can merge to main.

If you are using the same site files from the original template, the plan to add 73 should match.

Select Merge pull request > Confirm merge.
Delete branch.
In your IDE Terminal, type the following:

git checkout main

git pull

Validation:

AWS Management Console:
- In the Search bar, search for "s3" > Select "Buckets"
- Select your Bucket, and you should see the files uploaded

You can validate the Content-Type by selecting any file. In this example, I have selected the generic file, which has no defined dot file extension. Scrolling to the Metadata section you will see the content type is text/html.

You have completed this module and uploaded your site files to your S3 bucket. In the next module you will configure your S3 Bucket to be a static hosting website, and tie in CloudFront for content delivery.

4. Cloud Resume Challenge: Upload Site Files to S3

Comments

AWS Cloud Resume

5. Cloud Resume Challenge: Creating CloudFront Distribution

More from this blog

12. Cloud Resume Challenge: Lessons Learned/ Clean Up

11. Cloud Resume Challenge: Configuring Amazon Certificate Manager (ACM) Certificate and Route53 Entries

10. Cloud Resume Challenge: Integrating API Gateway

9. Cloud Resume Challenge: Creating an API Gateway and API Resources/Methods

8. Cloud Resume Challenge: Creating Lambda Functions

Create a new branch in GitHub

Modifying s3_object files

main.tf

variables.tf

outputs.tf

Modifying my_portfolio files

main.tf

variables.tf

outputs.tf

Modifying ./infra/main.tf

main.tf

variables.tf and outputs.tf

Pushing to GitHub

Create Pull Request

Validation:

Command Palette

Comments

AWS Cloud Resume

5. Cloud Resume Challenge: Creating CloudFront Distribution

More from this blog

Create a new branch in GitHub

Modifying s3_object files

main.tf

variables.tf

outputs.tf

Modifying my_portfolio files

main.tf

variables.tf

outputs.tf

Modifying ./infra/main.tf

main.tf

variables.tf and outputs.tf

Pushing to GitHub

Create Pull Request

Validation: