T68 prod dev test configs

[alepbloyd]

What type of PR is this? (check all applicable)

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update

Description

This PR updates the Docker infrastructure and adds Terraform infrastructure for production deployments. Docker infrastructure is now split into a docker-compose-init.yml file to handle startup and shared directory configuration and permissions, and then separate docker-compose files for dev, test, and prod.

Related Tickets & Documents

• Related Issue Prod configuration #68

QA Instructions, Screenshots, Recordings

To test:

• Prod: follow instructions in the README update to try deploying to AWS. Try depositing a work and ensure that you can see if in the resulting S3 bucket. You will likely see an error in the viewer on a work show page about UV routes - that is not a problem at the moment and will be addressed with the coming switch to other AV viewers.
• Development: locally, run bin/dev to test local development environment.
• Test: run bin/test. There will likely be failing tests, but those are addressed in coming PR that is less about infrastructure. You should be able to check the logs of the rspec container after running and see test results.

THINGS TO NOTE:

• The prod deployment is slow (think, 15-20 minutes) because it builds the images when it runs. Ultimately this should pull from an image repository, but this is a step in that direction.
• This does not handle DNS and just deploys with public IP.
• This creates a self-signed certificate on start up that should be replaced on EC2 instance.
• On production, this does NOT seed the database with required admin sets, collections, or an admin user. You can SSH to the EC2 instance, attach to Rails container, and run the steps in db/seeds.rb but with a different admin email/password.
• This currently deploys on a t2.large EC2 instance with a 100 gb root volume. @kilahimm @kerchner - this is particularly where I could use extra input on what we are hoping for, and this can be configured in tfvars.

[kerchner]

Great work on this @alepbloyd . I ran into a couple of issues that, I believe, may be because main.tf assumes that the AWS resources don't already exist, but some do and probably should. In particular:

Error: creating IAM Role (scholarspace-tf-test-prod-web-server-role): EntityAlreadyExists: Role with name scholarspace-tf-test-prod-web-server-role already exists.

and

Error: creating Amazon S3 (Simple Storage) Bucket (scholarspace-tf-test): BucketAlreadyOwnedByYou: Your previous request to create the named bucket succeeded and you already own it.

(not sure who "you" is referring to in the above error).

I tried creating a script to, instead, import existing resources:

#!/bin/bash
terraform import aws_iam_role.web_server_role scholarspace-tf-test-prod-web-server-role
terraform import aws_s3_bucket.app_bucket scholarspace-tf-test

but these are resulting in:

╷
│ Error: Resource already managed by Terraform
│
│ Terraform is already managing a remote object for aws_iam_role.web_server_role. To import
│ to this address you must first remove the existing object from the state.

and

╷
│ Error: Resource already managed by Terraform
│
│ Terraform is already managing a remote object for aws_s3_bucket.app_bucket. To import to
│ this address you must first remove the existing object from the state.
╵

I'll keep working through this but wanted to share findings so far. I would be interested to know if any of these may be due to failures on my part to follow the README.md

[alepbloyd]

Thanks for taking a look! You're right that this assumes none of the resources already exist.

For the error:
Error: creating IAM Role (scholarspace-tf-test-prod-web-server-role): EntityAlreadyExists: Role with name scholarspace-tf-test-prod-web-server-role already exists.

I think this is because in the process of creating/testing this, the instance I deployed with these terraform settings created that scholarspace-tf-test-prod-web-server-role. Being able to import the role once it's configured would definitely be an improvement, or you could adjust the role name when you try this, which I think will just end up with a role with duplicate permissions but a different name.

For Error: creating Amazon S3 (Simple Storage) Bucket (scholarspace-tf-test): BucketAlreadyOwnedByYou: Your previous request to create the named bucket succeeded and you already own it.

I think a similar issue. scholarspace-tf-test is the S3 bucket name I used when doing this, so that bucket already exists. In this case I would just try using a different bucket name. I have it set up to be a versioned bucket, which makes it hard(/impossible?) to delete through terraform destroy, so once you create a bucket with this workflow, you need to either delete it through the console or the AWS CLI.

I'm going to be focusing on the derivative/display stuff today, but I'll try to do some research on importing/using existing AWS resources in terraform.

[kerchner]

I'll choose new, unique names for the resources and re-test the prod deployment with those.

[kerchner]

Prod deployment works. A few comments:

• terraform destroy is unable to destroy the Fedora S3 bucket; this requires aws cli commands. Example:

  aws s3api delete-objects --bucket scholarspace-tf-test-dk --delete "$(aws s3api list-object-versions --bucket scholarspace-tf-test-dk --query '{Objects: Versions[].{Key:Key,VersionId:VersionId}}' --max-items 1000)"```
  

• Is the intention that this would be used in a context where S3 content, DB, etc. are pre-existing but we'd use Terraform to stand up a new EC2 server etc.? If so, the terraform config/script will need to be modified somehow to gracefully work with pre-existing components (where the names are those referenced in the config/script)

I will try dev and test setup next.