Security is important. You know it’s true. But implementing good security practices is a challenge. In particular, when you’re managing resources in a shared cloud environment, you need to keep particular considerations and best practices in mind. Examples include locking down your accounts with multi-factor authentication, minimizing your blast radius by segregating resources, and using the principle of least privilege when assigning user permissions.
Secure AWS Account Structure
To achieve a more secure AWS account structure, we've recently reviewed our AWS footprint and security practices. We decided that we wanted to take advantage of AWS Organizations to structure our AWS accounts and resources in a more manageable way with more granular security controls. After reviewing suggestions from Amazon about possible multiple account strategies, we chose to implement a hybrid structure that provides substantial security benefits by separating Identity and Access Management (IAM) from actual AWS resources.
In this secure AWS account structure, a Master account manages the billing for the organization. All IAM users exist in an InfoSec account, and users use role assumption to access resources in the various accounts. Resources are provisioned in either the Production or Non-Production accounts. CloudTrail is configured in each account for auditing activity within the account, and a log trail stores the logs in the InfoSec account. We then lock down the root credentials with multi-factor authentication. (Soon, we’ll be adding alarms and notifications to let us know when potentially nefarious activity occurs in the logs, such as anytime a root user logs in.)We also wanted a reproducible way to set up a secure AWS account structure and associated policies in order to share them with our clients and easily make changes in the future. Terraform was an obvious choice. In addition, we decided to leverage the awesome Terragrunt tool from Gruntwork. Terragrunt makes it easier to write DRY (don’t repeat yourself) Terraform code and manage remote state.
With Terragrunt, we organize our Terraform configurations into subfolders to make them easier to manage; however, they can still share some common settings and variables. The configurations for the initial organization setup and the temp-admin user are in the master folder. The accounts folder has all the configurations for each of the three sub-accounts (infosec, non-prod, and prod). The environments folder is where we’ll eventually put configurations for the actual resources. The modules and utility folders contain some additional configurations that are used across accounts (they could probably be pulled out into their own repo at some point).
├── accounts
│ ├── infosec
│ ├── non-prod
│ └── prod
├── environments
│ ├── non-prod
│ └── prod
├── master
│ ├── organization
│ └── temp-admin
├── modules
└── utility
Which Came First: the Terraform or the Terraform?
In trying to automate a solution, we quickly discovered a “chicken or egg” dilemma. Well, actually two related dilemmas. First, in order to run Terraform modules to create accounts, policies, users, etc., we needed to already have a user attached to policies that allow Terraform to perform the required actions.
Second, in order to configure an S3 bucket to store the Terraform state, we needed the account where that bucket would be stored to exist before applying the configurations. Our goal was to have no resources actually managed within the Master account, but that was the account where our initial Terraform would be run. (Terragrunt automatically creates the S3 bucket on init, but the S3 bucket would be created in the Master account, which isn’t what we wanted.)
In the end, we got the process down to just a few manual steps, as follows:
- Create the AWS account that will serve as the Master account.
- Lock down the root credentials immediately.
- Create an IAM policy that allows organization, account management, and role assumption in the child accounts.
- Create an “init” user that will be used to run a base setup script.
That’s it for manual steps. We created a bash script that takes in the credentials of the init user and does the rest of the work. That’s where the fun happens.
The Initialization Script
The initialization script first configures Terragrunt to use a local backend for state and applies the Terraform configurations that create the sub-accounts by using an override file and a specific Terragrunt config. The script then configures Terragrunt to use the S3 remote backend and re-inits Terraform to copy the state, which solves the problem of getting the remote state stored in the right account.
The script runs the temp-admin configurations to create the temp-admin user. It then uses the output of the apply to retrieve the secret key and encrypted secret access key for the temp-admin user.
Next, it applies all the configurations in the three accounts folders as the temp-admin user. These configurations create the resources we need within the three sub-accounts, including the initial set of users, groups and policies as well as the cross-account roles and CloudTrail logging.
If you pass the -u parameter, the script generates a one-time password for the specified user.
Finally, it deletes the temp-admin user.
The Result: Secure AWS Account Structure
Be sure to customize shared.tfvars and accounts/infosec/users.tf before you run the script. Then once you’ve run the script, you’ll have three sub-accounts within your Master account’s organization — Infosec, Production, and Non-Production — with users assigned to one or more of the following groups that restrict their access to the particular accounts you’ve specified:
- InfosecAdmins group – access to manage users and policies
- ProdAdmins group – access to manage production account
- NonProdAdmins group – access to manage non-prod account
- ProdDevelopers group – access to production resources
- NonProdDevelopers group – access to non-prod resources
- MasterBilling group – access to manage billing for the Master account
We’ve made the basic configurations and scripts available for creating a secure AWS account structure in a public repo on GitHub called aws-accounts-terraform. We look forward to feedback and suggestions for improvement!
Special shout out to Emii Khaos, whose blog post on Automated AWS account initialization with Terraform and OneLogin SAML SSO inspired this work.
