AWS

The AWS bundle installs a catalog scraper that:

• Scrapes AWS Resources and detects changes in the resource definition
• Ingests changes from CloudTrail
• Ingests cost data from AWS Cost & Usage Reporting
• Links AWS EKS resources to the corresponding Kubernetes resources

Prerequisites

• IAM Role for scraping AWS API's
• Mission Control configured to use Pod Identity/IRSA or an AWS access key

Setup

helm repo add flanksource https://flanksource.github.io/charts
helm repo update
helm install mission-control-aws flanksource/mission-control-aws

When you go to the catalog now, you can now see all the AWS Resources

Create an IAM Role

Depending on how you want to use Mission Control you need to create an IAM role for mission control to use:

Use Case	Role
Read Only Scraping	arn:aws:iam::aws:policy/ReadOnlyAccess
Playbooks to create and update AWS Resources	arn:aws:iam::aws:policy/PowerUserAccess

Create new IAM Policy (Alternative)

You can also create a new policy with just the permissions required by Mission Control

iam-policy.json
{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "mission-control-config-role",
			"Effect": "Allow",
			"Action": [
				"acm:Describe*",
				"acm:Get*",
				"acm:List*",
				"cloudtrail:LookupEvents",
				"config:BatchGetAggregateResourceConfig",
				"config:BatchGetResourceConfig",
				"config:Describe*",
				"config:Get*",
				"config:List*",
				"config:SelectAggregateResourceConfig",
				"config:SelectResourceConfig",
				"ec2:Describe*",
				"ecr:Describe*",
				"eks:Describe*",
				"eks:ListClusters",
				"elasticfilesystem:Describe*",
				"elasticloadbalancing:Describe*",
				"guardduty:Describe*",
				"guardduty:Get*",
				"guardduty:List*",
				"iam:GetAccountName",
				"iam:GetAccountSummary",
				"iam:GetGroup",
				"iam:GetGroupPolicy",
				"iam:GetInstanceProfile",
				"iam:GetLoginProfile",
				"iam:GetPolicy",
				"iam:GetRole",
				"iam:GetRolePolicy",
				"iam:GetUser",
				"iam:List*",
				"lambda:List*",
				"rds:Describe*",
				"sts:GetCallerIdentity"
				"trustedadvisor:Describe*",
				"trustedadvisor:DownloadRisk",
				"trustedadvisor:Get*",
				"trustedadvisor:List*",
			],
			"Resource": "*"
		}
	]
}

Configure IAM Roles for Mission Control

IAM Roles for Service Accounts

eksctl

2. Setup variables

   # The name of the EKS cluster mission control is being deployed to
   export CLUSTER= <CLUSTER_NAME>
   # the default namespace the mission-control helm chart uses
   export NAMESPACE=mission-control
   export ACCOUNT=$(aws sts get-caller-identity  --query 'Account' --output text)
   

3. Enable EKS IAM Roles for Service Accounts

   eksctl utils associate-iam-oidc-provider --cluster=$CLUSTER
   

4. Create the IAM Role mappings

   eksctl.yaml
   iam:
   	withOIDC: true
   	serviceAccounts:
   	- metadata:
   			name: mission-control-sa
   			namespace: mission-control
   		roleName: MissionControlRole
   		roleOnly: true
   		attachPolicyARNs:
   		- "arn:aws:iam::aws:policy/ReadOnlyAccess"
   	- metadata:
   			name: canary-checker-sa
   			namespace: mission-control
   		roleName: CanaryCheckerRole
   		roleOnly: true
   		attachPolicyARNs:
   		- "arn:aws:iam::aws:policy/ReadOnlyAccess"
   	- metadata:
   			name: config-db-sa
   			namespace: mission-control
   		roleName: ConfigDBRole
   		roleOnly: true
   		attachPolicyARNs:
   		- "arn:aws:iam::aws:policy/ReadOnlyAccess"
   

   eksctl create iamserviceaccount --cluster $CLUSTER -c eksctl.yaml
   

5. Choose a routable DOMAIN for Mission Control

   See Ingress for more options on configuring the ingress including generating certs with cert-manager

   See Local Testing for testing using a kind or minikube without a routable domain

6. Install Mission Control

   Helm

   cat > values.yaml << EOF
   
   serviceAccount:
   	annotations:
   		# used by mission control for notifications / playbooks
   		eks.amazonaws.com/role-arn: arn:aws:iam::$ACCOUNT:role/MissionControlRole
   
   canary-checker:
   	serviceAccount:
   		annotations:
   			# used for cloudwatch, S3 and other AWS health checks
   			eks.amazonaws.com/role-arn: arn:aws:iam::$ACCOUNT:role/CanaryCheckerRole
   
   config-db:
   	serviceAccount:
   		annotations:
   			# used to scrape AWS resources, change history via AWS CloudTrail and cost via Athena
   			eks.amazonaws.com/role-arn: arn:aws:iam::$ACCOUNT:role/ConfigDBRole
   EOF
   
   helm repo add flanksource https://flanksource.github.io/charts 
   helm repo update
   helm install mission-control flanksource/mission-control \
    --set-file values.yaml \
    -n mission-control --create-namespace \
    --wait 

   Flux

   apiVersion: v1
   kind: Namespace
   metadata:
     name: mission-control
   ---
   apiVersion: source.toolkit.fluxcd.io/v1beta1
   kind: HelmRepository
   metadata:
     name: flanksource
     namespace: mission-control
   spec:
     interval: 5m0s
     url: https://flanksource.github.io/charts
   ---
   apiVersion: helm.toolkit.fluxcd.io/v2beta1
   kind: HelmRelease
   metadata:
     name: mission-control
     namespace: mission-control
   spec:
     chart:
       spec:
         chart: mission-control
         sourceRef:
           kind: HelmRepository
           name: flanksource
           namespace: mission-control
         interval: 1m
     values:
        
      serviceAccount:
      	annotations:
      		# used by mission control for notifications / playbooks
      		eks.amazonaws.com/role-arn: arn:aws:iam::$ACCOUNT:role/MissionControlRole
      
      canary-checker:
      	serviceAccount:
      		annotations:
      			# used for cloudwatch, S3 and other AWS health checks
      			eks.amazonaws.com/role-arn: arn:aws:iam::$ACCOUNT:role/CanaryCheckerRole
      
      config-db:
      	serviceAccount:
      		annotations:
      			# used to scrape AWS resources, change history via AWS CloudTrail and cost via Athena
      			eks.amazonaws.com/role-arn: arn:aws:iam::$ACCOUNT:role/ConfigDBRole
   See values.yaml

Pod Identity

eksctl

1. Ensure the AWS Pod Identity Agent is configured and running

2. Create a mapping file for eksctl

   eksctl.yaml
   podIdentityAssociations:
   - namespace: mission-control
   	serviceAccountName:  mission-control-sa
   	permissionPolicyARNs: arn:aws:iam::aws:policy/ReadOnlyAccess
   
   - namespace: mission-control
   	serviceAccountName:  config-db-sa
   	permissionPolicyARNs: arn:aws:iam::aws:policy/ReadOnlyAccess
   
   - namespace: mission-control
   	serviceAccountName:  canary-checker-sa
   	permissionPolicyARNs: arn:aws:iam::aws:policy/ReadOnlyAccess
   iam:
   # note withOIDC is not required for Pod Identity
   serviceAccounts:
   # used by mission control for notifications / playbooks
   - metadata:
   		name: mission-control-sa
   		namespace: mission-control
   	attachPolicyARNs:
   	- "arn:aws:iam::aws:policy/ReadOnlyAccess"
   # used for cloudwatch, S3 and other AWS health checks
   - metadata:
   		name: canary-checker-sa
   		namespace: mission-control
   	attachPolicyARNs:
   	- "arn:aws:iam::aws:policy/ReadOnlyAccess"
   # used to scrape resources, AWS CloudTrail and AWS Cost & Usage Reports
   - metadata:
   		name: config-db-sa
   		namespace: mission-control
   	attachPolicyARNs:
   	- "arn:aws:iam::aws:policy/ReadOnlyAccess"
   

   Using an existing IAM Role

   If you are using a pre-existing IAM role when creating a pod identity association, you must configure the role to trust the newly introduced EKS service principal (pods.eks.amazonaws.com)

   iam-trust-policy.json
   {
   	"Version": "2012-10-17",
   	"Statement": [
   		{
   			"Effect": "Allow",
   			"Principal": {
   				"Service": "pods.eks.amazonaws.com"
   			},
   			"Action": ["sts:AssumeRole", "sts:TagSession"]
   		}
   	]
   }
   

3. Apply the Pod Identities using eksctl

   eksctl create podidentityassociation  -c eksctl.yaml
   

4. Choose a routable DOMAIN for Mission Control

   See Ingress for more options on configuring the ingress including generating certs with cert-manager

   See Local Testing for testing using a kind or minikube without a routable domain

5. Install Mission Control

   Helm

   helm repo add flanksource https://flanksource.github.io/charts 
   helm repo update
   helm install mission-control flanksource/mission-control \
    -n mission-control --create-namespace \
    --wait 

   Flux

   apiVersion: v1
   kind: Namespace
   metadata:
     name: mission-control
   ---
   apiVersion: source.toolkit.fluxcd.io/v1beta1
   kind: HelmRepository
   metadata:
     name: flanksource
     namespace: mission-control
   spec:
     interval: 5m0s
     url: https://flanksource.github.io/charts
   ---
   apiVersion: helm.toolkit.fluxcd.io/v2beta1
   kind: HelmRelease
   metadata:
     name: mission-control
     namespace: mission-control
   spec:
     chart:
       spec:
         chart: mission-control
         sourceRef:
           kind: HelmRepository
           name: flanksource
           namespace: mission-control
         interval: 1m
     values:
     
   See values.yaml

Terraform

1. Ensure the AWS Pod Identity Agent is configured and running

2. Create main.tf

   main.tf
   variable "cluster" {
     type = string
   }
   
   variable "role" {
     type = string
   }
   
   variable "namespace" {
     type = string
   	default = "mission-control"
   }
   
   variable "policy" {
   	type = string
   	default = "arn:aws:iam::aws:policy/ReadOnlyAccess"
   }
   
   locals {
     service_accounts = [
   		"mission-control-sa",
   		"canary-checker-sa",
   		"config-db-sa"
     ]
   }
   
   data "aws_caller_identity" "current" {}
   
   resource "aws_iam_role" "mission-control" {
     name = "MissionControlRole"
     assume_role_policy = jsonencode({
       Statement = [{
   			Action = [
   				"sts:AssumeRole",
   				"sts:TagSession"
   			]
         Effect = "Allow"
         Principal = {
           Service: "pods.eks.amazonaws.com"
         }
       }]
       Version = "2012-10-17"
     })
   }
   
   resource "aws_iam_role_policy_attachment" "mission-control" {
     policy_arn = var.policy
     role       = aws_iam_role.mission-control.name
   }
   
   resource "aws_eks_pod_identity_association" "pod_identities" {
     for_each   			= local.service_accounts
     cluster_name    = var.cluster
     namespace       = var.namespace
     service_account = each.value
     role_arn        = var.role
   }
   

3. Apply the terraform

   TF_VAR_role=$CLUSTER  terraform apply
   

4. Choose a routable DOMAIN for Mission Control

   See Ingress for more options on configuring the ingress including generating certs with cert-manager

   See Local Testing for testing using a kind or minikube without a routable domain

5. Install Mission Control

   Helm

   helm repo add flanksource https://flanksource.github.io/charts 
   helm repo update
   helm install mission-control flanksource/mission-control \
    -n mission-control --create-namespace \
    --wait 

   Flux

   apiVersion: v1
   kind: Namespace
   metadata:
     name: mission-control
   ---
   apiVersion: source.toolkit.fluxcd.io/v1beta1
   kind: HelmRepository
   metadata:
     name: flanksource
     namespace: mission-control
   spec:
     interval: 5m0s
     url: https://flanksource.github.io/charts
   ---
   apiVersion: helm.toolkit.fluxcd.io/v2beta1
   kind: HelmRelease
   metadata:
     name: mission-control
     namespace: mission-control
   spec:
     chart:
       spec:
         chart: mission-control
         sourceRef:
           kind: HelmRepository
           name: flanksource
           namespace: mission-control
         interval: 1m
     values:
     
   See values.yaml

Cloudformation

1. Setup variables

   # The name of the EKS cluster mission control is being deployed to
   export CLUSTER= <CLUSTER_NAME>
   # the default namespace the mission-control helm chart uses
   export NAMESPACE=mission-control
   

2. Create a cloudformation template

   mission-control-iam-cloudformation.yaml
   AWSTemplateFormatVersion: "2010-09-09"
   Description: CloudFormation template for Mission Control IAM Role using EKS Pod Identities
   
   Parameters:
     Cluster:
       Type: String
       Description: Name of the EKS cluster
     Namespace:
       Type: String
       Default: "mission-control"
       Description: Kubernetes namespace
     PolicyArn:
       Type: String
       Default: "arn:aws:iam::aws:policy/ReadOnlyAccess"
       Description: ARN of the IAM policy to attach to the role
   
   Resources:
     MissionControlRole:
       Type: "AWS::IAM::Role"
       Properties:
         RoleName: "MissionControlRole"
         AssumeRolePolicyDocument:
           Version: "2012-10-17"
           Statement:
             - Effect: "Allow"
               Principal:
                 Service: "eks.amazonaws.com"
               Action:
                 - "sts:AssumeRole"
                 - "sts:TagSession"
   
     MissionControlRolePolicyAttachment:
       Type: "AWS::IAM::Policy"
       Properties:
         PolicyName: "MissionControlPolicy"
         Roles:
           - Ref: "MissionControlRole"
         PolicyDocument:
           Version: "2012-10-17"
           Statement:
             - Effect: "Allow"
               Action: "*"
               Resource: "*"
   
     MissionControlServiceAccount:
       Type: "AWS::EKS::PodIdentityAssociation"
       Properties:
         ClusterName: !Ref "Cluster"
         Namespace: !Ref "Namespace"
         RoleArn: !GetAtt MissionControlRole.Arn
         ServiceAccount: mission-control-sa
   
   CanaryCheckerServiceAccount:
       Type: "AWS::EKS::PodIdentityAssociation"
       Properties:
         ClusterName: !Ref "Cluster"
         Namespace: !Ref "Namespace"
         RoleArn: !GetAtt MissionControlRole.Arn
         ServiceAccount: canary-checker-sa
   
     ConfigDBServiceAccount:
       Type: "AWS::EKS::PodIdentityAssociation"
       Properties:
         ClusterName: !Ref "Cluster"
         Namespace: !Ref "Namespace"
         RoleArn: !GetAtt MissionControlRole.Arn
         ServiceAccount: config-db-sa
   

3. Create a new stack

   aws cloudformation deploy \
   	--stack-name mission-control-roles \
   	--template-file file://mission-control-iam-cloudformation.yaml \
   	--parameter-overrides Cluster==$CLUSTER Namespace=$NAMESPACE
   

4. Choose a routable DOMAIN for Mission Control

   See Ingress for more options on configuring the ingress including generating certs with cert-manager

   See Local Testing for testing using a kind or minikube without a routable domain

5. Install Mission Control

   Helm

   helm repo add flanksource https://flanksource.github.io/charts 
   helm repo update
   helm install mission-control flanksource/mission-control \
    -n mission-control --create-namespace \
    --wait 

   Flux

   apiVersion: v1
   kind: Namespace
   metadata:
     name: mission-control
   ---
   apiVersion: source.toolkit.fluxcd.io/v1beta1
   kind: HelmRepository
   metadata:
     name: flanksource
     namespace: mission-control
   spec:
     interval: 5m0s
     url: https://flanksource.github.io/charts
   ---
   apiVersion: helm.toolkit.fluxcd.io/v2beta1
   kind: HelmRelease
   metadata:
     name: mission-control
     namespace: mission-control
   spec:
     chart:
       spec:
         chart: mission-control
         sourceRef:
           kind: HelmRepository
           name: flanksource
           namespace: mission-control
         interval: 1m
     values:
     
   See values.yaml

Access Key

Security Warning

Using Access Keys and Secrets is not recommended from a security perspective

First we create a secret called aws containing the access key and secret.

1. Choose a routable DOMAIN for Mission Control

   See Ingress for more options on configuring the ingress including generating certs with cert-manager

   See Local Testing for testing using a kind or minikube without a routable domain

2. Install Mission Control

   Helm

   helm repo add flanksource https://flanksource.github.io/charts 
   helm repo update
   helm install mission-control flanksource/mission-control \
    -n mission-control --create-namespace \
    --wait 

   Flux

   apiVersion: v1
   kind: Namespace
   metadata:
     name: mission-control
   ---
   apiVersion: source.toolkit.fluxcd.io/v1beta1
   kind: HelmRepository
   metadata:
     name: flanksource
     namespace: mission-control
   spec:
     interval: 5m0s
     url: https://flanksource.github.io/charts
   ---
   apiVersion: helm.toolkit.fluxcd.io/v2beta1
   kind: HelmRelease
   metadata:
     name: mission-control
     namespace: mission-control
   spec:
     chart:
       spec:
         chart: mission-control
         sourceRef:
           kind: HelmRepository
           name: flanksource
           namespace: mission-control
         interval: 1m
     values:
     
   See values.yaml

3. Create a new IAM User and Access Key

   USER_NAME="mission-control-sa"
   
   aws iam create-user --user-name $USER_NAME
   aws iam attach-user-policy \
     	--user-name $USER_NAME \
    	--policy-arn arn:aws:iam::aws:policy/ReadOnlyAccess
   key=$(aws iam create-access-key --user-name $USER_NAME)
   

4. Create a new secret aws containing the access and secret key

   kubectl create secret  generic aws \
   	--from-literal=AWS_ACCESS_KEY_ID=$(echo $key | jq -r '.AccessKey.AccessKeyId') \
   	--from-literal=AWS_SECRET_ACCESS_KEY=$(echo $key | jq -r '.AccessKey.SecretAccessKey')
   

5. Create a new connection

   aws-connection.yaml
   apiVersion: mission-control.flanksource.com/v1
   kind: Connection
   metadata:
   	name: aws
   	namespace: mission-control
   spec:
   	region: eu-west-1
   	accessKey:
   		valueFrom:
   			secretKeyRef:
   				name: aws
   				key: AWS_ACCESS_KEY_ID
   	secretKey:
   		valueFrom:
   			secretKeyRef:
   				name: aws
   				key: AWS_ACCESS_KEY_ID
   

6. When creating Scrapers / Registry bundles you can now refer to connection://mission-control/aws

values.yaml

The following table lists the configurable parameters and their default values:

Parameter	Description	Default
labels	Additional labels to apply to resources.
scraper.name	Name of the AWS scraper.	"aws"

Parameter	Description	Schema	Default
connection	AWS connection details.	Connection	""
accessKey	Name of the access key.	EnvVar	""
secretKey	Name of the secret key.	EnvVar	""
regions	AWS regions to pull from	[]string	[]
endpoint	AWS endpoint.	string	""
skipTLSVerify	Skip TLS verification.	bool	false
assumeRole	Assume AWS role.	string	""
cloudtrail.maxAge	Maximum age for CloudTrail.	""
cloudtrail.exclude	List of excluded items for CloudTrail.	[]
compliance	Enable or disable compliance.	true
costReporting.enabled	Enable or disable cost reporting.	false
costReporting.database	Cost reporting database.	""
costReporting.region	Cost reporting region.	""
costReporting.s3BucketPath	S3 bucket path for cost reporting.	""
costReporting.table	Table for cost reporting.	""
includeResources	List of resources to include.	[]
excludeResources	List of resources to exclude.	[]

6Cost & Usage Reporting

Prerequisites

• Cost and Usage Reports are configured with an Athena table
• The AWSQuicksightAthenaAccess policy or similar is attached to config-db IAM role