This module:
- deploys an IAM Role assumed by the DAG with permissions to execute the Jobs
- deploys a k8s
namespace
dedicated to simulations - deploys an IAM Role and associated k8s ServiceAccount that Jobs will use for execution
- deploys Airflow DAGs to the shared MWAA that execute k8s Jobs to mock simulations
simple_mock
DAG: uses a simplebash
script to sleep for a random amount of time and randomly fail the conatainercoarse_parallel_mock
DAG: uses an SQS Queue to orchestrate a mock Coarse Parallel Processing workflow, where each Pod takes a message off the queue, processes it, then exitsfine_parallel_mock
DAG: uses an an SQS Queue to orchestrat a mock Fine Parallel Processing workflow, where each Pod continuously processes messages off the queue until it is empty, then exits
dag-bucket-name
: name of the Bucket configured in the shared MWAA Environment to store DAG artifactsdag-path
: name of the path in the Bucket configured in the shared MWAA Environment to store DAG artifactsmwaa-exec-role-arn
: ARN of the MWAA Execution Roleeks-cluster-name
: name of the EKS Cluster to send Jobs toeks-cluster-kubectl-role-arn
: ARN of the IAM Role used to executekubectl
commands on the EKS Clustereks-oidc-arn
: ARN of the OpenID Connect Provider assigned to the EKS Clustereks-cluster-admin-role-arn
: ARN of the IAM Role configured as a Cluster Admin and associated with thesystem:masters
Kubernetes Group
DagRoleArn
: ARN of the DAG Execution Role created by the Stack- `
{
"DagRoleArn": "arn::::",
"EksServiceAccountRoleArn": "arn::::"
}