class RayJob (construct)
Language | Type name |
---|---|
.NET | Amazon.CDK.AWS.Glue.Alpha.RayJob |
Go | github.com/aws/aws-cdk-go/awscdkgluealpha/v2#RayJob |
Java | software.amazon.awscdk.services.glue.alpha.RayJob |
Python | aws_cdk.aws_glue_alpha.RayJob |
TypeScript (source) | @aws-cdk/aws-glue-alpha 禄 RayJob |
Implements
IConstruct
, IDependable
, IResource
, IJob
, IGrantable
Ray Jobs class.
Glue Ray jobs use worker type Z.2X and Glue version 4.0. These are not overrideable since these are the only configuration that Glue Ray jobs currently support. The runtime defaults to Ray2.4 and min workers defaults to 3.
Example
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.RayJob(stack, 'ImportedJob', { role, script });
Initializer
new RayJob(scope: Construct, id: string, props: RayJobProps)
Parameters
- scope
Construct
- id
string
- props
Ray
Job Props
RayJob constructor.
Construct Props
Name | Type | Description |
---|---|---|
role | IRole | IAM Role (required) IAM Role to use for Glue job execution Must be specified by the developer because the L2 doesn't have visibility into the actions the script(s) takes during the job execution The role must trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions. |
script | Code | Script Code Location (required) Script to run when the Glue job executes. |
connections? | IConnection [] | Connections (optional) List of connections to use for this Glue job Connections are used to connect to other AWS Service or resources within a VPC. |
continuous | Continuous | Enables continuous logging with the specified props. |
default | { [string]: string } | Default Arguments (optional) The default arguments for every run of this Glue job, specified as name-value pairs. |
description? | string | Description (optional) Developer-specified description of the Glue job. |
enable | boolean | Enables the collection of metrics for job profiling. |
glue | Glue | Glue Version The version of Glue to use to execute this job. |
job | string | Name of the Glue job (optional) Developer-specified name of the Glue job. |
job | boolean | Specifies whether job run queuing is enabled for the job runs for this job. |
max | number | Max Concurrent Runs (optional) The maximum number of runs this Glue job can concurrently run. |
max | number | Max Retries (optional) Maximum number of retry attempts Glue performs if the job fails. |
number | number | Number of Workers (optional) Number of workers for Glue to use during job execution. |
runtime? | Runtime | Sets the Ray runtime environment version. |
security | ISecurity | Security Configuration (optional) Defines the encryption options for the Glue job. |
tags? | { [string]: string } | Tags (optional) A list of key:value pairs of tags to apply to this Glue job resources. |
timeout? | Duration | Timeout (optional) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. |
worker | Worker | Worker Type (optional) Type of Worker for Glue to use during job execution Enum options: Standard, G_1X, G_2X, G_025X. |
role
Type:
IRole
IAM Role (required) IAM Role to use for Glue job execution Must be specified by the developer because the L2 doesn't have visibility into the actions the script(s) takes during the job execution The role must trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.
See also: https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html
script
Type:
Code
Script Code Location (required) Script to run when the Glue job executes.
Can be uploaded from the local directory structure using fromAsset or referenced via S3 location using fromBucket
connections?
Type:
IConnection
[]
(optional, default: [] - no connections are added to the job)
Connections (optional) List of connections to use for this Glue job Connections are used to connect to other AWS Service or resources within a VPC.
continuousLogging?
Type:
Continuous
(optional, default: continuous logging is enabled.)
Enables continuous logging with the specified props.
See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
defaultArguments?
Type:
{ [string]: string }
(optional, default: no arguments)
Default Arguments (optional) The default arguments for every run of this Glue job, specified as name-value pairs.
See also: [https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters)
description?
Type:
string
(optional, default: no value)
Description (optional) Developer-specified description of the Glue job.
enableProfilingMetrics?
Type:
boolean
(optional, default: no profiling metrics emitted.)
Enables the collection of metrics for job profiling.
See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
glueVersion?
Type:
Glue
(optional, default: 3.0 for ETL)
Glue Version The version of Glue to use to execute this job.
jobName?
Type:
string
(optional, default: a name is automatically generated)
Name of the Glue job (optional) Developer-specified name of the Glue job.
jobRunQueuingEnabled?
Type:
boolean
(optional, default: no job run queuing)
Specifies whether job run queuing is enabled for the job runs for this job.
A value of true means job run queuing is enabled for the job runs. If false or not populated, the job runs will not be considered for queueing. If this field does not match the value set in the job run, then the value from the job run field will be used. This property must be set to false for flex jobs. If this property is enabled, maxRetries must be set to zero.
maxConcurrentRuns?
Type:
number
(optional, default: 1)
Max Concurrent Runs (optional) The maximum number of runs this Glue job can concurrently run.
An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.
maxRetries?
Type:
number
(optional, default: 0)
Max Retries (optional) Maximum number of retry attempts Glue performs if the job fails.
numberOfWorkers?
Type:
number
(optional, default: 10)
Number of Workers (optional) Number of workers for Glue to use during job execution.
runtime?
Type:
Runtime
(optional, default: Runtime version will default to Ray2.4)
Sets the Ray runtime environment version.
securityConfiguration?
Type:
ISecurity
(optional, default: no security configuration.)
Security Configuration (optional) Defines the encryption options for the Glue job.
tags?
Type:
{ [string]: string }
(optional, default: {} - no tags)
Tags (optional) A list of key:value pairs of tags to apply to this Glue job resources.
timeout?
Type:
Duration
(optional, default: 2880 (2 days for non-streaming))
Timeout (optional) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
Specified in minutes.
workerType?
Type:
Worker
(optional, default: WorkerType.G_1X)
Worker Type (optional) Type of Worker for Glue to use during job execution Enum options: Standard, G_1X, G_2X, G_025X.
G_4X, G_8X, Z_2X
Properties
Name | Type | Description |
---|---|---|
env | Resource | The environment this resource belongs to. |
grant | IPrincipal | The principal to grant permissions to. |
job | string | The ARN of the job. |
job | string | The name of the job. |
node | Node | The tree node. |
role | IRole | The IAM role Glue assumes to run this job. |
stack | Stack | The stack in which this resource is defined. |
env
Type:
Resource
The environment this resource belongs to.
For resources that are created and managed by the CDK (generally, those created by creating new class instances like Role, Bucket, etc.), this is always the same as the environment of the stack they belong to; however, for imported resources (those obtained from static methods like fromRoleArn, fromBucketName, etc.), that might be different than the stack they were imported into.
grantPrincipal
Type:
IPrincipal
The principal to grant permissions to.
jobArn
Type:
string
The ARN of the job.
jobName
Type:
string
The name of the job.
node
Type:
Node
The tree node.
role
Type:
IRole
The IAM role Glue assumes to run this job.
stack
Type:
Stack
The stack in which this resource is defined.
Methods
Name | Description |
---|---|
apply | Apply the given removal policy to this resource. |
metric(metricName, type, props?) | Create a CloudWatch metric. |
metric | Return a CloudWatch Metric indicating job failure. |
metric | Return a CloudWatch Metric indicating job success. |
metric | Return a CloudWatch Metric indicating job timeout. |
on | Create a CloudWatch Event Rule for this Glue Job when it's in a given state. |
on | Return a CloudWatch Event Rule matching FAILED state. |
on | Create a CloudWatch Event Rule matching JobState.SUCCEEDED. |
on | Return a CloudWatch Event Rule matching TIMEOUT state. |
setup | Setup Continuous Loggiung Properties. |
to | Returns a string representation of this construct. |
applyRemovalPolicy(policy)
public applyRemovalPolicy(policy: RemovalPolicy): void
Parameters
- policy
Removal
Policy
Apply the given removal policy to this resource.
The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you've removed it from the CDK application or because you've made a change that requires the resource to be replaced.
The resource can be deleted (RemovalPolicy.DESTROY
), or left in your AWS
account for data recovery and cleanup later (RemovalPolicy.RETAIN
).
metric(metricName, type, props?)
public metric(metricName: string, type: MetricType, props?: MetricOptions): Metric
Parameters
- metricName
string
鈥� name of the metric typically prefixed withglue.driver.
,glue.<executorId>.
orglue.ALL.
. - type
Metric
鈥� the metric type.Type - props
Metric
鈥� metric options.Options
Returns
Create a CloudWatch metric.
See also: https://docs.aws.amazon.com/glue/latest/dg/monitoring-awsglue-with-cloudwatch-metrics.html
metricFailure(props?)
public metricFailure(props?: MetricOptions): Metric
Parameters
- props
Metric
Options
Returns
Return a CloudWatch Metric indicating job failure.
This metric is based on the Rule returned by no-args onFailure() call.
metricSuccess(props?)
public metricSuccess(props?: MetricOptions): Metric
Parameters
- props
Metric
Options
Returns
Return a CloudWatch Metric indicating job success.
This metric is based on the Rule returned by no-args onSuccess() call.
metricTimeout(props?)
public metricTimeout(props?: MetricOptions): Metric
Parameters
- props
Metric
Options
Returns
Return a CloudWatch Metric indicating job timeout.
This metric is based on the Rule returned by no-args onTimeout() call.
onEvent(id, options?)
public onEvent(id: string, options?: OnEventOptions): Rule
Parameters
- id
string
鈥� construct id. - options
On
鈥� event options.Event Options
Returns
Create a CloudWatch Event Rule for this Glue Job when it's in a given state.
See also: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types
onFailure(id, options?)
public onFailure(id: string, options?: OnEventOptions): Rule
Parameters
- id
string
鈥� construct id. - options
On
鈥� optional event options.Event Options
Returns
Return a CloudWatch Event Rule matching FAILED state.
onSuccess(id, options?)
public onSuccess(id: string, options?: OnEventOptions): Rule
Parameters
- id
string
鈥� construct id. - options
On
鈥� optional event options.Event Options
Returns
Create a CloudWatch Event Rule matching JobState.SUCCEEDED.
onTimeout(id, options?)
public onTimeout(id: string, options?: OnEventOptions): Rule
Parameters
- id
string
鈥� construct id. - options
On
鈥� optional event options.Event Options
Returns
Return a CloudWatch Event Rule matching TIMEOUT state.
setupContinuousLogging(role, props?)
public setupContinuousLogging(role: IRole, props?: ContinuousLoggingProps): any
Parameters
- role
IRole
鈥� The IAM role to use for continuous logging. - props
Continuous
鈥� The properties for continuous logging configuration.Logging Props
Returns
any
Setup Continuous Loggiung Properties.
toString()
public toString(): string
Returns
string
Returns a string representation of this construct.