Format and upload your batch inference data - Amazon Bedrock

Format and upload your batch inference data

To prepare inputs for batch inference, create a .jsonl file in the following format:

{ "recordId" : "11 character alphanumeric string", "modelInput" : {JSON body} } ...

Each line contains a JSON object with a recordId field and a modelInput field containing the request body for an input you want to submit. The format of the modelInput JSON object must match the body field for the model that you use in the InvokeModel request. For more information, see Inference request parameters and response fields for foundation models.

Note
  • If you omit the recordId field, Amazon Bedrock adds it in the output.

  • You specify the model that you want to use when you create the batch inference job.

For example, you might provide a JSONL file containing the following line if you plan to run batch inference using the Anthropic Claude 3 Haiku model:

{ "recordId": "CALL0000001", "modelInput": { "anthropic_version": "bedrock-2023-05-31", "max_tokens": 1024, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Summarize the following call transcript: ..." } ] } ] } }

After preparing your input files, upload them to an S3 bucket. Attach the following permissions to your batch inference service role and replace ${{s3-bucket-input}} with the bucket that you uploaded the input files to and ${{s3-bucket-output}} with the bucket that you want to write the output files to.

{ "Version": "2012-10-17", "Statement": [ { "Action": [ "s3:GetObject", "s3:PutObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::${{s3-bucket-input}}", "arn:aws:s3:::${{s3-bucket-input}}/*", "arn:aws:s3:::${{s3-bucket-output}}", "arn:aws:s3:::${{s3-bucket-output}}/*" ], "Effect": "Allow" } ] }

If you are using batch inference for video understanding with the Nova Lite and Nova Pro models, ensure that the Amazon S3 path provided in the InputDataConfig field of the CreateModelInvocationJob API input is an Amazon S3 folder that contains all the input JSONL files you want to process and all the video objects referenced by the input records.

For example, if you provide the Amazon S3 path s3://batch-inference-input-data/job-input-folder-data/ as the InputDataConfig for a batch job, an appropriate data setup would resemble the following:

batch-inference-input-data/ job-input-folder-data/ input.jsonl video.mp4

A sample s3://batch-inference-input-data/job-input-folder-data/input.jsonl with invocation record for processing a video object is shown below. Note how the video object’s Amazon S3 URI is located under the same Amazon S3 object as the JSONL file here.

{ "recordId": "RECORD01", "modelInput": { "messages": [ { "role": "user", "content": [ { "text": "You are an expert in recipe videos. Describe this video in less than 200 words following these guidelines: ..." }, { "video": { "format": "mp4", "source": { "s3Location": { "uri": "s3://batch-inference-input-bucket/job-input-folder-data/video.mp4", "bucketOwner": "123456789012" } } } } ] } ] } }

Assuming valid Amazon S3 URI paths, the video object can be nested anywhere under the provided InputDataConfig Amazon S3 URI path. Refer to this Amazon S3 URL Parsing blog to learn how to proactively validate of Amazon S3 URIs in an input request.

Refer to Amazon Nova vision prompting guidelines for more information on how to set up invocation records for video understanding with Nova.