1. About this repo
2. Scenarios covered
3. How does it work?
   1. Bulk import
   2. Bulk export
4. Pre-requisites
5. How to use?
   1. Configure
   2. Run bulk import scenarios
   3. Run bulk export scenarios
6. Clean up

AWS IoT SiteWise now supports the bulk import, export, and update of industrial equipment metadata for modeling at scale. These bulk operations are accessible through new API endpoints such as CreateMetadataTransferJob, ListMetadataTransferJobs, GetMetadataTransferJob and CancelMetadataTransferJob.

This repository provides code samples that allow you to explore the bulk operations feature and understand its application in typical real-world customer scenarios.

The table below outlines common scenarios for customers with industrial manufacturing facilities and how bulk operations can address their needs.

The following steps explain the process involved in both bulk import and bulk export operations.

1. Prepare a bulk definitions JSON file for SiteWise resources such as asset models and assets, following the AWS IoT SiteWise metadata transfer job schema, and upload this file to an Amazon S3 bucket.
2. Make a bulk import call to AWS IoT SiteWise, referencing the uploaded definitions JSON file.
3. AWS IoT SiteWise will import all the resources specified in the definitions JSON file.
4. Upon completion, AWS IoT SiteWise will return the status and a pre-signed S3 URL for any failures encountered.
5. In the event of failures, access the provided report to investigate and understand the root cause.

1. Specify the asset models and assets to be exported, the destination S3 bucket, and initiate a bulk export call to AWS IoT SiteWise
2. AWS IoT SiteWise will export the definitions of all resources that meet the specified filtering conditions.
3. Download the exported definitions JSON file for further use.
4. Upon completion, AWS IoT SiteWise will return the status and provide a pre-signed S3 URL for any failures encountered.
5. In the event of failures, access the provided report to investigate and determine the root cause.

1. Configure AWS credentials.
2. An S3 bucket to store the bulk definitions
3. To perform bulk operations, you must create an AWS Identity and Access Management (IAM) policy with permissions that allow the exchange of AWS resources between Amazon S3, AWS IoT SiteWise, and your local machine. Refer to IAM permissions.
4. Ensure Python 3.x is installed on your system, you can verify by running python3 --version or python --version (on Windows).
5. Clone this Git repository and install required Python packages by running pip3 install -r requirements.txt

Update config/project_config.yml to provide necessary information for the job.

• s3_bucket_name: Name of the S3 bucket where bulk definitions will be stored.
• job_name_prefix: Prefix to be used for the bulk operations jobs.

Create an initial set of asset models and assets in AWS IoT SiteWise for an automotive manufacturing company.

python3 src/import/main.py --bulk-definitions-file 1_onboard_models_assets.json

Define relationships between assets and create an asset hierarchy.

python3 src/import/main.py --bulk-definitions-file 2_define_asset_hierarchy.json

Associate data streams with assets for contextualization.

python3 src/import/main.py --bulk-definitions-file 3_associate_data_streams_with_assets.json

Create additional assets Sample_Welding Robot 3 and Sample_Welding Robot 4 and a new production line Sample_Welding Line 2.

python3 src/import/main.py --bulk-definitions-file 4_onboard_additional_assets.json

Add a new property Joint 1 Temperature to Sample_Welding Robot asset model.

python3 src/import/main.py --bulk-definitions-file 5_onboard_new_properties.json

Correct the serial number of Sample_Welding Robot 1 by replacing the old serial number S1000 with S1001.

python3 src/import/main.py --bulk-definitions-file 6_fix_incorrect_datastreams.json

Re-arrange assets to adapt to changes in production line operations. Move Sample_Welding Robot 3 asset to Sample_Welding Line 2 production line.

python3 src/import/main.py --bulk-definitions-file 7_relocate_assets.json

Promote the resources from developement to QA environment. The definitions file such as 7_promote_to_another_environment.json can be obtained using bulk export operation.

Redo the Pre-requisites and Configure in your target account, then run the following command.

python3 src/import/main.py --bulk-definitions-file 9_promote_to_another_environment.json

Apply external identifier to an existing asset, for example, Sample_Welding Robot 4.

python3 src/import/main.py --bulk-definitions-file 10_apply_external_identifier.json

Compose Sample_Welding Robot asset model by independently modeling components in a welding robot such as robot joints.

python3 src/import/main.py --bulk-definitions-file 11_compose_models.json

You can backup all or specific resources using a bulk export operation. Learn more at Running a bulk export job.

Replace <YOUR_ASSET_ID> in 6_backup_models_assets.json with an asset ID of your choice.

Run the following to export the definitions of the given asset and all the assets below it in the hierachy.

python3 src/export/main.py --job-config-file 8_backup_models_assets.json

If you no longer require the solution, remove the resources.

Run the following to remove all the asset models and assets created using this sample repository.

python3 src/remove_sitewise_resources.py --asset-external-id External_Id_Company_AnyCompany