The Pipeline in this zipped example, MetadataCatalog_Insert_Read_Example.zip, demonstrates how you can:

• Use the Catalog Insert Snap to update metadata tables.
• Use the Catalog Query Snap to read the updated metadata information.

• File Reader
• JSON Parser
• Parquet Writer
• Catalog Insert
• Catalog Query

Data Catalog Partition Overwrite Pipeline

The following Pipeline demonstrates how to overwrite an existing partition by using the three Data Catalog Service Snaps: Catalog Query , Catalog Delete , and Catalog Insert . Because you cannot edit the Table Assets in Manager, you can use this Pipeline as a way for overwriting partitions in the Data Catalog.

• Catalog Query
• Catalog Delete
• JSON Generator
• Parquet Writer
• Catalog Insert

This example Pipeline queries a partition in the table, deletes the partition, generates a JSON document, and writes it to S3, inserting the metadata into the Table Asset in Manager.

• Catalog Query
• Catalog Delete
• JSON Generator
• Parquet Writer
• Catalog Insert