Data Automation

New Features

  • The ELT Snap Pack supports the latest JDBC drivers across CDWs—Azure Synapse, BigQuery, DLP, Redshift, and Snowflake. See Configuring ELT Database Accounts or the respective Account page for the exact versions.
  • The end Snap SQL query in the ELT Insert Select Snap’s preview output now displays the appropriate CREATE TABLE... or the DELETE/DROP TABLE statements to be run before the query that inserts/loads data into a new table in the Snowflake target CDW.
  • The Value List field in the ELT Pivot Snap displays a dynamic list of suggestions to select from.
  • The ELT SCD2 Snap:
    • Allows replacing an existing target table with a new table and loading the SCD2 entries into it from the source table/files. Use the new Overwrite existing table option in the Target Table action field to perform this action.
    • Displays the final SQL query in its output preview upon Pipeline validation.

Fixed Issues

  • Field names in ELT Aggregate, ELT Cast Function, ELT String Function, and ELT Transform Snaps are normalized.
  • The ELT Insert Select, ELT Merge Into, ELT Load, and the ELT SCD2 Snaps now run successfully even when the specified target table does not exist. The Snaps now create a new target table if it does not exist during Pipeline validation.
    Warning: The new table thus created will not be dropped in the event of a subsequent/downstream Snap failure during validation.
  • The ELT Load Snap does not cause:
    • Any null values in new columns added to the target table through the Alter table Load Action.
    • The SQL exception [Simba][SparkJDBCDriver](500051) ERROR processing query/statement when reading from a CSV file in S3 mount point on DBFS in the case of a DLP target instance.