Snowflake
Allows the connection to Snowflake to automatically retrieve from or publish data to a Snowflake table.
Code repository: https://github.com/AmadeusITGroup/dataio-framework/tree/main/src/main/scala/com/amadeus/dataio/pipes/snowflake
Useful links:
Common
The following fields are available for all Snowflake components:
| Name | Mandatory | Description | Example | Default |
|---|---|---|---|---|
| options | No | Snowflake options to specify such as key = value pairs. These options are then passed as option to the Spark connector for Snowflake | options {"sfDatabase" = "my_db" } |
Batch
Input
Type: com.amadeus.dataio.pipes.snowflake.batch.SnowflakeInput
Output
Type: com.amadeus.dataio.pipes.snowflake.batch.SnowflakeOutput
| Name | Mandatory | Description | Example | Default |
|---|---|---|---|---|
| mode | true | Writing mode on the Snowflake table | mode = "append" |
No streaming processing is currently available for Snowflake in Data I/O.