Transfer · From Tasks · from_snowflake

Purpose

Imports data from Snowflake. If you have a Snowflake database configured as your client_cloud you should probably use the do task.

Method of use

Before you can download data from Snowflake you have to give acces.

Configuration

Example usage

extract:
    query: SELECT 1 AS int_field, '{{ task.run_id }}' AS string_field, CURRENT_TIMESTAMP() AS timestamp_field
    db_conn_id: snowflake
    storage_conn_id: aws_s3
    database: PRODUCTION
    schema: OSB_RAW
    bucket: data-exchange-bucket
    folder: some-folder-that-holds-the-data

Properties

propertytypemandatorydescription
conn_idstringnoDefault is snowflake. Connection string as handed to you by the Onesecondbefore team.
querystringnoUse either query or template (below). Query to be executed, whose results will be uploaded to the destination.
templatestringnoUse either query(above) or template. Contains a link to a file in the `includes` folder in your repository that contains the SQL statement. This query will be executed and the results will be uploaded to the destination.
db_conn_idstringnoConnection to use to connect to the BigQuery database
storage_conn_idstringnoConnection to use to connect to blob storage.
databasestringnoName of the database of the table to extract to blob storage.
schemastringnoName of the schema in which the temporary table will be created. The table id will be _tmp_{{ task.run_id }}
bucketstringnoBucket to which the data will be extracted from the temporary table.
folderstringnoFolder to which the data will be extracted from the temporary table.

Details

itemdescription
Pre-formatted schemaNo. Use config schema if you want to add descriptions.