Workflows · Do Tasks · do_profiles

Purpose

The do_profiles task is used to synchronize the profiles table in the analytical database with the profiles in the profile database. Only when this step is done, the profiles are available in realtime.

Example usage

task:
    type: do_profiles

extract:
    table_id: profiles
    dataset_id: transfer_testdata
    project_id: factbase-augment
    limit: 10

deduplicate:
    type: truncate

load:
    project_id: factbase-7dj2
    log_result: no

Root properties

Extract
Deduplicate
Load

Properties: Extract

property	type	required	description
`project_id`	string	no	BiqQuery only. Default is `project_id` of `client_cloud`
`dataset_id`	string	yes	BiqQuery only. Dataset ID of the profiles table.
`table_id`	string	yes	BiqQuery only. Table ID of the profiles table.
`database`	string	yes	Snowflake only. Default is `database` of `client_cloud`. Database of the profiles table.
`schema`	string	yes	Snowflake only. Schema the profiles table.
`database`	string	yes	Snowflake only. Table of the profiles table.
`limit`	integer	no	Limits the amount of profiles to be synced with the profile server. Use in development mode.

Properties: Deduplicate

property	type	required	description
`type`	enumerator(replace, truncate)	no	Either `truncate` or `merge`. If set to truncate the complete profiles table will be truncated, before the profiles will be imported. This basically means a reset of the profile table. If not set or set to `replace` the profiles of the analytics database will replace the profiles in the profiles database.

Properties: Load

property	type	required	description
`project_id`	string	no	BiqQuery only. Default is `project_id` of `client_cloud`
`log_result`	yesno (boolean)	no	If set to yes, the task will count all profiles in the profile database and log the result. Extra charges by your cloud may apply. Use `no` in production situation.