Symptom
You are looking for a way to use the option [Delete All Before Loading] in Replication Flow when replicating into Google BigQuery target connection, but this feature is not currently available.
When using Replication Flow to replicate data into Google BigQuery target:
- The target is always written in Append mode.
- Option [Delete All Before Loading] is not available.
- Each execution of the replication job appends new data into BigQuery without removing existing data.
- As a result, duplicate records may appear.
Environment
SAP Datasphere
Cause
- Google BigQuery target connections in SAP Datasphere only support Append write mode.
- The option Delete All Before Loading is not available for this target.
- This is a product limitation.
Resolution
The currently available options are:
- Manual cleanup: Drop or truncate the target table in BigQuery before data loading.
Also there is/are ongoing campaign(s) for this enhancement request on the SAP Customer Influence Portal for SAP Datasphere.
- Enable Truncate data for Bigquery Export - Customer Influence
- "Delete All Before Loading " is not enabled when Google BigQuery table is selected as Target - Customer Influence
Customers can vote for these requests on the SAP Customer Influence Portal to increase visibility and help prioritize this feature.
Please contact with your Customer Success Partner (CSP) to get updates on this enhancement request.
Furthermore, stay tuned about upcoming features in SAP Datasphere Roadmap.
See Also
- Google BigQuery Connections
- Using Google BigQuery As the Target
- KBA 3332382 - How to create an enhancement request for SAP Datasphere?
- SAP Customer Influence Portal for SAP Datasphere
- Roadmap for SAP Datasphere
Keywords
Overwrite Target Settings at Object Level, GBQ, UPSERT, CDS view keys, replication flow, Google BigQuery, Append , KBA , DS-DI-RF , Replication Flow Runtime , CA-DI-IS-RMS-CON , RMS connectivity , Problem
SAP Knowledge Base Article - Public