Symptom
When a source table or joined table is modified or updated, the transformation flow run will only continue in delta mode. This means that instead of performing an initial load, the system will only consider the delta changes made to the data.
Resolution
When running a transformation flow that loads delta changes to a target table, the system uses a watermark (a timestamp) to track the data that has been transferred.
-
If a table that can capture delta changes is joined with a second table, and columns in the second table have been updated, you can reset the watermark to ensure that these changes are reflected in records that have already been transferred.
-
If corrupt data is present in the target table, you can reset the watermark for the transformation flow to ensure that the latest data from the source table is loaded to the target table.
To find out more about Resetting The Watermark, please refer to our Help Portal.
Keywords
resetting the watermark, timestamp, transformation, delta, initial, load , KBA , DS-DI-TRF , Transformation Flows , Problem