3736857 - Sharing a Data Product from SAP Application with Null date values are changing to negative dates in Databricks

- Null values Date type fields in SAP Application table are automatically changing to negative dates in the Bronze Data product within Databricks.
- The issue occurs during the execution of a Transformation Flow.

• SAP Datasphere
• SAP Databricks

In SAP Application , a blank/uninitialized date is stored as the string 00000000 (8 zeros, in YYYYMMDD format) rather than a true SQL NULL. 
This is a SAP convention — it means "no date set."
                                                                  
During ingestion the SAP BDC extracts 00000000, and Databricks/Spark tries to parse the value as follows:                      

1. 00000000 → year 0000, month 00, day 00
2. Month 00 and day 00 are invalid in the Gregorian calendar
3. Spark/Java's date arithmetic overflows backwards: day 0 of month 0 → rolls back to Nov 30 of year -1 (i.e., -0001-11-30 in ISO 8601 proleptic Gregorian calendar)

This is essentially "day zero" in Java's internal date representation when SAP's zero-date gets numeric-cast to 0.   

• Please confirm whether the date is being saved as `00000000` in SAP Applications for NULL values.
• Please ensure that dates are non-null in SAP Applications.
• If null values are unavoidable, then handle them on the Databricks side for downstream operations.
  
  Filter records from Databricks for downstream processing, something like below:
  SELECT
    CASE
    WHEN date_col < '0001-01-01' OR date_col = date('0000-01-01') THEN NULL
    ELSE date_col
    END AS date_col
   FROM your_table; 

For more details regarding date representations, please check the following product document links:

SAP Product Documents
Design considerations for SAP application null behavior
Built-In ABAP Types

Databricks Product Documents
Dates and timestamps

SAP Product Documents
Design considerations for SAP application null behavior
Built-In ABAP Types

Databricks Product Documents
Dates and timestamps

Null Dates, Negative Date in Databricks , KBA , BDC-DBX-XFR , BDC Databricks Transfer , BDC-DBX-CON , Databricks Connect Service , BDC-CON , SAP Business Data Cloud Connect , Problem