Symptom
- Streaming projects are unable to process records when data type mismatch occurs on column values. The entire row gets skipped.
- Consider the below scenario:
- Data is being input to the project from a CSV or XML file using the File/Hadoop CSV Input or File/Hadoop Event XML Input Adapter.
- The Streaming schema is using the "time" data type for the field.
- Some records in the field contains value ‘0’ (which is not in time format).
- The input adapter is unable to parse the value ‘0’.
- The input adapter skips the entire row of data and processes the next row.
- In the CSV or XML file, suppose there are 40000 records, out of which 2000 rows have the value ‘0’ for the field X.
- When starting the Streaming project, the input adapter processes only 38000 rows.
- Because of one field with invalid data for the data type, the entire row of data is skipped and not processed.
- How to determine what data rows were skipped in this scenario?
Read more...
Environment
- SAP Event Stream Processor (ESP) 5.1 SP12 PL03
- SAP HANA Streaming
Product
SAP Event Stream Processor 5.1 ; SAP HANA 1.0 smart data streaming ; SAP HANA streaming analytics 2.0
Keywords
streaming, processing , KBA , BC-SYB-ESP , Sybase ESP (Event Stream Processor) , HAN-SDS , SAP HANA Smart Data Streaming , Problem
About this page
This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.