Sparkflows never converts invalid numeric values into strings once a column is inferred as numeric.
With the updated engine:
-
Valid numeric formats are parsed correctly
-
Percentages, accounting formats, and grouped numbers are normalized
-
Invalid numeric representations become
null -
The column type is preserved
Examples that become null:
-
Invalid comma placement
-
Mixed symbols in numeric columns
-
Formula errors
-
Text embedded in numeric data
Why this is intentional
Silently converting bad numeric values to strings causes:
-
Broken aggregations
-
Incorrect joins
-
Downstream data corruption
Sparkflows prefers correct schema with nulls over incorrect data.
Recommendation
If nulls are unexpected:
-
Inspect the original Excel formatting
-
Or enforce schema explicitly for full control