What users observe
Columns with mixed values behave inconsistently.
Sparkflows’ column-level decision model
Sparkflows makes decisions at the column level, not per cell.
-
If all sampled values are numeric → numeric type
-
If mixed numeric and text →
STRING -
If numeric type chosen and text appears later →
null
Sparkflows does not change column type mid-stream.
Why this matters
Dynamic type switching would:
-
Break Spark schemas
-
Make pipelines unpredictable
-
Introduce silent failures
Recommendation
Separate mixed-purpose columns in Excel, or enforce schema explicitly.