Why does Tokenize to Rows keep null rows?

Because row explosion should never drop data implicitly.

If a row has:

  • No matches

  • Or a null input

The node emits a row with null instead of removing it.

Why?

Dropping rows during tokenization can:

  • Break row counts

  • Corrupt joins

  • Mask missing data

Think of it as a LEFT explode, not an INNER explode.