Skip to content

fix(expressions): reject fractional strings converting to int/long - #4383

Open
Shahzeb-DataEngineer wants to merge 6 commits into
apache:mainfrom
Shahzeb-DataEngineer:issue-string-literal-fractional-truncate
Open

Shahzeb-DataEngineer wants to merge 6 commits into
apache:mainfrom
Shahzeb-DataEngineer:issue-string-literal-fractional-truncate

Conversation

@Shahzeb-DataEngineer

Copy link
Copy Markdown
Contributor

Closes #4382

Rationale for this change

StringLiteral.to(IntegerType) and StringLiteral.to(LongType) parsed with Decimal then truncated with plain int(), so "2.6" became 2 and a bound predicate such as x < "2.6" on an int column silently became x < 2, wrongly dropping rows with x = 2 and feeding the same wrong literal into partition pruning. The sibling DecimalLiteral path already rejects fractional values via _to_integral, and the Java reference implementation refuses the conversion entirely. This change reuses _to_integral in both StringLiteral paths.

Are these changes tested?

Yes. New test test_fractional_string_to_integral_type_raises mirrors test_fractional_decimal_to_integral_type_raises; it fails on unpatched code and passes with the fix. test_string_literal updated to no longer assert the truncating behavior. tests/expressions/test_literals.py: 180 passed. tests/expressions/: 594 passed, 2 failed (pre-existing pyparsing-version failures on clean main, verified via git stash). ruff check and ruff format --check clean on both changed files.

Are there any user-facing changes?

Yes, behavior change for the better: converting a fractional string such as "2.6" to an int/long column now raises ValueError at bind time instead of silently truncating. Integral strings ("2.0", "1e3") still convert exactly.

Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 15, 2026 04:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>
Signed-off-by: Shahzeb Ali <126076773+Shahzeb-DataEngineer@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

StringLiteral to IntegerType/LongType silently truncates fractional strings, changing predicate meaning

2 participants