You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
StringLiteral to IntegerType/LongType silently truncates fractional strings, changing predicate meaning #4382
literal("2.6").to(IntegerType()) returns LongLiteral(2) instead of raising, while DecimalLiteral(Decimal("2.6")).to(IntegerType()) raises ValueError. A bound predicate such as x < "2.6" on an int column therefore becomes x < 2 and wrongly drops rows with x = 2 (verified with expression_evaluator); the same bound literal feeds partition pruning, so files can be skipped too.
The sibling DecimalLiteral.to(IntegerType/LongType) already rejects fractional values via the existing _to_integral helper, with a test spelling out why: rounding would change the predicate (e.g. x > 2.6 becoming x > 3 drops x = 3). The StringLiteral path was missed. The reference Java implementation has no INTEGER/LONG case in StringLiteral.to() and refuses the conversion.
Suggested fix: use the existing _to_integral helper in both StringLiteral.to(IntegerType) and StringLiteral.to(LongType), and update test_string_literal to expect ValueError for fractional strings.
Activity
SurenderguliaEnginner commentedon Nov 15, 2026
Hi @Shahzeb-DataEngineer, I'd like to work on this and submit a PR if you aren't already working on it!
Shahzeb-DataEngineer commentedon Nov 15, 2026
Hi @SurenderguliaEnginner, thanks for offering! I already have a fix in progress, PR #4383 is open for this. Appreciate it though.