[kdeldycke]

Description

Add a text_len parameter to textwrap.wrap(), textwrap.fill(), textwrap.shorten() and textwrap.TextWrapper. It sets how the width of a string is measured, so text with double-width or zero-width characters wraps to the right number of columns:

>>> import textwrap, unicodedata
>>> def display_width(text):
...     return sum(2 if unicodedata.east_asian_width(c) in 'FW' else 1 for c in text)
...
>>> textwrap.wrap('東京 大阪 京都 札幌', width=10)
['東京 大阪 京都', '札幌']
>>> textwrap.wrap('東京 大阪 京都 札幌', width=10, text_len=display_width)
['東京 大阪', '京都 札幌']

text_len must be additive: the width of a string is the sum of the widths of its characters. ANSI escape sequences do not follow this rule, so this PR does not touch the argparse fix of #154634.

Context

I am a maintainer of Click and had to maintain my own local fork of TextWrapper to make it strip ANSI strings properly. See my initial code at: pallets/click#3420 .

This PR was initially a direct port of that code to Python standard library. But after the fact I realized that a width function cannot handle escape sequences, so this PR was fixes to only focus on variable-width characters.

This PR:

• Closes Add text_len to TextWrapper for variable character width awareness #154686
• Closes textwrap.wrap: add control for fonts with different character widths #56708
• Supersede gh-56708: support custom len function in textwrap.wrap #28136 which has been stuck for 5 years
• Edits the same lines as gh-158979: Fix infinite loop in textwrap when indent is wider than width #158988, gh-160043: Don't reject fitting text in textwrap.shorten() and wrap(max_lines) #160044 and gh-160633: Break the last chunk of a broken long word in textwrap truncation #160634: I will rebase if we see progress on these

• Issue: Add text_len to TextWrapper for variable character width awareness #154686

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #35702498 | 📁 Comparing 56ea2d3 against main (f1c5363)

  🔍 Preview build  

453 files changed · + 13 added · ± 440 modified

+ Added

• builtins/constants.html
• builtins/exceptions.html
• builtins/functions.html
• builtins/index.html
• builtins/stdtypes.html
• builtins/threadsafety.html
• builtins/time-complexity.html
• c-api/pyexpat.html
• library/asyncio-tools.html
• library/time-complexity.html
• and 3 more...

± Modified

• glossary.html
• index.html
• license.html
• py-modindex.html
• _static/tachyon-example-heatmap.html
• c-api/arg.html
• c-api/bool.html
• c-api/buffer.html
• c-api/bytearray.html
• c-api/bytes.html
• and 430 more...

[github-actions]

This PR is stale because it has been open for 90 days with no activity.

[kdeldycke]

I just reworked this PR to focus on the cases of wrapping variable-width characters, add more edge-cases around these and drop all references to ANSI so I do not need to touch the argparse workaround introduced in #154634.

This PR relates to the following issues and does not try to fix them but need them for full coverage of edge-cases:

• gh-158979: Fix infinite loop in textwrap when indent is wider than width #158988
• textwrap.wrap() and fill() loop forever when the indent is wider than width #158979
• gh-160043: Don't reject fitting text in textwrap.shorten() and wrap(max_lines) #160044
• textwrap.shorten() raises "placeholder too large" for text that already fits #160043
• gh-160633: Break the last chunk of a broken long word in textwrap truncation #160634
• textwrap truncation discards a broken long word (and, for shorten(), the whole text) instead of breaking it further #160633

So if any of the one above get merged first I will need to rebase this one and rework it a bit.