[C API] Add PyTupleWriter API #161694

Currently, to build a tuple, PyTuple_New() should be used with PyTuple_SetItem() or PyTuple_SET_ITEM(). This API has multiple issues: it modifies an immutable tuple and the tuple is directly tracked by the GC. There is also _PyTuple_Resize() function to resize an immutable tuple.

I propose adding a new PyTupleWriter C API to clarify the "build" phase which ends with a Finish() call (or Discard() on error or if the builder is no longer needed).

The problem of designing such API is that it should be as efficient or even more efficient than the existing PyTuple_New() + PyTuple_SET_ITEM() pattern. I made a first attempt in 2023. Last year, I made a second attempt which didn't convince other developers. The API was based on PyTupleWriter_Create() + PyTupleWriter_Add() + PyTupleWriter_Finish(), and PyTupleWriter_Create() allocated a structure on the heap memory.

I'm back with a different design: PyTupleWriter_Create() no longer allocates a structure on the heap memory, it's just a PyTupleObject* pointer hidden as an opaque PyTupleWriter* type. So PyTupleWriter_Create() and PyTupleWriter_Finish() are fast. The API basically allocates an array of PyObject* and the caller has to fill the array. The array is not initialized for best performance. The caller is responsible to fill all items, otherwise the tuple contains uninitialized memory.

See #139888 for the rationale on adding PyTupleWriter API.

Linked PRs

• gh-161694: [C API] Add PyTupleWriter API #161639