THRIFT-6069: python: use a flat fastbinary encode buffer

[markjm]

Hi - I figured I'd share a few perf optimizations we are using internally. We are still on an older thrift version (😢 ), so I did use some agent magic to port these to the head of this repo. My testing was primarily on my branch, so buyer beware on that front!

This one is 1 of 3 PRs

⚠️ I did use AI tools to investigate and address feedback, but I am a real human ready to collaborate 😄

---

Replace std::vector back_inserter writes in the fastbinary encoder with a malloc/realloc buffer so encoded bytes can be copied in bulk while preserving the existing growth behavior.

Performance (50k iterations, warmed)

Message size	apache/master	branch	speedup
8 KiB	12.19 us	1.11 us	11.0x
64 KiB	101.58 us	5.48 us	18.5x
256 KiB	564.34 us	25.22 us	22.4x
1 MiB	2188.97 us	136.22 us	16.1x

Decode performance is unchanged because this only affects the encode path.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR adds coverage for Thrift’s accelerated binary protocol round-tripping a large TApplicationException, and refactors the Python C-extension encode buffer implementation away from std::vector to a manual malloc/realloc-backed buffer.

Changes:

• Add a unit test that exercises _fast_encode/_fast_decode via TBinaryProtocolAcceleratedFactory for a large exception payload.
• Replace EncodeBuffer’s std::vector<char> storage with a manual heap buffer and update protocol write logic to use memcpy.
• Add required C/C++ headers for the new buffer implementation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
lib/py/test/thrift_TBinaryProtocol.py	Adds accelerated large-message roundtrip test coverage for the fastbinary path.
lib/py/src/ext/types.h	Refactors EncodeBuffer to raw malloc/realloc storage with destructor-based cleanup.
lib/py/src/ext/protocol.tcc	Updates encode buffer allocation and write path to use the new EncodeBuffer API.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Jens-G]

I did use AI tools to investigate and address feedback, but I am a real human ready to collaborate

As long as they adhere to AGENTS.md all is fine.

[Jens-G]

Code review

Found 1 issue:

1. AI tool use is acknowledged in the PR body but neither commit contains the required Co-Authored-By: or Generated-by: label (AGENTS.md §4 says "Always label AI-assisted commits and PRs … Apply this label even when AI only generated a portion of the change").

thrift/AGENTS.md

Lines 57 to 71 in 35c1a53

	## 4. AI-Generated Contributions
	Per [`CONTRIBUTING.md § AI generated content`](CONTRIBUTING.md#ai-generated-content) and the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html):
	- **Always** label AI-assisted commits and PRs. Use one or both of:
	```
	Co-Authored-By: <AI tool name and version>
	Generated-by: <AI tool name and version>
	```
	Example:
	```
	THRIFT-9999: Fix connection timeout handling in Go client
	Client: go
	Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[Jens-G]

Hi, any plans to continue here?