Skip to content

feat: Retry event delivery and close the client with a timeout - #253

Open
flagsmithsmith[bot] wants to merge 5 commits into
mainfrom
openhands/sdk-spec-v0.9.0
Open

flagsmithsmith[bot] wants to merge 5 commits into
mainfrom
openhands/sdk-spec-v0.9.0

Conversation

@flagsmithsmith

Copy link
Copy Markdown
Contributor

Contributes to Flagsmith/sdk-spec#13
Contributes to Flagsmith/sdk-spec#23

renovate Bot and others added 4 commits November 11, 2026 15:33
Its entry in Flagsmith/sdk-spec's compatibility matrix at v0.9.0: `core events local-evaluation`.
Spec v0.7.1 runs the `@events` scenarios for this SDK: getting an
experiment flag, tracking an event and tracking an exposure event, all
with events enabled. The SDK already implements these (#226), so the
steps file wires each step to FlagsmithClient.getExperimentFlag,
trackEvent and trackExposureEvent, and creates the events-enabled client
through FlagsmithConfig, next to the local evaluation client.

(cherry picked from commit 0c48af8)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 29cbe7b)
(cherry picked from commit 723a712)
Spec v0.8.0 specifies how SDKs deliver events. The event processor now:

- Times out each events request after 3 s, and gives a batch 3 attempts,
  backing off from 1 s, doubling, with full jitter. Network errors,
  timeouts, 408, 429, 500, 502, 503 and 504 are retried.
- Keeps a batch whose last attempt failed transiently in the buffer,
  ahead of newer events, for the next flush, unless that attempt got a
  500. Any other error response drops the batch.
- Stops event tracking, dropping the buffer, on a 401 or 403.
- Sends one batch at a time, and caps the buffer at 1000 events by
  dropping the oldest, kept events included.
- Never fails `flushEvents()`, which waits for the send, retries
  included.

`FlagsmithClient.close(timeout, unit)` sends the buffered events for at
most the timeout, then abandons the request in flight. `close()` waits
for one batch's worst case, 12 s.

The spec steps file wires the new steps: flushing events, closing with
a timeout, tracking a range of events, and a client with both local
evaluation and events enabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 88fa258)
@flagsmithsmith flagsmithsmith Bot added the sdk-spec Bumps of the Flagsmith SDK behavioural spec label Nov 11, 2026
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 65dcecf2-a063-4254-97b4-0e721ad6bd10

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@codspeed

codspeed Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 15 untouched benchmarks


Comparing openhands/sdk-spec-v0.9.0 (f3c57d7) with main (1c633a1)

Open in CodSpeed

@flagsmithsmith
flagsmithsmith Bot marked this pull request as ready for review November 11, 2026 15:51
@flagsmithsmith
flagsmithsmith Bot requested a review from a team as a code owner November 11, 2026 15:51
@flagsmithsmith
flagsmithsmith Bot requested review from emyller and removed request for a team November 11, 2026 15:51

@Zaimwa9 Zaimwa9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. Spec passes and the structure is good, but events behind a kept batch are never sent on flush or close, and a full buffer after a kept batch retries on every track. Please also retitle: the tracking methods are already on main.

waiting = nextBatch;
nextBatch = null;
// A kept batch means the events API is failing: the buffer waits for the next flush.
if (waiting != null && outcome != Outcome.KEPT && !buffer.isEmpty()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A flush waiting behind a KEPT batch completes without sending newer events; on close they are dropped unattempted. Always send the buffer, kept batch first.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f3c57d7. When a flush (from the application, a timer tick or close()) is waiting behind the batch in flight and that batch is kept, settle now sends the whole buffer with the kept batch first. Before, it completed the flush without sending anything. close() therefore tries those events within its timeout instead of dropping them. New tests: flush_sendsTheEventsBehindAKeptBatchWithItWhenRequestedWhileItWasInFlight and close_sendsTheEventsBehindAKeptBatchWithIt. Both check that the sent events are add_to_cart ×4, then checkout.

droppedToReport = recordDrop(1);
}
droppedToReport = dropOverflow();
isFull = buffer.size() >= maxBufferItems;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a kept batch every trackEvent starts a new retry cycle. Hold size-triggered flushes until the next tick or explicit flush.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f3c57d7. A full buffer now calls flush(false). After a batch is kept, a size-triggered flush neither starts a send nor, if it was registered while that batch was in flight, sends the buffer when the batch settles. The buffer waits for the next timer tick or explicit flush (flush() / close()). A batch that gets through, or is dropped, lifts the hold. Oldest-first dropping still caps the buffer at 1000 while it waits. New tests: trackEvent_holdsAFullBufferAfterAKeptBatchUntilTheNextFlush (it also checks that size-triggered sends resume after a success) and trackEvent_holdsABufferThatFilledBehindABatchThatIsThenKept.

…hile

A flush requested while a batch was in flight completed without sending
the events tracked since, when that batch was kept, and close() then
dropped them without trying. Such a flush now sends the whole buffer,
the kept batch first, and so does close().

After a kept batch, a full buffer no longer starts a new retry cycle on
every tracked event: it waits for the next timer tick or explicit flush.
A batch that gets through lifts the hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@flagsmithsmith flagsmithsmith Bot changed the title feat: Support getExperimentFlag, trackEvent and trackExposure methods feat: Retry event delivery and close the client with a timeout Nov 12, 2026
@flagsmithsmith

Copy link
Copy Markdown
Contributor Author

Thanks @Zaimwa9. Addressed in f3c57d7.

events behind a kept batch are never sent on flush or close

Fixed. A flush waiting behind a kept batch now sends the whole buffer, kept batch first. close() does the same within its timeout. Details in the inline reply.

a full buffer after a kept batch retries on every track

Fixed. After a kept batch, a full buffer waits for the next timer tick or explicit flush. The next batch that gets through lifts the hold. Details in the inline reply.

Please also retitle: the tracking methods are already on main.

Retitled to "feat: Retry event delivery and close the client with a timeout".

Locally, mvn clean install passes (539 tests, with the engine-test-data submodule checked out). So do the release build and the spec at v0.9.0 with core events local-evaluation (24/24).

@Zaimwa9 Zaimwa9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. Both earlier points are fixed, but with the flush timer disabled (withEventsFlushIntervalMillis(0)) one kept batch stops size-triggered delivery for the rest of the process.

nextBatchRequested |= requested && nextBatch != null;
return nextBatch == null ? inFlight : nextBatch;
}
if (buffer.isEmpty() || (!requested && holdFullBuffer)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the timer disabled nothing ticks, so after one kept batch only an explicit flushEvents() or close() lifts this hold; on main it recovers. Let the hold expire after a cool-down when the interval is 0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sdk-spec Bumps of the Flagsmith SDK behavioural spec

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants