Skip to content

feat(flags): report unresolved local flags on evaluate_flags snapshots - #319

Merged
gustavohstrassburger merged 6 commits into
mainfrom
posthog/unresolved-local-flags
Oct 29, 2026
Merged

gustavohstrassburger merged 6 commits into
mainfrom
posthog/unresolved-local-flags

Conversation

@gustavohstrassburger

@gustavohstrassburger gustavohstrassburger commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Why: sdk-specs #85 defines how a server SDK reports flags that local evaluation could not resolve. Today such a flag is absent from the evaluate_flags snapshot, so it looks the same as a flag that does not exist, and $feature_flag_called reports flag_missing for both. With only_evaluate_locally: true this is permanent. posthog-node already implements this (PostHog/posthog-js#5234); posthog-python (#1133) and posthog-server (#896) are in review.

Changes:

  • FeatureFlagEvaluations#unresolved_flags maps each unresolved flag to a PostHog::UnresolvedFlagReason (experience_continuity, unsupported_definition, unresolved_dependency, missing_context). When several causes apply, the spec's fixed precedence picks one. only / only_accessed snapshots have no entries, and reading the hash is not an access.
  • A snapshot read of an unresolved flag reports local_evaluation_inconclusive instead of flag_missing.
  • InconclusiveMatchError and RequiresServerEvaluation take an optional reason:, set at every raise site. A bad person value is missing_context; a bad filter value is unsupported_definition.
  • When definitions load, the SDK logs a WARN line for flags local evaluation can never resolve. Each flag warns once until its definition changes; 304 and repeated shared-cache loads do not repeat it.

Behavior changes (all in the changeset):

  • A condition or cohort property group no longer stops at the first filter it cannot check. If another filter does not match, the flag can resolve locally. A static cohort still forces the fallback only when it is the first filter in its condition that cannot be checked, as in posthog-node.
  • An inactive flag with experience continuity resolves to false locally.
  • A condition that depends on a flag with experience continuity is inconclusive locally. A later condition can still resolve the flag.

Divergences from the spec or posthog-node:

  • The SDK has no device-id bucketing, so the device-id acceptance row does not apply. A missing group key still resolves to false, as in posthog-node.
  • A regex that only the server can evaluate reports unsupported_definition, even when the person value causes the deferral. Patterns that always need the server are in the load-time warning; posthog-node does not warn about regexes.

💚 How did you test it?

New RSpec examples mirror every scenario tagged @unresolved_flags_capable and @unresolvable_flag_warning_capable in acceptance/public/evaluate-flags.feature, plus precedence, the condition loop, dependencies, warning dedup, failed fallback and unexpected errors. Run locally:

  • bundle exec rspec (2007 examples, 0 failures, 40 pending for the optional OpenTelemetry bundle)
  • bundle exec rubocop
  • bundle exec rake public_api:check

No manual testing in an app.

📝 Checklist

  • All commits are signed
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

  • Written with Claude Code in PostHog Desktop, from a cross-SDK plan to apply sdk-specs Add Action workflow for publishing Ruby gem #85 to all server SDKs. It follows posthog-node and the posthog-python and posthog-server PRs from the same plan.
  • PostHog::Utils.convert_to_datetime (@api private) now takes a required reason: so date parsing can tell a bad person value from a bad filter value.

Created with PostHog Desktop

🤖 Generated with Claude Code

Implements the "Snapshot unresolved flag reporting" and "Warning for flags local evaluation never resolves" requirements of the evaluate-flags spec (PostHog/sdk-specs#85).

- Add `FeatureFlagEvaluations#unresolved_flags`, with a `PostHog::UnresolvedFlagReason` for each flag and the spec's fixed precedence.
- Report `local_evaluation_inconclusive` instead of `flag_missing` when a snapshot read hits an unresolved flag.
- Do not stop a condition at the first inconclusive filter, so a filter that does not match resolves the condition locally.
- Check `active` before experience continuity, also for dependencies.
- Log one warning per flag that local evaluation can never resolve, repeated only when its definition changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: 39793b31-809e-4689-a5d8-b4359cbb22c1
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

posthog-ruby-sync Compliance Report

Date: 2026-10-29T17:20:04.739245+00:00
Duration: 56025ms

⚠️ Some Tests Failed

39/47 tests passed, 8 failed


Capture Tests

⚠️ 23/30 tests passed, 7 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 11ms
Format Validation.Event Has Uuid ✅ 6ms
Format Validation.Event Has Lib Properties ✅ 9ms
Format Validation.Distinct Id Is String ✅ 5ms
Format Validation.Token Is Present ✅ 6ms
Format Validation.Custom Properties Preserved ✅ 5ms
Format Validation.Event Has Timestamp ✅ 8ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 7ms
Retry Behavior.Retries On 503 ✅ 5326ms
Retry Behavior.Does Not Retry On 400 ❌ 10ms
Retry Behavior.Does Not Retry On 401 ❌ 6ms
Retry Behavior.Respects Retry After Header ✅ 8016ms
Retry Behavior.Implements Backoff ❌ 351ms
Retry Behavior.Retries On 500 ✅ 5117ms
Retry Behavior.Retries On 502 ✅ 5118ms
Retry Behavior.Retries On 504 ✅ 5120ms
Retry Behavior.Max Retries Respected ❌ 542ms
Deduplication.Generates Unique Uuids ✅ 18ms
Deduplication.Preserves Uuid On Retry ✅ 5115ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10308ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5128ms
Deduplication.No Duplicate Events In Batch ✅ 20ms
Deduplication.Different Events Have Different Uuids ✅ 9ms
Compression.Sends Gzip When Enabled ✅ 5ms
Batch Format.Uses Proper Batch Structure ✅ 6ms
Batch Format.Flush With No Events Sends Nothing ✅ 3ms
Batch Format.Multiple Events Batched Together ❌ 16ms
Error Handling.Does Not Retry On 403 ❌ 5ms
Error Handling.Does Not Retry On 413 ❌ 5ms
Error Handling.Retries On 408 ✅ 5112ms

Failures

retry_behavior.does_not_retry_on_400

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

retry_behavior.does_not_retry_on_401

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

retry_behavior.implements_backoff

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

retry_behavior.max_retries_respected

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

batch_format.multiple_events_batched_together

Expected 1 requests, got 5

error_handling.does_not_retry_on_403

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

error_handling.does_not_retry_on_413

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 10ms
Request Payload.Flags Request Uses V2 Query Param ✅ 7ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 7ms
Request Payload.Flags Request Omits Authorization Header ✅ 7ms
Request Payload.Token In Flags Body Matches Init ✅ 9ms
Request Payload.Groups Round Trip ✅ 8ms
Request Payload.Groups Default To Empty Object ✅ 8ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 6ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 6ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 6ms
Request Lifecycle.No Flags Request On Init Alone ✅ 2ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 6ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 11ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 6ms
Retry Behavior.Retries Flags On 502 ✅ 147ms
Retry Behavior.Retries Flags On 504 ✅ 139ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 9ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate', 'token']

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

posthog-ruby-async Compliance Report

Date: 2026-10-29T17:20:38.170955+00:00
Duration: 98205ms

⚠️ Some Tests Failed

46/47 tests passed, 1 failed


Capture Tests

✅ 30/30 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 10ms
Format Validation.Event Has Uuid ✅ 106ms
Format Validation.Event Has Lib Properties ✅ 110ms
Format Validation.Distinct Id Is String ✅ 107ms
Format Validation.Token Is Present ✅ 107ms
Format Validation.Custom Properties Preserved ✅ 111ms
Format Validation.Event Has Timestamp ✅ 106ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 8ms
Retry Behavior.Retries On 503 ✅ 5309ms
Retry Behavior.Does Not Retry On 400 ✅ 2109ms
Retry Behavior.Does Not Retry On 401 ✅ 2110ms
Retry Behavior.Respects Retry After Header ✅ 8013ms
Retry Behavior.Implements Backoff ✅ 15509ms
Retry Behavior.Retries On 500 ✅ 5213ms
Retry Behavior.Retries On 502 ✅ 5212ms
Retry Behavior.Retries On 504 ✅ 5209ms
Retry Behavior.Max Retries Respected ✅ 15623ms
Deduplication.Generates Unique Uuids ✅ 112ms
Deduplication.Preserves Uuid On Retry ✅ 5213ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10412ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5214ms
Deduplication.No Duplicate Events In Batch ✅ 113ms
Deduplication.Different Events Have Different Uuids ✅ 108ms
Compression.Sends Gzip When Enabled ✅ 106ms
Batch Format.Uses Proper Batch Structure ✅ 107ms
Batch Format.Flush With No Events Sends Nothing ✅ 5ms
Batch Format.Multiple Events Batched Together ✅ 111ms
Error Handling.Does Not Retry On 403 ✅ 2109ms
Error Handling.Does Not Retry On 413 ✅ 2109ms
Error Handling.Retries On 408 ✅ 5213ms

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 108ms
Request Payload.Flags Request Uses V2 Query Param ✅ 108ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 108ms
Request Payload.Flags Request Omits Authorization Header ✅ 109ms
Request Payload.Token In Flags Body Matches Init ✅ 109ms
Request Payload.Groups Round Trip ✅ 108ms
Request Payload.Groups Default To Empty Object ✅ 108ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 107ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 108ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 107ms
Request Lifecycle.No Flags Request On Init Alone ✅ 4ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 106ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 111ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 107ms
Retry Behavior.Retries Flags On 502 ✅ 210ms
Retry Behavior.Retries Flags On 504 ✅ 249ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 109ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate', 'token']

@gustavohstrassburger
gustavohstrassburger marked this pull request as ready for review October 28, 2026 16:18
@gustavohstrassburger
gustavohstrassburger requested a review from a team as a code owner October 28, 2026 16:18

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed fa4eb3d: no findings. This pull request is large, so the review ran in 4 parts with less depth than usual.

@dustinbyrne dustinbyrne left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unresolved-flag diagnostics and supporting coverage look sound overall. One compatibility issue remains: returned failed flags now lose their documented false snapshot value. Please preserve that behavior or explicitly resolve the contract and release change described inline.

AI-assisted review.

Comment thread lib/posthog/client.rb Outdated
@dustinbyrne
dustinbyrne requested a review from a team October 28, 2026 19:25

@dustinbyrne dustinbyrne left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The returned-failed-record compatibility issue is fixed, including caller defaults and capture enrichment. No required findings remain; one optional release-note clarification is inline.

AI-assisted review.

Comment thread .changeset/unresolved-local-flags.md Outdated
@github-project-automation github-project-automation Bot moved this from In Review to Approved in Feature Flags Oct 29, 2026
@gustavohstrassburger
gustavohstrassburger merged commit 7d6cbd6 into main Oct 29, 2026
26 checks passed
@gustavohstrassburger
gustavohstrassburger deleted the posthog/unresolved-local-flags branch October 29, 2026 17:26
@github-project-automation github-project-automation Bot moved this from Approved to Done in Feature Flags Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants