Deletion vector reader#2
Open
Shreyas220 wants to merge 99 commits into
Open
Conversation
Shreyas220
force-pushed
the
deletion_vector-reader
branch
3 times, most recently
from
abe5341 to
d8ed2a7
Compare
…tlptracehttp from 1.41.0 to 1.43.0 (apache#864) Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp](https://github.com/open-telemetry/opentelemetry-go) from 1.41.0 to 1.43.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md">go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp's changelog</a>.</em></p> <blockquote> <h2>[1.43.0/0.65.0/0.19.0] 2026-04-02</h2> <h3>Added</h3> <ul> <li>Add <code>IsRandom</code> and <code>WithRandom</code> on <code>TraceFlags</code>, and <code>IsRandom</code> on <code>SpanContext</code> in <code>go.opentelemetry.io/otel/trace</code> for <a href="https://www.w3.org/TR/trace-context-2/#random-trace-id-flag">W3C Trace Context Level 2 Random Trace ID Flag</a> support. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8012">#8012</a>)</li> <li>Add service detection with <code>WithService</code> in <code>go.opentelemetry.io/otel/sdk/resource</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7642">#7642</a>)</li> <li>Add <code>DefaultWithContext</code> and <code>EnvironmentWithContext</code> in <code>go.opentelemetry.io/otel/sdk/resource</code> to support plumbing <code>context.Context</code> through default and environment detectors. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8051">#8051</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Add support for per-series start time tracking for cumulative metrics in <code>go.opentelemetry.io/otel/sdk/metric</code>. Set <code>OTEL_GO_X_PER_SERIES_START_TIMESTAMPS=true</code> to enable. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8060">#8060</a>)</li> <li>Add <code>WithCardinalityLimitSelector</code> for metric reader for configuring cardinality limits specific to the instrument kind. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7855">#7855</a>)</li> </ul> <h3>Changed</h3> <ul> <li>Introduce the <code>EMPTY</code> Type in <code>go.opentelemetry.io/otel/attribute</code> to reflect that an empty value is now a valid value, with <code>INVALID</code> remaining as a deprecated alias of <code>EMPTY</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Improve slice handling in <code>go.opentelemetry.io/otel/attribute</code> to optimize short slice values with fixed-size fast paths. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8039">#8039</a>)</li> <li>Improve performance of span metric recording in <code>go.opentelemetry.io/otel/sdk/trace</code> by returning early if self-observability is not enabled. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8067">#8067</a>)</li> <li>Improve formatting of metric data diffs in <code>go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8073">#8073</a>)</li> </ul> <h3>Deprecated</h3> <ul> <li>Deprecate <code>INVALID</code> in <code>go.opentelemetry.io/otel/attribute</code>. Use <code>EMPTY</code> instead. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> </ul> <h3>Fixed</h3> <ul> <li>Return spec-compliant <code>TraceIdRatioBased</code> description. This is a breaking behavioral change, but it is necessary to make the implementation <a href="https://opentelemetry.io/docs/specs/otel/trace/sdk/#traceidratiobased">spec-compliant</a>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8027">#8027</a>)</li> <li>Fix a race condition in <code>go.opentelemetry.io/otel/sdk/metric</code> where the lastvalue aggregation could collect the value 0 even when no zero-value measurements were recorded. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8056">#8056</a>)</li> <li>Limit HTTP response body to 4 MiB in <code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp</code> to mitigate excessive memory usage caused by a misconfigured or malicious server. Responses exceeding the limit are treated as non-retryable errors. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li> <li>Limit HTTP response body to 4 MiB in <code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp</code> to mitigate excessive memory usage caused by a misconfigured or malicious server. Responses exceeding the limit are treated as non-retryable errors. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li> <li>Limit HTTP response body to 4 MiB in <code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code> to mitigate excessive memory usage caused by a misconfigured or malicious server. Responses exceeding the limit are treated as non-retryable errors. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li> <li><code>WithHostID</code> detector in <code>go.opentelemetry.io/otel/sdk/resource</code> to use full path for <code>kenv</code> command on BSD. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8113">#8113</a>)</li> <li>Fix missing <code>request.GetBody</code> in <code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code> to correctly handle HTTP2 GOAWAY frame. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8096">#8096</a>)</li> </ul> <h2>[1.42.0/0.64.0/0.18.0/0.0.16] 2026-03-06</h2> <h3>Added</h3> <ul> <li>Add <code>go.opentelemetry.io/otel/semconv/v1.40.0</code> package. The package contains semantic conventions from the <code>v1.40.0</code> version of the OpenTelemetry Semantic Conventions. See the <a href="https://github.com/open-telemetry/opentelemetry-go/blob/main/semconv/v1.40.0/MIGRATION.md">migration documentation</a> for information on how to upgrade from <code>go.opentelemetry.io/otel/semconv/v1.39.0</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7985">#7985</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/9276201a64b623606e3eaa0d61ae8ee6d62756c0"><code>9276201</code></a> Release v1.43.0 / v0.65.0 / v0.19.0 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8128">#8128</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/61b8c9466c4e6b17e69b622279fe9b63fb15c89a"><code>61b8c94</code></a> chore(deps): update module github.com/mattn/go-runewidth to v0.0.22 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8131">#8131</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/97a086e82ffe01502f4c620e9c447efa229e2a23"><code>97a086e</code></a> chore(deps): update github.com/golangci/dupl digest to c99c5cf (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8122">#8122</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/5e363de517dba6db62736b2f5cdef0e0929b4cd0"><code>5e363de</code></a> limit response body size for OTLP HTTP exporters (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/35214b60138eac8dec97a2d2b851d8c8471680c7"><code>35214b6</code></a> Use an absolute path when calling bsd kenv (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8113">#8113</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/290024ceaf695f9cdbf29a0c6731a317d92bc361"><code>290024c</code></a> fix(deps): update module google.golang.org/grpc to v1.80.0 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8121">#8121</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/e70658e098033d6bb5ec1b399de16bbb2642f6dc"><code>e70658e</code></a> fix: support getBody in otelploghttp (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8096">#8096</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/4afe468e3b4859c949a1c1e8d92684d43d86ef8a"><code>4afe468</code></a> fix(deps): update googleapis to 9d38bb4 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8117">#8117</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/b9ca729776309e3c08fe700c131797a3b4d10634"><code>b9ca729</code></a> chore(deps): update module github.com/go-git/go-git/v5 to v5.17.2 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8115">#8115</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/69472ec56cb7674d55ca2e2bcb04dea73228ab79"><code>69472ec</code></a> chore(deps): update fossas/fossa-action action to v1.9.0 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8118">#8118</a>)</li> <li>Additional commits viewable in <a href="https://github.com/open-telemetry/opentelemetry-go/compare/v1.41.0...v1.43.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-go/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Add CompactionConfig and Plan() that groups FileScanTasks by partition, classifies files as candidates based on size thresholds and delete file counts, and bin-packs candidates into CompactionGroups using the existing SlicePacker. This is the planning layer for RewriteDataFiles (apache#832). - Oversized files skipped unless delete count exceeds threshold - Deterministic partition key via sorted field IDs - Config validation (target between min/max, positive thresholds) - Ceiling division for output file estimation
…otlpmetrichttp from 1.38.0 to 1.43.0 (apache#862) Bumps [go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp](https://github.com/open-telemetry/opentelemetry-go) from 1.38.0 to 1.43.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md">go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp's changelog</a>.</em></p> <blockquote> <h2>[1.43.0/0.65.0/0.19.0] 2026-04-02</h2> <h3>Added</h3> <ul> <li>Add <code>IsRandom</code> and <code>WithRandom</code> on <code>TraceFlags</code>, and <code>IsRandom</code> on <code>SpanContext</code> in <code>go.opentelemetry.io/otel/trace</code> for <a href="https://www.w3.org/TR/trace-context-2/#random-trace-id-flag">W3C Trace Context Level 2 Random Trace ID Flag</a> support. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8012">#8012</a>)</li> <li>Add service detection with <code>WithService</code> in <code>go.opentelemetry.io/otel/sdk/resource</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7642">#7642</a>)</li> <li>Add <code>DefaultWithContext</code> and <code>EnvironmentWithContext</code> in <code>go.opentelemetry.io/otel/sdk/resource</code> to support plumbing <code>context.Context</code> through default and environment detectors. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8051">#8051</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Support attributes with empty value (<code>attribute.EMPTY</code>) in <code>go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Add support for per-series start time tracking for cumulative metrics in <code>go.opentelemetry.io/otel/sdk/metric</code>. Set <code>OTEL_GO_X_PER_SERIES_START_TIMESTAMPS=true</code> to enable. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8060">#8060</a>)</li> <li>Add <code>WithCardinalityLimitSelector</code> for metric reader for configuring cardinality limits specific to the instrument kind. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7855">#7855</a>)</li> </ul> <h3>Changed</h3> <ul> <li>Introduce the <code>EMPTY</code> Type in <code>go.opentelemetry.io/otel/attribute</code> to reflect that an empty value is now a valid value, with <code>INVALID</code> remaining as a deprecated alias of <code>EMPTY</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> <li>Improve slice handling in <code>go.opentelemetry.io/otel/attribute</code> to optimize short slice values with fixed-size fast paths. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8039">#8039</a>)</li> <li>Improve performance of span metric recording in <code>go.opentelemetry.io/otel/sdk/trace</code> by returning early if self-observability is not enabled. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8067">#8067</a>)</li> <li>Improve formatting of metric data diffs in <code>go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8073">#8073</a>)</li> </ul> <h3>Deprecated</h3> <ul> <li>Deprecate <code>INVALID</code> in <code>go.opentelemetry.io/otel/attribute</code>. Use <code>EMPTY</code> instead. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8038">#8038</a>)</li> </ul> <h3>Fixed</h3> <ul> <li>Return spec-compliant <code>TraceIdRatioBased</code> description. This is a breaking behavioral change, but it is necessary to make the implementation <a href="https://opentelemetry.io/docs/specs/otel/trace/sdk/#traceidratiobased">spec-compliant</a>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8027">#8027</a>)</li> <li>Fix a race condition in <code>go.opentelemetry.io/otel/sdk/metric</code> where the lastvalue aggregation could collect the value 0 even when no zero-value measurements were recorded. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8056">#8056</a>)</li> <li>Limit HTTP response body to 4 MiB in <code>go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp</code> to mitigate excessive memory usage caused by a misconfigured or malicious server. Responses exceeding the limit are treated as non-retryable errors. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li> <li>Limit HTTP response body to 4 MiB in <code>go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp</code> to mitigate excessive memory usage caused by a misconfigured or malicious server. Responses exceeding the limit are treated as non-retryable errors. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li> <li>Limit HTTP response body to 4 MiB in <code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code> to mitigate excessive memory usage caused by a misconfigured or malicious server. Responses exceeding the limit are treated as non-retryable errors. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li> <li><code>WithHostID</code> detector in <code>go.opentelemetry.io/otel/sdk/resource</code> to use full path for <code>kenv</code> command on BSD. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8113">#8113</a>)</li> <li>Fix missing <code>request.GetBody</code> in <code>go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp</code> to correctly handle HTTP2 GOAWAY frame. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8096">#8096</a>)</li> </ul> <h2>[1.42.0/0.64.0/0.18.0/0.0.16] 2026-03-06</h2> <h3>Added</h3> <ul> <li>Add <code>go.opentelemetry.io/otel/semconv/v1.40.0</code> package. The package contains semantic conventions from the <code>v1.40.0</code> version of the OpenTelemetry Semantic Conventions. See the <a href="https://github.com/open-telemetry/opentelemetry-go/blob/main/semconv/v1.40.0/MIGRATION.md">migration documentation</a> for information on how to upgrade from <code>go.opentelemetry.io/otel/semconv/v1.39.0</code>. (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/7985">#7985</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/9276201a64b623606e3eaa0d61ae8ee6d62756c0"><code>9276201</code></a> Release v1.43.0 / v0.65.0 / v0.19.0 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8128">#8128</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/61b8c9466c4e6b17e69b622279fe9b63fb15c89a"><code>61b8c94</code></a> chore(deps): update module github.com/mattn/go-runewidth to v0.0.22 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8131">#8131</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/97a086e82ffe01502f4c620e9c447efa229e2a23"><code>97a086e</code></a> chore(deps): update github.com/golangci/dupl digest to c99c5cf (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8122">#8122</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/5e363de517dba6db62736b2f5cdef0e0929b4cd0"><code>5e363de</code></a> limit response body size for OTLP HTTP exporters (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8108">#8108</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/35214b60138eac8dec97a2d2b851d8c8471680c7"><code>35214b6</code></a> Use an absolute path when calling bsd kenv (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8113">#8113</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/290024ceaf695f9cdbf29a0c6731a317d92bc361"><code>290024c</code></a> fix(deps): update module google.golang.org/grpc to v1.80.0 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8121">#8121</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/e70658e098033d6bb5ec1b399de16bbb2642f6dc"><code>e70658e</code></a> fix: support getBody in otelploghttp (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8096">#8096</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/4afe468e3b4859c949a1c1e8d92684d43d86ef8a"><code>4afe468</code></a> fix(deps): update googleapis to 9d38bb4 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8117">#8117</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/b9ca729776309e3c08fe700c131797a3b4d10634"><code>b9ca729</code></a> chore(deps): update module github.com/go-git/go-git/v5 to v5.17.2 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8115">#8115</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-go/commit/69472ec56cb7674d55ca2e2bcb04dea73228ab79"><code>69472ec</code></a> chore(deps): update fossas/fossa-action action to v1.9.0 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-go/issues/8118">#8118</a>)</li> <li>Additional commits viewable in <a href="https://github.com/open-telemetry/opentelemetry-go/compare/v1.38.0...v1.43.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…sts (apache#868) ## Problem `NewManifestReader` in `manifest.go` calls `strconv.Atoi` unconditionally on `metadata["format-version"]`. The Java Iceberg library omits this key from v1 manifest file Avro OCF headers — `format-version` is optional for v1 per the spec. When iceberg-go reads those files, `metadata["format-version"]` returns `nil` → empty string → `Atoi("")` → error: ``` manifest file's 'format-version' metadata is invalid: strconv.Atoi: parsing "": invalid syntax ``` ## Fix Default `formatVersion` to `1` and only parse the metadata key when it is present. This matches the spec (`format-version` is optional for v1, required for v2+) and the same pattern already used in `ReadManifestList` since apache#826. The existing `formatVersion != file.Version()` check below is unaffected and still catches real version mismatches. ## Testing Added `TestNewManifestReaderMissingFormatVersion`: writes a v1 manifest Avro file without the `format-version` metadata key (simulating Java Iceberg output) and verifies `NewManifestReader` succeeds and reports version 1. ## Related - Follows the same pattern as apache#826 which fixed `ReadManifestList` - Spec reference: `format/spec.md` Avro manifest file metadata table — `format-version` is `optional` for v1, `required` for v2
…pache#887) Add a contribution guideline rules for new-contributors, especially how to claim issue, work with reviews and where to seek for feedback and ping people.
Bumps the gomod_updates group with 4 updates: [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go), [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2), [github.com/testcontainers/testcontainers-go/modules/compose](https://github.com/testcontainers/testcontainers-go) and [google.golang.org/api](https://github.com/googleapis/google-api-go-client). Updates `cloud.google.com/go/storage` from 1.61.3 to 1.62.0 <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/googleapis/google-cloud-go/commit/01df6e18f052807c7e0b5cc92cb329b339e8ba37"><code>01df6e1</code></a> chore(main): release spanner 1.62.0 (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/10073">#10073</a>)</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/8e675cd0ccf12c6912209aa5c56092db3716c40d"><code>8e675cd</code></a> fix(spanner): Add ARRAY keywords to keywords (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/10079">#10079</a>)</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/db8216e54c8d9d23dd90bc47a081eb2754f7c92a"><code>db8216e</code></a> feat(pubsub/pstest): add support to register other servers into grpc.Server (...</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/292e81231b957ae7ac243b47b8926564cee35920"><code>292e812</code></a> docs(maps/fleetengine): mark TerminalPointId as deprecated (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/10130">#10130</a>)</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/0e6413db32f2a27269602fa88afa762abdb837c0"><code>0e6413d</code></a> feat(bigtable): Add ignore_warnings flag to SetGcPolicy (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/9372">#9372</a>)</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/2d34bf378974c64ec599a89a496cee1490f58996"><code>2d34bf3</code></a> chore(all): update deps (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/10156">#10156</a>)</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/73716659ed3caa636f29a89248506b4076dac6a7"><code>7371665</code></a> test(pubsub): make error checking of receive less brittle (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/10165">#10165</a>)</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/fa3bfdb23aaa45b34394a8b61e753b3587506782"><code>fa3bfdb</code></a> fix(auth): handle non-Transport DefaultTransport (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/10162">#10162</a>)</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/1320d7db0df61940c5e4ad705d6658474c7c38c4"><code>1320d7d</code></a> chore(main): release vertexai 0.9.0 (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/10163">#10163</a>)</li> <li><a href="https://github.com/googleapis/google-cloud-go/commit/b452014e0a8ea81ef6b3363a0832da6384092ba1"><code>b452014</code></a> chore(vertexai): skip tests that check blocked inputs for model (<a href="https://redirect.github.com/googleapis/google-cloud-go/issues/10152">#10152</a>)</li> <li>Additional commits viewable in <a href="https://github.com/googleapis/google-cloud-go/compare/storage/v1.61.3...spanner/v1.62.0">compare view</a></li> </ul> </details> <br /> Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.98.0 to 1.99.0 <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/159f4d23fd3150aa3b14c231c434f46f42b72c2a"><code>159f4d2</code></a> Release 2026-04-07</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/50223dc5168c2eb6316a108f23a7fd0803afba76"><code>50223dc</code></a> Regenerated Clients</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/af25337605d0de4dc7f4141c6b1627ec31254415"><code>af25337</code></a> Update endpoints model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/277dc017b3b8ea944865ffb901eea3bf82121f5d"><code>277dc01</code></a> Update API model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/22bc35ca5a2af77140a47680f80873b4d61b6551"><code>22bc35c</code></a> Release 2026-04-06</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/9a35fedd65d0cdad42281199a7659146234893a1"><code>9a35fed</code></a> Regenerated Clients</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/867b085ecb8579affb591d0e98bbe362376ce911"><code>867b085</code></a> Update endpoints model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/61d367c87617dbe79f6cb8fe8809d09ce949d2fa"><code>61d367c</code></a> Update API model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/8560a07e8b7ed78b0962b6df18050500a32705e7"><code>8560a07</code></a> Release 2026-04-03</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/a5aafdff76b34debb6940077fc4377ed80d24805"><code>a5aafdf</code></a> Regenerated Clients</li> <li>Additional commits viewable in <a href="https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.98.0...service/s3/v1.99.0">compare view</a></li> </ul> </details> <br /> Updates `github.com/testcontainers/testcontainers-go/modules/compose` from 0.41.0 to 0.42.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/testcontainers/testcontainers-go/releases">github.com/testcontainers/testcontainers-go/modules/compose's releases</a>.</em></p> <blockquote> <h2>v0.42.0</h2> <h1>What's Changed</h1> <h2>⚠️ Breaking Changes</h2> <ul> <li>chore!: migrate to moby modules (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3591">#3591</a>) <a href="https://github.com/thaJeztah"><code>@thaJeztah</code></a></li> </ul> <h2>🔒 Security</h2> <ul> <li>chore(deps): bump moby/client v0.4.0, moby/api v1.54.1 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3634">#3634</a>) <a href="https://github.com/thaJeztah"><code>@thaJeztah</code></a></li> </ul> <h2>🐛 Bug Fixes</h2> <ul> <li>fix: return an error when docker host cannot be retrieved (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3613">#3613</a>) <a href="https://github.com/ash2k"><code>@ash2k</code></a></li> </ul> <h2>🧹 Housekeeping</h2> <ul> <li>chore: gitignore Gas Town agent artifacts (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3633">#3633</a>) <a href="https://github.com/mdelapenya"><code>@mdelapenya</code></a></li> <li>fix(usage-metrics): include last release in the legend pop over (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3630">#3630</a>) <a href="https://github.com/mdelapenya"><code>@mdelapenya</code></a></li> <li>chore: update usage metrics (2026-04) (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3621">#3621</a>) @<a href="https://github.com/apps/github-actions">github-actions[bot]</a></li> <li>fix(usage-metrics): order of actions matters (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3623">#3623</a>) <a href="https://github.com/mdelapenya"><code>@mdelapenya</code></a></li> <li>fix(usage-metrics): reduce rate-limit cascade errors (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3622">#3622</a>) <a href="https://github.com/mdelapenya"><code>@mdelapenya</code></a></li> <li>fix(usage-metrics): replace the per-version inline retry with a multi-pass approach (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3620">#3620</a>) <a href="https://github.com/mdelapenya"><code>@mdelapenya</code></a></li> </ul> <h2>📦 Dependency updates</h2> <ul> <li>chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp from 1.28.0 to 1.43.0 in /modules/grafana-lgtm (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3639">#3639</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp from 1.42.0 to 1.43.0 in /modules/compose (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3641">#3641</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.42.0 to 1.43.0 in /modules/compose (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3645">#3645</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump mkdocs-include-markdown-plugin from 7.2.1 to 7.2.2 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3626">#3626</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.51.2 to 1.97.3 in /modules/localstack (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3638">#3638</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.41.0 to 1.43.0 in /modules/grafana-lgtm (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3643">#3643</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump go.opentelemetry.io/otel/sdk from 1.41.0 to 1.43.0 in /modules/milvus (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3644">#3644</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore: update to Go 1.25.9, 1.26.9 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3647">#3647</a>) <a href="https://github.com/thaJeztah"><code>@thaJeztah</code></a></li> <li>chore(deps): bump bump github.com/klauspost/compress v1.18.5, github.com/docker/compose v5.1.2 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3646">#3646</a>) <a href="https://github.com/thaJeztah"><code>@thaJeztah</code></a></li> <li>chore(deps): bump moby/client v0.4.0, moby/api v1.54.1 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3634">#3634</a>) <a href="https://github.com/thaJeztah"><code>@thaJeztah</code></a></li> <li>chore(deps): bump golang.org/x/sys from 0.41.0 to 0.42.0 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3629">#3629</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump github.com/moby/patternmatcher from 0.6.0 to 0.6.1 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3628">#3628</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump github.com/shirou/gopsutil/v4 from 4.26.2 to 4.26.3 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3627">#3627</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>fix(localstack): accept community-archive as a valid tag (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3601">#3601</a>) <a href="https://github.com/johnduhart"><code>@johnduhart</code></a></li> <li>chore(deps): bump github.com/go-jose/go-jose/v4 from 4.1.3 to 4.1.4 in /modules/gcloud (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3632">#3632</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump actions/upload-artifact from 6.0.0 to 7.0.0 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3625">#3625</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump pygments from 2.19.2 to 2.20.0 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3615">#3615</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump google.golang.org/grpc from 1.67.0 to 1.79.3 in /modules/milvus (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3612">#3612</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump google.golang.org/grpc from 1.67.0 to 1.79.3 in /modules/etcd (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3611">#3611</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump google.golang.org/grpc from 1.79.1 to 1.79.3 in /modules/ollama (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3610">#3610</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump google.golang.org/grpc from 1.67.0 to 1.79.3 in /modules/pinecone (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3609">#3609</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump google.golang.org/grpc from 1.67.0 to 1.79.3 in /modules/couchbase (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3608">#3608</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump requests from 2.32.4 to 2.33.0 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3604">#3604</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump google.golang.org/grpc from 1.79.1 to 1.79.3 in /modules/meilisearch (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3607">#3607</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> <li>chore(deps): bump github.com/moby/buildkit from 0.27.1 to 0.28.1 in /modules/compose (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3605">#3605</a>) @<a href="https://github.com/apps/dependabot">dependabot[bot]</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/6e584183e1623fb83a78db902462eb9ee255327b"><code>6e58418</code></a> chore: use new version (v0.42.0) in modules and examples</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/f713dc0c78c37558ed13c985f5ed15101a6cba08"><code>f713dc0</code></a> chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetr...</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/300827abcec0b6e2d8c10c8360d335f29962f701"><code>300827a</code></a> chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetr...</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/7a15ac1b7214c0bc5cb14c31a979a166ac5c088c"><code>7a15ac1</code></a> chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptrace...</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/5bae3d2aa7281a71d1c24b5f8bde1391e2e8dfcb"><code>5bae3d2</code></a> fix: return an error when docker host cannot be retrieved (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3613">#3613</a>)</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/fc19484a8ae088b714a9e9d7d9bba2ae383e3e69"><code>fc19484</code></a> chore(deps): bump mkdocs-include-markdown-plugin from 7.2.1 to 7.2.2 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3626">#3626</a>)</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/95bdc0c4a0b368564f052bd7a533f055adab694a"><code>95bdc0c</code></a> chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3638">#3638</a>)</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/75aa22665b03ec8d597341575001d71ff693f1fc"><code>75aa226</code></a> chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptrace...</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/2f59938fb9ce365d974fe21a426ffffe7231caca"><code>2f59938</code></a> chore(deps): bump go.opentelemetry.io/otel/sdk in /modules/milvus (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3644">#3644</a>)</li> <li><a href="https://github.com/testcontainers/testcontainers-go/commit/580abf68d44021bc7e54f8655a2fb66578205527"><code>580abf6</code></a> chore: update to Go 1.25.9, 1.26.9 (<a href="https://redirect.github.com/testcontainers/testcontainers-go/issues/3647">#3647</a>)</li> <li>Additional commits viewable in <a href="https://github.com/testcontainers/testcontainers-go/compare/v0.41.0...v0.42.0">compare view</a></li> </ul> </details> <br /> Updates `google.golang.org/api` from 0.274.0 to 0.275.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/googleapis/google-api-go-client/releases">google.golang.org/api's releases</a>.</em></p> <blockquote> <h2>v0.275.0</h2> <h2><a href="https://github.com/googleapis/google-api-go-client/compare/v0.274.0...v0.275.0">0.275.0</a> (2026-04-07)</h2> <h3>Features</h3> <ul> <li><strong>all:</strong> Auto-regenerate discovery clients (<a href="https://redirect.github.com/googleapis/google-api-go-client/issues/3557">#3557</a>) (<a href="https://github.com/googleapis/google-api-go-client/commit/2b2ef99cb9f245743690a4d26e4fdc65287253e0">2b2ef99</a>)</li> <li><strong>all:</strong> Auto-regenerate discovery clients (<a href="https://redirect.github.com/googleapis/google-api-go-client/issues/3560">#3560</a>) (<a href="https://github.com/googleapis/google-api-go-client/commit/9437d4d741a6ae9e1c20a6f727b9c8f64e1bc19e">9437d4d</a>)</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md">google.golang.org/api's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/googleapis/google-api-go-client/compare/v0.274.0...v0.275.0">0.275.0</a> (2026-04-07)</h2> <h3>Features</h3> <ul> <li><strong>all:</strong> Auto-regenerate discovery clients (<a href="https://redirect.github.com/googleapis/google-api-go-client/issues/3557">#3557</a>) (<a href="https://github.com/googleapis/google-api-go-client/commit/2b2ef99cb9f245743690a4d26e4fdc65287253e0">2b2ef99</a>)</li> <li><strong>all:</strong> Auto-regenerate discovery clients (<a href="https://redirect.github.com/googleapis/google-api-go-client/issues/3560">#3560</a>) (<a href="https://github.com/googleapis/google-api-go-client/commit/9437d4d741a6ae9e1c20a6f727b9c8f64e1bc19e">9437d4d</a>)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/googleapis/google-api-go-client/commit/d43aa15bdf02279f1beaa366b551587391355265"><code>d43aa15</code></a> chore(main): release 0.275.0 (<a href="https://redirect.github.com/googleapis/google-api-go-client/issues/3558">#3558</a>)</li> <li><a href="https://github.com/googleapis/google-api-go-client/commit/9437d4d741a6ae9e1c20a6f727b9c8f64e1bc19e"><code>9437d4d</code></a> feat(all): auto-regenerate discovery clients (<a href="https://redirect.github.com/googleapis/google-api-go-client/issues/3560">#3560</a>)</li> <li><a href="https://github.com/googleapis/google-api-go-client/commit/0a62c64ae95b23c6ecb9fc71db89f09c479b0442"><code>0a62c64</code></a> chore(all): update cloud.google.com/go/auth to v0.20.0 (<a href="https://redirect.github.com/googleapis/google-api-go-client/issues/3559">#3559</a>)</li> <li><a href="https://github.com/googleapis/google-api-go-client/commit/2b2ef99cb9f245743690a4d26e4fdc65287253e0"><code>2b2ef99</code></a> feat(all): auto-regenerate discovery clients (<a href="https://redirect.github.com/googleapis/google-api-go-client/issues/3557">#3557</a>)</li> <li>See full diff in <a href="https://github.com/googleapis/google-api-go-client/compare/v0.274.0...v0.275.0">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…shots (apache#873) ## Summary This PR fixes two related bugs: **1. `buildCommonMetadata` silently dropped all statistics (primary bug)** `buildCommonMetadata` in `metadata.go` was not copying `StatisticsList` or `PartitionStatsList` into the rebuilt metadata. This meant *every* `Build()` call — not just `RemoveSnapshots` — silently dropped all statistics files. Any builder operation (add snapshot, update schema, set properties, etc.) would lose statistics on the resulting metadata. **2. `RemoveSnapshots` left orphaned statistics entries** Even after fixing the builder, `RemoveSnapshots` would leave stale `StatisticsList`/`PartitionStatsList` entries pointing at snapshots that had just been removed. Added pruning logic consistent with the existing `snapshotList`/`snapshotLog` pattern. Fixes apache#836 ## Test plan - [x] `go test ./...` - [x] `gofmt` clean - [x] `TestRemoveSnapshotsPrunesStatistics` — seeds statistics for two snapshots, removes one, asserts the right entry is pruned and the result survives `Build()`. - [x] `TestBuildPreservesStatistics` — no-op `MetadataBuilderFromBase → Build()` round-trip, asserts statistics are preserved. Pins the `buildCommonMetadata` fix independently of `RemoveSnapshots`. Signed-off-by: Ali <alliasgher123@gmail.com> --------- Signed-off-by: Ali <alliasgher123@gmail.com>
…shots (apache#877) ## Summary `ExpireSnapshots` required callers to always pass `WithRetainLast` and `WithOlderThan`; it never consulted the table-level retention properties (`min-snapshots-to-keep`, `max-snapshot-age-ms`, `max-ref-age-ms`), even though their keys and defaults are defined in `properties.go`. Tables that set these properties via a catalog would not honor them during snapshot expiry, diverging from the Java implementation. **Fix:** read the three retention properties from table metadata and use them as the last-resort fallback in the `cmp.Or` chains for `maxRefAgeMs`, `minSnapshotsToKeep`, and `maxSnapshotAgeMs`. When a property is absent the `math.MaxInt` constants apply, preserving current behaviour for callers that pass explicit options. Fixes apache#839 ## Test plan - [x] `go test ./...` - [x] `gofmt -l` clean - [x] New test `TestExpireSnapshotsUsesTableProperties`: creates a table with `min-snapshots-to-keep=2` and `max-snapshot-age-ms=0` set as properties, calls `ExpireSnapshots()` with no options, and asserts exactly 2 snapshots survive. Signed-off-by: Ali <alliasgher123@gmail.com> --------- Signed-off-by: Ali <alliasgher123@gmail.com>
…#878) ## Summary `GetWriteProperties` read compression, page size, row group size etc. but silently ignored the two bloom filter properties defined in the same file: - `write.parquet.bloom-filter-max-bytes` — caps the per-column bloom filter size (default 1 MB) - `write.parquet.bloom-filter-enabled.column.<name>` — enables bloom filters per column Wire them through to the Arrow Parquet writer: - `WithMaxBloomFilterBytes` is always applied (defaults to the existing `ParquetBloomFilterMaxBytesDefault` constant). - All table properties whose key starts with `write.parquet.bloom-filter-enabled.column.` are scanned; for each match `WithBloomFilterEnabledFor` is called with the column name and parsed boolean. Fixes apache#844 ## Test plan - [x] `go test ./...` - [x] `gofmt` clean - [x] New `TestGetWritePropertiesBloomFilter` covering default max-bytes, custom max-bytes, and per-column enable/disable. Signed-off-by: Ali <alliasgher123@gmail.com> --------- Signed-off-by: Ali <alliasgher123@gmail.com>
…he#875) ## Summary The `SortOrderID` field on `WriteTask` was declared but never set by any producer, so every data file written via the standard/partitioned/rolling/equality-delete paths ended up with a nil `sort_order_id` in the manifest entry. This leaves readers with no way to know which sort order the file was written against. This PR threads `SortOrderID` end-to-end: * `WriteFileInfo` gains a `SortOrderID` field and `DataFileStatistics.ToDataFile` now calls `DataFileBuilder.SortOrderID` so the value lands on the final `DataFile`. * `arrow_utils.filesToDataFiles`, `writer.writeFile`, and `writerFactory` all pass the table's default sort order id down to `WriteFileInfo`, and the equality-delete / position-delete / rolling data writer paths populate the `WriteTask.SortOrderID` accordingly. Fixes apache#842 ## Test plan - [x] `go test ./table/...` - [x] `go vet ./...` - [x] New assertion: `withSortOrderIDMatching` added to `defaultPositionDeleteMatching` in `TestPositionDeletePartitionedFanoutWriterProcessBatch`; `mockDataFile` gains an overridable sort order id so this path is covered end-to-end. Signed-off-by: Ali <ali@kscope.ai> --------- Signed-off-by: Ali <ali@kscope.ai> Signed-off-by: Ali <alliasgher123@gmail.com>
…e#879) ## Summary `NewSchema` silently accepted multiple fields sharing the same positive integer ID. The Java implementation throws `IllegalArgumentException` at construction time with _"Multiple entries with same key: 6=struct.inner_req and 6=struct.inner_op"_. Adds `checkDuplicateFieldIDs` — a lightweight recursive walk over `NestedField`, `*StructType`, `*ListType`, and `*MapType` that only reads field IDs without dereferencing type pointers. It is called from `NewSchemaWithIdentifiers` before returning the schema. A duplicate positive ID panics with `ErrInvalidSchema`, consistent with the `indexByName` panic for duplicate field names (schema.go line 851). Non-positive IDs (`-1` etc. used as "unassigned" placeholders in schema builders) are skipped so existing internal usage is unaffected. Fixes apache#593 ## Test plan - [x] `go test ./...` - [x] `gofmt` clean - [x] `TestNewSchemaPanicsOnDuplicateFieldIDs`: top-level duplicate + the exact nested-struct case from the bug report. Signed-off-by: Ali <alliasgher123@gmail.com> --------- Signed-off-by: Ali <alliasgher123@gmail.com>
…pache#845) (apache#881) Replace the bytes fallback with avro.NewFixedSchema of the correct size. Update write path to encode []byte as [N]byte fixed arrays, and read path to decode them back. Adds round-trip test for fixed partition columns.
…che#837 (apache#882) PostCommit was collecting manifest lists, manifests, and data files for deletion when expiring snapshots, but never collected StatisticsFile or PartitionStatisticsFile paths. Statistics files for expired snapshots were never deleted, leaking storage. Note: This PR depends on apache#873 (MetadataBuilderFromBase preserving StatisticsList/PartitionStatsList). Until that merges, the subtraction at PostCommit (lines 517–523 of updates.go) relies on postTable.Metadata().Statistics( returning surviving entries — which it won't in the current main for a mixed expiry scenario. Fixes apache#837
…aFileOpts (apache#888) ## Summary Follow-up to the review comment on [apache#875](apache#875 (comment)) — `DataFileStatistics.ToDataFile` grew to eight positional arguments including two adjacent `int`s (`filesize` and `sortOrderID`), which is hard to read at call sites and easy to swap by mistake. This PR collapses the parameters into a `DataFileOpts` struct: ```go type DataFileOpts struct { Schema *iceberg.Schema Spec iceberg.PartitionSpec Path string Format iceberg.FileFormat Content iceberg.ManifestEntryContent FileSize int64 PartitionValues map[int]any SortOrderID int } func (d *DataFileStatistics) ToDataFile(opts DataFileOpts) iceberg.DataFile ``` All six call sites (two production, four tests) are updated to the named-field form. Future additions are source-compatible. ## Notes - Stacked on apache#875 — this PR will show both diffs until apache#875 merges, at which point only the refactor remains. - No behaviour change. ## Test plan - [x] `go test ./...` - [x] `gofmt` clean Signed-off-by: Ali <alliasgher123@gmail.com> Signed-off-by: Ali <alliasgher123@gmail.com>
The fanout function currently defer's `record.Release()`. This isn't a big deal if the iterable it reads from contains a small amount of data. However, if it's a long lived streaming channel or contains a lot of data, the fanout function will hold on to **all ** the records until it processes all of them. This dramatically increases memory use, especially in a streaming use case. This PR extracts out the code to process each record into it's own function, and calls `defer record.Release()` there. No functional modifications were made. This will release each record after successfully processing the data.
…m Long to Integer to match spec (apache#880) ## Description The `equality_ids` field (field-id 135) in the manifest entry's `data_file` struct is currently defined with `LongSchema` (Avro `long` / 64-bit) for its array element type. The [Iceberg spec](https://iceberg.apache.org/spec/#manifests) defines this field as `list<int>` (Avro `int` / 32-bit), and the [Java implementation](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/BaseFile.java#L365) reads it as `int[]` via `ArrayUtil.toIntArray`. This mismatch causes failures when other Iceberg implementations read manifests containing equality delete files written by iceberg-go: **Apache Spark / Java runtime:** ``` java.lang.ClassCastException: class java.lang.Long cannot be cast to class java.lang.Integer at org.apache.iceberg.util.ArrayUtil.toIntArray(ArrayUtil.java:50) at org.apache.iceberg.BaseFile.internalSet(BaseFile.java:336) ``` ### Spec reference [Iceberg Table Spec — Manifests — Data File fields](https://iceberg.apache.org/spec/#manifests): | Field id | Field name | Type | |----------|----------------|------------| | 135 | equality_ids | list\<int\> | ### How to reproduce 1. Write an Iceberg V2 table with equality delete files using iceberg-go (e.g., via `RowDelta.AddDeletes`) 2. Read the table with Spark: `SELECT * FROM table` → `ClassCastException` ### NOTE This would fix compatibility only for new manifests, old ones need to migrated somehow.
Drop github.com/hamba/avro/v2 in favor of github.com/twmb/avro v1.5.0 for manifest reading, writing, and partition schema construction, replacing the archived hamba/avro. # Behavior differences visible to callers - DataFile.Partition() returns int32 for plain integer partition fields (was int). Correct per Avro spec — Avro int is 32-bit signed. Go code doing v.(int) on partition values needs to switch to v.(int32). - NewDataFileBuilder takes map[int]string instead of map[int]avro.LogicalType. The corresponding field on dataFile and the hasFieldToIDMap interface also now use string. Logical type constants live in the new github.com/twmb/avro/atype subpackage (atype.Date, atype.TimestampMicros, etc.), untyped string constants, drop-in wherever a string is expected. All other hamba/avro types used by the public surface are replaced: avro.Schema → *avro.Schema on the handful of exported helpers that referenced them; avro.SchemaCache (internal.AvroSchemaCache) → map[string]*avro.Schema (internal.AvroSchemaMap). # Dependency footprint Removes five transitive dependencies: json-iterator/go, modern-go/concurrent, modern-go/reflect2, ettle/strcase, and go-viper/mapstructure (hamba's chain). twmb/avro's only direct dependency is klauspost/compress, which iceberg-go already transitively requires. # Tests New TestPartitionTypeToAvroSchemaDuplicateNamedTypes covers partition spec scenarios with multiple fields of the same named Avro type (two UUIDs, two same-size Fixed, two same-precision Decimal). Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#893) Add compaction.Analyze(ctx, tbl, cfg) as a read-only entry point that scans the table via PlanFiles and produces a compaction Plan without modifying anything. Useful for dry-run analysis, CLI tooling, and deciding whether compaction is worthwhile before executing. E2e tests with real Parquet files: - Small files detected as compaction candidates - Optimal files skipped when below MinInputFiles - Empty table produces empty plan - Invalid config returns error Part of apache#832 (table compaction).
…ally (apache#869) ## Problem `fastAppendFiles.existingManifests()` filters parent manifests using: ```go if m.HasAddedFiles() || m.HasExistingFiles() || m.SnapshotID() == fa.base.snapshotID { existing = append(existing, m) } ``` `HasAddedFiles()` returns `AddedFilesCount != 0` and `HasExistingFiles()` returns `ExistingFilesCount != 0` (v2 manifest file). Both return `false` when a manifest list entry has `added_files_count=0` and `existing_files_count=0`, which is the standard Iceberg v2 representation for **inherited manifests** written by external writers (Athena, Spark, Trino, etc.). As a result, any data written by an external writer is **silently dropped** from the snapshot on the next iceberg-go fast-append. After the append, queries return only the iceberg-go-written rows; all previously existing data becomes invisible. ## Root Cause Confirmed Reproduction: create a table with Athena, insert 2 rows, append 1 row with iceberg-go. After the append the parent snapshot's manifest list entries have `added_files_count=0, existing_files_count=0`. Both manifests are filtered out. Athena queries the new snapshot and sees only the 1 iceberg-go row. Diagnostic output from the new test (before fix): ``` Parent has 2 manifests: [0] snapshot_id=... added_files=0 existing_files=0 HasAdded=false HasExisting=false [1] snapshot_id=... added_files=0 existing_files=0 HasAdded=false HasExisting=false ``` This was discovered and confirmed during a production Iceberg table remediation at Docker. See [docker/data-platform#406](docker/data-platform#406) for the full investigation. ## Fix A fast-append never removes or overwrites data files, so all parent manifests should be inherited unconditionally. Remove the filter and return `previous.Manifests()` directly. ## Testing - New test `TestFastAppendInheritsZeroCountManifests` reproduces the bug (FAIL before patch, PASS after): creates two zero-count manifests simulating an Athena-written snapshot, fast-appends one new data file, asserts all 3 manifests are present in the resulting snapshot. - Full `./table/...` suite passes with no regressions.
Add Transaction.RewriteDataFiles() that reads data with deletes applied, writes new consolidated files, and atomically replaces old files via ReplaceFiles. Position delete files matched to rewritten data files are removed in the same commit. - CompactionTaskGroup bridges compaction planner and executor (avoids circular import between table/ and table/compaction/) - collectSafePositionDeletes only removes pos deletes with explicit content type check; equality deletes preserved (may apply outside scope) - Context cancellation checked between groups - Empty groups skipped gracefully E2e tests with real Parquet files: - Small file compaction (5→1 file, row count preserved) - Position delete cleanup (delete applied, delete file removed) - Empty plan / empty group handling - Partial progress mode - Context cancellation Part of apache#832 (table compaction).
…lts (apache#898) Fixes apache#324. `mergeConf` was checking `len(resConfig.Field) == 0` before applying values from the YAML config file. Since `docopt` populates `resConfig` with default values (e.g. `Output="text"`) before `mergeConf` runs, the file-config was silently ignored for any option that has a docopt default. **Fix:** scan `os.Args` to record which flags were explicitly provided, then apply the file-config value only when the corresponding flag was absent from the command line. CLI-provided values always take precedence; file-config fills in everything else. **Precedence after this change:** 1. Explicit CLI flag (highest) 2. YAML config file 3. docopt default (lowest) Signed-off-by: alliasgher <alliasgher123@gmail.com>
…apache#899) Fixes apache#714. Adds `Table.NewTransactionOnBranch(branch string)` so callers can commit to a named branch. The existing `NewTransaction()` becomes a convenience wrapper for `NewTransactionOnBranch(MainBranch)`. The branch name flows through `Transaction.branch` into `snapshotProducer`, replacing the two hardcoded `"main"` strings in `NewSetSnapshotRefUpdate` and `AssertRefSnapshotID`. ```go // Commit to a feature branch txn := tbl.NewTransactionOnBranch("my-feature-branch") if err := txn.Append(ctx, reader, nil); err != nil { ... } if _, err := txn.Commit(ctx); err != nil { ... } ``` Signed-off-by: alliasgher <alliasgher123@gmail.com>
…e#901) AddDataFiles unconditionally scans every manifest in the current snapshot to check whether any file being added already exists in the table. Each manifest requires a storage read (e.g. an S3 GET). For tables with many commits and manifests this means hundreds of sequential reads, easily pushing the operation past reasonable processing time. AddFiles method already has an ignoreDuplicates parameter that skips this scan, but AddDataFiles has no equivalent. WithoutDuplicateCheck() fills this gap. Callers who can guarantee the files being added are new (e.g. freshly written by a compaction job or an ingestion pipeline with unique naming) can opt out of the scan and avoid the I/O cost entirely.
…check) (apache#904) Adding these 3 checks for all apache/iceberg* repos as part of apache/iceberg#15742 + include missing dependabot for `github-actions`
…#834) (apache#891) After stats-based row group filtering, apply an additional bloom filter check for EqualTo and In predicates. Row groups where none of the queried values appear in the column bloom filter are skipped, reducing I/O for selective point lookups. - Add RowGroupBloomPred and ParquetRowGroupTester types to parquet_files.go; GetRecords runs stats then bloom filter checks per row group using the physical-byte hasher from the bloom filter itself to guarantee algorithm consistency with the writer - Add literalToPhysBytes and bloomPredicateCollector in evaluators.go to extract bloom-filterable predicates from bound expressions; And merges predicates from both sides, Or suppresses collection - Wire ParquetRowGroupTester in arrow_scanner.go processRecords - Add TestBloomFilterRowGroupPruning covering present/absent/In/unknown field ID cases; add TestLiteralToPhysBytes and TestBloomPredicateCollector
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.6.0 to 6.0.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/checkout/releases">actions/checkout's releases</a>.</em></p> <blockquote> <h2>v6.0.2</h2> <h2>What's Changed</h2> <ul> <li>Add orchestration_id to git user-agent when ACTIONS_ORCHESTRATION_ID is set by <a href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2355">actions/checkout#2355</a></li> <li>Fix tag handling: preserve annotations and explicit fetch-tags by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2356">actions/checkout#2356</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v6.0.1...v6.0.2">https://github.com/actions/checkout/compare/v6.0.1...v6.0.2</a></p> <h2>v6.0.1</h2> <h2>What's Changed</h2> <ul> <li>Update all references from v5 and v4 to v6 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2314">actions/checkout#2314</a></li> <li>Add worktree support for persist-credentials includeIf by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2327">actions/checkout#2327</a></li> <li>Clarify v6 README by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2328">actions/checkout#2328</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v6...v6.0.1">https://github.com/actions/checkout/compare/v6...v6.0.1</a></p> <h2>v6.0.0</h2> <h2>What's Changed</h2> <ul> <li>Update README to include Node.js 24 support details and requirements by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2248">actions/checkout#2248</a></li> <li>Persist creds to a separate file by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2286">actions/checkout#2286</a></li> <li>v6-beta by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2298">actions/checkout#2298</a></li> <li>update readme/changelog for v6 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2311">actions/checkout#2311</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v5.0.0...v6.0.0">https://github.com/actions/checkout/compare/v5.0.0...v6.0.0</a></p> <h2>v6-beta</h2> <h2>What's Changed</h2> <p>Updated persist-credentials to store the credentials under <code>$RUNNER_TEMP</code> instead of directly in the local git config.</p> <p>This requires a minimum Actions Runner version of <a href="https://github.com/actions/runner/releases/tag/v2.329.0">v2.329.0</a> to access the persisted credentials for <a href="https://docs.github.com/en/actions/tutorials/use-containerized-services/create-a-docker-container-action">Docker container action</a> scenarios.</p> <h2>v5.0.1</h2> <h2>What's Changed</h2> <ul> <li>Port v6 cleanup to v5 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2301">actions/checkout#2301</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v5...v5.0.1">https://github.com/actions/checkout/compare/v5...v5.0.1</a></p> <h2>v5.0.0</h2> <h2>What's Changed</h2> <ul> <li>Update actions checkout to use node 24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li> <li>Prepare v5.0.0 release by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2238">actions/checkout#2238</a></li> </ul> <h2>⚠️ Minimum Compatible Runner Version</h2> <p><strong>v2.327.1</strong><br /> <a href="https://github.com/actions/runner/releases/tag/v2.327.1">Release Notes</a></p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/actions/checkout/blob/main/CHANGELOG.md">actions/checkout's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <h2>v6.0.2</h2> <ul> <li>Fix tag handling: preserve annotations and explicit fetch-tags by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2356">actions/checkout#2356</a></li> </ul> <h2>v6.0.1</h2> <ul> <li>Add worktree support for persist-credentials includeIf by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2327">actions/checkout#2327</a></li> </ul> <h2>v6.0.0</h2> <ul> <li>Persist creds to a separate file by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2286">actions/checkout#2286</a></li> <li>Update README to include Node.js 24 support details and requirements by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2248">actions/checkout#2248</a></li> </ul> <h2>v5.0.1</h2> <ul> <li>Port v6 cleanup to v5 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2301">actions/checkout#2301</a></li> </ul> <h2>v5.0.0</h2> <ul> <li>Update actions checkout to use node 24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li> </ul> <h2>v4.3.1</h2> <ul> <li>Port v6 cleanup to v4 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2305">actions/checkout#2305</a></li> </ul> <h2>v4.3.0</h2> <ul> <li>docs: update README.md by <a href="https://github.com/motss"><code>@motss</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li> <li>Add internal repos for checking out multiple repositories by <a href="https://github.com/mouismail"><code>@mouismail</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li> <li>Documentation update - add recommended permissions to Readme by <a href="https://github.com/benwells"><code>@benwells</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li> <li>Adjust positioning of user email note and permissions heading by <a href="https://github.com/joshmgross"><code>@joshmgross</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2044">actions/checkout#2044</a></li> <li>Update README.md by <a href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2194">actions/checkout#2194</a></li> <li>Update CODEOWNERS for actions by <a href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2224">actions/checkout#2224</a></li> <li>Update package dependencies by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2236">actions/checkout#2236</a></li> </ul> <h2>v4.2.2</h2> <ul> <li><code>url-helper.ts</code> now leverages well-known environment variables by <a href="https://github.com/jww3"><code>@jww3</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1941">actions/checkout#1941</a></li> <li>Expand unit test coverage for <code>isGhes</code> by <a href="https://github.com/jww3"><code>@jww3</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1946">actions/checkout#1946</a></li> </ul> <h2>v4.2.1</h2> <ul> <li>Check out other refs/* by commit if provided, fall back to ref by <a href="https://github.com/orhantoy"><code>@orhantoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1924">actions/checkout#1924</a></li> </ul> <h2>v4.2.0</h2> <ul> <li>Add Ref and Commit outputs by <a href="https://github.com/lucacome"><code>@lucacome</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1180">actions/checkout#1180</a></li> <li>Dependency updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a>- <a href="https://redirect.github.com/actions/checkout/pull/1777">actions/checkout#1777</a>, <a href="https://redirect.github.com/actions/checkout/pull/1872">actions/checkout#1872</a></li> </ul> <h2>v4.1.7</h2> <ul> <li>Bump the minor-npm-dependencies group across 1 directory with 4 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1739">actions/checkout#1739</a></li> <li>Bump actions/checkout from 3 to 4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1697">actions/checkout#1697</a></li> <li>Check out other refs/* by commit by <a href="https://github.com/orhantoy"><code>@orhantoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1774">actions/checkout#1774</a></li> <li>Pin actions/checkout's own workflows to a known, good, stable version. by <a href="https://github.com/jww3"><code>@jww3</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1776">actions/checkout#1776</a></li> </ul> <h2>v4.1.6</h2> <ul> <li>Check platform to set archive extension appropriately by <a href="https://github.com/cory-miller"><code>@cory-miller</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1732">actions/checkout#1732</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/actions/checkout/commit/de0fac2e4500dabe0009e67214ff5f5447ce83dd"><code>de0fac2</code></a> Fix tag handling: preserve annotations and explicit fetch-tags (<a href="https://redirect.github.com/actions/checkout/issues/2356">#2356</a>)</li> <li><a href="https://github.com/actions/checkout/commit/064fe7f3312418007dea2b49a19844a9ee378f49"><code>064fe7f</code></a> Add orchestration_id to git user-agent when ACTIONS_ORCHESTRATION_ID is set (...</li> <li><a href="https://github.com/actions/checkout/commit/8e8c483db84b4bee98b60c0593521ed34d9990e8"><code>8e8c483</code></a> Clarify v6 README (<a href="https://redirect.github.com/actions/checkout/issues/2328">#2328</a>)</li> <li><a href="https://github.com/actions/checkout/commit/033fa0dc0b82693d8986f1016a0ec2c5e7d9cbb1"><code>033fa0d</code></a> Add worktree support for persist-credentials includeIf (<a href="https://redirect.github.com/actions/checkout/issues/2327">#2327</a>)</li> <li><a href="https://github.com/actions/checkout/commit/c2d88d3ecc89a9ef08eebf45d9637801dcee7eb5"><code>c2d88d3</code></a> Update all references from v5 and v4 to v6 (<a href="https://redirect.github.com/actions/checkout/issues/2314">#2314</a>)</li> <li><a href="https://github.com/actions/checkout/commit/1af3b93b6815bc44a9784bd300feb67ff0d1eeb3"><code>1af3b93</code></a> update readme/changelog for v6 (<a href="https://redirect.github.com/actions/checkout/issues/2311">#2311</a>)</li> <li><a href="https://github.com/actions/checkout/commit/71cf2267d89c5cb81562390fa70a37fa40b1305e"><code>71cf226</code></a> v6-beta (<a href="https://redirect.github.com/actions/checkout/issues/2298">#2298</a>)</li> <li><a href="https://github.com/actions/checkout/commit/069c6959146423d11cd0184e6accf28f9d45f06e"><code>069c695</code></a> Persist creds to a separate file (<a href="https://redirect.github.com/actions/checkout/issues/2286">#2286</a>)</li> <li><a href="https://github.com/actions/checkout/commit/ff7abcd0c3c05ccf6adc123a8cd1fd4fb30fb493"><code>ff7abcd</code></a> Update README to include Node.js 24 support details and requirements (<a href="https://redirect.github.com/actions/checkout/issues/2248">#2248</a>)</li> <li><a href="https://github.com/actions/checkout/commit/08c6903cd8c0fde910a37f88322edcfb5dd907a8"><code>08c6903</code></a> Prepare v5.0.0 release (<a href="https://redirect.github.com/actions/checkout/issues/2238">#2238</a>)</li> <li>Additional commits viewable in <a href="https://github.com/actions/checkout/compare/v3.6.0...de0fac2e4500dabe0009e67214ff5f5447ce83dd">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Looks like the current config is incorrect based on the updated labeler action --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com>
… (apache#975) Two issues caused 301 PermanentRedirect errors when writing data files to S3 Table Buckets: 1. The SigV4 signing region configured on the REST catalog was not propagated to the S3 I/O layer. When s3.region is absent from vended credentials, the S3 client falls back to the SDK default region, which may differ from the bucket's actual region. Fix: toProps() now sets client.region from the SigV4 signing region as a fallback. 2. The S3 client defaulted to path-style addressing, which is deprecated by AWS. Virtual-hosted style is now the default for standard AWS S3, with path-style reserved for custom endpoints. Fixes apache#974
…ache#979) newConflictContext returned an empty concurrent list when the writer's base had no snapshot on the branch, silently bypassing RowDelta SERIALIZABLE validation. Walk the branch head's ancestry instead and treat truncation as ErrCommitDiverged. Fixes apache#977 Signed-off-by: laskoviymishka <laskoviymishka@gmail.com>
…log (apache#968) Set up Docker and Spark infrastructure for Hadoop catalog cross-compatibility testing with Java's HadoopCatalog. - Add hadoop_validation.py: SparkSession configured with spark.sql.catalog.hadoop_test (type=hadoop, warehouse=/home/iceberg/hadoop-warehouse) - Add shared volume mount in docker-compose.yml: /tmp/iceberg-hadoop-warehouse (host) <-> /home/iceberg/hadoop-warehouse (Spark) - Copy hadoop_validation.py into Spark container via Dockerfile - Add make integration-hadoop target Depends on: apache#963 Relates to apache#798
) Replace flat Config struct and boolean dispatch with typed subcommand structs. Each command owns its flags, subcommand help is auto-generated, and adding new commands is self-contained. Closes: apache#965
[3: Namespace operations](apache#798 (comment)) Implement all six namespace interface methods. CreateNamespace uses os.MkdirAll and errors if the directory already exists or if non-empty properties are passed. DropNamespace checks that there are no tables or child namespaces before calling os.Remove. CheckNamespaceExists uses os.Stat. ListNamespaces reads directory entries and filters out table dirs via isTableDir. LoadNamespaceProperties returns a synthetic {"location": path} (not persisted). UpdateNamespaceProperties returns an unsupported error. Tests cover create, create duplicate, create nested (a.b.c), create with properties (error), drop, drop non-existent, drop non-empty with tables and child namespaces, list empty and mixed entries, check exists, load properties, and update properties error. Relates to apache#798
) `FetchEntries` materializes every manifest entry into a `[]ManifestEntry` before returning, so a caller pays the full per-manifest entry slice even when it only needs to filter, count, or break out on first match. On tables with thousands of entries per manifest this shows up as avoidable memory pressure.
…waysTrue (apache#983) Fixes apache#978 ## Problem `RowDelta.validate` passes `iceberg.AlwaysTrue{}` to `validateNoConflictingDataFiles` whenever equality-delete files are present. This means any concurrent append to **any** partition is treated as a conflict, even when it lands in a completely different partition from the equality-deletes. Under serializable isolation this causes spurious `ErrConflictingDataFiles` errors for workloads that write to multiple independent partitions concurrently. ## Fix `RowDelta` now collects the partition tuples of all equality-delete files it adds (`eqDeletePartitions`). A new validator, `validateNoConflictingDataFilesInPartitions`, checks concurrent data files only in those specific partition tuples: - If the partition set is empty (no eq-deletes), the check is skipped. - If any eq-delete is unpartitioned (empty tuple), it falls back to the conservative `AlwaysTrue` check, preserving existing safety. - Otherwise, only concurrent files in a matching partition are flagged. ## Files changed - `table/row_delta.go`: collect `eqDeletePartitions` instead of `hasEqDeletes` - `table/conflict_validation.go`: `validateNoConflictingDataFilesInPartitions` + `partitionTupleKey` - `table/partition_conflict_test.go`: unit tests for both new functions
## Summary The hash `634933edcd8ababfe52f92936142cc22ac488b1b` resolves to tag `v6.0.1`, but the version comment said `# v6`. This caused zizmor's `ref-version-mismatch` audit to fail in CI. Updates the comment to match the actual tag the hash points to.
…isticsUpdate (apache#1018) Adds SetPartitionStatisticsUpdate and RemovePartitionStatisticsUpdate in table/updates.go with MetadataBuilder methods and UnmarshalJSON wiring. Mirrors the existing SetStatisticsUpdate / RemoveStatisticsUpdate from apache#902. Closes apache#1009
Validates that initial-default / write-default on struct, list, and map fields match the expected JSON shape (object for struct/map, array for list). Walks the full schema tree including nested fields within list/map elements. Mirrors the existing unknownTypeValidator pattern. Closes apache#1011
## Summary - Adds `MemFS`: an in-memory `IO` implementation with no external dependencies - `MemFS` implements `WriteFileIO` and `ListableIO` - Moves mem tests from `io/gocloud` to `io/` and removes the now-unnecessary gocloud import from `table/snapshot_producers_test.go` Part of apache#696.
…stics (apache#1012) (apache#1019) Adds a Java-produced v3 metadata.json fixture with partition-statistics entries and a round-trip test that parses, re-serializes, and asserts no fields are silently dropped. Closes: apache#1012
) Similar to apache/iceberg#16229 The workflow files use SHA-pinned actions (immutable), but the human-readable comments referenced only major versions (e.g., `# v6`, `# v5`). **When maintainers move these mutable tags to a new commit, zizmor fails in CI because the SHA no longer matches the stated tag.**
…nifests (apache#1030) ## Problem `ManifestListWriter.AddManifests` requires every input manifest file's version to match the writer's version exactly, which contradicts the Iceberg spec: a v2 manifest list must be able to reference v1 manifest files so that v1 tables can be upgraded without rewriting historical manifests. Java's [`ManifestListWriter`](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/ManifestListWriter.java) handles this; `iceberg-go` does not. The practical symptom for downstream users is that any commit to a table whose metadata has been upgraded from v1 to v2 — whether via `Transaction.UpgradeFormatVersion` or out-of-band by another engine — fails with `invalid argument: ManifestListWriter only supports version 2 manifest files`. The v1 manifest files written before the upgrade are never rewritten and continue to surface via `snapshotProducer.existingManifests()` on every subsequent commit, so the failure is permanent. ## Fix `manifest.go:1410-1414`: replace the `file.Version() != m.version` exact-match gate with `file.Version() > m.version`. Newer-than-writer inputs are still rejected — the v2 entry schema has no place for v3 fields such as `first_row_id`, and accepting them would silently drop data. Lower-than-writer inputs are accepted because the in-memory `manifestFile` produced from a v1 input (via `manifestFileV1.toFile()` or `NewManifestFile(1, ...)`) already carries the spec's inheritance values — `Content = data` and `SeqNumber = MinSeqNumber = 0` — so it can be encoded directly against the v2/v3 entry schema. The existing `first_row_id` assignment for data manifests with `FirstRowIDValue == nil` covers v1 inputs in v3 lists without further changes. ## Testing - `TestV2ManifestListAcceptsV1Manifests` — round-trips a v1 manifest through a v2 manifest list and asserts that the decoded entry has `content = data` and `sequence_number = min_sequence_number = 0` per spec inheritance. - `TestV3ManifestListAcceptsV1AndV2Manifests` — writes a v1 data manifest and a v2 delete manifest into a v3 list; asserts inheritance for the v1 entry, asserts that `first_row_id` is assigned to the v1 data manifest and is left unset on the v2 delete manifest (data-only assignment per the existing v3 writer rules). - `TestV2ManifestListRejectsV3Manifests` — confirms the downgrade direction is still blocked. - `TestWriteManifestListClosesWriterOnError` — existing test updated to drive its `AddManifests` failure path through a v3-in-v2 input (still rejected) instead of v1-in-v2. Confirmed failing on `main` before the fix and passing after. `go test ./...` passes. Fixes apache#1029
…1022) ## Summary This PR gates validator registration on a new `needsValidation() bool` method on the `producerImpl` interface: - `fastAppendFiles` returns `false`: appends are commutative and need no conflict checks - `mergeAppendFiles` inherits `false` via embedding - `overwriteFiles` returns `true`: real conflict validation required - `snapshotProducer.commit()` skips registration when `needsValidation()` is false The no-op `validate()` methods are kept as belt-and-suspenders. Behavior is locked with two unit tests asserting that fast-append and merge-append commits leave `txn.validators` empty. Parent: apache#830 --------- Signed-off-by: hectar-glitches <hectar@uni.minerva.edu>
5: ListTables + DropTable + RenameTable Implement the remaining table catalog methods. ListTables verifies the namespace exists then scans the namespace directory for table subdirs using isTableDir. DropTable verifies the table exists then calls os.RemoveAll on the table directory. RenameTable returns an unsupported error. Tests cover list empty namespace, list with tables, list non-existent namespace, drop existing table and verify directory is removed, drop non-existent table, and rename returns error. Depends on apache#968 apache#963 Relates to apache#798
Closes apache#1003. Maps Iceberg unknown fields to plain Arrow null fields, and maps Arrow null fields back to Iceberg unknown. Checked with `go test ./table -run TestArrow`.
[7: CLI integration](apache#798 (comment)) Added a case catalog.Hadoop: branch to the catalog switch in cmd/iceberg/main.go, wire the cfg.Warehouse property, and add the blank import for catalog/hadoop.
fix(occ): rebuild manifest list on OCC retry to inherit concurrent writes Fixes apache#976 ## Problem When `doCommit` retries after an OCC conflict, it re-uses the stale snapshot that was built before the first attempt. That snapshot's manifest list was written against the original parent and does not include any data files committed by concurrent writers in the meantime. On catalogs that perform server-side snapshot validation (e.g. AWS S3 Tables), this causes **silent data loss**: the retried commit succeeds (HTTP 200) but the stale manifest list effectively drops the concurrent writer's files from the table history. ## Fix Each `snapshotProducer` now records which manifests it wrote itself (`ownManifests` — those not inherited from the original parent). A `rebuildManifestList` closure is attached to the `addSnapshotUpdate` and called by `doCommit` on every retry. The closure: 1. Loads the fresh branch head's manifest list (concurrent manifests). 2. Concatenates `ownManifests` with the fresh inherited manifests. 3. Writes a new manifest list file with a retry-attempt suffix so each attempt gets a unique path. 4. Adjusts `sequence-number` relative to the fresh parent. After a successful commit, `doCommit` deletes the manifest list files produced by superseded retry attempts (orphan cleanup). ## Files changed - `table/snapshot_producers.go`: `computeOwnManifests`, `rebuildFn` closure - `table/updates.go`: `ownManifests` / `rebuildManifestList` fields on `addSnapshotUpdate`, propagated through `Apply` - `table/table.go`: `rebuildSnapshotUpdates` helper, orphan cleanup after commit - `table/rebuild_manifest_test.go`: unit tests for `rebuildSnapshotUpdates` --------- Signed-off-by: mzzz-zzm <80936176+mzzz-zzm@users.noreply.github.com> Signed-off-by: masa hoashi <mhoashi@ea.com> Co-authored-by: masa hoashi <mhoashi@ea.com>
Adds MetadataCompressionCodecZstd alongside gzip/none and wires it through both read (decompress .metadata.json.zstd) and write (write.metadata.compression-codec=zstd) paths. Uses klauspost/compress/zstd already present as a transitive dependency. Closes apache#1005
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
laskoviymishka
force-pushed
the
deletion_vector-reader
branch
from
2d06d13 to
036db40
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.