Skip to content

fix: make migrate-to-javascript.sh stop when a listing does not page - #11

Merged
marevol merged 1 commit into
masterfrom
fix/migrate-script-paging-guard
Oct 7, 2026
Merged

marevol merged 1 commit into
masterfrom
fix/migrate-script-paging-guard

Conversation

@marevol

@marevol marevol commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Problem

bin/migrate-to-javascript.sh lists scheduled jobs and data configs with fessctl <resource> list --page N --size 100 and stops once it has total entries. Fess 15.9's admin list endpoints read size and page only from the JSON body of the request, and fessctl (0.3.0, the current release) sends them as URL parameters, so every call returns the first page of the server's default size (25). Fix for fessctl itself: codelibs/fessctl#38.

With more than 25 jobs or data configs the loop therefore read page 1 again and again, counted the duplicates toward total, and left the remaining entries on Groovy. The script exited 0.

Reproduction with the released fessctl against a stand-in for the admin list API that ignores paging, 60 jobs + 60 data configs (script before this change): exit 0, 150 update calls (100 of them repeats), only 25 jobs + 25 data configs switched.

Change (bin/migrate-to-javascript.sh only)

  • list_all keeps each id once. When a page adds no new id and fewer than total distinct entries were read, it exits 1 with a message that names the resource and the counts, says that the installed fessctl probably does not apply --size/--page, and tells the user to upgrade fessctl or reduce the entries to one page.
  • Both lists are read before anything is updated, so a listing failure on the data configs cannot leave the jobs already switched.
  • Unchanged: printed output, --dry-run, exit codes with a working listing.

Verification

  • Stub fessctl scenarios (offline): honoured paging with 60+60 and 250+250 entries → every entry updated exactly once; a server that caps the page size at 25 but honours --page → all updated; paging ignored with exactly one page (25+25) → still works; paging ignored with 60+60, 26+26, and 3 jobs + 60 data configs → exit 1, zero updates, message checked. The same scenarios fail on the unchanged script.
  • Released fessctl against the stand-in server: 60+60 → exit 1 with the new message and 0 updates; 25+25 → exit 0, all 50 switched.
  • Live: full 15.8.0 → 15.9 upgrade (stored Groovy jobs and data configs): dry run listed 14 jobs and 2 data configs, the real run switched all 16, the Default Crawler is ok afterwards, 288 documents unchanged, and a third run prints "Nothing to migrate".

list_all asks fessctl for "--page N --size 100" until it has read "total"
entries. A fessctl that sends --page/--size as URL parameters (fessctl 0.3.0)
gets the first page of the server's default size
(25) on every call, because the Fess 15.9 admin list API reads them only
from a JSON body. With more than 25 jobs or data configs the loop read
the same 25 entries again, left the rest on Groovy, and still reported
success.

Count each id once and exit with a message when a page adds no new
entry before "total" is reached. Read both lists before changing
anything, so a failure on the data configs cannot leave the jobs already
switched. Behaviour with a working listing (output, --dry-run, exit
codes) is unchanged.
@marevol marevol self-assigned this Oct 7, 2026
@marevol marevol added this to the 15.9.0 milestone Oct 7, 2026
@marevol
marevol merged commit 973a1e9 into master Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant