Repository navigation
fix: make migrate-to-javascript.sh stop when a listing does not page - #11
Merged
Merged
Conversation
list_all asks fessctl for "--page N --size 100" until it has read "total" entries. A fessctl that sends --page/--size as URL parameters (fessctl 0.3.0) gets the first page of the server's default size (25) on every call, because the Fess 15.9 admin list API reads them only from a JSON body. With more than 25 jobs or data configs the loop read the same 25 entries again, left the rest on Groovy, and still reported success. Count each id once and exit with a message when a page adds no new entry before "total" is reached. Read both lists before changing anything, so a failure on the data configs cannot leave the jobs already switched. Behaviour with a working listing (output, --dry-run, exit codes) is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
bin/migrate-to-javascript.shlists scheduled jobs and data configs withfessctl <resource> list --page N --size 100and stops once it hastotalentries. Fess 15.9's admin list endpoints readsizeandpageonly from the JSON body of the request, and fessctl (0.3.0, the current release) sends them as URL parameters, so every call returns the first page of the server's default size (25). Fix for fessctl itself: codelibs/fessctl#38.With more than 25 jobs or data configs the loop therefore read page 1 again and again, counted the duplicates toward
total, and left the remaining entries on Groovy. The script exited 0.Reproduction with the released fessctl against a stand-in for the admin list API that ignores paging, 60 jobs + 60 data configs (script before this change): exit 0, 150 update calls (100 of them repeats), only 25 jobs + 25 data configs switched.
Change (bin/migrate-to-javascript.sh only)
list_allkeeps each id once. When a page adds no new id and fewer thantotaldistinct entries were read, it exits 1 with a message that names the resource and the counts, says that the installed fessctl probably does not apply--size/--page, and tells the user to upgrade fessctl or reduce the entries to one page.--dry-run, exit codes with a working listing.Verification
fessctlscenarios (offline): honoured paging with 60+60 and 250+250 entries → every entry updated exactly once; a server that caps the page size at 25 but honours--page→ all updated; paging ignored with exactly one page (25+25) → still works; paging ignored with 60+60, 26+26, and 3 jobs + 60 data configs → exit 1, zero updates, message checked. The same scenarios fail on the unchanged script.okafterwards, 288 documents unchanged, and a third run prints "Nothing to migrate".