Merge branch 'feature/LF-3531/merge-third-party-MR' into 'master'

Implement missing features from the latest version of the fhirpath spec

See merge request lfor/fhirpath.js!62

Author: yuriy-sedinkin

.gitignore

@@ -5,6 +5,7 @@ src/scratch.js
 build
 .DS_Store
 .editorconfig
+.codex
 Makefile
 test/benchmark/prev-fhirpath
 test/benchmark/prev-src

AGENTS.md

@@ -1,60 +1,131 @@
 # AGENTS.md
 
 ## Project map (read this first)
-- This project implements the FHIRPath language in JavaScript and supports DSTU2, STU3, R4, and R5 model contexts.
-- Entry point is `src/fhirpath.js`: public API is `parse`, `compile`, `evaluate`, `resolveInternalTypes`, `types`.
-- `src/fhirpath.js` also exports `version`, `FP_Decimal`, `ucumUtils`, and `util` (used by external callers and custom function implementations).
-- Evaluation flow is `parse()` -> AST -> `_compile()` -> `applyParsedPath()` -> `engine.doEval()`; see `src/fhirpath.js` and `src/parser/index.js`.
-- Core behavior is table-driven: `engine.invocationTable` in `src/fhirpath.js` maps function/operator names to handlers and arity checks.
-- Most function families live in separate modules (e.g. `src/aggregate.js`, `src/filtering.js`, `src/math.js`) and export an `engine` object.
-- FHIR version-specific typing/model metadata is in `fhir-context/{dstu2,stu3,r4,r5}` and is required for choice-type/type resolution.
-- CLI entry is `bin/fhirpath`; behavior is covered by `test/bin_fhirpath.test.js`.
+
+- This project implements the FHIRPath language in JavaScript and supports
+  DSTU2, STU3, R4, and R5 model contexts.
+- Entry point is `src/fhirpath.js`: public API is `parse`, `compile`,
+  `evaluate`, `resolveInternalTypes`, `types`.
+- `src/fhirpath.js` also exports `version`, `FP_Decimal`, `ucumUtils`, and
+  `util` (used by external callers and custom function implementations).
+- Evaluation flow is `parse()` -> AST -> `_compile()` -> `applyParsedPath()` ->
+  `engine.doEval()`; see `src/fhirpath.js` and `src/parser/index.js`.
+- Core behavior is table-driven: `engine.invocationTable` in `src/fhirpath.js`
+  maps function/operator names to handlers and arity checks.
+- Most function families live in separate modules (e.g. `src/aggregate.js`,
+  `src/filtering.js`, `src/math.js`) and export an `engine` object.
+- FHIR version-specific typing/model metadata is in
+  `fhir-context/{dstu2,stu3,r4,r5}` and is required for choice-type/type
+  resolution.
+- CLI entry is `bin/fhirpath`; behavior is covered by
+  `test/bin_fhirpath.test.js`.
 
 ## Architecture patterns that matter
-- Internal values are wrapped as `ResourceNode` / `FP_Type`; final conversion happens in `prepareEvalResult()` (`src/fhirpath.js`).
-- `evaluate()` mutates input resources by attaching hidden `__path__` metadata (documented in `README.md`); do not assume input objects stay pristine.
-- Base-path evaluation for partial resources uses `{base, expression}` and model normalization via `model.pathsDefinedElsewhere`/`model.path2Type` in `_compile()`.
-- Async operations (`resolve`, `memberOf`, `%terminologies.*`) require `options.async`; guards are enforced via `util.checkAllowAsync` (see `src/additional.js`, `src/terminologies.js`).
-- `%factory.*` is implemented as a class with its own invocation table in `src/factory.js` and injected as `%factory` in eval context.
+
+- Internal values are wrapped as `ResourceNode` / `FP_Type`; final conversion
+  happens in `prepareEvalResult()` (`src/fhirpath.js`).
+- `evaluate()` mutates input resources by attaching hidden `__path__` metadata (
+  documented in `README.md`); do not assume input objects stay pristine.
+- Base-path evaluation for partial resources uses `{base, expression}` and model
+  normalization via `model.pathsDefinedElsewhere`/`model.path2Type` in
+  `_compile()`.
+- Async operations (`resolve`, `memberOf`, `%terminologies.*`) require
+  `options.async`; guards are enforced via `util.checkAllowAsync` (see
+  `src/additional.js`, `src/terminologies.js`).
+- For mixed sync/async flows, `util.checkAllowAsync` belongs in the function
+  that creates an async promise; do not add it to pure Promise combiners such
+  as `sort` using `Promise.all(...)`.
+- `evaluate()`/`compile()` accept `options.debugger` for step-level tracing;
+  the callback receives `(ctx, parentData, result, node)` after each eval step
+  (see `engine.doEvalSync()` in `src/fhirpath.js`).
+- With `resolveInternalTypes: false`, `prepareEvalResult()` keeps
+  `ResourceNode` wrappers in output (instead of flattening to plain values), so
+  downstream evaluations can retain type/path metadata.
+- `%factory.*` is implemented as a class with its own invocation table in
+  `src/factory.js` and injected as `%factory` in eval context.
 
 ## Testing workflow (project-specific)
-- Spec-style unit tests are YAML-driven: `test/fhirpath.test.js` loads `test/cases/*.yaml` and generates Jest tests dynamically.
-- `test/cases/fhir-r4.yaml` and `test/cases/fhir-r5.yaml` are generated from `converter/dataset`; refresh them with `node converter/bin/index.js -s` (skip download, use local dataset files).
-- Additional Jest suites in `test/*.test.js` cover APIs and behaviors outside YAML cases (e.g. `test/async-functions.test.js`, `test/user-invocation-table.test.js`, `test/bin_fhirpath.test.js`).
-- Many tests run in both math modes automatically (`preciseMath` true/false) unless a case sets `preciseMath` explicitly.
-- `npm run test:unit` runs Jest three times across time zones (`default`, `America/New_York`, `Europe/Paris`) to catch datetime regressions.
-- Test helpers in `test/test_utils.js` auto-load models (`r5`, `r4`, `stu3`, `dstu2`) and deep-clone input resources to avoid cross-test pollution.
-- Add new spec-style cases in `test/cases/*.yaml`; use keys like `model`, `inputfile`, `variables`, `context`, `error`, `result`.
-- Type declaration checks use `npm run test:tsd` with tests in `test/typescript/fhirpath.test-d.ts`.
-- End-to-end browser checks run via Cypress under `test/cypress/` (script: `npm run test:e2e`, which builds the demo first).
+
+- Spec-style unit tests are YAML-driven: `test/fhirpath.test.js` loads
+  `test/cases/*.yaml` and generates Jest tests dynamically.
+- `test/cases/fhir-r4.yaml` and `test/cases/fhir-r5.yaml` are generated from
+  `converter/dataset`; refresh them with `node converter/bin/index.js -s` (skip
+  download, use local dataset files).
+- Additional Jest suites in `test/*.test.js` cover APIs and behaviors outside
+  YAML cases (e.g. `test/async-functions.test.js`,
+  `test/user-invocation-table.test.js`, `test/bin_fhirpath.test.js`).
+- Many tests run in both math modes automatically (`preciseMath` true/false)
+  unless a case sets `preciseMath` explicitly.
+- `npm run test:unit` runs Jest three times across time zones (`default`,
+  `America/New_York`, `Europe/Paris`) to catch datetime regressions.
+- Test helpers in `test/test_utils.js` auto-load models (`r5`, `r4`, `stu3`,
+  `dstu2`) and deep-clone input resources to avoid cross-test pollution.
+- Add new spec-style cases in `test/cases/*.yaml`; use keys like `model`,
+  `inputfile`, `variables`, `context`, `error`, `result`.
+- Type declaration checks use `npm run test:tsd` with tests in
+  `test/typescript/fhirpath.test-d.ts`.
+- End-to-end browser checks run via Cypress under `test/cypress/` (script:
+  `npm run test:e2e`, which builds the demo first).
 
 ## Build/dev commands you will actually use
-- Install: `npm install` (if `node` is missing in this environment, run `source bashrc.fhirpath` and retry).
-- Lint: `npm run lint` (targets `src/parser/index.js`, `src/*.js`, `converter/`).
+
+- Install: `npm install` (if `node` is missing in this environment, run
+  `source bashrc.fhirpath` and retry).
+- Lint: `npm run lint` (targets `src/parser/index.js`, `src/*.js`,
+  `converter/`).
 - Unit tests: `npm run test:unit`; debugger mode: `npm run test:unit:debug`.
 - Type tests: `npm run test:tsd`.
 - Demo build: `npm run build:demo` (`npm run build` + `demo` webpack build).
 - E2E tests: `npm run test:e2e` (builds demo + runs Cypress).
 - Full CI-like local run: `npm test` (lint + tsd + unit + e2e).
-- Parser regeneration: `npm run generateParser` (uses `src/parser/FHIRPath.g4`, `antlr-4.9.3-complete.jar`, then `scripts/fix-parser-imports.js`).
-- Browser artifacts: `npm run build` (webpack outputs in `browser-build/`, plus zipped distributable).
+- Parser regeneration: `npm run generateParser` (uses `src/parser/FHIRPath.g4`,
+  `antlr-4.9.3-complete.jar`, then `scripts/fix-parser-imports.js`).
+- Browser artifacts: `npm run build` (webpack outputs in `browser-build/`, plus
+  zipped distributable).
 - Performance comparison: `npm run compare-performance` (`test/benchmark.js`).
+  Use `-- -r <git-ref>` to compare against a local baseline commit and check the
+  generated report at `test/benchmark/results/compare-performance-report.html`.
 
 ## Sandbox and escalation
-- If a command that is important to the task fails due to sandbox restrictions (for example EPERM/spawn permission errors), rerun it with escalated permissions and include a brief justification request.
+
+- If a command that is important to the task fails due to sandbox restrictions (
+  for example EPERM/spawn permission errors), rerun it with escalated
+  permissions and include a brief justification request.
 
 ## Conventions to preserve when editing
-- Performance is a top priority: prefer lower-allocation, lower-overhead designs, and treat measurable performance regressions as blockers unless explicitly approved.
-- Use CommonJS (`require`/`module.exports`) and 2-space indentation (see `eslint.config.js`).
-- Use modern JavaScript syntax (`const`/`let`, arrow functions, destructuring) where it matches existing files.
-- In AI chat responses, prefer project-relative file references in `path:line` format (e.g. `src/fhirpath.js:19`).
-- Keep two blank lines between declarations and above JSDoc blocks.
-- Treat JSDoc as part of a declaration: keep two blank lines above the JSDoc block, and no blank line between JSDoc and the declaration.
-- Keep project layout conventions: core logic in `src/`, FHIR-version-specific metadata in `fhir-context/`, tests in `test/`, and docs in `docs/` or `README.md`.
-- New FHIRPath functions are typically added as module functions plus registration in `engine.invocationTable` (`src/fhirpath.js`).
-- For feature work, update tests together with implementation (`test/cases/*.yaml` for spec-style behavior, plus `test/*.test.js` when behavior is API-specific).
-- For custom function hooks, follow `userInvocationTable` conventions in `_compile()`; use `internalStructures: true` only if handler needs raw `ResourceNode` access.
-- For async server calls, thread options via `terminologyUrl`, `fhirServerUrl`, `httpHeaders`, and optional `signal` (see `docs/auth.md`, `docs/abort.md`).
-- If public API signatures or exports change, update `src/fhirpath.d.ts` and validate with `npm run test:tsd` (`test/typescript/fhirpath.test-d.ts`).
-- Keep behavior standards-compliant with FHIRPath and aligned with the selected FHIR model version when model-aware behavior is involved.
-- Contributors **MUST NOT** hand-edit `test/cases/fhir-r4.yaml` or `test/cases/fhir-r5.yaml` (these files are generated).
+
+- Performance is a top priority: prefer lower-allocation, lower-overhead
+  designs, and treat measurable performance regressions as blockers unless
+  explicitly approved.
+- Use CommonJS (`require`/`module.exports`) and 2-space indentation (see
+  `eslint.config.js`).
+- When generating code, keep lines to no more than 80 characters when practical.
+- Use modern JavaScript syntax (`const`/`let`, arrow functions, destructuring)
+  where it matches existing files.
+- In AI chat responses, prefer project-relative file references in `path:line`
+  format (e.g. `src/fhirpath.js:19`).
+- Blank lines: keep two between declarations; for JSDoc, keep two above the
+  block and none between the block and its declaration.
+- In authored `test/*.test.js` files, keep two blank lines before and after
+  `describe(...)` blocks and between `it(...)` blocks; inside each
+  `describe(...)`, keep one blank line before the first `it(...)` and after
+  the last `it(...)`.
+- Keep project layout conventions: core logic in `src/`, FHIR-version-specific
+  metadata in `fhir-context/`, tests in `test/`, and docs in `docs/` or
+  `README.md`.
+- New FHIRPath functions are typically added as module functions plus
+  registration in `engine.invocationTable` (`src/fhirpath.js`).
+- For feature work, update tests together with implementation (
+  `test/cases/*.yaml` for spec-style behavior, plus `test/*.test.js` when
+  behavior is API-specific).
+- For custom function hooks, follow `userInvocationTable` conventions in
+  `_compile()`; use `internalStructures: true` only if handler needs raw
+  `ResourceNode` access.
+- For async server calls, thread options via `terminologyUrl`, `fhirServerUrl`,
+  `httpHeaders`, and optional `signal` (see `docs/auth.md`, `docs/abort.md`).
+- If public API signatures or exports change, update `src/fhirpath.d.ts` and
+  validate with `npm run test:tsd` (`test/typescript/fhirpath.test-d.ts`).
+- Keep behavior standards-compliant with FHIRPath and aligned with the selected
+  FHIR model version when model-aware behavior is involved.
+- Contributors **MUST NOT** hand-edit `test/cases/fhir-r4.yaml` or
+  `test/cases/fhir-r5.yaml` (these files are generated).

CHANGELOG.md

@@ -3,6 +3,29 @@
 This log documents significant changes for each release.  This project follows
 [Semantic Versioning](http://semver.org/).
 
+## [4.10.0] - 2026-04-21
+### Added
+- Added `sort()` support with optional multi-key sort arguments and
+  `asc`/`desc` direction controls, including async sort expressions.
+- Added variadic `coalesce(...)` with short-circuit evaluation for both
+  sync and async arguments.
+- Added `pathname([short])` for returning element paths within the input
+  resource.
+- Added string functions `lastIndexOf()`, `matchesFull()`, `escape()`, and
+  `unescape()`, and added optional flags support to `matches()`.
+- Added date/time component helpers: `yearOf()`, `monthOf()`, `dayOf()`,
+  `hourOf()`, `minuteOf()`, `secondOf()`, `millisecondOf()`,
+  `timezoneOffsetOf()`, `dateOf()`, and `timeOf()`.
+
+### Changed
+- Updated parser grammar and generated parser artifacts for the latest
+  FHIRPath syntax additions used by these features.
+- Updated the unit test scripts to call Jest via a Windows-compatible path.
+
+### Fixed
+- Fixed a rare error that occurred during unit tests when the time value was
+  missing milliseconds.
+
 ## [4.9.3] - 2026-03-31
 ### Changed
 - Updated package repository metadata in `package.json` to an explicit git

README.md

@@ -479,6 +479,32 @@ implementation (e.g. `userInvocationTable.pow.fn`) can be an array of
 value as is, you can use the `fhirpath.util.valData` or
 `fhirpath.util.valDataConverted` function (see src/utilities.js).
 
+
+If a user-defined function performs an asynchronous operation (returns a
+`Promise`), it should verify that async evaluation is allowed by calling
+`fhirpath.util.checkAllowAsync`.
+
+```js
+const userInvocationTable = {
+  asyncIdentity: {
+    fn: function(inputs) {
+      fhirpath.util.checkAllowAsync(this, 'asyncIdentity');
+      return Promise.resolve(inputs);
+    },
+    arity: {0: []}
+  }
+};
+
+// Throws without { async: true }.
+const resultPromise = fhirpath.evaluate(
+  { value: [3, 2, 1] },
+  "value.sort(asyncIdentity())",
+  {},
+  null,
+  { async: true, userInvocationTable }
+);
+```
+
 ## fhirpath CLI
 
 bin/fhirpath is a command-line tool for experimenting with FHIRPath.
@@ -652,33 +678,38 @@ open browser on localhost:8080
 ### Updating the FHIR module on a FHIR release
 * Download the FHIR StructureDefinitions (into the `fhir-context` directory - *don't check these in*)
   ```
-  > wget http://hl7.org/fhir/profiles-types.json -O profiles-types.json
-  > wget http://hl7.org/fhir/profiles-others.json -O profiles-others.json
-  > wget http://hl7.org/fhir/profiles-resources.json -O profiles-resources.json
-  > wget https://hl7.org/fhir/search-parameters.json -O search-parameters.json
+  > wget 'http://hl7.org/fhir/6.0.0-ballot3/profiles-types.json' -O profiles-types.json
+  > wget 'http://hl7.org/fhir/6.0.0-ballot3/profiles-others.json' -O profiles-others.json
+  > wget 'http://hl7.org/fhir/6.0.0-ballot3/profiles-resources.json' -O profiles-resources.json
+  > wget 'https://hl7.org/fhir/6.0.0-ballot3/search-parameters.json' -O search-parameters.json
   ```
 * Create the new folder for the version you are importing
   ```
-  > mkdir r5
+  > mkdir r6
   ```
-* Run the script `` with NodeJS
+* Run the script `extract-model-info.js` with NodeJS
   ```
-  > node ./extract-model-info.js --outputDir r5 --fhirDefDir .
+  > node ./extract-model-info.js --outputDir r6 --fhirDefDir .
   ```
 * Compare the output files in the new folder to those of the last release
   (looking for issues that might be due to changes in the StructureDefinition format)
 * Copy the `index.js` file from the last release into the new folder
   ```
-  > cp ../r4/index.js r5
+  > cp ../r5/index.js r6
   ```
-* Update the `/index.d.ts` file to include the new module as an export
+* Update the `/index.d.ts` file to include the new module as an export (can copy from previous version)
   ``` js
-  declare module "fhirpath/fhir-context/r5" {
+  declare module "fhirpath/fhir-context/r6" {
     export const {
+      version,
+      score,
       choiceTypePaths,
       pathsDefinedElsewhere,
       type2Parent,
-      path2Type
+      path2Type,
+      resourcesWithUrlParam,
+      path2TypeWithoutElements,
+      path2RefType
     }: Model;
   }
   ```