Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
f6532e8
add example verifies_requirement
carrils Dec 3, 2024
2f54b3e
add verifies_requirements reference for roughly half of requirements …
carrils Dec 9, 2024
9b359be
add rest of requirements and updated coverage csv
carrils Dec 10, 2024
37c3a4b
move verifies_requirements placement, address comments
carrils Dec 13, 2024
b5d36be
remove duplicates, remove @32, address comments for @33-36
carrils Dec 18, 2024
56f5159
update requirements references for status_check_test.rb and output_ch…
carrils Dec 18, 2024
8eb42a9
add @152
carrils Dec 18, 2024
f9e6a42
removed client reqs, removed should/may reqs, added new reqs 297+, mo…
carrils Jan 8, 2025
7cd06db
moved @230-@274 to operation-specific locations
carrils Jan 9, 2025
2cef8a1
move async reqs
carrils Jan 10, 2025
9004ee3
remove rest of should reqs
carrils Jan 13, 2025
996cd74
update coverage file
carrils Jan 13, 2025
531cd3f
Merge branch 'main' into v2-verifies-requirements
elsaperelli May 16, 2025
2017629
Update inferno core, run requirements coverage, remove MAY requiremen…
elsaperelli May 16, 2025
0bdbd3a
Remove 230, move hidden annotations, regenerate reqs and coverage, ad…
elsaperelli May 19, 2025
92528da
add client reqs config
Jul 1, 2025
18b9a28
Add reqs 15 and 16
elsaperelli Jul 1, 2025
f8f326c
move v1 suite requirements to v1 requirements
Jul 1, 2025
7b81420
Merge branch 'main' into v2-verifies-requirements
karlnaden Jul 1, 2025
2c4415c
add roo back in to gemfile.lock
Jul 1, 2025
3fe3d94
add platform
Jul 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 6 additions & 14 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ PATH
bulk_data_test_kit (0.12.2)
bloomer (~> 1.0.0)
colorize (~> 0.8.1)
inferno_core (>= 0.6.7)
inferno_core (>= 0.6.8)
json-jwt (~> 1.15.3)
mime-types (~> 3.4.0)
ndjson (~> 1.0.0)
Expand Down Expand Up @@ -197,7 +197,6 @@ GEM
mime-types (3.4.1)
mime-types-data (~> 3.2015)
mime-types-data (3.2025.0514)
mini_portile2 (2.8.9)
minitest (5.25.5)
msgpack (1.8.0)
multi_json (1.15.0)
Expand All @@ -213,13 +212,8 @@ GEM
ndjson (1.0.0)
netrc (0.11.0)
nio4r (2.7.4)
nokogiri (1.18.8)
mini_portile2 (~> 2.8.2)
racc (~> 1.4)
nokogiri (1.18.8-arm64-darwin)
racc (~> 1.4)
nokogiri (1.18.8-x86_64-darwin)
racc (~> 1.4)
nokogiri (1.18.8-x86_64-linux-gnu)
racc (~> 1.4)
oauth2 (1.4.11)
Expand Down Expand Up @@ -264,6 +258,9 @@ GEM
mime-types (>= 1.16, < 4.0)
netrc (~> 0.8)
rexml (3.4.1)
roo (2.10.1)
nokogiri (~> 1)
rubyzip (>= 1.3.0, < 3.0.0)
rouge (4.5.2)
rspec (3.13.0)
rspec-core (~> 3.13.0)
Expand Down Expand Up @@ -291,10 +288,7 @@ GEM
json-jwt (~> 1.15.3)
jwt (~> 2.6)
tls_test_kit (~> 0.3.0)
sqlite3 (1.7.3)
mini_portile2 (~> 2.8.0)
sqlite3 (1.7.3-arm64-darwin)
sqlite3 (1.7.3-x86_64-darwin)
sqlite3 (1.7.3-x86_64-linux)
stringio (3.1.7)
strings (0.2.1)
Expand Down Expand Up @@ -326,10 +320,7 @@ GEM
zeitwerk (2.7.2)

PLATFORMS
arm64-darwin-21
ruby
x86_64-darwin-19
x86_64-darwin-20
arm64-darwin-23
x86_64-linux

DEPENDENCIES
Expand All @@ -338,6 +329,7 @@ DEPENDENCIES
debug
factory_bot (~> 6.1)
rack-test (~> 2.2)
roo (~> 2.10.1)
rspec (~> 3.10)
webmock (~> 3.11)

Expand Down
3 changes: 2 additions & 1 deletion bulk_data_test_kit.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Gem::Specification.new do |spec|
spec.license = 'Apache-2.0'
spec.add_runtime_dependency 'bloomer', '~> 1.0.0'
spec.add_runtime_dependency 'colorize', '~> 0.8.1'
spec.add_runtime_dependency 'inferno_core', '>= 0.6.7'
spec.add_runtime_dependency 'inferno_core', '>= 0.6.8'
spec.add_runtime_dependency 'json-jwt', '~> 1.15.3'
spec.add_runtime_dependency 'mime-types', '~> 3.4.0'
spec.add_runtime_dependency 'ndjson', '~> 1.0.0'
Expand All @@ -24,6 +24,7 @@ Gem::Specification.new do |spec|
spec.add_development_dependency 'factory_bot', '~> 6.1'
spec.add_development_dependency 'rspec', '~> 3.10'
spec.add_development_dependency 'webmock', '~> 3.11'
spec.add_development_dependency 'roo', '~> 2.10.1'
spec.required_ruby_version = Gem::Requirement.new('>= 3.3.6')
spec.metadata['homepage_uri'] = spec.homepage
spec.metadata['source_code_uri'] = spec.homepage
Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1,23 @@
Req Set,ID,Reason,Details
hl7.fhir.uv.bulkdata_2.0.0,31,Not Tested,This seems more like a requirement that the server can handle the alternative notation.
hl7.fhir.uv.bulkdata_2.0.0,48,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,54,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,61,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,74,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,84,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,91,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,95,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,102,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,104,Not Tested,Depends on MAY params requirements
hl7.fhir.uv.bulkdata_2.0.0,113,Not Tested,Depends on MAY params requirements
hl7.fhir.uv.bulkdata_2.0.0,114,Not Tested,Depends on MAY params requirements
hl7.fhir.uv.bulkdata_2.0.0,116,Not Tested,Same as 31 (but specific to `_typeFilter`)?
hl7.fhir.uv.bulkdata_2.0.0,124,Not Tested,See Question
hl7.fhir.uv.bulkdata_2.0.0,221,Not Verifiable,
hl7.fhir.uv.bulkdata_2.0.0,252,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,255,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,258,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,261,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,264,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,267,Not Tested,MAY param requirement
hl7.fhir.uv.bulkdata_2.0.0,270,Not Tested,MAY param requirement

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ class BulkDataExportCancelTest < Inferno::Test
After a bulk data request has been started, a client MAY send a delete request to the URL provided in the Content-Location header to cancel the request.
Bulk Data Server MUST support client's delete request and return HTTP Status Code of "202 Accepted"
DESCRIPTION
# link 'http://hl7.org/fhir/uv/bulkdata/STU1.0.1/export/index.html#bulk-data-delete-request'
# link 'http://hl7.org/fhir/uv/bulkdata/STU1/export/index.html#bulk-data-delete-request'

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@305',
'hl7.fhir.uv.bulkdata_1.0.0@306'

input :smart_auth_info,
type: :auth_info,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ class BulkDataKickOffTest < Inferno::Test
DESCRIPTION
# link 'http://hl7.org/fhir/uv/bulkdata/STU1.0.1/export/index.html#response---success'

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@28',
'hl7.fhir.uv.bulkdata_1.0.0@300',
'hl7.fhir.uv.bulkdata_1.0.0@301'

input :smart_auth_info,
type: :auth_info,
options: { mode: 'access' },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class BulkDataExportOperationSupportTest < Inferno::Test
declares the standard bulk data export OperationDefinition provided in the
Bulk Data specification, nor does it attempt to resolve any non-standard
OperationDefinitions to verify if it is a constrained version of the
standard OperationDefintion.
standard OperationDefinition.

This test will provide a warning if no operations are declared via the
`CapabilityStatement.rest.resource.operation.name` element. It will
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ class BulkDataNDJSONDownloadTest < Inferno::Test
DESCRIPTION
# link 'http://hl7.org/fhir/uv/bulkdata/STU1.0.1/export/index.html#file-request'

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@201'

input :bulk_download_url
input :requires_access_token
input :smart_auth_info,
Expand Down
2 changes: 2 additions & 0 deletions lib/bulk_data_test_kit/v1.0.1/bulk_data_no_auth_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ class BulkDataExportNoAuthRejectTest < Inferno::Test
DESCRIPTION
# link 'http://hl7.org/fhir/uv/bulkdata/STU1.0.1/export/index.html#bulk-data-kick-off-request'

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@27'

input :smart_auth_info,
type: :auth_info,
options: { mode: 'access' },
Expand Down
8 changes: 8 additions & 0 deletions lib/bulk_data_test_kit/v1.0.1/bulk_data_output_check_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,14 @@ class BulkDataOutputCheckTest < Inferno::Test
DESCRIPTION
# link 'http://hl7.org/fhir/uv/bulkdata/STU1.0.1/export/index.html#response---complete-status'

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@164',
'hl7.fhir.uv.bulkdata_1.0.0@165',
'hl7.fhir.uv.bulkdata_1.0.0@166',
'hl7.fhir.uv.bulkdata_1.0.0@168',
'hl7.fhir.uv.bulkdata_1.0.0@169',
'hl7.fhir.uv.bulkdata_1.0.0@170',
'hl7.fhir.uv.bulkdata_1.0.0@175'

input :status_response

output :status_output, :bulk_download_url
Expand Down
17 changes: 17 additions & 0 deletions lib/bulk_data_test_kit/v1.0.1/bulk_data_status_check_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,23 @@ class BulkDataStatusCheckTest < Inferno::Test
DESCRIPTION
# link 'http://hl7.org/fhir/uv/bulkdata/STU1.0.1/export/index.html#bulk-data-status-request'

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@132',
'hl7.fhir.uv.bulkdata_1.0.0@144',
'hl7.fhir.uv.bulkdata_1.0.0@145',
'hl7.fhir.uv.bulkdata_1.0.0@147',
'hl7.fhir.uv.bulkdata_1.0.0@150',
'hl7.fhir.uv.bulkdata_1.0.0@151',
'hl7.fhir.uv.bulkdata_1.0.0@152',
'hl7.fhir.uv.bulkdata_1.0.0@155',
'hl7.fhir.uv.bulkdata_1.0.0@156',
'hl7.fhir.uv.bulkdata_1.0.0@157',
'hl7.fhir.uv.bulkdata_1.0.0@159',
'hl7.fhir.uv.bulkdata_1.0.0@160',
'hl7.fhir.uv.bulkdata_1.0.0@161',
'hl7.fhir.uv.bulkdata_1.0.0@191',
'hl7.fhir.uv.bulkdata_1.0.0@192',
'hl7.fhir.uv.bulkdata_1.0.0@193'

input :polling_url, :bulk_timeout
input :smart_auth_info,
type: :auth_info,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ class BulkDataValidResourcesTest < Inferno::Test
number of error messages it will display to 20.
DESCRIPTION

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@207',
'hl7.fhir.uv.bulkdata_1.0.0@208',
'hl7.fhir.uv.bulkdata_1.0.0@209'

input :smart_auth_info, type: :auth_info
input :status_output
input :requires_access_token
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ class BulkDataGroupExportGroup < Inferno::TestGroup
description <<~DESCRIPTION
Verify that group level export on the Bulk Data server follow the Bulk Data Access Implementation Guide
DESCRIPTION

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@224',
'hl7.fhir.uv.bulkdata_1.0.0@249'

id :bulk_data_group_export_group

input :smart_auth_info,
Expand Down Expand Up @@ -54,7 +58,7 @@ class BulkDataGroupExportGroup < Inferno::TestGroup
declares the standard Group export OperationDefinition provided in the
Bulk Data specification, nor does it attempt to resolve any non-standard
OperationDefinitions to verify if it is a constrained version of the
standard OperationDefintion.
standard OperationDefinition.

This test will provide a warning if no operations are declared at
`Group/[group_id]/$export`, via the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ class BulkDataPatientExportGroup < Inferno::TestGroup
description <<~DESCRIPTION
Verify that patient level export on the Bulk Data server follow the Bulk Data Access Implementation Guide
DESCRIPTION

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@272'

id :bulk_data_patient_export_group

input :smart_auth_info,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ class BulkDataSystemExportGroup < Inferno::TestGroup
description <<~DESCRIPTION
Verify that system level export on the Bulk Data server follow the Bulk Data Access Implementation Guide
DESCRIPTION

verifies_requirements 'hl7.fhir.uv.bulkdata_1.0.0@229'

id :bulk_data_system_export_group

input :smart_auth_info,
Expand Down
4 changes: 4 additions & 0 deletions lib/bulk_data_test_kit/v2.0.0/bulk_data_export_cancel_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ class BulkDataExportCancelTest < Inferno::Test
http://hl7.org/fhir/uv/bulkdata/STU2/export.html#bulk-data-delete-request
DESCRIPTION

verifies_requirements 'hl7.fhir.uv.bulkdata_2.0.0@122',
'hl7.fhir.uv.bulkdata_2.0.0@308',
'hl7.fhir.uv.bulkdata_2.0.0@309'

id :bulk_data_export_cancel_stu2

input :cancelled_polling_url
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ module BulkDataTestKit
module BulkDataV200
class BulkDataGroupOutputFormatParamTest < BulkDataV200::BulkDataOutputFormatParamTest
id :output_format_in_group_export_response
verifies_requirements 'hl7.fhir.uv.bulkdata_2.0.0@47',
'hl7.fhir.uv.bulkdata_2.0.0@49',
'hl7.fhir.uv.bulkdata_2.0.0@51',
'hl7.fhir.uv.bulkdata_2.0.0@52',
'hl7.fhir.uv.bulkdata_2.0.0@251'

input :group_id

config(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ module BulkDataTestKit
module BulkDataV200
class BulkDataGroupSinceParamTest < BulkDataV200::BulkDataSinceParamTest
id :since_in_group_export_response

verifies_requirements 'hl7.fhir.uv.bulkdata_2.0.0@53','hl7.fhir.uv.bulkdata_2.0.0@254'

input :group_id

config(
Expand Down
4 changes: 3 additions & 1 deletion lib/bulk_data_test_kit/v2.0.0/bulk_data_since_param_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ class BulkDataSinceParamTest < Inferno::Test
This test verifies that the server accepts an export request with the
`[_since](http://hl7.org/fhir/uv/bulkdata/STU2/export.html#query-parameters)`
query parameter. It initiates a new export using a _since parameter of
one week ago, and ensures that the export was initiated succesfully.
one week ago, and ensures that the export was initiated successfully.

The test does not attempt to verify that resources returned were
modified after the _since date that was requested, because the Bulk Data
Expand All @@ -27,6 +27,8 @@ class BulkDataSinceParamTest < Inferno::Test
After the export was successfully initiated, it is then cancelled.
DESCRIPTION

verifies_requirements 'hl7.fhir.uv.bulkdata_2.0.0@53'

input :smart_auth_info,
type: :auth_info,
options: { mode: 'access' },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class BulkDataSmartBackendServicesV200Group < Inferno::TestGroup
}

group from: :backend_services_authorization,
verifies_requirements: ['hl7.fhir.uv.bulkdata_2.0.0@2', 'hl7.fhir.uv.bulkdata_2.0.0@19'],
config: {
inputs: {
url: { name: :bulk_server_url },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ class BulkDataPatientExportParameters < Inferno::TestGroup
id: :output_format_in_patient_export_response,
config: {
options: { resource_type: 'Patient', bulk_export_url: 'Patient/$export' }
}
},
verifies_requirements: ['hl7.fhir.uv.bulkdata_2.0.0@47','hl7.fhir.uv.bulkdata_2.0.0@49','hl7.fhir.uv.bulkdata_2.0.0@51','hl7.fhir.uv.bulkdata_2.0.0@52','hl7.fhir.uv.bulkdata_2.0.0@274']

test from: :since_in_export_response,
id: :since_in_patient_export_response,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,15 @@ class BulkDataSystemExportParameters < Inferno::TestGroup
id: :output_format_in_system_export_response,
config: {
options: { resource_type: 'system', bulk_export_url: '$export' }
}
},
verifies_requirements: ['hl7.fhir.uv.bulkdata_2.0.0@47','hl7.fhir.uv.bulkdata_2.0.0@49','hl7.fhir.uv.bulkdata_2.0.0@51','hl7.fhir.uv.bulkdata_2.0.0@52','hl7.fhir.uv.bulkdata_2.0.0@231']

test from: :since_in_export_response,
id: :since_in_system_export_response,
config: {
options: { resource_type: 'system', bulk_export_url: '$export' }
}
},
verifies_requirements: ['hl7.fhir.uv.bulkdata_2.0.0@53','hl7.fhir.uv.bulkdata_2.0.0@234']
end
end
end
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ class BulkClientDataAuthVerification < Inferno::TestGroup
title 'Review Authentication Interactions'
run_as_group

test from: :bulk_data_client_token_smart_bsca_verification
test from: :bulk_data_client_token_smart_bsca_verification,
verifies_requirements: ['hl7.fhir.uv.bulkdata_2.0.0@15', 'hl7.fhir.uv.bulkdata_2.0.0@16']
test from: :smart_client_token_use_verification,
config: {
options: { access_request_tags: [KICKOFF_TAG, STATUS_TAG, OUTPUT_TAG, DELETE_TAG] }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ class DeleteTest < Inferno::Test

id :bulk_data_client_delete

verifies_requirements 'hl7.fhir.uv.bulkdata_2.0.0@119'

run do
assert load_tagged_requests(DELETE_TAG).any?, fail_message
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ class OutputTest < Inferno::Test

id :bulk_data_client_output

verifies_requirements 'hl7.fhir.uv.bulkdata_2.0.0@200'

run do
assert load_tagged_requests(OUTPUT_TAG).any?, fail_message
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ class StatusTest < Inferno::Test

id :bulk_data_client_status

verifies_requirements 'hl7.fhir.uv.bulkdata_2.0.0@123'

run do
assert load_tagged_requests(STATUS_TAG).any?, fail_message
end
Expand Down
9 changes: 6 additions & 3 deletions lib/requirements_config.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
test_kit_id: bulk-data-test-kit

suites:
# - id: bulk_data_v101
# class_name: BulkDataTestKit::BulkDataV101::BulkDataTestSuite
# suite_actor: Server
- id: bulk_data_v101
class_name: BulkDataTestKit::BulkDataV101::BulkDataTestSuite
suite_actor: Server
- id: bulk_data_v200
class_name: BulkDataTestKit::BulkDataV200::BulkDataTestSuite
suite_actor: Server
- id: bulk_data_v200_client
class_name: BulkDataTestKit::BulkDataV200Client::BulkDataClientTestSuite
suite_actor: Client

requirement_sets:
- id: hl7.fhir.uv.bulkdata_2.0.0
Expand Down