Background:
Some runtimes (like AWS Lambda) are time limited in their execution for client code. To perform data ETL for SO-RAD from these runtimes it is necessary to page the client side and be able to control the number of results and quit before all data is fetched and resume from a bookmark at a later time.
Current behaviour:
All data is collated into a list and returned from the results API even when count is set to a fixed number.
Expect behaviour:
Count or an additional parameter send to the get_wfs function allows paging and returns a fixed number of results to allow higher level code to perform paging outside the library as discussed in the background.
Background:
Some runtimes (like AWS Lambda) are time limited in their execution for client code. To perform data ETL for SO-RAD from these runtimes it is necessary to page the client side and be able to control the number of results and quit before all data is fetched and resume from a bookmark at a later time.
Current behaviour:
All data is collated into a list and returned from the results API even when count is set to a fixed number.
Expect behaviour:
Count or an additional parameter send to the
get_wfsfunction allows paging and returns a fixed number of results to allow higher level code to perform paging outside the library as discussed in the background.