New Issue - Unable to get all model objects from Snipe-IT instance

[kwalshPPU]

Grok,

I haven't set up any scheduled runs of this yet, so I was just manually running the task. The same one I ran yesterday (multiple times, as I was testing something), is now showing me this error and nothing else helpful from --verbose or --debug

Any idea whats broken here?

[kwalshPPU]

I found the issue - I had a Windows script running that was errantly creating duplicate models, so my model list grew out of control. I can only assume the API didn't like looking through that amount of models and failed. After I deleted the duplicates and got down to ~300 models rather than 700+, it is running again.

[jerm]

Someone just submitted a PR for adding pagination support to kandji2snipe, which is based on this codebase. Once I've tested and merged grokability/kandji2snipe#13, I'll port it over here, and that should solve this sort of issue going forward.

[guruevi]

I have too many models for Snipe-IT to return. I fixed it with my updated code: #119

[mattcarras]

I just ran into this issue in our own snipe-it instance, which recently got up to 501 models. Since the 2nd pull in get_snipe_models() is missing the offset parameter, it ends up pulling down the first 500 rows again before throwing an error:

logging.info("We didn't get enough results so we need to get them again.")
api_url = '{}/api/v1/models?limit={}'.format(snipe_base, jsonresponse['total'])

I think this is intended to be:

logging.info("We didn't get enough results so we need to get them again.")
api_url = '{}/api/v1/models?offset={}'.format(snipe_base, len(jsonresponse['rows']))

Ideally the recursive solution already used in get_snipe_users() would be better though.

Edit:

Turns out the original code does not work either, only returning the offset rows. I ended up adapting the code from get_snipe_users(), which seems to work for our environments:

# Recursive function to get all the asset models in a Snipe Instance.
def get_snipe_models(previous=[]):
    api_url = '{}/api/v1/models'.format(snipe_base)
    payload = {
        'offset': len(previous)
    }
    logging.debug('Calling against: {}'.format(api_url))
    response = session.get(api_url, headers=snipeheaders, params=payload, verify=user_args.do_not_verify_ssl, hooks={'response': request_handler})
    if response.status_code == 200:
        jsonresponse = response.json()
        logging.info("Got a valid response of {} rows that should have {} models.".format(len(jsonresponse['rows']),jsonresponse['total']))
        current = jsonresponse['rows']
        if len(previous) != 0:
            current = current + previous
        if jsonresponse['total'] > len(current):
            logging.debug('We have more than {} models, get the next page - current: {}'.format(jsonresponse['total'], len(current)))
            return get_snipe_models(current)
        else:
            return current
    else:
        logging.error('When we tried to retreive a list of models, Snipe-IT responded with error status code:{} - {}'.format(response.status_code, response.content))
        raise SystemExit("Snipe models API endpoint failed.")

Minor change is needed in the main loop to remove the ['rows'] reference:

# Get a list of known models from Snipe
logging.info("Getting a list of computer models that snipe knows about.")
snipemodels = get_snipe_models()
logging.debug("Parsing the {} model results for models with model numbers.".format(len(snipemodels)))
modelnumbers = {}
for model in snipemodels:
    if model['model_number'] == "":
        logging.debug("The model, {}, did not have a model number. Skipping.".format(model['name']))
        continue
    modelnumbers[model['model_number']] =  model['id']
logging.info("Our list of models has {} entries.".format(len(modelnumbers)))
logging.debug("Here's the list of the {} models and their id's that we were able to collect:\n{}".format(len(modelnumbers), modelnumbers))

I'll work on making a pull request after testing it some more.