Skip to content

Update OpenAlex journal sync: dedupe IDs, add error handling#197

Merged
nuest merged 2 commits into
mainfrom
feature/add-journal-data-5
Jul 15, 2025
Merged

Update OpenAlex journal sync: dedupe IDs, add error handling#197
nuest merged 2 commits into
mainfrom
feature/add-journal-data-5

Conversation

@BharatVe

@BharatVe BharatVe commented Jul 8, 2025

Copy link
Copy Markdown
Collaborator

Resolve AttributeError: 'str' object has no attribute 'get':

  • Split full OpenAlex URI into openalex_url (full URI) and openalex_id (short ID) to avoid redundant data
  • Added works_api_url persistence
  • Wrapped per-source update logic in try/except to prevent one failure from stopping the entire run

openalex_id = full_id.rsplit("/", 1)[-1] if isinstance(full_id, str) else None

# update fields
safe_upd("openalex_id", openalex_id)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am happy that you check if there actually was a change.

But you can check for changes first, and then if you find a single change, then you update the whole record. You could try doing that via the hash, for example.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored the loop to collect all changed fields into an updates dict and issue one Source.objects.filter(pk=…).update(**updates) call per record

Comment thread publications/management/commands/update_openalex_journals.py Outdated
@nuest nuest merged commit e13a902 into main Jul 15, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants