Skip to content

Deduping nanomem import for large directories #4

@cadentj

Description

@cadentj

Hi! Thanks for making this tool, I've been playing around with it today and it's pretty cool.

When getting started with this repo, I find myself calling nanomem import <dir> to load memories from my Obsidian vault or other personal notes.

Then, I guess the intended use case is to never do this again. nanomem becomes a tool for whatever agent I'm interacting with to manipulate memories manually with nanomem add / retrieve.

Curious what you think of a slightly different use case where I (the human) am still actively contributing to the Obsidian vault and running nanomem add is annoying to do with every file edit. In this case, it would be nice to be able to call nanomem import after I'm done editing all my files.

However, nanomem import is wasteful since there's no deduplication on files that haven't changed. Is it within the scope of this project to add better caching?

If yes, would you be open to a PR improving nanomem import? One proposal is to hash all files in a directory on import and skip files whose hashes match a stored value. Happy to discuss the design before writing code.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions