Persistent op store

[maackle]

I'm considering trying the latest version of kitsune. It looks like there is still only the in-memory op store. I was able to hack some persistence together previously by just dumping the memory store's state to disk after every change but obviously this is not sustainable. Any plans yet to include a basic persisted op store in kitsune, or is there an implementation in holochain or elsewhere that could be easily reused? It's tempting to reach for kitsune, but it seems not quite ready to be used in a real project.

If not I'll consider this the tracking issue for it and if you ever do it, let me know here!

[jost-s]

Hi @maackle! The in-memory op store is only for testing. One aspect of not implementing a persisted op store is that the schema of the ops is unknown. It would just store byte blobs. But the byte blobs are unlikely to be usable by the actual host application, which would rather use columns with names and types. If kitsune implemented an op store based on byte blobs, all data would be stored redundantly.

How do you hope an op store in kitsune to work?

[maackle]

It seemed from the conversation in #277 that there was a possibility of some kind of reusable persistent op store being made available at some point, and since that PR is closed I just wanted to have an open issue to keep that possibility open. If there is no possibility, then this can be closed too.

One aspect of not implementing a persisted op store is that the schema of the ops is unknown

Yes, I would hope that a reusable op store would be generic over the op type via a trait, so that the implementor can use whatever structure they need, while kitsune can know only what it needs to know about each op (timestamp, hash, etc.)

[jost-s]

This time around we're convinced ;-)

Seriously, we think it's a reasonable request and are going to prioritize an implementation of a persistent op store over the coming weeks.