Adding Embedding to TensorSharp

[Rpgdudester]

Good morning,

I was the one who had posted on Reddit to see if Tensorsharp was going to support embedding models and you had asked me to put in a discussion or open an issue about it.

Currently what I am doing is I'm going through all my projects and embedding all my code files into a sqlite db so I can search through either I or my ai Kira can search the embedded codebase for certain things. It makes it a lot easier to do that than to grep the entire database looking for something.

the link to the reddit discussion is here:
Reddit Discussion

The model I am currently using for embedding is snowflake-arctic-embed-l-v2.0-q8_0 gguf.

Thank you so much!

[vvdb-architecture]

I'm a bit in the same situation as you, but I'm still delving into the TensorSharp code and learning by contributing to some minor PRs.

In the meantime, I'm using https://github.com/elbruno/elbruno.localembeddings/tree/main as a separate embedding vector computation, which is simple and fast enough for my needs: I use the CPU, so that TensorSharp can use Cuda.

I'm quite sure that the code to retrieve an embedding vector is not difficult, but I'd rather spend some time to increase my understanding of the evaluation path in TensorSharp for the completion case before embarking on the embedding adventure...

[zhongkaifu]

@Rpgdudester @vvdb-architecture Thanks for these ideas. I will prioritize it. Besides snowflake-arctic-embed-l-v2.0-q8_0 gguf, any other model you want to be supported ? I would like to add them into test cases.

[zhongkaifu]

@Rpgdudester @vvdb-architecture Here is a PR for embedding models: #213 I'm testing it now, and you could also test it if you like. Any feedback will be welcome.

[zhongkaifu]

@Rpgdudester @vvdb-architecture Text embeddings have been merged into main. You can check with this document to understand how to use it: https://github.com/zhongkaifu/TensorSharp/blob/main/docs/embeddings.md