Replies: 1 comment
-
|
(Maybe new OWUI option resolve it, as it's bad idea to hack file names instead of fix an issue in OWUI. Anyway, it is critical that KB tools can see directories OR if we can tag files using their directory location} I have an example. We have a lot of files in Contracts directory, and they are organized by customer names. Files themselves have customer names, etc, too, but it was not always done this way plus search in directory is way faster vs search in the flat space. It can be very useful if system can navigate KB by the directories / or search files by names using customer name, and then only make RAG search using name template or TAG. And it is not just our problem, most data are organized in the file trees and when we index them, it is essential to maintain this information. Sure, we have some workaround (like creating directory lists manually and adding them into KB) but it all cause more and more context usage and context is already a bottleneck (as small KB of 3000 files use more than 50% of 256K contexts). |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
We see one issue with KB indexing. It do not add directory name into the file name. So we have directory in file system
Contracts/'Customer-Name'/'Customer-contracts'
As for now, OWUI create files with name 'Customer-contacts' and model can not see / use directory name in the file search (or in the index search).
It can be great if we have a way to compose file name in KB so it preserve directory names. I am not sure if it can be 'Directories'/'File' or 'Directories'--'File' or similar (for example we may want to add only one element here) but in any way, almost any solution which allows to preserve directory in KB will be great.
Just now, oikb creates directory structure in KB. But this structure is not visible to the model. SO if we have files in
ABC/'file-names'
model can not request _give me all files by the pattern ABC/.*
Of course, it can be solved on different level - make directory structure visible to the internal tools model can use to query KB. It will be even better. But it must be done in open-webui itself.
Beta Was this translation helpful? Give feedback.
All reactions