Auditing visual vs. text embedding space isomorphism using Mantel/HKS

[joinsnipe]

Hi PixelRAG team,

First of all, congratulations on this amazing project! Implementing pixel-native visual RAG to preserve document layouts, tables, and charts is a massive leap forward.

We have been working on a similar challenge regarding document ingestion pipelines. In our open-source project.
Motor 15 (Geometric Audit)
[]https://github.com/joinsnipe/motor15-geometric-audit.

We use mathematical and spectral validation metrics (such as the Mantel Test, Wasserstein-1 distance on MST, and Heat Kernel Signature spectral correlation) to audit and certify the geometric and topological consistency between different embedding projection spaces.

Since PixelRAG retrieves documents visually, we believe it would be highly valuable to build a benchmark/validation tool that tests the metric isomorphism between:

1. The original text-based embedding space.
2. PixelRAG's visual/pixel-based embedding space.

This would mathematically certify that shifting from text to pixel-native representation preserves the geometric structure of semantic relationships without introducing metric distortions or latent space deformations.

Would you be open to a discussion or a contribution demonstrating how we can apply these geometric audit methods to PixelRAG's embedding spaces?

Best regards,
Rubén Abella / TRACE Team

[ZubairAli999]

To really make work well we need to create a Geometric Consistency and Retrieval Validation Framework for PixelRAG. This framework for PixelRAG will check if PixelRAG keeps the meaning of words and ideas when it changes documents from text to pictures.

The framework for PixelRAG will make two sets of pictures from the documents. One set will be made using a good text model and the other set will be made using PixelRAG. Then it will compare these two sets using methods like KNN Preservation and Trustworthiness to see if PixelRAG keeps the same relationships between words and ideas.

We also want to know if PixelRAG can really find the documents when we search for something. So the framework for PixelRAG will use Recall@K and Precision@K to test this.

At the end we will get a score that shows how well PixelRAG keeps the meaning of words and ideas. This score is called the Geometric Consistency Score for PixelRAG. We will also get a report that tells us if PixelRAG is working well.

If we add this framework to our work process every time we make a change to PixelRAG it will be checked to make sure it still works well. This will help us make sure that PixelRAG always keeps the meaning of words and ideas and can find the documents when we search for something. The Geometric Consistency and Retrieval Validation Framework for PixelRAG will be very important, for PixelRAG.