V-JEPA, Expansion, License

[Sharpz7]

Hey, really like what you have done, but a few questions:

• I would be interested in collaborating with you on extending this repo to cover V-JEPA. Is that of interest?
• It would also be cool to extend the codebase to be easier to use (make training and inference as easy as possible for non-researchers). I would be happy to start this effort on my own.

My final concern is the license. Something like MIT is definitely ideal (See https://opensource.org/licenses), but I am not sure if you are allowed to put it under this license.

Reading the terms of the original I-JEPA license, (https://github.com/facebookresearch/ijepa?tab=License-1-ov-file#readme), I think you would need to put this code under the same license if you have shared the code. I am not sure about the case where you have just used the paper to build your own codebase.

My collaboration would be dependent on the license being MIT or equivalent, as the use I can make of anything else is limited (https://opensource.org/osd)

Thanks again for your efforts and I hope to hear from you soon ;))

[gaasher]

@Sharpz7, I would love to collaborate on transferring this to V-JEPA! I currently have a MIT license attached, since all the code is mine and I just transcribed what I read in the paper. In fact I released this code before any official implementation was released. What do you envision the final result for easy inference looking like? Also, I'm pretty busy right now so I don't have much time to work on this repo further. With that said - what can I do to help you work fastest/best?

[Sharpz7]

Hey! Thanks for the quick response, that is great!

I as well do not have much bandwidth right now, but I will track this for when I do because I think it would be a great contribution to the community.

Does your implementation include the conditional latent z variable from the JEPA architecture? I have struggled to find any references on what that actually is outside of the first H-JEPA Paper.

In terms of "easy inference" - I would love to build out the codebase to support using JEPA in different ways - from CLI, from an import etc, as well as having sensible defaults, easy data processing, and connections to things like tensorboard, W&B, MlFlow. These things, although small, would make it much easier to make cool things with JEPA as a software engineer with limited ML experience.

As well as this, documentation on how to make decoders, and again an easy way to "build" a decoder from this codebase would also be cool.

If you agree, maybe I could open a ticket about scoping all this work, into smaller "bite-sized" chunks?

[gaasher]

Sorry for the late reply - just started classes again for this term.

My implementation does not use the z variable. My understanding is this is just a stand-in to explain the fact that JEPA models don't learn a known probability distribution like generative models. Also your plan sounds good. We could even just work to integrate this with huggingface. Feel free to divvy this up into bite size tickets as you mentioned.