Question about cla module

[jayLeo456]

Hello, this is a good project.
I would like to ask a question about cla module. cla module modifies several self-attention layers in up_blocks, why not modify all self-attention layers?

[YujieOuO]

Thanks for your attention.

The CLA module aims to stabilize the generation of the background lighting source. We have conducted some experiments, and find applying CLA on Upblock 1 and 2 helps to achieve a stable light source while avoiding excessive temporal smoothing.
Previous works like MotionClone and FreeControl also only extract control signals from the first few layers of the upblock.
Of course, you can also try different combinations of layers, which should yield some differences in effect.