Hi, thank you for the great work!
I believe there may be a small discrepancy between Eq. 7 in the paper and the actual implementation in mamba_vision.py.
The paper states the final linear as $\text{Linear}(\frac{C}{2}, C)$, where $\frac{C}{2}$ is $C_{in}$ and $C$ is $C_{out}$. However, each branch outputs a $\frac{C}{2}$ tensor, and after concatenation, the resulting tensor has dimension $C$. This makes $C_{in} = C$, not $\frac{C}{2}$, which is consistent with the code:
# d_inner = self.expand * d_model = 1 * d_model
self.out_proj = nn.Linear(self.d_inner, self.d_model)
Please correct me if I am wrong, the paper may have a small typo in Eq. 7, where $\text{Linear}(\frac{C}{2}, C)$ should read $\text{Linear}(C, C)$.
Thank you for your time!
Hi, thank you for the great work!
I believe there may be a small discrepancy between Eq. 7 in the paper and the actual implementation in
mamba_vision.py.The paper states the final linear as$\text{Linear}(\frac{C}{2}, C)$ , where $\frac{C}{2}$ is $C_{in}$ and $C$ is $C_{out}$ . However, each branch outputs a $\frac{C}{2}$ tensor, and after concatenation, the resulting tensor has dimension $C$ . This makes $C_{in} = C$ , not $\frac{C}{2}$ , which is consistent with the code:
Please correct me if I am wrong, the paper may have a small typo in Eq. 7, where$\text{Linear}(\frac{C}{2}, C)$ should read $\text{Linear}(C, C)$ .
Thank you for your time!