Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalization will not work #3

Open
zhangtemplar opened this issue Nov 25, 2019 · 5 comments
Open

Normalization will not work #3

zhangtemplar opened this issue Nov 25, 2019 · 5 comments

Comments

@zhangtemplar
Copy link

I tested it and the normalization will not working:

       for W in self.fc.parameters():
            W = F.normalize(W, p=2, dim=1)
@gregunz
Copy link

gregunz commented Dec 17, 2019

This should fix it:

for _, module in self.fc.named_modules():
    if isinstance(module, nn.Linear):
        module.weight.data = F.normalize(module.weight, p=2, dim=1)

@bnu-wangxun
Copy link

bnu-wangxun commented Oct 25, 2020

This should fix it:

for _, module in self.fc.named_modules():
    if isinstance(module, nn.Linear):
        module.weight.data = F.normalize(module.weight, p=2, dim=1)

thanks

@JisongXie
Copy link

I find this problem, too. I modify it like this:

self.fc.weight.data = F.normalize(self.fc.weight.data, p=2, dim=1)

@Charlyo
Copy link

Charlyo commented Oct 13, 2021

@cvqluu can you fix it in the code? thanks

@xdn-github
Copy link

xdn-github commented Dec 20, 2024

I think use weight normalize layer may be a more simlple way:
in line 35 use
self.fc = weight_norm(nn.Linear(in_features, out_features, bias=False), name='weight')
instead of
self.fc = nn.Linear(in_features, out_features, bias=False)
and delete

       for W in self.fc.parameters():
            W = F.normalize(W, p=2, dim=1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants