Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NFResNet model to models package #57

Open
zaleslaw opened this issue May 18, 2021 · 2 comments
Open

Add NFResNet model to models package #57

zaleslaw opened this issue May 18, 2021 · 2 comments
Assignees
Labels
research Research or reproducing code from science paper

Comments

@zaleslaw
Copy link
Collaborator

The new family and approach of BatchNorm-free NN architectures look very perspective due to the lack of BatchNorm training support.

In the paper "High-Performance Large-Scale Image Recognition Without Normalization" the new approach is proposed.

We should apply this approach to the traditional ResNet architecture to prevent gradient exploding without BatchNorm layers.

The pre-trained weights will be a plus but are not nessesary.

@zaleslaw zaleslaw added the research Research or reproducing code from science paper label May 18, 2021
@zaleslaw zaleslaw self-assigned this May 18, 2021
@zaleslaw zaleslaw added this to the 0.4 milestone Sep 13, 2021
@zaleslaw
Copy link
Collaborator Author

How to obtain pre-trained weights https://reposhub.com/python/deep-learning/benjs-nfnets_pytorch.html

@zaleslaw
Copy link
Collaborator Author

An example of pytorch implementation https://nfnets-pytorch.readthedocs.io/en/latest/

@zaleslaw zaleslaw modified the milestones: 0.4, 0.5 Dec 15, 2021
@ermolenkodev ermolenkodev removed this from the 0.5 milestone Sep 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
research Research or reproducing code from science paper
Projects
None yet
Development

No branches or pull requests

2 participants