Code

[WIP] Make FastDEQs fast again#45

Code

Merged

avik-pal merged 76 commits intomainSciML/DeepEquilibriumNetworks.jl:mainfrom ap/eflSciML/DeepEquilibriumNetworks.jl:ap/eflCopy head branch name to clipboardJun 6, 2022

+2,952-1,709

Commits

Commits on Apr 9, 2022

• move towards efl

  avik-pal

  committed
  d593c41

  Copy full SHA for d593c41

• Some model code

  avik-pal

  committed
  a032635

  Copy full SHA for a032635

Commits on Apr 10, 2022

• Update model

  avik-pal

  committed
  8ec596d

  Copy full SHA for 8ec596d

• Update examples

  avik-pal

  committed
  c6fd5e5

  Copy full SHA for c6fd5e5

• add cifar10 training script

  avik-pal

  committed
  9b6c74c

  Copy full SHA for 9b6c74c

• Add dates dep

  avik-pal

  committed
  cbc2adf

  Copy full SHA for cbc2adf

• Fix training

  avik-pal

  committed
  7f7312c

  Copy full SHA for 7f7312c

• RIP ordering

  avik-pal

  committed
  2c296a1

  Copy full SHA for 2c296a1

• Some printing

  avik-pal

  committed
  98a4003

  Copy full SHA for 98a4003

• Some printing

  avik-pal

  committed
  b3a6a08

  Copy full SHA for b3a6a08

• Stop printing

  avik-pal

  committed
  47f9ea3

  Copy full SHA for 47f9ea3

• missing acc

  avik-pal

  committed
  6ec4c7b

  Copy full SHA for 6ec4c7b

Commits on Apr 11, 2022

• Add distributed training

  avik-pal

  committed
  0844f88

  Copy full SHA for 0844f88

• Make float32

  avik-pal

  committed
  7f3273a

  Copy full SHA for 7f3273a

• HPC safe precompile

  avik-pal

  committed
  37f5c87

  Copy full SHA for 37f5c87

• Change solver from VCABM4 since it changes inputs to FP64

  avik-pal

  committed
  30da3c5

  Copy full SHA for 30da3c5

Commits on Apr 12, 2022

• ADAMW syncing not working for MPI

  avik-pal

  committed
  b3f301e

  Copy full SHA for b3f301e

• Reduce using MPI

  avik-pal

  committed
  ed4c40f

  Copy full SHA for ed4c40f

Commits on Apr 13, 2022

• Fixed depth for pretraining

  avik-pal

  committed
  e89b401

  Copy full SHA for e89b401

• Fixed depth testing

  avik-pal

  committed
  07634bc

  Copy full SHA for 07634bc

• Missing state

  avik-pal

  committed
  3968f1c

  Copy full SHA for 3968f1c

• Imagenet model configuration

  avik-pal

  committed
  a951f98

  Copy full SHA for a951f98

• Imagenet model configuration

  avik-pal

  committed
  005f35f

  Copy full SHA for 005f35f

• Update config

  avik-pal

  committed
  3745309

  Copy full SHA for 3745309

• Int divide

  avik-pal

  committed
  f67a776

  Copy full SHA for f67a776

• exptconfig

  avik-pal

  committed
  7a401be

  Copy full SHA for 7a401be

• Oops

  avik-pal

  committed
  e51a483

  Copy full SHA for e51a483

• Remove nothing

  avik-pal

  committed
  e3f411f

  Copy full SHA for e3f411f

• Warmup the pretraining mode

  avik-pal

  committed
  43fd1ac

  Copy full SHA for 43fd1ac

Commits on Apr 14, 2022

• updates

  avik-pal

  committed
  1b57511

  Copy full SHA for 1b57511

Commits on Apr 16, 2022

• Update training

  avik-pal

  committed
  b57102d

  Copy full SHA for b57102d

Commits on Apr 17, 2022

• Update defaults

  avik-pal

  committed
  8e84ae2

  Copy full SHA for 8e84ae2

Commits on Apr 18, 2022

• Run for different configurations

  avik-pal

  committed
  ab2afcd

  Copy full SHA for ab2afcd

• Fix model construction

  avik-pal

  committed
  a016807

  Copy full SHA for a016807

Commits on Apr 19, 2022

• Better scheduling

  avik-pal

  committed
  c3cb712

  Copy full SHA for c3cb712

Commits on Apr 20, 2022

• Dont track statistics

  avik-pal

  committed
  27703a5

  Copy full SHA for 27703a5

• Temp fix for keys error

  avik-pal

  committed
  ce8cddf

  Copy full SHA for ce8cddf

• Move to component arrays

  avik-pal

  committed
  d12347f

  Copy full SHA for d12347f

Commits on Apr 21, 2022

• Use named tuple

  avik-pal

  committed
  c019b20

  Copy full SHA for c019b20

• Add termination.jl

  avik-pal

  committed
  1ba6736

  Copy full SHA for 1ba6736

• extra in train

  avik-pal

  committed
  94f8fb8

  Copy full SHA for 94f8fb8

• Update config

  avik-pal

  committed
  5e45c4f

  Copy full SHA for 5e45c4f

Commits on Apr 23, 2022

• Modify architecture to make it GPU friendly

  avik-pal

  committed
  7934860

  Copy full SHA for 7934860

• Update

  avik-pal

  committed
  4953087

  Copy full SHA for 4953087

Commits on Apr 24, 2022

• Use pretraining

  avik-pal

  committed
  0aef672

  Copy full SHA for 0aef672

• Fix backpass for variational autoencoder

  avik-pal

  committed
  3a12e0a

  Copy full SHA for 3a12e0a

• Fix dep

  avik-pal

  committed
  9c2626d

  Copy full SHA for 9c2626d

Commits on May 3, 2022

• Partial update to Lux

  avik-pal

  committed
  0b6a927

  Copy full SHA for 0b6a927

• Update FastDEQExperiments

  avik-pal

  committed
  336a19a

  Copy full SHA for 336a19a

• Compat entries

  avik-pal

  committed
  cad992e

  Copy full SHA for cad992e

Commits on May 6, 2022

• Dep fixes

  avik-pal

  committed
  db98546

  Copy full SHA for db98546

• Dep fixes

  avik-pal

  committed
  19f2911

  Copy full SHA for 19f2911

Commits on May 12, 2022

• Update src

  avik-pal

  committed
  0eca98f

  Copy full SHA for 0eca98f

• Docs

  avik-pal

  committed
  25cea0f

  Copy full SHA for 25cea0f

• Docs

  avik-pal

  committed
  c0ed127

  Copy full SHA for c0ed127

• Docs

  avik-pal

  committed
  7b71612

  Copy full SHA for 7b71612

Commits on May 13, 2022

• make it val

  avik-pal

  committed
  ea24bfc

  Copy full SHA for ea24bfc

• Type Inference fixes

  avik-pal

  committed
  afc8b79

  Copy full SHA for afc8b79

Commits on May 14, 2022

• Cant typeassert for state

  avik-pal

  committed
  34517ea

  Copy full SHA for 34517ea

• Remove typeasserts

  avik-pal

  committed
  fdc460b

  Copy full SHA for fdc460b

Commits on May 16, 2022

• Update script

  avik-pal

  committed
  be6f72d

  Copy full SHA for be6f72d

Commits on May 17, 2022

• Pretraining

  avik-pal

  committed
  ff9462c

  Copy full SHA for ff9462c

• Config

  avik-pal

  committed
  57a7abb

  Copy full SHA for 57a7abb

Commits on May 18, 2022

• Fix data augmentation

  avik-pal

  committed
  513172a

  Copy full SHA for 513172a

Commits on May 23, 2022

• Wandb logging

  avik-pal

  committed
  d709846

  Copy full SHA for d709846

• modify normalization layers

  avik-pal

  committed
  50b879d

  Copy full SHA for 50b879d

Commits on May 24, 2022

• modify normalization layers

  avik-pal

  committed
  d709d7f

  Copy full SHA for d709d7f

Commits on May 26, 2022

• Modify model architecture

  avik-pal

  committed
  6897ec3

  Copy full SHA for 6897ec3

• Modify model architecture

  avik-pal

  committed
  4555e10

  Copy full SHA for 4555e10

• Use weight norm

  avik-pal

  committed
  9fa1c32

  Copy full SHA for 9fa1c32

• Update model

  avik-pal

  committed
  0ef3ffa

  Copy full SHA for 0ef3ffa

• Relax types

  avik-pal

  committed
  3df322d

  Copy full SHA for 3df322d

• Minor fix

  avik-pal

  committed
  1cd9a6e

  Copy full SHA for 1cd9a6e

• Make post_deq type-stable

  avik-pal

  committed
  edbb7a3

  Copy full SHA for edbb7a3

• No MPI for cifar10

  avik-pal

  committed
  a9af2e3

  Copy full SHA for a9af2e3

Commits on May 27, 2022

• Decay the weight of skip

  avik-pal

  committed
  7e42c90

  Copy full SHA for 7e42c90