Code

GRPO RL for Reasoning#500

Code

Closed

SahilJain314 wants to merge 17 commits intoNVIDIA:mainNVIDIA/NeMo-Aligner:mainfrom SahilJain314:math_rlSahilJain314/NeMo-Aligner:math_rlCopy head branch name to clipboard

+2,552-68

Commits

Commits on Dec 18, 2024

• Added star and grader client

  SahilJain314

  committed
  202c23b

  Copy full SHA for 202c23b

Commits on Dec 30, 2024

• Functional RLOO math training

  SahilJain314

  committed
  d7add3a

  Copy full SHA for d7add3a

• Added actor model

  SahilJain314

  committed
  75e8bc3

  Copy full SHA for 75e8bc3

• adding length penalty WIP

  SahilJain314

  committed
  1de56e0

  Copy full SHA for 1de56e0

Commits on Jan 2, 2025

• Added support for communication collectives on rollout batches with strings.

  Show description for a0d1b68
  SahilJain314

  committed
  a0d1b68

  Copy full SHA for a0d1b68

Commits on Jan 6, 2025

• Wroking improving validation rewards on MATH, llama3

  SahilJain314

  committed
  52d1024

  Copy full SHA for 52d1024

Commits on Jan 10, 2025

• Added unsolved rate

  SahilJain314

  committed
  190ca8d

  Copy full SHA for 190ca8d

Commits on Jan 17, 2025

• Add online filtering for prompts based on the accuracy of the current policy

  parthchadha

  committed
  af04715

  Copy full SHA for af04715

• Fix usage of prompt_mask in masking

  parthchadha

  committed
  656e1a5

  Copy full SHA for 656e1a5

• Merge pull request #1 from SahilJain314/online-filtering-for-rollouts

  Show description for 2938273
  SahilJain314

  authored
  2938273

  Copy full SHA for 2938273

• Added fast generation logging and baseline disable flag

  SahilJain314

  committed
  d1bf125

  Copy full SHA for d1bf125

Commits on Jan 21, 2025

• Merge pull request #2 from SahilJain314/fast_generation_log

  Show description for 4695648
  SahilJain314

  authored
  4695648

  Copy full SHA for 4695648

Commits on Jan 23, 2025

• Added GRPO loss form to REINFORCE trainer

  SahilJain314

  committed
  793a5e0

  Copy full SHA for 793a5e0

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 27c4935
  

  pre-commit-ci[bot]

  authored and

  SahilJain314

  committed
  27c4935

  Copy full SHA for 27c4935

• bugfix in GRPO config

  SahilJain314

  committed
  b6196fd

  Copy full SHA for b6196fd

• Fixed PP for 70B

  SahilJain314

  committed
  8c47fd0

  Copy full SHA for 8c47fd0

Commits on Feb 4, 2025

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 3d12947
  pre-commit-ci[bot]

  committed
  3d12947

  Copy full SHA for 3d12947