Welcome to Open-LLM-Leaderboard-Report Discussions!

[dsdanielpark]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[dsdanielpark]

We welcome discussions on the analysis of updates in the following revision. If there are any inaccuracies or inappropriate sections, please point them out for correction and we will incorporate the changes. Alternatively, if you initiate a new discussion, we can continue engaging in an interesting conversation on that topic.

https://github.com/dsdanielpark/Open-LLM-Leaderboard-Report/blob/main/REVISION.md

[RonanKMcGovern]

Thanks for putting this together.

Is MPT-30B here? I don't see it, but maybe it's under some other name? Any reason not to include it?

[dsdanielpark]

Thank you for your interest and questions!

For the MPT-7B model, it can be found in the Hugging Face repository, but it appears that the evaluation for the 30B model has not been completed yet. Furthermore, the visualization analysis in this repository is based on the top 60 models (which can be adjusted in config.py) according to their average scores.

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

[RonanKMcGovern]

k, will be great to get 30B evaluated because it fits on a GPU and has big context extendible by alibi.

[dsdanielpark]

I hope so.
Once the evaluation is completed and if it ranks high on the leaderboard, you will be able to see it on this repository as well.

Please stay tuned and check for updates!