Train roberta-base from scratch for Nepali language using the CC-100 subset.
To start the training, run:
python train.pyThe default configuration used is stored at config/default.yaml. You can also view all the configuration options using the --help command.
python train.py --helpYou can override any configuration from the CLI using the hydra syntax. For example, to train using only 100 sentences for 1 epoch, run:
python train.py dataset.portion=100 model.epochs=1Our model has been featured in the following papers:
- Nepali Transformers@NLU of Devanagari Script Languages 2025: Detection of Language, Hate Speech and Targets (Khadka et al., CHiPSAL 2025)
- Pande, Bishal Debb, et al. "Named Entity Recognition for Nepali Using BERT Based Models." International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Cham: Springer Nature Switzerland, 2023.
- Niraula, Nobal, and Jeevan Chapagain. "DanfeNER-Named Entity Recognition in Nepali Tweets." The International FLAIRS Conference Proceedings. Vol. 36. 2023.
- Timilsina, Sulav, Milan Gautam, and Binod Bhattarai. "NepBERTa: Nepali language model trained in a large corpus." Proceedings of the 2nd conference of the Asia-Pacific chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing. Association for Computational Linguistics (ACL), 2022.
- Tamrakar, Suyogya Ratna, and Chaklam Silpasuwanchai. "Comparative Evaluation of Transformer-Based Nepali Language Models." (2022).