RedPajama-Data-1T togethercomputer
RedPajama is a clean-room, fully open-source implementation of the LLaMa dataset.
- Type
- dataset
- Language
- en
- Downloads
- 1,939
- Likes
- 1,187
- Access
- public
- Files
- 0
Tags
- 预训练语料
- 文本数据集
- 多语言语料
- 大语言模型
- 代码数据
- NLP
- 深度学习
- 数据处理
README
--- task_categories: - text-generation language: - en pretty_name: Red Pajama 1T --- ### Getting Started The dataset consists of 2084 jsonl files. You can download the dataset using HuggingFace: ```python from datasets import load_dataset ds = load_dataset("togethercomputer/RedPajama-Data-1T") ``` …