mbpp google-research-datasets
Dataset Card for Mostly Basic Python Problems (mbpp) Dataset Summary The benchmark consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in the paper, a subset of the data has been hand-verified by us. Released here as part of… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/mbpp.
- 类型
- dataset
- 许可
- cc-by-4.0
- 语言
- en
- 下载量
- 243,239
- 点赞
- 235
- 访问
- public
- 文件
- 0
标签
- 评测基准
- 代码数据
- 代码生成
- 数据处理
- 测试数据
摘要
MBPP是Google发布的Python基础编程问题评测基准，包含约1000个由众包生成、面向入门程序员的编程题目，每个题目含任务描述、标准代码解答和3个自动化测试用例。数据集提供full与sanitized两个版本，主要用于评测大语言模型的代码生成能力，适合用于代码生成、程序综合等任务的基准测试与模型评估。
README
--- annotations_creators: - crowdsourced - expert-generated language_creators: - crowdsourced - expert-generated language: - en license: - cc-by-4.0 multilinguality: - monolingual size_categories: - n<1K source_datasets: - original task_categories: - text2text-generation task_ids: [] pretty_name: M…