openai_humaneval openai

Dataset Card for OpenAI HumanEval Dataset Summary The HumanEval dataset released by OpenAI includes 164 programming problems with a function sig- nature, docstring, body, and several unit tests. They were handwritten to ensure not to be included in the training set of code generation models. Supported Tasks and Leaderboards Languages The programming problems are written in Python and contain English natural text in comments and docstrings.… See the full description on the dataset page: https://huggingface.co/datasets/openai/openai_humaneval.

类型
dataset
许可
mit
语言
en
下载量
291,534
点赞
401
访问
public
文件
0

标签

  • 评测基准
  • 代码数据
  • 代码生成
  • 文本生成

摘要

OpenAI HumanEval 是由 OpenAI 发布的代码生成评测数据集，包含164道手写的 Python 编程问题，每道题配有函数签名、文档字符串、规范解法和单元测试。该数据集专门用于评估代码生成模型的正确性，设计时刻意避免出现在代码生成模型的训练集中。它是衡量大语言模型代码生成能力的行业标准评测基准。

README

--- annotations_creators: - expert-generated language_creators: - expert-generated language: - en license: - mit multilinguality: - monolingual size_categories: - n<1K source_datasets: - original task_categories: - text2text-generation task_ids: [] paperswithcode_id: humaneval pretty_name: OpenAI H…

查看完整页面 · 查看原文