NatureBench JoeLiu996
Dataset Card for NatureBench NatureBench is a cross-discipline benchmark of 27 tasks distilled from peer-reviewed Nature-family publications, spanning 6 scientific domains. It is designed to evaluate whether AI coding agents can move beyond reproduction toward discovery: each task asks an agent to solve a real scientific machine-learning problem and is scored against the source paper's reported state of the art. 📄 arXiv paper: https://arxiv.org/abs/2606.24530 💻 GitHub code… See the full description on the dataset page: https://huggingface.co/datasets/JoeLiu996/NatureBench.
- 类型
- dataset
- 许可
- other
- 语言
- en
- 下载量
- 255,741
- 点赞
- 0
- 访问
- public
- 文件
- 0
标签
- 评测基准
- Agent
- 代码生成
- 科学数据
- 机器学习
- 多任务
- 测试数据
摘要
NatureBench 是一个跨学科评测基准，从 Nature 系列期刊论文中提炼出 27 个真实科学机器学习任务，涵盖细胞组学、蛋白质生物学、生物医学建模、物理建模、分子设计与关系推理等6大科学领域。它用 SOTA 归一化相对差距来评测 AI 编码智能体（coding agents）能否超越复现、走向科学发现，通过隔离容器与有效性审查机制防止取巧解。适用于衡量大型语言模型驱动的编码智能体在真实科研问题上的推理与解决能力。
README
--- language: - en license: other license_name: mit-with-third-party-data license_link: LICENSE pretty_name: NatureBench size_categories: - n<1K tags: - coding-agents - benchmark - scientific-machine-learning - nature configs: - config_name: default data_files: - split: train path: manifest.jsonl -…