鹈鹕骑车:同一句话测模型结构还原
同一句话喂所有模型,比较谁能画对自行车的结构和鹈鹕的喙——社区用它给新模型做体检。
做出来什么
一份汇集各模型生成 Pelican riding a bicycle SVG 的对比清单。
安装
把 simonw/pelican-bicycle 的 SKILL.md(上游没给可定位的 commit) 装到 ~/.claude/skills/pelican:上游没有可定位的 commit,装最新版后把版本记下来,装完把目录树发我核对。
无来源元数据(没有 commit 与许可)。一句话交给 agent,它会补齐这一步。上游原文
# Pelicans on a bicycle For more on this benchmark check out my talk [The last six months in LLMs, illustrated by pelicans on bicycles](https://simonwillison.net/2025/Jun/6/six-months-in-llms/) or review the [pelican-riding-a-bicycle tag](https://simonwillison.net/tags/pelican-riding-a-bicycle/) on my blog. Prompt: > `Generate an SVG of a pelican riding a bicycle` ## Cerebras Llama models ### Cerebras Llama 3.1 70B <img src="cerebras-llama3.1-70b.svg" width="600" alt="Cerebras Llama 3.1 70B"> ### Cerebras Llama 3.1 8B <img src="cerebras-llama3.1-8b.svg" width="600" alt="Cerebras Llama 3.1 8B"> ## Claude models ### Claude 3.5 Sonnet (2024-06-20) <img src="claude-3-5-sonnet-20240620.svg" width="600" alt="Claude 3.5 Sonnet (2024-06-20)"> ### Claude 3.5 Sonnet (2024-10-22) <img src="claude-3-5-sonnet-20241022.svg" width="600" alt="Claude 3.5 Sonnet (2024-10-22)"> ### Claude 3 Haiku <img src="claude-3-haiku-20240307.svg" width="600" alt="Claude 3 Haiku"> ### Claude 3 Opus <img src="claude-3-opus-20240229.svg" width="600" alt="Claude 3 Opus"> ## Gemini models ### Gemini 1.5 Flash 001 <img src="gemini-1.5-flash-001.svg" width="600" alt="Gemini 1.5 Flash 001"> ### Gemini 1.5 Flash 002 <img src="gemini-1.5-flash-002.svg" width="600" alt="Gemini 1.5 Flash 002"> ### Gemini 1.5 Flash 8B 001 <img src="gemini-1.5-flash-8b-001.svg" width="600" alt="Gemini 1.5 Flash 8B 001"> ### Gemini 1.5 Pro 001 <img src="gemini-1.5-pro-001.svg" width="600" alt="Gemini 1.5 Pro 001"> ### Gemini 1.5 Pro 002 <img src="gemini-1.5-pro-002.svg" width="600" alt="Gemini 1.5 Pro 002"> ### gemini-exp-1114 <img src="gemini-exp-1114.svg" width="600" alt="gemini-exp-1114"> ### gemini-exp-1121 <img src="gemini-exp-1121.svg" width="600" alt="gemini-exp-1121"> ### gemini-exp-1206 <img src="gemini-exp-1206.svg" width="600" alt="gemini-exp-1206"> ## OpenAI models ### GPT-3.5 Turbo <img src="gpt-3.5-turbo.svg" width="600" alt="GPT-3.5 Turbo"> ### GPT-4o mini <img src="gpt-4o-mini.svg" width="600" alt="GPT-4O Mini"> ### GPT-4o <img src="gpt-4o.svg" width="600" alt="GPT-4O"> ### o1-mini <img src="o1-mini.svg" width="600" alt="O1 Mini"> ### o1-preview <img src="o1-preview.svg" width="600" alt="O1 Preview"> ## AWS models ### Amazon Nova Micro <img src="us.amazon.nova-micro-v1-0.svg" width="600" alt="Amazon Nova Micro"> ### Amazon Nova Lite <img src="us.amazon.nova-lite-v1-0.svg" width="600" alt="Amazon Nova Lite"> ### Amazon Nova Pro <img src="us.amazon.nova-pro-v1-0.svg" width="600" alt="Amazon Nova Pro">