Prompt Evals Pack

Small evaluation sets for comparing prompt quality, cost and failure modes.

Compatibility87

Build Stack With This Pack

Compatible Models

GPT-5.5
Claude
DeepSeek
Qwen

Inside The Pack

rules
eval templates
scoring rubric

Included Workflows

Task fixture set
Rubric scoring
Cost notes
Regression comparison

Install Skill Pack

Download through the server-checked manifest API, run the installer from this project root, then restart Codex.

Download JSON

Install command

node scripts/install-skill-pack.mjs ./prompt-evals.skillpack.json

Codex skills install into $CODEX_HOME/skills. The installer does not overwrite existing skill folders.

{
  "schemaVersion": 1,
  "kind": "artificial-search.skill-pack",
  "id": "prompt-evals",
  "name": "Prompt Evals Pack",
  "summary": "Small evaluation sets for comparing prompt quality, cost and failure modes.",
  "tier": "Pro",
  "compatibilityScore": 87,
  "notice": "This installs Codex skill files from the pack manifest. MCP servers still require a separate verified mcp.json configuration.",
  "install": {
    "target": "$CODEX_HOME/skills",
    "command": "node scripts/install-skill-pack.mjs ./prompt-evals.skillpack.json",
    "steps": [
      "Download this manifest as prompt-evals.skillpack.json.",
      "Run the installer from the Artificial Search project root.",
      "Restart Codex after installation so the new skills are loaded."
    ]
  },
  "skills": [
    {
      "id": "prompt-evals-rules",
      "name": "Prompt Evals Pack Rules",
      "description": "Rules skill for Prompt Evals Pack. Small evaluation sets for comparing prompt quality, cost and failure modes.",
      "files": [
        {
          "path": "SKILL.md",
          "content": "---\nname: prompt-evals-rules\ndescription: \"Rules skill for Prompt Evals Pack. Small evaluation sets for comparing prompt quality, cost and failure modes.\"\n---\n\n# Prompt Evals Pack Rules\n\nUse this skill when a task matches the Prompt Evals Pack pack.\n\n## Pack Summary\n\nSmall evaluation sets for comparing prompt quality, cost and failure modes.\n\n## Compatible Models\n\n- GPT-5.5\n- Claude\n- DeepSeek\n- Qwen\n\n## Pack Contents\n\n- rules\n- eval templates\n- scoring rubric\n\n## Operating Rules\n\n- Use the pack scope before choosing a model, MCP server or prompt chain.\n- Keep generated outputs tied to explicit user intent and validated project context.\n- Do not expose secrets, private files or credentials through MCP tools.\n"
        }
      ]
    },
    {
      "id": "prompt-evals-workflow",
      "name": "Prompt Evals Pack Workflow",
      "description": "Workflow skill for Prompt Evals Pack. Small evaluation sets for comparing prompt quality, cost and failure modes.",
      "files": [
        {
          "path": "SKILL.md",
          "content": "---\nname: prompt-evals-workflow\ndescription: \"Workflow skill for Prompt Evals Pack. Small evaluation sets for comparing prompt quality, cost and failure modes.\"\n---\n\n# Prompt Evals Pack Workflow\n\nUse this skill when a task matches the Prompt Evals Pack pack.\n\n## Pack Summary\n\nSmall evaluation sets for comparing prompt quality, cost and failure modes.\n\n## Compatible Models\n\n- GPT-5.5\n- Claude\n- DeepSeek\n- Qwen\n\n## Pack Contents\n\n- rules\n- eval templates\n- scoring rubric\n\n## Operating Rules\n\n- Task fixture set\n- Rubric scoring\n- Cost notes\n- Regression comparison\n"
        }
      ]
    },
    {
      "id": "prompt-evals-validation",
      "name": "Prompt Evals Pack Validation",
      "description": "Validation skill for Prompt Evals Pack. Small evaluation sets for comparing prompt quality, cost and failure modes.",
      "files": [
        {
          "path": "SKILL.md",
          "content": "---\nname: prompt-evals-validation\ndescription: \"Validation skill for Prompt Evals Pack. Small evaluation sets for comparing prompt quality, cost and failure modes.\"\n---\n\n# Prompt Evals Pack Validation\n\nUse this skill when a task matches the Prompt Evals Pack pack.\n\n## Pack Summary\n\nSmall evaluation sets for comparing prompt quality, cost and failure modes.\n\n## Compatible Models\n\n- GPT-5.5\n- Claude\n- DeepSeek\n- Qwen\n\n## Pack Contents\n\n- rules\n- eval templates\n- scoring rubric\n\n## Operating Rules\n\n- Check the recommended model and agent against the task constraints.\n- Verify every MCP server permission before using the generated config.\n- Run the relevant lint, build, smoke or review gate before delivery.\n"
        }
      ]
    }
  ]
}

How to use it

Pick the model and agent that match the task pressure: coding, documents, automation or private local work.

Attach only the MCP servers needed for that workflow and verify permissions before exposing files or secrets.

Run the pack checklist before delivery so prompts, rules and validation steps stay consistent.