# MCP tool-use eval

- Commit `868524f259`, gpt-6.1-sol at low effort, user instructions drop, 1 samples per case, template `codex.json.gz`
- T3 context: system text 7163 chars (~1791 tokens), tool descriptions 15011 chars (~3753 tokens), full tool schemas 48085 chars (~12021 tokens) across 71 tools.
- Pass: 100% of 1 samples.
- Cost: not applicable (subscription through proxy).
- Agent tokens: input 16397, output 260, cacheWrite 0, cacheRead 31104.
- Shell simulator (claude-haiku-5-5) tokens: input 2, output 2746, cacheWrite 27717, cacheRead 0 (API-price estimate $0.00).
- Code-mode tools use case mocks and a simulated machine. Native subagent prompts and asynchronous notifications are approximate.

## Cases

| Group | Pass |
| --- | --- |
| NODEV-02 | 100% of 1 |

## By kind

| Group | Pass |
| --- | --- |
| control | 100% of 1 |

## By pattern

| Group | Pass |
| --- | --- |
| repo question needs no device or SSH | 100% of 1 |
# MCP tool-use eval

- Commit `c0a65526e8`, gpt-6.1-sol at low effort, user instructions drop, 1 samples per case, template `codex.json.gz`
- T3 context: system text 7139 chars (~1785 tokens), tool descriptions 14620 chars (~3655 tokens), full tool schemas 47144 chars (~11786 tokens) across 69 tools.
- Pass: 100% of 1 samples.
- Cost: the agent is not priced. `vibeproxy` in `~/.codex/config.toml` may use a subscription or bill an API key per token; agent tokens are below.
- Agent tokens: input 4536, output 312, cacheWrite 0, cacheRead 43392.
- Shell simulator (claude-haiku-5-5) tokens: input 2, output 3230, cacheWrite 24630, cacheRead 3316 (API-price estimate $0.00).
- Code-mode tools use case mocks and a simulated machine. Native subagent prompts and asynchronous notifications are approximate.

## Cases

| Group | Pass |
| --- | --- |
| NODEV-02 | 100% of 1 |

## By kind

| Group | Pass |
| --- | --- |
| control | 100% of 1 |

## By pattern

| Group | Pass |
| --- | --- |
| repo question needs no device or SSH | 100% of 1 |
# MCP tool-use eval

- Commit `c0a65526e8`, gpt-6.1-sol at low effort, user instructions drop, 1 samples per case, template `codex.json.gz`
- T3 context: system text 7139 chars (~1785 tokens), tool descriptions 14620 chars (~3655 tokens), full tool schemas 47144 chars (~11786 tokens) across 69 tools.
- Pass: 100% of 1 samples.
- Cost: the agent is not priced. `vibeproxy` in `~/.codex/config.toml` may use a subscription or bill an API key per token; agent tokens are below.
- Agent tokens: input 901, output 266, cacheWrite 0, cacheRead 46592.
- Shell simulator (claude-haiku-5-5) tokens: input 2, output 1582, cacheWrite 24425, cacheRead 3316 (API-price estimate $0.00).
- Code-mode tools use case mocks and a simulated machine. Native subagent prompts and asynchronous notifications are approximate.

## Cases

| Group | Pass | Baseline | Delta |
| --- | --- | --- | --- |
| NODEV-02 | 100% of 1 | 100% | 0 pts |

## By kind

| Group | Pass |
| --- | --- |
| control | 100% of 1 |

## By pattern

| Group | Pass |
| --- | --- |
| repo question needs no device or SSH | 100% of 1 |