Book · O’Reilly
AI Engineering
Chip HuyenBuilding applications with foundation models: evaluation, prompt engineering, RAG, agents, latency, and cost trade-offs.
Working library
Read, build,
verify
Books, open guides, standards, and benchmarks for designing and evaluating AI coding agents.
Suggest a resourceA working shelf rather than an endless link list—from agent-system design to reproducible result verification.
Books and engineering guides
Open book · GitHub
深入理解 AI Agent
Bo Jie LiAn open book on AI-agent design: planning, memory, tools, multi-agent systems, and engineering practice.
Practical repository
Everything Claude Code
Affaan MustafaSkills, security practices, memory, and research-first workflows for Claude Code, Codex, Cursor, and other harnesses.
Survey · arXiv
The Landscape of Emerging AI Agent Architectures
Masterman et al.A survey of agent architectures for reasoning, planning, and tool calling—from simple chains to autonomous organizations—with open challenges like verification and reproducible evaluation.
Paper · arXiv
OpenCodeInterpreter
Zheng et al.Research on code-generation systems with a closed loop of compiler and execution feedback, enabling iterative refinement and verification of generated code.
Architecture and standards
Framework · GitHub
LangGraph
LangChainAn industrial framework for building cyclic, controllable multi-agent systems with routing, state, and checkpointing for agent-based development.
Framework · GitHub
Microsoft AutoGen
MicrosoftA foundational ecosystem and standard for orchestrating, collaborating, and conversing across multiple AI agents in a shared workflow.
Framework · GitHub
LlamaIndex Workflows
LlamaIndexAn event-driven architecture for decomposing heavy AI tasks and strictly controlling context handoffs between steps and agents.
Documentation · OpenAI
Agents guide
OpenAIOfficial guidance for agents, tools, orchestration, tracing, and safety controls.
Open standard
Model Context Protocol
MCPThe protocol specification for connecting AI applications to data, tools, and external systems.
Coding-agent evaluation
Article · Towards Dev
AI Agents Are Failing at Their Most Important Test
ArXiv In-depth AnalysisAnalysis of the WideSearch benchmark (ByteDance): top agents from Google, OpenAI, and Anthropic score near 0% on broad information-seeking tasks. Why reliability and recall fail at scale, and why multi-agent collaboration beats single agents.
Benchmark · GitHub
SWE-bench
Princeton NLPReal GitHub issues and repositories for testing whether models can resolve software-engineering tasks with reproducible evaluation.
Leaderboard · Aider
Aider LLM Leaderboard
AiderThe leading practical leaderboard for how well models edit complex repositories from instructions, judged on strict git diffs and tests.
Toolkit · GitHub
SWE-agent & mini-SWE-agent
Princeton NLPAn open-source toolkit and environment that turns base LLMs into full software engineers through specialized agent-computer interfaces.
Benchmark · GitHub
LiveCodeBench
LiveCodeBenchAn open, continuously updated benchmark that guards against test-data leakage, checking generation, self-repair, and execution on fresh tasks.
Benchmark · GitHub
BigCodeBench
BigCode ProjectA benchmark for testing AI agents on following compound instructions and working with many third-party libraries.
VOLY methodology
How we verify ‘done’
VOLY ResearchOur metrics, compared architectures, and protocol from agent report to accepted change.