Free for open source projects and schools. AI Council costs nothing for OSS maintainers and educational institutions. Apply →

Blog / AI development in practice

When CLAUDE.md stops helping

Almost every team that starts working with Claude Code, Codex or Cursor ends up in the same place: they create a rules file. CLAUDE.md, AGENTS.md, Cursor rules. It says how to run the tests, which libraries to avoid and what not to touch in the old module. For the first few weeks it works great.

The problems come later, and at first it isn't clear where they come from.

The file falls behind the code

A module moved to another folder, but the rule in the file still points to the old path. A decision was revisited in a meeting, and nobody updated the file. The agent dutifully follows an instruction that no longer matches the project, and the reviewer is the last to find out.

A rule in the file has no link to the code it's about. So nobody notices when it goes stale.

Everyone has their own file

On a team of eight, after a couple of months you have six versions of the rules: in different repositories, in personal settings, in notes. Different developers' agents behave differently, and the "how we do things here" argument moves into code review.

The agent reads everything at once

The file keeps growing. To fix one function, the agent loads all of it into context: rules for the frontend, for migrations, for the payments module. That costs tokens. Worse, the one rule that matters gets lost among the ones that don't.

Advice without enforcement

The file suggests, but it doesn't check. If the agent breaks a convention, you find out in review, which is the most expensive place to catch mistakes. According to CodeRabbit, AI-assisted pull requests get about 1.7 times more review comments than human-written ones. And LinearB data shows that agent PRs wait noticeably longer for review to start: reviewers don't trust that code until they've gone through it.

One version for every branch

When agents work on several tasks in parallel, a decision made in one branch goes straight into the shared file. Another task starts relying on something that hasn't been merged yet and may still be reverted.

What to do about it

If the project is small and one or two people work on it, a rules file is a fine solution. There's no need to make things more complicated.

As the team grows, your rules need four things a file doesn't have:

  • A link to specific modules and functions, so you can see when a rule is out of date.
  • One shared base for the team instead of personal files.
  • Delivery of only what's relevant to the current task.
  • Checks of changes against the rules before the merge request.

That's what we built in AI Council, and we also keep knowledge separate per git branch. You can keep your CLAUDE.md. Usually a short project description stays there, and the conventions move to the knowledge base as MUST and SHOULD rules.

Detailed comparison of AI Council and CLAUDE.md

Want to see it on your project?

Connect your repository and get the first review on your own code.

Made on
Tilda