The Three Foundations of Reliable Evaluation

Domain knowledge is the first foundation. You have to know enough about the subject to know when something is wrong. This seems obvious, but it's often overlooked. Many people manage or oversee work in areas where they have less expertise than they think. They can't evaluate because they can't tell the difference between something that's technically sound and something that's plausible but wrong. They rely on the executor to know the field, and they hope for the best. This is management without direction.

If you're directing work, you need real domain knowledge. Not necessarily at the level of someone who could execute the work themselves, but enough to catch major errors, to recognize when something is incomplete or logically inconsistent, to know what the hard problems in the domain actually are. If you don't have this knowledge, you can develop it — by reading, by asking questions, by working with experts. But you have to do the work.

Standards are the second foundation. You have to have thought about what good means in this context. What does a good analysis look like? What does a good design look like? What does a good document look like? These standards can be implicit when you're executing — you know good work when you see it. But when you're directing, you have to make them explicit. You have to be able to articulate what you're looking for.

Good standards are specific but not prescriptive. They name what matters without dictating how to achieve it. For a presentation: "The argument builds logically from evidence to conclusion, and the conclusion is supported by the specific data presented" is a useful standard. "Use bullet points and talk slowly" is not. Standards name the quality you want. They let the executor find their own path to that quality.

Practice is the third foundation. Evaluation is a skill that improves with use. The more you evaluate, the better you get at it — but only if you evaluate honestly and correct yourself. If you evaluate carelessly and never follow up, you don't improve. If you evaluate harshly and never examine your own standards, you don't improve either. Real practice means: evaluate the work, look carefully at what was actually produced, compare it to what you expected, and adjust your understanding.