On Judgment, Reality, and the Limits of Verification

1. Correctness Is Not the Same as Judgment Quality

A decision can be reasonable at the time it is made and still lead to an unexpected outcome.

The world contains uncertainties that cannot always be anticipated. A decision may be based on the available evidence, a sound understanding of the situation, and a reasonable assessment of risk, yet still be affected by external events or consequences that were not foreseeable.

The outcome alone does not tell us whether the original judgment was good.

We need to distinguish the quality of a decision at the time it was made from the result it produced, and from the ability to reconsider that decision when new evidence emerges.

The goal is not to guarantee correctness. It is to improve the quality of judgment under uncertainty.

2. Does a Model Have to Be Complete?

A model does not have to contain everything to be useful.

People in earlier periods worked with less knowledge and fewer tools than we have today. That does not mean their decisions were necessarily defective. They had to make choices using the information, resources and capabilities available to them.

If completeness were the standard for usefulness, every model we use today could be dismissed by a future generation.

The more relevant question is whether a model is suitable for the task and conditions in which it is being used.

An omission matters when it affects the decision at hand. Not every omission is a flaw, and not every limitation prevents a model from being useful.

3. Verification Within a World of Assumptions

Verification can establish whether a model behaves consistently within a defined set of rules and assumptions.

But consistency within a model does not automatically establish that those assumptions accurately represent the world.

A business decision-maker may use a model to estimate demand, allocate resources or evaluate possible outcomes. The model can provide a useful direction, but its assumptions must still be considered against the actual conditions, available resources and constraints of the business.

A reasonable decision does not require a model to account for everything. It requires understanding what the model can support, what remains uncertain, and when the decision needs to change.

The boundary of verification is not necessarily the boundary of reality.

4. A Model Can Be Useful Before It Becomes Better

An earlier model can still be useful even when a newer version offers greater accuracy or more capabilities.

Early versions of ChatGPT, for example, were less capable than the systems available today. Yet people could already use them to explore ideas, solve problems and complete meaningful tasks.

Later improvements do not erase the value those earlier systems provided.

A model should be evaluated in relation to what it could reasonably support at the time, not only by comparing it with what becomes possible later.

Improvement expands what a model can do. It does not automatically invalidate what came before.

5. The Difference Between Using a Model and Replacing Reality With It

A model can provide a starting point, a reference or a proposed direction.

But applying it in the real world requires attention to the conditions in which the decision will actually operate.

A plan that works under assumed conditions may need adjustment when resources are limited, circumstances change or new evidence becomes available.

Using a model means drawing on what it can offer while retaining the ability to assess its relevance.

Replacing reality with a model means treating its assumptions as if they were the conditions themselves.

These are not the same thing.

6. What Should Change When Reality Disagrees?

When reality diverges from a model, the first response should not automatically be to reject the model or to force reality to fit it.

The disagreement is information.

It may reveal that an assumption needs revision, that a relevant condition was not considered, or that the original model remains useful but requires a different application.

Sometimes the decision needs to change. Sometimes the model needs to change. Sometimes both need to be reconsidered.

The important question is not simply whether the model was wrong, but what the difference reveals about the decision, the model and the conditions in which both were used.

Judgment includes the ability to revise.

7. What Are We Actually Trying to Improve?

If the objective is to eliminate every possible error, the standard may be impossible to satisfy.

No model can anticipate every future event, and no decision can be separated entirely from uncertainty.

But judgment can improve.

We can become more careful about the assumptions we rely on, more aware of the limits of our information, and more willing to revise a decision when circumstances change.

The value of a model is not measured only by how closely it approaches completeness. It is also measured by whether it helps us make better decisions within the conditions that actually exist.

Enough for What?

A model does not need to be complete to be useful.

A decision does not need to produce the desired outcome to have been reasonable.

And a model does not become useless simply because reality eventually demands something different from it.

The question is not whether a model is enough in the abstract.

It is whether it is enough for the task, the conditions and the decision at hand — and whether we can recognise when that is no longer the case.

Case 68 | 模型，何時才算足夠？

關於判斷、現實與驗證的邊界

1. 正確的結果，不等於高品質的判斷

一個決定在作出當下可能是合理的，最後卻仍然得到意料之外的結果。

現實世界存在許多無法預先掌握的不確定性。一個決定可以根據當時可用的證據、對情況的合理理解，以及對風險的評估而作出，卻仍然受到外部事件或不可預見的後果影響。

單憑結果，無法判斷當初的決策品質。

我們需要區分三件事：作出決定當下的判斷品質、決定最終產生的結果，以及新證據出現後重新審視決定的能力。

我們追求的不是保證正確，而是在不確定性之中改善判斷的品質。

2. 模型一定要完整嗎？

模型不必包含一切，才能發揮作用。

過去的人所掌握的知識與工具，比今天少得多。這不代表他們當時的決策必然有缺陷。他們同樣需要運用當時可用的資訊、資源與能力，解決眼前的問題。

如果完整才是有用的標準，那麼今天使用的每一個模型，都可能被未來的一代以同樣理由否定。

更重要的問題，是模型是否適合當下的任務與使用條件。

當某項遺漏影響了眼前的決策，它才成為需要處理的問題。並非所有遺漏都是缺陷，也不是每一項限制都會令模型失去價值。

3. 在充滿假設的世界裡，驗證能證明什麼？

驗證可以確認一個模型，在既定規則與假設之內，是否保持一致。

但模型內部的一致性，並不能自動證明這些假設準確反映了現實。

商業決策者可以利用模型估算需求、分配資源，或評估不同結果的可能性。模型可以提供有用的方向，但仍然需要把它的假設，放回企業實際面對的環境、可用資源與限制之中考量。

合理的決策不要求模型解釋一切，而是需要理解模型能支持什麼、仍有哪些不確定性，以及何時需要改變決策。

驗證的邊界，不一定就是現實的邊界。

4. 模型可以在變得更好之前，先發揮作用

即使新版本的模型更準確、能力更強，較早期的模型仍然可能具有實際價值。

例如，早期的 ChatGPT 不如今天的系統成熟，但人們當時已經可以用它探索想法、解決問題，並完成有意義的工作。

後來的改進，不會抹去早期系統曾經提供的價值。

評估一個模型，應該考慮它在當時能夠合理支持什麼，而不只是拿它與後來出現的能力比較。

改進會擴大模型能做到的事情，但不會自動否定過去已經做到的事情。

5. 使用模型，與用模型取代現實

模型可以提供起點、參考依據，或一個建議的方向。

但要把它應用於現實，就必須考慮決策真正運作的環境與條件。

一個在假設條件下成立的計劃，當資源有限、情況改變或出現新證據時，可能就需要調整。

使用模型，是運用它能提供的價值，同時保留判斷其適用性的能力。

用模型取代現實，則是把模型的假設當成現實本身。

兩者並不相同。

6. 當現實與模型不一致時，應該改變什麼？

當現實與模型出現偏差，第一個反應不應該自動是拒絕模型，也不應該強迫現實符合模型。

這種不一致本身就是資訊。

它可能表示某項假設需要修正、某個重要條件尚未被考慮，也可能表示原有模型仍然有用，只是需要換一種方式應用。

有時需要改變的是決策，有時是模型，有時兩者都需要重新審視。

重要的問題不只是模型是否出錯，而是這個差異揭示了什麼：關於決策本身、模型的適用範圍，以及兩者所處的現實條件。

判斷也包括修正的能力。

7. 我們真正想改善的是什麼？

如果目標是消除所有可能的錯誤，這個標準可能根本無法達成。

沒有任何模型能預見所有未來事件，也沒有任何決策能完全脫離不確定性。

但判斷的品質可以改善。

我們可以更謹慎地看待自己依賴的假設，更清楚地理解資訊的限制，也更願意在環境改變時重新審視決定。

模型的價值，不只取決於它距離完整有多近，也取決於它能否幫助我們在真實存在的條件下，作出更好的決策。

對什麼而言，才算足夠？

模型不必完整，才能發揮作用。

決策不必得到預期的結果，當初的判斷才算合理。

模型也不會因為現實最終要求不同的做法，就自動失去價值。

問題不在於模型抽象而言是否足夠。

而在於，對眼前的任務、條件與決策而言，它是否足夠——以及當它不再足夠時，我們能否察覺。