Troops from the IDF’s 98th Division operating in Jabalia, the northern Gaza Strip, May 2024. Photo: Israel Defense Forces.

The IDF’s current mechanism for determining and providing intelligence assessments rigidifies the thinking of intelligence analysts, and increases the risk that they will not recognize changes in the behavioral patterns of the adversary that affect the predictive ability and relevance of the forecast. Providing assessments should be the beginning of the process, not the end. A structured, open, and ongoing discussion of an assessment can make both the process and the product more dynamic in the face of changing conditions.

The current intelligence assessment process increases the risk of forecast error

At the core of intelligence assessment stands the forecast. If the adversary’s behavioral dynamics form a linear regression — that is, they adhere to familiar patterns — then it is possible to draw a straight line from the present to the future and score high prediction levels (90% or better). This applies to most current and tactical engagements of intelligence. Assessment based on familiar patterns, combined with reasonable risk analysis, makes intelligence an effective support tool for political and operational decision-making.

Difficulties arise when the adversary’s behavior diverges from the expected. In this situation, the ability to predict decreases significantly; and the risk grows that the intelligence forecast will lead to decisions that do not correspond to reality. In such a situation, decision-makers must rely to a greater degree on risk management.

It is a serious problem when familiar patterns are disrupted and can no longer serve as a basis for assessment. To make matters worse, limitations on human thinking make it difficult for analysts to identify this problem when it arises. This increases the risk that intelligence predictions will actually serve the enemy, because our side will make decisions based on those predictions that the enemy is prepared to disrupt, circumvent, or even exploit.

Once an intelligence assessment has been formulated, it becomes an inflexible entity. Predictions about the future become a fixed reality (a “concept”) about the present. After the assessment is given to the client (the decision-maker, the commander), it becomes even more fixed among intelligence analysts. This is in order to prevent situations that are perceived as unprofessional, including changing the assessment as a result of dynamics with the client or frequently updating the assessment in a way that makes it difficult for the decision-maker to form policy decisions.

Because intelligence assessments tend to set like concrete after they have been turned over to the client, intelligence analysts generally shift their focus to the implications of the assessment for intelligence gathering, assimilating the assessment among the clients, and providing recommendations for policy and action stemming from it. All this activity diverts focus away from the implications of the assessment for the intelligence assessment process itself.

The problem can be summarized as follows: The mechanism for determining and providing intelligence assessments rigidifies the thinking of intelligence analysts, and increases the risk that they will not recognize changes in the behavioral patterns of the adversary that affect the predictive ability and relevance of the forecast.

A solution: Assessment as an ongoing process

To overcome this problem, the providing of an assessment should be viewed as the beginning of the process, not the end. To paraphrase Eisenhower on military planning, “Assessment is everything and nothing”. Intelligence analysts should remain within the assessment process even after providing it to the client.

What would this mean in practice? It would mean examining the assessment against many parameters and continuing to do so systematically, even after the assessment has been submitted. Discussions about the assessment should be open, transparent, and structured for the participation of both the intelligence analysts and the clients so that gaps in the forecast can be identified early.

Parameters for the ongoing review of an assessment can include:

1. Verification and validity: These are recognized existing parameters within which the analyst examines whether developments with the subject strengthen or weaken the assessment. As we have seen many times, these criteria are the first victims of cognitive biases that make it difficult for the analyst to detach himself from the assessment.
2. Evaluation context: What circumstances underlying the prediction are connected not to the adversary but to other circles, like the adversary’s partners, regional factors, international factors, and the internal context? Variations in the broader circumstances can affect the validity of the evaluation, even if no apparent change is identified in the subject.
3. Assessment levels: If a strategic early warning has been issued, what are its implications for operational and tactical early warnings, and how does the sequence of forecasting change between levels? Does a sequence of tactical early warnings indicate a strategic early warning? Are the actions of the adversary consistent with its policy? What is the meaning of continuity or lack of continuity in the approach between different levels?
4. Projection: This refers to the projection of an assessment of a particular issue onto assessments of other issues and topics. For example, is the predicted behavior of an adversary likely to affect the behavior of one of his allies? Is the predicted weakening of an adversary likely to lead to a deeper and broader change in other circles?
5. Impact of responding to opportunities and risks: If the assessment indicates an opportunity, what are the implications of acting or not acting in response to it? Does inaction in response to the opportunity influence the forecast? And conversely, with regard to risks: If the assessment indicates a risk, is it a self-fulfilling prophecy? In other words, would caution in the face of the risk lead to its realization?
6. Meaning of continuity or change: If the forecast indicates continuity, what are the indications if change in fact occurs? If the forecast indicates change, what needs to happen so that change does not materialize?
7. The price of error: How does the cost of a prediction error affect the validity of the assessment? Isn’t the validity of a prediction that excludes extreme scenarios (military attack, nuclear weapons proliferation, regime collapse) weakened by the heavy cost of an error?
8. Use of the assessment: What is the intelligence assessment being used for? If it is not used, or if, in the intelligence analysts’ view, its use is contrary to its content, what does that say about the assessment itself and its logic and clarity?
9. Realization: If the prediction comes to pass, why did it do so? Did this show that the estimate was correct, or was it merely a coincidence? Conversely, does the failure of a prediction necessarily indicate that the estimate was incorrect?

Continuous engagement in the assessment process will improve its quality

A structured, open, and ongoing discussion of an assessment during and after its construction can free intelligence analysts from fixations, create more dynamism in the process and product, and improve and refine the product over time and in the face of changing conditions. This takes broader contexts than the adversary himself into account. It is not about changing estimates due to external influences. Rather, it is about developing a more comprehensive, broad, and rich view of the act of intelligence assessment.

The products of such an improvement could be more accurate and nuanced assessments, constant examination of the validity of assessments, continuous engagement with lower probability scenarios and not neglecting them after an assessment has been submitted, and expanding the potential for identifying problems in assessments as a result of more time spent on them.

Col. (res.) Shay Shabtai is a senior researcher at the BESA Center and an expert in national security, strategic planning, and strategic communication. He is a cyber security strategist and a consultant to leading companies in Israel. A version of this article was originally published by The BESA Center.