• 📖 Book Update

• 🎤 Upcoming Talks

• 📝 The Complete Guide to Cyber Risk Dashboard Design

📖 Book Update

Hey, it’s here! It’s in my hand! My book, From Heatmaps to Histograms, just arrived on my doorstep from Springer Nature. It’s an amazing feeling that I don’t really have words for. It’s been quite a journey, to say the least. In the book, I call risk measurement and quantitative analysis a “quiet revolution” of rational thought, so as it goes out into the world it feels “done” in a sense, but it also feels like a beginning. I keep coming back to this quote from T.S. Eliot, one of my favorite poets:

Here is where things stand with orders and shipping as of April 1:

• The book is available worldwide, anywhere books are sold, online and in local bookstores

• Available now via the publisher, Springer — ebook $29.99, print $39.99

• Available now via Barnes & Noble — ebook and print $39.99, I am hearing that orders through B&N have shipped

• Still on preorder via Amazon 🤷‍♂️ — print $39.99, showing an April 9 date, but I would imagine copies ship within a few days

Book signings and events are in the works, and I’ll share details as they come together.

As you get your copies, please send me pictures. I would love to post them on LinkedIn. If you leave a review, which helps enormously, let me know, and I will share it. And thank you to everyone who preordered.

🔗 Book website

🎤 Upcoming Talks

April 21, 2026 — “Did We Solve the Data Problem? Judgment, Beliefs, and Risk in the AI Age” SIRAcon 2026 — Keynote

I’m keynoting SIRAcon this year with a talk on why the real bottleneck in risk quantification was never data, it was judgment, and why AI makes that distinction more important than ever. SIRAcon remains my favorite conference because it is model-neutral, vendor-neutral, and entirely dedicated to advancing risk practices.

🔗 More info

📝 The Complete Guide to Cyber Risk Dashboard Design

This issue is a masterclass in building the ultimate cyber risk dashboard. I’m going to teach you how to build the example below, which has every critical element a risk team needs to present to executive leadership:

• A peer benchmarking figure from the IBM/Ponemon Cost of a Data Breach Report, a trusted source that gives you a perfect baseline for what a data breach would cost your organization

• An aggregated cyber risk rating across all risks and portfolios, so leadership has a single clear picture of where we stand

• Control coverage and open risk items, the operational metrics every board wants to see

• The centerpiece: a risk matrix showing your top risks for quick, easy decisions on where to focus mitigation efforts

• A 3D pie chart of top threats for fast visual recognition

• And much more….

ꜜ

April Fools.

I hope I didn’t give everyone a heart attack. You probably thought I lost my mind. The real topic of this issue is a walkthrough of very common data visualization and measurement techniques, how they can be misused or misinterpreted, and what the better alternatives look like.

What is entirely absent from this dashboard, before you look at any individual element, is any statement of purpose. There is no decision it is designed to support, no organizational objective it connects to, and no decision-relevant question a board member could answer after reading it that they could not have answered before. The dashboard is a collection of indicators assembled because indicators can be assembled, a snapshot of activity presented as “risk.” When a dashboard has no decision to serve, there is no principled way to ask whether any given metric belongs on it at all.

A dashboard that tells you what happened is a report. A dashboard that tells you what to do requires a decision at its center, and this one has none.

Hubbard defines measurement as a quantitatively expressed reduction in uncertainty based on one or more observations. By that standard, not one element on this dashboard qualifies as a measurement. Each one fails on at least one of four criteria: it lacks a defined construct, it has no unit, it is not reproducible across observers, or it has no demonstrable connection to the phenomenon it claims to represent.

Let’s dive in, one at a time.

① $4.44M: The Wrong Number from the Wrong Companies Measuring the Wrong Thing

I have seen this figure used two ways, and both are wrong.

1. Using it as a proxy for your projected breach cost: “if we have a breach, it will cost us $4.44M on average.”

2. Taking the per-record figure from the same report and multiplying it by the number of records at your organization to produce a “personalized” breach cost estimate. The report’s own FAQ section explicitly states that the per-record cost cannot be used this way, because the study only covers breaches up to 113,620 records, but people still do it. All the time. (Who reads the fine print, anyway?)

Neither approach is defensible, and you should not use Ponemon figures for either purpose. The methodological problems with this report could fill an entire issue (and I will probably do that at some point), but here are the three that matter most:

• The sample is non-scientific. The report surveys 600 organizations that experienced a breach, selected in a way that is, by the report’s own admission, “biased toward organizations with more mature privacy or information security programs.” Your organization may look nothing like that population, and because the sample is non-scientific, the results cannot be extrapolated to your organization.

• The cost data is estimated, not measured. Ponemon asks individuals to estimate their breach costs after the fact using activity-based extrapolation, and the report acknowledges this may introduce bias and inaccuracies. A survey is not the right instrument for gathering this kind of data.

• The headline figure conceals enormous variance. The US average in 2025 was $10.22M. Healthcare averaged $7.42M. Retail averaged $3.54M. Public sector averaged $2.86M. A single global average summarizing those figures tells you almost nothing useful about any of them, and averaging a distribution this wide is a flaw of averages in its own right.

For next time: If you need a benchmark figure, the Cyentia Institute’s IRIS 2025 report is built on more than 150,000 real cyber incidents spanning fifteen years of empirical loss data, segmented by sector and organization size. Unlike Ponemon, it is not survey-based. You can get a credible, defensible range calibrated to your revenue band and sector. Disclaimer: I am a Fellow at the Cyentia Institute, but I was advocating for this data long before that relationship existed.

② “Medium”: A Label That Cannot Be Computed

Think about ten hot sauces lined up on a table, ranging from almost ketchup to face-melting. You could rank them from mildest to hottest. What you could not do is average them into a single meaningful spiciness score, because the distance between each level is undefined. The gap between a Tabasco and a habanero is not the same as the gap between a habanero and a Carolina Reaper, and no amount of arithmetic on your ranking will tell you that.

This is exactly what happens when you take a portfolio of risks rated High, Medium, and Low and collapse them into a single aggregate rating. The “Overall Risk Rating: Medium” on this dashboard is not a measurement of anything. It is an artifact of an operation that should not have been performed. Ordinal scales encode rank order only, saying nothing about the distance between levels. You can sort them, but you cannot add them, average them, or combine them into a summary figure.

Then there is the label itself. Consider what a leader is supposed to do with the word Medium. There is no defined threshold, no probability, no magnitude, and no financial unit attached to it. Someone decided this was Medium rather than High or Low, and that decision implies a process, but the process almost always involves additional undefined terms: likelihood judged on a five-point scale, impact estimated against vague categories. Undefined inputs produce undefined outputs, and the label that emerges carries the weight of a decision without the substance of one.

The sub-label reads “Unchanged from Q4.” The absence of change is being presented as a finding worth reporting.

For next time: A label that connects to a decision tells a different story. “There is a 35% probability of a loss event exceeding our $5M retention threshold in the next 12 months” is a statement that means something. It tells you whether to buy more insurance, change your controls, or accept the exposure, and it gives leadership an actual basis for a conversation.

Borrowing from finance, a quantitative approach also opens the door to a genuine portfolio view of risk, one where the results of individual scenario analyses can be legitimately aggregated, because you are working with probabilities and loss ranges rather than ordinal labels. The math is valid in a way that averaging Highs and Mediums simply is not. That said, I do not personally recommend collapsing a risk portfolio into a single aggregate number either, for reasons I covered in a reader question a few issues back. The portfolio view is most useful when you can see the individual scenarios and their relative contributions, not when they disappear into another composite score.

③ 37 Open Risk Items: You Cannot Close the Future

Of everything on this dashboard, this one may be the hardest to let go of, even for practitioners who have been doing quantitative risk analysis for years. The open/close model is not a simple rookie mistake. It is a way of thinking that runs through incident response and almost every other operational process in security. It also feels good; it feels productive in a way that probabilistic thinking sometimes does not, because closing a ticket is a visible act of completion.

The problem is that it rests on a logical error. A risk is not a task. It is a statement about the future: given what we know about our environment, our controls, and the threat landscape, there is some probability that a loss event will occur. You cannot fix that and close it. You can change your controls, accept the exposure, transfer it to an insurer, or decide the scenario is outside your scope, but none of those actions make the future go away. The probability changes. The scenario does not disappear from the register because someone marked it resolved.

When a risk register behaves like a ticketing system, the incentive follows: teams work to close items rather than to reduce exposure. A risk that gets closed without any corresponding change in the probability or magnitude of loss is a risk that got administered, not managed. The count goes down, the dashboard looks better, and the organization’s actual exposure is unchanged.

The 37 is not a measurement of risk. It is a count of tickets.

For next time: A risk register entry should persist as long as the scenario is plausible, updated as the threat environment and your controls change. The number worth tracking is not how many items are open but whether the probability and expected loss for your top scenarios are moving in the right direction, and by how much.

④ 84% Control Coverage: Coverage of What, Exactly?

Coverage metrics are legitimate. Knowing whether a control is present is a valid input to vulnerability estimation, and FAIR practitioners use control presence as part of probabilistic risk analysis all the time. The interesting question is what coverage means when it is aggregated across a generic framework checklist and presented as a standalone indicator of risk posture.

At that level of abstraction, 84% cannot be converted to a probability, an expected loss, or a decision. The question it answers is “what percentage of framework line items are checked,” not “how much have we reduced the likelihood or magnitude of our top loss scenarios.” Those are different questions, and only the second one carries much weight in a board conversation.

A 16-point gap in a framework checklist could represent almost nothing, or it could represent the specific controls most relevant to your highest-consequence scenarios. The coverage percentage cannot tell you which, because it was never anchored to scenarios in the first place. Aggregate coverage against a checklist, with no scenario weighting, has no reliable relationship to scenario-specific loss reduction, and without that connection, the number has nowhere to go.

For next time: Anchoring coverage to scenarios changes the conversation entirely. “Our ransomware scenario controls are 94% in place, reducing expected annual loss for that scenario from $8.4M to $5.1M. Closing the remaining gap to 99+% would cost $600K and reduce expected loss by a further $1.2M” is a coverage metric that connects directly to a decision about where to invest next. The percentage is still there, and now it means something.

⑤ The Risk Matrix: A Math Crime Dressed in Traffic Light Colors

Look at this chart for a moment before reading anything else. It has a grid, coordinates, color-coded zones, and named threats plotted at specific positions. It looks exactly like something that was calculated. That appearance is the problem.

The risk matrix is the most visually convincing measurement instrument in security that doesn't measure anything, and there is a substantial body of research to back that up. A good summary of that literature can be found in Hubbard’s The Failure of Risk Management, but the three core problems are worth unpacking here because they are not obvious from looking at the chart.

1. The ordinal scale problem. You already know this one from the hot sauce problem above. Likelihood and Impact in a risk matrix are ordinal scales: they encode rank order only, saying nothing about the distance between levels. “Likely” is ranked above “Possible,” but by how much? Nobody knows, because the scale does not define it. You could relabel the five likelihood levels as 1, 7, 103, 500, and 8,000,000 and the ranking would be identical, but the positions on the grid would be wildly different. Any arithmetic that combines those levels is operating from faulty logic, and the grid coordinates that result are meaningless as a consequence.

2. Range compression. A 5x5 matrix has 25 cells, which means every risk your organization faces gets forced into one of 25 buckets. A 3x3 grid obviously has even fewer buckets. A scenario with a 15% probability of occurring lands in the same cell as one with a 24% probability. A loss of $800K looks identical to a loss of $4M if they share a cell. The matrix not only fails to capture that variance; it actively hides it, which means the decisions made from it are based on less information than the underlying analysis contains.

3. False precision. This is the one that makes the other two worse. Because the matrix produces a position, a color, and a label, it creates the appearance of a calculated result. The grid implies that something was measured and plotted. The colored zones imply that the boundaries between them were defined. None of that is true, and the visual language of the chart makes it harder to notice, not easier.

For next time: Probability and loss ranges per scenario give you something the matrix cannot. “Ransomware: 60% probability, $2M to $9M expected loss range” is a single row of a table. It is less dramatic than a 5x5 grid and considerably more useful, because it is a format that connects directly to a decision.

⑥ 37% Reduction in Cyber Risk: A Semi-Attached Figure

You finish a long run, step on the scale, and you are down three pounds. The number is real and it went in the right direction, but it is measuring sweat and fluid loss, not fat. You did not get three pounds leaner. The scale measured something real, just not the thing you wanted to know.

Darrell Huff called this the semi-attached figure in the classic book How to Lie with Statistics, and the 37% reduction on this dashboard is a textbook example. The bar chart shows a reduction in “Risk Score.” Risk Score is a composite of weighted indicators whose components and weights are not defined anywhere on the dashboard. The reduction of 37% tells you that a number went down, derived from other numbers, combined in a way that is not disclosed. Before you can measure a reduction in X, you need to define what X is, and “cyber risk” as expressed by an undisclosed composite is not a defined construct. The 37% is not a measurement of risk reduction; it is a measurement of a score that was named after something. The further you trace the number back, the further you get from the thing you want to know, which is whether the organization is less likely to suffer a material loss than it was a year ago.

“On Track” sits in a badge in the corner. Toward what, though? There is no target state anywhere on the dashboard, no defined destination, and no criterion for what “arrived” would look like. The chart is reporting progress toward a place it has never described.

For next time: Define the target in real units first. “We aimed to reduce the probability of a loss exceeding $5M from 40% to 25% and are currently at 31%” is a bar chart that works well here because it attaches the figure to the thing you are trying to manage.

⑦ 72/100 Cyber Resilience Score: Goodhart’s Law in a Speedometer

There is a principle in economics called Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. When organizations find ways to move the number they are being measured on, the connection between that number and the underlying reality it was meant to represent loosens.

A cyber resilience score on a board dashboard is close to an ideal Goodhart trap. It is visible, it is tracked quarter over quarter, and it is labeled Good or Needs Improvement in terms that make optimization feel like progress. A score that can be influenced by checking off controls or shifting which inputs get counted, without any corresponding reduction in the actual probability or magnitude of loss, will reliably go in the right direction while telling you nothing about whether the organization has become more resilient in any meaningful sense.

The speedometer graphic makes this worse. The gauge is borrowed from physical systems like fuel levels and engine temperature, where the underlying measurement is real, continuous, and directly connected to what the needle is showing. Resilience has none of those properties, and the visual language of the gauge implies a precision and directness that the composite index behind it cannot deliver.

Underneath all of this is a measurement problem that the design problem obscures. When you aggregate heterogeneous indicators into a single number, you lose the information that would tell you how to act. For example, if your temperature, your blood pressure, and your white cell count are each concerning, a composite health score of 68 does not tell you whether to take aspirin, call a cardiologist, or start antibiotics. The more interesting question is what decision changes if this score is 68 instead of 72. That question has no answer, because the score was never meant to have one. It was designed to communicate that things are being monitored, which is a different purpose and a lower bar.

For next time: The metrics that compose a score are almost always more useful than the score itself. “Ransomware scenario: 60% probability, $3.2M expected annual loss. Closing the top three control gaps would reduce expected loss to $1.8M.” Three numbers, each in a unit that means something, each connected to a decision. That is what disaggregation looks like on a board slide.

⑧ The 3D Pie Chart: An Optical Illusion with a Data Problem

Phishing accounts for 28% of the data. Now look at how much of the chart it appears to occupy at the front of that tilted disk. It occupies more space than Ransomware, which is 34%.

Before we get to the chart, look at the three lines above. They are identical in length. Your visual system insists otherwise, because it is responding to the arrow geometry rather than the actual measurement. This is the Müller-Lyer illusion, and it is not a quirk or a trick that careful observers can avoid; it’s just how human vision works. The visual system applies depth cues automatically and universally, and almost always gets this wrong.

The 3D pie chart exploits exactly the same mechanism. What you are seeing when you look at that Phishing slice is Tufte’s Lie Factor in action: the ratio of the size of the visual effect in a graphic to the size of the actual effect in the data. A Lie Factor of 1 means the graphic represents the data faithfully, and a Lie Factor greater than 1 means the graphic is exaggerating. The 3D pie chart structurally cannot achieve a Lie Factor of 1, because the perspective projection that creates the illusion of depth physically inflates front slices and compresses back slices independently of what the data says. Rotate it 180 degrees, and Phishing would appear to be much smaller, with the data unchanged and only the geometry different.

3D Pie charts persist because they are intuitive and available in every charting tool. Nobody ever got fired for using one (that I know of). However, intuitive and accurate are not the same thing, and this one is working against you.

For next time: A horizontal bar chart lets the eye compare lengths along a common baseline, which is the one perceptual task humans perform with any reliability. The comparison is immediate and exact; the 3D pie requires you to estimate it through distorted geometry.

⑨ Risk Posture Over Time: Improving Relative to Nothing

The trend line descends from 82 to 52 across eight quarters, the footnote reads “Composite score based on weighted risk indicators,” and the label over the chart reads “Improving.”

The word “Improving” is doing a lot of heavy lifting on this chart. The organization’s own prior score on an undefined composite index went down, but threat actors did not stand still during those eight quarters. Ransomware groups got new tools and techniques, supply chain attack surfaces expanded considerably, and AI-assisted phishing scaled to the point where generating a convincing attack email dropped from hours to minutes. A score that improves monotonically through eight quarters of a changing threat landscape may indicate that risk declined, maybe via reductions in frequency or magnitude, or both, or it may indicate that the measurement system is insensitive to the things that changed most. The trend line cannot distinguish between those two explanations, and neither can anyone reading this dashboard.

For next time: Trending risk metrics against the total risk landscape, rather than just against themselves, adds the context that makes a trend line meaningful. If expected loss for a ransomware scenario declined from $8M to $5M while ransomware frequency in your sector increased, that context belongs on the same chart. A trend line without a reference point is a line going somewhere, and somewhere is not enough for a board conversation.

Happy April Fools

Happy April 1st. Please do not use this dashboard.

If any of the nine callouts sparked something and you want to go deeper, these are some additional resources:

• Darrell Huff, How to Lie with Statistics — short, funny, and still the best introduction to how numbers mislead

• Douglas Hubbard, The Failure of Risk Management — the most thorough takedown of risk matrices and qualitative methods in print

• Douglas Hubbard and Richard Seiersen, How to Measure Anything in Cybersecurity Risk — the practical companion, written specifically for this field

• Edward Tufte, The Visual Display of Quantitative Information — everything you need to know about why some charts work and most don’t

• I wrote an essay here on the Semi-Attached Figure

✉️ Contact

Have a question about this, or anything else? Here’s how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

🌐 Elsewhere

I share shorter thoughts on risk, metrics, and decision-making on LinkedIn.

Book updates, chapter summaries, tools, and downloads are at www.heatmapstohistograms.com

My longer-form essays and older writing live at www.tonym-v.com

❤️ How You Can Help

✅ Tell me what topics you want covered: beginner, advanced, tools, AI use, anything ✅ Forward this to a colleague who’s curious about CRQ
✅ Click the ❤️ or comment if you found this useful

If someone forwarded this to you, please subscribe to get future issues.

Thanks for reading.

—Tony

A step-by-step guide to the chart that connects your analysis to executive decisions

In This Issue:

• 📖 Book Update

• 🎤 Upcoming Talks

• 📝 How to Read a Loss Exceedance Curve

• ❓ Reader Question

📖 Book Update

The finish line is here. From Heatmaps to Histograms is in its final production stages and the release is imminent. I have several book signings planned that I’ll announce in future issues, so there will be plenty of opportunities to connect in person.

If you’ve been thinking about pre-ordering, this is your last window. Pre-orders matter more than most people realize: they signal demand to retailers, influence how the book gets promoted, and give the launch real momentum from day one.

I also have a special offer for newsletter subscribers: if you pre-order and send me a note with your mailing address, I’ll mail you a set of my CRQ stickers. International mailing addresses are perfectly fine.

🔗 Pre-order on Amazon | 🔗 Book website

🎤 Upcoming Talks

March 24, 2026 — “The Future of Cyber Risk Intelligence” FAIR Institute Seminar at RSAC 2026, San Francisco

I’m closing out the FAIR Institute’s seminar at RSA with a session on the shift from backward-looking risk reporting to forward-looking risk intelligence. The FAIR seminar is always one of the best events of RSA week, and I’d encourage anyone attending the conference to make time for it.

🔗 More info

April 21, 2026 — “Did We Solve the Data Problem? Judgment, Beliefs, and Risk in the AI Age” SIRAcon 2026 — Keynote

I’m keynoting SIRAcon this year with a talk on why the real bottleneck in risk quantification was never data, it was judgment, and why AI makes that distinction more important than ever. SIRAcon remains my favorite conference because it is model-neutral, vendor-neutral, and entirely dedicated to advancing risk practices.

🔗 More info

📝 How to Read a Loss Exceedance Curve

I’ve shown the same chart to hundreds of people across different fields. Finance teams nod and start asking questions about the tail. MBAs recognize it from business school, and engineers start interpreting it and reading their interpretation back to me before I finish explaining the axes. When I show the exact same chart to information security professionals, I usually get blank stares.

The chart is a loss exceedance curve, and I’d argue it’s one of the most important visualizations in cyber risk quantification. It’s also the one that confuses security people the most, which is a problem because it’s the chart that executives already think in. They’ve seen it in insurance models, in business forecasts, in scenario planning. They just haven’t seen it come from a security team.

This might reframe how you feel about this chart if it’s unfamiliar to you: it’s not new. The exceedance curve has been around for over a century. Hydrologists were using it to model flood risk long before anyone was quantifying cyber threats, plotting the probability that a river would exceed a given height in a given year. Insurance companies have relied on it for decades to price policies and set reserves. Weather forecasters, structural engineers, and financial analysts all use the same underlying concept. The x-axis represents a magnitude of something (water height, wind speed, financial loss) and the y-axis represents the probability of exceeding that magnitude.

Cyber risk quantification didn’t invent this visualization. It borrowed a proven tool from fields that have been reasoning about uncertainty far longer than information security has existed. If the chart feels foreign to you, that’s on us, not on the chart. Security has spent decades trapped in qualitative frameworks, heat maps and red-yellow-green dashboards, and we never learned how other disciplines communicate risk along the way.

This issue is my attempt to fix that, at least for this one chart. It’s not complicated. It just hasn’t been taught well to security practitioners.

Start with the Shape: The Histogram

Before you can understand a loss exceedance curve, you need to know what it’s built from.

When you run a quantitative risk assessment using a framework like FAIR, the engine underneath is a Monte Carlo simulation. You provide estimated ranges that define probability distributions for your inputs, and the simulation samples from those distributions across thousands of trials, sometimes 10,000, sometimes 50,000 or more. A frequency distribution might be centered between 1 and 4 times per year, and a loss magnitude distribution might span $50,000 to $500,000. Each trial represents one possible version of next year. Most simulated years are unremarkable, a few are catastrophic, and the spread between those outcomes is where the interesting information lives.

The first way to visualize those results is a histogram. It groups the simulated outcomes into bins and displays them as a bar chart. The x-axis shows loss amounts and the y-axis shows the percentage of simulations that landed in each range.

The histogram shows you the shape of risk: where most losses cluster, how wide the spread is, and whether there’s a long tail of expensive outcomes stretching to the right. A narrow peak means the losses are fairly predictable. A wide spread or a long tail means there’s real uncertainty about how bad things could get, and that uncertainty demands different planning than a tight distribution does.

The y-axis on a histogram shows the percentage of simulations that produced different loss amounts. It does not show the probability of an incident happening. Think of each bar as answering the question “if something bad happens, here’s how bad it’s likely to be.” The frequency of the event itself is already baked into the annual loss exposure calculation.

The histogram is useful, though it has a limitation: it doesn’t answer the question that executives ask most often, which is “what are the chances we lose more than $X?” For that, you need the loss exceedance curve.

How a Loss Exceedance Curve Is Built

The math behind a loss exceedance curve is simpler than most people expect. If you can sort a column of numbers and do basic division, you already know how this works.

Imagine your Monte Carlo simulation produced 50,000 results, and they’re sitting in a spreadsheet column, one per row. Each number represents the total annual loss from one simulated year. Some are zero, some are modest, and a few are enormous.

To build the curve, you conceptually sort those 50,000 results from smallest to largest. Then, for any dollar amount you want to examine, you count how many of the 50,000 results exceeded that amount and divide by 50,000. That gives you the exceedance probability for that dollar amount.

If 10,000 of your 50,000 results exceeded $2 million, the exceedance probability at $2 million is 10,000 divided by 50,000, which is 20%. If 2,500 results exceeded $5 million, the exceedance probability at $5 million is 5%. Plot every dollar amount against its exceedance probability, and you get the curve.

That’s the entire construction. Just “what fraction of my simulated outcomes were worse than this amount?” repeated across every dollar value on the x-axis. The curve slopes downward from left to right because fewer and fewer simulations produce extreme outcomes as you move toward higher loss amounts.

I often spend extra time on the construction with skeptics because people trust a chart more when they understand what’s underneath it. When you can explain to someone that the curve is built from 50,000 simulated scenarios and each point represents a simple fraction, you’re building trust in not only the chart, but the underlying methodology and process.

One caveat: the curve will faithfully reflect whatever you feed it. If your input distributions are poorly reasoned or pulled from thin air, the LEC will produce a polished, confident-looking result that points you in the wrong direction. The visualization doesn’t validate the analysis; that’s something you still need to do.

I built an interactive tool for the book’s companion site that lets you run through this entire construction yourself, from setting inputs to watching the histogram transform into an exceedance curve. Try it here.

What the Loss Exceedance Curve Shows

The histogram and the LEC are built from the same simulation data, though they answer different questions. The histogram answers “what does the spread of outcomes look like?” The LEC answers “what are the chances we lose more than a given amount?”

I call the LEC the “What Are the Chances?” chart because that’s the question it answers in every conversation where it matters. What are the chances we lose more than $10 million? What are the chances losses exceed our insurance coverage? What are the chances this risk is bigger than the project we’re considering? Every one of those questions maps to a single point on the curve.

This is why executives respond to it immediately. They’re accustomed to reasoning in terms of exposure and probability because every other risk function in the organization already communicates this way. Finance, insurance, and operational risk all speak this language, and security has been the holdout, bringing heat maps to a conversation where everyone else brings probabilities. The LEC gets you a seat at that table.

How to Read It: Three Steps

Reading a loss exceedance curve takes three steps.

The y-axis shows probabilities, running from 0% at the bottom to 100% at the top. The x-axis shows dollar amounts, increasing from left to right. The curve itself shows the probability of losses exceeding any given dollar amount.

Step 1. Find your dollar amount of interest on the x-axis. Let’s use $35 million as our example.

Step 2. Move straight up from that point until you hit the curve.

Step 3. Move horizontally to the left and read the probability on the y-axis. In this example, you’d land at approximately 20%.

That gives you: “There is a 20% chance that losses will exceed $35 million.”

If you look at the shaded area above and to the right of the $35 million mark, that region represents the 20% probability zone, all the outcomes where losses exceed $35 million. The larger the shaded area at any given point, the higher the probability of exceeding that loss amount. As you move further to the right toward larger losses, the shaded area shrinks because fewer and fewer simulated outcomes reach those extremes.

The Y-Axis Trap

Misunderstanding the Y-axis is the most common mistake I encounter when people are new to the LEC.

The y-axis does not show how often incidents occur. It shows the probability that losses exceed a particular dollar amount. The simulation already accounts for how often events happen; that’s built into the annual loss exposure before the curve is ever drawn.

The confusion makes sense when you think about where security professionals come from. We’re trained to think in terms of “will an attack happen?” The LEC answers a different question: “if something bad happens, how bad could it get?” If you mix those two up, the y-axis will mislead you.

Loss Exceedance Statements

I always accompany a loss exceedance curve with written statements that serve as a voiceover for people who aren’t comfortable reading probability graphs.

The format goes something like this:

• “There is a 20% chance that losses will exceed $2 million.”

• “We have a 50/50 chance of losses exceeding $500,000.”

• “There is only a 5% chance we would see losses above $8 million.”

Each statement is a single point on the curve translated into plain language. You pick the dollar amounts that matter to your audience, read the corresponding probabilities, and write the sentences.

Here’s the presentation technique that I’ve found lands well: start with the loss exceedance statements, then show the curve. When you lead with the narrative, you prime your audience so they know what to look for before the chart appears on screen. They understand what the axes mean, they’re oriented, and the visual confirms what they’ve already absorbed. I’ve been doing it this way for years, and the difference in how people engage is night and day compared to showing the chart cold and trying to explain it after the fact.

Anchoring the Curve to Real Decisions

You should not pick an arbitrary dollar amount to point out to audiences. Pick amounts that matter to your organization. I use five anchors regularly.

Risk tolerance. Ask your board or CFO what the maximum loss the organization can absorb before it threatens operations. Use that number as your primary anchor point on the curve. If there’s a 30% chance of exceeding your stated risk tolerance, that’s a conversation worth having.

Cyber insurance limit. Losses beyond the policy limit are uninsured, and the organization bears the full impact. Plot your insurance limit on the curve and read the exceedance probability. Keep in mind that exclusions and sub-limits for things like ransomware or regulatory fines may reduce the effective coverage, so the real exposure might start earlier than the headline policy number suggests.

Budget thresholds. Compare the curve to your contingency or emergency reserves. If there’s a 60% chance of exceeding those reserves, your planning needs attention regardless of what the risk appetite statement says.

SEC materiality. For public companies, ask legal at what loss level disclosure would likely be required. There’s no fixed dollar rule since materiality depends on context, though establishing a working threshold makes the curve actionable for your compliance team.

Project costs. Compare loss probabilities to the cost of proposed security investments. If there’s a 40% chance of losses exceeding the cost of a $3 million project, the investment starts to look justified in a way that a heat map could never communicate.

Once you start overlaying these thresholds on the curve, the conversation changes. You’re answering questions your stakeholders already have, using numbers they already care about, and in my experience that’s the moment the LEC earns its place in every presentation you give.

Closing

The loss exceedance curve isn’t exotic. Hydrologists, insurers, and financial analysts have relied on it for over a century because it answers the question that matters most when you’re making decisions under uncertainty: what are the chances things get worse than this?

Cyber risk quantification adopted it for the same reason. If this chart is new to you, you’re in good company. Most security professionals never encountered it because our field spent decades in qualitative frameworks that didn’t require it. Once you learn to read it, the way your executives engage with your work will look very different.

🔗 Interactive tool: Build and read a Loss Exceedance Curve

❓ Reader Question

“I’m a security analyst thinking about moving into risk quantification. Is it worth getting the FAIR certification, or should I just start doing the work?”

Both, but start with the work. The certification will make more sense after you’ve worked on a few real assessments, because the concepts stick differently when you’ve already felt the pain of scoping a scenario or arguing over a frequency estimate. If you study for the exam first with no practical context, it reads like theory. If you do two or three assessments first, even rough ones, the material reads like answers to questions you already have.

That said, the FAIR certification is worth pursuing. It gives you a shared vocabulary and understanding with other practitioners, it signals to hiring managers that you’re serious about the discipline, and the study process will tighten your understanding of the taxonomy in ways that doing the work alone won’t. It’s not expensive, and it’s not a months-long commitment.

The bigger career advice is this: build a portfolio of work product. Run an assessment, document it, present the results internally, and refine your approach based on the questions you get. Three completed assessments with clear write-ups will do more for your career trajectory than any credential on its own.

✉️ Contact

Have a question about this, or anything else? Here’s how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

🌐 Elsewhere

I share shorter thoughts on risk, metrics, and decision-making on LinkedIn.

Book updates, chapter summaries, tools, and downloads are at www.heatmapstohistograms.com

My longer-form essays and older writing live at www.tonym-v.com

❤️ How You Can Help

✅ Tell me what topics you want covered: beginner, advanced, tools, AI use, anything ✅ Forward this to a colleague who’s curious about CRQ
✅ Click the ❤️ or comment if you found this useful

If someone forwarded this to you, please subscribe to get future issues.

Thanks for reading.

—Tony

• 📖 Book Update: A Forecast, Revised

• 🎤 Upcoming Talks

• 📝 The Obscured Target: Why Measurement Quality Matters

• ❓ Reader Question

📖 Book Update: A Forecast, Revised

The official Amazon release date for From Heatmaps to Histograms is April 29, though we’d been pushing hard to get it out by early-March in time for RSA. I had a book signing planned at the RSA bookstore and a few other events lined up around the conference.

I got word late last week that mid-March isn’t going to happen. I’ll be honest, I went through all five stages of grief on this one. Denial was particularly convincing for about 48 hours.

The good news is that we’re only talking a few weeks, not months. Early April is the current target, which means RSA attendees will just have to settle for my sparkling personality instead of a signed copy.

Those of you who have been reading since Issue 8 may remember that I said I was “90% confident” the book would land between March 4 and 13. This is a good teaching moment. A 90% forecast means I believed there was roughly a one-in-ten chance the real date would fall outside that range, and that’s what happened. The Bayesian mindset doesn’t treat that as a defeat. New information came in, I updated my belief, and the estimate moved accordingly. That's forecasting working the way it should. If you never land outside your forecast ranges, your ranges are probably too wide, and you’re not being honest about your uncertainty.

In other news, I redesigned the website at heatmapstohistograms.com and added full chapter descriptions, so if you want to know what’s inside the book, check out the “Inside the Book” section. I also sent the manuscript to a number of people across the industry and received some incredibly generous feedback. Here are three of my favorites:

From Heatmaps to Histograms is a significant contribution to our profession, and would be required reading for anyone in my organization if I was still a CISO. Brilliantly written for those with little or no background in quantitative risk measurement, it also will be very useful to those with years of experience. This is just further evidence that Tony is one of the leading contributors to the future of our profession.

-Jack Jones, Creator of FAIR

I’ve been measuring various aspects of cyber risk for over 20 years. In the early days, I saw the lack of reliable data as the primary roadblock to the broad adoption of quantitative approaches to managing risk. I’m still a strong proponent of better data, but I now view practicality as the chief impediment. A recurring “Yeah, but how?” dominates conversations on cyber risk quantification, but practical answers and examples remain scarce. That’s why I’m so excited about this book. Tony strikes the perfect balance of what you need to know and what you need to do to make CRQ work for you.”

-Wade Baker, Ph.D., Partner, Cyentia Institute and Professor, Virginia Tech

Tony Martin-Vegue makes a strong case for replacing certain popular risk assessment methods and he goes further with practical approaches needed to implement better methods. He adds multiple case examples and step-by-step procedures with the primary goal of making even more quantitative concepts accessible to every reader. I highly recommend his book.

-Douglas Hubbard, author and measurement expert

I’m thrilled that they liked the book, and I’m grateful for the time they spent reading it.

🔗 Pre-order on Amazon | 🔗 Book website

🎤 Upcoming Talks

I have a few events planned:

March 24, 2026 “The Future of Cyber Risk Intelligence” FAIR Institute Seminar at RSAC 2026, San Francisco

I’m closing out the FAIR Institute’s seminar at RSA with a session on what boards and regulators will expect next from cyber risk programs. The talk covers the shift from backward-looking risk reporting to forward-looking risk intelligence, where AI fits (and doesn’t), and the skills that will define the next generation of risk analysts. The FAIR seminar at RSA is always one of the best events of the week, and I’d encourage anyone attending the conference to make time for it.

🔗 More info

April 21, 2026 “Did We Solve the Data Problem? Judgment, Beliefs, and Risk in the AI Age” SIRAcon 2026 — Keynote

I’m honored to be keynoting SIRAcon this year. My talk argues that we never had a data problem in risk quantification; we had a judgment problem wearing a data costume. Now that AI has made data abundant and cheap, the real bottleneck is fully exposed: judgment under uncertainty. The session develops three ideas: risk estimates are beliefs rather than truths, uncertainty is not a data gap AI can close, and data must earn the right to influence belief before it shapes decisions.

SIRAcon is my favorite conference because it is model-neutral, vendor-neutral, and entirely dedicated to advancing risk practices. I’d encourage anyone serious about the craft to attend.

🔗 More info

📝 The Obscured Target

Why Measurement Quality Matters

A few months ago, I asked an LLM to help me research breach frequency data for a risk assessment I was running. It gave a very convincing response: a specific industry, a specific incident type, a time range, and a precise frequency. I was almost persuaded by the exact number, the clean citations, and a reference to what appeared to be a Ponemon report I hadn’t seen before. The output had that satisfying feeling of precision.

Nonetheless, I did what I always tell people to do. I checked the sources.

The Ponemon report didn’t exist. It wasn’t a miscited year or a different title. The report was completely fabricated (a hallucination). In human parlance, we would call that a lie.

Here’s the disturbing thing: I am very well-versed in industry reports, especially data breach frequency. My spidey sense tingled, though I think for many other subjects I would have a hard time distinguishing a good output from a hallucination. My brain was ready to accept it and move on. If I’d been any less disciplined, those fake numbers would have gone straight into my model.

That experience sent me back to something I’d been teaching for years without fully appreciating its implications. AI introduces a new dimension into how we think about the accuracy and precision of our measurements.

Accuracy and Precision: The Four Targets

If you’ve spent any time studying metrology, the science of measurement, or any field that draws from it, like decision science, risk quantification, and many others, you’ve probably seen the four targets.

It’s a teaching aid used to illustrate a foundational concept in measurement: accuracy and precision. The typical image shows four bullseyes, four shot patterns with each one illustrating a different combination of accuracy and precision. It’s one of those visuals that’s been around forever because it works so well, and you see it once and you get it.

As I mentioned earlier, this concept comes from metrology. Metrology is about how we know what we know when we measure things. How do you know your scale is correct? How do you know your thermometer isn’t drifting? How do you know the number you’re looking at reflects reality?

These questions matter every time you make a decision based on a measurement. In cyber risk quantification, we’re making decisions based on continuous measurements, such as loss estimates, frequency ranges, and confidence intervals. It all depends on whether our measurement process is working. Keep in mind that risk quantification, including FAIR, is a forecast, and a forecast is a type of measurement.

I teach the concepts of accuracy and precision using the four-targets analogy whenever I teach FAIR. It’s one of the first concepts I cover because it reframes how people think about estimates. Most people walk in thinking the goal is to get the “right” number, and the four targets help them see that measurement quality is more nuanced than that.

Let me walk through them quickly, because you need all four to understand the fifth.

Accurate and Precise. Tight grouping, dead center. This is the ideal. Your estimates cluster close together and they’re centered on the true value. In CRQ terms, your model is well-calibrated and your inputs are solid. If and when the results of forecasts are known, the forecasts land consistently where they should.

This can be expensive, though. Getting here requires good data, experienced analysts, validated models, and time. If you’re chasing this standard for every assessment, you’ll burn through resources fast. It’s like insisting on a gourmet meal when a solid sandwich would get you through the day just fine. Save the five-star treatment for the risks that justify the investment.

Accurate and Imprecise. Scattered shots, centered on the target. Your estimates vary widely, though on average you’re in the right zone.

This is the sweet spot for early-stage analysis. You’re not nailing down exact numbers, you’re just figuring out if this risk matters. Does it change your decision if the loss is $2 million or $8 million when you’re comparing it to something that might cost $200 million? Probably not. Wide yet accurate ranges can drive good decisions, so don’t let the pursuit of precision slow you down when direction is what you need.

Precise and Inaccurate. Tight grouping, off to the side. This is bias. Your estimates are consistent, consistently wrong. Maybe you’re anchoring on a vendor report, your SME has a blind spot, or your measurements have a structural flaw.

This one is dangerous because it feels like you’re doing well. The numbers look clean and cluster nicely, but you’re missing the target entirely, and I’ve seen this one in the wild many times: a team producing beautifully consistent assessments quarter after quarter, all anchored in the same flawed assumption no one questions. The precision gives you false confidence, and you might never go looking for the problem.

Inaccurate and Imprecise. Scattered shots, nowhere near the center. You’re all over the place and not even close. Your measurement process is broken.

The irony is that this is also expensive. You’re spending time and effort producing numbers that mean nothing. You might as well not measure at all, or rather it’s worse than not measuring, because at least then you know you’re guessing. Here, you think you’re doing analysis.

The Obscured Target

These four states cover most of what can go wrong with a measurement process. Bias, variance, systematic error, random noise: it all maps somewhere on this framework.

All four targets share one assumption, though: you can see where the shots landed.

You shoot. You walk up to the target. You look. The holes are there. Maybe they’re clustered, maybe they’re scattered, though you can verify the result against reality. You have access to ground truth.

With AI as your measurement instrument, that assumption breaks down.

When you ask an LLM to estimate breach frequency, pull loss data from a report, or synthesize threat intelligence, it gives you an answer, often a confident-sounding answer with citations. Unless you are disciplined enough to verify, you have no idea if those shots hit anything real.

This isn’t a precision problem. It’s not a bias problem. It’s something else entirely.

I call it the obscured target.

Picture a tight grouping, dead center. It’s textbook accurate and precise, except the target is hidden behind fog. You can’t see where the shots landed. You’re trusting the instrument’s report of where it hit.

Here’s an example: imagine stepping on a scale that always reads 165 pounds. Every time you weigh yourself, you get a consistent number. It looks accurate, it looks precise. There’s a problem you don’t know about, though: the display is disconnected from the sensor. It’s just showing you a stored value. The number has the shape of a measurement and feels like a measurement, yet nothing is being weighed in reality. Day to day, as your weight fluctuates, the instrument is wrong. It doesn’t have a measurement problem you can fix, like bias you can correct or imprecision you can average out. It has something worse: no measurement is happening at all. That’s what happened to me with those fake Ponemon numbers. The output wasn’t inaccurate in the normal sense. It was invented.

The “Obscured Target” Is a Validity Threat

The four targets describe properties of a measurement system: accuracy, precision, and bias. These properties are diagnostic, meant to help you evaluate whether your instrument is working. Metrologists would tell us the correct response to AI hallucination is the same response you’d give to any suspect instrument: calibrate it. Check the output against known values. Validate. If it’s wrong, fix it or stop using it.

They would be right.

If you can’t verify the output directly, decompose the estimate into components you can verify. Don’t treat the LLM as an oracle. Treat it as one input and calibrate it against known reference data, the same way you’d calibrate any instrument. This is a calibration and decomposition problem, and measurement science has solved such problems for a long time.

The framework works; the question is whether practitioners know to apply it when the output looks this good.

The obscured target isn’t a new measurement quality quadrant. It’s a category error to put it on the same level as accuracy and precision. What it is, in measurement theory terms, is a validity threat: a condition that undermines your ability to evaluate the instrument at all.

Think of it this way. The four targets all assume you can walk up to the bullseye and look. That’s the prerequisite. If you can see where the shots landed, you can diagnose accuracy, precision, bias, whatever. The obscured target breaks that prerequisite. It’s not telling you something new about the instrument’s performance. It’s telling you that you can’t evaluate the instrument’s performance in the first place.

When someone hands you a biased estimate, you can detect it and correct it. When someone hands you an imprecise estimate, you can invest in better data. The danger comes when the output looks so complete that you don’t think to verify it. You can always pull back the curtain, check the source, find the original data, validate the citation. The problem isn’t that verification is impossible. It’s that nothing about the output signals that verification is necessary.

Before you ask whether your AI-assisted estimate is accurate or precise, ask the more basic question: can I see the target? That question comes first.

What This Means for CRQ

We’re increasingly using AI to support risk analysis: estimating frequencies, interpreting incident data, extracting figures from documents, summarizing threat reports, just to name a few. I’ve written extensively about how AI can accelerate risk work and I use it every day.

The problem isn’t that AI is inaccurate. Sometimes it is, sometimes it isn’t. The harder question is whether we can tell which situation we're in.

The risk isn’t uniform across all AI-assisted tasks, either. Asking an LLM to extract a number from a report you provide is relatively low risk, since the source document is right there and you can check. Asking it to generate a breach frequency estimate from its training data is where the obscured target lives. The AI is producing a number with no source document behind it. Know which situation you’re in before you decide how much verification the output needs.

The classic four targets assume observability. We can validate results and compare what the instrument reported to what happened. AI breaks that feedback loop by giving you high apparent precision with no built-in signal that verification is needed.

In my book, From Heatmaps to Histograms: A Practical Guide to Cyber Risk Quantification, I have an easy-to-remember rule on how to think about AI: Jack Sparrow from Pirates of the Caribbean. AI, like Jack Sparrow, is charming, fast, and brilliant at what it does, though it’s optimized for completing the adventure (or the conversation). Neither will tell you when it’s making things up. At least Jack Sparrow knows when he’s lying. Current AI systems will hand you fabricated data with the same confidence as verified facts, and there’s no flashing warning light. The supervision has to come from you.

Before you evaluate whether your AI-assisted estimate is accurate or precise, ask a more basic question: can I see the target?

If the answer is no, you’re not measuring. You’re trusting.

We’re relying on AI more and more to assist with, and in many cases perform, our measurements. That’s a massive improvement in CRQ and a real time saver. As we hand more of the measurement process to AI, though, we take on a new responsibility, which is making sure what comes back is real.

❓ Reader Question

I’ve started using AI to help with risk assessments and my team loves it because it’s fast, but my manager is skeptical and says we can’t trust the outputs. How do I find the middle ground?

Your manager isn’t wrong, and you’re not wrong either. The answer depends on which part of the workflow you’re handing to AI.

I think about this as a question of where the information is coming from. When the AI is working with something you provided, like summarizing a report you uploaded or generating Monte Carlo code from parameters you specified, the risk is low because you can check the output against your own inputs. The source material is sitting right there. If the AI gets something wrong, you’ll probably catch it.

The risk changes when the AI is generating information from its training data. Estimating breach frequencies, citing industry studies, producing loss figures: these are the tasks where the obscured target lives. You have no way to verify the output without going and finding the original source yourself, which is the step that feels redundant when the AI’s answer looks so polished. That feeling is the danger.

The middle ground your manager is looking for is a workflow where verification scales with risk. Use AI freely for the tasks where your inputs are the source of truth. When the AI is the source, treat the output the way you’d treat a claim from a colleague you’ve never worked with before: interesting, possibly useful, not yet trusted. Verify before it goes into your model.

If you can explain that distinction to your manager, you’ll probably find more common ground than you expect.

✉️ Contact

Have a question about this, or anything else? Here’s how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

🌐 Elsewhere

I share shorter thoughts on risk, metrics, and decision-making on LinkedIn.

Book updates, chapter summaries, tools, and downloads are at www.heatmapstohistograms.com

My longer-form essays and older writing live at www.tonym-v.com

❤️ How You Can Help

✅ Tell me what topics you want covered: beginner, advanced, tools, AI use, anything ✅ Forward this to a colleague who’s curious about CRQ
✅ Click the ❤️ or comment if you found this useful

If someone forwarded this to you, please subscribe to get future issues.

Thanks for reading.

—Tony

• 📖 Book Update: Production and Pre-Orders

• 🎤 Upcoming Webinars

• 📝 Why "Risk-Based Security" Remains Elusive

• ❓ Reader Question

📖 Book Update: Production and Pre-Orders

From Heatmaps to Histograms has officially entered production with Apress. The Amazon pre-order date has moved up to Friday, April 3, but we’re still targeting an early March release. I’m 90% confident it’ll land between March 4–13, just in time for RSA. Ebook and international ordering options are coming soon.

It even briefly hit #1 in Amazon’s Network Security category, which was a fun surprise.

🔗 Pre-order on Amazon 🔗 Book website

🎤 Upcoming Webinars

January 21, 2026 at 12:00 PM EST “The Six Levers That Actually Move Risk (Hint: It’s Not Just Controls)” Greater Ohio Chapter, FAIR Institute

This was one of the top-rated talks at FAIRcon in November. If you missed it, here’s your chance to see it. I’ll share lessons learned from building Netflix’s FAIR-based CRQ program, starting from the premise that controls are just one lever, and often not the biggest one. Most changes in risk come from forces far outside your walls. This presentation identifies six environmental forces acting on cyber risk and how each affects loss event frequency and loss magnitude.

This virtual webinar is open to anyone. You don’t need to be a member of the Ohio chapter or live in Ohio.

Register here

January 29, 2026 at 12:00 PM EST “From Gut Feel to Good Data: How AI Can (and Can’t) Transform Risk Management” FAIR Institute

Artificial intelligence promises faster, richer insights for cyber risk quantification, but it also brings hallucinations, biases, and overconfidence. This talk explores where AI truly adds value in transforming gut-feel estimates into usable data, and where human judgment and validation remain essential. Attendees will leave with practical strategies to integrate AI into their risk workflows without losing rigor or credibility.

Register here

📝 Why “Risk-Based Security” Remains Elusive

Over the holidays, I did a Reddit AMA on r/cybersecurity with three CISOs. The topic was :“I’m a security professional who transitioned our security program from compliance-driven to risk-based. Ask Me Anything.”

I expected the usual questions about tools, metrics, and getting executive buy-in, and perhaps some pushback on quantitative methods in general. I expected to dig deep into techniques and maybe a bit of mythbusting. It is Reddit, after all, a notoriously tough crowd. What I got instead was a bit more revealing and personally enlightening.

The AMA brought into the forefront something I wrote about extensively in the book, but sometimes lose sight of on a day-to-day basis, because I work with clients almost exclusively who are already doing some level of quantification. The gap between “compliance-based security programs” and “risk-based thinking” is wider than even the quantification community acknowledges. We’ve been focused on teaching people how to build models, even though many still don’t understand why models are necessary. That’s a pedagogy problem, not a technical one.

What made the week interesting wasn’t that everyone was confused. The thread was divided between people who immediately understood what we were talking about and people who believed they were already doing a risk-based program, but weren’t. Understanding what separated those two groups taught me more about the state of the field than any conference presentation I’ve given.

The Scoring System That Wasn’t Risk Management

Early in the thread, someone asked for practical examples. I walked through a ransomware scenario with frequency estimates, loss components in dollars, and the options leadership was choosing between. Basic and standard CRQ work.

Another commenter jumped in with their own example: an elaborate scoring system where applications were assessed against control frameworks, weighted by criticality, filtered through review processes, then bucketed into high, medium, and low categories. The description was detailed and clearly took real effort. They walked through how apps scored on a 1-100 scale using weighted formulas for vulnerabilities, control implementation, test cadence, and audit findings. The scores were precise, applications ranked cleanly, leadership got quarterly reports showing movement.

They believed they were describing risk management. I had to explain why they weren’t:

“This is a well thought out scoring and governance system, but it’s still measuring control posture, not risk. All the ‘math’ happens in ordinal space, so at the end you know which app is ‘higher’ than another, not how much loss you’re exposed to or what you bought down by fixing it. High/Medium/Low doesn’t tell leadership whether Control A was a better investment than Control B, only that something moved. That’s fine for standardization and audits. It hits a ceiling the moment the question becomes tradeoffs, ROI, or ‘was this worth the money.’”

Risk, at its core, has two components: probability (how often something happens; likelihood, frequency, chances) and impact (how bad it is when it does; magnitude, loss, consequence). If your assessment can’t tell you both of those things in terms that support decisions, you’re measuring something else. This scoring system measured control coverage. It’s useful for passing audits, but it couldn’t answer the question every executive cares about: If we spend money here, how much safer are we?

This pattern repeated throughout the week. People used “risk-based” to describe prioritization frameworks, vulnerability triage workflows, compliance programs that measured control posture, and GRC tools that tracked remediation. It’s all useful work, some even necessary, but none of it was risk management.

The Questions That Never Came

Midway through I decided to stop and write a definitional post that described what “risk-based security” is in practice. I realized the conversations were drifting and we weren’t talking about the same thing:

“At its core, risk management is about decision-making under uncertainty. Risk itself is a future event. A risk assessment is a forecast about something adverse that might happen, how often it could happen, and how bad it would be if it does. It’s not a list of issues, gaps, concerns, audit findings, controls, or aspirations. Those are inputs, not risk. When a risk register starts to look like a to-do list, what you really have is a compliance tracking system with risk language layered on top.”

The questions I wanted never really materialized:

• How do you validate your loss estimates?

• When does quantification provide enough decision value to justify the effort?

• How do you handle scenarios where you have no data?

• What’s the right level of precision for different types of decisions?

These are hard questions with nuanced answers. They require deep thinking about uncertainty, imperfect information, reasoning through problems, and the limits of modeling. These are the conversations that move the field forward. However, you can’t ask them if you think “risk-based” means moving your compliance tracking from red-yellow-green to a 1-5 scale with decimals.

The Skeptical Argument Worth Engaging With

Near the end of the week, someone posted something completely different. User Competitive-Coma shared a conversation they’d had about CRQ that had stumped them:

“Risk quantification is a poor forecasting tool without deep, high quality historical data -- the kind actuaries rely on. This is why the forecasts are never right.

If it were an effective forecasting method, stock traders would already use it successfully. Historical results show that those applying quantitative risk models to trading fail to outperform the market over time (e.g., S&P 500).

Its primary contribution appears to be limited to structured discussion of risks, which is a benefit that does not require formal risk quantification to achieve. If anything, the $$$ distracts from the points at hand.”

They added: “The problem I have is that this jives with my experience with FAIR and HDR, so I don’t have a rebuttal. That said, I am seeking to understand so please correct my thinking if I got any of that wrong.”

This was the best question in the entire thread. It was a genuine challenge and the original skeptic and the person asking understood what CRQ claims to be. They weren’t confusing prioritization with risk management. They understood CRQ attempts to characterize uncertainty to support decisions, and they’d encountered someone skeptical that it delivers enough value to justify the complexity. They wanted help engaging with that argument.

That’s the conversation the field should be having. My response tried to address each point directly. Here’s my response to the stock trading challenge:

“The comparison to stock trading is a category error. Markets are adversarial and reflexive systems. The moment you act on a model (buy stock), you change the system itself. A ransomware attacker does not change their behavior because you ran a Monte Carlo simulation, but markets absolutely respond to trading strategies. They also answer very different questions. In trading, the question is ‘can this beat the S&P 500.’ In CRQ, the question is ‘can this model help us make better tradeoffs under uncertainty?’ Those are fundamentally different problems.”

And then I had to acknowledge the elephant in the room:

“Finally, there is snake oil everywhere. Cybersecurity has plenty of it, and cyber risk quantification is not immune. If a person, vendor, or product is selling FAIR or HDR as accurate predictions or precise forecasts, they are overselling it at best or lying at worst. That is not what CRQ is for. CRQ is a decision support tool. It helps compare options, test sensitivity, understand uncertainty, and identify where uncertainty matters, where and why we care. When it is used that way, it is doing exactly what it is supposed to do.”

What separated this exchange from the rest of the thread was conceptual depth. This person had encountered a real critique of risk quantification and wanted to understand how to respond. The scoring system folks from earlier in the week didn’t know they were missing anything.

The Pedagogical Problem

The practitioners who posted thoughtful scoring systems aren’t the problem. They’re doing good work within real constraints. The problem is we, as a field, told them that better compliance tracking is risk management without changing how they think about the underlying problem.

What separated the people who got it from those who didn’t wasn’t intelligence or experience. It was exposure to decision science, actuarial thinking, forecasting, and statistics, disciplines that most cybersecurity training doesn’t cover. The CISSP still teaches asset-based quantitative risk methods from the 1980s that make CRQ seem either an impossible level of precision or mathematical theater. No wonder people are confused.

This is why I wrote From Heatmaps to Histograms. I’m not trying to convince sophisticated skeptics that quantification is perfect, because it isn’t, but I can help practitioners see that the “risk programs” they’re running are often compliance programs with better labels, and show them what the actual shift looks like. The conversation we need isn’t “is quantification worth it?” It’s “do we understand what risk management actually is?” Until we get that foundation right, we’re still passing audits, not managing risk.

The full Reddit AMA is still up on r/cybersecurity. It’s worth reading to see where the field actually is versus where we think it is.

❓ Reader Question

“I’m a college student studying cybersecurity and I’m interested in risk management, but I’m afraid AI will make it obsolete. Should I still pursue this path?”

I think risk quantification is one of the better bets for job security right now, for a specific reason.

Compliance-based risk work is already being automated, and AI is surprisingly good at it. Scoring vulnerabilities, tracking controls, filling out risk registers, mapping to frameworks are pattern-matching exercises. If your job is maintaining heat maps or running compliance reports, that work is disappearing fast.

Decisions under uncertainty are different. Judgments about tradeoffs, opportunity costs, and what risks are worth taking are harder to automate because they require business context that changes constantly. CRQ means translating technical risk into financial terms and helping executives choose between competing uses of capital. AI helps with parts of that workflow, but it can’t replace the judgment required to do it well.

If I were starting out today, I'd learn FAIR. It teaches you to think about risk in terms of decisions rather than scores. FAIR is widely recognized, has a thriving and growing community through the FAIR Institute, is taught in colleges and universities, and has extensive documentation and books available. It's easy to find help and mentors, and it fits well with existing risk frameworks. There's solid documentation on how to integrate it with ISO, NIST, and others. FAIR isn't the only model of course, but if I were starting out, I'd immerse myself in this. Understanding probability, impact, and how to communicate uncertainty to non-technical leaders is what keeps you relevant as the field changes.

I think in the next few years, practitioners doing rote compliance work will struggle. If you can translate risk into business language and help organizations make better decisions, you'll have work.

✉️ Contact

Have a question about this, or anything else? Here’s how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

Elsewhere

• I share shorter thoughts on risk, metrics, and decision-making on LinkedIn.

• Book updates, chapter summaries, tools, and downloads are at
  www.heatmapstohistograms.com

• My longer-form essays and older writing live at www.tonym-v.com

❤️ How You Can Help

✅ Tell me what topics you want covered: beginner, advanced, tools, AI use, anything
✅ Forward this to a colleague who’s curious about CRQ
✅ Click the ❤️ or comment if you found this useful

If someone forwarded this to you, please subscribe to get future issues.

Thanks for reading.
—Tony

• 📖 Book Update: The Manuscript Is Done

• 🔨 The Risk Workshop Survival Guide

• 👓 What I’m reading

• ❓Reader Question

📖 Book Update: The Manuscript Is Done

I crossed a big finish line earlier this week, and it still feels a little unreal to say it out loud: the manuscript is officially complete. After living with this book every day for months, in outlines, in messy notes, in drafts that did not survive the edit, there was a strange mix of relief and momentum when I typed the final line. The work is done, and somehow things are moving even faster now.

The manuscript also went through technical review by Robert D. Brown III, one of the sharpest thinkers in decision science. Working with Rob strengthened the book in ways I did not expect. Some of his comments made me laugh and wonder what I was thinking when I first drafted a chapter; others made me rethink how I explain key ideas so they land cleanly for readers. It was the kind of collaboration that leaves the work stronger and leaves you a better thinker than when you started.

From pre-orders alone, From Heatmaps to Histograms hit #1 in Amazon’s Network Security category.

Next up is copyediting, then printing, then launch. Ok, it’s real now.

🎉 Preorders are open for From Heatmaps to Histograms: A Practical Guide to Cyber Risk Quantification.
(Apress / Springer Nature, publishing March 2026)

If you have been following this journey, preorders help more than you might think. They signal demand to the publisher and the retailers.

🔗 The book’s official website
👉 Amazon preorder link

The Risk Workshop Survival Guide

A risk workshop exists for a simple reason: before you can analyze risk, you have to decide which risks are worth analyzing. Its job is to surface candidate scenarios, align on what matters, and provide structure to uncertainty before anyone starts modeling.

When it works, a risk workshop creates shared understanding. People leave with a clearer sense of what could happen, why it matters, and where analysis would inform a decision.

I have hosted more of these sessions than I can count. Over time, I learned that the difficult part is not the modeling that comes later, but the conversation in the room. If no one actively takes responsibility for the shape of that conversation, it will drift toward whatever is most interesting, provocative, or philosophically defensible in the moment.

That is how otherwise well-intentioned workshops end up debating nation-state-sponsored supply chain attacks and mega tsunamis in Nebraska.

These are the moves I have learned, often the hard way, to keep risk workshops focused and productive.

Survival Skill 1

Eliminate the impossible

After a workshop or GenAI brainstorm, you often end up with a very healthy list of risk statements. Sometimes too healthy. I have seen sessions produce fifty statements for a single asset class. No analyst is going to run fifty full assessments, nor should they.

Before you invest hours in scoping, run a quick quality check. Look at each risk statement and ask:

“Could this actually happen in the real world, given how the world works now?”

Some scenarios are impossible in the sense that matters for our work.

• A hurricane cannot be the cause of a data breach via SQL injection.

• A tsunami cannot destroy a datacenter in Omaha.

• Natural disasters do not directly exploit technical vulnerabilities.

This first pass should be quick. You are not assigning probability. You are filtering reality from noise.

Survival Skill 2

Handle “impossible” pushback with philosophy

There is always someone who will ask, “But what if you are wrong?”

Fair question. When analysts say “impossible,” we do not mean logically impossible, like a square circle, or physically impossible, like faster than light travel. We mean practically impossible: so far outside the range of the world we live in that modeling it would not help anyone make a decision.

Practical impossibility is about stewardship of attention. We focus on possibilities that can inform decisions within the constraints of the next quarter, next year, or the current budget cycle. As William James put it, beliefs are judged by their practical consequences. A mega tsunami reaching Omaha does not help a CISO allocate next year’s security budget.

Risk analysis is an exercise in practical wisdom. We bracket extreme theoretical scenarios not because we can prove they will never happen, but because including them makes our analysis less useful.

Survival Skill 3

Prioritize plausible before possible

Now you are left with things that could happen. The trick is choosing which ones deserve your time.

Consider two data breach scenarios:

• A: Cybercriminals exploit an unpatched web application vulnerability.

• B: A nation-state deploys a custom zero-day against proprietary middleware.

Both are possible. The question is which is more plausible for your organization.

Unless you are a defense contractor, Scenario A is usually the better use of time. It requires fewer special conditions and matches patterns we see in actual incident reports. Scenario B is not wrong. It is simply a poorer investment if you are trying to understand your most likely risks.

Survival Skill 4

Use Ockham’s Razor before you scope

When choosing between scenarios that could happen, ask:

• Which happens more often in organizations like ours

• Which requires fewer unusual circumstances

• Which a reasonable security professional would worry about first

This is Ockham’s Razor applied to risk analysis. Favor the explanation that makes the fewest assumptions. If a scenario requires multiple unlikely events to align just right or assumes sophisticated attackers for a very typical target, it probably belongs lower on your list. You are not trying to impress anyone with complexity. You are trying to be useful.

(Note: I wrote more about Ockham’s Razor on my blog.)

Survival Skill 5

Correct for “cool threat” bias

Security teams are uniquely talented at jumping straight to the sophisticated edge case. It is part of the job and part of the fun. But many of these scenarios are the cybersecurity equivalent of an espionage thriller. They look great on the whiteboard, but they are rarely what hurts us.

If the room is stuck, pull up your own incident logs or skim the Verizon DBIR. In most organizations, the most likely threat is not an APT. It is Sam from Accounting, who clicks on anything that looks like an invoice.

Availability bias pulls us toward the dramatic. Workshop discipline pulls us back to reality.

Survival Skill 6

Bracket and move on

Every so often a workshop takes a philosophical turn. Someone will say:

• “Nothing is truly knowable.”

• “Why model a breach at all? The world might end tomorrow.”

• “What if the real threat is something we have not imagined yet?”

Technically, none of these is wrong. They are simply not helpful.

This is where bracketing is your friend. It suspends judgment on the unresolvable questions so you can focus on what is actionable today.

My go-to line:

“You are right that anything could happen. For the purpose of this exercise, let us bracket that. Given what we know, today, what is the most plausible and decision useful scenario we can build?”

It validates the concern and keeps the session moving.

What You Do Next

Next workshop you run, use this mental checklist to keep things on track.

• Eliminate the impossible.
  Don’t debate philosophy when judgment will do.

• Prioritize the plausible.
  Possible is cheap. Attention is not.

• Favor simpler explanations.
  Start with what requires fewer assumptions.

• Watch for “cool threat” bias.
  Interesting is not the same as likely.

• Bracket what we can’t resolve here.
  Set it aside and keep moving.

• Aim attention at the risks that matter.
  The rest can wait.

When someone takes responsibility for the room, the workshop usually does what it’s supposed to do.

📚 What I’m Reading

Here are a few pieces I’ve really enjoyed the last few weeks

How to Model Enterprise Operational Risk | Graeme Keith
I treat Graeme’s pieces the same way my wife treats her Sunday New York Times crossword. When one appears in my feed, I save it for the right window so I can really think and absorb it. Thoughtful and worth your full attention.

Six Common Challenges in Cyber Risk Quantification | Prometheus Yang
A useful summary of the organizational friction points that quietly derail CRQ programs. Always helpful to see how others frame the same challenges we face.

Against the Gods | Peter Bernstein (re-read)
I revisited parts of this classic while finishing the early chapters of the book. Bernstein’s storytelling about probability, uncertainty, and the invention of risk still holds up. A rewarding slow burn.

❓ Reader Question (via LinkedIn)

“I’m comfortable doing one-off quantitative analyses, but leadership keeps asking why we don’t have a single “cyber risk number” for the whole organization. Is that a reasonable expectation, or a misunderstanding of what CRQ can actually do?”

My Answer

This question comes up a lot, especially once leadership starts to see value in quantitative analysis.

The short answer is that a single enterprise-wide cyber risk number is usually the wrong abstraction, even if the question behind it is reasonable.

What leaders are really asking is to understand scale. Are we talking about hundreds of thousands, tens of millions, or something that could threaten the business? That’s a fair question, and CRQ can absolutely help answer it.

Where things go wrong is when that curiosity gets collapsed into a single number.

In practice, cyber risk does not behave like one thing. It behaves like a portfolio of distinct loss scenarios: ransomware, data breaches, operational outages, fraud, regulatory action. Each has different drivers, different controls, different time horizons, and different decision levers. Rolling all of that into a single figure hides trade-offs, masks opportunity costs, and makes capital conversations harder, not easier.

What I’ve found works far better is a small, enterprise-wide portfolio of material loss scenarios, usually six to ten scenarios, expressed in financial terms and tied directly to real decisions. That structure makes it possible to have meaningful capital conversations: how much loss the organization could absorb, how much is transferred through insurance, and where coverage stops short.

That nuance matters, because not all cyber losses are insured, and not all policies respond the same way. Ransom payments, business interruption, regulatory fines, legal costs, and recovery expenses behave very differently under coverage. Collapsing all of that into a single number hides exactly the trade-offs leaders need to see.

If someone insists on a single number, I’m careful about how it’s framed. At best, it’s a rough summary of exposure, not a scorecard, target, or KPI. The moment it becomes something you try to optimize directly, it starts obscuring more than it reveals.

In my experience, once leaders see the portfolio view, they stop asking for a single number. Not because it’s impossible to calculate, but because they can finally see what’s driving risk, where money changes outcomes, and what trade-offs they’re being asked to make.

That’s a much more useful conversation.

✉️ Contact

Have a question about this, or anything else? Here’s how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

Elsewhere

• I share shorter thoughts on risk, metrics, and decision-making on LinkedIn.

• Book updates, chapter summaries, tools, and downloads are at
  www.heatmapstohistograms.com

• My longer-form essays and older writing live at www.tonym-v.com

❤️ How You Can Help

✅ Tell me what topics you want covered: beginner, advanced, tools, AI use, anything
✅ Forward this to a colleague who’s curious about CRQ
✅ Click the ❤️ or comment if you found this useful

If someone forwarded this to you, please subscribe to get future issues.

Thanks for reading.
—Tony

• 📖 Book Update: Book cover and pre-order links are out!

• 💡 Metrics for Thanksgiving Dinner

Hi everyone,
I have several very exciting book updates in this issue. Things are moving quickly with the book, and I’m excited to share the latest news with you all.

Here in the US, it’s Thanksgiving, my favorite holiday of the year. To celebrate, I updated one of my old blog posts from years ago on metrics for a typical American Thanksgiving meal, complete with a key risk indicator (KRIs) and several key performance indicators (KPIs). It’s a fun, lighthearted post, but the underlying message is demonstrating how to measure both tangibles and intangibles.

To those who celebrate, Happy Thanksgiving!

Tony

📖 Book Update: Cover & Pre-Order Links!

It feels so strange to make this post. This book, as a concept, notes, and blog posts to test ideas, has been rattling in my head for years, and it finally feels real to me.

When I started writing From Heatmaps to Histograms, I just wanted to make sense of cyber risk quantification; to write the book I couldn’t find anywhere else. Along the way, it became something bigger: a practical, readable guide for anyone who wants to really learn this stuff. I start from the very beginning, and we walk through a quantitative risk analysis step-by-step, one concept at a time, until we’ve built a real-life analysis together. I do remain a bit worried that the book will frustrate advanced practitioners, but I’ve sent out draft chapters as early reads to folks, and the feedback has been very positive. There is something for everyone, with techniques you won’t find in other cyber risk books.

🎉 Preorders are officially live for my book, From Heatmaps to Histograms: A Practical Guide to Cyber Risk Quantification.
(Published by Apress/Springer Nature; release date March 2026)

👉 You can pre-order it on Amazon here.

The release date on Amazon (June) is a placeholder; we’re pushing hard to get it released before the RSA Conference in late March. The price and release date will be updated as we get closer.

What’s Inside

A big portion of the book tackles the hardest problem in CRQ: data. I show you how to find, normalize, and blend internal, external, and SME data using practical tools and AI-assisted techniques. Part 3, Solving the Data Problem, spans about 100 pages and focuses on these concepts. Here’s what’s covered:

Finding and Understanding the Right Data

• How to identify the minimum viable dataset for any scenario

• The Three-Source Model (External, Internal, SME) and when each one matters

• Data you already have but don’t realize you can use

• How to find external benchmarks quickly, cheaply, and with quality checks

Vetting and Trusting Data (The Missing Skill in CRQ)

• A complete data-quality scoring method (Relevance, Verifiability, Applicability, Coverage)

• How to widen or tighten ranges depending on confidence

• How to spot bias and overfitting in vendor reports

• How to handle conflicting data without freezing or restarting

Normalizing and Transforming Evidence

• Turning messy operational logs into analyzable frequency signals

• Converting external single-point medians into three-point ranges

• Using internal telemetry to adjust industry-based rates

• When (and how) to drop unreliable data without breaking the model

Working With Subject-Matter Experts (Without Getting Garbage Inputs)

• A repeatable elicitation method: P5/P50/P95, how to run a workshop, lightning-fast calibration

• How to correct for anchoring and overconfidence

• Structured interviews that produce defensible inputs

• The “SME → Range Converter” method that turns intuition into modeling data

Blending Data Using Bayesian Reasoning (In Plain English)

• A simple process for merging external data, internal signals, and SME judgment

• How to update your belief transparently as new evidence arrives

• Examples of real-world blended frequency and magnitude inputs

• How Bayesian thinking prevents analysis paralysis and perfectionism

AI-Forward Tools and Techniques

• Using LLMs to standardize data vetting, summarize telemetry, and generate SME prompts

• Safe workflows to minimize hallucinations and maintain auditability

• AI-assisted parsing of reports, logs, audit findings, and threat intel

• Using GenAI to simulate synthetic incidents for practice and modeling

Preparing Data for Modeling

• How to structure inputs for Monte Carlo simulations

• Methods for checking logic before touching a spreadsheet

• How to document traceability so your work survives scrutiny

Why This Matters

Part 3 is the heart of the book because data, not math, is what so many people tell me is their barrier to CRQ.

And this section gives readers the one thing practitioners desperately lack:
a practical, step-by-step playbook for finding, fixing, trusting, and blending data in the real world.

Gratitude

This book wouldn’t exist without the SIRA and FAIR communities, the book’s technical reviewer Rob Brown, and the many people who pushed, challenged, and inspired me over the years. Rob’s depth of experience in decision analysis and quantitative modeling shaped some of the most important chapters in this book. His feedback was clear, candid, and always grounded in real practice; the kind that makes the work stronger, not just different. I’m grateful for his rigor, his patience, and his willingness to work through ideas with me until they held up under real-world scrutiny.

Thank you all for believing that better risk analysis is possible

This isn’t just a book about risk, it’s about decisions, data, and how we understand uncertainty.

If my work has ever helped you think differently about risk, I hope you’ll preorder a copy and help spread the word.

The Most Basic Thanksgiving Turkey Recipe - with Metrics!

I love Thanksgiving. Most cultures have a day of gratitude or a harvest festival, and this is ours. I also love cooking. I am moderately good at it, and when we host Thanksgiving, I always take on the turkey. It brings me great joy, not only because it tastes great, but because it is a genuinely hard problem. Cooking a turkey is easy. Cooking a great turkey is not.

I have gathered years of evidence from my own attempts and from watching my mother and grandmother. I treat the turkey like a high-stakes project with risk factors, mitigations, and real metrics. Metrics let me evaluate how things went and improve year over year.

Turkey Cooking Objectives

A successful Thanksgiving turkey checks four boxes:

• The bird is fully cooked and has no undercooked pockets.

• It avoids the opposite failure mode of a dry, inedible breast. The challenge is navigating the narrow corridor between raw and dry.

• It tastes good and has real flavor.

• It finishes inside a predictable window, so the turkey and the sides all hit the table together.

My Turkey Golden Rules

Brining is optional, not mandatory
People swear by wet brines, dry brines, or minimal seasoning. All can work. The only way to know what you prefer is to practice periodically throughout the year. I personally like a wet brine with salt, herbs, and spices.

Keep the cavity mostly empty
Stuffing the cavity with apples or onions smells amazing, but it slows cooking. Faster cooking helps keep the breast moist, so I recommend skipping this.

Skip basting
Opening the oven drops the temperature and lengthens cooking time. That creates more variation in the breast and thigh temperatures. Butter under the skin does more for moisture than basting.

The Most Basic Recipe

Tools

• Turkey lacer kit

• Roasting pan and rack

• Real thermometer, probe, or instant read

Ingredients

• Turkey

• Salt

• Herb butter (butter mixed with thyme, rosemary, sage or whatever you like)

Prep Work

• Thaw thoroughly. The USDA guidance is 24 hours in the refrigerator per 4 to 5 pounds.

• Preheat to 325 F.

• Remove packaging or brine bag and make sure the cavity is empty.

• Lightly salt the inside and outside. Go lighter if you brined.

• Loosen the breast skin and insert herb butter underneath.

• Brush the outside with melted butter.

• Pin the wings and tie the legs.

• Estimate cooking time. At 325 F, 13 to 15 minutes per pound is a reasonable baseline.

• Optional: add a small amount of herbs or lemon to the cavity. Not too much or it slows airflow.

• Not Optional: calibrate your oven. Accuracy matters more than people think.

Cooking

• Place the turkey in the oven.

• Halfway through, tent the breast with foil to keep it from overcooking.

• About 15 minutes before your projected finish time, start taking temperatures.

  • Innermost part of the thigh

  • Thickest part of the breast

• Target temperature: USDA says 165 F in the breast. Some cooks pull the turkey at 160 F and let it rest, since carryover heat will finish the job. I usually stick to 165 for simplicity and food safety.

• Let the turkey rest for 15 to 20 minutes before carving.

The Metrics (KPIs and KRIs)

If you are going to approach a turkey like a project, measurement is part of the fun. Metrics tell you whether the turkey hit the intended outcomes. KPIs tell you how the performance went. KRIs help you predict failures before they happen.

Here are the KPIs and one KRI I use.

KPI #1: Cooking time accuracy

The turkey should finish within plus or minus 15 minutes of your forecasted cooking time. Too early or too late means your thermometer, your oven, or your recipe was off.

KPI #2: Undercooked areas

This is a binary metric. If any slice reveals pink or translucent meat, the KPI fails. Either the thermometer was off, you measured the wrong place or the bird was still partially frozen.

KPIs for the subjective quality of the turkey

Intangibles can be measured using observable signals from your guests. The next 4 KPIs measure guest sentiment.

KPI #3: Percentage of people getting second helpings

Some guests always get seconds and some never do. Compare to your historical baseline. If fewer than about 20 percent of guests get seconds, moisture or flavor were off target.

KPI #4: Percentage of people overusing gravy

Gravy is a masking agent for dry turkey. If more than about 40 percent of people are drowning their plates, the turkey is on the dry side. Adjust your threshold based on how gravy obsessed your family is.

KPI #5: Percentage of kids refusing to eat it

Kids under 10 do not hide their opinions. If half of them will not take a second bite, the turkey is dry, bland or both.

KPI #6: Leftover disposition

Great turkey gets eaten cold from the fridge. Mediocre turkey becomes soup. Bad turkey gets thrown out after a few days. If more than about 60 percent of your leftovers convert to soup or trash, the outcome missed the mark.

KPI 7: Oven Temperature Stability

This metric measures how much the oven’s actual temperature fluctuated around the target of 325°F during cooking. Even well-calibrated ovens drift, and that variability affects cooking time, moisture, and predictability. This KPI captures how stable the heat source was throughout the roast.

Predictive KRI: October sentiment check

If in late October more than 50 percent of your household says things like “Let’s just order Chinese this year” or “Maybe we can keep it simple,” that is a valid early warning indicator. Past performance influences stakeholder confidence.

Final Thoughts

Adjust thresholds based on your family’s preferences. Your KPIs will look different if you have gravy lovers, leftover hoarders or children who only eat food shaped like nuggets. The goal is not perfection. The goal is a predictable, enjoyable, low risk Thanksgiving.

Wishing you a delicious and successful holiday.

✉️ Contact

Have a question about this, or anything else? Here’s how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

❤️ How You Can Help

✅ Share your questions or feedback in the comments below
✅ Forward this to a colleague
✅ Click the ❤️ if you liked this issue

Thank you for reading. This Thanksgiving, I’m grateful for the support this community has given me.

—Tony

• 📋 Book update: Scenario building and measurable scope

• 🧠 Three ways to quantify never-happened risks

• 📗 Book excerpt: Handling “impossible” scenarios in workshops

• ✉️ Contact and how you can help

Hey there,

Welcome to Issue 5! If you’ve ever been asked to quantify a risk your organization has never experienced, you know that sinking feeling. No internal data. No precedent. Just a blank Word doc and a decision-maker waiting for an answer.

This issue gives you the framework. It won’t make the work easy, but it will make it possible.

📖 Book Update: The Art of Keeping Scope Tight

I just finished up a rewrite of the scenario-building chapter, and it is turning into one of the most important parts of the entire book. This isn’t about dreaming up dramatic threat stories. It is about creating scenarios that keep stakeholders focused and give analysts something they can actually measure.

A beautifully crafted scenario is worthless if you cannot quantify it. The entire book builds from this foundation. Once we define a clear, measurable scenario, the next chapters show exactly how to estimate each component.

This is where risk analysis turns from speculation into numbers that support real decisions.

A Cover, A Reviewer, and Some Gratitude

Last week I spent a few hours going through cover image options with the publisher, and we picked a great one. I can’t share it yet, but I love it!

I’m incredibly grateful for the team at Apress for all their support throughout this process. Special thanks to Robert Brown, who is serving as the book’s technical reviewer. You might know him from his fantastic book, Business Statistics with R. He’s been an invaluable sanity check on everything I’m writing.

The Post-it Note Philosophy

I posted this on LinkedIn a few weeks ago, and it encapsulates my philosophy behind my writing process.

There’s a Post-it note on my monitor that says: “Don’t melt the reader’s brain.”

This simple reminder drives every page I write in my new book on cyber risk quantification.

Einstein said it best: “If you can’t explain it simply, you don’t understand it well enough.” In cybersecurity risk, we’ve become so accustomed to complexity that we’ve forgotten this fundamental truth.

My book “From Heatmaps to Histograms” takes the Lego block approach:

→ Start with fundamentals (coin flips, basic probability)
→ Build one concept on the last
→ No brain-melting allowed

I’ve found that if you break complex ideas into small enough pieces, anyone can understand them. I learned this the hard way, struggling through textbooks at my kitchen counter until probability finally clicked.

Writing this book taught me that the biggest breakthrough in cyber risk isn’t more sophisticated math. It’s making quantitative thinking accessible to practitioners who need it most.

I’m constantly looking at that Post-it note, reminding myself who I’m writing for: it’s me 15 years ago, and everyone else who’s been there too.

🧠 How to Quantify What You’ve Never Seen: A Taxonomy of Unknown Risks

A Practical Taxonomy for Risks We Haven’t Lived Yet

Most risk programs live comfortably inside the obvious: phishing, ransomware, and BEC. But what about the risks that haven’t happened to you, or to anyone? This post explores how to think and model at the edge of experience, where data ends.

This idea surfaced while I was writing the risk scenario chapter of my book. That chapter focuses on the obvious: phishing, ransomware, business email compromise, the kinds of risks that, in most organizations, happen most often.

But as I wrote, a question kept lingering in the margins. What about the other kinds of risk, the ones that haven’t happened yet? Do we have a place for them?

We do, but it’s harder to get there. That’s the edge of scenario thinking, where imagination meets quantification and where the data ends, but the analysis doesn’t.

Every risk analyst eventually faces the same uncomfortable question: How do we quantify a risk that’s never happened to us?

It’s one of the most complex problems in our field because it exposes a truth we don’t often say out loud. We’re expected to be rigorous and data-driven, yet much of our work extends beyond firsthand experience.

We model what we’ve seen: incident tickets, logs, audit findings, but we also have to model what we haven’t seen: a supply chain attack that’s hit peers but not us, an AI-driven threat that hasn’t materialized yet, or a failure type no one has imagined.

These aren’t the same analytical problem. Treating them as if they are is where many risk programs go wrong. A risk that’s happened to others but not to you demands a very different approach than one no one has experienced, which again differs from one no one can even conceive of.

Through experience, I’ve come to see these as three distinct levels of “never happened” risk, each requiring its own mindset and method.

The Three Levels of “Never Happened” Risk

1. Unexperienced but Observable

(It hasn’t happened to us, but it has to others.)

This is the most common and the easiest to quantify. These are the zero-event risks in your organization, scenarios that exist elsewhere in the world but haven’t touched you yet.

Your stance here is empirical. Treat the world as your dataset. Pull base rates from peers, incident databases, or industry research. Adjust for your own context, such as size, control maturity, and exposure, and you can build a defensible range.

When SolarWinds was disclosed in 2020, many companies hadn’t been hit but were clearly exposed. We built models using published incident counts, vendor integration data, and internal telemetry to estimate local probability and impact. The result wasn’t perfect, but it was grounded in evidence.

The mistake is assuming that because it hasn’t happened here, it can’t happen here.

Key mindset: the absence of local evidence doesn’t mean the risk is zero.

2. Unrealized but Conceivable

(It hasn’t happened to anyone yet, but we can describe how it could.)

This is the analytical frontier. Plausible, but unobserved. Examples include quantum decryption of current algorithms or AI-generated polymorphic malware that learns as it attacks.

Your stance here is analogical. You reason by pattern and proximity. Look at adjacent technologies or transitions: how quickly did similar systems in the past move from secure to compromised? Use expert elicitation to fill data gaps, but structure it using calibrated ranges rather than gut-check opinions.

In 2013, I modeled the risk of blockchain wallet compromise before any major exchange hacks had occurred. I used online banking fraud frequencies as a proxy, then adjusted for immaturity and control gaps. A year later, the first major breaches confirmed the model's direction.

The trap here is cherry-picking analogies that confirm fear or bias. The antidote is breadth: collect multiple comparables, build a distribution, and reason from the middle.

Key mindset: analogical reasoning isn’t guessing; it’s disciplined extrapolation.

3. Inconceivable and Unimaginable

(We can’t yet describe it because we can’t yet imagine it.)

This is the domain of true surprise, the “how did no one see this coming?” events. Before 9/11, few imagined coordinated hijackings as guided missiles. Before Stuxnet, few imagined air-gapped industrial systems being sabotaged via USB. This is the core subject of Taleb’s book, The Black Swan.

You can’t model the probability of something you can’t conceive, but you can model your resilience to it.

Your stance here is architectural. Focus on designing systems that can absorb shocks and adapt. Build redundancy, strengthen detection, and practice incident response. Model impact categories rather than specific causes, such as availability, integrity, confidentiality, productivity, fines, and reputation.

When you can’t quantify the odds, quantify your capacity to recover.

Key mindset: when you reach the edge of prediction, move to resilience.

Universal Lessons

No matter which level of “never happened” risk you’re dealing with, three principles hold true.

1. Start with the decision. Know what choice the analysis is meant to inform. If the decision wouldn’t change whether the probability is 0.1% or 1%, stop polishing decimals.
   

2. Use structured expert judgment. Calibrate experts with seed questions, capture ranges (P5, P50, P95), and reward accuracy, not seniority.
   

3. Measure when more precision matters. If perfect information wouldn’t change the decision, it’s not worth chasing.

The Deeper Lesson

The most dangerous risks aren’t always the unimaginable ones. They’re the obvious ones we’ve seen elsewhere, but we convince ourselves that they don’t apply.

What makes great risk analysis isn’t omniscience, it’s clarity. Knowing what kind of “never happened” risk you’re facing tells you which tools will work and which will waste time.

Quantification doesn’t end where experience stops. It just changes form.

📗 Book Excerpt: The Philosophy of “Impossible” Risk Scenarios

“You just said a tsunami can’t destroy a datacenter in Omaha. What about a mega-tsunami?”

If you have ever run a risk brainstorming workshop, you know how quickly the conversation can drift into philosophical debates about what is truly possible.

I’ve been in this meeting many times. We're talking about website outage scenarios, and suddenly, we are discussing asteroid impacts and mega-tsunamis instead of the actual decisions a CISO needs to make next quarter.

This piece is about getting the room back on track. It explains why we bracket remote scenarios and focus our limited time and attention where it matters. I ended up cutting and condensing this section for the chapter, but it stands well on its own. I will share a couple of the other trimmed pieces in the next issue.

This pushback on absolute claims is philosophically valid. When analysts say something is “impossible,” they’re not claiming logical impossibility (like a square circle) or even physical impossibility (like faster-than-light travel).

They’re talking about practical impossibility: scenarios so remote that modeling them would waste resources better spent on actionable risks. This connects to practical reason: cognitive and analytical resources are finite, so analysts must focus on possibilities that could meaningfully inform decisions.

As philosopher William James noted, beliefs should be judged by their practical consequences. A “mega-tsunami reaching Omaha” scenario might be theoretically possible via Earth-altering events (e.g., planetary impact, complete crust displacement) but it fails the pragmatic test: it won’t help a CISO allocate next year’s security budget.

Risk analysis is an exercise in practical wisdom (what Aristotle called phronesis), not exhaustive enumeration of every conceivable threat. Analysts set aside (called bracketing) extremely remote possibilities not because they can prove they’re impossible, but because doing so helps them focus on scenarios that actually matter for business decisions.

📚 What I’m Reading

• Curiosity Didn’t Kill the Cat by Erin Eilers

  A reminder that great risk identification starts with great questions. AI can surface patterns, but human curiosity reveals the blind spots.

• Outside In and Inside Out Superforecasting by Rick Howard

  A great reminder that even strong models need human judgment. Forecasting is not about precision for its own sake. It is about making decisions with the uncertainty we have.

✉️ Contact

Have a question about risk analysis, or have a general question? Here’s how to contact me:

• Reply to this newsletter if you received it via email

• Comment below

• Connect on LinkedIn

• Contact form

What specific risk analysis challenges are you facing? Hit reply and let me know.

❤️ How You Can Help

✅ Tell me what topics you want covered: beginner, advanced, tools, AI use, anything
✅ Forward this to a colleague who’s curious about CRQ
✅ Click the ❤️ or comment if you found this useful

If someone forwarded this to you, please subscribe to get future issues.

— Tony

• 📖 Book Update: Part 2 Complete

• 🎯 The First Thing I Do When Someone Asks for a Risk Analysis

• 💡 Risk Analyst Skills Evolution

• 💬 Reader Question: Which skills do you think will matter most for risk analysts in the next five years?

Hi!
I’m still experimenting with shorter, more frequent newsletter issues. Please let me know your thoughts. Drop a comment below, or just like this post.

📖 Book Update: Part 2 Complete

Part 2 sets the foundations for quantitative risk analysis, demystifies the Monte Carlo method, and walks readers through what may be their first quantitative risk analysis. I’m writing the book out of order, tackling the hardest chapters first. It’s 20 chapters, and I have 14 done. Getting close!

Here’s a quick snapshot of what Part 2 covers.

Chapter 4: Foundations

Building the Right Mindset for Risk Assessment

This chapter establishes the core philosophy that risk assessment exists to support better decisions, not create compliance theater. Key concepts include adopting a "less wrong" mindset that acknowledges all models are imperfect yet useful, understanding that uncertainty is valuable information rather than a flaw, and learning essential vocabulary, such as the distinction between frequency and probability. The chapter emphasizes starting small with one decision and one scenario rather than trying to revolutionize everything at once.

Chapter 5: Your First Quantitative Risk Assessment

From Theory to Practice with Monte Carlo

Chapter 5 walks readers through their first hands-on quantitative assessment using a practical example of forecasting lost or broken mobile phones. It introduces the fundamental concepts of frequency (how often bad things happen) and magnitude (how much they cost), then combines them using Monte Carlo simulation in Excel. This chapter bridges the gap between theory and practice, showing how to build confidence through a simple, real-world exercise that demonstrates the basic risk equation in action.

Chapter 6: Interpreting and Communicating Results

Turning Numbers into Clear Risk Stories

This chapter focuses on what to do with quantitative results once you have them. It teaches essential statistical concepts like mean, median, and percentiles, then introduces key visualization tools including histograms, box plots, and loss exceedance curves. The chapter also provides practical guidance on communicating findings to executives while avoiding common reporting pitfalls.

Pre-order links are coming any day now. Check www.heatmapstohistograms.com

The First Thing I Do When Someone Asks for a Risk Analysis

When someone asks me to “do a risk analysis,” the first thing I try to do is talk myself out of it.

That may sound strange. Risk analysis is what I do, and I have built my career around it. I have also learned the hard way that risk analysis is not always the right tool. Sometimes the most helpful thing I can do is pause and ask a few questions before I ever start modeling.

A Painful Lesson

Early in my career, I ran what I thought was an excellent risk assessment. I spent weeks pulling data, interviewing stakeholders, and running simulations. When I finally presented the results, the decision maker looked at me politely and said, “This is interesting, but we already decided what we are going to do.”

That moment stuck with me. My analysis was technically solid but practically useless, because the decision had already been made. I had answered a question nobody was asking.

Ever since, I have developed a habit: when someone requests a risk analysis, I first try to talk myself out of it.

The Hammer and the Nail

There is a bias I try to keep in check. Psychologists call it the Law of the Instrument, sometimes referred to as Maslow’s Hammer. If all you have is a hammer, everything looks like a nail. For me, risk analysis is that hammer. It is powerful. However, not every problem is a nail.

Before I swing it, I ask: What problem are we really trying to solve?

My Field Notes: If You Are Asking for That… Try This Instead

Over time, I have built a giant notebook filled with scribbles on the alternatives to risk analysis. In other words: “if you ask for this, what you may need is this.” Here are some of the most common ones I see:

• Document a decision you already made → Decision log

• Could this threat group break in? → Threat modeling or red team

• What’s the consequences if a system goes down? → Business impact analysis or continuity plan

• What happens if ransomware hits us? → Tabletop exercise

• Which vulnerabilities should we prioritize? → Vulnerability management and threat intel

• Are we compliant with regulation X? → Gap assessment or audit

• Prove the program is effective → Metrics and KPIs

• Where are we most exposed? → Attack surface mapping

• Compare vendors → Third-party risk management

• Forecast budget needs → Scenario planning or financial modeling

• Help us choose between two strategies → Decision matrix or Multiple-criteria decision analysis

• Where are our process weaknesses? → SWOT or root cause analysis

• Justify an initiative to executives → Business case

• Estimate the cost of an incident after the fact → Post-incident review

• Show how we compare to peers → Benchmarking

These tools are not dead ends. They are on-ramps. A tabletop may surface uncertainty about the probability and magnitude of ransomware losses. A business case may expose unclear alternatives or competing preferences. A compliance gap assessment may highlight trade-offs between risk reduction and business friction.

Each of these is an opportunity to turn the conversation into a quantitative risk analysis once the uncertainty is clear and the decision is tied to objectives.

The real payoff is cooperation. Working with continuity teams on a BIA, with compliance on a gap assessment, or with finance on scenario modeling creates shared language. It shows security is not just a critic on the sidelines, but a partner helping the business connect the dots. That is how we expand influence and make risk analysis something the whole organization values.

When a Risk Analysis Is Warranted

A risk analysis is justified only when a decision is made that matters, tied to organizational objectives, and faces material uncertainty about a future event whose probability and impact can be estimated well enough to inform action.

That means several conditions need to be true:

1. Clear decision statement – The problem can be framed as a real choice between alternatives (“Should we do A, B, or C?”).

2. Link to objectives – The decision is directly connected to the organization's goals, mission, or priorities.

3. Decision owner – Someone is accountable for making the call and acting on the results.

4. Uncertainty that matters – There is genuine uncertainty about the outcome, and reducing it would influence the decision.

5. Defined preferences – We understand what matters in choosing between alternatives (cost, uptime, safety, growth).

6. Meaningful information – We have, or can generate, enough data to credibly estimate probability and impact.

7. Actionability – The results will drive behavior or resource allocation, not sit on a shelf.

8. Stopping rule – We know when additional analysis no longer changes the decision.

If a request does not pass these tests, it usually belongs in one of the alternative tools above. If it does, then a risk analysis can truly add value.

When Risk Analysis Really Matters

When those tests are satisfied, that is when risk analysis shines.

• It is the right tool when we need to allocate resources and compare competing investments.

• It is the right tool when we are setting risk tolerance and need to understand plausible loss ranges.

• It is the right tool when we are evaluating a high-stakes initiative like a cloud migration or an acquisition.

• It is the right tool when leadership asks, “If I give you $X, how much risk does it reduce?”

• It is the right tool when we want to see how risks add up across the enterprise.

• It is the right tool when we are negotiating insurance.

• It is the right tool when we need to show the board not just what might happen, but how likely and how big.

Those are the moments when probabilistic modeling earns its keep.

The Discipline of Saying No

Trying to talk myself out of a risk analysis is not a sign of cynicism. It is discipline. It forces me to check decision quality, ownership, and alignment with organizational priorities. It helps me resist the hammer-and-nail bias. It also ensures that when I do deliver a risk analysis, it answers a real question that matters.

The result is sharper insights, stronger cooperation across teams, and decisions that leaders are willing to act on.

That, to me, is the craft.

The Risk Analyst Skills Evolution

I recently gave a talk at the Society of Information Risk Analysis (SIRA) annual conference, SIRAcon 2025. My session was titled “Quantifying in the Age of Hallucination: How I Learned to Stop Worrying and Trust the AI (Sometimes).” It explored how AI is changing the way we do risk analysis, what it is good at, where it is dangerous, and how our skills as analysts are evolving.

If you are not a member of SIRA, you should consider joining. It is a great community focused on advancing the practice of risk analysis. If you are already a member, the video of my talk should be up soon.

When I think about the future of our field, this is the slide that keeps me up at night.

AI is already handling a lot of what used to be core to our jobs. It excels at routine calculations, summarizing reports, creating first drafts, building simple visualizations, and even assisting with compliance checks. These are things I used to spend hours on earlier in my career, but now I can ask an AI tool to take a first pass, and it often does a decent job. The time savings are real, but it also makes me pause. If AI is doing this now, what will be left in five years?

The skills that are rising in value are the ones that are harder to automate:

• Critical thinking and synthesis

• Framing risk in a strategic context

• Deciding which risks actually matter

• Influencing decision makers

• Bringing in ethical judgment

These are not just nice to have; they are the very things that will define the best risk analysts in the future.

In my own work, I already see this shift. Two years ago, I would spend a full afternoon finding peer company incident data for use in an analysis. Today, I let AI do the research, and then I spend my time refining the narrative and making sure the analysis actually connects to the business decision at hand. The AI is faster at the chart, but it cannot decide which story matters. That is still on me, and that is where my real value lies.

One other point I made in the talk is about prompt engineering. Right now it feels like an essential skill. Knowing how to craft the right prompt can make the difference between a useful output and complete nonsense. But we should not fool ourselves into thinking prompt engineering will remain a differentiator for long. Models are already starting to generate and refine their own prompts. AI will eventually take this work out of our hands. For now it is important, but soon it will fade into the background.

The bottom line: risk analysts who cling to tasks that are being automated will find themselves automated out. Risk analysts who invest in human differentiators like judgment, synthesis, and influence will thrive.

AI will not replace risk analysts, but risk analysts who use AI effectively will replace those who do not.

Reader Question

✉️ Contact

Have a question about this, or anything else? Here's how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

❤️ How You Can Help

✅ Share your questions or feedback in the comments below
✅ Forward this to a colleague
✅ Click the ❤️ if you liked this issue

Thank you for reading, and remember: quantitative risk analysis is a valuable tool, but not the only tool.

—Tony

• 📖 Book Update

• 🎯 A Practical Introduction to Risk Measurement

• 📚 What I’m Reading

Hey there,

This issue is focused on one core technique, with immediate applications you can try this week, instead of several ones.

Let me know what you think of this format. I'm genuinely curious whether this hits better than the longer, packed emails.

📖 Book Update: Part 1 Complete!

A big milestone this month: Part 1 of "From Heatmaps to Histograms" is now complete and has been submitted to the publisher. Additionally, I'm more than halfway through the first drafts of the entire book. Here's what's in the can:

Part 1 covers everything, from my embarrassing "red/yellow/green" moment at a San Francisco bank (while other teams spoke in actual dollars) to why our industry inadvertently created two completely different things, both called "risk management."

The best part might be Chapter 3 on GenAI; current AI is more like Jack Sparrow (charming but needs boundaries) rather than Data from Star Trek (perfectly trustworthy) and proper AI guardrails for use in risk analysis.

Next up: Part 2 gets into the actual step-by-step techniques you need to do this work.

Timeline update: Still targeting early 2026 for publication, with pre-orders likely available in a few weeks.

Saving the best for last…

The book’s website is up! Check it out: www.heatmapstohistograms.com

🎯 A Practical Introduction to Risk Measurement

If you're learning cyber risk quantification (CRQ), you've probably heard this before: "You can't measure cyber risk. It's too uncertain, too complex, too subjective."

This mindset stops many risk professionals from even trying quantitative approaches. However, we simply need to apply concepts from metrology, the science of measurement, and techniques from decision science, as well as other relevant disciplines, to this problem.

Measurement is a core discipline utilized across various fields, including science, medicine, finance, and engineering. Astronomers calculate the mass of planets they'll never touch. Epidemiologists estimate infection risks across populations they can't fully observe. Engineers estimate failure rates for systems with millions of components. None has perfect information; all work with uncertainty ranges and probabilistic thinking.

The same techniques work for cyber risk. The goal isn't precision for its own sake; it's moving from "we know nothing" to "we know enough to make better decisions."

The Challenge: From LinkedIn Skepticism to Practical Measurement

Here’s the inspiration for this issue. I recently saw this post on LinkedIn that perfectly captures measurement skepticism:

"The advocates of quantitative risk analysis point to sophisticated math, but ultimately how do you measure the likelihood of any given adverse event? It's a judgement call. How likely is it your phone will get hacked at DEFCON? ...oh...somewhere between 0 and 100 percent depending on who you ask. The quality of the SMEs matter more than the tools they use."

Challenge accepted! Let's demonstrate core cyber risk quantification (CRQ) techniques using this real-world scenario and watch our estimate evolve from complete ignorance to actionable insight.

What Is Fermi Estimation?

Before diving into techniques, understand our core approach: Fermi estimation. Named after physicist Enrico Fermi, this involves making reasonable estimates about seemingly impossible-to-measure quantities by breaking them into smaller, more estimable parts.

Classic example: "How many piano tuners are in Chicago?"

• Chicago population: ~5 million people

• Average household size: 2 persons per household

• Households: ~2.5 million (5M ÷ 2)

• Households with pianos: ~5% (1 in 20) = 125,000

• Tuning frequency: once per year = 125,000 tunings annually

• Tuner productivity: ~1,000 tunings per year (50 weeks × 5 days × 8 hours ÷ 2 hours per tuning)

• Result: ~125 piano tuners

In the famous Chicago piano tuner problem, each step is estimated using data, logic, and reasoning. The goal isn't precision, it's moving from "impossible to know" to "reasonable ballpark estimate" using structured reasoning. We'll apply this same approach to the phone hacking question, and the same principles are often used to measure cyber risk.

Foundation: The Measurement Mindset

If someone claims something can't be measured, recall Douglas Hubbard's Clarification Chain from "How to Measure Anything”:

• If it matters, it's detectable

• If it's detectable, it can be measured as a range

• If it can be measured as a range, it can be quantified

Core principles for any risk measurement:

• Embrace uncertainty: Every measurement has error; our job is credible ranges, not false precision

• Make judgment explicit: Risk analysis makes assumptions transparent so others can challenge them

• Think Bayesian: Form initial beliefs, then update systematically as evidence emerges

Starting Point: Complete Ignorance

Current estimate: 0-100% (All possibilities equally likely)

The LinkedIn post claimed "0–100%" probability: complete ignorance where all possibilities are equally likely. This flat prior isn't a prediction; it's an acknowledgment that we have no information. Every outcome from "impossible" to "certain" is treated as equally plausible.

Technique #1: Precise Scoping (Define What You're Measuring)

The Problem: Vague definitions create meaningless measurements. "Phone hacking" could mean anything.

The Technique: Start every risk assessment by precisely defining the adverse outcome you're measuring.

Application: "Phone hacking" could mean various things. For this analysis, our scope includes device-based attacks that result in unauthorized access to data or device functionality.

In Scope:

• Persistent malware installation: Ongoing device control or monitoring

• Data theft: Accessing photos, contacts, files, messages without ongoing access

• Privacy violations: Location tracking, microphone/camera access, credential harvesting

• Temporary device control: Remote command execution, device manipulation

Out of Scope:

• WiFi traffic eavesdropping and credential capture from unencrypted protocols (Wall of Sheep scenarios). These represent insecure communication, not device compromise

Current estimate: 0-100% (Still complete uncertainty, but now we know what we're measuring)

Scoping doesn't narrow our range yet, but it's crucial foundation work. We've defined our target precisely, which prevents scope creep during analysis.

Key Principle: Always ask "probability of what, exactly?" before starting any risk measurement. Vague definitions guarantee useless results.

Technique #2: Eliminate Absurd Extremes

The Problem: Starting with "0-100%" is saying “we know knowing!” and provides no decision value. But, we do know something, don’t we?

The Technique: Use logical reasoning and available evidence to systematically rule out impossible or highly implausible values before detailed calculation.

Test the extremes systematically:

100%: If every phone among ~30,000 attendees got compromised, it would be international headlines and kill the conference. People go to jail. No hotel or convention center would dare to host it. Clearly absurd.

75-90%: 22,500+ device compromises would be the biggest cybersecurity story of the decade. Still absurd.

50%: 15,000 victims would generate a massive industry response and likely lead to changes or the cancellation of the conference. Extremely unlikely.

25%: 7,500 compromised devices would create an unprecedented security industry crisis. Very unlikely, but starting to enter the realm of possibility.

10%: 3,000 incidents would still generate major headlines and community outcry. Possible, but would be widely reported.

5%: 1,500 compromises would generate some industry discussion. Getting into plausible territory.

On the other end of the range:

0%: DEF CON represents a unique high-threat environment with demonstrated attacks. Not zero risk.

Updated estimate: 0.5-5% (90% confidence interval)

Confidence change: We've eliminated 90% of the original possibility space using pure logic. This is the power of logical thinking: most of human intuition about "impossible" vs "possible" is actually quite good.

Key Principle: Start every risk estimate by ruling out the obviously absurd answers. This technique applies to any uncertain scenario, from data breach likelihood to business continuity risks.

Technique #3: Threat Modeling with Attack Vectors

The Problem: "Phone hacking" is still too vague to estimate precisely. We need specific pathways.

The Technique: Enumerate concrete attack vectors, then estimate each separately.

Credible attack vectors at DEF CON:

1. Cellular interception (IMSI catchers): Rogue cell towers demonstrated at conferences

2. Malicious WiFi networks: Rogue WiFi at DEF CON

3. Physical attacks (BadUSB): Modified charging cables demonstrated at DEF CON 27

4. Social engineering: Malicious QR codes, fake apps, manipulation tactics

5. Zero-day exploits: Premium mobile exploits worth $15-20 million

6. Bluetooth exploits: Proximity-based attacks

Current estimate: 0.5-5% (Range unchanged but confidence increased)

Threat modeling doesn't narrow our range much, but it significantly increases our confidence by grounding the analysis in concrete attack scenarios rather than vague "hacking" concepts.

Key Principle: Always decompose vague risks into specific attack scenarios. "Data breach" becomes "phishing → credential compromise," "malware → lateral movement," etc. This makes the seemingly immeasurable measurable.

Technique #4: Pipeline Analysis (Fermi Decomposition)

The Problem: Complex events seem impossible to estimate because so many things must align.

The Technique: Model the event as a pipeline where multiple stages must all succeed, then multiply the conditional probabilities.

Pipeline stages for device compromise:

Stage 1: Sophisticated Attacker Presence (60-80%)

Question: Are sophisticated attackers actively targeting mobile devices with premium tools?

Key insight: DEF CON isn't random people. It's an exceptional concentration of high-value targets:

• Fortune 500 CISOs and security executives

• Government cybersecurity officials

• Security researchers with unreleased intelligence

• Critical infrastructure employees

Economic reality: A $15-20 million zero-day investment could yield intelligence worth hundreds of millions from the right targets.

Stage 2: User Vulnerability Exposure (20-40%)

Question: Do targets engage in risky behavior when attacks are active?

Even security-conscious professionals take some risks over a 4-day conference exposure: experimental culture, social pressure to participate, and inevitable human errors.

Stage 3: Technical Attack Success (2-8%)

Question: Do attacks overcome modern mobile defenses?

Our broader scope includes temporary access and data theft (easier than persistent compromise). But modern mobile security is substantial, and time constraints limit attackers.

Pipeline calculations (Fermi estimates):

• Conservative: 60% × 20% × 2% = 0.24%

• Moderate: 70% × 30% × 5% = 1.05%

• Aggressive: 80% × 40% × 8% = 2.56%

These represent my Fermi estimates for each stage based on available evidence and structured reasoning about DEF CON's unique environment.

Updated estimate: 0.24-2.6% (Major narrowing through decomposition)

Confidence change: Pipeline analysis dramatically narrows our range by breaking an impossibly complex question into three estimable components. This is where structured reasoning really shows its power.

Key Principle: Use pipeline analysis whenever measuring complex events that require multiple conditions. Supply chain attacks, insider threats, ransomware incidents all benefit from this decomposition approach.

Technique #5: Detection Bias Analysis (Reality Check)

The Problem: My pipeline analysis assumed successful attacks would be detected and reported, but sophisticated mobile compromises are designed to be undetectable. It was at this stage that I realized the upper bound of the estimates so far did not account for this, and I need to correct this. I need to raise the upper bound slightly.

The "Dark Figure" Challenge:

• Advanced persistent threats remain hidden for months/years

• Even security professionals may not detect sophisticated compromises quickly

• Nation-state actors specifically design stealth attacks

Why the upper bound increases: Pipeline analysis gave us estimates based on detected incidents only. But sophisticated mobile compromises are specifically designed to avoid detection. Detection bias analysis asks: "What if many successful attacks remain completely hidden?"

The logic:

• Pipeline analysis: "Based on visible evidence, attacks succeed 2.6% of the time"

• Detection bias: "But what if we're only seeing 70% of successful attacks? Then the true rate could be 2.6% ÷ 0.7 = 3.7%"

Why this is methodologically sound: We're not changing our assessment of attack mechanics (pipeline stages remain the same). We're adjusting for measurement limitations - the possibility that our "no widespread reports" evidence reflects successful concealment rather than low attack rates.

Detection bias adjustment: Apply a modest multiplier for undetected incidents while respecting bounds analysis:

• Conservative: 0.24% × 1.5 = 0.36%

• Moderate: 1.05% × 2 = 2.1%

• Aggressive: 2.56% × 1.5 = 3.84%

My analytical assessment of how detection limitations affect our estimates, constrained by earlier bounds analysis, suggests that anything approaching 5%+ would generate a visible community response.

Updated estimate: 0.36-3.84% (Upper bound increases due to measurement limitations)

Key Principle: Sometimes ranges should widen when you realize your measurement approach has limitations. Detection bias is a reality check that says "our estimates could be too low if attacks are more stealthy than we assumed."

Always consider whether your risk measurement depends on incident detection and reporting. Many cyber risks suffer from significant underreporting bias.

Technique #6: Conditional Risk Modeling (Behavior Matters)

The Problem: Treating all attendees identically ignores the biggest risk factor, individual behavior.

The Technique: Create distinct risk profiles based on key behavioral variables, while respecting the logical bounds established earlier.

Risk profiles with detection bias included (capped at bounds analysis limits):

Cautious Attendee: 2G disabled, burner phone, minimal connectivity

• My estimate: 0.1-1%

Curious Attendee: Standard security practices, some experimental behavior

• My estimate: 1-3%

Reckless Attendee: Primary device, connects freely, ignores warnings

• My estimate: 3-5%

Current estimate: 0.1-5% depending on behavior (Maintains logical consistency with our bounds)

Why this makes sense: I can't simultaneously claim that 5%+ would generate visible community discussion (bounds analysis) and some attendees face higher risk without contradicting my own reasoning.

Key Principle: Always identify the key behavioral or environmental variables that drive risk differences. Create distinct profiles while respecting constraints from earlier analysis.

Technique #7: Bayesian Updating (Evidence Integration)

The Technique: Treat risk estimates as living hypotheses that update systematically based on new data.

Evidence assessment:

• Limited public reports: Could indicate either low rates or successful stealth

• Modern mobile security: Continues to improve, but is countered by sophisticated adversaries

• DEF CON's unique environment: Justifies higher estimates than normal mobile security statistics

Bayesian update: Current evidence supports the moderate end of our range.

Current estimate: 0.5-4% for typical attendees (Evidence-weighted)

Critical Reality Check: The Empirical Evidence Problem

Our systematic analysis suggests meaningful device compromise rates, but there's a glaring empirical problem: the security community isn't talking about widespread device compromises at DEF CON.

The 5-year silence problem: If even our lower estimates were accurate, we'd expect some community discussion among security professionals who actively share threat intelligence and study these exact threats.

Why the silence doesn't necessarily invalidate our estimates:

1. Detection bias is even more severe than we thought: Truly sophisticated nation-state attacks remain completely hidden even from security professionals. Modern mobile APTs are designed for multi-year persistence without detection.

2. Incidents occur but aren't discussed openly: Classification concerns, legal issues, or professional embarrassment prevent sharing. Government officials and corporate executives might be specifically instructed not to disclose incidents.

Honest final assessment: 0.1-4% with low confidence

• Lower bound (0.1%): ~30 incidents annually - explains the community silence

• Upper bound (4%): ~1,200 incidents annually - possible if detection bias and non-disclosure are severe

• Low confidence: The wide range reflects fundamental uncertainty

Final Estimate Progression Summary

Here's how our estimate evolved through systematic analysis:

Key Takeaways: The Realistic Middle Ground

It's not an urban myth, but it's also not inevitable.

1. Device compromise CAN happen at DEF CON - We have credible attack vectors, high-value targets, and sophisticated adversaries. The analysis suggests 0.1-4% individual risk, which is meaningfully above zero.

2. Historical attacks were mostly legacy exploits - The famous demonstrations involved 2G/3G cellular vulnerabilities that modern devices can now disable. Many classic attack vectors have been significantly hardened.

3. Wall of Sheep creates false perceptions - Hundreds of credential captures annually at the Wall of Sheep represent WiFi eavesdropping, not device compromise. People conflate seeing "lots of hacking" with device takeover, but these are fundamentally different threats.

4. The "100% probability" claim is nonsense - Our systematic analysis moved from 0-100% (useless) to 0.1-4% (actionable). Even in a worst-case scenario, most attendees' devices remain uncompromised.

5. Detection bias could explain the silence - The security community's lack of discussion about widespread device compromises over the last 5 years suggests either: (a) actual rates are toward the lower end of our range, or (b) truly sophisticated attacks remain completely hidden.

6. Behavior matters enormously - Cautious attendees (burner phones, disabled connectivity) face much lower risk than those who connect freely to conference networks and experimental services.

The bottom line: DEF CON device compromise is a real but relatively rare risk that's often confused with much more common eavesdropping on unsecured connections. It's worth taking basic precautions, but not worth avoiding the conference entirely.

For risk professionals: This demonstrates how structured reasoning can cut through both "it's impossible to measure" and "it definitely happens to everyone" extremes to reach a nuanced, evidence-based middle ground.

Techniques You Can Apply Tomorrow

For any "unmeasurable" cyber risk scenario:

1. Start with bounds: Before any analysis, rule out absurd extremes using basic logic

2. Define scope precisely: "Data breach" isn't specific enough. Breach of what, leading to what outcome?

3. Use pipeline decomposition: Break complex events into stages (Attacker → Access → Impact)

4. Make it conditional: Different behaviors/controls should yield different risk estimates

5. Account for detection bias: Consider what you might not be seeing

6. Update with evidence: Treat estimates as hypotheses that improve with new data

7. Reality check: When estimates seem implausible in absolute terms, acknowledge uncertainty

8. Maintain logical consistency: Each step should respect constraints from previous analysis

Key insight for your organization: Even rough estimates are better than "we can't measure it." The goal isn't perfect precision. It's moving from complete ignorance to informed decision-making.

When stakeholders claim cyber risk "can't be measured": Show them this progression. We went from 0-100% (useless) to 0.1-4% (actionable) using techniques any risk professional can learn.

Do You Think I’m Wrong?

Great! Create your own analysis and estimations and share them with me.

📚 What I’m Reading

• Vibe Coding and Basyesian Thought Experiment by Rick Howard
  I love Rick Howard’s First Principles newsletter. I linked to one of his posts in my last issue, and I keep coming back to his writing. This latest piece brings Bayes’ classic billiards thought experiment to life with vibe coding, showing why Bayesian thinking is so powerful for risk forecasting. I constantly reference and use this approach in my own work and highly recommend subscribing.

• Scheer Memos - by Alexander Scheer
  I just connected with the author of Scheer Memos on LinkedIn and I am glad I did. I have been really enjoying his blog; thoughtful takes on tech and cybersecurity with a mix of experience and perspective that feels fresh. I recommend subscribing if you want smart writing that makes you think.

• Curious About GRC? Here’s How I Got Started - by Nikita Gupta
  Have you ever wondered how people actually get started in GRC, or if you have been asked that question yourself, this post is an honest take. Nikita Gupta shares her journey from “no clue what GRC was” to a thriving career, with practical steps anyone can follow. It is approachable, encouraging, and a great share for students, career changers, or even seasoned pros looking to mentor the next wave.

• Presilience by Dr. Gavriel Schneider
  I just finished Presilience and found it eye-opening. The mix of personal stories makes the lessons real, and the practical advice is something I have already started using. Dr. Gavriel Schneider blends resilience tools with proactive risk management and leadership insights, showing how to turn uncertainty into growth. It is thoughtful, thorough, and very thought-provoking.

✉️ Contact

Have a question about this, or anything else? Here's how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

❤️ How You Can Help

✅ Share your estimation and measurement stories in comments
✅ Forward this to a colleague who's stuck in analysis paralysis
✅ Click the ❤️ if this helped you measure the seemingly immeasurable

Thanks for reading, and remember: anything that matters can be measured.

—Tony

• 📖 Book update: Getting Unstuck

• 🎯 The Risk Analysis Scope Creep Death Spiral

• 📄Book Excerpt: Defending Your Scope Against “What Abouts”

• 🔍 Risk Analysis and the Gambler's Fallacy: Why Your Inside View Isn’t Enough

• 📚 What I’m Reading: antifragility in risk models, the economics of security failures, and the joy of deep reading

• 🗂 From the Archives: two posts on probability language and expert disagreement

Hey there,

Welcome to Issue 2! Thanks for continuing this journey with me. This month, I'm tackling a common problem I see in the field: risk assessment scopes that grow and grow and grow until they strangle the very decisions they were meant to support.

Whether you're new to cyber risk quantification or a seasoned practitioner, you've probably felt this pain. Let's fix it.

I'll be speaking at SIRAcon next month, the annual conference hosted by the Society of Information Risk Analysts. It's a model-neutral and vendor-neutral non-profit that advocates for risk quantification in the technology field.

This year's conference is in Boston on September 9-11. Whether you're a beginner or advanced practitioner, there's something for everyone. I'll be discussing how to leverage Generative AI and LLMs to enhance your risk management practice. Hope to see you there!

More on my talk here, and you can learn more and register here.

📖 Book update: Getting Unstuck

My book, "From Heatmaps to Histograms: A Practical Guide to Cyber Risk Quantification," comes out in early 2026. Pre-order links and a release date will come in a few months. I'm about half done with the book, tackling the hardest chapters first. Editing is proving to be the most challenging part and often takes longer than writing the initial draft. This process is a reminder of the famous quote, often attributed to Mark Twain:

“I didn't have time to write a short letter, so I wrote a long one instead.” - Mark Twain

Through this process, I've put an enormous amount of thought into why risk quantification programs fail - and it happens at three distinct levels.

• At the individual level, practitioners often become overwhelmed by the apparent complexity, fail to commit to personal learning, or lack the basic statistical thinking required to get started.

• At the team level, failures often result from using the wrong tools, applying inadequate techniques, skipping proper team-based training, or attempting to retrofit quantitative concepts onto qualitative frameworks that can't support them.

• At the organizational level, programs fail due to poor stakeholder engagement, lack of executive buy-in, or perverse incentives that reward compliance theater over genuine risk reduction.

The root cause behind most of these failures is simple to state, but it's hard to grasp and even harder to devise meaningful solutions.

Here's my hypothesis: we've been treating two fundamentally incompatible disciplines as if they're just different flavors of the same thing. Qualitative and quantitative risk management appear side-by-side in textbooks and frameworks, but they come from completely different intellectual traditions: one rooted in 300 years of probability theory, the other emerging from industrial management optimization in the 1970s. They have different philosophies, success criteria, and tools. People get "stuck" when they fail to recognize this divide and attempt to quantify the unquantifiable, such as calculating ROI from heat map movements, using risk scores to allocate budgets, or assuming that moving from red to yellow signifies a measurable reduction in risk.

My goal in the first portion of my book is to help individuals, teams, and organizations get unstuck. Most people don't realize they're trying to blend fundamentally incompatible approaches. Whether you're ready to embrace quantitative methods or need to work within existing qualitative constraints, the path forward starts with recognizing which discipline you're actually practicing; then aligning your tools, metrics, and expectations accordingly.

I'll cover this divide in my book, but if you can't wait and want to explore these ideas now, there are two books every risk analyst should read.

Against the Gods: The Remarkable Story of Risk by Peter Bernstein shows you that risk management isn't some modern invention born from compliance frameworks. It's a centuries-old intellectual tradition stretching from Renaissance gamblers to Wall Street quants. Understanding this pedigree helps you see why real risk analysis requires mathematical thinking, not just following an ISO standard or filling out templates.

The Failure of Risk Management: Why It’s Broken and How to Fix It by Douglas Hubbard systematically dismantles the qualitative methods most organizations use, demonstrating with brutal clarity why heat maps and risk matrices don't actually manage risk. He doesn't just critique; he provides a practical path forward, showing how to apply proven quantitative methods to the very problems organizations claim are "unmeasurable."

Together, these books reveal both where we came from and where we need to go. Against the Gods shows the intellectual heritage you're part of when performing a quantitative risk assessment. The Failure of Risk Management shows why most of what passes for "risk management" today is theater, not analysis.

🎯 The Risk Analysis Scope Creep Death Spiral

Lately, I've been thinking a lot about risk analysis scope creep. Earlier in my career as a risk analyst, I didn't have the technical know-how and rhetorical skills to recognize and rein in this common problem. What starts as "let's assess our ransomware exposure" somehow morphs into modeling every conceivable threat from insider trading to asteroid impacts.

This reminds me of those old DirecTV commercials from years back. Here's how one went:

When your cable company keeps you on hold, you get angry. When you get angry, you blow off steam. When you blow off steam, accidents happen. When accidents happen, you get an eye patch. When you get an eye patch, people think you're tough. When people think you're tough, they want to see how tough. And when they want to see how tough, you end up in a roadside ditch.

Don't wake up in a roadside ditch. Get rid of cable and upgrade to DirecTV.

Now, the risk assessment version:

When you start a simple ransomware assessment, you realize you need to model supply chain risk. When you model supply chain risk, you discover fourth-party vendor dependencies. When you discover fourth-party dependencies, you start mapping the entire global semiconductor supply chain. When you map the global semiconductor supply chain, you're analyzing geopolitical tensions in Taiwan. When you're analyzing geopolitical tensions, you're building World War III scenarios. When you're building World War III scenarios, you're calculating the cyber risk impact of nuclear winter.

Don't calculate the cyber risk impact of nuclear winter. Scope your assessment properly.

Why This Matters (Especially for Beginners)

I once watched a 6-month risk assessment die on the vine because it took too long. The team had produced incredible work: sophisticated threat modeling, multiple risk scenarios, and analysis of every vector that stakeholders even casually mentioned in passing. "What about USB drives?" became a multi-page appendix. "Could nation-states target us?" spawned geopolitical risk research.

By the time they delivered their report, the CISO had already allocated next year's budget. The decision they were supposed to inform was made two months prior, without the benefit of a risk analysis.

Risk assessments aren't academic exercises; they're decision support tools operating under real-world constraints. When you lose sight of that, you're not being thorough; you're being counterproductive.

The Warning Signs

Watch for these scope creep red flags:

• "While we're at it, let's also look at..."

• Risk scenarios requiring multiple independent failures, plus a solar flare

• Modeling threats that would require your company to first become 10x more important

• Analysis timeframe exceeding the decision timeframe

• Stakeholders who've stopped asking when you'll be done

The Antidote

Before you start any risk assessment, get crystal clear on three things:

1. What decision needs to be made? Not "understand our risk" but "should we invest $X in Y?"

2. When does that decision need to be made? Work backwards from this date, not forwards from today

3. What would change their mind? If the answer is "nothing," you're writing a report, not doing risk analysis

Remember: A good-enough assessment delivered in time to influence a decision beats a perfect assessment delivered to the void.

📄Book Excerpt: Defending Your Scope Against “What Abouts”

Here's an excerpt from my book that ties in nicely to the previous section on scope creep. I call all these additional requests that can sink an assessment "what abouts." This is from my chapter on risk scenario building:

Be very protective of the in-scope and out-of-scope items in your risk assessment. As you circulate the risk scope to various teams for feedback, you will inevitably receive a flood of "what abouts" - the scope creep requests that arise once people see your risk scenario. It goes something like this:

• "What about insider threats? Shouldn't we include those too?"

• "What about our mobile app? That has customer data too, right?"

• "What about that new ransomware group we read about in the threat intel brief?"

• "What about social engineering? That's how most breaches start."

• "What about our cloud infrastructure? That could be a risk too."

These are all valid concerns, but your scope doesn't necessarily need to extend to these items. You do have to listen to each and every one, but bring it back to two things: the risk statement and the business decision the assessment will inform. Do the additional what abouts inform those?

The opportunity cost matters: Every additional piece of scope costs something: your time to collect and analyze data, stakeholder time for interviews, potential delays to decision-making, and sometimes real money for external research or tools. Before expanding your scope, ask:

• Will this additional analysis change the decision we're trying to make?

• Is the cost (time, effort, money, delay) of including this worth the potential insight?

• Could the decision-maker act confidently without this extra information?

• Does this address the original business concern, or is it just intellectually interesting?

Every hour you spend analyzing tangential risks is an hour not spent on the core question. Every week you delay the assessment while gathering "nice-to-have" data is a week the business operates without the insights it needs.

Remember: The goal isn't to analyze every possible factor of every single risk in the universe. It aims to provide decision-makers with sufficient insight to act confidently on the specific concern that prompted this assessment.

How to respond to scope creep: "That's an interesting point. Let's capture that as a separate risk to analyze later, but for this assessment, we're focused on [restate your original scope] because it directly addresses the [business decision]."

Stay focused, stay decisive, and protect your scope. You can always do another assessment later.

Risk Analysis and the Gambler's Fallacy: Why Your Inside View Isn’t Enough

I’m writing this from the Mandalay Bay in Las Vegas, taking a break from Black Hat. Last night, walking through the casino, I stopped at a roulette table where a small crowd had gathered.

Above the wheel, an electronic board flashed “HOT” and “COLD” numbers, showing the recent winners and the “overdue” losers. Black had hit six times in a row.

A guy next to me slid a stack of chips onto red.
“It’s due,” he told his friend.

The wheel spun. Black again. Seven.

That board is a master class in psychological misdirection. The casino knows exactly how people will read it. They are not offering insight. They are using a bias that keeps people betting.

The Inside View Trap

What I saw was the Gambler’s Fallacy: the belief that past results somehow change the odds in an independent game. In reality, every spin is the same, with eighteen chances for red, eighteen for black, and two for green in an American game. About 47.4 percent either way. The past does not matter.

Daniel Kahneman talks about this in terms of the inside and outside view. The inside view is personal. All the details are right in front of you. Seven blacks in a row - black must be “hot.” The outside view zooms out and looks at the bigger picture. Spins are independent, and the odds have not moved an inch. The outside view starts with what Kahneman and Tversky first described, later expanded by Kahneman and Lovallo, as a “reference class” approach: finding similar past cases to establish a base rate, then adjusting for your own specifics.

In risk work, the inside view feels compelling because it is your data and your history. But without the outside view to anchor it, it can be dangerously misleading.

From Roulette to Cybersecurity

The next day at Black Hat, I overheard someone say, “We haven’t had a major incident in three years. Our security program is clearly working.”

It is the same mental trap. The absence of bad news feels like proof of success. But in cybersecurity, events are not like roulette spins. They are connected in messy ways: shared vulnerabilities across common platforms, attackers reusing tools, exploit kits spreading, supply chains concentrating risk, and defenses that work for a while until attackers shift tactics.

So, unlike roulette, where past spins tell you nothing, in cyber, we should update on external events and peer data, not just our own quiet streak. Three quiet years could mean you have done a great job. It could also mean the attackers were busy elsewhere, or that someone is already inside and you have not found them yet. Sometimes it is just normal variation. Without looking beyond your own history, you cannot improve your forecast about which of these is actually happening.

The Data Blind Spot

This is where loss event frequency estimates often go wrong. If you base them only on your own history, a quiet stretch might make the probability look close to zero. But take the outside view, look at industry data, and you might find that in many circumstances, annual probabilities for material incidents land somewhere in the single-digit to low double-digit range. The exact number depends on your sector, size, and exposure.

Think about SolarWinds or Kaseya. Many companies had no frame of reference for a supply chain compromise of that scale. Their models, built entirely on internal experience, underestimated both the frequency of such events and their potential severity. Well-run risk programs that took the outside view already had this kind of event in their scenarios long before those breaches made headlines. The 2013 Target breach through an HVAC vendor, the 2017 NotPetya attack spread via a compromised accounting software update, and the 2017 CCleaner incident all showed that trusted channels can be turned into attack vectors. Programs built on that foundation could see risks to their company that had never happened within their own walls. Zero incidents in your history do not mean zero incidents in your future.

Mixing the Views

The best risk models start with the outside view: industry data, base rates, and what is happening in the wild. Then they layer in the inside view: your controls, your attack surface, targeting patterns you have seen, and the current threat landscape.

You do not assume the outside view is perfect, and you do not throw away the inside view just because it is biased. You blend them. You keep your ranges wide enough to be honest about uncertainty. And you write down your reasoning so you can revisit it later or change it when new data arrives.

Humility in the Model

Roulette is easy to describe. The odds are fixed, the events are independent, and the math is clean. Cyber risk is the opposite. Events influence each other. The threat landscape changes constantly. The data is incomplete and often noisy.

The teams that get this right do not confuse a lucky streak with a winning strategy. They use their own data but never rely on it alone. They check it against the outside view because “we have never seen it happen” is not the same as “it cannot happen here.”

In risk, as in the casino, betting on patterns that do not exist is the fastest way to lose. The strongest risk analysis programs follow the Three-Source Framework I wrote about in Issue 1 of this newsletter. Start with external data because it gives you the outside view: industry context and base rates that are bigger than your own experience and immune to the false comfort of a quiet streak. Then bring in internal data to ground the analysis in your specific reality, and subject matter expert input to bridge past and future with judgment about current threats, new controls, and changing conditions. Each source has blind spots, but together they create something far more reliable than any single input. Do this consistently, and you will stop betting on luck and start making better, more defensible decisions.

Further reading:

• Kahneman, D., & Tversky, A. (1979). Intuitive prediction: Biases and corrective procedures. In Forecasting

• Kahneman, D., & Lovallo, D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk taking. Management Science, 39(1), 17–31.

📚 What I'm Reading

Antifragile by Nassim Taleb

I’ve been rereading Antifragile through the lens of cyber risk quantification, asking: what makes a risk model fragile? Taleb’s core idea is that fragility isn’t just about being vulnerable. It’s about being harmed by volatility, randomness, and stressors. Many traditional risk models, especially those built on single-point estimates or assumptions of normality, fall apart when hit with real-world uncertainty.

Taleb is particularly critical of what he calls fragilistas: people who try to control complex systems with overly simplified models, often creating more harm than good. If that sounds familiar in cyber risk management, it should.

In contrast, he argues we should look for systems that benefit from disorder. In cyber risk terms, that means favoring models that expose uncertainty with distributions, adapt to new data, and resist overfitting. The book is dense, digressive, and a bit too confrontational for my comfort, but its insights are powerful. It reminds us that precision without resilience is a liability.

Why Information Security is Hard – An Economic Perspective by Ross Anderson

This paper is over 20 years old, but it’s still one of the most useful things you can read if you're trying to understand why so many security problems persist. Better tools, better talent, and bigger budgets haven’t solved them. Anderson’s core argument is that most security failures aren’t technical; they’re economic.

He explains how misaligned incentives, externalities, and information asymmetries create a world where the people who could fix problems often aren't the ones who suffer from them. It’s a foundational concept in security economics, and it helps explain everything from underinvestment in basic hygiene to why insecure software is still everywhere.

This matters for risk quantification, too. If we’re trying to measure and model risk, we need to understand not just what’s vulnerable, but why it stays that way. Incentives shape outcomes. Without that lens, even the best models will miss the mark.

The topic is still very relevant. I recommend it to anyone who wants to understand and solve real-world security problems.

The Joy of Deep Reading by Rick Howard

I recently went back and read Rick Howard’s July First Principles newsletter on the joy of deep reading, and it hit uncomfortably close to home. I love reading, but I am guilty of skimming far too much in my rush to consume as much as possible each day. Rick makes a compelling case for slowing down, picking one great book, and going deep, really engaging with the material instead of chasing bullet points and AI summaries.

What I loved most was how he blends personal stories, history, and curated recommendations from the CyberCanon project. If you have ever felt the pressure to “keep up” and ended up sacrificing depth for speed, this will remind you why deep reading is worth it.

If you care about cybersecurity, strategy, or just sharpening your thinking, I highly recommend subscribing to Rick’s First Principles newsletter. You will come away with ideas that stick and a reading list that is actually worth your time.

🗂 From the Archives

To keep decisions from drifting, here are two older posts of mine that fix two quiet common failures: fuzzy language and mismatched expert input.

Probability & the words we use: why it matters

I wrote a while back about how words like likely, high chance, or probably can completely derail a risk discussion. The CIA and NATO have both done research showing that people interpret these words in wildly different ways, sometimes by 40 percentage points or more.

You see this all the time in boardrooms, forecasts, and risk reports. Everyone thinks they’re on the same page, but they’re not.

The fix? Stop thinking like a storyteller and start thinking like a weather forecaster. Put numbers on it. High likelihood becomes 70% chance. Now it’s clear, and you can actually make a decision with it.

When Experts Disagree in Risk Analysis

Ever had one expert give you a risk estimate that’s way outside what everyone else is saying? It’s more common than you might think, and it’s worth understanding why before you throw their number out.

From what I’ve seen, it usually comes down to one of four things:

• They’re not calibrated

• They misunderstood the question

• They have a different worldview

• They know something the others don’t

Sometimes it’s noise, but sometimes it’s the most important data point in the room. The only way to know is to dig in, ask follow-up questions, and be honest about what you find.

If you work with SMEs in your modeling or forecasts, this is one to bookmark.

✉️ Contact

Have a question about scoping risk assessments? Here's how to reach me:

• Reply to this newsletter if reading via email

• Comment below

• Connect with me on LinkedIn

• Contact form

❤️ How You Can Help

✅ Share your scope creep horror stories - they might become examples in the book
✅ Forward this to a colleague who's stuck in analysis paralysis
✅ Click the ❤️ if this helped you escape a scope death spiral

Thanks for reading, and remember: the perfect risk assessment that never gets finished is infinitely less valuable than the good one that informs a decision this week.

—Tony

📊 Incident-Specific Frequencies: Use IRIS 2025 data to move beyond overall probabilities and understand which threats actually affect organizations like yours

🎯 The Complete IRIS Formula: Layer incident types from IRIS 2025 onto your base rates for targeted threat modeling

🗣️ Conversation Transformation: New conversation unlocks that are enabled with IRIS research instead of guesswork

🏠 Your Risk Analysis "Home Lab": Practice these IRIS-based skills even if your company doesn't do quantitative risk yet

Hi!

A few weeks ago in Issue 1, we explored Bayesian thinking and showed you exactly how to extract your organization's base rates from IRIS 2025. We walked through four use cases: establishing base rates, applying sector adjustment, estimating loss magnitude using revenue-tier and sector benchmarks, and factoring in risk trajectory. You learned the methodology - how to start with what you have and get better over time.

I enjoyed working with the IRIS report so much, I wanted to do one more practical application before Issue 2 drops in early August. Think of this as a quick follow-up that builds naturally on your foundation.

Thank you for reading,
Tony

The Missing Layer in Your Risk Analysis

Where We Are: In Issue 1, you've established your organization's sector-specific base rates using IRIS 2025 data. You now have solid, research-backed probabilities for experiencing a significant cyber incident.

The Next Layer: Those base rates tell you about overall incident likelihood, but they don't help you understand which types of incidents are most likely to affect organizations like yours. That's where incident-specific frequencies come in.

The Unlock: By layering incident-specific frequencies onto your base rates, you can move from "we have a 9.9% annual severe incident probability" to "we're most likely to face system intrusion (46% of incidents), followed by ransomware (35%), with accidental disclosure making up 3%."

This enables much more targeted conversations with leadership about where to focus security investments, what scenarios to prepare for, and how different threat types might impact your specific business operations.

IRIS 2025: Calculate Your Incident-Specific Risk

You've built your sector-specific base rates. Now let's drill one layer deeper to understand which types of incidents organizations in your sector actually experience.

The Complete Formula

Up to this point, we’ve been talking about overall incident probability. That’s useful, but it still leaves an important question unanswered. If an incident does occur, what kind of incident is it most likely to be?

This is where incident-specific frequencies come in.

At a high level, the idea is simple. You start with your organization’s overall probability of experiencing a significant cyber incident, then you decompose that probability based on the types of incidents that actually occur in the real world.

Here’s the formula we’ll use:

Incident-Specific Probability
= Base Rate × Sector Adjustment × Incident Type Percentage

Before we apply it, one clarification matters.

The incident type percentages from IRIS are conditional distributions. They describe how incidents break down given that a significant incident has occurred. They are not independent risks, and they are not additive. This formula doesn’t create new risk. It simply partitions an existing annual probability into more actionable pieces.

In other words, we’re taking one overall likelihood and asking, “If something happens, what does it most likely look like?”

Each part of the formula plays a specific role.

The base rate represents the annual probability that an organization experiences at least one significant cyber incident that enters the public record.

The sector adjustment is a relative likelihood factors drawn from IRIS. IRIS does not prescribe a single official way to combine revenue tier and sector effects, so we use the sector factor here as a transparent heuristic on top of a revenue-conditioned base rate.

The incident type percentage comes from IRIS’s observed distribution of incident types. Applied correctly, it lets us move from a single headline probability to a set of incident-specific probabilities that still sum back to the original total.

This distinction is subtle, but important. We’re not stacking risks on top of each other. We’re breaking one risk apart so it can be discussed, prioritized, and acted on.

With that foundation in place, we can walk through a concrete example.

Step 1: Get Your Base Rate

Current annual probability (base rate): approximately 8–9% (Figure 7, p. 12)

This is your starting point: the likelihood that an organization in this revenue tier experiences at least one significant incident that enters the public record.

Because Figure 7 already conditions on revenue tier, we do not apply an additional revenue adjustment in this example.

Step 2: Adjust for Your Industry

Source: Figure A1, p. 34

Apply your sector-specific multiplier:

• Healthcare: 1.34x

• Financial: 1.44x

• Information: 1.55x

• Education: 1.60x

• Professional: 1.50x

• Manufacturing: 1.03x

• Utilities: 0.62x

• Retail: 1.19x

Note: These are a few of the sectors on page 34; refer to the report for the full list)

Step 3: Apply Incident Type Frequency

So far, we’ve been talking about how often an organization might experience a significant cyber incident. That’s useful, but it still leaves a practical gap. Even when leadership accepts the overall probability, the next question is almost always the same.

“What kind of incident are we actually talking about?”

IRIS helps answer that by showing how real-world incidents break down by type.

To see this, turn to Figure 2 on Page 5 of the IRIS 2025 report. This figure shows the distribution of incident types observed in the IRIS dataset for the most recent year.

One important clarification before we go further. These percentages represent a conditional distribution. They answer the question: given that a significant cyber incident occurs, what type of incident is it most likely to be? They are not independent annual risks, and they should not be added together. Used correctly, they simply partition the total incident probability we already calculated.

Based on a visual reading of Figure 2 for 2024, the incident pattern looks roughly like this:

• System intrusion: ~46%

• Ransomware: ~35%

• Denial-of-service attacks: ~8%

• Accidental disclosure: ~3%

• Insider misuse: ~2%

• Defacement: ~1%

• Scam or fraud: ~1%

• Physical threat: ~1%

• System failure: ~1%

These values are approximate and may not sum to exactly 100 percent due to rounding. That’s fine. Precision isn’t the goal here. Direction and relative weight are.

Another important label. This incident-type distribution is aggregated across the IRIS dataset. It is not sector-specific and it is not revenue-specific. Think of it as a general conditional pattern: when a significant incident happens, this is how those incidents tend to show up across organizations.

Despite that limitation, this breakdown is extremely useful. When you apply it to your organization’s overall incident probability, it allows you to move from a single abstract risk number to a short list of concrete scenarios that actually drive most of the risk.

With these incident-type frequencies in hand, we can now walk through a complete example and see how the math translates into something you can use in real conversations.

Real-World Example

Let’s walk through a concrete example to see how this works in practice.

We’ll stick with the same hypothetical organization: a healthcare company with approximately $800 million in annual revenue.

From earlier, we already have what we need to build a firm-specific starting point.

First, we estimate the overall probability of a significant cyber incident. Based on IRIS 2025 Figure 7 and a visual reading of the $100M to $1B revenue tier, the base annual probability appears to be roughly 8 to 9 percent.

Next, we apply a sector-relative adjustment. Using the Healthcare factor from Appendix 3, Figure A1, and treating it as a transparent heuristic, we arrive at a firm-specific prior of approximately 11 to 12 percent per year for at least one significant, publicly reported cyber incident.

At this stage, we stop adjusting. That 11 to 12 percent is the total probability we’re working with. Everything that follows is about breaking that single number into more useful pieces, not adding new risk on top of it.

Now we apply the incident-type distribution.

Using the incident-type percentages from IRIS Figure 2, we can decompose that overall probability into incident-specific probabilities. For simplicity, we’ll work with a midpoint estimate of 9.9 percent, recognizing that the real answer is a range.

Here’s what that looks like.

• System intrusion:
  9.9% × 0.46 ≈ 4.6%

• Ransomware:
  9.9% × 0.35 ≈ 3.5%

• Denial-of-service attack:
  9.9% × 0.08 ≈ 0.8%

• Accidental disclosure:
  9.9% × 0.03 ≈ 0.3%

• Insider misuse:
  9.9% × 0.02 ≈ 0.2%

• All other incident types combined:
  Each on the order of 0.1% annually

These numbers are not independent risks. They are slices of the same overall probability. If you add them back up, you end up right where you started.

What matters isn’t the second decimal place. What matters is the shape of the risk. In this example, system intrusion and ransomware together account for more than 80 percent of the organization’s incident likelihood. Everything else trails far behind.

That single insight is often enough to change the conversation.

Instead of debating whether cyber risk is “high” or “medium,” you can now say something much more concrete. If this organization experiences a significant incident in the next year, it is far more likely to involve credential compromise or ransomware than any other category. That gives you a defensible basis for prioritizing controls, planning scenarios, and focusing leadership attention.

With the math out of the way, we can now look at what this enables in real conversations with finance, business leaders, and executives.

The Conversations This Enables

The real payoff of this approach isn’t the math. It’s the conversations it unlocks.

Once you’ve decomposed overall cyber risk into incident-specific probabilities, you stop arguing about abstractions and start talking about concrete, defensible scenarios. That shift alone changes how people engage with the analysis.

Here’s what that looks like in practice.

With Finance and the CFO

Before, the conversation often sounds like this:

“We need more security budget because cyber threats are increasing.”

After applying this approach, the conversation changes shape:

“Based on IRIS 2025 data, organizations like ours face roughly an 11 to 12 percent annual probability of a significant cyber incident that enters the public record. Over 80 percent of that likelihood is driven by system intrusion and ransomware. Those are the scenarios that dominate our risk exposure, and they’re the ones this investment is designed to reduce.”

At this stage, you don’t need to pretend you have perfect loss estimates. What matters is that you can explain where risk is concentrated and why certain investments are aimed at specific, high-probability scenarios. When finance teams see that you’re reasoning from data rather than fear, the tone of the conversation changes.

With Business Unit Leaders

Before:

“Everyone needs to be more security conscious because cyber risk is high.”

After:

“Industry research shows that when significant incidents occur, nearly half involve system intrusion, often driven by credential compromise. For organizations like ours, that translates into several percentage points of annual risk concentrated in how access is managed day to day. The practices your team controls directly influence that exposure.”

This reframing connects abstract cyber risk to operational behavior. Business leaders may not care about probability theory, but they understand cause and effect. Incident-specific frequencies give you a credible bridge between the two.

With the Board and Executives

Before:

“Cyber risk is one of our top concerns.”

After:

“Industry data suggests our overall likelihood of a significant cyber incident is on the order of ten percent annually, with most of that risk concentrated in a small number of scenarios. We’re prioritizing controls and resilience measures accordingly, and we can show how those choices change the shape of our risk profile over time.”

Notice what’s missing here. There’s no overconfident claim of precision, and no reliance on a single scary number. Instead, you’re showing that risk is being actively managed, measured, and revisited as conditions change. That’s what boards are actually looking for.

What Really Changes

In each of these conversations, something subtle but important happens. You stop defending your instincts and start explaining your analysis.

You’re no longer the person saying, “I think cyber risk is high.” You’re the person saying, “Here’s how industry data breaks down, here’s where our exposure really sits, and here’s why we’re focusing where we are.”

That shift builds credibility, not because the numbers are perfect, but because the reasoning is visible and disciplined.

🎯 Your Next Steps

1. Calculate your top 3 incident probabilities using this method

2. Multiply by potential loss estimates (see IRIS 2025 loss data by sector/revenue)

3. Upgrade your conversations: budgeting, insurance planning, and board presentations

4. Update annually as baseline probabilities and threat landscapes evolve

🏠 Homework: Pick One Conversation

Your assignment: Choose one conversation you need to have in the next two weeks where you've been struggling with credibility or budget approval.

1. Calculate your incident-specific probabilities using the formula above

2. Use the conversation frameworks to build your specific analysis

3. Lead with "Industry research shows..." instead of "I think..."

4. Watch how the conversation changes

The goal isn't perfection - it's moving from opinion to analysis.

Your Risk Analysis "Home Lab"

What if your company doesn't do quantitative risk analysis, but you still want to try these techniques?

Here's my recommendation: do this stuff at home anyway.

Other people in the security field always talk about having a "home lab" - spinning up VMs, practicing penetration testing, learning new tools. This is our version of a home lab: practicing quantitative risk analysis.

Why this matters:

• Skill development: These analytical skills transfer to every risk conversation you'll ever have

• Pattern recognition: The more assessments you build, the better you get at spotting what matters

• Confidence building: When you do get the chance to do this professionally, you're already practiced

• Career advancement: Quantitative risk skills are becoming table stakes for senior security roles

Practice scenarios to try:

• Pick a company from recent breach headlines and reverse-engineer their risk profile

• Model your previous employer's risk using public information

• Build risk comparisons between different sectors you're curious about

• Practice the conversation frameworks with industry colleagues

Think of it as professional development, not just a theoretical exercise. Every assessment you build makes you better at the next one.

🔑 Key Takeaways

For Leadership Conversations: Stop saying "we have high cyber risk." Start saying "Based on IRIS 2025 data, organizations like ours face a 4.6% chance of system intrusion and 3.5% chance of ransomware this year. These aren't generic estimates; they're based on what actually happened to similar organizations in our sector."

For Security Planning: Focus defensive investments on your top 2-3 calculated risks rather than spreading resources equally across all possible threats. In our healthcare example, system intrusion (4.6% probability) and ransomware (3.5% probability) represent over 80% of the incident risk profile.

For Budget Conversations: Use the complete formula (Base Rate (revenue-conditioned) × Sector Factor × Incident Type %) to move from opinion-based requests to real risk analysis. "Without this $200K investment, we have a 15% chance of losses exceeding $5M annually. With it, that drops to 8% chance of exceeding $2M" is a CFO conversation.

The way I framed risk above is called a loss exceedance statement. They are much stronger than heat maps because they:

• Show tail risk (the really bad scenarios CFOs worry about)

• Use probability language that executives understand from other business decisions

• Focus on the "what if we get unlucky" scenarios that keep leadership awake at night

• Mirror how insurance and financial risk is typically discussed in boardrooms

For Professional Development: Practice these calculations even if your company doesn't do quantitative risk yet. These analytical skills transfer to every risk conversation you'll ever have and are becoming table stakes for senior security roles.

What's Coming

A note on IRIS focus: I know we've spent significant time on IRIS 2025 between Issues 1 and 1.5. That's intentional. The last IRIS was published in 2022, and it's genuinely one of the most important resources a cyber risk analyst has. Getting the most value out of this research felt worth the deep dive.

But we're moving on from here. Issue 2 will shift focus to completely different things.

✉️ Contact

Have a question about this issue or risk analysis in general? Here's how to reach me:

• Reply to this newsletter, if you receive via email

• Comment below

• Connect on LinkedIn

• Contact form

Had success with any of the content here? Hit reply and share your story. These become case studies for future issues.

❤️ How You Can Help

• Forward this to someone who needs to upgrade their risk conversations

• Click the ❤️ or comment if you found this useful

• Tell me what topics you want covered next

If someone forwarded this to you, please subscribe to get future issues.

- Tony

• 📖 Book update: progress on data collection, vetting, and the three-source model for CRQ

• 🧠 Why every risk analyst should think like a Bayesian (and how to start)

• 📊 How to use IRIS 2025 to establish base rates and build sector-specific priors

• 🛠️ Step-by-step GenAI prompts for applying IRIS data in your own models

• 📗 Book Excerpt: My Experience with Prediction Markets

• 📚 What I’m reading: favorite picks on risk sin eater, security-first controls, and financial discipline in CRQ

• 🗂 From the archives: two blog posts on pushback in risk analysis and decision framing

• 🙋 Reader Q&A: what to do when your org has 1,500 risks in the register

Hey there,

Welcome to Issue 1! Thanks for being part of this journey from the very beginning. This newsletter accompanies my upcoming book "From Heatmaps to Histograms: A Practical Guide to Cyber Risk Quantification" (Apress, early 2026). Each month, I share practical techniques, behind-the-scenes insights from the book writing process, and field-tested CRQ tactics that help you build better risk models.

If you missed my launch post (Issue 0), you can check it out here. I shared the full abstract of the upcoming book as well as a guide to your first quantitative cyber risk assessment.

📖 Book Update: Progress on the Big Middle

First, a quick update on the book project. The deal is signed, and we're targeting early 2026 for publication. Pre-order links should be available later this year (likely Q4 2025), and I'll share those with you as soon as they're live.

I'm currently focused on what I thought would be the most challenging section: data collection, vetting, and preparation for quantitative analysis. I tackled this middle chunk first because getting the foundation right makes everything else click into place.

The Three-Source Framework

One of the core principles I'm building the book around is what I call the "three essential data sources" for cyber risk quantification:

• External data provides industry context and base rates - breach frequencies, cost studies, enforcement patterns. Your "what typically happens to organizations like ours" baseline.

• Internal data grounds everything in your specific reality - incident logs, actual recovery times, real costs from past events. Your "what actually happens here" update.

• Subject matter expert (SME) input bridges past and future with forward-looking judgment about current threats, new controls, and changing conditions. Your "what's likely to happen next" forecast.

The magic happens when you systematically combine all three. Each source has blind spots, but together they create something far more reliable than any single input.

Getting Unstuck

People get stuck on data collection for three main reasons: the "perfect data myth," not knowing where to look, and analysis overwhelm when facing messy information. I'm addressing each with practical methods that work with real-world constraints.

Vetting and Blending

The book includes a simple framework for evaluating any data source in minutes, plus time-tested quality adjustment methods borrowed from fields like climate science, nuclear safety, and actuarial science that specialize in high-stakes decisions under uncertainty. When you find that vendor survey claiming "average breach cost is $2.1M," you'll know exactly how to convert it into an honest range that reflects what you actually know.

For combining sources, I use a math-free Bayesian approach - start with your best external baseline, then systematically update it with internal evidence and expert judgment. Like checking traffic before heading to the airport, but structured for risk data.

This foundation work enables everything else - scenarios, simulations, and actual business decisions. Getting the data piece right makes the rest straightforward.

🧠 Why Every Risk Analyst Should Think Like a Bayesian

Speaking of Bayesian thinking… let me share a tool that's transformed how I approach risk analysis, and I believe it can do the same for you. It’s more than a tool, though; it’s a way of thinking.

You don't need to be a statistics wizard to benefit from Bayesian thinking. Yes, there are sophisticated mathematical applications that belong in any serious risk analysis toolkit, but today I want to focus on Bayesian reasoning as a mental model. This shift in mindset can revolutionize how you frame risk analyses, keep projects focused, and avoid the dreaded "boiling the ocean" syndrome.

More importantly, if you embrace this thinking early, you'll sidestep the common CRQ myths that paralyze many analysts:

• Believing you need to collect ALL the data before starting

• Thinking a risk analysis is invalid without perfect information

• Getting stuck in analysis paralysis

Bayesian thinking also acts as a natural defense against cognitive biases we all carry: overconfidence, anchoring, and the IKEA effect (overvaluing something because we built it ourselves).

Bayesian Thinking as a Mental Model

Prior: Your initial belief or estimate, based on current knowledge, experience, or data.

Evidence: New information, observations, or data relevant to the belief.

Posterior: Your updated belief after factoring in the new evidence. It becomes the new prior for next time.

The Process:

1. Start with a belief: based on what you already know (your prior).

2. Gather evidence: actively look for new data or observations.

3. Update your belief: revise your estimate in light of the evidence.

4. Repeat : treat your updated belief as the new starting point for the next cycle.

The beauty is in the cycle: today's posterior becomes tomorrow's prior. You're always learning, always updating, never claiming to have the final answer.

A Real-World Example

Let's say you're assessing the risk of a data breach at your company.

Your Prior: Based on industry reports, you estimate a 15% chance of a significant breach in the next year.

New Evidence: Your security team reports they've detected 3x more phishing attempts this quarter, and a recent vulnerability scan found several unpatched systems.

Your Posterior: This evidence suggests higher risk. You update to around 25% chance of a breach.

Next Cycle: A month later, phishing training shows dramatic improvement in employee click rates, and all critical patches are applied. Your posterior (25%) becomes your new prior, and with this positive evidence, you adjust down to around 18%.

Notice what happened? You started with imperfect information, made decisions, gathered more data, and refined your thinking. You never stopped to collect "all possible data." You started with what you had and improved from there.

This is exactly how effective risk analysis works in the real world.

The key insight: Without this mindset, beginners often try to "get it perfect" before they start and never actually do a risk analysis. Bayesian thinking gives you permission to start with what you have and get better over time. You're not ignoring uncertainty, you're structuring it. That's what makes it so powerful for real-world risk.

📊 Using IRIS 2025 Data as Your Risk Analysis Starting Point

Now that you’re thinking like a Bayesian, it’s time to put that mindset to work. The Cyentia Institute’s IRIS 2025 report is one of the best starting points we have for establishing base rates, those initial priors that anchor a quantitative risk analysis before we ever touch a scenario or a simulation.

Instead of beginning with a guess, or a color on a heat map, you can start with probabilities grounded in what has happened to organizations like yours. This is where Bayesian thinking stops being abstract and starts being practical. You’re not trying to be perfect. You’re trying to be honest about what you know, and then improve from there.

One important caveat up front. IRIS focuses on significant cyber incidents that make their way into the public record. That means it should be treated as a lower-bound prior, not a complete census of all incidents. Over time, these priors should be updated with your internal incident data and structured expert judgment. That’s not a limitation of the method. That’s the method working as intended.

Below are four practical ways you can start using IRIS 2025 right now.

Four Ways to Use IRIS 2025 Right Now

Use Case 1: Establish Your Organization’s Base Rate

Let’s start by estimating a base rate for a hypothetical healthcare organization with $800 million in annual revenue. This base rate represents the annual probability that the organization experiences at least one significant security incident, meaning an event that would reasonably be expected to require public disclosure.

• First, find your revenue tier. In IRIS 2025, turn to Page 12, Figure 7.

• Next, locate the appropriate category. For our example organization, that’s the “$100M to $1B” revenue band.

• Now read the chart. Follow the line for this revenue tier to the rightmost point, which corresponds to 2024.

Based on a visual reading of the chart, organizations in the $100M to $1B revenue range appear to face roughly an 8-9% annual probability of a significant, publicly disclosed cyber incident.

That range becomes your Bayesian prior for modeling significant cyber incidents. It’s not a final answer, and it’s not a claim of precision. It’s a reasonable starting point.

⚠️ These probabilities reflect publicly disclosed events, not all security incidents. I strongly recommend working with ranges rather than single point estimates to account for both reading uncertainty and real-world variability.

🔍 GenAI Prompt:

[upload the PDF into the prompt]

See the attached IRIS 2025 report, Page 12, Figure 7. Help me extract the incident probability range for an organization with $800M in annual revenue. Walk me through how to read the chart and provide a reasonable range rather than a precise percentage. These probabilities are for significant cyber incidents requiring disclosure.

Use Case 2: Build Sector-Specific Frequency Models

Now let’s tailor that base rate to your industry. We’ll stay with the same example, a healthcare organization with about $800M in annual revenue.

Before we do the math, a quick note on what we’re doing conceptually. IRIS gives us two useful ingredients: an overall base rate by revenue tier, and a sector-relative likelihood factor. IRIS does not spell out a single “official” way to combine those two, so what follows is a practical heuristic for building a firm-specific prior. It’s simple, transparent, and easy to update later when you bring in internal data.

Here’s what we’ll use from IRIS 2025.

First, the base annual incident probability by revenue tier from Figure 7 on page 12. This is our starting point for estimating the likelihood of at least one significant incident entering the public record.

Second, the sector relative probability factor from Appendix 3, Figure A1 on page 34. For Healthcare, the factor is 1.34x. Think of it as “more or less likely than the median sector,” not a magic conversion that makes the data perfect.

Step-by-Step Exercise

Start with your base rate. From Figure 7 (p. 12), organizations in the $100M to $1B revenue range appear to have roughly an 8 to 9 percent annual probability of a significant, publicly reported incident.

Find your sector factor. In Appendix 3, Figure A1 (p. 34), Healthcare’s relative probability factor is 1.34x.

Apply the factor to your range.

Lower bound: 8% × 1.34 ≈ 10.7%
Upper bound: 9% × 1.34 ≈ 12.1%

So for a healthcare organization with roughly $800M in annual revenue, a reasonable firm-specific prior for a significant, publicly reported incident is about 11 to 12 percent per year, using this simple heuristic.

One last bit of humility, because it matters. If you compare this derived prior to the healthcare time series elsewhere in IRIS, you may see differences. That’s not a contradiction. It’s a reminder that we’re combining two different lenses on the data. In practice, you treat this as a starting point, then update it with what you know about your organization.

🔍 GenAI Prompt:

Using IRIS 2025, help me calculate a sector-adjusted incident probability range. My organization is in the Healthcare sector with $800M annual revenue.

• From Figure 7 on p. 12, extract the base annual incident probability range for organizations with $100M–$1B in revenue.

• From Appendix 3, Figure A1 on p. 34, find Healthcare’s sector relative probability factor (1.34x).

• Apply that factor to both ends of the base probability range, show each calculation step, and clarify these probabilities are for significant cyber incidents that enter the public record.

Use Case 3: Sector and Revenue Loss Benchmarks

Now let’s talk about impact. Frequency tells you how often you might get hit. Loss magnitude tells you what it looks like when you do.

This is also where it’s easy to accidentally overclaim what the data can do. IRIS gives us two useful, but different, views of loss magnitude.

One is organized by revenue tier (Table 1 on page 16). The other is organized by sector (Appendix 3, Figure A3 on page 35). IRIS does not provide sector “loss multipliers,” and it does not prescribe a single way to combine the two into a perfectly sector-and-revenue-conditioned loss estimate. So in this issue, we’re going to keep it honest and use the sector loss statistics directly as our starting point, then refine later with internal data.

Here’s how to do it.

Step-by-Step Exercise

Start with revenue-tier loss benchmarks (Table 1, p. 16). For organizations with $100M to $1B in annual revenue, IRIS reports:

• Median (50th percentile): $466.7K

• High-end (95th percentile): $12.3M

These numbers are useful because they keep you anchored to firm size, but they’re not sector-specific.

Now pull sector loss benchmarks (Appendix 3, Figure A3, p. 35). For Healthcare, IRIS reports sector-level loss statistics for significant, publicly reported incidents. Use those values as your sector prior for impact.

Your Result: For a healthcare organization, a reasonable starting point for loss magnitude is:

• Typical loss (median): approximately $557K

• Extreme loss (95th percentile): approximately $14M

A quick but important label. These sector figures are benchmarks at the sector level. They’re not conditioned on revenue tier. That’s fine for a starting point, and in practice it’s often what you want early on, because it is easier to defend and easier to update. If you have internal incident cost data, or a strong reason to believe your organization is systematically above or below the sector, this is exactly where Bayesian updating earns its keep.

🔍 GenAI Prompt:

Using the IRIS 2025 report, help me extract defensible loss magnitude benchmarks for significant cyber incidents for a Healthcare organization.

1. From Table 1 on p. 16, extract the median (50th percentile) and high-end (95th percentile) loss values for organizations in the $100M–$1B revenue range.

2. From Appendix 3, Figure A3 on p. 35, extract the Healthcare sector median and 95th percentile loss values.

Explain the difference between revenue-tier and sector benchmarks, and clarify that IRIS reports on significant incidents that enter the public record. Use ranges or “approximately” language rather than implying precision.

Use Case 4: Factor in Sector Risk Trajectory

So far, we’ve treated risk as a snapshot. A base rate. A set of priors. That’s necessary, but it’s not sufficient. Risk is not static, and one of the most useful questions you can ask is whether your sector’s risk profile is trending upward, stabilizing, or starting to bend downward.

IRIS 2025 gives us a way to look at that over time.

Step-by-Step Exercise

To understand sector trajectory, turn to Page 13, Figure 8 in the IRIS 2025 report and focus on the Healthcare panel.

This chart shows the estimated annual probability of a significant, publicly reported cyber incident over time for healthcare organizations. The exact values are not tabulated, so what follows is based on visual interpretation of the plotted trend rather than precise point estimates.

Over the long term, the direction is clear. Healthcare incident probability rises steadily from the late 2000s through the early 2020s, moving from low single-digit percentages to roughly nine percent by the early part of this decade.

In the most recent years shown, however, the pattern changes. From about 2022 through 2024, the healthcare curve flattens and shows signs of stabilization rather than continued acceleration. The most recent value shown is approximately 9.1 percent in 2024.

How to Interpret This

There are two signals here, and both matter.

The long-term signal tells you that healthcare cyber risk increased substantially over the last fifteen years. That context helps explain why today’s base rates are meaningfully higher than they were a decade ago.

The short-term signal is more cautious. Recent years suggest a plateau, or at least a pause in growth. That does not mean risk is going away, and it certainly doesn’t mean it can’t rise again. It does mean you should be careful about assuming continued straight-line growth into the future.

Practical Takeaway

For forward-looking planning, treat healthcare risk as elevated relative to the past, but not necessarily accelerating year over year. This is a reminder to revisit your priors regularly. IRIS is published on a multi-year cadence, and sector trajectories can and do change.

Bayesian thinking fits naturally here. Today’s posterior becomes tomorrow’s prior. When new data arrives, you update. You don’t lock in assumptions simply because they once felt directionally true.

🔍 GenAI Prompt:

Using IRIS 2025 Page 13, Figure 8, help me analyze the risk trajectory for healthcare organizations from 2008–2024. I need to determine whether the trend is rising, falling, or stabilizing and estimate the long-term rate of change as well as the short-term directional signal. Use this to inform 2025+ planning assumptions.

Final Output: Your Sector Risk Profile

At this point, you’ve built a defensible, research-backed starting profile for a healthcare organization with approximately $800 million in annual revenue. Nothing here is exotic. Nothing requires perfect data. It’s simply a structured way to turn published research into something you can actually use.

Here’s what that profile looks like.

Incident frequency

Based on IRIS 2025 Figure 7, organizations in the $100M to $1B revenue range appear to face roughly an 8 to 9 percent annual probability of experiencing at least one significant cyber incident that enters the public record.

Applying IRIS’s sector-relative probability factor for Healthcare as a practical heuristic, a reasonable firm-specific prior is approximately 11 to 12 percent per year. This is not an official IRIS statistic. It’s a transparent, derived starting point that can and should be updated with internal data and expert judgment.

Loss magnitude

For loss magnitude, IRIS provides sector-level benchmarks that are well suited for early modeling.

According to IRIS 2025 Appendix 3, Figure A3, Healthcare organizations show:

• A median (typical) loss of approximately $557K

• A 95th percentile (extreme) loss of approximately $14M

These figures reflect significant, publicly reported incidents at the sector level. They are not conditioned on revenue tier, and that’s intentional here. As a starting point, sector benchmarks are easier to explain, easier to defend, and easier to update once you incorporate your own incident cost data.

Risk trajectory

Looking at IRIS 2025 Figure 8, Healthcare risk increased substantially over the long term, rising from low single-digit probabilities in the late 2000s to roughly nine percent by the early 2020s.

In more recent years, the trend appears to flatten. The most recent estimate shown is approximately 9.1 percent in 2024, suggesting stabilization rather than continued acceleration. That doesn’t mean risk is declining, but it does mean you should be cautious about assuming straight-line growth into the future.

What This Means for You

Taken together, this gives you a usable, defensible risk profile:

• Frequency: roughly an 11 to 12 percent annual probability of a significant, publicly reportable cyber incident

• Impact: typical losses around $500K, with extreme but plausible losses on the order of $10M to $15M

• Trajectory: elevated relative to the past, but not clearly accelerating year over year

More importantly, you now have inputs that support real decisions.

You can test whether proposed security investments meaningfully reduce expected loss. You can evaluate insurance retention levels against tail risk instead of gut feel. You can explain to executives not just that cyber risk exists, but how often it shows up, how bad it tends to be, and how that compares to what the business is willing to tolerate.

These are not final answers. They are informed starting points.

And that’s the whole point of Bayesian thinking. You don’t wait for perfect information. You start with what you can defend, then you update as you learn more about your own environment, your own controls, and your own incidents.

💡The Bayesian Unlock

Notice what's happening here: You're not using IRIS data as the end goal; something you skim once and never open again. You're using it as your informed starting point, then systematically updating with organization-specific evidence.

This prevents the classic risk analysis traps:

• Starting with gut feelings instead of data

• Ignoring industry baselines

• Treating your organization as completely unique

• Getting paralyzed by the need for "perfect" data

Your risk analysis becomes a conversation between industry research and organizational reality - exactly what Bayesian thinking is designed to handle.

🏠 Homework: Put This Into Practice

Your Assignment: Download IRIS 2025 and build your organization's risk priors using the GenAI prompts above.

1. Use the four exercises to calculate your organization's base rates

2. Try all four GenAI prompts - see how they help extract and organize the data

3. Document your priors - you'll need them as your Bayesian starting point

4. Compare your current risk assessments with research-backed priors - how do they differ from your existing estimates?

Bonus Challenge: Take your calculated ranges and ask yourself: "What organizational-specific evidence could move me up or down within these ranges?" This is exactly how Bayesian updating should work.

Remember: These aren't your final risk assessments - they're your informed starting points. The real power comes when you start updating these priors with your organization's specific threat intelligence, control effectiveness, and incident history.

📗 Book Excerpt: My Experiment with a Prediction Market

One of the cooler experiences in my career happened when I worked with Richard Seiersen, co-author of How to Measure Anything in Cybersecurity Risk and author of The Metrics Manifesto. Together, we set up a functioning prediction market at a large financial services company in San Francisco.

The Setup

We created an open-to-the-public event that was part Shark Tank, part American Idol: cybersecurity startups pitched to a panel of CISO judges in our arena-style auditorium. But here's where it got interesting: we set up a prediction market so attendees could "trade" on which startup they thought would win at the end of the night.

A prediction market is essentially a crowdsourced forecasting tool where participants buy and sell contracts based on event outcomes. The trading price reflects the collective probability that participants assign to each outcome. Think of it as real-time betting on future events; the more demand for a particular outcome, the higher its implied probability.

To make it more engaging and test the concept, we added other prediction questions: Will it rain tomorrow? Who will win the Giants game? What's the probability of a major data breach in healthcare this quarter?

The Key: Proper Incentives

Prediction markets work best when participants have skin in the game: something to gain or lose. We offered prizes and leaderboard clout to the best forecasters, which incentivized people to use their genuine knowledge and research rather than just guessing randomly.

The Results

The accuracy was remarkable. The market consistently predicted not just the startup pitch winners (judged at the end of the evening), but also sports games, weather patterns, traffic conditions, and short-term cyber events that we could verify within a month.

When we talked to the most accurate forecasters, I found something interesting: many were fantasy football players, poker enthusiasts, or people with backgrounds in meteorology, statistics, or microeconomics. What they had in common was probabilistic thinking: the ability to reason about uncertainty and update beliefs based on new information.

Why This Matters for CRQ

I've often thought about extending this to regular cybersecurity use cases. Imagine a public prediction market where the entire cybersecurity community collectively forecasts cyber events. Done right, this could provide base rates (or better) for our quantitative risk assessments.

The challenge? Confidentiality. Most companies can't share the detailed information needed to make such markets work effectively. It becomes a tragedy of the commons; everyone wants to use the data, but few can contribute without violating privacy or competitive concerns.

Still, the concept shows the power of structured, incentivized expert judgment. Even if we can't build industry-wide prediction markets, we can apply the same principles in our SME workshops: create the right incentives, structure the process, and tap into the collective wisdom of people who think probabilistically.

Want to Try It? Check out Good Judgment Open, a publicly accessible prediction market where you can practice forecasting on everything from politics to technology trends.

📚 What I'm Reading

Each issue, I'll share a few blog posts, research articles, or tools worth your time. Here are three standouts I've been diving into:

The Role of the Risk Sin Eater by Jack Freund: This one is a little older, but I still share it often with friends and coworkers. Jack is one of my favorite writers on risk, and in this piece he draws a sharp parallel between modern cyber risk approval practices and the centuries-old tradition of sin eaters, figures in rural Europe who symbolically absorbed the sins of the dead, urging us to abandon this outdated folklore and let business leaders own their risks.

The GRC Engineer Newsletter by Ayoub Fandi: I really enjoy the GRC Engineer newsletter by Ayoub Fandi, and this issue is one of my favorites. It offers a clear, practical framework for designing security-first controls that reduce real business risk, rather than just satisfying compliance requirements, and shows how this approach can make compliance easier and more meaningful.

Bringing Financial Discipline to Cyber-Risk Decisions by Laura Voicu: This is one of the most practical and accessible deep dives into aligning cyber risk decisions with financial rigor. Laura walks through a ransomware scenario using FAIR, NPV, IRR, and the Gordon-Loeb model to show how to move beyond ROSI and make smarter, defensible investments in security. If you're trying to speak the language of finance while making risk-informed decisions, this is a must-read.

Got recommendations? Send them my way, and I'll feature reader picks too.

🗂 From the Archives

Here are a few blog posts of mine you might enjoy:

“That Feels Too High”: A Risk Analyst's Survival Guide: This post explores why risk estimates often get challenged with comments like “that feels too high,” despite being backed by solid analysis. It outlines three common reasons for pushback: missing information, cognitive bias, and communication gaps, and offers a practical framework to diagnose and respond constructively, turning discomfort into a valuable part of the decision-making process.

Using Risk Assessment to Support Decision Making: This post reframes risk assessments as tools that support decision-making, emphasizing that without a clear decision to inform, defined by choice, preference, and information, a risk assessment often falls short. It offers a practical framework to help analysts focus their efforts, using real-world examples to show how clarifying the underlying decision can make assessments more useful and aligned with business goals.

🙋 Questions

So far in the industry, I've realized that companies still have the appetite to do cyber risk quantification (monetary terms) for a one-off event like a ransomware or a data breach. However, doing it for each and every cyber risk in their register seems difficult.

For example, I worked with an organization that had almost 1500 risks in their cyber risk register and they did it using qualitative analysis. How do you propose an org can use quantitative analysis for a big number like 1500 risks?
- From Varun W.

Great question! You've hit on one of the biggest practical barriers to adopting decision-based risk. The answer isn't to quantify all 1500 risks - it's to fundamentally rethink what belongs in a "risk register."

The problem: That 1500-item list likely contains a mix of vulnerabilities, controls, scenarios, and compliance items that aren't actually decision-relevant risks. Most organizations treat their risk register like a comprehensive inventory rather than a decision-support tool.

The solution: Start with portfolio thinking, not individual risk scoring:

1. Aggregate first - Group those 1500 items into major loss scenarios (ransomware, data breach, system outage, etc.). You probably have 8-12 actual business-impacting scenarios.

2. Quantify the scenarios that matter - Focus on the loss events that could actually influence executive decisions about budget allocation, insurance, or strategic direction.

3. Use the 80/20 rule - A small number of scenarios likely drive most of your actual risk exposure. Quantify those first.

The mindset shift is moving from "we have 1500 risks to manage" to "we have $X million in annual loss exposure across Y major scenarios - where should we invest to reduce it?"

Most of those 1500 items are probably controls or vulnerabilities that feed into the major scenarios, not separate risks requiring individual quantification.

What decision was that 1500-item register actually helping executives make?

✉️ Contact

Have a question about risk analysis or have a general inquiry? Here’s how to contact me:

Reach me at:

• Reply to this newsletter, if you receive via email

• Comment below

• Connect on LinkedIn

• Contact form

What specific risk analysis challenges are you facing? Hit reply and let me know - your question might become the focus of a future deep dive.

❤️ How You Can Help

✅ Tell me what topics you want covered: beginner, advanced, tools, AI use, anything
✅ Forward this to a colleague who's curious about CRQ
✅ Click the ❤️ or comment if you found this useful

If someone forwarded this to you, please subscribe to get future issues.

- Tony

Thanks for being one of the very first subscribers. I'm really glad you're here.

✉️ Why Issue 0?

This is a special celebration edition: part book deal announcement, part welcome note, and part "let's get you started on your first CRQ assessment right now."

From Heatmaps to Histograms: Field Notes officially kicks off in July 2025, with monthly issues featuring CRQ tactics, prompts, tools, and deep dives. Consider this your early-access sneak peek. Thanks for being here from the start.

🎉 HUGE NEWS: I Signed a Book Deal!

I'm absolutely thrilled to share that I've officially signed with Apress to publish my first book! After years of writing, testing, and refining these ideas in the field, From Heatmaps to Histograms: A Practical Guide to Cyber Risk Quantification will hit shelves in early 2026 in both print and ebook formats, wherever books are sold.

This feels surreal and I'm incredibly excited to get this knowledge into your hands. Here's the book description:

Cyber risk quantification (CRQ) is the practice of measuring cybersecurity risk using numbers —not colors or guesswork. Instead of labeling risks “high,” “medium,” or “low,” CRQ uses probabilities, ranges, and impact estimates to help organizations make better, data-informed decisions about risk.

In a world where ransomware gangs operate like small businesses, every core function of an organization is digital, and Boards and regulators are demanding meaningful, defensible risk metrics, CRQ has never been more relevant than now. And thanks to AI, it’s about to scale fast.

At the same time, CRQ is often misunderstood as expensive, technical, or just “voodoo math.” People assume you need a stats degree, six-figure software, or a room full of analysts. This book is here to prove otherwise.

From Heatmaps to Histograms: A Practical Guide to Cyber Risk Quantification is a hands-on, plain-English guide written by a seasoned practitioner who’s built CRQ programs at top global companies. It’s packed with step-by-step instructions, practical tips, templates, shortcuts, AI prompts, and plenty of myth-busting to take you from CRQ skeptic to CRQ champion—even if you’ve never cracked open a statistics book.

All techniques in this book can be performed in Excel or Google Sheets—no coding required. But for readers who want to go further, you’ll find dozens of GenAI prompts that help you generate risk scenarios, clean messy data, or even “vibe-code” your way through a Monte Carlo simulation in Python or R. You'll also get guidance on when to not use AI, how to spot hallucinations, and how to integrate it responsibly into your risk practice.

CRQ is no longer optional. This is your roadmap for making it work—cheaply, ethically, and effectively.

🎯 Your First CRQ Assessment: Start Here

The key insight: Your first CRQ assessment should analyze a risk that informs a concrete business decision that's already on the table. Don't pick a random "hot risk" and hope it becomes relevant later.

Ask: "What decision do we need to make where risk is genuinely a factor?"

Look for upcoming decisions like:

• "Should we buy cyber insurance this year?"

• "Is this $200K security tool worth it?"

• "Is it safer to move this system to the cloud?"

Why this works: You get built-in stakeholders who already care about the outcome, clear success metrics, and immediate relevance.

Pick the Right Risk to Quantify

Once you've identified that business decision, your next step is choosing which risk to analyze that would most influence that choice.

Start with what's already keeping people up at night. Your assessment should tackle something leaders are already discussing:

• "How bad could a ransomware attack really get?"

• "What if we get hit with a data breach like [recent headline]?"

• "What's our realistic exposure if this vendor gets compromised?"

Pick a topic that's already on someone's mind, and you'll get faster traction and better engagement.

Keep it simple and contained. Don't pick abstract risks like "third-party risk" or scenarios with endless branches. Good starter risks have clear loss paths:

High-Impact, Manageable Scenarios

• Ransomware attacks - Plenty of public data on frequency, ransom amounts, recovery costs

• Business email compromise - Clear financial impact, documented rates by industry

• Data breaches - Well-researched cost data available (Verizon DBIR, Cyentia IRIS 2025)

Gather Your Data

Ask "Where can I get numbers?" You don't need perfect data, but pick something you can estimate:

• Has this happened to companies like yours?

• Can you find public research or get SME opinions?

• Can you describe the loss in 4-5 clear steps?

The goal isn't precision; it's building a reasonable model that informs your decision.

Report Risk that Supports the Decision

Your analysis should directly feed into the business choice you identified at the start. The best CRQ assessments answer specific questions:

• Will this help leadership say yes or no to a specific investment?

• Could it inform budget, policy, or insurance decisions?

• Does it feed into an upcoming board presentation or audit?

Remember: CRQ works best when it supports actual business choices, not when it sits in a report that nobody reads.

🤖 Need help brainstorming? Try these GenAI prompts:

For finding business decisions:

I work in [role] at a [company size/industry] organization. Based on typical corporate calendars and security priorities, what are 5 major business decisions coming up in the next 6-12 months where cybersecurity risk analysis could influence the outcome? For each decision, explain: who typically makes it, what risk factors are usually considered, and what data would be most persuasive to decision-makers.

For identifying relevant risks:

I work in [industry sector] and our technology stack includes [list key systems/platforms]. Research the most significant cyber incidents in my industry from 2024-2025. For each major incident, help me understand: what was the initial attack vector, how much did it cost the victim organization, what controls might have prevented it, and what frequency data exists for this type of attack in our sector. Focus on incidents that resulted in quantifiable losses and include the source data.

Don’t forget to double-check everything GenAI comes back with. Ask for citations and references and verify yourself.

The Sweet Spot

A concrete business decision that executives need to make soon, where analyzing a specific cyber risk they're already worried about, with decent available data, would genuinely influence the choice.

Pro Tips for Success

• Start small. Your first assessment doesn't need to be comprehensive. Pick one decision, one risk, one analysis. Do it well, show the value, then expand.

• Focus on "good enough" data. Industry averages plus educated estimates beat pure guesswork every time. Don't let perfect be the enemy of good.

• Make it visual. A simple slide showing "current risk vs. risk after investment" speaks louder than spreadsheets full of numbers.

• Practice the pitch. Before you present, rehearse explaining your analysis in 2 minutes. If you can't make it clear and compelling quickly, simplify further.

Remember: The goal isn't to predict the future perfectly. It's to make better decisions under uncertainty by putting numbers on trade-offs that were previously just gut feelings.

🧭 What You Can Expect Going Forward

That guide above is exactly the kind of practical, field-tested content you'll get every month starting in July.

This newsletter is for anyone trying to do cyber risk quantification in the real world - with limited time, limited budget, and a high bar for quality.

I'll be sharing:

🧪 CRQ tips, tools, and workflows: beginner to advanced, all field-tested

📊 Breakdowns of public reports and datasets (like the DBIR and industry studies)

🧠 Prompts, scripts, and techniques for using GenAI in risk analysis (strategically, not recklessly)

📬 Q&As and AMAs: answering your questions and surfacing the ones we all have but rarely ask

📎 Book previews and behind-the-scenes cuts that didn't make it into the final pages

🔁 Occasional links to great writing and thinking from others in the CRQ and risk world

Some issues will be deep dives. Some will be quick hits. But my promise is this: it'll always be practical.

📚 What I'm Reading

Each issue, I'll share a few blog posts, research articles, or tools worth your time. Here are two standouts I've been diving into:

Operationalizing Cybersecurity Risk Quantification by James Hanbury: This 5-part LinkedIn series tackles the practical challenges of implementing CRQ at scale. What I love about James's approach is how he addresses the organizational dynamics that make or break CRQ initiatives. It's not just about the math - it's about building consensus, creating shared language, and designing metrics that actually drive decisions. Essential reading if you're trying to move CRQ from spreadsheet exercises to business strategy.

Quantifying Cyber Financial Impact: Step-by-Step Guide with Triangular Distribution by Mehdi Kiani: A solid walkthrough of using triangular distributions for modeling cyber losses when you have limited data. Kiani does a nice job showing the actual mechanics. If you're looking for concrete examples of how to move from expert estimates to probability distributions, this hits the sweet spot between accessible and rigorous.

Got recommendations? Send them my way and I'll feature reader picks too.

🗂 From the Archives

Here are a few recent blog posts of mine you might enjoy:

• Six Levers That Quietly Change Your Risk and How to Spot Them: Controls are just one lever. Most changes in risk come from forces far outside your walls, from M&A activity to threat actor capability shifts to regulatory changes. A framework for spotting what's actually moving your risk math.

• AGI Dreams: What Keeps a Risk Professional Up at Night: What happens when a quantitative risk analyst tries to model something that doesn't exist yet? A personal take on artificial general intelligence, uncertainty, and why fictional AI might be our best frame of reference for the real thing.

• Why Ransomware Isn't Just a Technology Problem (It's Worse): Ransomware persists not because of tech failures, but because it exploits economic misalignments between victims, insurers, law enforcement, and vendors. When everyone has competing incentives, the only winners are the criminals.

  ---

🙋 How You Can Help

This is just the beginning, and I'd love to make this newsletter something genuinely useful for you.

✅ Tell me what topics you want covered: beginner, advanced, tools, AI use, anything

✅ Forward this to a colleague who's curious about CRQ

✅ Click the ❤️ or comment if you're into this

If someone forwarded this to you, you can subscribe here to get future issues.

Thanks again for being here. The real fun starts in July.

- Tony

Subscribe now