1,179 Products and No Agreed Upon Dictionary
What the AIDE report gets right, and the question underneath it that predates AI entirely.
It's National Preparedness Month, and next week I'll be in Austin for the AIDE Summit, speaking on a panel about AI approaches purpose-built for emergency management — alongside Sean Griffin, Tony Fung, Joel Wish, and Tyler Felous, moderated by Matthew Watkins of the Texas Tribune.
I'm glad it's that group. Every one of us has had to make these calls with our own roadmap and our own money: what to build, what to refuse to build, which part of the lifecycle to serve, and what to tell a customer who wants something the technology can't responsibly do yet. Those are concrete decisions with consequences attached, and they make for a more honest conversation than a panel of people theorizing about the market from outside it. Builders should be held to a higher standard of specificity here, not a lower one.
The AIDE report — AI for Disasters + Emergencies: A Way Forward, from the Markle Foundation with Aspen Digital and RAND — came out at the beginning of August. Normally I'd have posted my reaction to something like this within a day or two. I've been sitting with this one for a month instead.
It earned that. 1,179 AI-enabled products from 717 companies, mapped against 45 emergency management task areas, alongside months of practitioner workshops. It is the most serious assessment we have of where this field actually stands, and a fast take would have been the wrong response to it. Some documents are worth reading twice before you say anything. If you work anywhere near this intersection, read it.
What follows is what a month of sitting with it produced. I want to do two things: credit what it gets right, which is a lot, and then sit with a question it doesn't quite ask — one that has nothing to do with AI, and everything to do with whether we'll be able to tell if AI is helping.
What the report gets right
Two findings have stayed with me.
The market is thinnest in preparedness. The report says plainly that this mirrors where our funding has always gone. Product coverage is heaviest in response, moderate in recovery, and thinnest in the work that happens before anything goes wrong. We have under-resourced preparedness for decades, and the product landscape simply hands that back to us.
And this line, which I'd put on a wall: "Multi-organization coordination is weakly automated across the product landscape, despite being the defining feature of how emergency management actually operates."
That is the sharpest sentence in the document. The thing our profession is for — getting organizations that don't report to each other to act coherently under pressure — is the thing the market has built least support for.
The report's diagnosis is that useful tools exist but agencies lack the capacity and trust to use them, and its answer is a capacity-building agenda: a summit, an education program, a playbook, capacity-building pilots, a research agenda, and an alliance. I support all of it. The education piece is where I'd invest hardest.
Why I built in preparedness
The thinness finding is why I built where I built.
There was ample encouragement to go sell into response. That's where the attention is, where budgets move fastest, and where demos look best. A response demo is thrilling. A preparedness demo is a person explaining their planning process.
I don't think AI has earned response yet.
Response is high-consequence, time-compressed, and unforgiving of confident wrong answers. We are nowhere near the evaluation standards that would justify putting a model near those decisions. There is real, working value in AI supporting the information work around response — synthesis, drafting, tracking, translation. That is different from AI near a time-compressed response decision, and we should stop letting the first one launder credibility for the second.
Preparedness, by contrast, is unglamorous, chronically under-resourced, and genuinely improvable right now. So that's where I went, and I'd make the same call again.
The coordination gap has a cause the report doesn't name
Coordinating across organizations requires software that talks to other software. For two decades, the commercial incentive in this space has run the other way.
Lock-in is a better business model than integration. That is the whole explanation. It's how you get a landscape where every tool owns a slice and almost none hand off cleanly — and where RAND found that one in five products depends on another product just to function.
The report describes the coordination gap as a market fact. I'd name it as a market design. Nobody set out to make the field harder to coordinate; a thousand rational commercial decisions did it anyway.
We went the other direction — open interfaces, APIs, agent-facing standards like MCP — so a customer's data and workflow can leave if they want them to. We're not the only ones anymore, and that shift is the most encouraging thing I've seen in this market in years.
The question underneath
Here's what I keep circling, and it has nothing to do with AI.
Preparedness produces documents that get marked complete. Plans, exercises, training records, capability assessments. Almost none of it is systematically evaluated against agreed standards. Mostly we count that something happened. An exercise was held. A plan was updated. A capability was worked on, a capacity was discussed. Very rarely is any of it tied back to how the response actually went.
So when the report asks — as its research agenda properly does — whether AI improves the work, I'd ask the prior question:
Improves it compared to what?
There is no answer key for a plan. And that isn't because AI made the problem hard. It's because we never wrote one.
This matters more than it sounds. Every evaluation conversation in this space eventually arrives at accuracy, and accuracy stalls immediately, because emergency management outputs describe futures that haven't happened. When a practitioner hears "accurate" with no number and no context attached, they assume total. Nearly every accuracy claim in this market is currently unfalsifiable — which, incidentally, punishes the vendors doing the harder work, because rigor is invisible when nothing is measured.
Which means that right now, the real measure is persuasion.
Any vendor can claim accuracy. Any vendor can claim safety. There is no common definition of either in this domain and no standard to be held against — so what actually decides procurements is relationships, sales skill, and marketing. Who you know. Who presented well. Who the emergency manager in the next county already bought from.
I want to be clear that I'm describing a market I'm in, not one I'm standing above. I cannot prove my own accuracy claims to the standard I'm asking for. Nobody can, because the standard doesn't exist. That is the entire problem.
So here's what I think happens next, and I'd rather say it before than after: parts of this field are going to get burned by what they are adopting right now. Not because the technology is bad, and not because anyone acted in bad faith. Because we are making high-consequence purchases on charisma, and some meaningful fraction of those decisions will turn out badly, in public, at the worst possible moment.
When that happens, the damage won't stay with the jurisdiction that bought it. It will land on the whole idea of AI in emergency management, and it will set back the people doing careful work by years. That is the real cost of an unfalsifiable market. Standards aren't a bureaucratic nicety here — they're how a field protects itself from its own enthusiasm, and how the vendors doing rigorous work get to prove it.
But the accuracy problem is still downstream. The root is that the field never built the habit of evaluating its own work against standards, long before any of this was automated.
Worth noticing: in the report's own tally of promising AI use cases, developing after-action reviews sits near the bottom. We are asking AI to help with nearly everything except the mechanism that would tell us whether any of it worked.
What that means for adding AI
Layering AI onto a field that runs on good vibes and requirements language does not make the field clearer. It makes it faster and more voluminous.
We will produce more documents, more quickly, still marked complete, with no better idea whether any of them work. The compliance artifact was never the point — a plan is a lossy compression of operational reality, a receipt for thinking that happened somewhere else. Automating the production of receipts makes them cheaper. It does not make a jurisdiction more prepared, and it can make things worse by making the receipt look more complete than the work behind it.
This is my honest worry about the next three years, and it isn't a worry about the technology. It's that we'll industrialize an unmeasured process and call the throughput progress.
The picture the report doesn't try to paint
One more thing. The report is an accurate picture of where we are, and it doesn't try to be a picture of where we're going. We need both, and the second one is harder.
Here's my version.
Software in this field will change more in the next few years than it has in the last twenty. Practitioners will stop buying an application and adapting themselves to it. They'll have agents build what they need, when they need it, shaped to their jurisdiction and their hazards. The interface stops being the product.
Which means the durable value moves down the stack, to services and infrastructure. The hard parts. The structured data, the operational context, the domain logic and the guardrails that an agent cannot conjure for a user out of nothing. Log in or don't — the point is that the hard bits should be available to you either way, wherever you and your agents decide to do the work.
I think that's good for the field, and I think vendors who are honest with themselves know it's coming.
Why the knowledge layer is the durable one
And that is exactly why the missing foundation matters more, not less.
If agents are going to build against something, this field had better have written down what it knows. What the terms mean. What good looks like. What counts as evidence. What separates a capability that exists from a capability that was discussed.
Otherwise we've automated the production of confident noise at a scale we've never seen — and we still won't have an answer key.
Right now we have 1,179 products and no dictionary. No agreed definition of accuracy for this domain. No competency framework an accredited program could teach from. No standard of evidence a clearinghouse could measure against. A capacity-building agenda is the right response to the adoption gap, and I'll support every part of it. But we should be honest that we're building the field's capacity to adopt AI into a discipline that hasn't finished defining itself.
Every profession that matured did it the same way. It wrote down what it knows — definitions, competencies, standards of evidence — somewhere durable enough to cite and argue with. Emergency management has that. AI in emergency management does not.
What this field needs is an independent body of work, and I mean each word.
Independent of any single vendor, including mine. The moment a company's proprietary framework becomes the field's vocabulary, the vocabulary stops being neutral and starts being a moat.
Independent of any one initiative or funding cycle. Programs end. Foundations change priorities. A profession's knowledge has to outlive whoever paid for the first version of it.
And genuinely a body of work — peer-reviewed, citable, and open to being argued with. That last part matters most. A body of knowledge earns authority by being contestable. Anything that can't be challenged in public isn't a standard, it's a marketing claim with better formatting.
Built by practitioners, researchers, and educators — not by the people selling into the field. I say that as one of the people selling into the field.
That is what I want to see exist, and what I'd want to help build. Some of that work is underway. I'll have more to say before long.
---
If you're at the AIDE Summit in Austin next week, find me — I'd rather have this conversation in person.
And if you're teaching this, researching it, or deploying it in a real jurisdiction: tell me what belongs in it. I'm collecting.
Justin Snair is the founder and CEO of Preppr.