Industry Thought

Before We Ask Whether AI Works for Emergency Management, We Have to Know What Works

The Journal of Emergency Management is building a three-volume body of knowledge on AI in emergency management. The hardest question it has to answer isn't about AI. It's about us.

I'm writing this from Austin at the AIDE Summit. And on the same morning, the Journal of Emergency Management announced something I've been arguing for in public for years: an independent, rigorous, peer-reviewed knowledge base for AI in EM. It's a three-volume scholarly and professional reference series, Artificial Intelligence in Emergency Management and the Disaster Sciences, and I think it's exactly what the field needs.

I've signed on as the Senior Technical Editor, so I'm not a bystander here. But the reason I said yes has less to do with AI than most people would expect.

What was announced

JEM, led by Managing Editor and Publisher Richard A. DeVito, Jr., is developing the series around a simple progression: UNDERSTAND, APPLY, ADVANCE.

  • Understand builds the foundations: AI concepts, human-AI collaboration, ethics, bias, transparency, privacy, cybersecurity, governance, organizational readiness, and workforce competencies. The common language the profession doesn't have yet.
  • Apply works through the mission: EOCs, situational awareness, forecasting, logistics, public warning, healthcare, public health, EMS, fire, critical infrastructure, damage assessment, training, and community resilience, across mitigation, preparedness, response, and recovery.
  • Advance is the part I care most about: evidence, validation, professional standards, emerging technologies, research priorities, and the unanswered questions.

The editorial team includes Attila J. Hertelendy, PhD; Sonny S. Patel, PhD, MPH, MPhil; Megan Snair, MPH; Emma Erwin, MPH, MSc; and Brenda DeVito, PTA. JEM is now inviting scholars, practitioners, researchers, educators, and subject-matter experts to participate as chapter authors, reviewers, and research collaborators.

That's the announcement. Here's the problem it has to solve.

We don't know what to hand over

Every conversation about AI in emergency management eventually arrives at the same question: what should AI do, and what should stay human?

It sounds like a question about AI. It isn't. It's a question about us. To answer it, you need to know which parts of the practice are understood well enough to be specified, measured, and checked. Most of emergency management never has been. The profession runs on doctrine, tradition, and professional judgment, most of it never independently tested against outcomes. That's not a knock on the people. It's the state of the evidence. We can't tell AI what to do reliably because we've never verified what works. And you shouldn't automate a practice you can't evaluate.

You shouldn't automate a practice you can't evaluate.

Take exercises. HSEEP has shaped how this field practices for more than twenty years, and in all that time it has never been independently verified as effective. Not once. I think it works. Prove it. Planning is the same story. CPG 101 tells you how to build a plan, step by step, and the field follows it. It can't tell you whether plans built that way hold up when the disaster arrives. No one has checked that either. The same holds for how we assess risk, how we write after-action reports, and how we decide a jurisdiction is ready. We have process frameworks. We have very few verified outcomes.

So when an AI tool speeds up any of that work, we face a question we can't answer: did it make the work better, or just faster? Better compared to what? And the question under that one: are we making something faster that shouldn't exist at all?

I know, because I did it

Preppr started as an exercise design platform. I understood exercise design deeply. HSEEP existed as a process. I could describe it, specify it, and encode that expertise into the product. That's what made exercise design automatable in the first place: a practice well enough understood to be written down. Specified, in other words. Not verified.

But look at who wrote it down. What the AI does at scale, across thousands of exercises, was largely defined by me. It had to be. For AI to do a task repeatedly well, someone has to decide what "well" means, and there was no verified standard to hand it. So I made the calls.

And even now, there is no evidence to say whether a one-hour exercise run eight times is better than one eight-hour exercise. Not better according to post-exercise vibes from participants. Better against the highest bar: outcomes when the disaster arrives.

That's the gap. Not whether AI can do the work. Whether anyone has proven the work is the right work.

Exercises are just the corner of the field I know best. Now ask the same question everywhere else. RAND mapped 45 emergency management task areas in the AIDE report. Practitioners named 40 use cases for AI. Go down that list, planning, threat and hazard assessment, situational awareness, public warning, resource management, damage assessment, response, recovery, and ask honestly: have we proven the best approach to any of it? Not vibes. Not feelings. Not "I've done this for twenty years and I just know."

For most of that list, the answer is no. Not because the approaches are wrong. Because no one has checked. Before we add AI to any of it, the field has to be willing to ask that question out loud, and to be uncomfortable with the answer.

What AI can take, and what it can't

The honest map looks something like this.

Where the practice is well specified and the output can be checked against something we already trust, AI is ready to do real work: synthesis, drafting, translation, retrieval, cross-referencing plans against requirements, tracking who has practiced what. This is administrative and analytical load, and it is exactly where AIDE's practitioner workshops found the strongest demand. The rule is clear, the output is checkable, the human stays accountable.

Where the practice is unverified, the risk changes shape. Adding AI to unproven practice is a risk of its own. You don't get better. You get faster. Faster plans, faster exercises, faster paperwork, and no idea whether any of it moved an outcome in a disaster. Speeding up an unproven process doesn't improve it. It scales it. An AI that produces ten times the exercises against a design methodology no one has validated produces ten times the unknowns. The bar can't be throughput. It has to be outcomes.

You don't get better. You get faster.

And where the work is judgment under uncertainty, with lives downstream of the decision, AI hasn't earned the seat. Not because the models can't produce an answer. Because we can't yet say what a good answer is, or prove one was made.

The line between these categories is the most important thing this field needs to draw, and right now it's being drawn by how we've always done it, procurement language, and product demos. It should be drawn by evidence.

Why a journal, and why now

The AIDE report says the thing holding adoption back isn't a shortage of tools. It's trust. That's right, and it raises a harder question: how do you scale trust? One jurisdiction can trust a tool because they piloted it, or because they know the people who built it. A whole field can't. Trust at that scale has to rest on something everyone can inspect: reliable, verified, evidence-based approaches, and AI held to the same standard. Trustworthy because it was tested, not because someone said so.

Drawing that line is research. It needs practitioners who know the work, academics who know how to test it, and technologists who know what the tools can actually do, working together and publishing what they find where it can be cited, reviewed, and argued with. It has to be built independently, and it has to be contestable. A body of knowledge earns authority by being open to argument. That's the only thing that turns "we think this works" into "we know what works, and here's what we still don't."

A body of knowledge earns authority by being open to argument.

The timing isn't optional. Policy, funding, and education are already behind the innovation curve. Tools ship faster than guidance, and guidance ships faster than evidence. That gap widens on its own. The only way it closes is if the field deliberately builds the evidence base, and does it now, while adoption is still early enough to be shaped by it.

What I'm asking

If you're a scholar, a practitioner, a researcher, or an educator with real expertise in AI, emergency management, or the disaster sciences, JEM wants you: chapter authors, reviewers, research collaborators. Details and how to participate are here. The work is slower than a product launch and harder than a conference panel. It also outlasts both.

The question isn't whether AI will enter emergency management. It already has. The question is whether we'll know what we handed it, and whether it worked.

Better compared to what? Let's go find out.