Public Comment Analysis: A Practical Framework for Project Teams
Public comment analysis is the disciplined process of reading, coding and summarizing what people submitted about a proposed decision, then reporting what it means for that decision. Done well, it turns a pile of letters, emails and form submissions into a clear account of themes, substantive concerns and the changes they justify. This guide walks through that process step by step, with a focus on land use, energy and infrastructure projects.
TL;DR
Public comment analysis is not vote-counting. Specific, well-supported content shapes decisions more than volume does.
Build a coding framework first, tag consistently, and cross-review your coding so that two analysts reach the same reading.
Treat identical, mass, computer-generated and falsely attributed comments with a published policy: de-duplicate, separate and disclose.
AI can give an aerial view of topics, but human validation is a prerequisite for reliable results.
Close the loop with a report showing who was heard, what themes emerged and what changed.
Table of Contents
1. What public comment analysis is, and what it reveals
At its core, public consultation data analysis reviews qualitative feedback, codes it into themes and reports the overall meaning of what was heard. For a land use project, that can reveal which concerns are widespread (traffic, water, noise, visual impact), which are narrow but technically serious, and where a misunderstanding about the project is driving opposition that clarification could resolve.
It also separates two things teams routinely confuse: how strongly people feel and what they actually said. Both matter, but they answer different questions. Intensity tells you where trust is thin. Content tells you what the record requires you to address.
2. Why it is not vote-counting
The Department of Energy’s guidance on the EIS comment-response process states the principle plainly: depth of feeling and interest provide context, but the appropriateness, specificity and factual accuracy of a comment’s content provide the basis for modifying planning documents and decisions. The same guidance describes a sliding scale, where focused comments receive brief responses and detailed, well-supported critiques receive more comprehensive ones.
The practice is visible in the field. In Minnesota’s PolyMet mining review, agency staff coded comments as substantive or non-substantive, and only the substantive ones were deemed to require a response and capable of leading to changes.
For project teams, the lesson is practical: a thousand identical postcards signal broad sentiment, while one engineer’s letter citing a hydrology study may change a design. Your analysis has to capture both without confusing them.
3. A step-by-step workflow
Step 1: Define the decision the comments serve
Before reading anything, write down the decision on the table and the questions comments could inform. A rezoning hearing, a permit application and a draft environmental review each call for different categories.
Step 2: Build a coding framework
The Community Engagement Hub’s feedback coding guide recommends a structured framework with pre-defined categories where possible, reading to identify themes, and consistent tags that make the data actionable. A starter set for a land use project might include:
Category | Example codes |
Environmental | Water, wildlife, air quality, wetlands |
Community character | Density, views, noise, traffic |
Economic | Jobs, tax revenue, property values |
Process | Notice, access to meetings, timing |
Technical | Cites data, study or regulation |
Support | Stated reasons for support |
Add a separate flag for substantive content, meaning comments that raise new facts, analysis, errors in the record or specific alternatives.
Step 3: Read, tag and norm
Thematic analysis means reading feedback for recurring patterns in what people experience and say. Mass.gov’s thematic analysis guidance recommends “norming”: team members compare how they assigned themes to the same messages, find discrepancies and build a common language. Practically, have two analysts code the same 50 comments, compare results, rewrite ambiguous code definitions, then proceed. Repeat the check periodically on large datasets.
Step 4: Summarize by theme, not by commenter
For each theme, record how many comments raised it, the strongest representative points, any factual claims that need verification and the suggested response. Keep verbatim excerpts so decision-makers can see the language people used.
Step 5: Link themes to decisions
Each substantive theme should map to an action: a design change, a mitigation measure, a clarification, or a documented reason for no change. That mapping is the part of the record that holds up later.
4. Handling mass, duplicate and automated comments
Campaign-driven submissions are a legitimate form of participation, and they are also a processing challenge. The Administrative Conference of the United States recommends that agencies manage mass, computer-generated and falsely attributed comments by using tools to de-duplicate, flagging or separating automated submissions, and giving people a way to flag false attribution. It also advises disclosing these policies publicly, which protects trust.
In practice:
Group identical and near-identical comments and count them as one position with a tally, while still reading any unique additions.
Encourage multiple-signature submissions where a group shares one message.
Store computer-generated comments separately from human-authored ones, not delete them.
Publish your handling rules before the comment period closes.
Our own view of how organized campaigns fit into civic participation is covered in the role of public comment campaigns in civic advocacy.
5. Using AI and text analytics responsibly
Natural language processing can compress weeks of sorting. The federal CDO Council’s comment analysis pilot report describes tools that help reviewers identify topics and themes and group duplicate, near-duplicate and semantically similar comments.
Large language models add speed, but they require oversight. Research on benchmarking large language models against qualitative analysis finds they can offer an aerial view of sentiment and topics, while human validation is needed to address hallucination risk, check equity in how different groups’ concerns are weighted and refine categories. Scholars writing on AI-assisted content analysis warn that fabricated output threatens reliability and validity, and recommend triangulating with human coders so invented material is never treated as evidence.
A workable division of labor: let software cluster and de-duplicate, let people define the codes, verify every quoted excerpt against the original and make the final call on what is substantive. For a broader look at tools in this space, see this overview of AI research tools matched to tasks. For the sentiment layer specifically, our guide to sentiment analysis for public affairs and the piece on why one sentiment score fails planners explain what a single number hides.
6. Reporting back so the analysis builds trust
Analysis only earns trust when people see what happened to their input. The World Bank Group’s consultation guidelines call for clear goals, transparent and accessible processes, and feedback to stakeholders on how their input was considered. They define accountability as collating and assessing input, sharing it back and formally conveying it to decision-makers.
A good consultation report records who was consulted, how engagement took place, the key themes, how feedback was considered and what changed. Documenting engagement and resulting changes in this way also gives your team a reusable record for the next project.
A one-page “you said, we did” summary, posted where the comments were collected and mailed to those who asked, is often the most effective single trust-building document a project team can produce.
7. Where analysis fits in a permitting timeline
Comment analysis feeds the moments when a project can still change. Our guide to critical junctures in NEPA public involvement and the article on permitting outreach that does more than notify cover how to time engagement so results arrive while design choices remain open. To understand who is likely to comment before the period opens, try stakeholder analysis methods.
We approach public comment analysis the way we approach a campaign: as a source of real-time intelligence, not an administrative afterthought. Comments tell you where trust exists, where it has broken down and which technical issues could stall a permit. Our view is that the strongest projects treat analysis as a prerequisite for strategy. Granular coding lets a team separate genuine design problems from misunderstandings, and it lets us recalibrate messaging, outreach and coalition work while there is still time to act. We also believe the record should be open. Explaining how comments were grouped and answered is part of building a pathway to implementation that communities can accept. Ignacio
Our perspective
Support with analysis and engagement
If your team is facing a comment period on a land use, energy or infrastructure project, AMAUTA Public Affairs combines community engagement, media relations, digital advocacy, coalition building and permitting support under one campaign-style approach, tailored to each project. Our services are built around evaluating the community landscape, and refining tactics based on real-time feedback.
To discuss your project, get connected with our team.
FAQ
How long does public comment analysis take?
It depends on volume and how many comments are unique. De-duplicating and grouping identical submissions first shortens the work considerably, since the team reads each unique position once. Plan analysis time into the schedule before the comment period closes, so findings can still influence decisions.
Should every comment get an individual response?
Not necessarily. The Department of Energy’s sliding-scale concept ties the depth of response to the content: brief answers for focused comments, fuller ones for detailed, well-supported critiques. Grouping comments by theme and responding to each theme is the common approach.
Can AI analyze public comments on its own?
It can help with clustering, topic identification and finding near-duplicates, but research stresses human validation to catch hallucinations and check that different groups’ concerns are weighted fairly. Use it for the first pass and have people verify every theme and excerpt.
What is the difference between substantive and non-substantive comments?
Substantive comments raise new facts, analysis, errors or specific alternatives relevant to the decision. Non-substantive comments express support or opposition without such content. Both are recorded, but in reviews such as PolyMet, only substantive ones were deemed to require a response.
Sources
Recommended
Comments