Improving Accuracy in Survey Response Coding Using AI

A behind-the-scenes look at RSG’s AI-powered data workflow and how it delivers deeper insights

August 2025

Business AI and Data Analytics Concept. Businessman analyzes enterprise data management by AI artificial intelligence technology. Big data analytics, dashboard charts, graphs, kpi, marketing analyst,

Household travel surveys remain one of the most powerful tools for understanding how communities move: where people go, how they get there, and what shapes those personal travel decisions. But travel behaviors often do not fit neatly into boxes. Hybrid work, shifting shopping patterns, and wide-ranging open-ended responses all challenge traditional questionnaires.

To meet this complexity, RSG’s data science team has recently started blending trusted survey methods with artificial intelligence (AI) in a way that is both innovative and intentional. Guided by our internal AI Steering Committee and our policies around governance, ethics, and acceptable uses, we apply AI where it adds the most value for our clients while maintaining human oversight, transparency, and client control.

In this article, we unpack how this novel AI application works in practice and why it leads to better community outcomes. We show how AI helps interpret the everyday language people use in surveys, transforming rich but “messy” text into decision-ready insights without losing the original nuance or transparency provided by survey respondents. It’s not just about making surveys faster. It’s about making sure the stories people tell become part of the dataset our clients use to make smarter, fairer planning decisions.

The Challenge with Open-Ended Survey Data

Our surveys, administered using rMove®, ask open-ended questions to capture real-world travel behavior in respondents’ own words. These free-text responses are full of detail, but they are also messy. People abbreviate, misspell, use shorthand, or mix multiple ideas in a single answer. Historically, this made it difficult to extract meaning at scale.

To classify responses into categories like “work,” “school,” or “errand,” we have relied on keyword searches and regular expressions, a kind of pattern-matching system that scans text for exact matches or patterns, like a checklist of keywords. It works well when people use predictable phrasing. But if someone misspells a word, uses slang, or strings together multiple ideas, those rules can break down. Regular expressions aren’t context-aware, so they often miss the meaning behind the words.

A woman at a transit hub checks her phone while transit departure boards glow in the background, representing real-world travel choices captured through modern survey tools.

Real-life travel behavior doesn’t always follow predefined categories. Open-ended survey responses help capture what standard forms can miss—but they require smarter tools to interpret at scale.

“There’s only so far we could get when we were using standard regular expression checks,” said Joseph Trost, a Senior Analyst at RSG and developer of RSG’s approach to AI integration in the data workflow. “Prior to using AI in this part of our data workflow, we found that our automations matched how a human reviewer would have coded the response only 30% of the time.”

That level of accuracy presented challenges and limited the utility of these text responses, especially given the scale of data RSG collects on behalf of our clients. Thousands of responses per study meant we could not rely on manual coding alone, however. And when too many responses are left uncategorized or dumped into “other,” valuable insights get lost. Compounding the downside is the fact those gaps often affect underrepresented communities.

“We couldn’t really churn through all that text-based data effectively and efficiently,” said Nick Fournier, a Data Scientist at RSG. “Without using AI, you can come up with a rules-based approach, but it’s not going to be very smart or insightful”. This strikes at the heart of RSG’s work in this space: Leverage the strength of large language models to summarize complex statements and accurately extract meaning.

To address this challenge, we developed a governed AI workflow—one designed not to chase trends, but to improve fidelity, capture nuance, and free our analysts to focus on more strategic tasks for our clients. For context, not all respondents type in open-ended trip purposes. Most choose from predefined categories, and only those selecting “Other” provide text. That’s where the AI model comes in.

By default, our pipeline applies AI categorization, meaning clients now receive more accurate classifications unless they specifically request we do not use AI (always an option available to them). In all cases, the original respondent’s answer is preserved alongside the AI-coded field. With this workflow in place, alignment with human-coded responses has jumped to over 84%. In practice, that means we are classifying more responses correctly without human intervention, reducing noise, and keeping more voices in the mix. The trip-purpose fields we deliver are now more accurate, more inclusive, and more useful for planning decisions.

How RSG’s AI-Powered Workflow Works

A fully automated, enterprise-grade data workflow sits at the heart of this approach, using AI to process open-ended responses. Built to scale across all our surveys, the system combines speed and consistency with transparency and human oversight at every step.

When a respondent includes a free-text answer in lieu of a listed category/option, our data-processing system flags fields that need classification. Those responses are batched and sent to the secure AI model that compares the free-text response to the categories we’re looking for (like trip purpose). This step is informed by real, human-coded examples to guide the model’s reasoning.

“It's a fully automated process that happens on a nightly basis while surveys are in field,” said Joseph Trost. The AI model then returns a single label per response (just like a human analyst would) and that label is imputed into the dataset alongside the original response. It’s fast, repeatable, and context aware. By the time analysts review the dataset, the most labor-intensive part is already done.

Importantly, the AI is not working in isolation. RSG requires a “human in the loop” for AI applications, meaning that human review plays a key role in keeping the system grounded. Analysts conduct regular quality assurance checks and reviews to catch edge cases or ambiguous language. We retrain the model as needed to reflect updated logic or shifting response patterns.

“I think an underexplored avenue is how we can use AI to decrease survey burden,” Trost added, pointing to opportunities where AI could simplify participation without sacrificing accuracy.

Throughout the survey process and well before, RSG’s clients have influence in this approach. If a project team prefers not to use AI-derived fields, our team can default to rule-based coding or deliver only the raw responses. We always include the original text in the final dataset, so analysts can trace and review every imputed value.

This blend of automation, oversight, and transparency defines how we apply AI responsibly. It allows us to process high volumes of data efficiently without losing sight of quality, context, or trust.

What’s Improved—and Why It Matters

The biggest gain from our AI-enhanced data workflow is simple: we’re getting more answers right. Compared to the 30% match rate from older keyword-based tools, alignment with human coding has jumped to over 84% since implementing AI as part of our workflow. That translates directly into cleaner data and clearer insights without asking more from respondents.

But this is not just about better data hygiene. It is about what that improved accuracy enables. “It’s effectively increasing your sample size,” said Joseph Trost. “And in today’s environment with declining response rates, we want to maximize that as much as possible.” In short, fewer responses get dropped into “other” bins, meaning more complete and representative datasets are delivered.

An urban street scene in motion: a cyclist rides alongside cars and yellow taxis on a wet road, with the Pershing Square bridge visible overhead and buildings reflected in a puddle. The image reflects the complexity of daily travel behavior that RSG’s AI-enabled surveys help classify and interpret.

Real-world travel is complex, often involving multiple modes, nonstandard routines, and overlapping purposes. Smarter data processing helps reveal the full picture, not just the most common trips.

That has ripple effects. The system now correctly categorizes patterns that analysts once lost due to nonstandard phrasing, spelling errors, or multilingual responses. “We’re not leaving people out because they don’t fit into our bins,” said Nick Fournier. “That helps us capture more complex patterns and make sense of more travel behaviors.”

AI also helps us deliver deeper insights. Because the AI applies logic consistently across all responses and all RSG surveys, it improves comparability over time and across regions. That is especially important to our team when aggregating findings or tracking changes.

Responsible AI, Real-World Impact

RSG’s team of data scientists has integrated AI into our data processing flow to transform open-ended responses into decision-ready insights. By doubling alignment with human coding and increasing the value of each response, we are helping clients make more confident decisions that directly shape daily life in our communities. Our process pairs proven methods with AI to bring clarity to complexity, anchored by strong internal governance, transparent workflows, and client-first flexibility.

As planning challenges grow more intricate and response rates decline, extracting full value from every data point becomes even more essential. That’s what RSG’s process delivers today, and we’re already exploring where it can go next. Emerging approaches in survey research hint at the potential for AI to play a more active role in data collection itself, offering a more natural, conversational experience that adapts in real time. From clarifying vague answers to reducing respondent burden, these methods could unlock new ways to engage the public and improve data quality. It is a space we’re looking at closely and one where we see great promise.

····························

This work reflects a collaborative effort across RSG. Matt Landis designed the AI-based workflow described here, and Erika Redding and Joseph Trost implemented it. Special thanks to Nick Fournier and Joseph Trost for sharing their insights for this article.

Curious how we’re using AI to support clearer decisions and more inclusive data? We’d be happy to walk through our approach or discuss ways to enhance your next survey. Contact us today to learn more.

Joseph Trost

Nick Fournier

Can we stay connected?

Sign up for RSG emails to keep up with our news & insights.