Using AI to Analyze Large Table Data

Published: at 02:08

Using AI to Analyze Large Table Data: Identifying Key Customer Concerns from 3,046 Words of Feedback

Background

AI is transforming how we work. Tasks that once required significant manpower can now be completed with just a single command.

📊 Data Overview

  • Total queries: 3,046
  • Total characters: 37,124
  • Goal: Extract key customer concerns and deeply categorize them

Recently, I needed to analyze a huge dataset—37,124 characters of user queries spread across 3,046 records. Most of these were filled with unnecessary words, greetings, and short responses with no real analytical value. These were not the real customer concerns.

Challenges

Traditional customer query analysis has major issues:

Using AI for Data Analysis

Now, with AI, we can review all content, categorize it, and identify top concerns—what users care about the most.

How It Works

Writing a “Prompt”

In the AI world, you’re the driver, and your commands (the “prompt”) are what guide the AI to work for you.

As the saying goes: “A good general with no strategy will tire out even the best army.” No matter how powerful an AI model is, if your prompt isn’t good, it won’t perform well.

Here’s the prompt I used to analyze customer concerns:

You are a customer service analyst for a company focused on XXX. Please analyze and categorize large volumes of customer feedback to help optimize our service strategies and product experience.

# Analysis requirements:
1. Parse each query and determine if it contains a genuine customer question. Only genuine questions should be counted; non-questions should be ignored.
2. Group similar questions together and count their occurrences.
3. Ignore irrelevant content (e.g., greetings, "thanks," "OK," short responses).
4. Analyze every single query and output comprehensive statistics.

# Output format:
| Similar Questions | Occurrences | Percentage | Priority | Sample Queries (up to 10 examples) |

# Customer Question List:
{....}

The prompt I wrote isn’t particularly advanced—just the basics. A good prompt should clearly define tasks, steps, and output format and be iteratively improved based on AI feedback.

Testing AI Models

ChatGPT: The “Top Contender”

ChatGPT is the most popular and widely regarded AI model. I use it daily for work and personal tasks.

But this time, it disappointed me.

When I copied and pasted the text, ChatGPT said the input was too long to process. Uploading the document didn’t help either, and its responses failed to meet my requirements.

Claude: The “Newcomer”

Claude has gained a strong reputation for code generation, but it struggled with long text inputs and refused to respond altogether.

Doubao: The “Best in China”

Doubao, developed by ByteDance, is highly praised in China. While its Python script-based analysis looked promising, it couldn’t handle semantic understanding, which was critical for this task.

WindSurf: The “Best Solution”

WindSurf specializes in handling large-scale text, as it’s designed for coding projects that often have thousands of lines. My 37,000 characters were no challenge for it.

WindSurf not only processed the input efficiently but also directly addressed my problem. It categorized, summarized, and provided detailed examples of user concerns, outperforming all the other models.

Key Takeaways

After testing multiple AI models, WindSurf was the clear winner for this specific task.

AI Model Recommendations

The key is finding the right AI tool for your needs and optimizing your prompt strategy!