Lessons learned from 17,000 user interviews

The title's a bit misleading. I didn't conduct 17,000 user interviews myself, but I built an AI user interview tool that did - Resubscribe.ai.

I spent several months working on Resubscribe alongside two friends. We took the product from idea to five paying customers before shutting it down in late 2024 due to lack of product-market fit.

Even though we pulled the plug, I wanted to share a bit about my experience and learnings from Resubscribe.

The product

Resubscribe was an AI user feedback tool that helped B2C founders gather continuous qualitative feedback at scale.

You can check out our Product Hunt launch video to learn a bit more.

As a previous app founder myself, I was solving a pain point I had experienced. When working on my previous app, Umi, I spent a ton of energy increasing conversion. Oftentimes I just wanted to directly ask some of my app's users why they didn't want to pay.

However when we reached out via email, users wouldn't respond. Think about it from your own experience - how often do you respond to feedback request emails?

An in-app AI feedback tool seemed like a perfect solution.

Idea maze

Before and after building the tool, we spent a majority of our time talking to founders to understand how they resonated with the problem.

We learned a few things from these user interviews:

1. Quite a few early-stage founders don't intuitively value user discovery. There's a reason why Y Combinator so often repeats "talk to your users".
2. Some early-stage founders already have direct connections with their early customers. They've set up discords or direct text chats and feel like they are receiving enough feedback already.
3. Some founders were able to effectively gather diverse feedback by offering incentives like $100 Amazon gift cards.

In hindsight, this makes sense. Founders are either already talking to customers or don't care.

The customers who would benefit most from our product, were problem unaware (The 5 Stages of Awareness) which is an uphill battle when going to market.

Unintuitive learnings

Even though we didn't find product-market fit, we did learn a bit about the psychology of user feedback in a B2C context.

'Good enough' channels

One trend we saw from B2C founders was that they believed that their current feedback channels were good enough. Many of them were already collecting feedback via some combination of:

• In-app open text field
• Support emails
• App store reviews

While these channels are useful, relying on a single one creates gaps in discovery.

For example, if you rely on receiving support emails for feedback, then you'll only solve obvious issues. You'll fail to discover opportunities and learn more about your users.

Feedback value ceiling

After we activated a few customers and started conducting interviews of their users, we noticed an interesting pattern. We hit a ceiling in the value of the feedback.

Collecting 20-50 user interviews consistently provided value, with each interview yielding new insights.

But when we collected 200 user interviews for the same customer, we found that we were gathering duplicative feedback.

This is obvious in hindsight, but we overestimated how much continuous value we could provide.

People love to share feedback with AI

Unsurprisingly we saw much higher engagement with Resubscribe AI than with a plain text field.

LLMs are great at affirmation and making people feel heard.

• On average, 83% of users who sent one message, sent at least one other message
• The number of responses peaks at 5 messages - that's strong engagement for B2C where users notoriously don't engage!