GenAI and the Feature Factory: Automating Away Collaboration

In 2016, John Cutler coined the phrase “Feature Factory” to describe a situation where team members churn out features all day long without a connection to the client, the discovery process, or even knowing what the product vision is. Fast forward to 2025 where we see the rise in the popularity of GenAI, with misused GenAI making our Feature Factory worse.

In Why AI Doesn’t Replace Your ScrumMaster, we shared Guidelines for the use of AI in Scrum Teams and then tested the promises made by various tool vendors regarding the use of AI. In this article, we will examine the good, the bad, and the truly ugly uses of GenAI in the world of Product Owners. (Note: In many of these examples, I’m summarizing what is on a vendor’s website, so remember that, as with all summaries (human or LLM-generated), essential details might have been accidentally missed.)

Speeding Product Backlog Refinement

It has been suggested that GenAI could save time in Backlog Refinement by generating User Stories. People aren’t using a special tool; they’re just asking Claude Sonnet or ChatGPT to create User Stories based on a prompt. One tool vendor even promises “Craft clear, concise, and actionable user stories that propel your project forward with the click of a button.”

My Take

But hang on, let’s consider the real value of User Stories. The goal isn’t to write product requirements with better language. Backlog Refinement is a tool to get everyone on the same page about what to build. User Stories are the outcome of that conversation between the Product Owner and the Team.

Let’s evaluate using our guidelines. Will generating User Stories from a prompt build a common understanding between the PO and the rest of the Team? Will it increase or decrease collaboration? … AI-generated User Stories would seem to harm both.

Further, all of these tools that promise to speed Product Backlog Refinement are helping to speed up an activity that typically only takes 2-3 hours in a two-week Sprint. So the AI is being used to speed up an activity that focuses on collaboration and understanding the Product we’re trying to build, all just so we can save a couple of hours in the entire Sprint? Generating User Stories speeds up the wrong thing, saving time that will harm the quality of the product and make it less likely to meet the customer’s needs. That seems completely of whack. (Harsh, but honest.)

Other tool vendors promise to speed up Story Map creation. This is another collaborative activity where saving a few minutes or hours at the start will ultimately cost us more in the long run.

How could GenAI help? Take a User Story the team has written, along with any additional context — persona description, summary of the vision and strategy, etc. Feed all of this to the tool and get it to ask the group clarifying questions. Most of the questions it generates will be useless, but a few will help reveal assumptions that haven’t been explored. I know because I’ve tried this myself.

AI Evidence-Based Estimation

This was copy and pasted from a tool vendor’s website:

AI brings a new level of objectivity to the estimation process by:

• Analyzing historical data for similar user stories or tasks
• Identifying patterns in estimation accuracy across different task types and team members
• Recognizing and accounting for common estimation biases
• Providing estimation ranges based on confidence levels derived from past performance. Machine learning models can process vast amounts of sprint data to predict effort and complexity with increasing accuracy over time. This data-driven approach helps teams transition from subjective assessments to evidence-based estimations grounded in actual performance history.

My Take

Using GenAI can help make the estimation process more accurate. But let’s consider why we estimate in the first place: we can make forecasts based on estimates; they help us decide how much work can fit in a Sprint; and the act of estimating helps people better understand what they’re trying to create.

Those are benefits of estimation, but what are the risks? Estimation processes are flawed because they create a false sense of precision, consume valuable time, ignore uncertainty, and often become a political tool rather than focusing on delivering value. Taking that view, if estimates are fundamentally flawed, then making them more accurate still leaves us with a flawed system.

Even if estimation is valuable, the vendors offering to remove people from the estimation process are missing some key points:

• Estimation as a Tool for Collaboration - For teams that estimate, a key value of the exercise is building a common understanding of what they’re estimating. An AI doing the estimation won’t help create that understanding.
• Variability - Inaccurate estimates don’t come from humans doing a poor job. The real problem is variability. When asked to build two similar-sounding features, the GenAI estimates should be the same. However, artifical intelligence wouldn’t consider variables like the quality of the code where the change is being made; the number of conversations required with the end users; or hidden dependencies.

AI doesn’t make Estimation better, because Estimation was never the real problem. If these vendors provide tools to help teams ask better questions, the team members themselves might provide better estimates without any automated assistance.

AI for Product Discovery

This is a summary of a real tool:

Product X is designed to streamline the product discovery process by leveraging artificial intelligence to analyze extensive customer feedback. It gathers customer interactions and comments from many sources, including communication channels, in-app feedback mechanisms, and CRM systems, consolidating them into a unified view. This centralization is key to understanding the broader customer narrative. The system employs AI to sift through this large volume of customer data, automatically identifying and categorizing key patterns and actionable insights. It can distinguish between various types of feedback, such as complaints, feature requests, emerging opportunities, or reasons for customer churn. The tool generates reports highlighting significant customer signals, segmenting them by factors like the source of feedback, customer demographics, potential revenue implications, and specific tags.

My Take

• Discovery is a time-consuming task, so automating parts of the process could be helpful.
• Discovery is about finding patterns in large quantities of data, so AI can be helpful for this.
• Risk of errors? All discovery processes have errors, so an AI-aided process isn’t creating a new risk. However, it does underscore the importance of a Product Owner listening to interviews and reading customer tickets themselves so they can spot false positives and hallucinations.

AI-Powered Prioritization

One tool vendor promises to help Product Owners with the insanely challenging task of prioritization. In our CSPO workshops, we dedicate 60-90 minutes to exploring this subject, providing attendees with at least five models.

The tool processes data (user feedback, project history, market trends) to identify critical requirements, analyze relationships, and predict outcomes. AI tools work well at interpreting unstructured data.

Key Details:

• Pattern recognition to categorize requirements
• Predictive analytics for risk assessment
• Automated scoring/ranking based on criteria
• Continuous learning to adapt to new data
• As conditions change, items can be reprioritized quickly and with limited effort
• Early detection of conflicts or inconsistencies

The vendor also highlights key risks, including data quality issues (inaccurate or incomplete data) and overreliance on AI without human validation.

My Take

I’ve reviewed over 30 tool vendors that promise AI will magically make your team better. This example is one of the few where they address risks.

How well does this do with our guidelines?

• Collaboration: It might hurt collaboration depending on how the product owner currently prioritizes.
• Bottlenecks/value: Prioritization can be a time-consuming task.
• Innovation: It enables more frequent prioritization experiments and would help to try different prioritization models.

AI-powered prioritization has potential, but only if it is used as an adjunct and not a replacement. Specifically, the product owner still needs to have a deep understanding of prioritization. In addition, take the time it promises to save and invest it in testing the recommendations.

So, how can Product Owners use LLMs effectively?

Along with the positive mentions above, GenAI/LLM tools are highly effective at generating a large volume of text quickly. They’re also excellent at processing a large amount of text. They’re excellent at distilling large amounts of data from customer interviews. They can help work through thousands of support requests to find recurring themes.

One Product Owner I know used Claude Sonnet to help write a prototype (i.e. throw-away code) for a product feature they wanted to test. Over several days, they went from a rough idea to a prototype that worked with the existing product. The prototype was used to start a conversation with executives and get buy-in for the feature. The same prototype also showed what worked well and what needed usability work. This usage checks many of the boxes in our guidelines because it increased collaboration with the execs and the developers, throw-away code was an acceptable risk and sustainable, and it was innovative in making the previously difficult work practical.

Conclusion

Using GenAI effectively isn’t just about copying and pasting prompts from the internet. Relying on someone else’s set of Product Owner prompts is likely to result in products that don’t solve the customer’s real problems. Instead of creating another Feature Factory, use GenAI to help you see things you might have eventually noticed, or maybe would have missed. Use it to run experiments that were previously impossible.