Validating a problem, an idea, or a solution is where most people begin discovery. The skipped judgment is whether to begin it at all.
“Customers keep asking us to add more dashboard widgets.”
Someone raises it in a product roadmap planning meeting, and it sounds less like feedback and more like an instruction. Twenty-three of the last forty calls mentioned it. The people in the room all nod. It is added to the feature backlog. Discovery, if it happens at all, will now be about how to build more widgets, not whether to build them.
Those nods are a decision. Nobody experiences it as one, which is exactly the problem.
By the time a team is running interviews and testing prototypes, the most consequential call has usually already been made upstream, in silence: the decision that this was worth investigating in the first place. Most teams never examine it. The loudest signal, or the one attached to the biggest customer, becomes the work. From there, a team can do everything downstream with real rigour, frame the problem, design the solution, validate it carefully, all in pursuit of something that was never worth investigating at all.
As building gets cheaper and faster, this becomes more dangerous, not less. When a wrong call costs a quarter of engineering, the judgment about what to pursue matters more than it did when slow building forgave the occasional wrong turn. Before the activities of discovery begin, there is a decision about whether to begin them, and it is the one most often skipped, because it does not look like a decision at all.
Discovery does not start with research
It is tempting to think discovery begins when you start talking to customers. It begins one step earlier, with the call to research this rather than something else. Every signal that reaches a team, a complaint, a feature request, a churn number, a stakeholder’s hunch, arrives carrying an implicit claim: this is worth your attention. Evaluating a signal is the act of testing that claim before you spend anything on it.
This decision has a name, and naming it is the first step to doing it well. Call it Signal Evaluation: the first point in discovery where human judgment, not activity, determines whether a team builds the right thing. Its core question is deceptively plain. Does this signal suggest an underserved job? Get the call right and everything downstream has a chance. Get it wrong, and the cost stays hidden, because it never shows up as a failed test. It shows up as months of good work pointed at the wrong target.
Most signals should not survive
Because the cost hides like that, the bar for letting something through has to be high. Signal Evaluation is not a box to tick on the way to the real work. It is a filter, and a good filter is defined by what it keeps out. If most of what reaches your team makes it into discovery, the filter is not working; it is just a turnstile.
A signal earns its way into discovery only by passing three tests. Each one removes signals that look worth pursuing but are not.
Signal Strength asks whether the signal is real or just loud. How often does it appear? How intensely is it felt? How many people does it affect? Does it correlate with anything you actually care about, like churn or expansion? A single passionate complaint is not the same as a pattern across fifty customers. Both might deserve a glance, but only one suggests a problem worth weeks of work. The most common failure here is mistaking volume for signal: the loudest voice in the room or the biggest logo is treated as representative when it is simply audible.
Job Connection asks what the signal is really about. This is the test that teams most often fail, because failing it feels like success. A signal can be strong, frequent, and genuine, and still point at the wrong thing. The distinction that matters is whether the signal is about the customer’s job or about your product, the shift from product features to the underlying job the customer is trying to accomplish that sits at the centre of jobs-to-be-done thinking (Gecis, 2021). “The export button is confusing” is a feature signal: it concerns your solution and usually warrants a quick fix rather than a discovery effort. “I cannot get my insights to my stakeholders” is a job signal: it is about what the customer is trying to accomplish, and it may hide an underserved need worth real investigation. Teams that skip this test pour discovery into polishing features, even though the actual job is breaking down elsewhere entirely.
Strategic Alignment asks whether to pursue the signal now. A valid signal that does not fit your strategy is still valid. It might be right for a different company, or right for you in a year. Alignment is not about whether the signal is true; it is about whether solving it advances where you are trying to go, whether you have the capability, and whether this is the moment. A real problem you are not positioned to solve is a distraction masquerading as an opportunity.
Pass all three, and you have something rare: a signal pointing to an underserved job worth discovering. Most signals do not get there, and that is the filter working, not failing.
One signal, three verdicts
Abstract tests are easy to nod along to and hard to apply. So return to the signal we opened with, the widgets request the room nodded through, and run it through the three lenses properly this time. Watch it come apart.
On Signal Strength, it passes and passes easily. Twenty-three of forty calls are high-frequency, broad across the base, and clearly real. That is precisely what made the room nod. A signal this strong feels like a mandate, and its strength alone tempts teams to skip evaluation and start building.
On Job Connection, it fails, and this is the catch. “More widgets” is a request about your product, not a description of the customer’s job. The job underneath, the thing they are actually struggling to do, is something closer to “I cannot see what matters at a glance.” Once you see that, the widget request inverts: more widgets might make the real job harder, not easier, by cluttering the very view the customer cannot read. A team that took the signal at face value would have built the opposite of what was needed, fast and with total confidence, because the signal was so strong.
On Strategic Alignment, you never reach it for the widget request, because it failed the previous lens. But the genuine job it points to, helping customers take in their data quickly, would be worth taking to the third lens to ask whether solving it fits your strategy now.
That is the whole discipline in one signal: strong, genuine, and pointing at the wrong thing. The judgment was never in gathering the signal. It was in refusing to take it at face value, even, especially, when it was loud.
A signal that earns its place
A filter that only ever rejects things you watch looks like cynicism. So here is the same process running the other way, on a surviving signal.
A team sees mid-market churn climbing. Before committing to anything, they run it through the lenses.
Signal Strength holds. Twenty-three percent of the quarter’s churned accounts are mid-market, compared with 15% of the base, and the exit interviews show frustration rather than indifference. That is intensity and frequency. Job Connection points somewhere real: churned customers keep saying a version of “we are still making decisions the same way we did before we bought the product.” That is not a complaint about a button. It is a job that is not getting done. And on Strategic Alignment, mid-market is the team’s growth segment, so solving this advances exactly where they are trying to go.
Three passes, three yeses. This signal earns its place in discovery. And notice the bonus: the act of evaluating it has already sharpened the problem. The team walked in assuming the issue was onboarding complexity. The lenses surfaced something quite different, a customer base that had not changed how it makes decisions. They have not run a single interview yet, and the problem is already better framed than it was an hour ago. That is what good Signal Evaluation buys you. Not just a yes or no, but a clearer view of what you would actually be researching.
Why teams skip it
If this decision is so consequential, why do teams skip it? Three reasons, all about pressure rather than ignorance.
The first is the pull of the loudest stakeholder. A signal attached to a big customer or a senior voice carries social weight that has nothing to do with its actual strength, and pushing back feels like a political act rather than a discovery one.
The second is that solution-first signals smuggle in their own answers. When a signal arrives phrased as a feature request, “add widgets,” “build a dashboard,” it has already skipped past the job and proposed the solution. Accepting the phrasing means accepting the solution without ever examining the problem.
The third is confirmation comfort. We evaluate the signals we secretly hope are false more deeply, and we wave through the ones we want to be true. The signal that confirms the roadmap we already love gets the gentlest scrutiny of all.
What doing it well looks like
None of this requires a research project, and that is the part that teams miss. Done properly, Signal Evaluation does not slow you down; it is what saves you the wasted weeks later. It is a quick, honest pass using evidence you mostly already have, support tickets, sales call notes, and the churn dashboard, before you commit anyone to new work.
In practice, it is three moves. First, run the signal through the lenses out loud, with whoever is in the room, so the reasoning is visible rather than living in one person’s head. Second, make an explicit call and write it down: proceed, hold, or stop, with a sentence on why. Writing it down matters more than it sounds, because it turns a silent nod into a decision you can revisit when you learn more. Third, treat a fast “not now” as a real outcome, not a failure to act. A signal you decline today is not gone; it is parked, with a reason, ready to be reconsidered when the evidence or the strategy shifts.
The whole discipline comes down to one refusal: not letting something become work simply because it was loud, recent, or came from someone important. That refusal costs an hour. Skipping it can cost a quarter.
Where AI changes the picture, and where it does not
This is exactly the kind of work AI seems built for, and that is precisely why it deserves care. AI can scan your entire feedback corpus, cluster signals by frequency, surface correlations with churn, and tell you in minutes that the widget request appeared in twenty-three of forty calls. That is genuinely useful and genuinely predictive: pattern-finding at a scale no human can match.
But notice what AI cannot do in that example. It can tell you the signal is strong. It cannot tell you the signal is pointing at the wrong job. That judgment, that “more widgets” is really “I cannot see what matters,” and that building the literal request might make things worse, is meaning-making, not pattern-matching. AI increases the volume of signals you can detect, but it does not reduce the need to evaluate them. It increases it. The more signals you can surface, the more judgment you need about which ones deserve to survive.
The decision before the work
The next time something lands in your discovery queue, try this before you plan a single interview. Take your current top candidate, the thing you are most sure is worth exploring, and run it through the three lenses honestly. Is it actually strong, or just loud? Is it about a job, or about your product? Does pursuing it now fit where you are going? If you cannot answer cleanly, you have found something more valuable than a research plan. You have found a decision you were about to make without realizing it.
Signal Evaluation is one of nineteen such decision points in the Discovery Judgment Framework, the moments across the discovery process where judgment, not activity, decides the outcome.
References
Gecis, Z. (2021, April 7). 8 things to use in “Jobs-To-Be-Done” framework for product development. UX Collective. https://uxdesign.cc/8-things-to-use-in-jobs-to-be-done-framework-for-product-development-4ae7c6f3c30b
About the author
Gale Robins helps software teams and solo founders strengthen discovery judgment, the ability to decide what is worth building when AI makes building faster and cheaper. She is the creator of the Discovery Judgment Framework and the author of its handbook.
Product discovery’s quietest, most consequential decision was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.
