Here are a couple of sample conversations that a product manager typically goes through, day in day out with different well meaning people in the organisation.
For building a new product/feature
TES (The Enthusiastic Stakeholder)
You (The Product Manager)
—
TES: We should build a section in the portal where people can compare their portfolio and top performers are showcased.
You: Yes, we have talked about it earlier, but do people need it? How do we know?
TES: Well, you won’t know until you give it to them and they try it!
You: Well, we did conduct some experiments around it by sending mailers to people asking whether they’d like to know how they are performing compared to others. Not many people showed interest.
TES: That experiment was flawed. Not many people open email. Unless you gamify it, you can’t really say that this people don’t like it. Gamification is THE thing. It can get people hooked very well.
You: Right, but we have to go by data after all?
TES: Sure, but if we blindly follow the data, how will innovation happen? Did Steve Jobs have data on people wanted iPods? You have to go with your instinct and steer people towards the right products and right behaviour. It is part of our long term strategy to take this route. We need to find ways to make it success. (And some mention of faster horses too in between)
(And it goes on..)
————————–
For enhancing a feature or increasing investment in a feature
TES: We should be making it possible for users to save their searches.
You: Sure. It is in pipeline, but surely we don’t need it right now? There is not much usage of the search screen itself.
TES: Sure, but this will enable users to do so much more and may increase the usage of the filters. Maybe without this, people are not feeling empowered enough. If I can’t save what I searched for, I need to repeat my work next time, hence I won’t do the search itself.
You: Sure, that sounds like a good logic. Maybe we can confirm this hypothesis though by checking the data as to out of the people using search, how many are first timers, and how many have used in past.
(After few days of digging through the data)
You: Well, it turns out we don’t track that dimension and hence we don’t know whether people are dropping off the searches because of not being able to save search.
TES: Okay, then let’s give benefit of doubt to my hypothesis and build the product and see?
You: Yeah, let’s do some quick experiment to see if people would want it and like it.
TES: Sure, but I’d suggest we make a well designed feature and release, because otherwise you can’t tell whether users didn’t want the feature, or they didn’t want it in this shape.
(And it goes on..)
————————–
There is no right or wrong here. Both the parties are probably right in their position. There are different ways of interpreting the data and multiple dimensions of an issue. And you can always counter argue the data with some new factor.
As Product Managers it is their duty to make sure resources are utilised optimally to bring most bang for the buck. But such conversations leave them in dilemmas. Should they go with data or instincts? What to do if data is not there? When to believe users?
Factors and Lines of thinking
There are no definitive answers, but I have some “lines of thinking” and “key factors” to offer, which can help in having a workflow which would lead to satisfactory results often.
These are best explained with examples with the two key factors.
- Nature of change
- Does the feature change the way users do things so far (in real life)?: In this case usually users won’t be able to tell you real feedback until they experience. It is better to build light version and let them use it.
- Does it enable the user to do something she couldn’t do earlier via your product? : In this case the interest can quite strongly be gauged by talking to the users. (Of course you need to study what was the alternative they used to use so far for achieving the same thing, and keep that in mind while designing solution)
- Is it a feature that user probably didn’t speak out loud about but had subconsciously wished/needed? Or maybe a habit that you want to inculcate in them? : In this case you have to lead the behaviour of the user with your conviction as they won’t be able to articulate their needs clearly. Their feedback is much more valuable after the first version or after prototype.
- Type of users
- Users use the product very frequently (Daily or so) or follow a strong pattern e.g. every weekend night: User behaviour data can be strong indicator in these cases. Prefer user interviews before new features. (Be careful about making frequent workflow changes here. Test very well before launching such changes.)
- Users use the product sporadically, and as per need: Need to be patient with the data here. Also, while interviewing users, need to find people in real need. Hence, can be aggressive in releasing features and can rely on behavioural data and instincts.
Of course these are general guidelines and sometimes you might have to do radically opposite things due to various factors being at play, like:
- Whether the product is b2b or b2c.
- Type of user interaction: New in industry or Old in industry but new to the our product
- Cost of experiment, or of gauging the user interest in the feature
- Soft factors like “Enthusiasm of team”, “Stakeholder convictions”
- Some factors would come in the context and call needs to be taken accordingly. E.g. A product might not be used heavily or be business generating, but not having it might cause strong negative emotion with people who need it. E.g. Resetting password
These guidelines are more to tell you about the thought approach one needs to have while making those decisions. Exact outcomes may vary.
Logistical Concerns
Then there are some practical and logistical concerns/questions like how often should product managers meet users? Should every product change be accompanied by user meetings/studies?
Here are my views on those:
- A Product Manager should meet at least 2-3 users in a month.
- Practically, not every small product change can be user tested. Try to utilise your user meetings such that you cover maximum of those with different users.
- Every key feature that involves a few screens better be user tested via prototypes
- In absence of user meetings, definitely try to study user sessions with tools like Inspectlet. They are very accurate in giving you real perspective of the usage. One should study at least 6 such sessions per month.
Workflow
In terms of workflow for when to involve user and the data, here is a simple workflow that can guide the lifecycle of a feature:
Hope this helps!
Would look forward to your thoughts/suggestions.