How Product Decisions Work
It’s all very simple.
But I thought I would write-up a quick post in case there are those of you out there struggling with how to either a) make product decisions or b) think a PM you are working with is taking too long.
The bottom line is this: product decisions are highly complex. In some organizations they take months to make - in others (start-ups for example) days or weeks. But that’s because they are hard choices to make and require gathering consensus. You can’t spitball your way to a successful market-driven product.
You also can’t play pump & dump strategies with features - it doesn’t work long-term (even if it you think it will) and it creates far too much make work for all of those involved.
You will end-up with a product that just is an amalgamation of cruft from all those failed attempts (regardless how small and meaningless they seem - this is a very slippery slope and once it starts) unless there are appropriate reasons in place for each of them and strategies to deal with them when they work and when they don’t work.
There is no argument with this. It happens all the time - so just stop it if you see it happening or starting to happen. It always, always ends badly.
Product decisions, when the inputs have been built up and are being used effectively take everything in to account, have the full benefit of the user in mind while still taking alignment with the overall problem and vision in to account.
The product definition, corporate strategy, user feedback, competitive analysis, analytic data, metrics, win/loss, industry research, and just ideas that are floating around in general, etc, etc… These all need to be consulted as you go. It’s the product manager’s job to track all that data and use it to make effective choices.
Again, trust me - you can’t rely too heavily on one or the other - they will always lead you astray when it comes to product. For example, if you follow what users say exactly you will end-up with features that dead-end because user’s won’t think things through entirely.
A great example is internal ideas. Everyone always knows what’s going to make the product successful - and they are always highly passionate about it. But remember, while they idea may make sense and see really great at the outset, once you start to attempt to map it against other market data, it may not align with what the market tells you.
And you ALWAYS trust the market before co-workers and yourself.
Another example is analytics. You look too hard at where users are clicking, and you may miss the underlying fundamentals - for example, maybe you have no value proposition. Analytics won’t tell you that and you could arrange a page and bucket test it now until the cows come home and it still won’t work.
When you are assessing a priority to give to something, or how to make something better, you need to take all these factors in to account.
Yes, there are circumstances where slapping stuff together may work - but ad-hoc product development, the further you go in the lifecycle, just doesn’t cut it. If it did, everyone would be able to make these decisions and know how to filter out all the noise from the data that has been gathered.
Everyone doesn’t.
So, there is an easy fix to all of this - just talk before taking action. It doesn’t have to be day long meetings - just 5-10 minutes. You’ll be amazed at what you discover is actually being done behind the scenes to improve a product when you have a PM who knows what they are doing.
Vote for PM Bloggers
My good friends Saeed, Alan, and Ethan over at On Product Management are up for a Best Professional / Career Blog award.
Make sure you swing by and vote for them - they run a great blog and always have outstanding posts!
PM Book Store
You may, or may not, have noticed that I have setup a book store on Amazon.com. This is intended for all product managers out there that are looking for some great things to read.
Most of these books I will have read - some I haven’t, but want to read. I will try and review all those that I have read so you get some idea (either on this site or on Amazon) - feel free to send in your own reviews to me if you want me to post them here so folks get a sense of what to read first / what can wait / etc…
And of course, if you have any suggestions for books I should add to the store, let me know!
TPMA SaaS Session
I’ll be headed to the TPMA session this evening, which is entitled: SaaS Dictates Different Product Management Priorities. Register now - if you can make it, make sure to hunt me down / Twitter me / call me / etc… if you are up for talking shop.
Conducting Effective User Sessions
There are certain ways to gather user feedback.
Some PM’s may like the focus group / formal approach. I’m much more of a fan of doing “discounted” or informal user research. I like to be right there, asking the questions, and watching the users get frustrated when something doesn’t work or they can’t figure something out. This is the best way to experience the pain with them so you can get a sense of how urgent or severe it is and prioritize it effectively.
The Problem with Focus Groups
The way that i see it (and I’m not the only one), focus groups leave PMs open to false data. Why? Well, it’s pretty easy. Typically, getting a group of strangers together in a foreign setting (where they feel uncomfortable) and asking them questions about some product they have probably never seen nor used creates a very contrived environment.
They will feel like, and try to, answer how they think they should instead of providing the truth. Of course, there are exceptions to this rule - not every participant will do this. But it will happen more often than not, potentially leaving you with incorrect guidance from your set of users.
So what to do?
Discounted User Feedback
The way I will schedule these sessions is very simple - with individual users. Time for me to sit down with them, asking them questions and really get involved with how they are experiencing the product and what it offers. I really am interested in knowing what sucks and what’s going to make them tear their hair out.
Once you get past the “it’s OK to tell me this is awful” threshold, you’ll find the feedback you receive is really honest. And for the most part, very informative. Of course, like with all sessions similar to this you will want to always ask “why?” to ensure you fully understand what the user is saying.
These types of sessions are called “discounted” because they aren’t heavily controlled - they are informal, and that’s what you want. Think about it. What’s better? Putting a user in a weird environment where they feel like they are being tested - and having a consultant ask them the questions (who doesn’t really know the product all that well to begin with) sitting with a group of their peers, answering really specific questions. The answers to which may be perceived as making them look really smart or really stupid.
You want your users to relax so they can speak freely. That’s going to give you the best feedback.
Collecting the Data
The best way, at least in my experience, is to use both a combination of note-taking and activity models. Don’t worry - it sounds a lot harder than it is.
Really, activity models are very basic sketches of what a user is doing and where they get blocked while using the product. You can see a rough / example activity flow below.
Again, this doesn’t have to be fancy; just capture the user activity (each major step) and then note where they got blocked / frozen so you can look for patterns throughout all user research that’s executed - and hopefully find & solve some problems.
Asking the Questions
The key to all this is knowing the right questions to ask. Ideally, at least to me, you would start out with asking the user (who presumably has never seen the product before) to accomplish the products main goal. For example, imagine working for Microsoft and you are doing this style of user feedback for the Word product. I’d probably start out with a question like (from a blank Desktop): “OK, create, save, and print a Word document with the text ‘1, 2, 3, 4′ in it.”
Some other example questions might be:
- Create a bulleted list
- Adjust the properties of the document to ensure the ‘Author’ field reads ‘Bill Smith’
- Insert a table with 4 columns and 4 rows and make sure the text is center aligned vertically, and left-aligned horizontally
These are some leading questions that should drive the user forward without them just sitting there staring at a blank screen. You want to keep the sessions moving, but you don’t want to push the user to answer for which you are seeking. If a user has difficulties, the user has difficulties - note it and move on. If they can’t overcome them and have to to continue the next pieces of the session, teach them.
They should be able to offer up some ideas to make things much easier - and even if they don’t, you would have just experienced a pain point with a real-life user. Much better than doing anything in a vacuum.
High Satisfaction is Good, but…
Remember, above all else, you want negative feedback / criticisms to come out of user feedback sessions. You don’t want to have to report back to your boss / management that everything is 100% - that’s like saying you don’t have any competitors in the marketplace.
You need to sometimes really hunt for the gems. Users don’t usually know what’s going to be the solution to the problem they are having - that’s not their job. But they (in concert with additional market research, diving in to product analytics, competitive analysis and other such inputs) should offer up a very clear picture of ways in which to proceed.
