Idea generation, rating, and testing

January 6, 2021

We all like to think that all great ideas start with a Newton-type lightbulb moment. But in reality, the majority involve solid detective work to find the raw issues and a distillation process to find clarity. At Eli5, there is a conflict of interest if we do this for a client, so we hand this work over to a trusted consulting partner. However, we still use this process for our own internal projects, so here are our recommendations for idea generation and rating.

Idea Generation

There are two ways to arrive at a new product: from ideas and from problems. The thing you need to look for are the ones that are leveraged by specific knowledge.

Specific knowledge is a term used by Naval Ravikant to describe the innate passions and learnings that make one person a) different from another and b) irreplaceable. Usually, these are highly creative or highly technical - and are the things that we learn through experience, therefore, these are also things that can't be taught. You can read more about specific knowledge here.

We recommend keeping the ideas behind specific knowledge in mind when following the idea generation structure below.

What problems do you have?: A really good place to start is thinking about the problems, roadblocks, and bottlenecks you come across in your daily life. What involves too many steps, what takes a little too long, what is inconvenient or repetitive?

What problems do others have?: Conduct some Problem Identification Interviews with teams. Find out not only what they think their problems are, but also what blockages they hit in their daily lives. These will help you to uncover underlying issues that may not seem like a problem, or are things people didn't realise could be improved on.

What problems do the people around you have?: Once you have spent time on your own problems, start asking those around you what issues they face at home, at work, or during their hobby/recreation time.

What are you an expert in?: The best founders are those who know their industry inside and out. If you don't already know a lot about the industry your product belongs in, you'll need to do some learning upfront

Establish a niche: This links to the problem and your expertise. Once you know the problem you are trying to solve and have advanced knowledge of that problem, then you have a niche that will lead you to your users and a focus that will help you stand out from the crowd.

Examine the market: Speaking of standing out from the crowd, you need to look at the current market to ensure you product ends up in the right place at the right time to achieve success. You also need to think about competitors, both so understand how to beat them and what you can learn about your business potential from things like their revenue, profits and growth

Monetisation: And speaking of revenue, how are you planning to make money? Will this be a short project that just needs to make enough to sustain itself, a side hustle to top-up your salary, or a profitable business?

Idea Development

Once you have your idea you need to think about three things:

  • The Product Concept
  • The Business Model
  • The Value Proposition

We use a Lean Model Canvas, but you can also use a Business Model Canvas to help with this. This article explains the difference between the two and also provides a Lean Model Canvas template for you to use.

These are the common assumptions that you'll need to confirm:

  • Who is your user? Be specific. Even the most popular apps, like Instagram, have a target demographic - 67% of 18 to 29-year-olds use the social media site in 2020 - and many of the newer features like stories, filters, reels and shopping were made for that audience.
  • What problems are you solving? Work out what problems your product solves, then check if your target users a) agree these are valid problems and b) think they are problems worth solving.
  • How does your product solve those problems? The value of your product is found in how it solves your users' problems or make their life better. Pinpoint the ways in which it will specifically do these things as quantitatively as possible. So how will it save your user time or money? How will it help them to perform a task (e.g. cook or exercise) better? We recommend using a Value Proposition Canvas to help.
  • What are the key features of the product? This part will help you to build a Minimum Viable Product. Strip down the product to the key features needed to prove its value. For example, the MVP for Uber would be an app that simply books you a taxi and allows you to pay via the app - it doesn't necessarily matter how this happens, even if it's a Mechanical Turk. All of the other planned features at this point are additional extras.

Idea Rating

You want to have as many, varied ideas as possible, but you want to filter those down so you have only a handful of solutions to test. At Eli5, we use the ICE method invented by growth guru Sean Ellis to help make rational choices.

ICE stands for Impact x Confidence x Ease = Score. Impact should directly relate to the key metric you are trying to improve. Confidence relates to the impact and implementation. Ease  relates to the effort and resources required. These are measured on a scale of 1-10. Confidence is the most subjective, so you should try to back it up with data, research, or some other opinions.

This is very much a 'System 1' method of thinking - fast, automatic, and intuitive. So this method has to lead into some 'System 2' thinking which is slower and more analytical. We need to test.


At the beginning, almost everything you think is an assumption. And because they are assumptions, you need to test them and prove if they are right, wrong, or need some adjustment.  The most critical assumption is usually that a particular audience need and want your product. Then that they will use it instead of their current solution. And finally, that they will use it in the way you think they will.

On the other hand, there will be some minor details that you should simply make temporary decisions about in order to get your product in front of users as soon as possible. So it is important to prioritise the assumptions you need to test, work through them methodically, and learn to become content with the discomfort you might feel knowing you're putting something less than perfect out there. Trust us, it's for the greater good as you'll save lots of time you could have put into an idea that wasn't even going to work out.

The most important ideas to test are those connected to the Value Proposition Canvas you did earlier and the problems they connect to , rather than focusing on the features of the product. So the big question when reviewing ideas is - how can they be tested?

The official name for this is 'Lean Market Validation' [link]. This is a broad term that covers any quick and cheap ways to confirm your product-market fit.

Your own network - Yes, your network will be a bit biased, but this is the easiest way to get some quick, basic feedback. I've even heard of some people doing a 'mum-test'. Generally, these people will be friendly and receptive, so it is crucial to keep asking why they think something is 'good', 'interesting', or 'useful'. Plenty of people could like your product, your focus needs to be solely on those who will actually be willing to use it consistently, or part with their cash for it.

Customer interviews - These are interviews specifically with potential end-users of your product. This is not a survey, but an interview. You should have prepared questions, but the idea is that you can deviate from them to keep following the 'why?' behind an initial response. [Read The 5 Why's] Your interest is in the user and their problem, not your solution, so giving them space to talk is hugely beneficial at this stage.

Expert interviews - If you can, also speak to people that work within your target industry. Specifically those with useful insights like analysts, consultants or C-Level. They can help you go further than just a user-level understanding of the problem you are trying to solve and see it instead from a managerial, financial, and business perspective.

Digital tests - A commonly used method of validation is to create a landing page for your product to get sign-ups prior to launch. If enough people sign up (and paying customers are essential if you're trying to validate an idea you plan to monetise) then you have some idea of the audience there is for your product. You'll need to create some buzz on social media or even pay for some cheap advertising to give it the visibility boost it needs to be a fair test.

Continue reading

Are you ready to take the plunge? Let’s chat.