{"id":4086,"date":"2019-03-08T05:21:38","date_gmt":"2019-03-08T10:21:38","guid":{"rendered":"https:\/\/blog.mozilla.org\/ux\/?p=4086"},"modified":"2021-06-14T12:13:13","modified_gmt":"2021-06-14T16:13:13","slug":"how-to-validate-an-idea-when-youre-not-working-in-a-startup-2","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/ux\/2019\/03\/how-to-validate-an-idea-when-youre-not-working-in-a-startup-2\/","title":{"rendered":"How to validate an idea when you\u2019re not working in a startup."},"content":{"rendered":"

I had a brilliant idea! How do I get stakeholders to understand whether the market sees it in the same\u00a0way?<\/p>\n

\n
\n
\n

People in startups have tried so hard to avoid spending time and money on building a product that doesn\u2019t achieve the product\/ market fit, so do tech companies. Resources are always limited. Making right decisions on where to put their resources are serious in organizations, and sometimes, it\u2019s even harder to make one than in a startup.<\/p>\n

ChecknShare, an experimental product idea from Mozilla Taipei for improving Taiwanese seniors\u2019 online sharing experience, has learned a lot after doing several rounds of validations. In our retrospective meeting, we found the process can be polished to be more efficient when we both validate our ideas and communicate with our stakeholders at the same time.<\/p>\n

Here are 3 steps that I suggest for validating your idea:<\/p>\n

Step 1: Define hypotheses with stakeholders<\/strong><\/h3>\n

Having hypotheses in the planning stage is essential, but never forget to include stakeholders when making your beautiful list of hypotheses. Share your product ideas with stakeholders, and ask them if they have any questions. Take their questions into consideration to plan for a method which can cover them all.<\/p>\n

Your stakeholders might be too busy to participate in the process of defining the hypotheses. It\u2019s understandable, you just need to be sure they all agree on the hypotheses before you start validating.<\/p>\n<\/div>\n<\/div>\n<\/section>\n

<\/div>\n
\n
\n
\n

<\/h3>\n

Step 2: Identify the purpose of validating your\u00a0idea<\/strong><\/h3>\n

Are you just trying to get some feedback for further iteration? Or do you need to show some results to your stakeholders in order to get some engagement\/ resources from them? The purpose might influence how you select the validation methods.<\/p>\n

There are two types of validation methods, qualitative and quantitative. Quantitative methods focus on finding \u201cwhat the results look like\u201d, while qualitative methods focus on \u201cwhy\/ how these results came about\u201d. If you\u2019re trying to get some insights for design iteration, knowing \u201cwhy users have trouble falling in love with your idea\u201d could be your first priority in the validation stage. Nevertheless, things might be different when you\u2019re trying to get your stakeholders to agree.<\/p>\n

From the path that ChecknShare has gone through, quantitative results were much easier to influence stakeholders as concrete numbers were interpreted as a representation of a real world situation. I\u2019m not saying quantitative methods are \u201cmust-dos\u201d during the validation stage, but be sure to select a method that speaks your stakeholders\u2019 language.<\/p>\n<\/div>\n<\/div>\n<\/section>\n

\n
<\/div>\n
\n

Step 3: Select validation methods that validate the hypotheses precisely<\/strong><\/h3>\n

With the hypotheses that were acknowledged by your stakeholders and the purpose behind the validation, you can select methods wisely without wasting time on inconsequential work.<\/p>\n

In the following, I\u2019m going to introduce the 5 validation methods that we conducted for ChecknShare and the lessons we\u2019ve learned from each of them. I hope these shared lessons can help you find your perfect one. Starting with the qualitative methods:<\/p>\n

Qualitative Validation Methods<\/strong><\/em><\/p>\n

1. Participatory Workshop<\/h4>\n

The participatory workshop was an approach for us to validate the initial ideas generated from the design sprint. During the co-design process, we had 6 participants who matched with our target user criteria. We prioritized the scenario, got first-hand feedback for the ideas, and did quick iterations with our participants. (For more details on how we hosted the workshop, please look at the blog I wrote<\/a> previously.)<\/p>\n

Although hosting a workshop externally can be challenging due to some logistic works like recruiting relevant participants and finding a large space for accommodating people, we see participatory workshop as a fast and effective approach for having early interactions with our target users.<\/p>\n

2. Physical pitching\u00a0survey<\/strong><\/h4>\n

In order to see how our target market reacts to the idea in the early stage, we hosted a pitching session in a local learning center that offered free courses for seniors to learn how to use smartphones. During the pitching session, we handed out paper questionnaires to investigate their smartphone behaviors, interests of the idea, and their willingness to participate in our future user testings.<\/p>\n

\"pitch

The pitching session in a local learning center<\/p><\/div>\n

\n
<\/div>\n<\/div>\n

It was our first time experimenting with a physical survey instead of sitting in the office and deploying surveys through virtual platforms. A physical survey isn\u2019t the best approach to get a massive number of responses in a short time. However, we got a chance to talk to real people, saw their emotional expressions when pitching an idea, recruited user testing participants, and pilot tested a potential channel for our future go-to-market strategy.<\/p>\n

Moreover, we invited our stakeholders to attend the pitching session. It provided a chance for them to be immersed in the environment and feel more empathy around our target users. The priceless experience made our post conversations with stakeholders more realistic when we were evaluating the risk and potential of our target users who the team wasn\u2019t quite familiar with.<\/p>\n

\"pitch

Our stakeholders were chatting with seniors during the pitching session<\/p><\/div>\n

<\/figure>\n

3. User\u00a0Testing<\/strong><\/h4>\n

During user testing, we were focusing on the satisfaction level of the product features and the usability of the UI flow. For the usability testing, we provided several pairs of paper prototypes for A\/B testing participants\u2019 understanding of the copy and UI design, and an interactive prototype to see if they could accomplish the tasks we assigned. The feedback indicated the areas that needed to be tweaked in the following iteration.<\/p>\n

\"user

A\/B Testing the product feature by using paper prototypes<\/p><\/div>\n

\n
\n
User testing can get various results as it depends on how you design it. From our experience of conducting a user testing that combined concept testing and usability testing, we learned that the usability testing could be postponed to the production stage since the detailed design polishment was too early before the production stage was officially kicked off by stakeholders.<\/div>\n
<\/div>\n<\/div>\n<\/figure>\n

 <\/p>\n

Quantitative Validation Methods<\/strong><\/em><\/p>\n

When we realized that qualitative results didn\u2019t speak our stakeholders\u2019 language, we started to recollect our stakeholders\u2019 questions holistically and applied quantitative methods to answer them. Here are the following 2 methods we applied:<\/p>\n

4. Online\u00a0Survey<\/strong><\/h4>\n

To understand the potential market size and the product value proposition which our stakeholders consider of great importance, we designed an online survey that investigated the current sharing behavior and the preference of the features among different ages. It helped us to see if there were any other user segments that were similar with seniors and the priority of the features.<\/p>\n

\"market

The pie chart and bar chart reveal the portion of our target users<\/p><\/div>\n

\n

 <\/p>\n<\/div>\n

\n
\n
\n
\"EDM\"

The EDM we sent out for spreading the online survey<\/p><\/div>\n<\/div>\n<\/div>\n<\/figure>\n

The challenge of conducting an online survey is to find an efficient deployment channel with less bias. Since the age range of our target responses were quite wide (from age 21 to 65, 9 segments), conducting an online survey became time-consuming and was beyond our expectations. To get at least 50 responses from each age bracket, we delivered survey invitations through Mozilla Taiwan\u2019s social account, sent out EDM by collaborating with our media partner, and also bought responses from Survey Monkey.<\/p>\n

When we reviewed the entire survey results with our stakeholders, we had a constructive discussion and progressed on defining our target audience and the value proposition based on solid numbers. An online survey can be an easier approach if the survey scope uses a narrower age range. For making constructive discussions happen earlier, we\u2019d suggest running a quick survey once the product concept is settled.<\/p>\n

 <\/p>\n<\/div>\n<\/section>\n

\n
<\/div>\n
\n
\n

5. Landing Page\u00a0Test<\/strong><\/h4>\n

We couldn\u2019t just use a survey to investigate a participant\u2019s app download willingness since it\u2019s very hard to avoid leading questions. Therefore, the team decided to run a landing page test and see how the real market reacted to the product concept. We designed a landing page which contained a key message, product introduction of the top 3 features, several CTA buttons for email signup, and a hidden email collecting section that only showed when a participant clicked on the CTA button. We intentionally kept the page structure similar to a common landing page. (Have no idea what a landing page test is? Scott McLeod published <\/em>a thorough landing page test guide<\/em><\/a> which might be very helpful for you\u00a0:))<\/em> Along with the landing page, we had an Ad banner which is consistent with our landing page design.<\/p>\n

\n
\n
<\/div>\n
<\/div>\n

We ran our ad on Google Display Network for 5 days and got 10x more visitors than the previous online survey responses, which is the largest number of participants compared to the other validations we conducted. The CTR and conversion rate was quite persuasive, so ChecknShare finally got support from our stakeholders and the team was able to start thinking about more details around design implementation.<\/p>\n<\/div>\n<\/figure>\n

Landing page test is uncommon in Taiwan\u2019s software industry, not to mention testing product concepts for seniors. We weren\u2019t quite confident with getting reliable results at the beginning, but it ended up reaching out to the most seniors we\u2019ve never had in our long validation journey. Here I summarized some suggestions for running a landing page test:<\/p>\n