Traditional community and government projects tend to move straight to the pilot stage, without testing and iterating on elements of the pilot project. Prototyping a solution, from idea to concept prototype, to rapid prototype to live prototype, allows us to test that idea in real life before committing significant resources to it.
Before investing in the costly development of a new product or service, low-fidelity prototypes allow us to test out an idea more quickly and cheaply than a high-fidelity one. For example, a low-fidelity prototype can be a number of people role-playing a service, without any technological infrastructure, in order to determine if the service is attractive to users.
Earlier prototypes inform the design of a (high-fidelity) pilot. The pilot phase of the solution allows us to conduct a full and robust longer-term test of the idea. We can collect robust and rigorous evidence for impact and outcomes, as well as viability and feasibility. Then, we can conduct an evaluation of the pilot to help us decide how to adopt the pilot and to bring it to scale. If there are significant issues with the solution, including funding/revenue-sustainability and political concerns, it would be cancelled or pivoted before making it to the pilot stage. Pilots that skip prototyping are prone to be dismissed altogether despite significant investments, or worse, renewed as perpetual zombies.
Concepts, Examples and Resources
Fidelity is how closely a prototype resembles the complete and final alternative or solution.
The degree of fidelity can vary along three dimensions:
Look & feel: How much does the prototype look and feel like the final product or service? How many differences are there in the way the user sees or feels (or is it consistent)?
Interactivity and functionality: How much can users interact with the prototype? How much of the functionality is working?
Substance: How much substance does the prototype have compared to the final product or service?
How to identify the level of fidelity needed for your idea?
The optimal prototype is the minimum amount of fidelity needed to achieve what you need to learn. That is, build the minimum required to test the most important questions with your users or stakeholders.
Concept prototypes help us to explore, probe and test solution ideas. They help us to learn about the:
Goals, problems, complexities, definitions, framings and scope
Eventual value proposition for users, beneficiaries and stakeholders
Experiences of users, beneficiaries, and stakeholders
Potential impacts, desired change and viability (acceptability)
Unique contexts, nuances and their implications where the solution idea may be used
Concept Prototyping, Design Kit:
Live prototypes help us to test out solutions in real life. We use them to learn about:
The desirability and value proposition for user, beneficiary and stakeholder
Operational and administrative feasibility
IT and systems compatibility
Unique contexts vs. features of solutions in real life
Diversity and variety versus optimization and precision
Aim for maximal interactivity so users can show you what works for them. Before designing live prototypes you will have already iterated through a series of concept prototypes (see above) and will have addressed enough of your concept prototyping learning goals.
There are no formulas to determining when to build a MVP. Deciding what to build and how much efforts and resources is also a judgement call. These questions are implicit learning objectives to achieve in all the previous prototypes.
MVPs for governments can be particularly challenging. What is the MVP for a piece of regulation, or policy? We propose that the MVP demonstrate:
Desirability and value for users - will people be impacted or changed as the regulation or policy intended? Will they appreciate and agree with the change?
Feasibility for implementors / providers - will government and service providers have the capacity needed to achieve the impact?
Viability for funders / payers - is the capacity and resources needed to achieve the impact worth paying for?
Viability for sponsors - is the social value and political capital created or expended worth it?
This is an area of active development for policy, regulatory and governance experimentation advocates around the world.
Degree of fidelity is the key difference between prototypes and pilot programs. Other differences relate to the level of investment, purpose, and degree of certainty.
Pilot programs have the following characteristics:
High Fidelity: pilot programs are extremely high fidelity. They have fully functioning features, complete content, and polished look and feel.
Resource Intensive: pilot programs are neither quick nor cheap. They require significant resources to develop and implement.
Purpose: a pilot is an extremely high fidelity way to ensure that everything is working as it should be, often as a precursor to scaling. Prototypes test hypotheses; pilots work out the kinks.
Higher Certainty: innovators should have a clear sense of desirability, viability, and feasibility before moving from prototyping to piloting.
While pilot programs can still help innovators test and learn, they are better suited to help innovators learn about implementation and operation rather than earlier stage user research and needs identification.
A well-designed pilot will have benefitted from many rounds of prototyping and iteration.
Prototypes and Pilots: What’s the Difference?
Prototype
Pilot
Purpose
-To quickly generate feedback and insights.
-To test whether an innovation works as intended in context.
Key Questions
-What’s the cheapest and quickest thing I can put in front of someone to get their feedback and start learning?
-Is everything working as it should be? Are we ready to launch?
-Potentially: Are we ready to scale?
Traits
-Quick
-Cheap
-Low Fidelity
-Generative
-Resource-Intensive
-High Fidelity
Use
-Low level of agreement on needs, challenges, and/or desirable solutions
-High level of agreement on needs, challenges, and desirable solutions
Timing
-Can be used at all stages of an innovation process.
-A result of an innovation process.
Examples
-Sketches
-Storyboards
-Wireframes
-LEGO models
-Role playing & Simulations
-Demonstration Projects
-Soft Launches
-Beta or User Experience Testing
-Minimum Viable Product
Prototype designs should:
Be relevant: they answer key questions and address learning goals
Be credible: they are considered legitimate to people using the data
Focus: on critical assumptions and uncertainties on the top 20% of the functions that you need to use 80% of the time
Have the appropriate ‘burden of proof’: different stages of prototyping should aim for progressively more robust evidence. Early concept prototypes should only be aiming for early indication (or hints), whereas pilots should aim to be repeatable and produce robust evidence
Produce timely data: made available in real time, as soon as possible
When creating a prototype plan, here are some considerations to take into account:
What do you really need? Where? When? What permissions do you need to obtain, from whom?
Who needs to be there to run the prototype, experience it and observe what happens?
How many iterations and variations are there on potential outcomes?
What are the sources of data? Can you build it into the prototype?
Who will use the resulting data? When and how will the data be used?
How can the data be presented to be most useful?
You may use the template below to help guide your prototype planning efforts.
Nesta’s DIY Toolkit also has a Prototype Testing Plan available here:
A minimum viable product (MVP) is a product with just enough features to satisfy early users or customers, built with the minimum efforts possible while providing maximum validated learning for future product development (See ). Earlier prototypes tend to focus on testing for different aspects of value proposition and attractiveness for users, technical feasibility and capacity in the field. A MVP is a complete product with a business or delivery/operation model. It is focused on testing the most minimum feasible solution that is attractive and viable - that provides enough value that enough people will really pay for it. A MVP while cheaper than a full product or pilot, is still a significant investment.
Adapted from Cabaj, Tamarack Institute. Small bets before big bets: a framework to evaluate prototypes, 2016.
Making sense of MVP (Minimum Viable Product) – and why I prefer Earliest Testable/Usable/Lovable. Henrik Kniberg. https://blog.crisp.se/2016/01/25/henrikkniberg/making-sense-of-mvp
