Merwyn Research, provided by Eureka! Research, is a market research company focused on innovation.
Merwyn is different, because it can get more information out of fewer people, which allows for more opportunities to learn at an overall lower cost than traditional market research.
Eureka! Research delivers the world’s most reliable innovation systems and can increase innovation speed up to 6 times and decrease the risk with innovation by 30 to 80%. Merwyn Research is one of the services Eureka! Research uses to increase speed and decrease risk with innovation.
Eureka Research’s tools are all focused on addressing the key problems with innovation projects such as:
- Which ideas should we commercialize?
- Which of our customer’s problems should be our focus?
- What do consumers think about our products, marketing concept, or packaging?
- What are our odds of success with this idea or marketing message and how do we improve those odds?
- How much sales are possible during the first year and then long-term?
- Can we use a Word of Mouth Marketing campaign and take advantage of social diffusion?
- How can we address the uncertainty in our Marketing Support assumptions?
Over 26,000 innovations ranging from industrial products to consumer products and services have been tested using Merwyn over the past 13 years. The system has been statistically validated in the USA, UK and Canada.
Merwyn Technology® is the decision support technology that powers the truth teller concept scores and sales forecasts. Parts of the technology is also used to power insight mining, idea coach, idea scan and rapid testing.
In its full deployment as part of truth teller the system forecasts the true odds of sales success for multiple sales, marketing and development scenarios. It does this by modeling both the “most likely” forecasting assumptions AND the uncertainty of each assumption at the current moment in time.
Development Team Pedigree: Merwyn Technology® was created by the Merwyn Research team. at They have over 30 years of experience forecasting odds of success for over 20,000 innovations.
Merwyn Research clients include such corporate leaders as: Procter & Gamble, Nike, Walt Disney, Hewlett Packard, Schlumberger, American Express, Ford Motor Company and Johnson & Johnson and the US Department of Commerce.
The Merwyn Research team is lead by Doug Hall, a Chemical Engineer by education who spent ten years at Procter & Gamble where he rose to the rank of Master Marketing Inventor. At P&G Doug led a small team that shipped a record 9 innovations in 12 months. The original rapid test system was implemented at P&G compressing the time for fielding concept and use tests from 3 months to 7 days. Note, today it’s 24 hours. Doug was the founder and inventor of AcuPOLL precision research and Merwyn Technology®.
Forecasting Methodology: Merwyn Technology® is a specialized version of the Fourt-Woodlock system of sales forecasting. It has been modified to make it effective early in development where there is significant uncertainty and to make it effective for both industrial and consumer products as well as services. (For academic details see: Fourt L.A., Woodlock J.W., 1960. Early prediction of market success for new grocery products. Journal of Marketing 25: 31–38.) Fourt Woodlock is also the basis of AC Nielsen/BASES forecasting methodology that holds an estimated 60% share of all innovation sales forecasting by large companies.
Fourt-Woodlock forecasting is based on the class of forecasting known as Decomposition. The bible of forecasting Principles of Forecasting edited by J. Scott Armstrong of the Wharton School and published by Kluwer Academic Publishers defines Decomposition as: “Decomposition is a method for breaking down (decomposing) the estimation tasks into a set of components that can be more readily estimated, and then combining the component estimates to produce a target estimate.”
The book documents the effectiveness of Decomposition especially when there is high uncertainty as exists with innovations.It states, “The research evidence in support of this general principle is enormous and cannot be adequately covered here. Over four decades of research in human judgment and decision making show that decomposition improves judgmental performance over unaided or holistic judgment.”
Six Factors that Drive The Intelligence of Merwyn Technology®
#1. Improved reliability by segmenting data into 3 streams and processing each differently.
A cornerstone of Merwyn methodology is to separate data forecasting inputs into three categories and to handle inputting of the data and processing of data differently for each data stream.
- Facts: Factual inputs are easy to input and use. Examples include Innovation Development Status and Proprietary Protection Level which are used when forecasting fair market royalty rates.
- Estimates: Estimates require four handling steps to aid accuracy and transparency: 1) Calibration, 2) Estimates of most likely values and confidence ranges, 3) Quantification of Uncertainty and 4) Documentation of Basis for Estimate. Each of these are the basis for the next four points of difference.
- Unknowables: Unknowable inputs are not likely to be correct even when the four special handling steps are applied by Fortune 500, small company innovators or independent inventors. To compensate for this, each unknowable is inputted into the Truth Teller model as a constant and reported as such.
In the case of sales forecasting the primary unknowable is Trial Rate. Specifically, the level of Marketing Support (awareness & distribution) as well as the concept persuasion or “pull”. Errors with these numbers from small company innovators and independent inventors are usually a result of ignorance in the quantification of marketing support and persuasion impact. Errors with Fortune 500 Clients are a result of over optimism especially early in the process when actual investment levels are highly variable due to lack of definition of actual marketing message impact, media plan design and efficiencies. This over optimism is generally accepted as the reason for the 25 to 80% error between sales forecasts at the moment of innovation development decisions and actual marketplace results. Obviously, all forecasts are highly accurate after the fact, when the true input data is known.
Instead of inputting marketing, advertising and promotion plans and then translating the plans into effective awareness and distribution impact levels – the Truth Teller system starts with industry standard stock assumptions for distribution and awareness. These are then optimized based on forensic analysis of past results.
Merwyn Concept Testing evaluates ideas by benchmarking versus 50 +/- success factors are evaluated. Each have been validated as predictive of marketplace results with direct links to marketplace success. A set of 5 raters as well as a “computer reading” evaluate each factor relative to predetermined benchmarks. Each rater scores only the presence or absence of each success factor. The Merwyn computer model assigns the weight and importance to each factor to generate an overall probability score. The Merwyn score is translated into a trial rate that is adjusted downward based on the assumed level of distribution and awareness. Research indicates that Merwyn Concept Testing is 7 times smarter than project leaders when evaluating concepts probability of success as it has no emotional connection or bias.
Research has validated Merwyn Concept scores versus AcuPOLL, BASES, Direct Mail Response Tests and other test systems in the USA and Internationally. It’s important to note, that Merwyn is an overall measure of the odds of sustained success – it is NOT a simple measure of purchase intent. The closest analogy to the Merwyn concept score is called “Meaningful Difference.” This is most closely replicated in classic concept research through a blended score that incorporates 60% Purchase Intent and 40% New and Different perception.
Comparison of truth teller Concept results to high precision Information Resources scanner data finds significant correlations with % trial, purchase frequency, percent repeat purchase and $/1000 customers. Impressively, these correlations are raw data to raw data i.e. they are not corrected for differences in marketing support and results.
To further improve the truthfulness of Merwyn Concept Testing both the “most likely” value and the variance (standard deviation) are used in the Merwyn Business Simulation to reflect the true dynamics associated with the innovation concepts persuasive power.
#2. Calibration of Input Estimates to Provide Marketplace Grounding.
Calibration data charts help the person providing the inputs assess, evaluate and determine the proper estimates. Calibration benchmarks include hundreds of listings of the number of final decision makers, number of additional repeats, EBIT percentages, repeat rates, proprietary protection and development status. In the case of custom corporate models calibration data is based on the company’s actual historical results.
#3. Estimates of most likely values and confidence ranges.
To model the true uncertainty of each data input three numbers area requested for each input.: 1) Most Likely Estimate, 2) Pessimistic Estimate and 3) Optimistic Estimate.
- Most Likely: This is the value that based on testing, experience and judgment is the most likely value the innovation will achieve.
- Pessimistic: This is the value where there is only a 1 in 5 chance (20%) that the true number is WORSE than this number. Note, this is not the “worst” possible value. Rather, it’s the mostly likely lowest value. In the case that an inventor has “measured data” on the question being asked they are instructed to provide the low end of one standard deviation.
- Optimistic: This is the value where there is only a 1 in 5 chance (20%) that the true number is BETTER than this number. Note, this is not the “best” possible value. Rather, it’s the mostly likely highest value. In the case that an inventor has “measured data” on the question being asked they are instructed to provide the high end of one standard deviation.
The three numbers are translated into a probability distribution. This natural mapping of the true uncertainty into dynamic forecasts is one of the reasons why Merwyn Business Simulation is significantly more truthful than point estimate forecasting systems. It’s important to note that Truth Teller is designed to be appropriately conservative. As such, the bias in the case of all modeling is towards protecting from “false positives” e.g. overly high estimates. Net, the Truth Teller system appropriately “rounds down” key dimensions.
#4. Detailing of Data Sources and Basis for Assumptions To Increase Transparency
To further enhance credibility of forecasts, innovators are required to document their data sources and basis for assumptions for each forecasting input. This complete transparency allows those viewing the report to quickly and easily asses the wisdom and integrity of the innovator’s assumptions
#5. Monte-Carlo Simulation of Inputs More Closely Simulates the Real World
All input data is processed using a Monte-Carlo simulation. In simple terms, Monte Carlo involves randomly selecting a value from a probability distribution for each variable in the sales forecasting equation. The values are randomly distributed across the range of possible values for each variable. The random values are then inserted into the volume equation and a sales forecast generated. The process is completed some 10,000 times for each marketing support and alternative development scenario (improved product, marketing concept or overall improvement of product and marketing concept).
The big benefit of Monte-Carlo simulation is that it takes into account the true individual and cumulative uncertainty associated with the forecast. In effect, Truth Teller translates innovation uncertainty into defined and quantified business probabilities.
6. Probability Matrix Report Format Aids Decision Making
To aid decision-making, the uncertainty of all inputs is translated into a report that details three estimates that together incorporate the variance of inputs and more closely model the real risk and uncertainty associated with the innovation.
Results from the 10,000 Monte-Carlo simulations for each level of marketing support are rank ordered and the 2,000th value from the bottom reported as the pessimistic estimate. This means there is an 80% chance that the actual sales for this innovation will be at least this amount. The median or 5,000th value is reported as the Most Likely (50%) value. The 8,000th value is reported as the optimistic forecast meaning there is a 20% chance that the true value will be higher than this.
Direct Questions to
Doug Hall 513 310-6374
Fail FAST Fail CHEAP
Innovation Research – Work Systems & Digital Tools
Purpose: Fail FAST Fail CHEAP Innovation Research – Work Systems & Digital Tools grow organizational courage through data driven customer, product and forecasting cycles.
Pedigree: The tools are powered by Merwyn Research technology. Merwyn research and forecasting systems have been validated by Fortune 500 Corporations, the USA National Institute of Standards & Technology and AC Nielsen/BASES. The technology is covered by granted and pending patents around the world.
Portions of the Merwyn Technology are documented in the academic article, The real-time response survey in new product research: it’s about time, by Lynn R. Kahle of the University of Oregon and Doug Hall and Mike Kosinki, Journal of Consumer Marketing Vol 14, No 3, 1997. It is also documented in Doug Hall’s books Jump Start Your Business Brain and Meaningful Marketing.
A comprehensive white paper describing Merwyn Technology is attached. In brief, research finds that when evaluating innovations the Merwyn Truth Teller system is 7 times more accurate than humans at predicting winners versus losers.
Design Premises: The system is built based on three design premises.
1. The primary design premise behind the Fail FAST Fail CHEAP suite of tools is that if customer feedback is made fast and easy then managers are likely to use it to help them make smarter decisions. Conversely, if it’s hard or difficult to do research, then to meet the demand for business speed they will rely on judgement, and judgement has been found to be very inaccurate.
2. The secondary design premise is to provide tools in three levels based on where the innovation is in the process. At the start of innovations use 1) Inspiration Tools. As ideas take form use 2) Smart Advice Tools. As it is time to make significant decisions use 3) Validated Decision Support tools.
3. The third design premise is to add intelligence and reliability into to the innovation research process. This is done through the design of automated systems that make it painless to create a survey, run a survey and analyze the results. What this means is that everyone, everywhere from front line worker to the CEO can create, field and evaluate results painlessly.
Overview of the Three Levels of Fail FAST Fail CHEAP Tools
Inspiration Tools: Innovation projects accelerate when they are founded in customer insights and fresh ideas. The Inspiration tools make it easy for managers to quickly and efficiently get the information needed to get started. They Include:
insight mining: This tool makes it easy to survey customers to understand the frequency and the importance of potential problems that they face. This focuses managers on focusing their investment of innovation effort on things that are meaningful to customers. As with all Fail FAST Fail CHEAP tools, automated reporting and statistical analysis is done painlessly. In addition to statistical data, Insight Mining also provides a patent pending Video Cross Tabs system that allows managers to instantly view video of actual customers reactions. The Insight Mining tool can also be used to evaluate beginning ideas or as we call them “idea starters.” Idea Starters include headlines for products, product forms, packaging, pricing options, etc.. Each is evaluated using the Merwyn measurement system and validated weighting for Meaningfulness and Uniqueness.
idea doubling: When managers get started with innovations it’s not uncommon for them to get “stuck” with one or more components of an idea. This can be their definition of the name, headline, customer, problem, promise, proof or even pricing. Idea Doubling makes it painless for managers to get Fresh Writing ideas for improving a couple of these areas. This can be implemented with internal teams, specially created customer/user panels or even the world wide Innovation Engineering Black Belt community. In addition to Fresh Writing the system can also be used to generate Fresh Math assumptions for all of the inputs that go into a fourt-woodlock sales forecast.
Smart Advice Tools: After an innovation project gains initial momentum it often goes through a valley where it slows down for a multitude of reasons. Smart Advice Tools are designed to provide advice to help navigate the challenges and continue the momentum. They Include:
idea coach: The longer a team works a project the more likely they are to over think or under think on their idea. idea coach provides an intelligent and non-emotional evaluation of the whole idea instantly. The system uses an advanced computer reading system and digital advice system to provide guidance. Because the system is digital and instant there is no embarrassment just fast and smart advice. Users report that after using the system, their ideas improve by 40%. idea coach system can also be customized for individual companies and industries.
idea scan: Within an innovation team there are often differences of opinions that are not openly shared. idea scan captures all team members feedback and wisdom. It uses a simplified version of the Merwyn Truth Teller system to provide quantitative calibration. idea scan delivers a risk adjusted concept score for each individual that is invaluable in leveraging team wisdom and addressing team concerns. The collective score can be used to power a first generation sales forecast.
Validated Decision Support: Validated Decision Support tools are valuable at major decision points such as when deciding: 1) to go into development, 2) to go to market or 3) to make a major investment in a new technology or innovation. These tools are arguably the most validated and intelligent innovation research tools available anywhere.
truth teller: truth teller provides a risk adjusted concept score and sales forecast. The research is powered by 50+/- market success factors by a team of 5 calibrated raters plus advanced computer “reading” of the idea. The concept score powers a series of monte-carlo sales forecasts.
rapid test: rapid test uses real world customer feedback on concept and product (sample or usage) to drive smarter inputs into the truth teller sales forecast and to give additional inputs into the truth teller evaluation. rapid test has been proven in 15 countries around the world in thousands of tests.
Key definitions for truth teller – powered by Merwyn Technology® model
AS – Annual Sales This is consumption sales. Or, said another way, the amount of revenue received from sales that went to Final Decision Makers. This does not include sales that are in process such as “pipeline”, or “inventory fill” associated wth distributor or retailer inventory.
FDM – Final Decision Makers. Final Decision Maker can be end users such as consumers or they can also be decision makers such as factory production managers, architects, doctors, etc. This term is used in place of the classical definition of number of consumers to make Truth Teller applicable for industrial and business to business situations. In the case of Truth Teller this value is the number of “POSSIBLE” Final Decision Makers NOT the net number that will purchase. The number of actual purchasers is calculated by the Truth Teller Model.
TR – Trial Rate The percentage of Final Decision Makers who will make an initial purchase. This number is calculated from the persuasive power of the idea itself as measured by concept testing adjusted for marketing support as measured by percent Awareness and percent Distribution. In the case of Truth Teller these values are customized for each company based on it’s history. Concept testing is based on benchmarking the potential for “customer pull” by evaluating the description on 50+/- factors that have each been found to correlate with marketplace trial at the 95% confidence level or higher. The concept score is translated to actual percent trial using an algorithm developed and confirmed through three independent data sets. The percent trial value is discounted based on the appropriate awareness and distribution levels.
FPR – First Purchase Revenue This is the amount that a Decision Maker will purchase the first time they purchase. For example if you were selling tires the Final Decision Maker might purchase 2 tires or 4 tires. Or 50% of customers might purchase 2 tires and 50% might purchase 4 tires thus the AVERAGE purchase would be 3. In this case, if tires cost 50.00 each the MOST LIKELY estimate of First Purchase Revenue would be 3 x $50 = $150.
RR – Repeat Rate The percentage of Final Decision Makers who will make a second purchase within 12 months of their first purchase. Examples of repurchases include: 1) If the innovation is an industrial or consumer consumable such as a cleaning product then final Decision Maker may purchase multiple times, 2) If the innovation is an industrial production product then Final Decision Makers might purchase a sample of it to test leading to a larger purchase order and 3) If an innovation is an industrial machine their could be a separate service contract or supplies for the machine. With many industrial products or consumer hard goods repeat rate is zero as the first purchase is the only purchase.
RPR – Repeat Purchase Revenue The revenue received when Final Decision Makers make additional purchases. This can be the same number as FIRST PURCHASE revenue; however, it doesn’t have to be. For example – many times a Final Decision Maker will buy a small amount first to try out a product then buy more on an on going basis. Or, sometimes the first purchase is for the equipment and the repeat revenue is for supplies or a service contract.
NRP – Number of Repeat Purchases The number of repeat purchases that are estimated to be made within a 12-month period of time.