A factorial survey shows each respondent a few short descriptions – vignettes – whose attributes are varied by design, and asks for a rating or a yes/no answer. Because the attributes are assigned experimentally, their effects can be estimated cleanly, and when one attribute is a cost, they can be expressed as willingness to pay.
This vignette walks through the workflow with take-away lunch boxes as the example: designing the vignettes, deciding how many respondents are needed, and analysing real ratings.
List the attributes and their levels. Levels may be text or numbers; every level of every attribute will appear in the design.
attrs <- list(
main = c("grilled fish", "teriyaki chicken", "tofu hamburger"),
side = c("no", "yes"),
info = c("no", "yes"),
price = c(700, 900, 1100, 1300)
)
design <- fs_defficient(attrs, n_vignettes = 16, n_blocks = 2,
seed = 1, verbose = FALSE)
design
#> Factorial survey design
#> Vignettes : 16
#> Blocks : 2 (optimal)
#> Attributes : 4
#> Candidates : 48
#> D-efficiency : 98.3%
#> A-efficiency : 96.6%
#>
#> main side info price block
#> 1 teriyaki chicken yes no 1300 1
#> 2 grilled fish yes yes 900 1
#> 3 teriyaki chicken yes no 700 1
#> 4 teriyaki chicken no no 900 1
#> 5 tofu hamburger no no 1100 1
#> 6 grilled fish yes yes 1100 1
#> 7 tofu hamburger no yes 1300 1
#> 8 grilled fish no yes 700 1
#> 9 tofu hamburger yes yes 700 2
#> 10 teriyaki chicken no yes 1300 2
#> ... 6 more row(s)The D-efficiency is reported relative to an orthogonal design of the
same size, so 100 percent is the best attainable.
fs_efficiency() computes the same measure for any design,
including ones built elsewhere; pass the intended levels so that a level
that never appears is caught.
fs_efficiency(design)
#> Design efficiency
#> Vignettes : 16
#> Parameters : 8
#> D-efficiency : 98.3%
#> A-efficiency : 96.6%Respondents are then given blocks and presentation orders that balance carryover, and the rows are turned into the text shown on screen.
design <- fs_vignette_text(
design,
"A {{main}} lunch box, side dish: {{side}}, nutrition label: {{info}}, ",
column = "part1"
)
schedule <- fs_assign(design, n_respondents = 200, seed = 1)
#> Using 8 Williams order(s) of 8 vignettes. Carryover is fully balanced when the number of respondents per block is a multiple of 8.
head(schedule[, c("respondent", "block", "position", "main", "price")])
#> respondent block position main price
#> 1 1 1 1 teriyaki chicken 1300
#> 2 1 1 2 grilled fish 700
#> 3 1 1 3 grilled fish 900
#> 4 1 1 4 tofu hamburger 1300
#> 5 1 1 5 teriyaki chicken 700
#> 6 1 1 6 grilled fish 1100fs_export() writes the design or schedule for a survey
platform, for example as a Qualtrics Loop & Merge table.
Before fielding, simulate the survey under plausible effects – from a pilot or a similar study – and see how often each effect would be detected and how precise the willingness to pay would be.
pw <- fs_power(
design, n_respondents = c(50, 150),
coef = c("(Intercept)" = 1, "mainteriyaki chicken" = 0.7,
"maintofu hamburger" = 0.1, sideyes = 0.3, infoyes = 0.15,
price = -0.0015),
formula = ~ main + side + info + price, price = "price",
n_sim = 50, seed = 1, verbose = FALSE
)
pw[pw$term %in% c("sideyes", "infoyes"), ]
#> Power analysis for a factorial survey
#> Response : rating
#> Simulations : 50 per sample size
#> Significance : 0.05 (two-sided, clustered SE)
#>
#> n_respondents term coef power mwtp mwtp_halfwidth mwtp_relative
#> 50 sideyes 0.30 1.00 200 97.7 0.49
#> 50 infoyes 0.15 0.54 100 96.2 0.96
#> 150 sideyes 0.30 1.00 200 60.3 0.30
#> 150 infoyes 0.15 0.96 100 58.4 0.58A modest effect such as the nutrition label is easily missed with 50
respondents. mwtp_relative is the half-width of the
confidence interval as a share of the true value: 0.3 means roughly plus
or minus 30 percent. Use more simulations (200 or more) for a final
decision.
The bento data hold real ratings of eight lunch-box
vignettes by 48 students, on a 0 to 10 purchase-intention scale.
head(bento)
#> respondent vignette main side info price rating
#> 1 1 1 grilled fish 1 1 700 7
#> 2 1 2 tofu hamburger 1 0 1300 1
#> 3 1 3 grilled fish 1 0 900 0
#> 4 1 4 tofu hamburger 1 1 900 2
#> 5 1 5 teriyaki chicken 1 0 1100 1
#> 6 1 6 teriyaki chicken 0 0 700 1fs_betareg() fits a beta regression, which respects the
bounds of the rating scale. Each student rated eight vignettes, so the
ratings of one person are correlated; cluster makes the
standard errors robust to this.
fit <- fs_betareg(rating ~ main + side + info + price, data = bento,
cluster = ~ respondent, compute_me = FALSE)
summary(fit)
#> Factorial survey beta regression
#> Call: fs_betareg(formula = rating ~ main + side + info + price, data = bento, cluster = ~respondent, compute_me = FALSE)
#> Standard errors: clustered by respondent (48 clusters)
#>
#>
#> Call:
#> betareg::betareg(formula = formula, data = work)
#>
#> Quantile residuals:
#> Min 1Q Median 3Q Max
#> -2.7799 -0.4580 0.2253 0.6149 3.9741
#>
#> Coefficients (mean model with logit link):
#> Estimate Std. Error z value Pr(>|z|)
#> (Intercept) 0.3326180 0.2517675 1.321 0.186457
#> mainteriyaki chicken 0.7632982 0.1504189 5.074 3.89e-07 ***
#> maintofu hamburger 0.1019424 0.1189826 0.857 0.391564
#> side 0.3418223 0.1028982 3.322 0.000894 ***
#> info 0.1379099 0.0838098 1.646 0.099865 .
#> price -0.0016285 0.0002745 -5.931 3.00e-09 ***
#>
#> Phi coefficients (precision model with identity link):
#> Estimate Std. Error z value Pr(>|z|)
#> (phi) 1.7937 0.3227 5.559 2.71e-08 ***
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Type of estimator: ML (maximum likelihood)
#> Log-likelihood: 127.9 on 7 Df
#> Pseudo R-squared: 0.1052
#> Number of iterations: 26 (BFGS) + 1 (Fisher scoring)
#>
#> Fit statistics
#> Log-likelihood : 127.9
#> Log-likelihood (intercept): 103.1
#> AIC : -241.7
#> BIC : -214.1
#> LR test vs intercept : 49.57 on 5 df, p = 1.7e-09
#> Pseudo R2 (Cox-Snell) : 0.1211
#> Pseudo R2 (betareg) : 0.1052The marginal willingness to pay (MWTP) values each attribute in yen: the price change that would offset it.
w <- fs_mwtp(fit, price = "price")
w
#> Marginal willingness to pay
#> Price coefficient : price
#> Interval method : delta method
#> Confidence level : 95%
#>
#> variable MWTP SE lower upper
#> mainteriyaki chicken 468.7263 82.2896 307.4416 630.0109
#> maintofu hamburger 62.6008 68.8856 -72.4125 197.6141
#> side 209.9063 56.8564 98.4699 321.3427
#> info 84.6877 49.3522 -12.0409 181.4163
plot(w)The expected willingness to pay (EWTP) values a whole lunch box – a complete scenario – rather than a single attribute. It is the mean of willingness to pay truncated at zero (Hanemann 1989), applied to complete scenarios as in Tanaka et al. (2027).
boxes <- data.frame(
main = c("grilled fish", "teriyaki chicken", "teriyaki chicken"),
side = c(0, 0, 1),
info = c(0, 0, 1),
row.names = c("Fish, plain", "Chicken, plain", "Chicken, full")
)
e <- fs_ewtp(fit, boxes, price = "price")
e
#> Expected willingness to pay
#> Price variable : price
#> Interval method : delta method
#> Confidence level : 95%
#>
#> scenario EWTP SE lower upper
#> Fish, plain 536.2 69.58 399.8 672.6
#> Chicken, plain 850.1 85.69 682.1 1018.0
#> Chicken, full 1083.0 101.50 884.1 1282.0
plot(e) + ggplot2::labs(x = "Expected willingness to pay (yen)")When respondents answer whether they would accept a scenario at the
stated cost, fs_logit() takes the place of
fs_betareg(), and fs_mwtp() and
fs_ewtp() work in the same way. Simulated data illustrate
it here.
yesno <- fs_simulate_ratings(
design, n_respondents = 300,
coef = c("(Intercept)" = 2, "mainteriyaki chicken" = 0.7,
"maintofu hamburger" = 0.1, sideyes = 0.3, infoyes = 0.1,
price = -0.002),
formula = ~ main + side + info + price, response = "binary",
name = "buy", seed = 2
)
lfit <- fs_logit(buy ~ main + side + info + price, data = yesno,
cluster = ~ respondent, compute_me = FALSE)
fs_mwtp(lfit, price = "price")
#> Marginal willingness to pay
#> Price coefficient : price
#> Interval method : delta method
#> Confidence level : 95%
#>
#> variable MWTP SE lower upper
#> mainteriyaki chicken 294.3987 55.6782 185.2715 403.5259
#> maintofu hamburger 71.9062 56.4467 -38.7272 182.5397
#> sideyes 97.6170 45.6579 8.1291 187.1049
#> infoyes 3.1653 43.9347 -82.9452 89.2758Hanemann, W. M. (1989). Welfare evaluations in contingent valuation experiments with discrete response data: Reply. American Journal of Agricultural Economics, 71(4), 1057–1061.
Kuhfeld, W. F., Tobias, R. D. and Garratt, M. (1994). Efficient experimental design with marketing research applications. Journal of Marketing Research, 31(4), 545–557.
Tanaka, K. (2025). A unified framework for estimating willingness to pay using factorial survey experiments. SSRN Working Paper. https://doi.org/10.2139/ssrn.5238343
Tanaka, K., Dill, J., McNeil, N., Geller, R. and Kondo, N. (2027). What are safer streets worth? Bridging resident preferences and policy design in traffic calming. Transportation Research Part A, 215, 105293. https://doi.org/10.1016/j.tra.2026.105293