Package {LLMRpanel}

Survey and experiment design with language model persona panels

Description

LLMRpanel administers survey and experimental instruments to panels of language model personas. panel_from_margins() draws attributes from supplied margins. panel_from_data() samples microdata rows, and panel_from_personas() uses a prepared persona data frame, such as the bundled anes_2024_personas.

Details

panel_instrument() combines survey items into an instrument. conjoint_design() and conjoint_instrument() create forced-choice conjoint tasks.

panel_administer() sends an independent request for each persona-item pair. It records the item's fixed instrument position and the displayed option order. Choice options can be permuted, and Likert scales can be displayed in their original or reversed order. panel_batch_submit() submits administrations through a provider's batch API.

panel_benchmark() compares response shares with a supplied benchmark. panel_bias_audit() counts parse failures and tests first-option sensitivity. conjoint_amce() estimates conjoint effects from the recorded respondent-level profile assignments.

Author(s)

Maintainer: Ali Sanaei sanaei@uchicago.edu [copyright holder]

See Also

Useful links:

• https://github.com/asanaei/LLMRpanel

• https://asanaei.github.io/LLMRpanel/

• Report bugs at https://github.com/asanaei/LLMRpanel/issues

Persona profiles of 100 respondents to the ANES 2024 Time Series Study

Description

A data frame describing 100 respondents to the American National Election Studies (ANES) 2024 Time Series Study by their demographics and their political attitudes, written as text a language model can read. It serves as ready-made persona material: panel_from_personas() turns its rows into a panel of language-model personas. FocusGroup ships an identical copy.

Usage

anes_2024_personas

Format

A data frame with 100 rows and 123 columns. Each row is one respondent; no ANES case identifier is kept. Column names are handles whose prefix names a block, so startsWith(names(x), "att_iss_") selects the issue positions:

• demo_: 17 demographic fields: age band, sex, education, race/ethnicity, marital status, household income, religion and its importance, religious attendance, census region, rural-urban self-identification, employment status, home ownership, children in the household, union household, military service, and attention to politics.

• att_id_: 9 fields of party identification, ideology, and vote choice.

• att_aff_: 18 feeling thermometers toward candidates, parties, institutions, and groups.

• att_eval_: 14 evaluations: approval of the President, Congress, and the Supreme Court, the economy, personal finances, feelings about the country, and which party would better handle the economy and immigration.

• att_iss_: 47 issue positions, among them taxes and spending, immigration, guns, climate, health care, abortion, policing, race, and foreign policy.

• att_val_: 17 fields of trust, values, and views of democracy.

• ideology_score: A number placing the respondent on one liberal-conservative dimension (see Details); the rows are sorted by it.

Every column but ideology_score holds character values:

• An answer with a value label is written as that label without its code ("Strong Democrat", "Favor a great deal"). Parenthetical qualifiers are kept, the interviewer notes "{VOL}" and "{SPECIFY}" are removed, and the four labels of the immigration policy item (V241386) that the Stata file cuts at 80 characters are restored in full from the codebook.

• The six 7-point scales that ANES labels only at their ends are written as sentences, endpoints included: "5 on a scale from 1 (Government should provide many fewer services) to 7 (Government should provide many more services)".

• Feeling thermometers are written as "70 on a 0-100 feeling thermometer".

• "Haven't thought much about this" is kept as an answer where ANES offered it.

• Age is given in six bands and children in the household as "Yes" or "No". Employment status is written from the full ANES labels without their "R " prefix ("Retired (if also working, working < 20 hrs/wk)"). Religious attendance is "Never" for respondents who said they never attend services (V241439 = 2), who were not asked how often (V241440).

• NA marks a missing-data code: refused, don't know, inapplicable, no post-election interview, and so on, and a thermometer rating outside 0-100 ("don't know where to rate", "don't recognize"). The 100 rows hold 246 such cells, most of them in the vote items, which do not apply to everyone.

The data frame carries these attributes:

• dictionary: A data frame with one row per column except ideology_score: handle; question, the short phrase shown with an answer ("Abortion position"); anes_variable, the ANES variable; domain, "demographic" or the attitude block; wording, the question as printed in the ANES codebook; wording_variable and wording_page, the variable whose question that is and the codebook page where it begins; and wording_status. A status of "verbatim" means the wording is the variable's own question; "from_source_item" means the variable is a summary without a question of its own and the wording is that of the item it summarizes (for party identification, V241221); "stem_only" means the codebook prints only a shared stem ("Where would you place yourself on this scale, or haven't you thought much about this?") and the subject of the scale is in its end labels; "none" means no single question exists (census region, age), and wording is NA.

• demographic_fields: The handles of the demographic columns.

• n_demographics, n_attitudes: 17 and 105.

• source, citation: The citation of the ANES release.

• build: The R and package versions, the md5 sums of the two ANES files, the seed, and the label overrides used to build the data.

• build_audit: For each column, the number of NA cells and the number of missing-data codes in the raw file for the same respondents; the two are equal.

Details

The respondents were drawn from the 4,710 of 5,521 who completed both the pre-election and the post-election interview, neither of them on paper, and answered at least 75 percent of the 122 fields. For each code of party identification, race/ethnicity, and liberal-conservative self-placement not yet covered, one respondent with that code was drawn at random (seed 110); the remaining places were filled by greedy maximin selection, each time adding the respondent farthest from those already chosen, under a Gower distance in which the demographic, identity, thermometer, evaluation, issue, and value blocks carry weights 0.25, 0.25, 0.20, 0.15, 0.15, and 0.15. Each respondent's answers are kept together.

The sample spans unusual combinations of answers and does not reproduce population proportions. It carries no survey weights and supports no population inference. Maximin selection over-represents people with no 2024 presidential vote recorded (38 of the 100, against 18.5 percent of the eligible respondents), and within each party it favors members whose other answers depart from the party's usual positions.

ideology_score comes from a one-dimensional graded-response item response model fitted to 89 attitude items over the 4,710 eligible respondents. Categories enter in the order of their ANES codes (thermometers grouped into five bands, off-scale answers such as "Haven't thought much about this" left out), and items are reversed where they correlate negatively with party identification and ideology. The expected a posteriori scores are standardized over the eligible respondents and oriented so that low is liberal and high is conservative. They correlate 0.89 with party identification and ideology among the eligible respondents and 0.58 among the 100, whose scores have a standard deviation of 0.55.

Items on sexual orientation and gender identity are excluded by design, as are restricted-use variables, geography below the census region, open-ended answers, and items asked only of the spouse/partner sample, only on paper, or only of the GSS re-interview sample, which this release does not include.

Source

American National Election Studies. 2026. ANES 2024 Time Series Study Full Release [dataset and documentation]. May 19, 2026 version. www.electionstudies.org. Work that uses these profiles should cite ANES. ANES bears no responsibility for the analyses or interpretations presented here.

Examples

dim(anes_2024_personas)
dictionary <- attr(anes_2024_personas, "dictionary")
dictionary[dictionary$handle == "att_iss_abortion_position",
           c("question", "wording", "wording_status")]
table(anes_2024_personas$att_id_party_identification)
issues <- anes_2024_personas[, startsWith(names(anes_2024_personas), "att_iss_")]
ncol(issues)

Attach the persona contract to a data frame

Description

Attaches persona metadata to a decoded data frame. When a persona_frame is passed to panel_from_data() without a template, demographic fields and stated answers are rendered separately. A question map supplies the wording used for answer fields. Plain data frames use the flat key-value rendering.

Usage

as_persona_frame(data, questions = NULL, demographics = NULL, answers = NULL)

Arguments

Value

data with the persona contract attached and class persona_frame.

See Also

panel_from_data(), panel_from_personas(), LLMR::llm_persona_split().

Examples

df <- data.frame(
  age = c("35-44", "65+"),
  pid = c("Strong Democrat", "Strong Republican"),
  ab  = c("Always legal", "Never legal"))
pf <- as_persona_frame(
  df,
  questions = c(pid = "Party identification", ab = "Abortion position"),
  demographics = "age")

AMCEs from a conjoint administration

Description

Average marginal component effects from a conjoint_instrument() administration: one OLS regression of profile choice on treatment-coded dummies for all attributes simultaneously, with CR1 cluster-robust standard errors clustered by persona and 95% intervals on the t distribution with G - 1 degrees of freedom (G personas). Under uniform, independent profile randomization this is the standard AMCE estimator. The regression uses the respondent-level profiles recorded during administration, not the profiles in the initial design table.

Usage

conjoint_amce(responses)

Arguments

Details

Each attribute's effects are relative to its reference level: the first level, in the design's order, that appears in the estimation data. That is the first declared level unless every profile showing it was lost, for example because the executions that drew it failed or their replies did not parse. A warning names the attribute and the level that took its place when this happens, and also names any other declared level that is absent and so has no row in the table.

Value

A conjoint_amce tibble: attribute, level, estimate, std_error, ci_lo, ci_hi. Reference levels appear with estimate 0 and std_error = NA, so the table feeds the familiar conjoint plot directly. The ordinary columns n_profiles, n_respondents, n_dropped_na, and n_execution_failures record the profile rows used, the respondents administered (including those whose responses failed), missing task responses dropped, and failed executions. The attribute reference is a character vector naming each attribute's reference level, and n_clusters is the number of personas with at least one estimated response, the clusters behind the standard errors.

References

Hainmueller, Jens, Daniel J. Hopkins, and Teppei Yamamoto (2014). "Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments." Political Analysis 22(1), 1-30.

Examples

set.seed(110)
panel <- panel_from_margins(list(group = c(A = .5, B = .5)), n = 6)
design <- conjoint_design(
  list(color = c("blue", "red"), cost = c("low", "high")),
  n_tasks = 6)
instrument <- conjoint_instrument(design)
cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b")

# A synthetic respondent who always takes the red profile: the AMCE for red
# is recovered, and cost is flat.
prefers_red <- function(experiments, ...) {
  shown <- vapply(experiments$messages, `[[`, "", "user")
  first <- sub("Profile 2.*", "", shown)
  experiments$response_text <- ifelse(grepl("red", first),
                                      "Profile 1", "Profile 2")
  experiments
}
r <- panel_administer(panel, instrument, cfg, .runner = prefers_red)
conjoint_amce(r)


if (nzchar(Sys.getenv("GROQ_API_KEY"))) {
  conjoint_amce(panel_administer(panel, instrument, cfg))
}

Conjoint tasks

Description

Random profile pairs (or k-tuples) over the supplied attributes, the design for a forced-choice conjoint. Profiles are sampled uniformly and independently per attribute; set a seed beforehand for a reproducible design (the function never sets one). At administration, fresh profiles are drawn independently for every respondent from the same attribute levels.

Usage

conjoint_design(attributes, n_tasks = 5L, profiles_per_task = 2L)

Arguments

Details

Within a task, a profile that repeats one already drawn is redrawn, up to 25 times. When the attribute space holds fewer distinct profiles than profiles_per_task, every task repeats a profile. Otherwise a repeat remains only when all 25 redraws repeat a profile, which can happen when the space is barely larger than the task. Either case draws a warning that says which occurred. The profiles drawn for each respondent at administration follow the same rule without a warning.

Value

A conjoint_design list with fields profiles, a tibble containing task, profile, and one column per attribute, and attributes, the named list of attribute levels. Render it into forced-choice items with conjoint_instrument() and estimate with conjoint_amce() after administration.

Examples

set.seed(110)
conjoint_design(
  list(price = c("$10", "$20"), speed = c("slow", "fast")),
  n_tasks = 4)

Build a conjoint instrument

Description

Converts a conjoint design into one forced-choice item per task. Administration draws profiles for each persona and asks the model to choose a profile by its label.

Usage

conjoint_instrument(design, question = "Which profile do you prefer?")

Arguments

Details

Each persona-task pair is a separate request. Its item_position is instrument metadata and does not describe an order of exposure. Profile choice options are permuted for each request.

Profile descriptions follow profile-number order. Attributes within a description follow the design's attribute order. The initial profile table in design is not reused across personas.

Value

A panel_instrument with task identifiers such as task_1 and profile options such as "Profile 1". Each item retains the attribute levels used for profile draws. The instrument's conjoint field retains the design metadata used by conjoint_amce().

Examples

set.seed(110)
panel <- panel_from_margins(list(group = c(A = 1)), n = 4)
design <- conjoint_design(
  list(economy = c("weak", "strong"), taxes = c("lower", "higher")),
  n_tasks = 3
)
instrument <- conjoint_instrument(
  design, "Which candidate do you prefer?"
)
cfg <- LLMR::llm_config("groq", "example-model")
runner <- function(experiments, ...) {
  experiments$response_text <- "Profile 1"
  experiments
}
panel_administer(panel, instrument, cfg, .runner = runner)

Administer an instrument to a panel

Description

Creates one independent request for each persona-item pair. The persona is placed in the system message, and the item and its options are placed in the user message.

Usage

panel_administer(
  panel,
  instrument,
  config,
  max_calls = 5000L,
  confirm = FALSE,
  price_table = NULL,
  tokens_per_call = NULL,
  .runner = NULL,
  ...
)

Arguments

Details

Closed-item replies are matched to the offered options. Unmatched replies receive NA. Open-item replies retain their text with surrounding whitespace removed.

Requests do not contain earlier items or responses. Instrument position therefore does not describe an order of exposure.

With "option_order" randomization, choice options are permuted for each request. Likert scales retain their supplied order or are reversed, each with probability one-half. Likert scores always use the supplied scale order.

Value

A panel_responses object.

• data contains one row per persona-item pair. Its identifiers are persona_id, item_id, and type.

• item_position is the item's fixed, 1-based instrument position. option_order records displayed options separated by |.

• response contains the matched option or open-item text. score gives the 1-based position on a Likert item's supplied scale.

• Execution columns retain response_text, response_id, success, error_message, finish_reason, model, and provider. Request hashes, model versions, and creation times are retained when supplied by the runner.

• Conjoint administrations include recorded profile draws in a profiles list-column.

• panel and instrument retain the supplied study objects. benchmark is NULL until panel_benchmark() is called.

• usage retains available token or duration records and their execution identifiers. It is NULL when no usage fields were returned.

Examples

set.seed(110)
panel <- panel_from_margins(
  list(party = c(left = .5, right = .5)), n = 6
)
instrument <- panel_instrument(
  item_likert("wk4", "A four-day work week would benefit society.")
)
cfg <- LLMR::llm_config("groq", "example-model")
runner <- function(experiments, ...) {
  experiments$response_text <- "agree"
  experiments
}
responses <- panel_administer(
  panel, instrument, cfg, .runner = runner
)
responses$data

Fetch and parse a completed panel batch job

Description

Retrieves the batch results and parses them into a panel_responses, identical in shape to a synchronous panel_administer() run. Responses are joined to the grid by request id, so the order the provider returns them in does not matter.

Usage

panel_batch_fetch(job)

Arguments

Value

A panel_responses object.

See Also

panel_batch_submit(), panel_batch_status().

Examples

# Fetching reads the job saved at submission, once the provider reports it
# complete; the result has the same shape as a synchronous run.
state <- file.path(tempdir(), "panel_job.rds")

if (file.exists(state)) {
  responses <- panel_batch_fetch(state)
}

Check the status of a panel batch job

Description

Check the status of a panel batch job

Usage

panel_batch_status(job)

Arguments

Value

The LLMR batch status (a one-row tibble).

See Also

panel_batch_submit(), panel_batch_fetch().

Examples

# A job saved by panel_batch_submit(state_path = ) is read back by path,
# so its progress can be checked from a later session.
state <- file.path(tempdir(), "panel_job.rds")

if (file.exists(state)) {
  panel_batch_status(state)
}

Administer a panel asynchronously through the batch API

Description

Submits one request per persona and item to a provider's batch API and returns a job handle. Use panel_batch_status() to inspect the job and panel_batch_fetch() to retrieve completed results. Provider services determine prices and completion times.

Usage

panel_batch_submit(
  panel,
  instrument,
  config,
  state_path = NULL,
  max_calls = 5000L,
  confirm = FALSE
)

Arguments

Details

All personas are administered under one config (one model). The handle carries the survey prompts and rendered persona text. When state_path is supplied, the API key must be referenced through an environment variable so its value is not written to the saved state.

Value

A panel_batch_job handle.

See Also

panel_batch_fetch(), panel_batch_status(), panel_administer().

Examples

# Three personas and one item make a batch of three requests.
panel <- panel_from_margins(list(party = c(left = .5, right = .5)), n = 3)
instrument <- panel_instrument(item_likert("wk4", "A four-day work week helps."))
cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b",
                        api_key = LLMR::llm_api_key_env("GROQ_API_KEY"))
state <- file.path(tempdir(), "panel_job.rds")


if (nzchar(Sys.getenv("GROQ_API_KEY"))) {
  job <- panel_batch_submit(panel, instrument, cfg, state_path = state)
  panel_batch_status(job)
}

Compare panel responses with a human benchmark

Description

Compares closed-item response shares with human benchmark shares. The comparison uses responses that parsed and were not reported as execution failures. It does not alter responses or adjust their shares.

Usage

panel_benchmark(responses, benchmark, benchmark_name = "benchmark")

Arguments

Details

Without a benchmark, response shares describe the configured model under the supplied personas.

The function refuses conjoint responses because profile assignments differ across personas. It also refuses a covered item when every execution failed. If executions occurred but no response parsed for an item, its panel shares and deviations are NA.

Value

responses with its benchmark field populated.

• table contains the panel share, human share, and their difference for each covered item-response pair.

• nonresponse contains execution-failure and nonresponse rates by item.

• items_covered and items_total record closed-item coverage.

• mean_abs_dev and max_dev summarize absolute deviations.

• benchmark_name and ts identify the comparison and its time.

Examples

panel <- panel_from_margins(list(group = c(A = 1)), n = 4)
instrument <- panel_instrument(
  item_choice("fund", "Choose an investment.", c("transit", "roads")),
  randomize = character(0)
)
cfg <- LLMR::llm_config("groq", "example-model")
runner <- function(experiments, ...) {
  experiments$response_text <- "transit"
  experiments
}
responses <- panel_administer(
  panel, instrument, cfg, .runner = runner
)
benchmark <- data.frame(
  item_id = "fund",
  response = c("transit", "roads"),
  share = c(.6, .4)
)
panel_benchmark(responses, benchmark, "Example human benchmark")

Summarize execution failures, parse failures, and first-option sensitivity

Description

Counts execution and parse failures by item. For closed items administered with randomized option order, it also applies a chi-squared test to the chosen response and the option shown first. The test does not use the full option permutation.

Usage

panel_bias_audit(responses)

Arguments

Value

A tibble: item_id, n, parse_failures, execution_failures, order_effect_p, and order_test_note. order_effect_p is the first-option chi-squared p-value. When it is NA, order_test_note says why: the item is open, its option order did not vary, fewer than four responses parsed, every parsed response is the same, one option was shown first in every parsed response, the expected cell counts are below five, or the test itself failed, in which case a warning is also issued. order_test_note is NA when a p-value is reported.

Examples

panel <- panel_from_margins(list(group = c(A = 1)), n = 4)
instrument <- panel_instrument(
  item_choice("pick", "Choose one.", c("A", "B")),
  randomize = character(0))
config <- LLMR::llm_config("groq", "example-model")
runner <- function(experiments, ...) {
  experiments$response_text <- "A"
  experiments$success <- TRUE
  experiments
}
responses <- panel_administer(panel, instrument, config, .runner = runner)
panel_bias_audit(responses)

Draw a persona panel from microdata rows

Description

Samples rows from a data frame with replacement, which preserves the joint distribution of the selected attributes, and renders each sampled row as a persona. This is the joint-distribution counterpart of panel_from_margins(), which samples attributes independently. The margins the report cites are computed from the source data, one prop.table(table()) per selected column over its values as the personas state them, so a missing-data label is not a category.

Usage

panel_from_data(
  data,
  n,
  persona_template = NULL,
  columns = NULL,
  weights = NULL,
  na_strings = NULL
)

Arguments

Details

Without a template, a plain data frame renders as "attribute: value" pairs, and a persona_frame (see as_persona_frame()) renders as background and stated answers, as panel_from_personas() does. Values are written as text by the rules under "Rendering of values" in panel_from_personas().

For a reproducible panel, set a seed before calling (the function never sets one itself).

Value

A silicon_panel: a tibble with persona_id, the selected attribute columns, and persona. Its source_rows attribute gives, for each persona in persona_id order, the row of data it was drawn from.

Examples

set.seed(110)
src <- data.frame(
  education = c("college", "college", "no college", "no college"),
  income    = c("high", "high", "low", "low"),
  weight    = c(2, 2, 1, 1))
panel_from_data(src, n = 10, columns = c("education", "income"),
                weights = "weight",
                persona_template = "A {education} respondent earning {income}.")

Draw a persona panel from population margins

Description

Samples n personas with attributes drawn independently from the supplied margins, and renders each persona's text from a template. Attributes are sampled independently, so the result does not preserve their joint distribution. Use panel_from_data() to sample complete microdata rows.

Usage

panel_from_margins(margins, n, persona_template = NULL)

Arguments

Details

For a reproducible panel, set a seed before calling (the function never sets one itself).

Value

A silicon_panel: a tibble with persona_id, one column per attribute, and persona (the rendered text).

Examples

set.seed(110)
panel <- panel_from_margins(
  list(cohort = c(young = .3, middle = .45, older = .25),
       party  = c(left = .45, right = .45, independent = .10)),
  n = 50,
  persona_template = "A {cohort} voter who leans {party}."
)
panel

Draw a panel from a persona data frame

Description

Turns rows of a persona data frame (one respondent per row, demographics plus survey or attitude answers) into a silicon_panel whose personas can be administered survey items. It is built for frames following the LLMR persona contract, such as the bundled anes_2024_personas: the demographic fields are those that LLMR::llm_persona_demographic_fields() names, every other column is a stated answer, answers are shown under their question wording when the frame carries a dictionary or questions gives one, and each persona is rendered as a person to answer in character ("You are this person: ... On a questionnaire you gave these answers: ...").

Usage

panel_from_personas(
  data,
  n = NULL,
  rows = NULL,
  weights = NULL,
  questions = NULL,
  na_strings = NULL
)

Arguments

Details

Unlike panel_from_margins() and panel_from_data(), this constructor keeps each selected respondent's answers together. The margins attribute contains the demographic distribution of the selected rows, and the source_rows attribute gives, for each persona in persona_id order, the row of data it was drawn from. A persona can therefore be traced to its source record with attr(panel, "source_rows")[panel$persona_id], which still holds after the panel's rows are subset.

For a reproducible draw, set a seed before calling (the function never sets one itself).

Value

A silicon_panel: a tibble with persona_id, the demographic columns as the persona text gives them (NA where a value is left out), and persona, with the attributes margins, source, and source_rows.

Rendering of values

Each value of a rendered field is written as text by these rules.

• A labeled column, such as haven::read_dta() and haven::read_sav() return, is decoded from its labels attribute; haven itself is not needed. A labeled code is shown as its label without the code prefix ("1. Male" becomes "Male"). A code without a label on a scale labeled only at its ends is shown with the scale, as in "5 on a scale from 1 (Fewer services) to 7 (More services)"; on a scale centered on zero the ends are the labeled codes -m and m, even when its midpoint is labeled too. A negative code without a label is left out as missing, with a warning. Any other code without a label is shown as the code, with a warning, unless the column labels at most one nonnegative code or only top and bottom codes ("80. Age 80 or older"), in which case the column holds numbers and the value is shown as one. Values that an SPSS file declares missing (na_values, na_range) are left out.

• A factor's levels are its labels. Levels as haven::as_factor() writes them ("1. Fewer services", "[1] 1. Fewer services", and bare numbers for unlabeled codes) are decoded by the rules above; a level without a code prefix ("Refused" from a file whose labels carry none) counts as a label whose code is unknown. A character column that mixes coded labels with bare numbers, as as.character(haven::as_factor()) writes it, is decoded the same way. Code prefixes are removed from factor levels and from character values.

• NA, empty values, and missing-data labels are left out. A value or a value label is one when it matches na_strings and either carries a missing-data code (negative, or 90 and above: "-9 Refused", "998. Don't know") or says nothing but the matched wording once codes, interviewer markers, and asides in parentheses are set aside ("Refused to answer", "Don't know (VOL)"), as FocusGroup decides, so a free-text answer such as "I refused to answer the door" is kept. A value with a negative code prefix that does not match na_strings is shown, with a warning, since many releases use negative codes for missing data; the labels of negative codes on a scale centered on zero ("-3. Far left" beside "3. Far right") draw none.

• Numbers are written in full: 100000, not 1e+05.

• A label exactly 80 characters long draws a warning. Stata files cut value labels at 80 characters, and the ANES 2024 release has labels cut there.

• Identifier columns (named, ignoring case, id, caseid, case_id, respondent_id, respid, resp_id, persona_id, uid, record_id, or row_id, or with a variable label that calls them a case or respondent ID, as the ANES label of V240001 does), the column named by weights, a column named persona, and ideology_score are never rendered.

The values left out as missing are reported in one message per call, naming each field and the number of personas whose text omits it. A stated answer shown under its column name, for want of a question wording, draws a warning.

See Also

anes_2024_personas, as_persona_frame(), panel_administer().

Examples

set.seed(110)
panel <- panel_from_personas(anes_2024_personas, n = 8)
panel

# A labeled column, as haven::read_dta() returns it, is decoded from its
# labels: the unlabeled 5 is placed on its scale and -9 is left out.
survey <- data.frame(age = c("18-29", "65 or older", "30-49"))
survey$services <- structure(c(1, 5, -9), labels = c(
  "1. Government should provide many fewer services" = 1,
  "7. Government should provide many more services" = 7,
  "-9. Refused" = -9))
labeled_panel <- panel_from_personas(
  survey, questions = c(services = "Government services and spending"))
labeled_panel$persona

Shared generic methods

Description

LLMRpanel provides LLMR::diagnostics(), LLMR::report(), and plot() methods for panel_responses objects. It provides tibble::as_tibble() methods for panel_responses and silicon_panel objects.

Value

diagnostics() returns the panel_bias_audit() tibble, one row per item, with four columns that repeat on every row: benchmark_state ("NOT BENCHMARKED", "PARTIALLY BENCHMARKED", or "BENCHMARKED"), items_covered and items_total, the numbers of closed items covered by the benchmark and administered, and mean_abs_dev, the benchmark's mean absolute deviation, or NA when there is no benchmark.

report() returns a panel_report object: a character vector that prints one line per element. The lines give the benchmark state, the panel's source and demographic fields, the response and failure counts, and each item's first-option sensitivity.

as_tibble() returns a tibble. For panel_responses it is the data field, one row per persona-item pair. For a silicon_panel it is the persona table without the silicon_panel class and without the margins, source, and source_rows attributes.

Assemble an instrument

Description

Combines survey items and records the option-order randomization used during administration.

Usage

panel_instrument(items, randomize = "option_order")

Arguments

Details

Each persona-item pair is administered as an independent request. The model does not receive earlier items or responses. "item_order" is therefore refused.

Administration records the displayed options in option_order. item_position is the item's fixed, 1-based position in the instrument. Likert scores use the supplied scale order, including when the displayed scale is reversed.

Value

A panel_instrument object containing items and randomize.

Examples

panel_instrument(list(
  item_likert("trust", "The city council acts in the public interest."),
  item_open("reason", "What would improve the city council's work?")
))

panel_instrument(
  item_choice("fund", "Choose an investment.", c("transit", "roads")),
  randomize = character(0)
)

Survey items

Description

Three item types cover most quantitative instruments: a Likert item (an agree-disagree battery row), a forced choice, and an open item (free text, returned verbatim). Likert responses also get a numeric score (position on the scale as given, 1-based).

Usage

item_likert(
  id,
  text,
  scale = c("strongly disagree", "disagree", "neutral", "agree", "strongly agree")
)

item_choice(id, text, options)

item_open(id, text)

Arguments

Details

Replies are matched to the labels after trimming surrounding spaces and, failing an exact match, without regard to case. Labels must therefore differ by more than case or surrounding spaces; labels that do not, and missing or empty labels, are refused when the item is built.

Value

An object of class panel_item.

Examples

item_likert("wk4", "A four-day work week would benefit society.")
item_choice("vote", "Which proposal do you prefer?", c("A", "B"))
item_open("why", "What would improve the city council's work?")

Usage diagnostics for an administered panel

Description

Summarizes token and outcome diagnostics recorded by panel_administer() or panel_batch_fetch(). The diagnostics are stored in the usage field of a panel_responses object and summarized by LLMR::llm_usage(). Model and provider remain in the returned frame. A supplied price_table adds a cost column. The package contains no price table. When the runner records per-call duration, its sum is returned as duration_s.

Usage

panel_usage(responses, price_table = NULL)

Arguments

Value

A one-row usage tibble, or a typed empty tibble when the runner returned no recorded usage fields.

See Also

panel_administer(), LLMR::llm_usage().

Examples

panel <- panel_from_margins(list(group = c(A = 1)), n = 2)
instrument <- panel_instrument(
  item_choice("pick", "Choose one.", c("A", "B")),
  randomize = character(0))
config <- LLMR::llm_config("groq", "example-model")
runner <- function(experiments, ...) {
  experiments$response_text <- "A"
  experiments$sent_tokens <- 4L
  experiments$rec_tokens <- 1L
  experiments$total_tokens <- 5L
  experiments$success <- TRUE
  experiments
}
responses <- panel_administer(panel, instrument, config, .runner = runner)
panel_usage(responses)

Plot a benchmark comparison

Description

Plots the comparison recorded by panel_benchmark(). Each covered response level has one point for the panel share and one for the benchmark share, joined by a segment. Response levels follow the instrument's option order, and items appear in separate panels. The method requires a benchmark record.

Usage

## S3 method for class 'panel_responses'
plot(x, ...)

Arguments

Value

A ggplot object.

Examples

if (requireNamespace("ggplot2", quietly = TRUE)) {
  set.seed(110)
  panel <- panel_from_margins(list(party = c(left = .5, right = .5)),
                              n = 12,
                              persona_template = "A voter who leans {party}.")
  instrument <- panel_instrument(
    item_choice("plan", "Which plan do you prefer?", c("Plan A", "Plan B")))
  cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b")
  by_party <- function(experiments, ...) {
    experiments$response_text <- ifelse(
      grepl("leans left", vapply(experiments$messages, `[[`, "", "system")),
      "Plan A", "Plan B")
    experiments
  }
  r <- panel_administer(panel, instrument, cfg, .runner = by_party)
  bench <- data.frame(item_id = "plan",
                      response = c("Plan A", "Plan B"),
                      share = c(.55, .45))
  plot(panel_benchmark(r, bench, "city survey 2025"))
}

Launch the LLMRpanel Shiny GUI

Description

Starts a Shiny application that builds a persona panel, administers a choice item or conjoint instrument, and presents the package's diagnostics and design analyses. Choice-item response shares can be compared with an optional benchmark. The application can download the responses and report in a zip file, with the benchmark table when one is available.

Usage

run_panel_studio(...)

Arguments

Details

The GUI is optional. It needs the suggested packages shiny, bslib, DT, and LLMR.shiny; install them first. Keys are read from environment variables only, never pasted into the app; a deterministic demo mode runs offline.

Value

Invisibly, the value of shiny::runApp(); called for the side effect of starting the app.

Examples

if (interactive() &&
    requireNamespace("shiny", quietly = TRUE) &&
    requireNamespace("LLMR.shiny", quietly = TRUE)) {
  run_panel_studio()
}