screenllm quickstart

This vignette walks through the six-call workflow screenllm implements. Everything below runs on a laptop against a locally-served Ollama backend. If you don’t yet have Ollama installed and the four default models pulled, check_setup() will tell you what’s missing.

1. Verify setup

library(screenllm)
check_setup()

2. Load a corpus

Any tibble with title and abstract columns works; read_records() also accepts CSV, XLSX, or RIS paths and normalises common column-name variations (Scopus, Web of Science, EndNote). The package ships with a 40-record toy dataset drawn from the Community-Based Fisheries Management (CBFM) review used in the manuscript, which we use here for demonstration.

library(screenllm)
#> screenllm loaded.
#>   * First time?          install_prereqs(preset = "light")   # installs Ollama + 4 small models
#>   * Ready to go:         launch_app()                         # open the browser workflow
#>   * Diagnose Ollama:     check_setup()
toy_path <- system.file("extdata", "toy_cbfm.csv", package = "screenllm")
records <- read_records(toy_path)
head(records[, c("id", "title")])
#> # A tibble: 6 × 2
#>   id         title                                                              
#>   <chr>      <chr>                                                              
#> 1 record_001 Evaluation of managed access approach in Anambas Islands Marine Re…
#> 2 record_002 Livelihoods and fisheries governance in a contemporary Pacific Isl…
#> 3 record_003 Guidelines for coastal and marine site-planning and examples of pl…
#> 4 record_004 Antimicrobial stewardship program (ASP): An effective implementing…
#> 5 record_005 Ecosystem services approach for community-based ecotourism: toward…
#> 6 record_006 Effects of community-based management on amazon river turtles: A c…

3. Define the inclusion criteria

criteria <- define_criteria(
  scope = "Articles potentially relevant to community-based fisheries management (CBFM) in Pacific Island contexts.",
  inclusions = c(
    "It is possible that the study includes a case study from a Pacific Island country (e.g. Fiji, Solomon Islands, Vanuatu, Papua New Guinea, Samoa, Tonga, or similar).",
    "It is possible that the study discusses fisheries and/or marine resource management.",
    "It is possible that the study discusses a community-based approach."
  )
)
print(criteria)
#> <screenllm_criteria 3 inclusions>
#> Scope: 
#>   Articles potentially relevant to community-based fisheries management
#>   (CBFM) in Pacific Island contexts.
#> 1. It is possible that the study includes a case study from a Pacific Island country (e.g. Fiji, Solomon Islands, Vanuatu, Papua New Guinea, Samoa, Tonga, or similar).
#> 2. It is possible that the study discusses fisheries and/or marine resource management.
#> 3. It is possible that the study discusses a community-based approach.

4. Rank the corpus

Real screening uses default_ensemble(), which talks to Ollama. For this vignette we use backend_mock() so the code runs without Ollama.

mock_ensemble <- custom_ensemble(
  models = c("gemma3:27b", "gpt-oss:20b"),
  replicates = 2,
  backend = backend_mock()
)
ranked <- rank_records(records, criteria, ensemble = mock_ensemble, verbose = FALSE)
head(ranked[, c("id", "title", "universal_best_score", "rank")])
#> # A tibble: 6 × 4
#>   id         title                                    universal_best_score  rank
#>   <chr>      <chr>                                                   <dbl> <int>
#> 1 record_023 The reimplementation of the Ra'ui: Cora…                 73       1
#> 2 record_010 Effectiveness of village-based marine r…                 72       2
#> 3 record_012 Marine Protected Areas in Fiji: A criti…                 71.5     3
#> 4 record_013 Enhancing community empowerment through…                 71.5     4
#> 5 record_004 Antimicrobial stewardship program (ASP)…                 71       5
#> 6 record_028 Fishers' perceptions, facilitating fact…                 71       6

For a real run, swap the mock for the default:

ranked <- rank_records(records, criteria, ensemble = default_ensemble())

5. Plan the human screening set

plan <- plan_screening(ranked)
plan
#> 
#> ── <screenllm_plan> ──
#> 
#> ℹ Stop at record 40 of 40 (expected workload 100.0%).
#> ℹ Settings: min coverage 50%, run length 50, spot check n = 200
#> ! Spot-check gate is placeholder (no labelled spot-check supplied). Once the reviewer labels the first ~200 records, re-run `plan_screening()` with `spot_check_labels` set to sharpen the stop point.

The to_screen element is the tibble of records the reviewer should inspect. Everything below the stopping point is treated as excluded.

6. Screen and report

Interactive screening via the Shiny app:

launch_screening_app(plan, ranked, out_file = "screening_decisions.csv")

Offline screening (spreadsheet round-trip):

export_worksheet(plan, path = "to_screen.xlsx")
# Reviewer fills in the human_decision column and saves as
# 'to_screen_completed.xlsx'.
decisions <- read_decisions("to_screen_completed.xlsx")

Summarise the run and surface any strong LLM-human disagreements as a manual audit queue:

report <- summarise_screening(ranked, decisions, plan = plan)
print(report)

disagreements <- audit_disagreements(ranked, decisions)
disagreements