When Stata and R Disagree: Expressions and Missing Values

library(statacompat)

An expression such as x > 5 looks the same in Stata and in R. When x is missing, the two languages give different answers. This vignette goes through the cases that matter when Stata data-management code is translated into R, and shows the statacompat helper that reproduces Stata’s result in each case.

Every rule is taken from Stata’s manuals; the relevant section is given in brackets. Where the manuals leave a case open, the text says so.

1. In Stata, missing is a very large number

Stata orders numeric values as all numbers < . < .a < .b < ... < .z ([U] 12.2.1). Relational operators always return 0 or 1 and never missing ([U] 13.2.3). In R, any comparison with NA returns NA.

x <- c(0, 1, 2, NA)

x > 1               # R
#> [1] FALSE FALSE  TRUE    NA
stata_gt(x, 1)      # Stata: x > 1
#> [1] FALSE FALSE  TRUE  TRUE

The table summarises the differences. Every row follows from the documented ordering.

Stata Stata result R NA equivalent statacompat
. == 1 0 NA stata_eq(NA, 1) → FALSE
. != 1 1 NA stata_ne(NA, 1) → TRUE
. > 1 1 NA stata_gt(NA, 1) → TRUE
1 < . 1 NA stata_lt(1, NA) → TRUE
. == . 1 NA stata_eq(NA, NA) → TRUE
. != . 0 NA stata_ne(NA, NA) → FALSE
. >= . 1 NA stata_ge(NA, NA) → TRUE

R’s three-valued logic then spreads the problem. NA | FALSE is NA, so a combined condition built from base R comparisons can become NA even when Stata’s version is a definite 0 or 1. The stata_*() helpers never return NA, so combining them with &, | and ! is safe.

The & x < . idiom

Careful Stata code excludes missing values explicitly:

gen high = income > 10000000 & income < .

The translation is literal: x < . becomes stata_lt(x, NA).

income <- c(5e6, 2.5e7, NA)
stata_gt(income, 1e7) & stata_lt(income, NA)
#> [1] FALSE  TRUE FALSE

!missing(x) and stata_lt(x, NA) are equivalent for ordinary missing values.

2. inlist()

inlist(z, a, b, ...) asks whether z equals one of the other arguments, using the same equality as ==. A missing value is therefore an ordinary member. The manual does not discuss missing values in inlist() explicitly; this rule follows from . == . being true ([U] 13.2.3).

kbli <- c(1, 4, NA)
stata_inlist(kbli, 1, 2, 3)
#> [1]  TRUE FALSE FALSE
stata_inlist(kbli, 1, NA)     # Stata: inlist(kbli, 1, .)
#> [1]  TRUE FALSE  TRUE

Stata evaluates expressions one observation at a time. When the candidates are variables, row i of x is compared with row i of each candidate. R’s %in% behaves differently: it pools every candidate value into a single set.

x <- c(0, 1, 2, NA)
a <- c(2, 0, 0, 0)
b <- c(0, 0, 1, 0)
stata_inlist(x, a, b)     # Stata
#> [1]  TRUE FALSE FALSE FALSE
x %in% c(a, b)            # pooled set: not what Stata computes
#> [1]  TRUE  TRUE  TRUE FALSE

A vector of codes is therefore not a set. To test membership in one, splice it:

codes <- c(1, 2, 3)
do.call(stata_inlist, c(list(kbli), as.list(codes)))
#> [1]  TRUE FALSE FALSE

3. inrange()

inrange(z, a, b) is “1 if it is known that a ≤ z ≤ b” ([FN]). Stata applies these rules in order:

  1. z missing → 0
  2. a and b missing → 1
  3. a missing → z <= b
  4. b missing → a <= z
  5. otherwise → a <= z & z <= b

A missing bound means unbounded. A missing z is never in range.

age <- c(12, 15, 64, 65, NA)
stata_inrange(age, 15, 64)
#> [1] FALSE  TRUE  TRUE FALSE FALSE
stata_inrange(age, 15, NA)      # 15 and older; missing age excluded
#> [1] FALSE  TRUE  TRUE  TRUE FALSE
stata_ge(age, 15)               # Stata's age >= 15 includes missing age
#> [1] FALSE  TRUE  TRUE  TRUE  TRUE

The common R translation !is.na(x) & x >= lower & x <= upper is correct when both bounds are known. When a bound is missing it returns NA, where Stata returns a definite 0 or 1.

4. cond()

Stata reads a numeric condition as false if it is zero and true otherwise, and a missing value counts as “otherwise”. So cond(., a, b) is a. An optional fourth argument gives the value to use when the condition is missing ([FN]).

wage <- c(0, 1500, NA)
ifelse(wage, "paid", "unpaid")                 # R: NA for missing
#> [1] "unpaid" "paid"   NA
stata_cond(wage, "paid", "unpaid")             # Stata: missing is true
#> [1] "unpaid" "paid"   "paid"
stata_cond(wage, "paid", "unpaid", "unknown")  # cond(wage, ..., ..., "unknown")
#> [1] "unpaid"  "paid"    "unknown"

A relational expression is never missing, so Stata never reaches the fourth argument when the condition is a comparison. The same holds when the condition is built with stata_*() predicates:

x <- c(1, 3, NA)
stata_cond(stata_gt(x, 2), 50, 70, 99)   # Stata: cond(x > 2, 50, 70, 99)
#> [1] 70 50 50

A logical NA condition is an error. In R such a value almost always comes from a base R comparison, and there is no safe way to guess what Stata would have done with it. Stata’s cond(x < 2, a, b) gives b for a missing x, because . < 2 is false. Treating R’s NA as Stata’s . would give a instead.

stata_cond(x < 2, "a", "b")
#> Error:
#> ! `stata_cond()`: `condition` is a logical vector containing NA. Stata conditions are never logical NA: build the condition with stata_eq(), stata_gt(), etc., which never return NA, or use as.numeric(condition) to apply Stata's rule that a missing number counts as true.
stata_cond(stata_lt(x, 2), "a", "b")
#> [1] "a" "b" "b"

Unlike ifelse(), stata_cond() keeps the class of a Date result and does not silently mix numbers with strings.

5. missing() and strings

Stata’s missing string is "". A string of spaces is not missing. Strings that come from Stata data never contain R’s NA_character_. For that reason, string comparisons reject it, and stata_missing() counts it as missing.

stata_missing(c("a", "", " "))
#> [1] FALSE  TRUE FALSE
stata_missing(c(30, NA, 40), c("Ari", "Prasojo", ""))   # missing(age, name)
#> [1] FALSE  TRUE  TRUE

Stata compares strings by their bytes ([U] 12.4.2), so every uppercase letter sorts before every lowercase letter. R’s < and > instead follow the collation rules of the session locale, so the same script can give different answers on different computers. The stata_*() helpers always use byte order, so "a" > "B" is 1, as in Stata.

stata_gt(c("apel", "Bayam"), "B")
#> [1] TRUE TRUE

6. egen row functions

r1 <- c(1, 1, NA)
r2 <- c(2, NA, NA)
r3 <- c(3, 3, NA)

rowSums(cbind(r1, r2, r3), na.rm = TRUE)
#> [1] 6 4 0
stata_rowtotal(r1, r2, r3)                  # egen rowtotal(): all-missing -> 0
#> [1] 6 4 0
stata_rowtotal(r1, r2, r3, missing = TRUE)  # , missing: all-missing -> .
#> [1]  6  4 NA

rowMeans(cbind(r1, r2, r3), na.rm = TRUE)   # NaN
#> [1]   2   2 NaN
stata_rowmean(r1, r2, r3)                   # .
#> [1]  2  2 NA

stata_rowmiss(r1, r2, r3)
#> [1] 0 1 3
stata_rownonmiss(r1, r2, r3)
#> [1] 3 2 0

stata_rowmiss() accepts string variables and counts "" as missing. stata_rownonmiss() accepts them only with strok, as [D] egen requires. The manual states that restriction only for rownonmiss(); that rowmiss() accepts strings is inferred from this.

stata_rowmiss(c(1, 1, NA), c("a", "", ""))
#> [1] 0 1 2
stata_rownonmiss(c(1, 1, NA), c("a", "", ""), strok = TRUE)
#> [1] 2 1 0

Inside mutate(), select the variables with pick():

dta <- data.frame(r1 = r1, r2 = r2, r3 = r3)
dplyr::mutate(dta, total = stata_rowtotal(dplyr::pick(r1:r3)))
#>   r1 r2 r3 total
#> 1  1  2  3     6
#> 2  1 NA  3     4
#> 3 NA NA NA     0

Precision

Stata’s default storage type for new variables is float (see set type), and egen without an explicit type is assumed to follow it. A float holds integers exactly only up to 16,777,216, so a total of 16,777,217 cannot be stored and becomes 16,777,216. With egen double t = rowtotal(a b) the exact value is kept. R computes in double precision, so stata_rowtotal() corresponds to the double version:

stata_rowtotal(16777217, 0)
#> [1] 16777217

Large monetary totals, such as household expenditure in Rupiah, can therefore differ by a few units from Stata output that was created without double. statacompat does not emulate float storage.

7. Extended missing values

Stata has 27 numeric missing values, . and .a to .z, and treats them as distinct and ordered ([U] 12.2.1), so .a == . is 0 and .a > . is 1. haven::read_dta() keeps the distinction as “tagged” NAs, but R itself cannot see it. To R, .a and .b are both NA, and identical() cannot tell them apart.

statacompat 1.0 reads the tags. It does not reproduce the ordering among missing values:

a <- haven::tagged_na("a")
stata_gt(a, 100)          # .a > 100: TRUE in Stata
#> [1] TRUE
stata_inrange(5, a, NA)   # missing bound = unbounded
#> [1] TRUE
stata_eq(a, NA)           # .a == . is 0 in Stata; not reproduced
#> Error:
#> ! `stata_eq()`: an extended missing value (.a-.z, from haven::tagged_na()) is compared with a different missing value. Stata treats these as unequal and ordered (. < .a < ... < .z); statacompat 1.0 does not reproduce that ordering. Convert extended missing values to ordinary NA first, e.g. `x[is.na(x)] <- NA_real_`.

To treat every missing value as ., remove the tags first:

v <- c(1, haven::tagged_na("a"), NA)
v[is.na(v)] <- NA_real_
stata_eq(v, NA)
#> [1] FALSE  TRUE  TRUE

8. Migrating from “missing-safe” helpers

Some R projects use helpers such as is_greater <- function(x, value) !is.na(x) & x > value. These are not literal Stata translations. They return FALSE whenever x is missing, which corresponds to the careful Stata idiom x > value & x < ., not to x > value.

Helper Matches Stata expression statacompat equivalent
!is.na(x) & x == v x == v (v not missing) stata_eq(x, v)
!is.na(x) & x != v x != v & x < . stata_ne(x, v) & stata_lt(x, NA)
!is.na(x) & x > v x > v & x < . stata_gt(x, v) & stata_lt(x, NA)
!is.na(x) & x >= v x >= v & x < . stata_ge(x, v) & stata_lt(x, NA)
!is.na(x) & x < v x < v stata_lt(x, v)
!is.na(x) & x <= v x <= v stata_le(x, v)
!is.na(x) & x %in% c(...) inlist(x, ...) only for scalar, non-missing candidates stata_inlist(x, ...)
!is.na(x) & x >= lo & x <= hi inrange(x, lo, hi) only if both bounds are known stata_inrange(x, lo, hi)

Before switching, check which Stata expression the original .do file actually used. If it was gen y = x > 5 without & x < ., the old helper and Stata disagree for missing x, and the replacement will change your results. In that case the change is a correction towards what the original Stata code computed.

9. What is not covered

Arithmetic (1/0 is . in Stata but Inf in R), float storage, Stata’s logical operators applied to missing numbers (5 & . is 1 in Stata), recode, _n and _N, bysort, and egen group() are outside the scope of version 1.0