RenyiExtropy

RenyiExtropy provides a comprehensive collection of entropy and extropy measures for discrete probability distributions. All measures use the natural logarithm (results in nats).

Functions

Function	Description
`shannon_entropy()`	Shannon entropy H(P)
`renyi_entropy()`	Rényi entropy H_q(P) of order q
`tsallis_entropy()`	Tsallis entropy S_q(P) of order q
`normalized_entropy()`	Shannon entropy normalised to [0, 1]
`extropy()` / `shannon_extropy()`	Classical (Shannon) extropy J(P)
`renyi_extropy()`	Rényi extropy J_q(P) of order q
`max_renyi_extropy()`	Maximum Rényi extropy for n outcomes
`conditional_renyi_extropy()`	Conditional Rényi extropy J_q(Y\|X)
`joint_entropy()`	Joint entropy H(X, Y)
`conditional_entropy()`	Conditional entropy H(Y\|X)
`kl_divergence()`	Kullback–Leibler divergence D_KL(P ‖ Q)
`js_divergence()`	Jensen–Shannon divergence
`cross_entropy()`	Cross-entropy H(P, Q)

Installation

Install the released version from CRAN:

install.packages("RenyiExtropy")

Or install the development version from GitHub:

# install.packages("remotes")
remotes::install_github("itsmdivakaran/RenyiExtropy")

Quick start

library(RenyiExtropy)

p <- c(0.2, 0.5, 0.3)

# Entropy measures
shannon_entropy(p)        # 1.030 nats
renyi_entropy(p, q = 2)   # 0.968 nats
tsallis_entropy(p, q = 2) # 0.620
normalized_entropy(p)     # 0.937

# Extropy measures
extropy(p)                # 0.775 nats
renyi_extropy(p, q = 2)   # 0.742 nats
max_renyi_extropy(3)      # maximum over uniform 3-outcome distribution

# Divergences
kl_divergence(p, c(0.3, 0.4, 0.3))   # KL(P || Q)
js_divergence(p, c(0.3, 0.4, 0.3))   # symmetric, bounded by log(2)
cross_entropy(p, c(0.3, 0.4, 0.3))   # H(P) + KL(P || Q)

# Joint / conditional measures
Pxy <- matrix(c(0.2, 0.3, 0.1, 0.4), nrow = 2, byrow = TRUE)
joint_entropy(Pxy)                     # H(X, Y)
conditional_entropy(Pxy)               # H(Y | X)
conditional_renyi_extropy(Pxy, q = 2)  # J_q(Y | X)

Limit / continuity behaviour

Functions with order parameter q automatically return their q → 1 limit (Shannon entropy or classical extropy) when |q − 1| < 1e-8:

renyi_entropy(p, 1)        # equals shannon_entropy(p)
renyi_extropy(p, 1)        # equals extropy(p)
tsallis_entropy(p, 1)      # equals shannon_entropy(p)

Citation

If you use RenyiExtropy in your research, please cite: