Compute Pointwise Mutual Information

MediumNLP

Implement compute_pmi(joint_counts, total_counts_x, total_counts_y, total_samples) returning the PMI of two events from counts, with $N$ = total_samples, $p(x) = c_x/N$ , $p(y) = c_y/N$ and $p(x,y) = c_{xy}/N$ :

$\text{PMI}(x, y) = \log_2 \frac{p(x,y)}{p(x)\,p(y)}$

Round the result to 3 decimal places.
If joint_counts is 0, return float('-inf').

Example:

Input:

compute_pmi(50, 200, 300, 1000)

Output:

-0.263

Reasoning:

The PMI calculation compares the actual joint probability (50/1000 = 0.05) to the product of the individual probabilities (200/1000 * 300/1000 = 0.06). Thus, PMI = log₂(0.05 / (0.2 * 0.3)) ≈ -0.263, indicating the events co-occur slightly less than expected by chance.

Learn About topic

Contributors:

saitiger

Contribute