How many $k$-bit words of a random bitstring are we expected to extract before all $2^k$ possible words occur?

lyrically wicked

6/16/23, 5:28 AM

Let $C(X)$ denote the cardinality of the set $X$. For example, $C(\{0\}) = 1, C(\{0, 2\}) = 2$ etc.

Let $S$ denote a (potentially infinite) sequence of random bits. Split $S$ into $k$-bit words $w_1, w_2, w_3, \ldots$ For example, if $k = 4$ and $S = 0001111010100100\ldots$, then $w_1 = 0001, w_2 = 1110, w_3 = 1010, \ldots$

At each step $i$ (here $i \geq 1$), do the following sub-steps:

Extract $w_i$ and go to sub-step 2;
If $C(X) < 2^k$ and $X$ contains $w_i$, do not do anything and go to step $(i+1)$; if $C(X) < 2^k$ and $X$ does not contain $w_i$, put $w_i$ in $X$ and go to sub-step 3;
If $C(X) < 2^k$, go to step $(i+1)$; if $C(X) = 2^k$, stop.

Question: what is the expected value of $i$ when the above algorithm halts? In other words, if $S$ is the source of truly random or unbiased pseudo-random bits, how many words should we expect to extract to fill $X$ with all possible $k$-bit elements?

0 + 0

statistical-test

probability

Score:4

Crypto

fgrieu

6/16/23, 6:27 AM

This known as the coupon collector's problem, with the number of coupons replaced by $2^k$ and $k$ is the number of independent bits drawn ($k=4$ in the question's example).

It's expected $(k\log(2)+\gamma)\,2^k+\frac12+\mathcal O(2^{-k})$ extractions, where $\gamma\approx0.5772$ is Euler's constant, and $\log(2)\approx0.6931$.

When few experiments are made, a rough approximation of the expectation is often good enough, e.g. $0.7\,k\,2^k$. The distribution has a long tail. I think the standard deviation is $\approx\frac\pi{\sqrt6}\,2^k\approx1.3\times 2^k$.

0 + 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: How many $k$-bit words of a random bitstring are we expected to extract before all $2^k$ possible words occur?

TH: จำนวน $k$-bit word ของ bitstring แบบสุ่มที่เราคาดว่าจะดึงออกมาก่อนที่ $2^k$ คำที่เป็นไปได้ทั้งหมดจะเกิดขึ้น?

RO: Câte $k$-biți cuvinte dintr-un șir de biți aleatoriu trebuie să extragem înainte de apariția tuturor $2^k$ cuvinte posibile?

RU: Сколько $k$-битных слов из случайной строки битов мы должны извлечь, прежде чем появятся все $2^k$ возможных слов?

VI: Có bao nhiêu từ $k$-bit của một chuỗi bit ngẫu nhiên mà chúng tôi dự kiến sẽ trích xuất trước khi tất cả $2^k$ từ có thể xảy ra?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.