Score:27

Can an AI really generate random numbers?

ru flag

I asked an AI the following question:

Can you provide me with random numbers of 30 digits in length?

And then the AI has generated these numbers for me.

563958422461839604397274590248
743298571529845197630149526734
962345019834590239458293827563
841259630492576302945836184025
129458392650293745092837563945
938475029835820943759284396284
284657492870476502834795836192
937453090283964506295830295830
293857692057983473950296395830
912465732491850173082913290227
107609347826170463295771141645
788604572134668302261094130853
302239005287747759752543651912
819043729526479834375692849203
503284901875216436897123740615
641375927104689705914038297506
474327512378904132524817120687
968370218489946036482905748215

Now I wonder:

  • From the point of view of information theory or cryptography, are these numbers random?
  • From an algorithmic point of view, how are these numbers predictable?

The AI in question is LLaMA from https://github.com/ggerganov/llama.cpp

Stef avatar
ng flag
When you say "I asked an AI", do you mean "I asked ChatGPT"?
DannyNiu avatar
vu flag
For those coming from HNQ unfamiliar with randomness in cryptographic context: the random numbers must be generated in such way that no algorithm can predict the next digit, even given VERY long stream of what's already generated; also even when the seed is compromised, past random numbers cannot be recovered or deduced. These 2 requirements are roughly known as: prediction-resistance and back-tracing-resistance.
swannty avatar
ru flag
@Stef, I use LLaMA from https://github.com/ggerganov/llama.cpp
swannty avatar
ru flag
@bmm6o, You're right, the question is more about "how the AI is generating these numbers" instead of "is the AI able to do it".
Maarten Bodewes avatar
in flag
That's not apparent from the title, and since you've not even indicated the AI in the question (initially) the answerers are not required to take above statement into account. How numbers are generated can usually not be concluded from looking at a few numbers.
swannty avatar
ru flag
@DannyNiu, interesting... so is there any way to predict the answer of an AI?
DannyNiu avatar
vu flag
@swannty Yes, answer by Joseph mentioned observing repeated pattern. That'll b a fatal wound on any PRNG in the cryptography context.
Philipp avatar
cn flag
@swannty There are lots of different methods to create what is commonly referred to as "AI". But most of them are just combinations of lots of different algorithms which are in the end deterministic. But while it is possible *in theory* to predict the decisions of an AI, it's often difficult in practice. It is often not easy to tell how "an AI" actually works, because the algorithms are often created through machine learning. Analyzing such AIs and designing AIs that are powerful but still built in a way that their decisions can be comprehended is a whole sub-discipline of computer science.
jp flag
@swannty yes, you can predict the answer of an AI by running the same AI again. Note that the GPT model outputs a score for each digit, and then the system picks one at random based on those scores. So you can see if the high scores are predicted more often, but you can't predict which one the random part will pick.
Brady Gilg avatar
sl flag
'AI' is a marketing term. There is nothing that can computationally distinguish them from any other computer program.
zaph avatar
pl flag
I find it peculiar that the 8th and 9th rows in the question end with the same five digits: "95830". Of course this could happen in a random number but my guess is not in this case.
isaacg avatar
de flag
It's much easier to get a program like chatGPT to output a program which, when run, will output the desired random numbers. I asked the query "Please write a Python program to generate 18 random 30-digit integers." and received the following program, which works as desired: [Try it online!](https://tio.run/##K6gsycjPM/7/PzO3IL@oRKEoMS8lP5eLKy2/SCFTITMPJJCeqmFooWnFpQAEeaW5CrYKSkp6WfmZeRrRxSVFGhAteiAqM69Ew0BHwVJTUwFkQDzCAGMDzVhNsAkFRSBVQHM0//8HAA "Python 3 – Try It Online")
Robbie Goodwin avatar
rw flag
Wasn't which AI or any particular AI merely a slip? Isn't the real Question whether any AI could generate random numbers?
Robbie Goodwin avatar
rw flag
I suggest the real Question underlying this one is whether an AI could provide a wholly different approach…
Score:49
ne flag

First of all, you have only 540 digits, and 540 digits seems a bit small. I would have asked the AI to generate as many digits as it can. Nevertheless, I was able to find the following sources of non-randomness here by casual observation followed by some statistical analysis.

The string 295830 repeats twice on the end of the 8th row, and we see the related string 395830 on the end of the 9th row. The string 30294583 appears in one place while the similar string 3029583 appears in another place.

There are only 13 instances out of 539 where a digit repeats. There should be about 54 instances of a repeating digit. It seems like the AI avoids repeating digits because they may appear 'non-random' or because the AI learned incorrectly from humans that random strings do not repeat characters very much.

We can actually calculate the probability of having at most 13 repeated digits out of 539 instances if the string were truly random since this distribution is a binomial distribution. Here, the probability of getting $k$ instances out of 539 instances of a repeated digit is $\binom{539}{k}\cdot 0.1^{k}\cdot 0.9^{539-k}$. Therefore, the probability of getting at most 13 out of 539 repeated digits is $\sum_{k=0}^{13}\binom{539}{k}\cdot 0.1^{k}\cdot 0.9^{539-k}\approx 4.90903\cdot 10^{-12}$. We can therefore safely conclude that these digits are not random/pseudorandom.

My word embedding algorithm detecting non-randomness

So I fed these 540 digits into my own complex matrix-valued word embedding algorithm with dimension $d=30$. I can tell that the word embedding converges to a good local maximum for two reasons (the word embedding does not always converge to a good local maximum):

  1. I trained the word embedding twice, and I got the same fitness level both times (with $1.31\cdot 10^{-14}$ difference in fitness).

  2. Even though the matrices were complex during the initialization and training, after the word embedding was fully trained, I checked that the matrices were real up-to-symmetry but computing the traces of their products. If I tried training using real matrices though the word embedding may converge to a suboptimal level of fitness (but it did not in this case which is good), so I needed to use complex matrices during training; the fitness function has a lot of near singularities and complex matrices are good at avoiding these near singularities while real matrices are not.

  3. I trained the word embedding using quaternionic and real matrices and the trained word embedding was the same thing (up-to-a quaternionic unitary matrix or real orthogonal equivalence) as I got when training with the complex matrices.

After training, I got a fitness of about -1.1372531160707808. A smaller fitness value after training means that the sequence is more random while a larger fitness value means that the sequence is less random. I have been searching for a random sequence that has a higher fitness than the fitness for the AI generated sequence, and I have not been able to find such a random sequence.

I call my word embedding algorithm an MPO word embedding which stands for matrix product optimized. Suppose that $A$ is a set. In our case, $A$ will be the set of digits from 0 to 9, but in natural language processing, $A$ will be the set of tokens. Let $a_1\dots a_n\in A^*$ be a string.

Suppose now that $f:A\rightarrow M_d(\mathbb{C})$ is a function. Then define the fitness $$L(a_1\dots a_n,f)=\log(\rho(f(a_1)\dots f(a_n))^{1/n})-\log(\|\sum_{a\in A}f(a)f(a)^*\|_p)/2.$$ Here, $\|X\|_p$ denotes the Schatten $p$-norm of a matrix $X$ while $\rho(X)$ denotes its spectral radius. We maximize the fitness using gradient ascent. In our case, $p=2$, so $\|X\|_p$ is just the Frobenius norm. One reason why we get the same fitness after training multiple times is that the dominant eigenvalue of $\sum_{a\in A}f(a)f(a)^*$ is substantially larger than the other eigenvalues which means that there is a rank-1 positive semidefinite matrix that somewhat approximates each $f(a)$. In other words, we are making a tradeoff between being able to use all $d=30$ complex dimensions equally and always converging to the same local optimum. The matrices $f(a)$ were also approximately low rank matrices. If the matrices $f(a)$ were all rank-1 matrices, then the MPO word embedding would reduce to a weighted digraph embedding where the weight of $(u,v)$ is the number of times the string $uv$ occurs in the cycle of 540 digits.

Here is a more detailed description of my word-embedding algorithm, and here is a link for code on my Github account.

Maarten Bodewes avatar
in flag
Well spotted. The AI may be also be a chatbot that tries to generate random numbers from previously seen values. That's generally the way they work after all.
Nordine Lotfi avatar
cn flag
Do you have it on your github? I looked for it, but didn't find it. Just curious.
Joseph Van Name avatar
ne flag
I gave a link on my Github account. But I think there is a slight bug in the code that I still need to clean up.
Aron avatar
in flag
`return 7` chosen from fair dice roll.
Fabio says Reinstate Monica avatar
cn flag
@Aron Surely you mean [return 4](https://xkcd.com/221/)?
Cruncher avatar
ru flag
So... amazingly it generates random numbers about as poorly as a human. Or maybe a little better
JimmyJames avatar
cn flag
"It seems like the AI is afraid of repeating digits". LLMs don't experience fear. They don't understand the questions they are asked or know if their answers are correct. There's enough nonsense and hype going around about these tools. Anthropomorphizing LLMs will not yield a better understanding of how they work.
Peter Cordes avatar
us flag
@JimmyJames: So more accurately we might say the LLM has learned that humans think "random" numbers are biased away from repeating digits. "Afraid" can work as short-hand for that idea, but fair point that it could be misleading for readers who don't understand the intended meaning.
JimmyJames avatar
cn flag
@PeterCordes I think it's illuminating to see how well GPT3 answers questions about simple addition. [100% accuracy for 2-digit numbers and less than 10% accuracy for 5-digit numbers](https://lambdalabs.com/blog/demystifying-gpt-3). At a really high level, these tools create answers that 'look' statistically like what it is trained with.
jp flag
@JimmyJames Humans do the same thing - avoid repeated numbers. Are humans "afraid" of repeating digits? I would say yes, although it is hardly as strong as, say, the fear of being eaten by a lion when you are hanging out in the lion pen.
JimmyJames avatar
cn flag
@user253751 I find it unsurprising that a model trained on human interactions and human written text and designed to mimic humans would produce results similar to what a human would create. The logical leap to the belief that the model is 'thinking' like a human is unfounded and well-documented as false.
Joseph Van Name avatar
ne flag
@JimmyJames. You are correct. I removed the anthropomorphism of LLMs since people on this site may not be very famility with AI/ML, and LLMs have received a lot of recent hype.
JimmyJames avatar
cn flag
@JosephVanName Thanks. This is my biggest worry about LLMs: that people will put too much faith into them. Some of the fabrications I've seen are pretty scary.
Score:11
fo flag

Answering the title - yes, it can.

A neural network can generate random output if it contains a layer introducing randomness. For example - when I use the TensorFlow framework I can consider tf.random.uniform function and put it inside the model. It will cause that "random" numbers will appear inside and the output will contain randomness, as well.

DannyNiu avatar
vu flag
This answer is okay, but not satisfactory in the context of cryptography. See my comment under OP.
Michael Mior avatar
lk flag
@DannyNiu I don't see why an AI couldn't have access to source of true randomness that is sufficient for cryptographic use. Whether it can actually use it appropriately is a different question.
BeB00 avatar
bd flag
I'm not sure how reasonable this is an as answer. This seems more like an answer for "if I put random numbers into a neural network, can they still be random when it comes out", since I wouldn't really count tf.random as being "part" of the neural network. The detail of "part" is pedantic, but its important because I think the much more interesting question is "can a neural network generate 'random' numbers without a random source" and the answer is probably no
Score:9
in flag

That will depend on the AI:

  • Currently it is likely that chatbots piece the random values together from previously observed values, which of course would make the result mostly non-random.
  • An AI may also request the random values from a separate API or service; AI's are using external services at a very high pace.
  • I'd say that it is currently unlikely that it would try and generate the random values itself (i.e. by using its neural network, in the same way as humans do). If it does then remember that humans are terrible when it comes to generating random values.

If it is using an API then it is unclear if the random number generator is secure and well seeded. Both are requirements for any cryptographically secure pseudo-random number generator. Nor is it clear if the values are obtained from an external service or if they are stored anywhere. As such, I would definitely not trust any the obtained values to be secure out of the box.

Note that it is impossible for anybody to tell you if these values are secure without more input. It might be possible to spot vulnerabilities, but you'd need to analyze the entropy source / true RNG and the PRNG algorithm to get anywhere "proving" security for these values.

It might be a good idea to ask the AI how it generates random numbers and then to verify the answer. If it doesn't integrate with an API to provide random values then the chances of getting cryptographically secure random numbers is so small you may safely ignore it.

Stef avatar
ng flag
I'm less optimistic than you about *"It will probably request the random values from a separate API or service. I'd say that it is unlikely that it would try and generate the random values itself (i.e. by using its neural network)."* AIs like Siri or Alexa are designed to only act as an interface, and call on external APIs, but a text-generating AI like ChatGPT or LLaMA is designed to produce its own text all on its own, which is why they are so bad at answering requests that are too logical and algorithmic.
jp flag
A GPT model could emit equal weights for digits 0,1,2,...,9 and then the next token selection part of the algorithm (which selects tokens randomly according to the weights given by the model) would select truly random (or truly pseudorandom) digits.
Maarten Bodewes avatar
in flag
Yeah right, well, without any proof that it does so, and the knowledge that ChatGPT is often wrong regarding statements even about itself, I'm not convinced that it can ever generate random values that way. It would be hard for it to prove that it would be secure anyway.
Score:7
pl flag

This question can be answered on a couple of levels.

First, on the level of basic intuitions, these AI systems are trained to imitate human text. We would therefore on a first order approximation expect them to perform similarly to unaided humans when generating pseudo-random numbers. Unaided humans are terrible at generating random numbers.

Second, we can (at least in the case of some AIs) successfully ask the AI itself. When I ask ChatGPT3.5 to "Write Python code that can run some common statistical tests on (this sequence of digits)", it suggests calculating chi-squared goodness of fit tests on the observed digit stream against the null hypothesis of even distribution for single digits and digit pairs. Eyeballing the code, the code seems largely correct (with a minor off-by-one error miscalculating the expected frequency of each pair, which should be totally inconsequential for results). When prompted accordingly, it admits this mistake and suggests a correction which looks good, but errors out. After having it fix that error, the code looks correct and produces the following output

Chi-squared test:
Chi-squared statistic: 15.26
P-value: 0.0841
Chi-squared test (digit pairs):
Chi-squared statistic: 190.71
P-value: 0.0000

So, digit-pairs look highly non-random.

Thirdly, the AI itself roughly speaking pseudo-randomly samples a model of human text generation. What RNG is used here depends on implementation choices. It is perfectly possible (and would not add significant computational work to the generation process of any reasonably general text-generation models) to use cryptographically secure random numbers here, but I would strongly expect that this will not be the case in practice (although maybe watermarking schemes for AI text could be a reason to use a cryptographic source of randomness in the sampling strategy). As such, the text generator only acts as an obfuscation layer over an insecure RNG here. At that, it is obviously an obfuscation layer that introduces statistical nonuniformity instead of removing it.

Fourth, if one asks e.g. ChatGPT3.5 for a way to generate cryptographically secure random numbers in Python, it outputs code using Python's secrets library. This is a standard library designed for generating cryptographic random numbers, and I would expect it to be a reasonable choice in many settings (although I haven't looked at its implementation choices). It is certainly by light-years more reasonable an approach than asking a neural network to do it.

Stef avatar
ng flag
When humans need to include 340 digits in their texts, I assume they're typically not *unaided*. But I'd be more afraid of the AI copying digits from not-random-at-all lists of digits. For instance, there must be more texts containing digits of pi than texts containing any other long sequence of digits; hence I would be afraid that an AI would be biased towards the digits of pi every time it's asked for sequences of digits.
wizzwizz4 avatar
in flag
The "underlying PRNG" is not the numpy PRNG. It's the PRNG used by the "driver" program for the model (see the [temperature setting](https://ai.stackexchange.com/a/32478/125)), passed through the model itself (which will tend to heavily bias the output sequence; that is, after all, its purpose).
Score:6
tl flag

Like any other RNGs based on deterministic algorithms, an AI will not be able to generate real randomness. Whether it meets the required level of randomness depends entirely on how it generates the numbers. But without any randomness as input into the AI, it will definitely be NOT RANDOM. Anyhow, since AI is usually easy to manipulate, it is not a good idea to use those random numbers in any cryptographic context.

AI only appears intelligent because its algorithms are too complex for a human to comprehend. That it is therefore capable of generating a safe random value is a fallacy.

Edit: I think the discussion in the comments is all due to different definitions of AI. Of course there are AI's that use real randomness as input and may make use of it. What I meant is that current approaches such as Machine Learning / Neural Networks are not able to generate randomness but still might make use of it from inputs it gets.

If the RNG is included in the "AI" definition, then AI may generate randomness. If the RNG is exluded from the definition, the AI is not able to generate any randomness, but still may output randmoness based on random inputs it gets. In the second case, the AI would be deterministic and the output would be predictable, when knowing the inputs into the AI. In the first definition case, it would not be deterministic.

Stef avatar
ng flag
*"AI only appears intelligent because its algorithms are too complex for a human to comprehend. "* Actually, the algorithms behind GPT2 are surprisingly simple. It's kind of a surprise that the output is as good as it is.
p_1092131280 avatar
tl flag
That's correct. Actually, I meant less the complexity of the algorithms (which are usually written by humans anyway) and more the complexity of understanding how a trained neural network maps the inputs to the outputs. Keyword "Explainable AI".
Michael Mior avatar
lk flag
There's no reason an AI couldn't have access to a source of entropy used to generate true random numbers.
jp flag
@Stef only if you do not consider all the essential data as part of the algorithm. It is like claiming that Microsoft Windows is simple because you can understand how a CPU executes opcodes.
jp flag
GPT is not based on deterministic algorithms. There is an essential part which uses randomness.
p_1092131280 avatar
tl flag
@user253751 But it does not generate the randomness, does it? To say that AI is able to produce randomness by using a randomness source as input does not make any sence to me (in terms of this question). An AI itself is deterministic. It might use randomness to produce random locking outputs. But this randomness is taken from any source and not created by the AI. In general, no computational algorithm is able to create randomness, it always has to use some kind of randomness source.
Nuclear Hoagie avatar
in flag
"But without any randomness as input into the AI..." Why can't you input true physical randomness? The Lavarand system, for example, is a hardware-based RNG that uses a snapshot of a bank of lava lamps as the random seed. The underlying algorithm which uses the image-generated seed is only pseudorandom, but the end-to-end system is considered truly random because of the randomness inherent in the seed.
p_1092131280 avatar
tl flag
There are two ways to look at the current question. In any way, of course it may use any random input. The question I wanted to answer is: "From an algorithmic point of view, how are these numbers predictable?". What I mean is that an AI is not capable of adding randomness to such random values. It can potentially be capable of outputting random values, but they are only as random as the input was. Thus, **everything that happens inside the AI is deterministic**.
p_1092131280 avatar
tl flag
Ultimately, it depends on the definition of an AI. Of course, one can take a physical RNG and call it an AI, which then is not deterministic. Logical algorithms, neural networks and in general machine learning approaches are deterministic.
Michael Mior avatar
lk flag
@p_1092131280 I'm not sure what you mean by "logical" algorithms. There are plenty of algorithms that make use of randomness (e.g. Monte Carlo algorithms). These will tend to use PRNGs, but there's no reason they couldn't use true sources of randomness.
Score:4
cn flag

The correct measure of randomness is entropy. Entropy is defined for probability distributions, e.g. a fair coin toss has more entropy than a biased one. It makes no sense to say "how random is this number" without knowing with what probability it is produced. For all we know, the AI might always produce the same strings in the same order when asked to provide randomness. Like another answer mentioned, a deterministic algorithm cannot produce randomness. The best you can hope for is to apply a statistical test to the string to test the hypothesis that it was generated from a uniform distribution.

On a side note, asking an AI for randomness seems overkill, much like opening a coconut using a bazooka.

Score:2
cy flag

Maybe it generates numbers at random the way a person does: it is well known that people who think they are generating random numbers actually introduce all sorts of biases. For instance, a person may conscientiously make sure that all ten digits have equal frequencies, but a sequence that is generated by a truly random process, such as Ernie, frequently violates this.

Score:2
jp flag

In general, a GPT AI architecture is capable of generating properly random numbers. That's because the overall design consists of two stages: a trained neural network which gives probabilities for the next possible tokens in the sequence, and then a simple piece of code which picks one at random based on those probabilities.

Due to the second stage, this system design intrinsically has plenty of randomness available to use. Indeed, every single token is the result of a random choice.

If the neural network was perfect, it would give the same probabilities for each of the digits 0-9, and then the random stage would pick one completely at random (subject to the quality of the randomness available to the system).

In practice, the neural network probably does not output exactly the same probabilities for the digits 0-9. But it could, so the answer to the title question is "yes."

Score:1
sj flag

Randomness is best understood, IMHO, as relative to information at hand. Perhaps there's true randomness at the quantum level, but at the macro-scale it's an illusion of ignorance.

For example, if I know all of the forces exerted on a coin when it's flipped, while it's in the air, and the properties of the surface where it will land, I could (conceivably) predict the outcome. Similarly, if I know the current state and and PRNG algorithm, I can predict the next number generated by a pseudo-random number generator (PRNG). If I know all about my CPU and its usage I could even predict the behavior of a heat/cpu based RNG.

The real issue with PRNG is not their lack of randomness (relative to the user's knowledgebase), but the fact that the numbers lack independence between draws (hence our ability to generate a repeatable series of number using a particular seed).

So even if an AI uses a PRNG, from the perspective of a user the number generated is random. The second one, if generated from the first, not so much...

Score:0
ht flag
WBT

While there are all sorts of limitations on AI systems being able to generate random numbers, as discussed in other answers (especially if a system is just regurgitating other content), an AI system should in theory be able to read values from an external source of effective randomness (including already-connected sensors) and report those values. Whether or not any particular AI system does that depends at least in part on its design.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.