Many near collisions but no full collision

Late important addtion: I now realize the code attempts to find collisions for a 64-bit $\operatorname{hash}$ accepting 64-bit messages. If that $\operatorname{hash}$ was a bijection, it would not collide. There is a continuum between a bijection and a random function, and no insurance that $\operatorname{hash}$ behaves mostly like the later. On the contrary, it's internal function $f(x)=C\,x\oplus D\,x\bmod2^{64}$ has no right-diffusion. That is, $x\equiv x'\pmod{2^i}\implies f(x)\equiv f(x')\pmod{2^i}$, thus $f$ is a poor hash round function. This might explain at least in part the difficulty to find collisions by methods designed for random functions. In the later I assume one of:

• the message space for $x$ is $b$-bit with $b$ sizably larger than the (64-bit output)
• we try to find collisions $x,x'$ such that $\operatorname{hash}(p\mathbin\|x)=\operatorname{hash}(p'\mathbin\|x')$ where $p$ and $p'$ are fixed distinct prefixes, $x,x'$ are $b=64$-bit, $x\ne x'$.

I suspect another important issue is that in

I populate (the arrays) by hashing Int values

it is hashed incremental Int values. It is quite feasible to make a function such that incremental values in a large interval do not collide, and it's quite possible the function $\operatorname{hash}$ searched for collisions behaves like that, thus any attempt to find collisions among consecutive values fails.

As an example of a function with no collisions for input in a small interval, consider $H(x)=\left(263x+\left(\operatorname{MD5}(x)\bmod256\right)\right)\bmod2^{64}$. It holds $H(x)-H(x')\equiv263(x-x')+(r-r')\pmod{2^{64}}$, with $r,r'\in[0,255]$ since they are obtained as the last byte of an MD5; thus $\lvert r-r'\rvert<256$. Therefore if $x\ne x'$, the only way to get $H(x)=H(x')$ is that $\lvert x-x'\rvert$ is large, at least $\lfloor 2^{64}/263\rfloor$, which won't be the case for consecutive $x$ in a small interval.

When trying to find collisions for such insufficiently random hash function $H$, an easy fix is to find collisions for the a more random function $x\mapsto H(G(x))$, constructed using some pseudo-random auxiliary bijection $G$, e.g. $G(x)=G_2(G_1(G_0(x)))$ with $G_i(x)=k_i(x\oplus(x\gg\lceil b/3+1\rceil))\bmod2^b$ for $k_i$ haphazard odd $b$-bit constants [where $\gg$ is right shift, and $b$ is the bit size of $x$]. Once a collision $x,x'$ is found with $H(G(x))=H(G(x'))$ but $x\ne x'$, a collision for $H$ is $G(x),G(x')$.

One advantage of searching collisions with Pollard's rho with distinguished points (rather than the method in the question's code) is that it's iterative nature often solves the issue of an insufficiently random function searched for collisions, without an auxiliary $G$; or, a relatively simple $G$ will do (here I think a 1-bit rotation in the feedback of Pollard's rho should do, compensating the lack of right diffusion). Also, Pollard's rho uses less memory, thus works for larger hashes; and for fast hash functions, it's faster, because it's cache-friendly.