Would anyone have a lead to the crypto function to do this? The challenge is for the server to compute a random HASH or MAC based on a digest only, which the client cannot guess.
Well, as some of the comments point out, the best we can do is prove that the firmware knows the correct image; not that it is actually running it.
However, I'll proceed ignoring that detail.
It doesn't look doable using symmetric crypto; the value $MAC(key, image)$ can only be computed with knowledge of the image, hence the server (who doesn't know the image) cannot verify it. I suppose that the server could have a number of precomputed $key, MAC(key, image)$ pairs; however that works only so many times (depending on the number of precomputed pairs).
On the other hand, it is a solvable problem using asymmetric crypto; here is a protocol that solves it (albeit in a way that is likely too expensive to be usable in practice) - this will show that this is a solvable problem (with the above caveat):
The IoT and the server share a value $n$, which is an integer of secret factorization.
The server transmit a challenge value $g$
The IoT translates the image into a (large) integer $i$ (so that if the image consists of 10kbyte = 80kbits, we have $i$ as an 80,000 bit integer); he computes $g^i \bmod n$ and sends that value to the server
The server has a precomputed $j = i \bmod \phi(n)$ stored; he computes $g^j \bmod n$ and compares that to the value that the IoT sends
The trick this uses is that the server has a summarized version of the image $i \bmod \phi(n)$; however the IoT cannot compute that summarized version (because he doesn't know the factorization of $n$); so he needs the entire image to compute it.
