Change paragraph object for rendering

I have a paragraph type with a node ref field that, if you leave it empty, is supposed to show a random node using some defined logic.

I have the code to do the query and get the node, and ideally I'd want to put it on the paragraph object in some pre-render hook so that the paragraph can render normally using its usual config. Something like this (warning: pseudo-codish):

function mymodule_paragraph_view($build, $paragraph, $display, $view_mode) {
  if (!is_my_paragraph_type($paragraph)) return;

  if (!$paragraph->field_mynoderef->isEmpty()) return;

  $node = getRandomNode();
  $paragraph->set('field_mynoderef', $node);
}

This is setting the value correctly, but it seems it's too late in the render process for it to affect the rendered output (i.e. it doesn't actually show the node). I've also tried doing that on the $build['#paragraph'] object, which contains the same paragraph, with similar results. How can I make this work?

BONUS: How to make it so it caches the output using a time-based configuration (e.g. expire after 60 seconds, so every minute it will show a different node and it won't run the query logic on every request).

You need to build the loaded node into a renderable array, and add it to the $build array.

$node = getRandomNode();

$entity_type = $node->getEntityTypeId();
$view_mode = 'teaser';
$view_builder = \Drupal::entityTypeManager()->getViewBuilder($entity_type);
$storage = \Drupal::entityTypeManager()->getStorage($entity_type);
$build['field_mynoderef'] = $view_builder->view($node, $view_mode);

Your question on caching is not related to this issue and should be separated into a separate question. Drupal Answers is a Q&A site with the format of one question to one answer, and adding the answer to that question would dilute this topic.