Score:1

Returning long-running api response as a "download"

ph flag

I'm working on a REST api endpoint which takes in an array of IDs, loads all of the entities associated with those IDs, does some processing on them, then returns them in a chosen format (csv, xls etc.).

This processing could take a long time with a large number of IDs, so I want to start returning data to the user immediately as if they are downloading a file (which they sort of are) and finish the download when the last entity is processed.

Is there a way to do that?

Kevin avatar
in flag
This sounds like streaming the file: https://symfonycasts.com/screencast/symfony-uploads/file-streaming
Score:0
cn flag

Excellent comment by @Kevin. Drupal core doesn't use StreamedResponse, but as always the Webform module is a good resource for examples, which also shows the difference between viewing and downloading the streamed result:

/modules/contrib/webform/modules/webform_submission_export_import/src/Controller/WebformSubmissionExportImportController.php

  /**
   * Returns the Webform submission export example CSV view.
   */
  public function view() {
    return $this->createResponse(FALSE);
  }

  /**
   * Returns the Webform submission export example CSV download.
   */
  public function download() {
    return $this->createResponse(TRUE);
  }

  /**
   * Create a response containing submission CSV example.
   *
   * @param bool $download
   *   TRUE is response should be downloaded.
   *
   * @return \Symfony\Component\HttpFoundation\Response
   *   A response containing submission CSV example.
   */
  protected function createResponse($download = FALSE) {
    $webform = $this->importer->getWebform();

    // From: http://obtao.com/blog/2013/12/export-data-to-a-csv-file-with-symfony/
    $response = new StreamedResponse(function () {
      $handle = fopen('php://output', 'r+');

      $header = $this->importer->exportHeader();
      fputcsv($handle, $header);

      for ($i = 1; $i <= 3; $i++) {
        $webform_submission = $this->generateSubmission($i);
        $record = $this->importer->exportSubmission($webform_submission);
        fputcsv($handle, $record);
      }

      fclose($handle);
    });

    $response->headers->set('Content-Type', $download ? 'text/csv' : 'text/plain');
    $response->headers->set('Content-Disposition', ($download ? 'attachment' : 'inline') . '; filename=' . $webform->id() . '.csv');
    return $response;
  }

When testing this module you won't see much of a streaming effect when viewing the result in the browser because of the buffering enabled in most web servers. You need much higher numbers of items, like $i <= 3000, to fill up the buffers and get a streaming effect for a few seconds.

Kevin avatar
in flag
Recently added StreamedResponse to the OpenAI module. Very cool in action.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.