Persistent anonymous user session with headless D9

In short: a new php session is started upon every request made through nextjs, I somehow must keep the anonymous user session alive.

For my anonymous users I need keep track of data submitted through nextjs. I Initially developed my app with Drupal 8.9, and many months later upgraded to 9.4-dev.

Before the upgrade to 9.4-dev, I was sure the sessions worked properly across several browsers and anonymous users at the same time; I could use the app, store and re-use unique data per anonymous user etc. Sadly its hard to verify this, downgrading to 8.9 will cause additional pain..

After the upgrade, the anonymous users suddenly share the same session. This might be related to https://www.drupal.org/project/session_based_temp_store/issues/3257214 and https://www.drupal.org/node/3006306 but not sure. Using session_based_temp_store:1.1 results in a shared session and session_based_temp_store:1.2 results in a non-persistent session.

The code below used to work:

 if ($this->currentUser->isAuthenticated()) {
  $user_preferences = unserialize($this->currentUser->get('field_preferences')->value);
  foreach ($values as $key => $value) {
    $user_preferences[$key] = $value;
  }
  $this->currentUser->set('field_preferences', serialize($user_preferences))->save();
}
else {
  $session = \Drupal::service('session_based_temp_store')->get('mymodule_user');
  $user_preferences = $session->get('d_user');
  foreach ($values as $key => $value) {
    $user_preferences[$key] = $value;
  }

  $session->set('d_user', $user_preferences);
}

Trying to fix the problem, I tried to use tempstore.private which resulted in the same problem (new session every request).

After this I tried to use the code below, which also resulted in the same problem:

$_SESSION['my_module']['data'] = 'My Data';
$request = $this->requestStack->getCurrentRequest();
$session = $request->getSession();

Even test var $_SESSION['my_module']['data'] is "reset" every time. So it doesn't really matter what kind of storage approach I try to use; the problem is clearly a non persistent session.

All of my rest resources are called with POST requests and authenticated with oauth2. Everything works nicely for authenticated users. Anonymous users can also successfully make requests.

Searching Google I can't really find anything useful, which makes me think i'm missing something obvious..

The main goal is: anonymous users must keep a persistent session in a headless drupal, also when the user closes the browser and comes back a week later, the same session must still be alive..

I do not want to store any user entered data in the frontend.

The problem is kind of logical I think..: how can Drupal know who is making the anonymous request? Should I have some sort of anonymous user cookie in the frontend, connected to an anonymous user session in the backend, passed with every request?

Can someone point me into the right direction please? Any help is much appreciated.

The main goal is: anonymous users must keep a persistent session in a headless drupal, also when the user closes the browser and comes back a week later, the same session must still be alive.

This is what cookies are for. I don't know about nextjs, but in my headless React app, I store all anonymous user preferences as a JSON object which I dump into the cookie.

Relying solely on Drupal's session handling to reliably track user preferences for anonymous users sounds like a recipe for disaster.

I do not want to store any user entered data in the frontend.

You're going to need to store the cookie. However, cookies that exclusively store user preferences and are not used to track users will not run afoul of the GDPR, etc.