Score:2

Why is my Google Cloud Function timing out when making a HTTP request to my AWS Fargate instance?

eu flag

I am working on a Google cloud function for beforeSignIn trigger which needs to fetch some data from a microservice hosted on a AWS Fargate instance. The request times out but only in the Google cloud function environment with this one particular domain. The code runs fine locally.

A simplified version of the code looks like this:

import { Auth } from 'gcip-cloud-functions';
import fetch from 'node-fetch';

const authClient = new Auth();

export const beforeSignIn = authClient.functions().beforeSignInHandler(async (userRecord, context) => {
  // ...
  const response = await fetch(process.env.MICROSERVICE_URL);
  // ...
});

The URL is read in from an environment variable. If I change this environment variable to another domain, like google.com or bbc.co.uk, or even a domain pointing to one of our Fargate instances from another project, the fetch works fine in the Google cloud function environment and I get a valid response.

Otherwise the fetch request times out and the cloud function aborts and the following is logged:

Function execution took 20006 ms, finished with status: 'error'
FetchError: request to {url} failed, reason: connect ETIMEDOUT
    at ClientRequest.<anonymous> (file:///workspace/node_modules/node-fetch/src/index.js:108:11)
    at ClientRequest.emit (node:events:513:28)
    at ClientRequest.emit (node:domain:489:12)
    at TLSSocket.socketErrorListener (node:_http_client:502:9)
    at TLSSocket.emit (node:events:513:28)
    at TLSSocket.emit (node:domain:489:12)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
{
  type: 'system',
  errno: 'ETIMEDOUT',
  code: 'ETIMEDOUT',
  erroredSysCall: 'connect
}

According the CloudWatch logs, the request is not reaching the instance. I have checked the network configuration of the AWS environment and couldn't find any obvious problems there.

Arrow Root avatar
sm flag
Did you check the security groups?
alsuvo avatar
eu flag
@ArrowRoot I did, the security group has inbound rules with `0.0.0.0/0` and `::/0` for ports 443 and 80 and outbound is the same but for all traffic
Score:1
eu flag

After some investigation I discovered that the Cloud Function was trying to use IPv6 but the AWS environment was not set up to handle this.

I solved the problem by adding a new rule to the VPC routing table with destination ::/0, targeting the internet gateway. Before, only 0.0.0.0/0 was targeting the internet gateway.

Score:0
th flag

I'm happy to hear you identified the problem's root cause and were able to fix it by including a new rule in the VPC routing table to accommodate IPv6 traffic. You enabled IPv6 communication between your AWS Fargate instance and the Google Cloud Function by adding the rule with destination::/0 and targeting the internet gateway.

To maintain effective connectivity between various systems, IPv6 support is crucial, especially when working with cloud services that may use IPv6 addresses. To allow IPv6 traffic to reach your AWS environment, adding the proper routing rule to the VPC routing table was the right course of action.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.