Join Log in
Score:0
avatar Server

Mounting Azure Files in Azure App Service

kh flag
JohnLBevan

Summary

I have an Azure App Service running a custom container. When I bind a path to an Azure Files share my container stops working. Looking at the Container Issues logs, I see the error: [BYOS] Custom storage volume(s) failed to initialize: [/var/LWASFiles/Sites/my-app/a3484543-39f9-45a3-816b-9524640dfd50].

Detail

  • My custom container has a volume defined /var/www/html/v3/uploads.

  • I'm attempting to map this to an Azure Files share, which is on a storage account in the same vnet as the hosting ASE, with rules permitting traffic between the ASE's subnet and the file share on ports 137, 138, 138, and 445.

  • The hosting ASE is configured with an internal load balancer (i.e. it's a private ASE).

  • The storage account has a private endpoint configured.

  • The volume is mapped under the App Service's Settings > Configuration > Path mappings, using a valid key and a mount path which matches the volume path on the container.

  • Before adding the path mapping, the app service runs as expected.

  • After adding the path mapping, the container fails to start / the only exception information I can find is from the Container Issues logs, as per the summary above.

  • If I remove the path mapping, the issue persists, even after forcing a restart. The only way to resolve the issue is to drop and redeploy the app service.

  • It successfully connects to a MySql DB (Azure Database for MySQL servers) via a private endpoint.

  • Using dig {privateEndpointFqdn} I'm able to prove that the Storage Account's private endpoint is resolved correctly (as is the MySQL's; as you'd expect)

  • Using tcptraceroute {privateEndpointFqdn} {destinationPort} I'm able to prove that I can connect to the storage account on port 445 (and to the MySQL DB on port 3306).

  • Note: I can't connect to the storage account on ports 137, 138, or 139 despite those being allowed via the NSG on the same in & outbound rules as are used for 445, above; though I suspect those ports are no longer required for CIFS (I only added these as a just-in-case measure after first hitting the issue in case they were somehow relevant as they were mentioned in some posts).

  • I'm running the above tests by SSHing into the container, so the commands are run from its context. Note: Because I can't start the container after adding the path mapping, these tests are performed after creating the app service but before adding the path mapping.

  • I've increased WEBSITES_CONTAINER_START_TIME_LIMIT to its max value: 1800

  • My container exposes port 80 (i.e. one of the default ports supported by App Service for container); and the website works when the path mapping doesn't exist, so this shouldn't be the issue. I've also set WEBSITES_PORT to 80, just for belt-and-braces.

  • WEBSITES_ENABLE_APP_SERVICE_STORAGE is set to false (though I've also experimented with setting it to true, just in case / it made no difference so I reverted to false).

  • My file share has several GBs of data. I've tried creating an identical file share with no content and mapping to that (immediately after recreating the app service and proving that that worked; to ensure my tests weren't impacted by the hangover of the larger file share). This gives the same issue as with the larger file share.

  • The image for my container is being loaded from an Azure Container Repo.

  • The image is based on the ubuntu:21.04

  • I've included the azure-cli and cifs-utils packages (I think this is only required when performing the mount operation from the container, rather than from the AppService's config; but I wanted to cover all assumptions)

  • The hosting ASE is in the UK South region (as is the storage account / all resources).

Note: This is my first experience with running containers in App Service; so PEBKAC is definitely a possibility.

Any suggestions or troubleshooting advise would be much appreciated. Thank-you.

872
0 + 0
ng flag
Sam Cogan
Just to confirm, this is mounting an Azure Files SMB share, not NFS right?
0
Reply
kh flag
JohnLBevan
Correct; SMB/CIFS.
0
Reply
Score:0
avatar Server
kh flag
JohnLBevan

I resolved the issue by changing the storage account's network from selected networks (in which I'd whitelisted by ASE's VNet and the outbound IPs for the web app) to all networks via the portal:

https://portal.azure.com/#@<myTentantName>.onmicrosoft.com/resource/subscriptions/<mySubscrption>/resourceGroups/<myStorageAccountsResourceGroup>/providers/Microsoft.Storage/storageAccounts/<myStorageAccountName>/networking

I then restarted my app (including dropping and re-adding the mount... not sure if that was required, but for good measure):

# stop the web app
az webapp stop \
    --subscription <sub id> \
    -g <resourcegroup> \
    -n <sitename> 

# drop the existing path mapping
az webapp config storage-account delete \
    --subscription <sub id> \
    -g <resourcegroup> \
    -n <sitename> \
    --custom-id <path-mapping-name> \

# re-add the path mapping
az webapp config storage-account add \
    --subscription <sub id> \
    -g <resourcegroup> \
    -n <sitename> \
    --custom-id <path-mapping-name> \
    --storage-type AzureFiles \
    --account-name [Azure storage account name] \
    --share-name   [Azure storage share name] \
    --access-key   [storage access key] \
    --mount-path   [/path/to/mount within the container] 

# restart the web app
az webapp start \
    --subscription <sub id> \
    -g <resourcegroup> \
    -n <sitename>
0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.