Azure Function App 502 Bad Gateway Fix - ZipFS setup failed

Overview

Deploying Function App is usually a simple task, but sometimes Azure Function Runtime is not able to properly start due to which you might start seeing weird behaviors while accessing your app.

In this post, I am going to find 502 Bad Gateway fix occurring due to “Failed to open siteversion.txt. ZipFS setup failed. Error: 0x80070002”.

Scenario

For my scenario, I encountered 502 bad gateway with a Function App that is deployed on a App Service Plan (ASP) integrated inside an App Service Environment v3 (ASE). This post will take you through as to how I was able to fix it.

Debugging Steps

Let me take you through some of the steps that I did to reach the potential source of problem.

Is Your Function Runtime Reachable?

Firstly, the most obvious check is to see whether your function runtime is reachable or not.

Accessing the host URL is probably one of the easiest way to see if your function runtime is properly working.

Type	URL
Non-ASE Function App	https://<function app name>.azurewebsites.net/
ASE Function App	https://<function app name>.<ASE name>.appserviceenvironment.net/

If your function runtime is reachable, you should ideally see the below page.

In my scenario, using the ASE integrated URL, I encountered 502 bad gateway as shown below. This points out to the fact that function runtime is not properly working.

Is Your Kudu Tools Working?

The next obvious step is to check whether your Kudu Tools are accessible.

You can reach Kudu tools using the below URLs.

Name	URL
Non-ASE Function App	https://<function app name>.scm.azurewebsites.net/
ASE Function App	https://<function app name>.scm.<ASE name>.appserviceenvironment.net/

In my scenario, my Kudu tools were working and I am able to access it.

If your Kudu tools are not accessible, then it might be some networking or DNS problem that is stopping your Function App proper functioning.

 

Checking DNS via Kudu Tools

If your Kudu tools (SCM) are working, then you have more options in place to debug further.

You can go to Debug Console → CMD/PowerShell to access the command line and use nameresolver.exe to check the DNS for your function app.

Once of the most common problem with 502 gateway is Function App not able to access your configured AzureWebJobsStorage Storage Account due to underlying networking issues which is DNS in most of the cases.

You can run the following command to check DNS for the storage account.

nameresolver.exe <storage account name>.blob.core.windows.net

You should get the private IP resolved by your configured DNS in the output. If it is correct, then you can assume your DNS is working.

In my scenario, DNS was working as expected.

Checking Logs via Kudu Tools

Next step is to check if you can spot something in Logs that can point out what’s going wrong with the function app.

These log files are available inside LogFiles folder which can be accessed by going to Debug Console → CMD/PowerShell as shown below.

For my scenario, I was able to spot something weird in the eventlog.xml as shown in the snippet below.

<EventData>
	<Data>Failed to open siteversion.txt. ZipFS setup failed. Error: 0x80070002</Data>
</EventData>

This log was getting repeated again and again pointing out the actual problem.

 

Actual Problem

So for our scenario, the problem was “Failed to open siteversion.txt. ZipFS setup failed. Error: 0x80070002”.

The potential reasons as to why this problem might occur is as follows:

• If you have WEBSITE_VNET_ROUTE_ALL set to 1, all outbound traffic will go through the VNET and subject them to Network Security Groups on your VNET which can block requests required to copy the ZIP from the remote source as mentioned in the error message.
• When using ZipDeploy with RUN FROM PACKAGE, the task that is supposed to mount the Zip package to a virtual mount is failing, so the code never updates from the last successful deployment.

Potential Solutions

The potential solutions or workaround for this that can work are as follows:

• Disable VNET integration in your function app if it is not required.
  • This was not applicable for my scenario.
• If you need VNET integration, check your Network Security Groups configured with the VNET as the rules might be blocking requests.
  • In my case, there was already a properly running function app in place which works with this ASE (VNET integration enabled), hence there should be no blocking rules in place.
• Removing & Re-adding WEBSITE_RUN_FROM_PACKAGE App Settings.
  • For my scenario, this instantly fixed the problem as the host was able to restart the function runtime where it was probably stuck in mounting the Zip package to a virtual mount all this while. After it worked, I re-added this key again and function app keeps on working.