GZip doesn't work with Accept-Encoding: "gzip, deflate"


#1

I have set up gzip compression as per instructions (auto gzip), but still there was still no compression. After some digging, I realized that the browser (Firefox) sends

Accept-Encoding: “gzip, deflate”

While automatically generated VCL script has the following line:

if (req.http.Accept-Encoding == “gzip”) {
set beresp.gzip = true;
}

Now it was obvious why it doesn’t work. Shouldn’t it be changed to

if (req.http.Accept-Encoding ~ “gzip”)

?

You propose an alternative solution in your blog: http://www.fastly.com/blog/best-practices-for-using-the-vary-header/ The solution is to normalize the header:

if (req.http.Accept-Encoding ~ “gzip”) {
set req.http.Accept-Encoding = “gzip”;
} else {
unset req.http.Accept-Encoding;
}

Two questions:

  1. Should default template be updated?
  2. Can I make gzip work for now?

#2

Hi @artemf we normalize Accept-Encoding as it comes in through our master configurations. We should surface that in a better way.

If you’d prefer, can you send me a few misbehaving URLs in a PM? I’ll take a look at them and see if there’s any changes that need to be made.


#3

Thanks @aspires! Next time I wouldn’t jump to conclusions too soon! :smile:

Maybe I’m dumb, but I can’t find PM here, so postng URLs in reply.

I can’t make gzip work with any kind of url:

  1. Assets: http://glavdigest.ru/assets/application-dbf66ede5cbd79b735916ac1a6c740c4.css
  2. Dynamic html (which is still cached in Varnish): http://glavdigest.ru/messages?page=1000

Thanks again!


#4

When I access both of these in the most recent Firefox for mac, I see Content-Encoding: gzip. Is there some other way you’re reproducing this?


#5

Hmm… Interesting. For simplicity, I can reproduce it with curl:

curl -vs --compressed http://glavdigest.ru/assets/application-dbf66ede5cbd79b735916ac1a6c740c4.css -o /dev/null

See my output here: https://gist.github.com/artemf/f2bc1896f82db56fdabd

I don’t get Content-Encoding: gzip. I wonder what your headers would be.


#6

Here’s Firefox (latest Mac version) screenshot for completeness: http://cl.ly/image/0X000x0c381S
(Firefox is in Russian, but you can clearly see headers - it’s the same as with curl)


#7

Hmm… Even more interesting. I’m in Bali, Indonesia. When I enable VPN (TunnelBear via Singapore or Australia), I’m getting gzipped contents correctly! Check the very same curl command run under VPN: https://gist.github.com/artemf/9e97c0074d29359959b2

Firefox also recieves gzipped content without issues.

Hmm… It sounds stupid for local Indonesian providers to mess with gzip (if that’s even possible)?.. I definitely recieve gzipped contents from other websites. Here’s the curl output to Fastly docs (without VPN), you can see that gzipping works: https://gist.github.com/artemf/e5c2025c66098be06c92

Do you have any ideas / pointers to what might be happening here?


#8

Additional info.

Tried it from different place / provider here (without VPN), also works fine. Seems like it was an issue with specific provider / network.

Still wonder though, what might be the reason. Especially since Fastly’s docs gzipping worked fine.


#9

Usually, this happens when an object gets cached before gzip gets turned on and an object purge usually flushes it out. There could be some sort of caching proxy in play with some remote ISPs though, so unsure as to the cause of this specific aberration. Either waiting for the ttl to expire or purge the object and retry will usually clear it up :slight_smile:


#10

@peter I did a lot of purges. Caching is definitely not the issue here - I was getting the fresh version all the time, just not gzipped.

Indonesian ISPs are notorious for messing with the data you get / receive. Maybe that was one of the cases. The strange thing is that it was only with my website.


#12

I have had the same situation while accessing Mozila on my Mac…


#13

Using HTTPS is a good way to prevent ISPs from interfering with your requests/responses.