hackage-security: file returned by server too large

[avdv]

Originally reported here: commercialhaskell/stack#5770

I have been seeing this error more and more often:

Selected mirror https://hackage.haskell.org/
Downloading root
Waiting to acquire cache lock on /home/runner/.stack/pantry/hackage/hackage-security-lock
Acquired cache lock on /home/runner/.stack/pantry/hackage/hackage-security-lock
Released cache lock on /home/runner/.stack/pantry/hackage/hackage-security-lock
Selected mirror https://hackage.haskell.org/
Downloading timestamp
Downloading snapshot
Downloading mirrors
Cannot update index (no local copy)
Downloading index
file returned by server too large: <repo>/01-index.tar.gz (expected exactly 115606991 bytes)

It started to occur yesterday, and today I have seen it a couple of times. In this CI run almost all jobs are failing because of it.

[phadej]

Right now the file is 115613771 bytes large, so not much larger. And cabal update works.

It would be nice if there were more debug output. Especially what is the mirror used and actual URLs downloaded, so one could manually expect them and check what is inconsistent.

[gbaz]

Note that stack uses via pantry its own http client, distinct from the one(s) used by cabal: https://github.com/commercialhaskell/pantry/blob/ab2eb910d3c7b71d89009ae8d410650f7c726489/src/Hackage/Security/Client/Repository/HttpLib/HttpClient.hs

This may be why the issue is tricky to reproduce -- I'm not sure precisely how that client is implemented, etc.

The filesize that hackage-security compares the returned file against is derived from snapshot.json and that currently is 115613771. So I suspect that stack, perhaps because of an issue with a mirror or a stale network cache in-between, is fetching an old snapshot.json with a different file size listed, but is fetching the current index file, and thus hackage-security reports a mismatch.

[ulidtko]

Subscribing, also got hit by this.

file returned by server too large: <repo>/01-index.tar.gz (expected exactly 119551581 bytes)

Didn't repro on rerun.