terraform/channels: disable caching for redirects and turn 301s into …

…302s

More context in:
- NixOS/nixos-channel-scripts#42
- NixOS/nixos-homepage#1181
- NixOS/nixos-homepage#1015
- and more linked in the above bugs

S3 object-level redirects can only generate 301s, which are cached
indefinitely by clients. S3 bucket-level redirects are limited to 50
rules, which is too few for our use case. This leaves the option of
using Fastly to doctor the responses from S3.

Using VCL, we:
- Turn 301s (indefinitely cacheable by clients) into 302s (temporary,
  usually not cached).
- Disable all Fastly-level caching for redirects, since some redirects
  need to be kept in sync (for example: a .iso and its associated
  .iso.sha256 should always point to the same version). Redirects should
  be cheap anyway, and the added latency is not a terrible cost compared
  to the potential issues of people not being able to checksum their
  downloads.

We could in theory re-enable caching if we modified the channel scripts
to do a partial Fastly cache invalidation for the impacted redirects.
This is left as an exercise to future readers.

terraform/channels.tf

@@ -212,6 +212,32 @@ resource "fastly_service_vcl" "channels" {
     type    = "recv"
   }
 
+  snippet {
+    content = <<-EOT
+      # S3 object-level redirects can only be 301s. We use them to point
+      # "latest" versions of various channel/release artifacts to the correct
+      # location. First, mark these redirects as temporary. Second, disable
+      # caching, since some of the artifacts need to have matching versions
+      # (e.g. a .iso and its checksum), which is near-impossible to guarantee
+      # with caching unless we explicitly perform invalidations.
+      #
+      # Note: we need to match on 301s and 302s here, since Fastly has multiple
+      # layers, and otherwise a redirect might still get cached at the second
+      # layer after the first layer turned a 301 into a 302.
+      if (beresp.status == 301 || beresp.status == 302) {
+        set beresp.status = 302;
+        set beresp.ttl = 0s;
+        set beresp.grace = 0s;
+        set beresp.cacheable = false;
+        return (pass);
+      }
+    EOT
+    name    = "Change 301 from S3 to 302"
+    # Keep close to last, since it conditionally returns.
+    priority = 999
+    type     = "fetch"
+  }
+
   logging_s3 {
     name              = "${local.channels_domain}-to-s3"
     bucket_name       = local.fastlylogs["bucket_name"]