Add SMTP brokerpak and supporting #5

jameshochadel · 2024-09-10T14:49:08Z

Changes proposed in this pull request:

I've been developing on a topic branch to avoid a PR workflow while in early stages of development, but I'm going to start PRing changes for each issue I resolve now.
Adds the SMTP brokerpak, including provision and bind code, manifest and service definition yaml files, and a Go client for testing service instances.
Adds local development tools for working with the CSB and brokerpaks locally.

Things to check

For any logging statements, is there any chance that they could be logging sensitive data?
Are log statements using a logging library with a logging level set? Setting a logging level means that log statements "below" that level will not be written to the output. For example, if the logging level is set to INFO and debugging statements are written with log.debug or similar, then they won't be written to the otput, which can prevent unintentional leaks of sensitive data.

Security considerations

Follows standard practices to avoid credential disclosure.

Also update the Dockerfile to build it. Source: https://github.com/GSA-TTS/datagov-brokerpak-smtp

They're the same underlying value, but looking at the outputs alone, you wouldn't know that.

https://github.com/cloudfoundry/cloud-service-broker/blob/main/docs/brokerpak-dissection.md

https://github.com/cloudfoundry/cloud-service-broker/blob/main/docs/opentofu_migration_guide.md#specifying-providers

Download the Terraform provider from GitHub to follow Hashi registry ToS

https://github.com/cloudfoundry/csb-brokerpak-aws/blob/799ab69e6439da1bcef542c9fafa9393af2dd7a5/manifest.yml#L11C1-L12C77

And Dockerfiles are unused

- Remove txt_verification_record which is redundant with DKIM records - Mirror tags from https://github.com/cloud-gov/go-broker-tags/blob/main/tags.go#L10

This reverts commit c2954f2.

See: cloud-gov/product#3107 (comment)

Related to cloud-gov/product#3118

Resolves cloud-gov/product#3120

Goal is to minimize disruption of updates. Resolves: cloud-gov/product#3123

Uses workaround to HIL parser limitation here: cloudfoundry/cloud-service-broker#1086 (comment)

Resolves cloud-gov/product#3107

Resolves cloud-gov/product#3118

brokerpaks/cg-smtp/client/bin/down.sh

markdboyd · 2024-09-10T14:55:42Z

brokerpaks/cg-smtp/terraform/bind/outputs.tf

+  value = aws_iam_access_key.access_key.id
+}
+
+# TODO should we only expose the SMTP interface?


What would this mean?

The binding provides a set of SMTP credentials and an AWS access key to the IAM user, which allows customers to use the AWS SES APIs instead of the SMTP interface. I'll take this TODO out, though; it's a desirable behavior so they can use the AWS SDK if they want, and the security impact is the same since the user has privileges limited to their identity only.

markdboyd · 2024-09-10T14:57:33Z

brokerpaks/cg-smtp/terraform/bind/provider.tf

+    tags = {
+      "broker"                = "Cloud Service Broker"
+      "client"                = "Cloud Foundry"
+      "environment"           = "local" # todo, parameterize at CSB level


I assume we'll follow up on this later

Yep, that's probably in my next issue, which is tackling Terraform for deploying the broker itself.

See discussion: #5 (comment)

jameshochadel added 30 commits March 22, 2024 10:20

oci-build step only supports settings args, not envs

19e0c96

Stop failing when zscaler.crt is not present by copying entire repo

15f7a19

Switch to OpenTofu, which upstream CSB now uses instead of Terraform

afa819d

Copy datagov-smtp-brokerpak verbatim

bedf1f8

Also update the Dockerfile to build it. Source: https://github.com/GSA-TTS/datagov-brokerpak-smtp

Ignore compiled brokerpaks

ec020d6

Debug failing COPY step

b762c3d

Brokerpak is using name from yaml, not directory

d8e0ccd

Regenerate cg-smtp using 'csb pak init' for new plan GUIDs

b153c75

Drop 'empty' buildpack

548a2dc

Fix using deprecated version constraint in provider block

f9c2fea

Allow granular DMARC reporting and update policy to p=reject

bcc6aa3

Make output usage more obvious by splitting username and access key ID

0695396

They're the same underlying value, but looking at the outputs alone, you wouldn't know that.

Update manifest to match new outputs

32ada9e

Switch to providers from the HashiCorp registry, per CSB docs

21b1477

https://github.com/cloudfoundry/cloud-service-broker/blob/main/docs/brokerpak-dissection.md

Drop debugging lines

49a6590

Try GitHub release URLs from OpenTofu

2833bc5

Make changes based on OpenTofu migration guide

87b18fe

https://github.com/cloudfoundry/cloud-service-broker/blob/main/docs/opentofu_migration_guide.md#specifying-providers

CSB is not yet compatible with OpenTofu providers

a1b2d4e

Download the Terraform provider from GitHub to follow Hashi registry ToS

Match example provider configuration exactly

74ee00d

Specify registry source in bind

82298ea

Try with replacements section

5c55d02

https://github.com/cloudfoundry/csb-brokerpak-aws/blob/799ab69e6439da1bcef542c9fafa9393af2dd7a5/manifest.yml#L11C1-L12C77

Match up AWS provider versions

15a8366

Test with terraform_upgrade_path set

d9a89d4

Add required default field

12153b6

Test with upstream terraform provider

cc5d57b

Migration guide recommends specifying url_template

10a41c9

And Dockerfiles are unused

Test brokerpak unzip with file later than versions.tf

5b6c9f0

Try again

877de0b

Add missing reference to versions.tf

372903d

<oh my god>

Add required 'sensitive' attributes to bind output

b93b38b

jameshochadel added 21 commits August 19, 2024 16:09

Revert IAM changes to troubleshoot

6032299

Switch to sesv2 module to support tagging SES identity

c64a1ba

- Remove txt_verification_record which is redundant with DKIM records - Mirror tags from https://github.com/cloud-gov/go-broker-tags/blob/main/tags.go#L10

Could not find space_name in context; try without

c2954f2

Revert "Could not find space_name in context; try without"

a4aa64a

This reverts commit c2954f2.

Try with map indexing syntax

278c1c0

Temporarily remove unusable tags

28e4b77

See: cloud-gov/product#3107 (comment)

Remove likely-unusable IP condition

19ff5fd

IAM user needs permission on the Configuration Set to send mail

5c3d466

Add empty placeholder tags

7448d37

Tag all resources created by brokerpak with provider default_tags

2de6a1c

Add Makefile to support local development

98b7df2

Related to cloud-gov/product#3118

Return FIPS endpoint in binding credentials

900667c

Resolves cloud-gov/product#3120

Prevent destruction of resources whose ARNs would change

c204e10

Goal is to minimize disruption of updates. Resolves: cloud-gov/product#3123

Add Go client for local or CF testing of SMTP interface

a3aec67

Improve descriptions and documentation in smtp plan

3f3d246

Improve documentation and variable naming

37f356b

Bump OpenTofu version to latest

df7c373

Extract tags from context in Terraform instead of service definition

722729a

Uses workaround to HIL parser limitation here: cloudfoundry/cloud-service-broker#1086 (comment)

Add final tags, meeting parity with go tagging library

012a2e5

Resolves cloud-gov/product#3107

Remove unused import

ac1c88f

Add scripts for creating and deleting resources locally

b714f7f

Resolves cloud-gov/product#3118

jameshochadel requested a review from a team as a code owner September 10, 2024 14:49

markdboyd reviewed Sep 10, 2024

View reviewed changes

brokerpaks/cg-smtp/client/bin/down.sh Show resolved Hide resolved

markdboyd reviewed Sep 10, 2024

View reviewed changes

jameshochadel added 2 commits September 10, 2024 16:20

Parameterize the runtime environment of the broker to support tagging

69ec352

Drop TODOs regarding SMTP interface

7b887c1

See discussion: #5 (comment)

markdboyd approved these changes Sep 10, 2024

View reviewed changes

jameshochadel merged commit fde3dd9 into main Sep 10, 2024

jameshochadel deleted the brokerpak-topic branch September 10, 2024 20:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add SMTP brokerpak and supporting #5

Add SMTP brokerpak and supporting #5

jameshochadel commented Sep 10, 2024

markdboyd Sep 10, 2024

jameshochadel Sep 10, 2024

markdboyd Sep 10, 2024

jameshochadel Sep 10, 2024 •

edited

Loading

Add SMTP brokerpak and supporting #5

Add SMTP brokerpak and supporting #5

Conversation

jameshochadel commented Sep 10, 2024

Changes proposed in this pull request:

Things to check

Security considerations

markdboyd Sep 10, 2024

Choose a reason for hiding this comment

jameshochadel Sep 10, 2024

Choose a reason for hiding this comment

markdboyd Sep 10, 2024

Choose a reason for hiding this comment

jameshochadel Sep 10, 2024 • edited Loading

Choose a reason for hiding this comment

jameshochadel Sep 10, 2024 •

edited

Loading