Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSS-10465 update trino secret format #57

Merged
merged 9 commits into from
Sep 17, 2024

Conversation

AmberCharitos
Copy link
Collaborator

@AmberCharitos AmberCharitos commented Sep 11, 2024

With the existing structure of the catalog-config.yaml file, this quickly becomes large enough that k8s secrets no longer support the size resulting in an error when adding additional catalogs. https://warthogs.atlassian.net/browse/CSS-10465

This PR:

  • Updates the config file format to use a backend structure to prevent repetition when adding multiple databases from the same server.
  • Adds logic to the charm code for reading this config and turning it into catalogs in trino.

Please note, for now this just supports the addition of postgresql connectors, in the future we can add additional backends and charm logic for other supported connectors but it is very hard to generalised for all as every connector differs considerably in configuration values.

@AmberCharitos AmberCharitos changed the title update trino secret format CSS-10465 update trino secret format Sep 13, 2024
@AmberCharitos AmberCharitos marked this pull request as ready for review September 13, 2024 10:51
Copy link

@kelkawi-a kelkawi-a left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

src/utils.py Show resolved Hide resolved
src/utils.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/utils.py Show resolved Hide resolved
README.md Show resolved Hide resolved
@AmberCharitos AmberCharitos merged commit 2397226 into main Sep 17, 2024
14 checks passed
@gtato gtato deleted the trino-catalog-config-reformat branch September 19, 2024 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants