Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support for unique where clauses for each relation passed to union_relations #897

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

support for unique where clauses for each relation passed to union_relations #897

wants to merge 1 commit into from

Conversation

thisismath
Copy link

@thisismath thisismath commented May 23, 2024 •
edited by dataders
Loading

resolves: #895

This is a:

  • documentation update
  • bug fix with no breaking changes
  • new functionality
  • a breaking change

All pull requests from community contributors should target the main branch (default).

Description & motivation

currently when a where clause is passed to union_relations, it is used for every relation that was passed. In some situations, this approach is not ideal. The proposed code change here maintains current functionality of passing strings to the where param but also allows for lists to be passed just like how the relations are passed to the relations parameter. Support for specific where clauses for each relation allows for more flexibility in the way the macro can be deployed

Checklist

  • This code is associated with an Issue which has been triaged and accepted for development.
  • I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
    • BigQuery
    • Postgres
    • Redshift
    • Snowflake
  • I followed guidelines to ensure that my changes will work on "non-core" adapters by:
    • dispatching any new macro(s) so non-core adapters can also use them (e.g. the star() source)
    • using the limit_zero() macro in place of the literal string: limit 0
    • using dbt.type_* macros instead of explicit datatypes (e.g. dbt.type_timestamp() instead of TIMESTAMP
  • I have updated the README.md (if applicable)
  • I have added tests & descriptions to my models (and macros if applicable)
  • I have added an entry to CHANGELOG.md

@thisismath
support for unique where clauses for each relation passed to union_re…
6da308b 
…lations

currently when a where clause is passed to union_relations, it is used for every relation that was passed. In some situations, this approach is not ideal. The proposed code change here maintains current functionality of passing strings to the where param but also allows for lists to be passed just like how the relations are passed to the relations parameter. Support for specific where clauses for each relation allows for more flexibility in the way the macro can be deployed
@thisismath
Copy link
Author

thisismath commented May 23, 2024 •
edited
Loading

For example:

prior functionality is maintained when passing a string as designed:

{{ dbt_utils.union_relations(
    relations=[ref('my_model'), source('my_source', 'my_table')]
    , exclude=["_loaded_at"]
    , where='1=1' 
) }}

and allows for specific where clauses when passing a list:

{% set max_dateframe = "'2023-06-01'" %}

{{ dbt_utils.union_relations(
    relations=[ref('my_model'), ref('my_model_view')]
    , where=[
      'ts >= '~max_dateframe
      , 'ts < '~max_dateframe
    ]
) }}

@dbeatty10 dbeatty10 added the enhancement New feature or request label Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

support for specific where clauses for each relation in the union_relations macro
2 participants
@thisismath @dbeatty10