Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Pydantic I/O types in workflow context #1189

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

Enable Pydantic I/O types in workflow context #1189

wants to merge 9 commits into from

Commits on Sep 24, 2024

  1. Factor out common _SCRIPT_PYDANTIC_IO_FLAG check 

    Code to check if _SCRIPT_PYDANTIC_IO_FLAG is set and error if not occurs
twice; factor out into a shared utility function.

Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    b372ba8 View commit details
    Browse the repository at this point in the history

  2. Stop disabling declaring in _create_subnode 

    _create_subnode temporarily disables _context.declaring to avoid
triggering the altered `__getattribute__` behaviour in
TemplateInvocatorSubNodeMixin in two situations:

 - Pydantic's implementation of `!=`, which is attempting to read field
   values, and will raise a validation error due to the altered field
   names being of incorrect type.
 - Accessing `__class__` on a Pydantic subnode object with no build_obj
   set.

Instead, fix this by:

 - Using `is not` instead of `!=`, which is correct where we use it,
   and also faster.
 - Fixing `__getattribute__` to no longer fail if used on an object with
   no build_obj set.

Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    d5bb695 View commit details
    Browse the repository at this point in the history

  3. Enable Pydantic I/O types in workflow context 

    Extend experimental Pydantic I/O support to allow passing Pydantic types
into `@script`-decorated functions when inside a `with` workflow context
block, and using fields on the returned Pydantic output as shorthand for
the associated Hera template in subsequent steps.

Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    7115e24 View commit details
    Browse the repository at this point in the history

  4. Add tests for the new Pydantic I/O syntax 

    Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    d2dd3d7 View commit details
    Browse the repository at this point in the history

  5. Clear current task dependencies in DAG._add_sub 

    Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    9055bb5 View commit details
    Browse the repository at this point in the history

  6. Allow passing depends in a @w.dag context 

    Currently, TemplateDecoratorFuncMixin._create_subnode unconditionally
adds a depends kwarg, causing a runtime error if the user also supplies
one. Instead, prefer the user-supplied one.

Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    6209b2e View commit details
    Browse the repository at this point in the history

  7. Move Step/Task creation to Steps/Parallel/DAG 

    Move the logic that creates the right leaf node for Steps, Parallel and
DAG to a _create_leaf_node method on those types. DAG now specifies how
to default the depends parameter to Task based on its
_current_task_depends field. This simplifies the duplicated logic in
_meta_mixins to a simple isinstance check for any of those three types,
followed by a _create_leaf_node call.

Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    c6a5bdc View commit details
    Browse the repository at this point in the history

  8. Move new syntax behind decorator_syntax feature 

    Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    f012868 View commit details
    Browse the repository at this point in the history

  9. Fix name of experimental feature message function 

    Signed-off-by: Alice Purcell <[email protected]>
    @alicederyn
    alicederyn committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    af1bcef View commit details
    Browse the repository at this point in the history