UiLayer for grouped UI effects and world-space UI

[UkoeHB]

This is a proposal for a UiLayer component that causes UI sub-trees to be rendered on textures using UiMaterials.

Objectives

• Make it easy to add fade in/out effects to large groups of entities.
• Enable complex group-wide UI effects like blur or that use custom shaders.
• Allow effects to be selectively applied throughout a UI hierarchy, without disrupting layout and with control over whether effects stack.
• Make it easier to write UI effects without needing to mess with render layers.
• Facilitate scaling UI sub-trees without affecting layout (e.g. an 'expand into place' behavior, or a 'pop-out on hover' behavior).
• Make it easier to set up world-space UI.

Background

Currently, if you want a fade-in/fade-out effect on UI then you need to define a custom render target and stick a UI tree on a camea with TargetCamera. This works well, but requires some boilerplate and technical knowledge to set up, and does not extend to the advanced case of applying effects in the middle of a UI tree on nodes that participate in layout. It also means world-space UI is clunky.

See:

• Added UI Opacity #11716
• UI Opacity #6956
• World-space UI #5476

Summary

The new UiLayer<M> component sets up a UI node so all children of the node are rendered to a UiMaterial stored on that node.

An entity with UiLayer<M> requires the following:

• Node: We are using UiMaterial, so we also need Node (and, transitively, Style/etc.).
• Handle<M: UiLayerMaterial>: Stores a handle to the UiMaterial that will present our texture. If no handle is provided by the user, then a new default material will be inserted.
• UiLayerImage: Stores a handle to the Image asset of our texture. If no handle is provided by the user, then a new default image asset will be inserted. Note that the image will be auto-resized every tick.

After a UiLayer<M> is added, we perform the following steps:

• Make a new Image asset if necessary.
• Make a new M material if necessary, and set its texture from the image asset (with UiLayerMaterial::set_texture).
• Make a new Camera and insert it to the entity in a UiLayerCamera component. This camera is hidden, we will update it manually.
  • Hidding the camera means it won't participate in visibilty checks of irrelevant entities, which makes it much more performant to have many of these cameras. It also decouples the cameras from render layers management.

Then, every tick, we:

• Iterate the children of every node with UiLayerCamera to propagate the camera entity to children in a UiLayerCamera component. The 'render camera' will receive nodes during UI extraction, while the TargetCamera and DefaultUiCamera will continue to be used for layout calculations (and as fallbacks in case there's no UiLayerCamera on a node).
• Update the image and camera/camera-projection sizes after UI layout is finished.

World-space 2D UI

Currently, any entity tree that starts with a Node will be rendered with the UI rendering pipeline. If a root node has UiLayer, then the UiMaterial on that node will be rendered to a normal UI camera. However, for world-space UI we want a UI tree to render to a texture but for that texture to not be rendered in a UI camera (and instead get used by some mesh in the world).

To support this, we can use the WorldUiRoot marker component to 'disable' nodes that have UiLayer so it collects children on its UiMaterial without the material being rendered by the UI pipeline.

• During layout, if a node has WorldUiRoot then a layout computation will start at the WorldUiRoot component (enabling you to stick UI directly on an entity with a mesh for world-space UI (pending analysis of compatibility with GlobalTransform propagation)).
  • If the node's parent has Node then a warning is emitted.
• During rendering, if a node has WorldUiRoot then it will not be rendered.
• TargetCamera will propagate from WorldUiRoot entities, which is useful for layout.

Note that this is not a complete solution to world-space UI, since we don't have a way to incorporate world coordinates automatically. But it should be relatively easy to expand on this concept in follow-up PRs.

API

/// Camera used for rendering UI sub-trees to textures.
///
/// This component is constructed and managed automatically on nodes with [`UiLayer`].
#[derive(Component)]
pub struct UiLayerCamera
{
    pub(crate) camera: Camera,
    pub(crate) projection: OrthographicProjection,
    pub(crate) visible_entities: VisibleEntities,

    /// Inferred from the destination camera that will render our render target (by default? let it be user-defined?).
    pub(crate) anti_alias: UiAntiAlias,

    // Need this? Can just use the default. Or infer from destination camera?
    pub(crate) graph: CameraRenderGraph,
    // Need this? Default, or infer from destination camera? Let it be user-defined?
    pub(crate) texture_usages: CameraMainTextureUsages,
    // Need Frustum? Not used in UI render pipeline and don't need it for frustum culling in this use-case.
    //     But might need it deeper in the rendering code...?
    // Presumably `DebandDither` and `ToneMapping` aren't needed here. Or infer them from destination camera?

    /// Flag indicating if the camera should render anything. Set automatically in [`UiLayerMaterialPlugin<M>`]
    /// if `UiLayerMaterial::is_transparent` returns true.
    pub(crate) dont_render: bool,
}

/// Helper trait for setting up a [`UiLayer`].
///
/// This trait requires `Default` so we can make a new material if one isn't provided.
pub trait UiLayerMaterial: UiMaterial + Default
{
    fn set_texture(&mut self, texture: Handle<Image>);
    fn is_transparent(&self) -> bool;
}

/// The default [`UiMaterial`] used by [`UiLayer`].
#[derive(AsBindGroup, Default, Asset, TypePath, Debug, Clone)]
pub struct DefaultUiLayerMaterial
{
    /// Color tint for the texture.
    #[uniform(0)]
    pub color: LinearRgba,
    /// The texture where descendents of this node will be rendered.
    #[texture(1)]
    pub texture: Handle<Image>,
}

impl UiMaterial for DefaultUiLayerMaterial { }
impl Material for DefaultUiLayerMaterial { }

impl UiLayerMaterial for DefaultUiLayerMaterial
{
    fn set_texture(&mut self, texture: Handle<Image>)
    {
        self.texture = texture;
    }
    fn is_transparent(&self) -> bool
    {
        self.color.alpha <= 0.0;
    }
}

/// Wraps [`UiMaterialPlugin`] for [`UiLayerMaterial`].
pub struct UiLayerMaterialPlugin<M: UiLayerMaterial>;

impl<M: UiLayerMaterial> Plugin for UiLayerMaterialPlugin<M>
{
    fn build(&self, app: &mut App)
    {
        app.add_plugin::<UiMaterialPlugin<M>>();

        // See "Scheduling and logic"
    }
}

/// Component that overrides the default render behavior of [`UiMaterial`]
/// nodes to force the position and bounds of the material.
///
/// Without this component, `UiMaterial` defaults to rendering in the node 'background color' area.
///
/// Computed by [`UiLayerPlugin`] to ensure child camera contents are properly positioned in the parent camera.
#[derive(Component)]
pub struct UiMaterialLayout
{
    pub size: Vec2,
    pub offset: Vec2,
}

/// Caches the texture where a UI sub-tree will be rendered.
#[derive(Component, Default, Deref, DerefMut)]
pub struct UiLayerImage(pub Handle<Image>);

/// Sets up a [`UiLayerMaterial`] on the current node where all children of this node
/// will be rendered.
#[derive(Component, Default)]
#[requires(Handle<M>, UiLayerImage, UiMaterialLayout, Node)]
pub struct UiLayer<M = DefaultUiLayerMaterial>
where
    M: UiLayerMaterial
{
    /// Extra space in pixels to add to each edge of the texture.
    ///
    /// Increase this value if you have [`Outline`] or shadows on children of this `UiLayer` that are not
    /// displaying properly.
    ///
    /// The margin will be scaled using [`UiScale`] and the window's scale factor.
    ///
    /// This field exists because the layout algorithm only provides a content size derived from the structure
    /// of the content. It doesn't include margin effects like [`Outline`] or shadows, which will extend outside
    /// the content region. Since increasing texture size has a perf cost, we assume no margin is needed, and
    /// provide this field for users who require it.
    pub margin: f32,
    /// Additional scale factor applied to the texture contents.
    ///
    /// This allows you to create accurate 'zoom' effects for UI subtrees.
    ///
    /// Scaling takes place relative to the center-point of this node's area. (TODO: make it configurable?)
    pub scale: f32,
    // TODO: Rotation/translation?
    _p: PhantomData<M>,
}

impl<M: UiLayerMaterial> UiLayer<M>
{
    /// Makes a new `UiLayer` with [`DefaultUiLayerMaterial`].
    fn new(margin: f32, scale: f32) -> UiLayer<DefaultUiLayerMaterial> {
        Self{ margin, scale, _p: PhantomData }
    }

    /// Makes a new `UiLayer` with a custom [`UiLayerMaterial`].
    fn custom<C: UiLayerMaterial>(margin: f32, scale: f32) -> UiLayer<C> {
        Self{ margin, scale, _p: PhantomData }
    }

    /// Sets the margin field.
    fn with_margin(self, margin: f32) -> Self {
        self.margin = margin;
        self
    }

    /// Sets the scale field.
    fn with_scale(self, scale f32) -> Self {
        self.scale = scale;
        self
    }
}

/// Manages entities associated with [`UiLayers`](UiLayer).
pub struct UiLayerPlugin;

impl Plugin for UiLayerPlugin
{
    fn build(&self, app: &mut App)
    {
        // See "Scheduling and logic"
    }
}

/// Defines which camera the UI node should be rendered to.
///
/// Propagated automatically when using [`UiLayer`].
/// The component is ignored if it has not been marked changed this tick.
#[derive(Component)]
pub struct UiLayerCamera
{
    pub(crate) camera: Entity,
    /// Optimization that indicates if the entity needs to render anything.
    pub(crate) should_render: bool,
    /// Offset subtracted from UI element transforms so they fit in the render camera properly.
    pub(crate) render_offset: Vec2,

    /// `GlobalTransform` correction to apply to entities in world-space UI. (??? is this the right solution ???)
    pub(crate) transform_correction: Transform,
}

/// Marker component to indicate a node is the root of a world-space UI tree.
/// - The node will not be rendered by any UI camera.
/// - Layout calculations will start at this node even if it is a child of some non-UI entity.
///
/// # Usage
///
/// World-space UI trees will compute layout relative to the default UI camera, or to a `TargetCamera`
/// if you include that. However, it is recommended to use a fixed-size root node with clipping enabled
/// so the UI lines up with your destination mesh.
///
/// You can use the computed [`UiMaterialLayout`] component to see how large the UI texture is.
#[derive(Component)]
#[requires(UiLayerImage)]
pub struct WorldUiRoot;

/// Controls which camera a node will render on, overriding any `UiLayerCamera` that is present.
#[derive(Component)]
#[requires(Node)]
pub enum UiLayerCameraOverride
{
    /// Priority order:
    /// - TargetCamera
    /// - DefaultUiCamera
    ///
    /// Can be used to render a nested [`UiLayer`] on the root view. This would allow you to, for example,
    /// show part of a UI during a tutorial, and then fade-in other elements as you progress.
    TargetCamera,
    /// Renders the node to a user-defined camera.
    Camera{
        entity: Entity,
        /// Offset to subtract from the material's position in order to render it on the specified camera.
        offset: Vec2,
    },
}

Implementation

Comments

• UiLayerCameraOverride is a 'bonus' feature that is not required for an MVP and can be implemented in a follow-up PR.
• The UiLayer node is not rendered on its own UiLayer component because otherwise the node would contain content rendered on two separate cameras (i.e. its UI components on the sub-camera, and its UiLayerMaterial on a parent camera).

Changes to existing code

• Add content_size field to Node using taffy::Layout::content_size. See update_uinode_geometry_recursive.
• In UiPlugin, replace check_visibility<WithNode> with check_visibility<(WithNode, Without<Changed<UiLayerCamera>>, Without<UiRenderOverrideTarget>)>.
  • Constrain UiUpdate::PostLayout to run between VisibilityPropagate and CheckVisibility.
• Update extract_ui_material_nodes to incorporate an optional UiMaterialLayout component on nodes.
• Update TargetCamera propagation to start from both root nodes and WorldUiRoot nodes.

Tricks

• We refresh UiLayerCamera components every tick, and then only access them for rendering if they were changed this tick. This avoids complex hierarchy-management code. The perf cost is partially amortized by shifting some check_visibility work into this traversal. See Add PropagateOpacity for controlling the alpha of entire UI node trees #15206 for the origin of this trick.
• During UiLayerCamera propagation, we can avoid duplicate traversals by checking if a given 'starter' node with UiLayerCamera has a UiLayerCamera already (which was presumably inserted by an ancestor with UiLayerCamera in a previous tick). If it does, then cache the camera entity for a second-pass traversal once all entities with UiLayerCamera and without UiLayerCamera have been traversed (and remove the entity from the cache if we encounter it during traversal down from an ancestor).

Scheduling and logic

To make everything fit together, we require precise organization of logic in the schedule.

• Update (user code)
  • Users can add UiLayer components to their UI nodes.
  • Users can mutate UiLayerMaterial assets to create effects.
• PostUpdate
  • UiSystem::Prepare
    • Detect Added<UiLayer<M>> (system in UiLayerMaterialPlugin<M>)
      • Fixup the material.
        • If Handle<M> is default, add a new material asset.
        • If UiLayerImage is default, add a new image asset.
        • Set the texture in the material.
      • Insert a new UiLayerCamera with the material's texture as render target.
        • Start with default values for fields that need to mimic the destination camera (i.e. the camera where the material on this node will be rendered). We will refresh those values every tick in the propagation system.
    • Refresh UiLayerCamera::dont_render field (system in UiLayerMaterialPlugin<M>).
  • UiSystem::Layout
    • ui_layout_system
      • Take into account WorldUiRoot markers.
      • Use TargetCamera for layout.
      • In update_uinode_geometry_recursive extract the taffy::Layout::content_size value and save it in Node.
  • UiSystem::PostLayout
    • Fixup ui texture state throughout hierarchy.
      • Get the new texture size.
        • We don't know where node content extends out, so take (content_size - node_size) as an offset from the upper left node corner (for offsetting the camera) and (content_size - node_size)*2 + node_size as total size.
      • Update UiLayerCamera (need equivalent of camera_system)
        • Set camera.computed.target_info from the new texture size.
        • Update the camera projection from the new texture size.
        • Update camera.computed.clip_from_view from the projection.
        • UiAntiAlias and possibly other attributes need to be inferred from the destination camera, which need to be transitively inferred all the way up to the root TargetCamera/DefaultUiCamera.
      • Update the UiLayerImage asset's size.
      • Update the UiMaterialLayout component.
      • Propagate UiLayerCamera to children of UiLayer (including but stopping at children who also have UiLayer).
        • UiLayer::camera: Ancestor with UiLayer.
        • UiLayer::should_render: All nested render targets are not transparent (use cumulative UiLayerCamera::dont_render values), taking into account UiLayerCameraOverride.
        • UiLayer::render_offset: Offset between render camera origin and target camera origin (taking into account scale factors), taking into account UiLayerCameraOverride.
      • Collect VisibleEntities for each UiLayerCamera (equivalent of check_visibility).
        • Skip entities with UiLayerCameraOverride.
  • After CheckVisibility
    • For each entity with UiLayerCameraOverride, push entities into the VisibleEntities component on the correct override camera.
• ExtractSchedule
  • UI nodes
    • Ignore nodes with !UiLayerCamera::should_render && !Has<UiLayerCameraOverride>.
    • Ignore nodes with WorldUiRoot.
    • Use target camera for sizing.
    • Use render camera (or render camera override) for assigning render destination (in extracted ui node).
    • The clip values need to be offset correctly so the clip rectangle lines up on the render target.
  • UI material nodes
    • Use UiMaterialLayout to correctly position and size the material.
  • UiLayerCamera
    • Need equivalent of extract_cameras, use collected VisibleEntities from UiSystem::PostLayout.

Testing

• Does it work as expected when you render to nested render targets?
• Scaling: appearance and picking should be correct
  • Root camera with scaling factor.
  • UiScale
  • Window scaling factor
  • UiLayer::scale (potentially nested)
  • Transform scaling (does this affect UI?)
• Resizing window doesn't break anything.
• Children extending outside the node area of the UiLayer:
  • Children should be rendered above the node's border and outline.
  • If the children have an Outline, then using UiLayer::margin should show/hide the outline at the edges.
  • If the node (or an ancestor) has clipping, then children should not be rendered outside the node's CalculatedClip.
• World-space UI should work as expected.

Questions

• Is it possible to render to nested render targets?
  • Answer: Yes, tested nesting UiMaterial and got equivalent results with nesting UiImage.
• Is it fine to remove the camera entity from UiBatch and UiLayerSlicerBatch? They don't seem to be accessed anywhere. If the entity is needed then it's an open question what to do, since the render camera may point to a UiLayerCamera rather than Camera.
• It is not 100% clear what fields the UiLayerCamera needs to function properly (i.e. seemlessly, as if it didn't exist), nor the exact setup code to ensure scaling behaves correctly.
• Is that #[requires(Handle<M>, ...) valid syntax?
• How should picking be handled if UiLayer::scale is not 1.0?
• Is a custom fragment shader required for DefaultUiLayerMaterial?
  • Answer: Yes, see this test gist, it only works with a fragment shader.
• Are UiMaterials drawn in front of or behind the outline and border of a node?
  • Answer: It looks like the TransparentUi::sort_key is tied to render-world entity index. Since extract_ui_material_nodes is in RenderUiSystem::ExtractBackgrounds which is after RenderUiSystem::ExtractBorders, the borders will presumably be drawn on top. It maybe necessary to add a separate step for extracting UiLayer materials...
• Do materials properly respond when their referenced textures' assets are mutated? A discord user recently reported having a problem here.
  • Tentative solution: Every tick, remove the Image assets from the world, update them, and reinsert to get new asset ids and update cameras/materials with the new ids.
• Will there be hierarchy traversal issues if world-space UI is inserted as a child of a scene? For example, GlobalTransform would be affected.
  • Tentative solution: Can add UiLayerCamera::transform_offset to force-correct the transform in render extraction. Alternatively, we could directly mutate GlobalTransform in the UiLayer hierarchy traversal.
• Is it problematic to implement Material on DefaultUiMaterial? The use-case is world-space UI.
• How to avoid pixel rounding issues when rendering to a texture that scales itself?
  • Tentative solution: Instead of scaling the texture directly, apply the texture scale to the UI nodes when rendering as part of an 'accumulated scale factor'. Then let the texture auto-expand to cover the scaled UI while matching texture pixels to screen pixels like in the unscaled case.
• How to detect if multiple UiLayer components for different materials are inserted to a node?
• Should the initial implementation include a DefaultUiLayerStyle component that automatically sets up UiLayer for default material and uses change detection to send style info (like opacity, scale, etc.) to the material and UiLayer component?

Future work

• Change WorldUiNode to be an enum that controls LayoutContext for taffy and also orientation/occlusion:
  • WorldUiNode::Billboard: Uses default camera or TargetCamera; render on that camera but use GlobalTransform from a scene hierarchy to position it; 100% dimension = 100% of viewport. Replaces bevy_mod_billboard without depth culling.
  • WorldUiNode::BillboardObstructed: Uses default camera or TargetCamera; render on that camera but use GlobalTransform from a scene hierarchy to position it; 100% dimension = 100% of viewport. Replaces bevy_mod_billboard with depth culling.
  • WorldUiNode::Sized(fixed size): Assume there is a mesh on the node where the material will be rendered. 100% dimension = 100% of specified size.
  • It might be the case that this enum is required, in which case the entire concept can be moved to a future PR.
  • Open question: what if multiple cameras can see a billboard?

[eidloi]

Do we need the additional camera? I recon in the extract logic we can just stop iterating the children when we encounter a UiTexture?

[UkoeHB]

A few things:

• Right now extraction doesn't walk the hierarchy, it just iterates nodes. Replacing the iteration with a hierarchy walk may be worse perf for the normal case when not using UiTexture.
• IIRC @cart has been pushing camera-driven UI for a while. That might be from a user-land DX perspective though (not relevant here since the camera would be hidden).
• The render world seems to want an ExtractedCamera, although UiPassNode only uses it to get the viewport. I don't know if it's required deeper in the rendering pipeline.

[eidloi]

My main worry is the hot copy-pasta that is the current UI render pipeline that is still growing today. Each rendered aspect is extracted separately and abuse the same struct to fill the data we send to the GPU. So even if all we need is a color we send the texture handle and a bunch of other values we just ignore (to solve opacity: technically, we could just change the color alpha in the render app if we schedule a system after the extracts, but the render data does not link properly to the source entity). Not sure what the maintainer plan for this is, but adding a new feature on top of the current mess might be counter-productive. In a dream I plan to rewrite and sanitize it, but that's that.

I like the interface though it also solves the issue of custom UI materials not being extensible, however the complexity of managing the state through the hierarchy is a bit high for my tastes (especially if we consider that the pipeline rewrite would probably solve it without the need to manage the sub-trees). I might just need to spend more time to dive into this though so don't let me stop you :D

[UkoeHB]

Not sure what the maintainer plan for this is, but adding a new feature on top of the current mess might be counter-productive.

Yeah I'd like to know this too. Based on my investigation, implementing UiTexture in the 'minimal architectural changes' way would be a little messy and far-reaching. For example, UiMaterial is designed to go in the background of nodes, but we'd need it to be in the foreground for UiTexture (i.e. above any border/outline/image on the node, since that's where children are located), so there'd be some extra spaghetti to differentiate material extraction.

the complexity of managing the state through the hierarchy is a bit high for my tastes (especially if we consider that the pipeline rewrite would probably solve it without the need to manage the sub-trees)

Right now the idea is to relocate a bunch of visibility-checking logic for sub-trees into the hierarchy traversal. Every ExtractedCamera (or ExtractedView?) needs a VisibleEntities vector, which we need to collect every tick. Managing the UiRenderCamera component on top isn't too big a lift.

I might just need to spend more time to dive into this though so don't let me stop you :D

Haha please do, I'm kind of at my limit in how far I can design this, considering my lack of knowledge in the rendering department.

[villor]

Right now extraction doesn't walk the hierarchy, it just iterates nodes. Replacing the iteration with a hierarchy walk may be worse perf for the normal case when not using UiTexture.

If ghost nodes ends up with retained UiChildren/UiParent components on the Nodes, this could still be just iteration.

One idea is to add a UiLeaf-marker or something that marks an entity as the end of the current UI hierarchy. The systems that updates UiChildren/UiParent would take this into account to stop the traversal at that node.

[Arrekin]

My use case was to make miniatures that can be animated(scaled) back into full menus - this would be perfect!

[villor]

I like this! Especially the ergonomics it brings to world space UI. bevy_ui always felt a bit too separate from the world to me. I want to be able to mix UI and 2d/3d “seamlessly” in one hierarchy.

It might also be worth to consider how this overlaps with (or can have similar APIs to) the reverse, e.g. UI space 2D/3D:

• Put any 2D node under a UI node to have it relatively positioned from the node, while ensuring correct z-ordering
  • Usecase: 2d primitives in UI (Geometric primitives for UI #14187)
• Put a whole 3D scene under a UI node
  • Useful for things like inventories, character creation screens, previews etc

Both of these might be able use a similar approach to UITexture (CanvasNode?) where the 2d/3d scene is rendered to a texture and used in the UI.

[villor]

Also, it looks like we’ve got some overlap with ghost nodes here! We are both changing what a UI root node is. We should probably synchronize this a bit.

In the current state of the ghost nodes PR (#15341 ) I’ve defined root nodes as “Node without a parent, or Node with only GhostNode ancestors”.

I’m starting to question whether there should be a GhostNode marker at all, or if we should just consider any spatial nodes without a Node ghost nodes… 🤔