Modules/Sculpt-Paint-Texture/Future Projects

Sculpt, Paint & Texture: Future Projects

(This page is still heavily under construction)

Here we list inactive or future projects & opportunities for the module before they become active projects on the Workboard. Additional docs can be linked for expanded info.


Brush Engine Refactor

This is one of the biggest blocking projects for any new feature and code maintainance. We should at some point soon focus on incrementally refactoring the entire brush and sculpting code.

Brush properties and various improvements are planned, especially for the goal to cherry-pick settings for brush assets. Could be done separately to the underlying refactor, but not wise to do so.

Sculpt-dev was used for prototyping and working out the design for the intended brush properties and unified settings. Since these mainly rely on RNA this would be doable to implement without causing to many breaking changes. But we still need to review the current implementation and design.

Moving forward we need to define a high level design first and then break them down into smaller goals.

High level design = Divide current code into logical blocks and how they need to be used. A global architecture that each tool/brush can access. How we handle the stroke, tool and resulting data. Brecht, Jeroen, Bastien and Joe have insights on this and will start to document their knowledge over 2023. Afterwards this can get a workshop and scheduled priority.

Port all painting improvements to Image Editor

With the current Paint Mode project and recent sculpt mode additions, it will become increasingly necessary to provide the same feature set to the image editor. This also relates to D5697

VDM Multires

Targets for this project are still being collected. This is a full refactor of the Multires system and how the displacement data is stored. The goal is to:

  • Make the code easier to maintain and support features
  • Fix stability issues
  • Make the displacement data more generic and easier to manipulate & access
  • Provide support for all existing features, including geometry nodes modifiers (T104262) and node group operators (T101778)
  • Fix multires baking issues

More tasks on current design topics are T84864, T93165, T88381, T81989, T80348 & T73047

More features could also be provided:

Sculpting Layers & Shape Keys

The goal is to implement a layer system for sculpting subdivision levels.

A temp doc assembled with Daniel Bystedt to collect some initial ideas:

Another current limitation that could be resolved is for transfering shape keys. Currently it reads the entire topology, which makes the oepration fail if the vertex count is off. If it only reads vertex IDs, then partial meshes could be joined as shape keys. As an example: Duplicating an arm, making changes on the arm and transferring the changes back onto the entire body.

Using shape keys on alembic caches is also an important use case. Supporting shapekeys as some sort of modifier or geo nodes could be beneficial.

Resolution Independent Sculpting

We discussed resolution independent smoothing (D9385). This is potentially relying on the upcoming dyntopo refactor. This could be a great thing to add for easier “resolution independent sculpting”.

A resolution picker could also replace the need of the Multires modifier in certain workflows by defining a resolution target instead of switching levels.

Notes from a meeting: The smooth code in D9385 reads the same vertex coordinates it's writing to. This makes it slightly non-deterministic. We can fix by having it write to a separate buffer. This would actually speed things up quite a bit, though it'd be too much work at the moment.

Related to think about: - This is somewhat in conflict with a remesh detail size. How would the user have intuitive control over both remesh detail and sculpting detail?

Mesh target & shading options

Partly based on D8925 The goal is to intuitively be able to deform modifiers with a preview, similar to edit mode.

Especially with deformation & subdiv modifiers this can give the additional option to either sculpt on the undeformed base or a close approximation of the deformed mesh.

This could be combined with the feature of choosing between optimized viewport shading or acurate shading. (Related to T103889 & T90598)

For an optimised performance there should also be testing or options for live mesh updates or only on confirming a stroke. (Related to T103871 and T101150)

Just like with Edit Mode it would be very beneficial to be able to sculpt on a live mirror modifier results and have the stroke be correctly applied on the original base geoemtry.

Other related tasks:

This might also relate to how EEVEE is being used and optimized for sculpt mode (D9522).

VDM Brushes

Based on #104570 There are various features that could improve the use of VDM brushes:

  • Support the "Tip Roundness" setting for the Draw brush. This will allow to use the whole square texture space (A workaround is currently to scale the texture to be smaller)
  • Discuss how and if we should support "View Plane" mapping
  • Test implementation of Brush.use_persistent and sculpt.set_persistent_base() for the Draw brush to prevent artifacts from overlapping VDM strokes.
  • Implement an additional method of mapping and applying the texture via smoothed surface normals instead of the area plane. This will help in the case of using textures to apply repeated patterns with a large brush radius. How this can be implemented still needs to be defined.
  • Implement vector displacement with "Bake from Multires". This needs additional research though, as the baking results are imprecise, just like "Normal" and "Displacement" baking
  • Add Dynamic Topology support. In theory, this is then meant to remesh the surface after the stroke is applied, and then correct any vector displacement inaccuracies between the initial vector displacement and the new topology. Long term this is should be the ideal way to use vector displacement brushes.
  • Prototype a way to choose a displacement direction via the stroke (Like the grab/pose brush). Would this lead to usable results?

Snapshots

Based on D6594 prototype. The goal is to store a mesh temporarily as a sculpting layer or shrinkwrap target for a Morph Brush. This mesh could be saved in a clipboard independent of the undo stack.

Attribute Conversion

Converting Masks or Face Sets to other named attributes is commonly requested. These are some use cases:

  • Saving and loading masks or face sets for later use
  • Using Face Sets in geometry nodes as an Face Integer Attribute
  • Converting Face Sets to Face Corner Color Attributes
    • for visibility outside of sculpt mode
    • to be exported as ID Maps

Paint & Sculpt at the same time

Adding support to every sculpt mode brush to also paint on the same stroke. Very useful for efficient concept sculpting. This was already prototyped a couple years ago and works as expected. Foreseeable to do's are:

  • A toggle to enable painting in the brush settings (Or for every brush where this makes sense)
  • (Unclear) A scene toggle to enable painting for all sculpting brushes
  • Properly implementing this with the canvas picker UI
  • Properly exposing the color picker in the header and sidebar for every brush that supports this

More issues might be part of this. So this feature needs more investigation and dev time.

Draw Materials Brush

There's a need for faster material slot assignment on complex and high density meshes. This could be done with a variant of the 'Draw Face Sets' brush, that draws new materials on faces or extends existing ones.

This might require improvements to the behavior of these brushes:

  • Use the Sampling shortcut on S to sample a face set (current face set shown in the header)
  • Using the brush always draws the sampled face set by default
  • Hold Ctrl instead to draw a new unique face set

This would allow for easier painting of unconnected face sets. It would also result in far less holding of the Ctrl key.

Geometry Nodes Brushes

Partly based on D8377 and D9380. Geometry nodes could be used to simulate unique tip shapes and insert various meshes.

Plane Trim Brush

Based on D9571

Auto Retopology

The goal is to improve the automatic results of the quad remesher to immediately convert a sculpt into a multires modifier

Sculpt Mode Performance

Improving the general performance, based on T81180 and D8983.

Performance Warnings & Scheduler

Preventing freezes and crashes from certain tools and operations by either warning the user in the UI based on critical mesh statuses, or giving the user the ability to cancel the operation on long waiting times.

Texture Filters

Based on D6156 We should investigate methods to quickly and roughly insert texturing details via Filters.

Voxel Remesher Improvements

  • Better re-projection of Colors & surface details
  • Performance improvements, espeically for Fix Poles
  • Remeshing individual mesh islands without merging them
  • Remeshing selective mesh islands
  • Checking for holes in the mesh and filling them (D6111)

A wider idea is to combine the concepts of voxel remeshign and dyntopo into simply Adaptive Sculpting. This way we can re-brand the remeshers into "Volume" and "Surface" remeshers with the same shortcuts and detail settings.

The overlay could also be improved by always showing the size (D8069) or even a way to show the wires on the mesh itself instead of via the bounding box (This often causes issues where the overlay is behind the camera).

Another way could be to show the wire overlay only within the brush cursor, similar to the dyntopo density overlay and the Density brush for curve sculpt mode.

Trim Tools Improvements

The current Trim and Project tools are not performant or useful enough. But doing proper changes would need heavy reworking of the tools.

A better way of approaching the trim tools is to integrate them with the voxel remesher. This would need to be less precise than exact boolean operations and could vastly increase performance. Implement a new remesher option for extreme performance boost. Use the bool operator on a lower remeshed version and then project the higher one on that.

Project Tools Improvements

Project tools could fair geometry instead of just snapping them . Assigning them a face set like the trim tools is also a good addition.

Wrap up Gesture Tools

Add missing Line, Lasso and Polyline versions. Investigate a way to combine them into one tool. Also add them as operators to the menus.

Filter UX

Filters provide very poor visual feedback currently and controlling the strength is difficult. Gizmos based on the generic pose slider could be used here.

Modal IP mask filters from the sculpt could also be implemented if they are performant enough on high res meshes.

Transform Tools Improvements

We’ll make a task for setting the sculpt-session pivot location via a shortcut and a tool settings. Also we need to make it possible to change the orientation of the sculpt session pivot. This is closely tied to D9540

The general idea is to use the shortcut Shift RC to set the pivot, just like setting the 3D cursor in other modes. The behavior for setting, snapping and transforming the pivot should be the same. When using the transform operations, the sculpt-session pivot is transformed as well.

A toggle will also be added to the transform tools tool settings to only transform the pivot.

We could also consider adding a popover/dropdown in the header to change the behavior of the sculpt-session pivot: - Automatic Pivot (Expand operations change the position and rotation of the pivot, in addition to manual placement) - Manual Pivot (Only the user is able to place the pivot) - Global Pivot (An optional feature to keep the pivot location & rotation consistent across multiple objects. This could also be implemented by using and transforming the 3D Cursor together with the pivot)

Automatic Sample Bias

The sample bias for brush textures needs to be manually set. This can be easily automated by checking the value of the image borders and matching the sample bias.

More Dyntopo Support

Add more support to operators. Also dyntopo remeshing support for brushes like Grab, Pose, Smooth and Draw Sharp. Revamp or remove Simplify Brush.

Kelvinlet Tools

Fix Elastic implementation to properly preserve volume. Based on Kelvinlets paper

Improved Xray

  • Option to exclude active object from Xray transparency
  • Option to highlight Fresnel for easier reading of shapes
  • Support for mask & face set overlay

Auto-Masking polishing

Use geodesic falloff instead of topology for boundary propagation steps. This would avoid artifacts.

We could also investigate settings to propagate only inwards or outwards of the auto-masked face set.

Incremental PBVH leaf nodes updates for brushes and operators

This could be a focus real soon as it can improve performance in various areas and fix bugs. For this we need to improve how we handle PBVH tree leaves. They need to be subdivided more. Then the actual geodesic distances could be used for checking falloffs along surfaces and making sculpting tools more aware of the mesh volume. In theory it’s possible, but needs investigation. Joe is interested.

This would have various benefits. It would help avoid sculpting on disconnected geometry within the brush radius on any brush (Not just Grab). Kelvinlets elastic brushes could be implemented more accurately by properly taking the volume of the mesh into account. Geodesic expand performance would also be heavily improved.

Color Expand/Gradients

We should add the current operator to the menus to make it more accessible and easier to assign a shortcut. We will need to soon add a “Paint” menu in sculpt mode too, especially once more operators are added.

Further research and prototyping needs to go into a Gradient Tool for expanding gradients across surfaces, space and the screen.

Texture Data Block UX

This needs some investigation. The data block is hard to work with and outdated.

Some examples:

Color Palette Overhaul

Various improvements were suggested for UI and how the data block works:

  • Each color slot is a saved color ID with RGB values, that can be accessed through nodes. This way complex node setups or pixel coordinates of textures can be avoided as a color palette substitute for materials & procedural systems. Instead the color palette data block is usable.
  • Drag and drop swaps colors instead of replacing them
  • Attach the color palette directly to the color picker UI (For popups)

Store Hue of Colors

Currently colors are stored as RGB values. This makes it impossible to change the Hue of a color that has 0 Saturation. (A case of sweeping away your footsteps as you move forward) We could store the Hue of a color, even if it's just in the current session.

Overlay Visibility

In any painting modes and sculpt mode some overlays are automatically hidden. While this was useful since the Origin point and the 3D Cursor or rarely if ever used, some users find them helpful. Adding support for these overlays could be implemented.

Questions:

  • Do we need to add the Cursor Tool to these modes then?
  • Should there still be an option to automatically hide them for certain modes?
  • Can this be customizable per mode?
  • Should this instead be tied to custom shading modes or custom workspaces?

Dynamic Overlays

Useful for painting to avoid clashing colors and values for mask and face set overlays. Needs more info.

Brush Behavior Overlays

Some brush settings can be very hard to understand, especially how the settings affect the brush. Additional overlays could be enabled to debug the brushes for authoring and editing their settings. Based on D6324

Geodesic falloff for Grab brush

This was already prototyped in the sculpt-dev branch. Could be implemented as a brush option. Based on this implementation D10068

Pressure sensitivity support for Anchored brushes

Based on D6603

Sculpt-dev Branch Features

This is a rough list of experimental features from 'sculpt-dev' branch of Blender. These are features and prototypes in various states of completion.

  • Draw Sharp brush: Sharp Mode (Simple / Plane)
  • Smooth brush: Deformatiom type (Directional / Uniform weights)
  • Enhance Details brush
  • Pose brush: Deformation type (Bend)
  • Boundary brush: Deformation type (Circle) (Not working in latest branch)
  • Twist brush
  • Cloth brush:
    • Deformation type (Elastic Drag)
    • Bending checkbox + Bending Stiffness slider
  • Fairing brush
  • Scene Project brush
  • Symmetrize tool
  • Array brush (Could be used as a
  • Sharpen Color Boundary brush
  • UV Smooth tool
  • Displacement Heal tool
  • Box / Lasso Project tools
  • Mask Select tool
  • Mesh Filter: Some filter types have additional options:
    • Smooth: (Weighted / Hard Edge Mode / Preserve Face Sets / Bevel Radius / Bevel Smoothness)
    • Sphere: (Sphere Center / Weighted)
    • Surface Smooth: (Weighted)
    • Sharpen: (Weighted)
  • Cloth Filter: Bending checkbox + Bending Stiffness slider
  • IPMask filter
  • Color Filter: Filter type (Random HUE / Random Saturation / Random Value)
  • Edit Face Set: modes (Fill component / Extrude / All Tangency)
  • Transform Tools (Move/Scale/Rotate): Deformation Target (Geometry / Cloth Simulation)
  • Brush Settings: (Hard Edge Mode / Topology Rake / Topology Rake Mode)
  • Brush Stroke Setting: (Smooth Raking)
  • Shift Smooth separate from the Smooth brush, with its options available in the tool settings/topbar (Options panel)
  • Reset All Brushes operator (bpy.ops.sculpt.reset_brushes())
  • Additional global options in the "Options" panel (Show Pivot / Ignore UVs / Set Limit Surface)
  • Initialize Face Sets: (By Face Maps)
Retrieved from "https://wiki.blender.org/w/index.php?title=Modules/Sculpt-Paint-Texture/Future_Projects&oldid=36924"