I just took a look at the source. The best way to do a refactor would be to change all the processing related to mask (mask_paint/smooth, undo, pbvh etc.) to act generally on customdata. Introduce Weight brush and Vertex color brush and then there are some issues with viewport drawing code which should be resolved.
Its a big project, however I might pursue it a bit just to see how it works.
Its a nice thing that Nicolas commented his code, a big difference to other parts of Blender where inline comments amount to:
/* a quick hack, will do for now */
:), i respect Blender developers but sometimes understanding the code amounts to running it through debugger and backtracking. I’ve done my share of reverse engineering, but it really shouldn’t be that way with open source projects.
I think there could be a lot ways to optimize Dyntopo. Its silly that javascript app has about the same performance, even doing a more complex calculations (enable Adaptive).
Btw. Stephane told me that he wouldn’t mind reusing his code for Blender. I don’t know if he changed his mind about this in the meantime. It may be worthwhile to check out the logic even though it may not be directly applicable. There is also c version of the app (a bit obsolete though).
BTW. what do you think Anthony about the merge of the sculpt/weight/vcolor as mentioned in the thread?
EDIT: here is sculptGL with 1mil+ triangles, still running fine
I think what you’re trying to argue for mostly is redesigning the vertex paint system to take inspiration from Ptex, which in a nutshell is like multires for vertex painting (due to how simply letting vertex paint work with multires-created geometry would basically give you a crippled version of what the library does anyway).
Perhaps Blender will still get Ptex at one point in the future, I know that OpenSubDiv is being integrated which will ease the effort for any such implementation (due to how they’re designed to work together).
Since you seem to understand the code more than most, I pose this question:
How hard would it be to code multi-function brushes that both add color data and displace the mesh at the same time?
Basically a sculpt brush tool that paints verts too. The brush could affect multiple color groups depending on what sort of material is used. So for example if you want to sculpt blue veins on your character you could do it in single strokes rather than in separate steps.
A Texture Paint version of this would be great too! Paint on a spatter of moles on your creature, so the brush adds pixel data to the bump map channel as well as the color channel (and any other sort of channel like glossy or whatever the material calls for). You could have brushes specifically for paint-carving scars onto skin, wrinkles, anything! Clearly this would require complete integration between all of the paint and sculpt features, and linking would have to be possible between Cycles material data and brush preset data. And maybe a new file type for brushes could be made which would just be a stack of layered PNGs with associated material data coded in.
Its most likely that for this kind of change (which I support, and think thats the way it should be) it would be easier to start from scratch than trying to fit years of segmented development and disjoint features into one uber-feature.
Its also funny that (sans Sculpting) this was exactly what Painter3d did 15 years ago.
Here is something I did in 1999 painted in 3d with all surface attributes at the same time.
Interesting! I was sort of expecting people to tell me some reason that the idea wouldn’t work. But it seems the biggest issue is the complete code rewrite. And that is pretty impressive for 15 years ago.
