Experimental fast-csg feature: it’s time to render more complex models :-D#4087
Experimental fast-csg feature: it’s time to render more complex models :-D#4087t-paul merged 18 commits intoopenscad:masterfrom
Conversation
…tions of utils for it
…ron_3 Provides CSG operations that seamlessly switch the underlying representation to use Polygon Mesh Processing (PMP) functions (on a Surface_mesh) when possible, or (usually) slower Nef_Polyhedron_3 functions when needed.
|
Oh, sorry, there's a likely conflict incoming via #4086. |
|
@t-paul merged, thanks for the heads up ✌️ |
t-paul
left a comment
t-paul
left a comment
There was a problem hiding this comment.
This is great, the split into smaller changes has helped a lot to see what's going on. Thanks a lot for that.
I'll still want to go through the test case diffs, but I don't expect any issues so I'll flag this as approved already. We can merge once the build issue is resolved. I'm not sure what it still complains about with the windows build, but I hope you know 😀.
The LGTM error might be caused by a very old base system they are using. If that's the case, we can ignore that and maybe just drop this. So far it has not helped much but caused a huge 2h delay on PR responses. Maybe we can find some better solution (also with Semmle being part of Github now, the Code-QL check might be the same engine anyway).
|
For reference, @thehans did some work on gmp earlier which included some extension for compatibility with |
|
@t-paul Thanks for the review! The gmp build error is still a bit mysterious to me, but {good, bad} news: I've found a workaround, and it's just to stick to CGAL 5.2.4 (which seems easy to hack in the cross mxe build but I'm not so sure about the native msys2 build yet / I don't have an environment to test it on apart from the CI itself). I haven't managed to reduce a simple test case to file the bug to CGAL yet but working on it. I'll probably add a build failure in cgal.h when on Win64 and with CGAL >= 5.3, if that sounds reasonable. Othewise I can reinstate all the |
|
If you need a simple way to access the MXE environment and have a system with docker, you could poke at the MXE CI failure via: |
Here's how to quickly reproduce the compilation bug in CGAL: docker run -it openscad/mxe-x86_64-gui:latest # Get the OpenSCAD sources *before* this workaround git clone https://github.com/ochafik/openscad.git workspace && cd workspace && git checkout 3b16f30 # Only build the offending file to speed things up sed -i 's/target_sources(OpenSCAD PRIVATE .*/target_sources(OpenSCAD PRIVATE src\/cgalutils-triangulate.cc)/' CMakeLists.txt export NUMCPU=1 export LIB3MF_INCLUDEDIR=/mxe/usr/x86_64-w64-mingw32.static.posix/include/lib3mf export LIB3MF_LIBDIR=/mxe/usr/x86_64-w64-mingw32.static.posix/lib export SUFFIX="" export OPENSCAD_VERSION="" ./scripts/release-common.sh -snapshot -mingw64 -v "$OPENSCAD_VERSION"
|
@t-paul hah thanks, I tinkered a bit and have got a relatively fast way to reproduce the bug for the CGAL folks (in this commit message, will file it shortly). Added a dirty hack to install 5.2.4 in the openscad-mxe-64bit build so we can get fast-csg binaries for all, and a version guard to compile the crashing code out for now. Might need to disable some test cases on the "real" windows build depending on what it tries to run. |
|
MXE is back to 5.2.4 for now and I think I convinced MSYS2 to download and install the older CGAL version stashed on the file server (https://repo.msys2.org/mingw/mingw64/ still has some older packages, but I don't see how to specify that version via normal installation). |
t-paul
force-pushed
the
fast_csg_final2
branch
from
d6efaa8 to
641c40d
Compare
|
I lost track, is this enabled for Windows? |
|
Yes, we had to revert to CGAL 5.2.4 for Windows hoping to get a fix via CGAL/cgal#6308. |
|
Woop woop, thanks @t-paul ! |
|
On the plus side, we can patch 5.4 for the windows builds if we get something. So we don't need to wait for CGAL to actually release a fix version. |
|
@ochafik thanks for that very cool stuff. I guess we'll have to fight some corner cases still, but on some of my models the difference is just amazing. I have one assembly which is missing a part after F6 though 😀. |
|
@t-paul my pleasure! Hah, I'll prepare a PR with the helpers to help forward this kind of disappearance bugs directly to CGAL 😜 |
|
Oh, note that there's a huge reformat is incoming later today. Finally running |
|
Blogged about this and related efforts here (includes summary of what this does and why). |
|
Fantastic work @ochafik et al, this speedup is exciting and very much needed! Sorry for cross-posting, I just noticed that in the blogpost you explicitly want feedback on this merged PR instead of twitter or HN comments, so here it comes:
Happy to open a new separate issue and point to this PR if that's what you really meant in the blogpost. Also, there were prior rendering issues with this model against master, so hiccups might be tangentially related to issue #4022 but not necessarily. /cc @markmaker |
|
I just tried this on the model I am currently working on. Old time: 0:04:50.025. New time (fast-csg without fast-csg-trust-corefinement): 0:00:00.161. New time (fast-csg with fast-csg-trust-corefinement): 0:00:00.156. It is extremely fast, but it revealed problems with non-manifold parts of my model. The interesting thing is that due to slow performance without this feature, what I was doing was to have a top-level string field identifying parts by name and I would have a switch in the code and render parts conditionally if the name was set to a specific part and before printing, I would go in and render the parts individually and pull them into the slicer one by one. With this new feature I no longer need to do this, I can render the whole model at once. However due to the non-manifoldness issues, my render is completely missing some parts. But when I return to the segmented rendering (without changing the geometry of the individual parts in any way), the parts that are missing in the complete render still render when rendered individually in isolation. Not a huge deal, I will fix my geometry and make sure all files are manifold, so it's basically a feature :) But I thought I'd share in case others see the same behavior. |
|
@brainstorm thanks for reporting this! The hang comes from the way corefinement functions generate more faces than Nefs do, so there's a need to simplify the meshes afterwards. There are two ways we could do this:
Good to know! |
|
@TomasHubelbauer thanks for reporting back! Please let me know if your non-manifoldness issues persist, it could be we need to help models close up again somehow. |
|
I have to steal @brainstorm's words: Fantastic work @ochafik et al, this speedup is exciting and very much needed! I just tried using this with rsheldiii/KeyV2 with some customizations and workarounds for special variables that tends to be very taxing for OpenSCAD to render(especially at scale). First test was a relatively simple set of 5 keys with symbols ("{ [" "| " ": ;" "? /" "_ -") . The second test was with a full-size 104-key layout with symbols. Computer: Here is how things went with different options and cache's flushed and files reloaded: 2021 (Full) - 3:33:21.817
|
|
Modeling a steam locomotive, well and good until the boiler/smokebox, which took 19 minutes to render. Did some googling, came across @ochafik's blog post, so I downloaded the nightly, enabled fast-csg and fast-csg-trust-corefinement and re-rendered: 2.51 seconds. But, I got four instances of this: Not surprised, the boiler is a concatenation of hollowed cylinders, with one segment being a canted cone segment just pushed into the adjacent segments, actually surprised it worked in the old render. The model was simple until I added rivets, which are cones embedded in the walls. Here's a view render: I'm a programmer, but I haven't done mesh graphics in 40 years, so not much technical help. Thought you might want a report on a model done without much thought to mesh integrity... Edit: Read the reddit thread, went back and enabled fast-csg-exact and fast-csg-exact-callbacks. Render time dropped just a few seconds: 2:48. |
|
Thanks @jonathan-dove and @butcherg for trying things out and reporting back! @butcherg I'm preparing a |
|
Sorry for the delay... Here's the script: |
|
So, I've got today's nightly build and ran it normally for a benchmark (101.1 seconds for F6), then tried and failed to find |
|
Command line parameters are normally not effective for GUI use. The experimental features are listed in |
|
Ah, thank you; not sure how I missed the Features menu. That worked perfectly and I've now done a proper benchmark against my Manifold library. Also, excellent work on fast_csg! https://elalish.blogspot.com/2022/03/manifold-performance.html |
|
This is a great improvement and most needed! I've been working on a project to build an open-source trackball for some time, and I modeled the enclosure in OpenSCAD. As the model got more complex, I saw rendering times measured in tens of minutes, which is pretty painful when trying to iterate on the design. These changes are a huge improvement, but I'm still saddened by the way the render pegs just one of the 20 cores in my machine for as long as it takes, so I'll be very excited to see any potential parallelism teased out of the algorithms for this. (I normally use the MacOS build of OpenSCAD, and I just upgraded from an ancient Mac Pro to a new M1 Ultra Mac Studio, so there's a lot of headroom there. ;) ) I suspect I'm doing lots of things wrong with this model (I basically taught myself OpenSCAD while creating it), but it might be a useful torture-test anyway. It currently throws a lot of warnings like this while rendering: The model is in my project's GitHub repository. If you clone that repo and open the file I've spent 30 years as a professional software developer, and I've done a fair bit of work over the years on threading and concurrency, so if you start digging into multithreading this code I'd love to lend a hand where possible. :) |
9 participants
This is a cleaned up, minimal version of fast-csg (incubated in #3641), which greatly speeds up a wide range of models when the feature is enabled (in the GUI’s settings or with the
--feature=fast-csgcommand-line flag), e.g. these tiny benchmarks. Fixes #2360.I've tried to itemize it in reasonable commits, but happy to send them as separate PRs instead.
Things I've left out for now:
fast-csg-exact-callbacksfeature (slows down some models but fixes some rare dramatically pathological cases like this one so quite worth the hassle). Coming soon in another PR. Before then, expect some slow corner cases.CGAL::Cartesian<CGAL::Gmpq>kernel in the hybrid polyhedron as in the current nef poly, which was fenced at compile time byFAST_CSG_SAME_KERNEL. While this produces a smaller executable and is functionally stable, it isn’t ready for prime time yet (much slower than using the different, lazyCGAL::Epeckkernel, and sometimes much slower than without fast-csg; Will need to try again to switch all our code to usingEpeckinstead, or maybe some better kernel - e.g. with filtering but not lazy?).FAST_CSG_DEBUG_SERIALIZE_COREFINEMENT_OPERANDSto help create self-contained bugs for CGAL, orSCOPED_PERFORMANCE_TIMERto debug perf)Some general notes: