Customising that mesh, however, is not a simple task. For example, adjusting the radius in one portion of the object requires individually tweaking the vertices and edges of each affected triangle. With complex models comprising thousands of triangles, customisation becomes daunting and time consuming. Traditional techniques to convert triangle meshes back into shapes don’t scale well to complex models or work accurately on low-resolution, noisy files.
Now, researchers from the Massachusetts Institute of Technology (MIT) have devised a system that applies a technique called ‘program synthesis’ to break down CAD models into their primitive shapes, such as spheres and cuboids.
Rather than build a CAD model as a designer would, by assembling individual shapes into the final object; program synthesis does the reverse, disassembling the CAD model into individual shapes that can be edited. As input, the system takes a 3D triangle mesh and first determines the individual shapes that make it up. Program synthesis then analyses the shapes, and how they were put together and assembled into the final model. In doing so, it breaks down the mesh into a tree of nodes that represent the primitive shapes and other nodes detailing the steps for how those shapes fit together. The final shapes contain editable parameters for users to tweak that can be re-uploaded to the mesh.
Foundational shapes
The researchers built a dataset of 50 3D CAD models of varying complexity. In experiments, the researchers demonstrated their system could reverse engineer CAD files composed of up to 100 primitive shapes. Simpler models can be broken down in around a minute. While run times can be quick, the key advantage of the system is its ability to distil very complex models into simple, foundational shapes, the researchers say.
“At a high level, the problem is reverse engineering a triangle mesh into a simple tree,” says Tao Du, a PhD student in the Computational Fabrication group of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). “Ideally, if you want to customise an object, it would be best to have access to the original shapes — what their dimensions are and how they’re combined. But once you combine everything into a triangle mesh, you have nothing but a list of triangles to work with, and that information is lost. Once we recover the metadata, it’s easier for other people to modify designs.”
The process could be useful in manufacturing or when combined with 3D printing software, Du adds. This is especially important in the age of design sharing, where amateur 3D printer users upload 3D print models to websites for online communities to download and modify. Uploads are mostly triangle meshes, because meshes are far more universally accepted across platforms than the original CSG-based CAD files.
“We have tons of mesh models, but comparatively few CAD files behind them,” Du says. “If users want to reproduce the design at home and customise it a little, then this technique could be useful.”
Program synthesis automatically finds candidate computer programs given a specific ‘grammar’, meaning the structure it must work within, such as trees, and mathematical specifications. Using those constraints, program synthesis works its way back and fills in the blanks to construct an algorithm that satisfies those specifications, given new input. The technique is used, for example, for simple components of software engineering.
In the researchers’ work, the grammar is CSG, represented as trees. Each final node (with no branching nodes) represents a primitive shape with clearly defined parameters, and intermediate nodes represent basic ways the shapes converge and relate.
Trees and triangles
The researchers developed a method that lets program synthesis scan an entire 3D mesh and, essentially, think of each possible CSG tree it could create as a new candidate program.
After the system receives an input mesh, a pre-processing step detects the possible locations, orientations, and parameters of all primitive shapes. This process creates a massive point cloud across the surface of the triangle mesh. A special ‘primitive-detection’ algorithm infers from these points the dimensions for each primitive shape that makes up the mesh.
The researchers then sample many points in the entire 3D space and flag them as either inside or outside the mesh. This helps determine how the shapes converge or relate to one another. A simple example is a mesh consisting of two spheresb (A and B) merged together. If one sampled point falls inside sphere A, one inside sphere B, and one at the intersection of the two (inside both A and B), it’s most likely a union of the two shapes.
Given this information, along with the primitive dimensions, program synthesis could potentially create a CGS tree. But, 3D meshes of even low complexity would require program synthesis to sample tens of thousands of points. This would create a massive search space that’s computationally impractical to handle. “Directly feeding all the samples will choke the program synthesiser,” Du says.
To ensure the system worked efficiently, the researchers designed a sampling method that creates small subsets of point samples across the 3D space, which is much easier for program synthesis to compute. By sampling these subsets, it creates a new candidate ‘program’, or CGS tree, that could be considered correct. After numerous iterations — and using techniques to eliminate certain points and trees — the system lands on the correct CGS tree for each shape, with correct intermediate steps and final parameters. Any edited shapes are fed back into the mesh as the system computationally follows the intermediate steps back to the final object.
Currently, the system only handles four primitive shapes — spheres, cylinders, cuboids, and tori (doughnut shapes). Next, the researchers aim to increase the complexity of CSG grammar to handle far more shapes and modifiers.
