Can Geometric Deep Learning lead eliminate the need of "Brute Force" pre-training [D]

I’ve been reading about Geometric Deep Learning lately (the whole grids, graphs, groups, manifolds idea), and something clicked that i wanted to get a clarity on, i don't think i'm an expert at GDL or anything mentioned here, so i can most definitely be wrong at a fundamental level as well, A lot of modern deep learning feels like we're throwing massive data and compute and we just hope the model learns the right invariances.