Olympiad programs are pure algorithms, which I agree are more math intensive than most "normal" programming done today (though I wouldn't dismiss math as "riddles"). But that's my whole point: most problem domains have some mathy parts, whether in the algorithms or from the problem domain. As we reduce the cognitive burden of plumbing and architecture that exists in programming today, these meaty parts will take up a larger portion of the programmers workload. The required skillset will shift somewhat, rather than rewarding the same mental juggling process at a larger scale.
Oh, but they are riddles. Hanoi tower, permutations, travelling salesman kind of problems. Algorithmically tricky (not hard, the problems are in textbooks for ages), but trivial from a programming perspective. The kind of problems that a physics major would think constitutes programming, but mostly irrelevant to complexity met in either the industry or CS academia.
I do some machining as a hobby; there are decades old teasers like turning a cube inside a hollow cube on a lathe from a piece of round stock. I feel like those are very much like Olympiad problems in nature, and just as remote from real-life machining exercises.