It is the compiler's responsibility to decide what reasonable padding (assuming typical access patterns) is. The compiler does know a whole lot more about your machine than you'll ever will. Besides, your machine will be with you a couple of years; the program will be around for decades, running on a wide range of platforms, subject to a mind boggling variety of usage patterns. What is the best for today's i7 could very well be the worst for tomorrow's i8 or ARMv11.
Obfuscating code in pursuit of elusive "performance" falls squarely into premature optimization. Always remember that your time (writing, testing, debugging, understanding again after a week's time, on tweaked code) is much, much more expensive than the possibly wasted computer time (unless said code is run thousands of times a day on millions of machines, that is). Code tweaking makes no sense at all until you have hard facts showing that the performance isn't enough, and measurements telling you that shuffling that structure around is a bottleneck worth worrying about.