Insects got smaller because only smaller insects survived the less oxygen-rich environments after the late Carboniferous and early Permian periods. And keep in mind all these biological and environmental changes occurred in tandem over many tens of millions of years, and more importantly, about 200 million years before the appearance of modern day ants.
While increased oxygen levels would permit larger insects to survive, it would still take countless generations of sexual reproduction for those less prevalent genes to reappear, and then the insect would still need to survive other possible threats in the current world.
And because reproduction of Formicidae is so slow—with most queen ants mating only once and then living and releasing sexual offspring for decades in many cases—that is why it would eons for any genetic mutations or dormant genes to become expressed in ants, compared to other insects with much faster reproductive cycles and shorter adult life spans, like the beetles you mentioned.
Edited by drtrmiller, April 17 2019 - 1:34 PM.
