I would like to shed some light on this topic as well. This is from my experiences working with these materials.
Plaster: easy to work with, moderate strength, fast water absorbtion, fast cast and cure time, smooth to touch, can get moldy easily.
Sanded Grout: strong and durable, slow-steady water absorbtion, can pool water, 24-48hr cure time, can be gritty to the touch, good for complex moulds, mostly drill friendly, very mold resistant, sand might degrade a tiny bit over time.
Unsanded Grout: moderately strong and durable, slow-moderate water absorbtion, 24-48hr cure time, smooth to touch, caution needed with drill, can be used for complex moulds, care needed working around narrow width edges, mold resistant, little to no degrading.
K23-Firebrick: easy to work with, no cast or cure needed, very dusty, strong and durable, very water absorbent, light weight(much more than the ones above), much like y-tong but softer, can be carved and cut with basic tools, mold resistant, chaulky to the touch, no water degrading but certain ant species might be able to chew it(not tested).
These are the only materials I have worked with. I find myself using primarily the sanded and unsanded grout although it has a slow cure time, but i’ve had the best results with it. Now my favorite to work with was the K23-firebrick for building quick modular nest. If you have some grinding bits for a drill then it’s a piece of cake. It just gives me a bit of anxiety that the ants might be able to chew it, however I think you can mix a small amount of grout and paint the inside parts of the nest with it to prevent potential chewing. I have not tried concrete because it might repel water, I think the biggest issue with all the materials, is which ones provide the best humidity control. IMO I think the firebrick evaporates much faster than grout and plaster, so I recommend painting or sealing the outside. And also I’m not talking about the concrete looking firebrick, it’s the ceramic kind. This is all I have to add really, hope I was able to give some useful feedback.