Dirt

I was making a 3' × 4' board for our Saga game next month and I wanted most of it to be dirt terrain. So I searched for what people use for "dirt." Well, it turns out people do not seem to use actual dirt. Really!? Well, being a rebel, I choose to. I dug up some clay from the yard, baked it for an hour at 450 F so as to dry it and kill the bacteria, and then sieved it on to the board coated with near full strength wood glue. It worked out well. (I then sort of ruined it with coats of clear acrylic unevenly applied. Damn.) So, my question is, why don't people use actual dirt for dirt terrain?