For more information look here, but some highlights include-
1. Caterpillars can be venomous
2. There is no antivenom (though so far no known deaths in the US are due to caterpillars)
3. Caterpillar spines can get embedded in your tongue/mouth/esophagus on down, requiring sedation/scopes to remove.
So when you see one of those oh-so-appetizing little guys, just say no.
PS don't let your children eat them either.