DE will work for potato bugs (roly polys) but I don't think it'll work with caterpillars. To apply DE, just dust it around on the plants, being careful not to inhale the dust.

If you don't mind chemical pesticides, rotenone is generated from certain plants and that's a dust you apply, it works I've used it before.