There are hot peppers hiding in this dirt, so you build a wall to protect yourself from them. Have you ever stepped on a really hot pepper? The oils bleed into your shoes and burn your feet! You don't want anyone stepping on these so you wall them up.
On the other side of this wall there are much less threatening cobs (かべ) of corn. Corn isn't spicy. It's delicious and sweet, so it's free from being walled in.