The dirt you use to go from place to place is the street. Back in the old days, all roads were made of dirt, not like the newfangled roads of today.
As you're walking down the street, staring at the dirt, a guy (がい) bumps into you, "Hey guy, watch where you're going!" As he says this he bumps into another guy. "Watch it guy, I'm walkin' in this street!" "Woah guy, this isn't your business, this is my street!"