Yajirobe and Korin are pretty closely connected. The former has been living in the latter's Tower of the same name since late Dragon Ball. Furthermore, while Korin cultivates Senzu Beans, he often tasks Yajirobe with delivering them to the Z Fighters. This is all fine for plot purposes, but it begs the question of what Yajirobe is doing in Korin Tower in the first place.
In reality, Yajirobe has no earthly business staying with Korin. There are only Senzu Beans to eat, and the two of them could easily get on each other's nerves with their snarky attitudes. Korin Tower doesn't even have a TV, according to Yajirobe. It's possible that a look back at the history of these two will better contextualize their relationship and explain what's kept them together all these years.
Korin first met Yajirobe during the King Piccolo Saga of the original Dragon Ball anime. At the time, he was carrying Goku up Korin Tower following his defeat at the hands of King Piccolo. The deal was that if Yajirobe did Goku this favor, he'd be treated to all the food he wanted.
Upon arriving at the top of the Tower, Korin offered Goku a Senzu Bean to recover his stamina. Yajirobe, disappointed by what looked like a meager offering of food, swiped the pot of Beans out of Korin's paws and stuffed handfuls of them into his mouth. The greedy samurai, unaware that one bean was enough to fill a man for 10 days, found himself rolling on the ground, bloated and in pain.
Yajirobe would take Goku up Korin's Tower once more after the latter had defeated King Piccolo. From here, Goku was advised to go visit Kami, the Guardian of Earth, so he could repair the destroyed Dragon Balls. This was around the time Yajirobe became a regular resident of Korin Tower.
As for why Korin allows Yajirobe to stay in the Tower, that's still a mystery. There's even a non-canon scene from Cooler's Revenge where the two ponder this question; neither of them reaches a definitive conclusion. Whatever the reason is, it's not something that's been expressly stated in the canon story.
It's possible that Korin lets Yajirobe stick around as his liaison with the rest of the world. Technically, Korin is a god of sorts; he's the "God of Martial Arts," to be precise. As such, it's possible that he's not allowed to interact too much with the world below his Tower. This would make Yajirobe integral for delivering Senzu Beans to the Z Fighters.
It's also possible that these two actually enjoy one another's company. They might have grown a fondness for one another in spite of their respective snark or even because of it. This would certainly explain why Korin didn't mind Yajirobe making the Tower his permanent abode.
Whatever the reason for Yajirobe's extended stay, it doesn't seem like he's going anywhere any time soon. He's more than made a home for himself, and Korin doesn't seem to mind the extra noise. If Dragon Ball thus far is any indication, Yajirobe will continue on as the Z Fighters' Bean Daddy for as long as he can.