The Fellowship's journey through Moria was one of the saddest moments in The Lord of The Rings. As the group fled from the powerful Balrog, Gandalf the Grey fell into the depths of Khazad-dûm. And if losing their leader wasn't difficult enough, Gandalf's death was also a turning point because everyone started to understand the gravity of their mission. If their trump card of a wizard couldn’t defeat the Balrog, there was little chance that the remainder of the Fellowship had any chance of defeating Sauron.
However, that wasn't the only tragedy in the Mines of Moria. To Gimli's knowledge, there was a functional Dwarvish kingdom in Khazad-dûm. That's why he got overcome when he found out otherwise. It wasn't all sadness, though, because before Gimli left Moria, he added to his weapons cache in a big way. Here's why Gimli wanted to go through Moria and the importance of the heirloom he took with him.
Midway through The Fellowship of the Ring, an argument arose about which route the Fellowship should take to Mordor. Among other options, Gimli heavily advocated taking the road to Moria. His cousin Balin, who had accompanied Bilbo on his quest in The Hobbit, had set up a kingdom and declared himself the Lord of Moria. And based on what Gimli knew, that kingdom hadn't fallen, so he wanted to see his relative and bask in the glory of Dwarvish hospitality.
But that all came crashing down as soon as the Fellowship entered the East Gate. It was immediately obvious that the limited Dwarvish presence in the ancient kingdom had gotten overrun, and there would be no hospitality awaiting the group. As they journeyed, Gimli probably held out hope of finding a remnant of Dwarves, but that came to an end in the Chamber of Mazarbul, where they found Balin's grave and the sight of the Dwarves' last stand. However, Gimli didn't have much time to mourn because after Pippin made an ungodly amount of noise, a group of Orcs assailed the Fellowship -- and they brought a Cave Troll with them.
As they quickly prepared for a fight, Gimli grasped his walking axe and throwing axe. However, during the conflict in Moria, he lost both of them while fighting the Cave Troll. So, mid-battle, he picked up the first thing he could get his hands on -- a large battle axe that belonged to Balin. And, apparently, Gimli liked it because he took it with him after the fight and used it for the rest of the War of the Ring.
In Peter Jackson's interpretation, Balin's axe was more than just a memento for Gimli -- it was actually Durn's Axe, one of the oldest weapons in Middle-earth. Forged in the First Age by Durin the Deathless (the first king of the oldest clan of Dwarves), the weapon was a sign of lordship for thousands and thousands of years. Balin had found the axe when he reclaimed Moria and taken it as his own, making Gimli locating it a huge stroke of luck.