Like the Tesseract, the Mind Stone wasn't initially introduced as an Infinity Stone. The stone was originally in the possession of Thanos, who gave it to Loki so that he could use it to get the Tesseract. The stone was hidden within the scepter, used by Loki to take over the minds of humans such as Hawkeye and Erik Selvig. The stone, still hidden inside the scepter, ended up in S.H.I.E.L.D. custody after the events of Avengers.
But the scepter was stolen by a disenfranchised S.H.I.E.L.D. agent and Baron Strucker in the comic tie-in Avengers: Age of Ultron Prelude. The Avengers reunited in Avengers: Age of Ultron to take the scepter away from Strucker and bring down the last of the Hydra leadership. But when Ultron was created, he stole the scepter to use as part of his perfect new body.
The stone ended up becoming the lifeforce powering the being known as Vision, with the apparent consciousness inside the Mind Stone helping to form his personality. The question of what to do with the Mind Stone played out throughout Infinity War, with Scarlet Witch even destroying it at one point. But Thanos used the Time Stone to reverse the destruction and took it for himself, killing Vision in the process.
The Time Stone has actually been used for good, proving essential to the protection of Earth from the dimensional conqueror Dormammu. The Time Stone was first in the possession of the ancient sorcerer Agamotto, who hid the stone within a device of his own creation. Known as the Eye of Agamotto, the tool could be used to manipulate the time stream of the entire world or even just individual moments. The Eye was kept in the mystic city of Kamar-Taj and studied by the sorcerer Cagliostro, who recorded his findings in the Book of Cagliostro.
When Doctor Strange arrived in the city, he showed particular interest in the Eye and trained with it. He utilized it against Kaecilius and his followers, reversing the ramifications of their attacks on the Hong Kong Sanctum. Strange was even able to use the stone to force Dormammu to retreat after trapping the two of them in a time loop. Strange took to wearing the Time Stone around his neck, utilizing it in Infinity War to fight against Thanos and explore what plans could help them stop the Mad Titan.
It's noticeable that he seemed reluctantly knowledgeable about the ramifications of turning the stone over to Thanos to save Tony Stark's life, hinting that he saw a path for the heroes to save the day that required half of existence to (at least temporarily) be killed.
The final Infinity Stone to be introduced, the Soul Stone was hidden away from almost every force in the universe. Infinity War Prelude #2 revealed that Wong was aware of the location of most Infinity Stones, but that he had no idea where the Soul Stone was located. Not even the Asgardians knew where it was. Only Gamora found the location of the stone, eventually giving up the location to Thanos when he tortured her adopted sister, Nebula. It is not specified how the stone eventually ended up on the planet Vormir, but that's where Gamora took Thanos.
When Thanos and Gamora arrive on the planet, they learned that the Soul Stone took Red Skull (who was teleported to Vormir by the Space Stone instead of being killed as it appeared in Captain America: The First Avenger) and made him the guardian for the stone. Incapable of loving anything enough to warrant the necessary sacrifice to claim the power, the Red Skull was humbled and became a servant of the Soul Stone. He advised Thanos about the stone, and Thanos sacrificed Gamora to claim it. Taking the Red Skull as his servant hints again that the Infinity Stones seem to have something of a consciousness, but their mysterious nature means their full power is still unclear.
We'll hopefully find out more about the stones in the upcoming Avengers: Endgame. Directed by Joe and Anthony Russo, Avengers: Endgame stars Robert Downey Jr., Josh Brolin, Mark Ruffalo, Tom Hiddleston, Chris Evans, Chris Hemsworth, Jeremy Renner, Chris Pratt, Elizabeth Olsen, Chadwick Boseman, Sebastian Stan, Benedict Cumberbatch, Paul Bettany, Samuel L. Jackson, Cobie Smulders, Benedict Wong, Zoe Saldana, Karen Gillan, Vin Diesel, Dave Bautista, Pom Klementieff, Scarlett Johansson, Tom Holland and Anthony Mackie. The film arrives April 26.