Damn, I guess I was really tired when I replied to the OP because it seems I misread the title's "expanding" as "exploring". Actually reading the OP changes a lot of things...
I think it's a great idea if done right. It'll be a long time until we can colonise another planet or moon. I already have experience travelling underwater.
Of course, we finished the (to my knowledge) first successful underwater tunnel in the 1990s, the Channel Tunnel. I've taken the Eurostar to visit Paris in the past.
Despite not having a view it was easy to forget that I was travelling underwater. I think an actual society could exist underwater too. Surely it would be inexpensive compared with space colonisation.
One of Saturn's moons, Titan, could actually be the best candidate for space colonisation. But think about how long it would take to get there. You'd be spending a significant amount of your life just going to visit family on Earth. Then there's the return journey. We're talking years. Unless there's no intention to visit Earth this will never be realistic without light-speed travel. I'm no expert but I doubt that will go beyond science fiction. I said Titan is a good candidate, but that isn't saying a lot since it's still a lethal environment.
On the other hand I can imagine a visit to the surface of Earth being a rather short journey. Inexpensive too when you consider the cheap (compared to flying) cost to visit another country with Eurostar. I think an underwater city would have both trains and lifts.
I like the idea and it's something that will in my opinion happen long before space colonisation. The world's population is at around 7.8 billion now. By 2030 it'll likely have grown to over 8.5 billion. By 2050 we'll have almost reached 10 billion. Underwater cities are simply an inevitability.