OP in situation I'd do a minivan. Don't even consider anything else. Used Honda or Toyota since you won't be driving it everyday.
We have it set up so our 2 older ones are in the back row, with baby in the middle by herself. With all 3 in the same row, it makes it hard and distracting for baby to fall asleep for longer drives.
And the fact that you live downtown is a huge reason to get a minivan.
No way I'd want to deal with loading both kids into the backseat, then baby and seat into the same row when there's not enough room to open the doors all the way.
I've had friends who's kids open the door themselves and swing it into the cars next to them.
Sliding doors are the best with young kids.
The upsides to a minivan with kids far outweigh the downsides to manoeuvring and parking downtown.