My current DSL speed is 10 Mbit with 1 Mbit upstream (approximately 1,250K/sec downstream and 125K/sec upstream).
That should work, in fact some providers claim the minimum required for their VoIP is 1 Mbit (and even less for the upstream). The question is what is your upstream speed? If you're uncertain of either, go to speedtest.net and run their test.Mine is only 3GB or a bit more - I wonder if that is fast enough for VoIP as well as internet use.
There are also different quality codecs to consider, the higher quality ones use more bandwidth. For example G729a uses about 24 kbps/sec, while G711u uses about 80 Kbps/sec. I wouldn't worry about that though. You just want to make sure you have a solid and stable Internet connection, and I'd recommend as close as 1 Mbit for your upstream as possible (the test above claims mine is about 0.92 Mbit). Your 3 Mbit downstream should be more than enough, so long as you're not doing a lot of heavy streaming, downloading or sharing the connection with others.