Yes that is what Google has to offer to you now. Basically now you can access Google voice account from your gmail account, its the integration of Google Voice + Gmail. Using this new Gmail feature you will be able to make free phone calls to USA and Canada.
You need to install a Google voice and video chat plugin to your Gmail account. After you have installed this plugin and restarted the browser, you will see an additional option "Call Option" in the chat window. To make a call, just click the Call phone link in Gmail and enter any number or name from your address book.
You can also receive phone calls in your gmail account now using this same feature. The calls made from Gmail will display your Google Voice phone number as the outbound caller ID. You can also make international phone calls using Gmail, the call rates will be same as that of Google voice service. So basically its just Google voice service which you can now directly access from gmail.
One thing which needs to be mentioned is that this feature is available only to people in USA. I am not sure how are they tracking it, it may be the IP address (will proxies work) or some say its the language set (English US). Keep trying and let us know whatever works for you.
Seems that Google is now all set to give a great competition to Skype, Yahoo voice, Vonage. Lets see how many are able to use this service and find this useful.