After using Google Voice as my exclusive phone number for about a year, I finally gave it up and went back to a regular ol' number on Verizon. There were a lot of things I loved about Google Voice -- being able to send online, having a history of all my texts, being able to send/receive SMS while traveling internationally (without crazy fees) -- but ultimately, the fact that messages would disappear was unacceptable. Why would they disappear? Because Google Voice doesn't support MMS. While I don't need to send my friends a pic every time I see a cute puppy, I do want to know when my friends send me a pic. Or, at the very least, they should know that the message was dropped. Google Voice fails to report any error message on either side. Not okay.

This issue became particularly problematic with iOS' new iMessage functionality which supports group chat. This works on the MMS protocol, so, again, messages would vanish. People thought I was just being rude and not responding.

So, after much debate, I switched off of Google Voice. Plus, I hated my Android and Google Voice doesn't work as well on an iPhone.

I couldn't find much online about what to expect when porting out of Google Voice, so I thought I'd write up my experience. [For simplicity, I'll call 610 my Google Voice number and 650 my Verizon number. Prior to porting, my 610 number forwarded to my 650 Verizon number.]

I unlocked my Google Voice (610) number at www.google.com/voice/unlock. Because I'd previously ported that number into Google Voice, I didn't have to pay anything. Unlocking appeared to happen instantly.
[Monday @2 pm PST] I went into the Verizon store to explain what I needed to do. I already had an existing Verizon contract, so they would need to switch my contract to the 610 Google Voice number. The representative appeared to understand what Google Voice was, but wasn't very clear on how to go about this.
The Verizon rep talked to a manager and various people and eventually learned that he needed to port the (610) Google Voice number as though it were a land line number.
Their system requires an account number. This is your Google Voice number (610). Apparently, this failed the first time he tried it (possibly he typed in the wrong number), but worked the second time. I spent about an hour in the shop total.
The Verizon rep explain what I should expect. He said it would take 2 to 10 business days, during which time I might not receive SMSs. When I left the shop, nothing had changed. My old non-Google Voice Verizon number was still active. I continued to get calls and SMSs to my (610) Google Voice number. Google Voice showed the same "status" of just being unlocked. There was basically no sign that anything had changed. Verizon explained that, at some point in the next 10 days, my phone would be suddenly disconnected. At that point, I needed to call *226 to attach my 610 number to my Verizon iPhone. It could still take a while after that to complete the porting though. Of course, their "what to expect" explanation was based on land lines, not Google Voice.
[Wednesday @ 6:30 pm PST] My phone just disconnected with no notice. Unfortunately, I didn't have the instructions with me right then, so I couldn't re-connect my phone.
[Thursday @ 1:00 am PST] I call *226 to complete the porting. With a few minutes, I received an SMS to my 610 number that I'd missed while my number was disconnected. Everything seemed to work instantly.
[Thursday @1:30am PST] I need a local phone number for my apartment buzzer, so I set up a new Google Voice number to forward to my 610 number. Although some people report having difficulties doing this, I was able to have my new Google Voice number immediately start forwarding to my 610 number.

As far as I could tell, everything started working instantly. Others report it taking several days (or even weeks) to receive SMSs from all carriers.

If you decide to port your Google Voice number to Verizon, AT&T, or any other carrier, I'd recommend that you do the following:

Unlock your Google Voice number before going into the store.
Start this process on a Monday, or maybe Tuesday, as this process is measured in business days.
After the porting is theoretically complete, have people on each major carrier test sending you a message. If you are using an iPhone, you should probably select "test senders" who are not on iPhones. iPhones will use iMessage to SMS between each other when possible. It's possible that iMessage is working when normal SMSs are not.

All in all though, it was a pretty fast, seamless process. (For what it's worth, porting from AT&T to Google Voice resulted in about 2 - 5 days when I couldn't receive any SMSs.)