You can definitely make your housing dream come true. The first thing to understand is that second-home mortgages carry interest rates that can be about half a percentage point higher than those on primary residences. Lenders figure that if you hit hard times, you're more likely to keep up payments on your primary home than on the vacation spot. The other issue is that your rate will be higher if the mortgage is above $333,700. Lenders refer to such loans as "jumbo mortgages."
So we want to get creative and avoid the problems of a big mortgage on a second home. Since you own your current home outright, I want you to refinance it or take out an equity line of credit, and use that to pay for the vacation home. And make sure the loan is paid off in 15 years. You don't want to face big mortgage payments in retirement.
Since you had a $300,000 mortgage on your current home, and your interest rate was probably about 7 percent, my guess is that you were paying $1,600 per month, plus the $1,000 you used to pay the loan off early. When I factor in typical property-tax and insurance numbers, you probably were paying a total of $3,000 each month in housing costs.
It's my firm belief that no one approaching retirement should stretch her budget to own a home, especially a second one. I don't want your dream house to cost you more than $3,000 per month. That means you can conservatively afford a $350,000 home, whose monthly housing costs would be about $2,860. If your dream comes with a bigger price tag, keep this in mind: For every $100,000 increase in cost, your monthly mortgage will rise by about $820 at today's interest rates. So if you buy a $450,000 home, you'll spend $3,680 per month. A $550,000 home would cost you $4,500 a month. With your income, you can afford that now, but what happens once you're retired? If I were you, I'd aim for the lower price range and use any extra cash to boost retirement savings.
Notice I didn't mention the impact of the mortgage-interest deduction. That's because vacation homes tend to require more upkeep. You aren't there 52 weeks a year to keep an eye on things. Perhaps you'll need to hire someone to look after the place. I figure an easy way to "budget" for such extra costs is to simply ignore the mortgage-interest deduction in your calculations. My guess is you'll end up spending those savings on upkeep.
Now all you have to do is prepare yourself for the onslaught of friends and relatives who'll want to be your weekend guests!