Taxes on IRA withdrawl question

[OnionPatch]

I don't know if this is the right place to ask this question. I'm not an avid investor and don't follow the markets much or anything. I'm just trying to survive the best I can during retirement and could use some advice...

My big burning financial question right now is: Is there a way I can use some of my IRA money to pay off my mortgage without getting bumped into a higher tax bracket? I'm at the 14% rate now but the next step up is 22%! Quite a jump. I'm retired so I'd have no withdrawal penalties for using the money and I don't mind paying taxes on that money since it was a 401K rollover that wasn't taxed when deposited, but I DO mind getting bumped into a higher tax bracket if I use it. I have more than enough to pay off my mortgage and really wish I could use some of it to do that. All I can think of is to withdraw a little bit each year to pay on the mortgage, but keep it just under the amount that would bump me up to 22%. But that's going to take years. is there any other way? I'd love to get rid of my mortgage ASAP. Thanks in advance.

[FBaggins]

2. Generally... no

Oh, there are ways to nibble around the edges, and you would have had an option of borrowing against the balance if you had left it in the 401k…

… but if I understand your scenario correctly (i.e., that the amount you need is many multiples of the income difference between your current income and the income that will drive you into the next tax bracket), the answer is very likely “no”

On a related note (pun intended), if you refinanced while rates were so low, you’re probably better off just keeping the mortgage. At current inflation rates it has a negative carrying cost.

[BlueBloodedAmerican]

6. I'm a retired financial advisor. Your response is absolutely correct.

I understand hating that mortgage payment. Who doesn’t? However, when most of your net worth is tied up in your home because you paid off your mortgage, you will have to pay fees and interest on YOUR OWN MONEY if you need to access it down the road. Also, if your property taxes and home insurance have been paid out of an escrow account accumulated monthly as part of your mortgage payment, you will now have to cough up those fees each year in one big chunk on your own. Just food for thought.

[OnionPatch]

9. Unfortunately I did not refinance.

I'm having a home built and the construction loan rolls over into a traditional mortgage once the home is complete. It will be subject to the rates at that time. That's partly why I wish I could get that principle lower.

I retired so I was not able to leave the money in the 401k. It had to be rolled over.

From looking at just the basics, I'd be able to pay off about 20K a year before I get kicked into a higher tax bracket. I don't think Social Security counts investments as earned income, do they? If not, it shouldn't affect my SSI payment, should it?

[Yonnie3]

3. If you get Social Security you may get taxed more on that.

I used to do my father's income tax and in a certain range of income he was taxed on $1.85 of income for every dollar of additional income. $0.85 of the SS became taxable. I've no idea what that range is now. If you use a tax software you can try several scenarios.

[OnionPatch]

17. I'm not sure if this is what you mean but I can make up to 19K of wages before my SSI is affected.

At least that's what they told me when I applied for SSI. Also, they said retirement income doesn't county. Only wages.

[Yonnie3]

19. no, that is not what I meant

I think you are talking of a reduction in the benefit if you are employed.

Part of your social security can be subject to income tax. How to calculate the amount can be found in the 1040 form instructions. There is a worksheet.

EDIT: I see progree has covered all the gotchas later in the thread.

[progree]

5. Your actual marginal tax rate could be higher, much higher ...

Last edited Mon Jul 4, 2022, 07:32 PM - Edit history (1)

One way, as Yonnie points out in #3, is more of your SS income may be taxable, and instead of 22% marginal rate, it COULD be effectively 1.50 * 22% = 33%, or a 1.85 * 22% = 40.7%.

Another gotcha is that if you have capital gains, additional ordinary income (and that's what a traditional IRA withdrawal is, tax-wise) can cause more of the capital gains to be taxable at the 15% capital gains rate. That can add 15 percentage points to your actual marginal rate.

and that's just federal hit. If you have state income taxes, that has to be considered too.

A 3rd way that one's marginal tax rate could be higher than the tables show is the surcharge on Medicare Part B and Part D premiums. Not something most people with a 12% nominal marginal tax rate have to worry about, but if you are talking about paying off a mortgage, that's a big chunk of money and that could lift your AGI into the range that causes you to pay a higher monthly premium.
https://rcsplanning.com/insights/medicare-irmaa-surcharges

I personally experience all three of the above, so I know of what I speak.

Also, a minor point, on the federal, the 2nd tax bracket is 12%

https://taxfoundation.org/2022-tax-brackets/
2022 taxes
TAX RATE FOR SINGLE FILERS FOR MARRIED INDIVIDUALS FILIN… FOR HEADS OF HOU…
10% $0 to $10,275 $0 to $20,550 $0 to $14,650
12% $10,275 to $41,775 $20,550 to $83,550 $14,650 to $55,900
22% $41,775 to $89,075 $83,550 to $178,150 $55,900 to $89,050
24% $89,075 to $170,050 $178,150 to $340,100 $89,050 to $170,050
32% $170,050 to $215,950 $340,100 to $431,900 $170,050 to $215,950
35% $215,950 to $539,900 $431,900 to $647,850 $215,950 to $539,900
37% $539,900 or more $647,850 or more $539,900 or more

Edited to add the remaining lines of the tax table

Edited to add after reading #16 -- just so that this post is somewhat complete as far as the tax / benefits / subsidies impact of additional "income" --

ACA premium subsidy amounts depends on one's income ... and I'm pretty sure a withdrawal from a Traditional IRA is going to count as income for this purpose. Not 100% sure, but pretty sure.

It was an issue back in my ACA days when I did a Roth conversion -- which is treated in part as a withdrawal from a Traditional IRA -- and I lost my ACA subsidy because of it.

[OnionPatch]

16. Lots to think about here.

I'm only 62 and don't get Medicare yet. I have a good plan through the ACA.

Right now I'm in right in the middle of the 12% tax bracket. I could make a little more and still stay put there. I have a daughter in college, too, so I'm head of household.

[progree]

20. Yeah for the ACA. But watch out, your premium subsidy amounts depends on your income ...

and I'm pretty sure a withdrawal from a Traditional IRA is going to count as income for this purpose. Not 100% sure, but pretty sure.

It was an issue back in my ACA days when I did a Roth conversion -- which is treated in part as a withdrawal from a Traditional IRA -- and I lost my ACA subsidy because of it.

Edited to add -- I bet it affects your SSI benefit too.

As far as I know, withdrawals from traditional IRAs count as income for ALL purposes. As silly as it may seem when all one is doing is just trying to access one's own money FFS!

As you mentioned in your OP, you don't mind paying the regular tax on the withdrawal, since you got a tax break when you deposited money in the 401k. But the "side effects", like those enumerated in post#5, can be overwhelming.

It's why some of us caution that one may actually have a higher EFFECTIVE marginal tax rate in retirement than when one was in the "working young" stage.

[OnionPatch]

22. Ok, so I just saw at the SSI site that IRA withdraws don't count as income when determining benefits

But the IRS does count it for tax purposes. Yep, that's the whole problem. I'm getting a lot of benefits by staying below a certain income. I'm too young for Medicare so I'd also hate to lose my subsidy for the ACA. I'm sorry that happened to you.

I guess if I'm going to use any of the IRA money, I could pay just a small extra amount on the mortgage each month but not enough to raise my "income" too much. I probably have a little bit of wiggle room there and a little each month might make a difference in the long run.

My problem is that I always imagined having my home paid for at this point in my life, so it wouldn't be a major expense. But I see now that it's not that simple and I appreciate all your input and sharing your knowledge. I've been trying to learn more about all of this. I probably should have done that before I retired, but life circumstances....you know.

[Phoenix61]

7. Please speak with a financial advisor.

Paying off a home is not always the best long-term financial plan. It can be difficult to get credit or loans after retirement.

[OnionPatch]

8. Yes, that's what I was advised but now I have a monthly mortgage

And it feels counterproductive when I have three times the amount needed to pay it off in my IRA. All that interest. But yes, I should speak to someone. My IRA investment consultant wants me to keep it invested of course. So someone different. Thanks.

[gainesvillenole]

10. Look At Your Interest Rates....

Simply, if your current mortgage interest rate (or your new rate if you refinance) is lower than your average rate on return on your 401K than why pay off the house?
We’re building a new house and originally wanted to simply pay the entire costs from the profit on the sale of our current home and pull the rest from our investment accounts. Luckily we have an excellent financial advisor who pointed out we can secure a 2.9% mortgage rate from the bank (this was last fall before rates rose a little, but they’re still very low) when our investment accounts averaged 5%-6% over the last 7 or 8 years…. You’d be taking money currently earning you (on average) DOUBLE what you’d pay in interest.

[OnionPatch]

11. That makes sense

But lately my IRA is losing money.

[Phoenix61]

12. It's a numbers game.

How much is the IRA making? What’s the interest rate on the mortgage? If you pay more than the monthly payment that will be applied to the principle effectively lowering the interest rate which is something to consider. Then there’s the issue of asset liquidity. What happens if your home depreciates in value?
The best article I read talked about how there isn’t a one size fits all answer for whether or not retirees should pay off their house.

[OnionPatch]

15. Thanks. The IRA isn't making anything right now, unfortunately.

But I hope it will in the future.

Maybe it would be best to pay at least some of it down when I can without bumping me up. I'd like to have lower payments once I get into my 70s. Right now I'm healthy and can make extra income with my skills. But maybe I won't be able to do that later.

[Phoenix61]

18. The only way to get lower payments would be to refinance

That can be a costly endeavor. The only thing paying more will do is shorten the time it takes to pay off the mortgage. This may be helpful.
https://www.bankrate.com/mortgages/amortization-calculator/

[OnionPatch]

24. Thanks, that looks really helpful.

I may want to refinance sometime in the future if interest rates come down significantly.

[mitch96]

13. Just brain storming here but could you split the payoff between two years?

Withdraw and pay ½ in Dec of one year and then withdraw and pay off the note in Jan of the next year. That would make it two year payoff in two months splitting the tax consequence between two years.. I don't know if you have a time constraint with the new house.
Talk to your finance guy is the best bet..
m

[OnionPatch]

14. Thanks. I may do something like this.

I might pay a little extra each month from it but not enough to bump me up. Maybe I could at least get it paid off sooner.

[PoindexterOglethorpe]

21. About ten or so years ago I asked my financial advisor about

selling one of my two annuities and paying my mortgage off completely. He explained that it was not a good idea, because then I would permanently lose the income from that annuity, once I started collecting on it, which was about five years ago now. I'm so glad I listened to him. My annuity amount is almost exactly what my mortgage is. And it will continue so long as I live, long past when the mortgage is paid off, I hope. And if I die sooner, my heirs will get the annuity, or whatever the value is when I pass.

I took a 30 year mortgage at age 60. I'm 73 now. For about two or so years I've been able to send an extra $300 a month, and it will be paid off around 7 years early. I haven't double checked the time line lately. Back when I bought my place, there was simply no way to afford a 15 year mortgage, but I could afford the payment on a 30 year one.

My advice would be not to take excess money out of your IRA. You may need that money down the road. However, if you do go ahead with that plan, do keep in mind that you only pay the higher taxes on the actual money that puts you into that bracket. It doesn't reach back to the earlier money.

Anyway, I'm sure you'll end up making a decision you feel good about.

[OnionPatch]

23. Yes, I'm in a similar situation as you were.

Going to be 63 in a few weeks and wishing I had my home paid for but yeah, everyone is telling me the same thing. My mortgage is also a 30 year mortgage. I really would like to get it paid off in 10 years or so though. I'll have to mess around and figure out what I can pay extra without it hurting me too much on everything else.

Good luck on getting yours paid off early. I really appreciate your advice and all of the advice I get here.

[PoindexterOglethorpe]

25. There's conventional advice out there that no one should retire with a mortgage.

Well, I don't know about your circumstances, but in my case I got divorced at age 60. I wasn't going to rent long term, and I got a sweet little place (2 bedrooms, 2 baths, 3 skylights) in my new city of Santa Fe. It's perfect for me.

While it certainly would be nice not to have a mortgage payment, look very carefully at what else you can do with that money. Investing in good mutual funds would grow that money, and you'd be better off down the road.

Because of careful investments and a good financial advisor, my income has increased in retirement.