Retirement Planning for $500K-$5M Households: When to Stop Saving and Start Spending (Without Tax & Medicare Traps)

You can be “fine” on paper and still feel like you can’t retire.

I see it all the time: households with $800,000, $1.6 million, even $3 million invested who keep working because the plan in their head is basically, “Save more and hope it works out.” That’s not a plan. That’s a delay strategy.

Here’s the tension: the moment you stop accumulating and start spending, the rules change. Taxes behave differently. Medicare premiums can jump based on income you didn’t realize “counted.” Social Security becomes a real lever, not a vague future benefit. And market declines early in retirement can do more damage than the same declines later.

As a fiduciary advisor, my job isn’t to sell you a product or tell you to “stay the course” and hope. My job is to help you make a clear decision: are you still working because it meaningfully improves your after-tax retirement life, or because you don’t yet have an integrated retirement income plan?

This pillar guide is a numbers-first framework for Retirement Planning for $500K-$5M Households—specifically for people close to retirement who want plain-English clarity on when to stop saving and start spending without triggering avoidable taxes, Medicare premium surprises, or withdrawal-sequence risk.

The goal is not to maximize your portfolio. The goal is to maximize your after-tax life.

The real finish line: “enough” is an after-tax paycheck, not a portfolio number

Most near-retirees are chasing the wrong target.

A portfolio balance feels concrete. “I’ll retire at $2 million” sounds responsible. But it’s incomplete because it ignores:

How much of that $2 million is pre-tax (like a 401(k) or IRA) versus after-tax (Roth) versus taxable brokerage.

How much income you need after taxes to live your life.

How Social Security changes the required withdrawals from your portfolio.

How Medicare premiums react to your taxable income.

How market risk early in retirement can change the outcome even if your long-term average return looks fine.

So the finish line isn’t a number on a statement. It’s a sustainable, inflation-adjusted, after-tax spending plan with guardrails.

Here’s the simple translation:

Your portfolio is a warehouse.

Retirement is the distribution plan.

If you distribute poorly, you can pay more tax than necessary, trigger higher Medicare premiums, and take more market risk than you intended.

If you distribute well, you can often retire earlier than you think—not because you “found” more return, but because you stopped leaking money through avoidable friction.

A fiduciary framework: the 5 numbers that decide “work longer” vs “retire now”

When a household asks me, “Can we retire?” I’m not looking for a magic withdrawal rate. I’m looking for five numbers that tell the truth quickly.

1. Your spending target, in today’s dollars

Not your “budget.” Your lifestyle cost.

Start with a clean annual number for what you want to spend after taxes. Include travel, hobbies, helping family, and the unsexy stuff like home repairs.

For many $500K–$5M households, the mistake is underestimating the “lumpy” years: a roof, a car, a big trip, a grandchild’s wedding gift.

2. Your guaranteed income floor

This is income that shows up regardless of the market.

Social Security.

Pensions (if you have one).

Certain annuity income (only if it’s already in place or being evaluated carefully).

This matters because it reduces how much your portfolio must produce.

Remember: Social Security is designed to supplement other income, not replace it. For many workers it’s roughly 40% of pre-retirement earnings on average, which is why your portfolio strategy still matters.

3. Your “gap” to be filled by investments

Gap = (after-tax spending need) minus (after-tax guaranteed income).

This gap is the paycheck your investments must create.

If your gap is small, you can take less risk.

If your gap is large, you need either more assets, less spending, more time working, or a different income design.

4. Your tax profile (not your tax bracket)

Near retirement, “tax profile” is more important than “tax bracket.”

Tax profile means:

How much is in pre-tax accounts (IRA/401(k)) that will be taxed when withdrawn.

How much is in Roth accounts (generally tax-free withdrawals if rules are met).

How much is in taxable brokerage (where capital gains rules apply).

How much future income will be forced on you via Required Minimum Distributions (RMDs).

RMDs currently begin at age 73 for many retirees. Miss an RMD and the penalty can be 25% of the amount you should have withdrawn. That’s not a “minor paperwork issue.” It’s a real planning landmine.

5. Your sequence risk exposure in the first 5–10 years

Sequence-of-returns risk is a simple idea with serious consequences.

If markets drop early in retirement while you’re withdrawing, you may be selling investments at depressed prices. That can permanently reduce the portfolio’s ability to recover.

This is why “average returns” are not the whole story. The order of returns matters when you’re taking money out.

If you want a deeper explanation and examples, we built a supporting guide here: /post/investment-allocation-in-retirement-sequence-of-returns-risk

A realistic example: the “we’re fine, but it doesn’t feel fine” couple

Let’s make this real.

Mark (63) and Denise (61) have:

$2.2 million invested

$1.6M in 401(k)/traditional IRA (pre-tax)

$350K in Roth IRAs

$250K in a taxable brokerage account

$0 debt, home paid off

They want to spend $120,000 per year after tax.

If Mark retires now, they plan to claim Social Security later (more on that soon). For the first few years, their portfolio will carry most of the income.

Their biggest risks aren’t “running out of money tomorrow.” Their biggest risks are:

Taking too much from the IRA early and accidentally pushing taxable income high enough to trigger Medicare premium surcharges later.

Waiting too long to do any Roth conversions, then getting hit with large RMDs at 73 that force high taxable income for the rest of their lives.

Investing too aggressively because they’re still thinking like accumulators, then getting a bad market sequence in the first few retirement years.

This is exactly where a numbers-first framework helps. It turns “I think we’re okay” into “Here’s the plan, here are the guardrails, and here are the tradeoffs.”

The underrated pivot: stop optimizing the portfolio and start optimizing the tax return

Here’s my strong opinion: for $500K–$5M households, the most overrated retirement strategy is obsessing over investment performance while ignoring tax structure.

You can have a perfectly reasonable portfolio and still lose the game through:

Poor withdrawal order

Unplanned Roth conversions

Accidental Medicare IRMAA surcharges

RMD-driven tax spikes

Uncoordinated charitable giving

The best retirement plans treat taxes as a multi-decade project.

Plain-English definitions (because jargon hides the ball)

Taxable income: the income number the IRS uses to calculate your tax. It’s not always the same as the cash you received.

Capital gains: profit when you sell an investment in a taxable account for more than you paid.

Roth conversion: moving money from a pre-tax IRA to a Roth IRA and paying tax now, to potentially reduce taxes later.

IRMAA: an extra Medicare premium charge if your income is above certain levels. Medicare looks back two years at your income to set these premiums.

RMD: required minimum distribution—forced withdrawals from certain retirement accounts starting at specific ages.

Why this matters: you don’t retire into a single tax bracket. You retire into a tax system.

Your income will change year to year.

Your deductions will change.

Your filing status may change.

And the government will eventually require distributions from pre-tax accounts.

So the question isn’t “What’s my bracket this year?”

The question is “How do we create the income we need while smoothing taxes and avoiding avoidable premium surcharges over 20–30 years?”

Designing your withdrawal paycheck: a tax-efficient order that matches real life

In retirement, you’re not “taking withdrawals.” You’re building a paycheck.

A good paycheck plan does three things at once:

Funds your lifestyle

Manages taxes intentionally

Protects the portfolio from early-retirement market shocks

There is no single perfect withdrawal order for everyone, but there is a common structure that works well when customized:

Use taxable accounts strategically (not automatically)

Taxable brokerage money is often the most flexible.

It can help bridge early retirement years before Social Security.

It can help keep taxable income lower if you’re selling assets with small gains.

It can also be used for large one-time expenses without creating ordinary income the way IRA withdrawals do.

But it’s not as simple as “spend taxable first.” Sometimes spending taxable first creates bigger capital gains later. Sometimes it wastes low tax brackets you could have filled with IRA withdrawals or Roth conversions.

Coordinate IRA withdrawals with Roth conversions

This is where many households either save or lose six figures over a lifetime.

If you retire at 62 and wait until 73 to think about taxes, you may miss an unusually valuable window: the years when you have lower earned income and before RMDs begin.

In those years, you can often:

Withdraw from pre-tax accounts up to a target tax bracket

Convert additional IRA dollars to Roth at a planned tax rate

Use taxable assets to cover the rest of spending

This is the heart of Roth conversion planning before RMDs. We have a deeper supporting article here: /post/roth-conversions-before-rmds-reduce-lifetime-taxes-irmaa

Use Roth as a shock absorber

Roth accounts are powerful in retirement because qualified withdrawals generally don’t add to taxable income.

That means Roth can help you:

Avoid jumping into a higher tax bracket in a high-spending year

Manage Medicare IRMAA thresholds

Pay for a large expense without triggering a tax spike

Leave tax-advantaged assets to heirs (depending on your estate goals)

If you only have pre-tax money, your flexibility is lower. That doesn’t mean you’re stuck. It means your plan needs to be more intentional.

If you want a full breakdown of withdrawal sequencing, start here: /post/tax-efficient-withdrawal-strategies-which-accounts-to-spend-first

Medicare IRMAA: the “stealth tax” that surprises high-savers

Medicare is not one price.

Most retirees know about Medicare Part B and Part D premiums. Fewer realize that Medicare can charge you more if your income is above certain thresholds. That surcharge is called IRMAA.

Here’s the simple version:

Medicare looks at your income from two years ago.

If it’s above certain levels, you pay higher monthly premiums.

Those higher premiums can last a full year.

Why this matters for $500K–$5M households

You don’t need to be “ultra-wealthy” to get hit by IRMAA.

Common triggers include:

Large IRA withdrawals

Big Roth conversions done all at once

Selling a highly appreciated asset

One-time income events (severance, business sale, large capital gain)

Even if the income event is a one-time thing, the premium increase can still apply.

The planning move is not “avoid income at all costs.” The move is to decide when higher income is worth it.

Sometimes paying IRMAA for one year is absolutely worth it if it reduces lifetime taxes.

Sometimes it’s an unforced error.

The difference is whether you modeled it.

If Medicare planning is on your mind, go deeper here: /post/medicare-planning-irmaa-how-income-raises-premiums

Social Security: the lever most people underuse (especially couples)

Social Security decisions are often treated like a one-time form you fill out.

In reality, it’s one of the biggest “guaranteed income” choices you’ll make.

And for couples, it’s not just two separate decisions. It’s a coordinated strategy.

Plain-English basics

Claiming earlier means smaller monthly checks.

Claiming later means larger monthly checks.

If you live a long time, delaying can pay off.

If you claim early, you may reduce the survivor benefit for the spouse who lives longer.

What’s commonly misunderstood

People focus on “break-even age” as if the only goal is to maximize total dollars received.

But retirees don’t live on total dollars received. They live on monthly cash flow and risk management.

Delaying Social Security can act like buying more guaranteed income later in life.

That can allow you to:

Spend more confidently from your portfolio early

Take less market risk later

Reduce the chance that one spouse is financially squeezed after the first spouse dies

On the other hand, claiming earlier can be reasonable when:

You retire early and need income

You have health concerns that shorten expected longevity

You have a high tax cost to bridge the gap (for example, large IRA withdrawals that spike taxes)

The right answer is usually not ideological. It’s coordinated.

We built a supporting guide specifically for couples here: /post/social-security-optimization-for-couples-claiming-strategies

The risk shift: why your portfolio needs different rules once you’re spending

Accumulation investing is about growing assets.

Decumulation investing (spending) is about funding a paycheck while controlling the damage from bad timing.

That requires a different set of rules.

The three-bucket idea (without turning it into a gimmick)

I’m not a fan of overly cute bucket systems, but the underlying concept is useful:

Near-term spending money should not be hostage to the stock market.

Mid-term money should be invested to refill the near-term bucket over time.

Long-term money can take more growth risk because it has time.

In practice, this often means having:

A cash reserve for upcoming spending

A bond or conservative allocation for the next several years

A stock allocation for long-term inflation protection

The exact mix depends on your gap, your guaranteed income, and your comfort with volatility.

What’s overrated

Chasing yield because “I need income now.”

High-yield strategies can concentrate risk and create unpleasant surprises. Retirement income is not just about generating dividends or interest. It’s about total return and controlled withdrawals.

What’s underrated

Simple guardrails.

For example:

A rule for how much you’ll spend from the portfolio when markets are down.

A plan for which account you’ll draw from first in a down year.

A rebalancing discipline that forces you to sell some of what went up and buy some of what went down.

If you want the deep dive on sequence risk and allocation tradeoffs, here again is the supporting piece: /post/investment-allocation-in-retirement-sequence-of-returns-risk

RMDs and the “tax wave” problem: plan before 73, not at 73

RMD planning is not about avoiding a penalty. It’s about avoiding a tax wave.

If most of your retirement savings are in pre-tax accounts, you may be sitting on a future tax bill.

At 73, the government requires you to start taking money out.

Those withdrawals add to taxable income.

Higher taxable income can:

Push you into higher tax brackets

Increase taxation of Social Security benefits

Trigger Medicare IRMAA surcharges

Reduce flexibility for charitable giving and estate planning

This is why the years between retirement and RMD age are so valuable.

You can often use that window to:

Do partial Roth conversions

Harvest capital gains strategically in taxable accounts

Fund spending in a way that keeps income within planned ranges

Coordinate charitable giving (including potentially using Qualified Charitable Distributions once eligible)

If RMDs are on your horizon, read our supporting guide: /post/required-minimum-distributions-rmds-what-changes-and-how-to-plan

Healthcare costs: the planning isn’t just the premium

Healthcare in retirement is one of the most emotionally loaded topics, and I’m not going to use scare tactics here.

The practical reality is simpler:

Healthcare costs are a line item that tends to rise over time.

The big expenses are not always predictable.

Your choices (Medicare, supplemental coverage, prescription plans) interact with your income.

The two planning mistakes I see most

Mistake 1: Treating Medicare as separate from tax planning

Medicare premiums can be affected by income (IRMAA). So a tax plan that ignores Medicare isn’t complete.

Mistake 2: Underestimating the “gap years” before Medicare

If you retire before 65, you may have several years of private health insurance.

Those years can be expensive.

They can also influence your Roth conversion strategy because higher income can affect subsidies (if applicable) and overall costs.

This is one reason the “retire at 62” decision is not just a portfolio decision. It’s a healthcare and tax decision.

Estate and legacy coordination: don’t let beneficiaries inherit a tax mess

If you’ve accumulated $500K–$5M, you’re not just planning for your own retirement. You’re also deciding what happens to assets and taxes after you’re gone.

This is where coordination matters:

Your beneficiary designations on IRAs and 401(k)s override your will.

Roth and pre-tax accounts have very different tax outcomes for heirs.

Charitable goals can be funded in ways that reduce taxes.

Trusts may or may not be appropriate depending on your goals, family dynamics, and state law.

A plain-English principle I use with clients

Don’t leave your family a scavenger hunt.

That means:

Accounts are titled correctly.

Beneficiaries are updated.

Powers of attorney and healthcare directives exist and match reality.

Your retirement income plan doesn’t accidentally create a tax bomb for the surviving spouse or heirs.

If you’re in the $500K–$5M range, estate planning is less about “estate tax” for most families and more about control, clarity, and income-tax consequences.

Most important retiree questions right now (with straight answers)

How can I create a tax-efficient withdrawal strategy for my retirement savings?

Start by mapping your accounts into three tax types: taxable, pre-tax, and Roth. Then design a yearly withdrawal plan that targets a tax range on purpose.

If you want a practical framework and examples, use this supporting guide: /post/tax-efficient-withdrawal-strategies-which-accounts-to-spend-first

What are the best ways to optimize my Social Security benefits?

For couples, treat it as one coordinated decision, not two separate ones. Consider longevity, survivor needs, and how claiming timing affects how much you must withdraw from investments.

Start here: /post/social-security-optimization-for-couples-claiming-strategies

How should I plan for healthcare expenses during retirement?

Plan in layers:

The gap years before 65 (if retiring early)

Medicare premiums and coverage choices

Income-related premium surcharges (IRMAA)

A reserve for unpredictable out-of-pocket costs

For IRMAA specifically: /post/medicare-planning-irmaa-how-income-raises-premiums

What estate planning strategies should I consider for my assets?

At minimum: updated beneficiaries, updated documents, and a clear plan for how pre-tax accounts will be handled. If you have complex family situations, business interests, or charitable goals, coordinate with an estate attorney and your tax advisor.

How can I manage investment risk as I approach retirement?

Shift from “How much can I earn?” to “How much risk do I need to take to fund the gap?” Then build guardrails for the first 5–10 years to reduce sequence risk.

Deep dive: /post/investment-allocation-in-retirement-sequence-of-returns-risk

The decision framework in practice: when “one more year” helps—and when it’s mostly habit

Working one more year can be powerful. It can also be pointless.

Here’s how I separate the two.

One more year tends to be high-impact when:

You’re still adding meaningful savings.

You’re delaying Social Security and increasing future guaranteed income.

You’re shortening the time your portfolio must fund spending.

You’re keeping employer health coverage and avoiding expensive pre-65 insurance.

You’re using the year to execute a tax plan (for example, doing planned Roth conversions after retirement but before Medicare starts).

One more year tends to be low-impact when:

Your spending goal is already well-covered.

Most of your “extra savings” are just piling into pre-tax accounts that will later create RMD pressure.

You’re taking market risk you don’t need because you’re still thinking like an accumulator.

You’re delaying retirement mainly because you don’t trust the plan.

The point isn’t to retire as early as possible.

The point is to stop trading years of your life for a bigger number that may not improve your after-tax outcome.

A practical action list: what to do in the next 30 days

If you’re within 0–5 years of retirement and you have $500K–$5M invested, here’s a clean set of actions that will move you forward quickly.

1. Write down your “after-tax lifestyle number”

One annual number. Not a category spreadsheet. Start simple.

2. List your income sources and when they start

Social Security (estimate at different claiming ages).

Pension (if any).

Rental income.

Part-time work.

3. Inventory accounts by tax type

Taxable brokerage

Traditional IRA/401(k)

Roth

HSA (if applicable)

This is the foundation for a Tax-Efficient Withdrawal Plan.

4. Ask: what is our RMD exposure?

If your pre-tax accounts are large, estimate what RMDs could look like at 73 and how that might affect taxes and Medicare premiums.

Supporting guide: /post/required-minimum-distributions-rmds-what-changes-and-how-to-plan

5. Identify your Medicare risk points

Are you likely to have high-income years from conversions, asset sales, or big withdrawals?

If yes, model IRMAA before you act.

Supporting guide: /post/medicare-planning-irmaa-how-income-raises-premiums

6. Set two spending guardrails

A “normal” spending amount.

A “down market” spending adjustment rule.

This is how you reduce sequence risk without living in fear.

7. Update beneficiaries and confirm your documents

Beneficiaries on retirement accounts.

Primary and contingent beneficiaries.

Powers of attorney and healthcare directives.

If you haven’t looked in years, assume something is outdated.

8. Decide what would make you feel ready

Not emotionally ready. Financially ready.

Examples:

“If we can fund $X after tax with a 90%+ success range in stress tests, we retire.”

“If we can retire without triggering long-term IRMAA surcharges, we retire.”

“If we can create a withdrawal order and Roth conversion plan through age 73, we retire.”

This turns retirement into a decision, not a vibe.

If you want help building your Retirement Income & Tax Map

If you’re sitting on real assets and real decisions, you don’t need more generic retirement tips. You need an integrated plan that answers, in numbers:

What “enough” looks like for your household

How to fund spending with a Tax-Efficient Withdrawal Plan

How Social Security timing affects your long-term safety and flexibility

How to avoid avoidable Medicare premium surprises

How to reduce withdrawal-sequence risk with the right investment allocation and guardrails

How to coordinate RMD planning, Roth conversions, and estate goals

If you’d like to build that plan with us at Grape Wealth Management, book a Retirement Income & Tax Map session here:

Book an appointment: Schedule appointment