A smaller home doesn’t always come with a smaller mortgage. In an effort to trim housing costs, some homeowners may be looking to switch to a smaller house, but economists say that’s not always a cost-saving move. “There are times when [moving to a smaller home] could have the opposite effect,” David Mele, told U.S. News & World Report. He points to a possible increase in taxes, moving expenses, insurance, and more that could make downsizing pricier.

“I have frequently had people sell huge, beautiful houses in bucolic suburbs near the city and pay more for their new two-bedroom apartment,” says Sheila Trichter, a real estate professional  in New York.

There are many factors that could make downsizing more expensive, and financial experts say home buyers need to consider them before taking the plunge. For example, downsizing may be pricier for those moving to a new home. Newer homes tend to have trendy decor, more resilient building materials, and energy-efficient features, pushing up the price—even if the home has less square footage. The location can have a big impact, too. Homeowners moving from rural to urban areas, in particular, may find that a smaller home isn’t necessarily cheaper.

A smaller home can also still come with higher property taxes. For example, in Michigan, the taxable value of a home is capped at the rate of inflation. But once the property sells, the cap is lifted and the taxable value is adjusted to equal the assessed value. “Property taxes are different in every state,” says Melanie Halstenberg, co-founder of Arch Financial Services, an advisory firm in Fayetteville, W.V.

Homeowners insurance premiums aren’t necessarily less with a smaller home either, financial experts note. The home’s location also could impact insurance: For example, moving to a coastal area may make buying supplemental flood insurance necessary.

Condos are a popular option for downsizing homeowners. But they also may come with added costs in association fees. Vincent Averaimo, a partner with Milford Law in Milford, Conn., says associations often charge $300 to $400 per month to go toward maintaining common areas and exterior upkeep and services. These costs can go higher over time. Averaimo advises that homeowners ask themselves before downsizing: “Do you already spend a total of approximately $3,600 per year in landscaping and snow removal?