Single em dashes can almost always be used interchangeably with semicolons—they typically separate independent clauses without a conjunction.

Paired em dashes—used to demarcate parenthetical expressions—can be replaced by commas, but not by semicolons.

It has less to do with what feels right and more to do with the mechanics of the sentence. There is a good bit of wiggle room, figuratively speaking, in deciding whether to use commas or paired em dashes—likewise, whether to use a single em dash or a semicolon is almost entirely a stylistic choice. But I feel like the way you explained it is a bit misleading to people still learning the difference.

An em dash can also be used to delineate an abrupt break in the direction or structure of a sentence or dialogue in a way that commas or semicolons simply—fuck, I just shit my pants.

Not trying to be a pedant, just sharing what I’ve learned over the years.