You can be against authorities limiting your freedom without thinking other people’s rights also limit your freedom. It’s a spectrum. I don’t know what it actually means to be libertarian but it makes sense to me that the polar opposite of authoritanism is selfishness. I think it’s the best to not take it to either extreme.