You can. Your downvote are not federated though. That means your instance knows about them, but can’t send it to lemmynsfw.com since it doesn’t want to know. Consequently no other instance will know about your downvote since they ask lemmynsfw.com for who voted and who didn’t and not your instance.