As you polarize prepare to feel the tug of insecurity. It's a big scary moment when people begin to understand who you are and what you believe in and they start to either pull away or draw closer.
Don't mistake the shift for what it is. Your aim is loyalty. Your target audience is those who believe what you believe. By it's very definition the exclusivity of an authentic following means that you will lose some "accidental" followers along the way. Those were the clients who were in it not for what you uniquely have to offer but rather for what you tempt them with, the manipulations rather than the inspirations.
What many brands get wrong is trying to "be all things to all people". If you are focused on trying to be everyone's cup of tea then the chances are that no-one will feel strongly about your particular variety of tea, which generates a "take it or leave it" attitude and again forces you to use manipulations to drive sales.
Take Marmite as an example: Marmite has never attempted to appeal to everyone. In fact the slogan "Love it or Hate it!" was cunningly used to polarize their brand even further. They created a community feeling which as social animals we are drawn to. Marmite lovers belong. They defend their common taste and feel a kinship with those who share it. This form of extreme polarization is hugely beneficial when creating an authentic brand.
Give your followers the feeling of belonging to something greater than the individual. Let them know that you and your fellow followers believe what they believe. Give them a sense of community, a cause, a movement. Don't aim to be accepted as "garden variety" or "dime a dozen".
Stand out! Be exceptional! And accept that this also means that some won't want to be with you on your journey. That's ok. Those who chose to be with you know you, love you, value you and will defend you.