Well argued article, though I disagree with how much you are equating conservatives with "TERFs"--a term used far too much as an insult. The quote above is an accurate description of a feminist thought, and that thought is, in my well-considered opinion, undeniably true. I don't know what it is like to be a woman because I have not grown up as a woman under patriarchy. Same with not knowing what it is to be Black, gay, a victim of colonization, and so on. That reality facing all women matters and it matters A LOT.
That said, I side with the rest of your argument. We do need to have a very open conversation about gender identity, and that has to include perspectives of multiple sides. I had written my own radical proposal last year: https://medium.com/inserting-philosophy/how-about-we-just-get-rid-of-gender-labels-altogether-7190b234eeb7