There is definitely a coordinated push to label the trans discussion as a left/right issue. The fact is that the trans lobby may be part of the left, but it does not own the left. Feminist TERFs, LGB folks, and people who prefer to live in reality are all part of the diverse left.
The feminist TERFs who question the eradication of sex-based rights under trans ideology are not conservative or fascist. Feminists generally don't get along with conservatives or fascists and vice versa. The most famous of this group is JK Rowling who was beloved as a promoter of LGB rights prior to standing up for the sex-based rights of women. These feminists come from the left. They remain part of the left no matter how much trans activists state otherwise. Feminists rightfully call the redefinition of the word "woman" by men, the invasion of female spaces by men, and the erasure of the meaning of womanhood by trans activists what it is: regressive, sexist misogyny.
Increasingly, we also hear complaints about transgender ideology coming from LGB folks. Lesbians are increasingly vocal about straight trans identified males calling themselves lesbian and insisting lesbians accept them in their dating pool. The lesbians are rightfully not interested in people who label themselves "chicks with dicks." Lesbians reject the insistence that they date biological males simply because those males label themselves as women. Lesbians rightfully call this trans behavior what it is: homophobic.
There is also a lobby of folks who like to live in scientific reality. These include academics, biologists, medical students, atheists, and performers such as Richard Dawkins, Sam Harris, Bill Maher, Dave Chappelle, Ricky Gervais, Macy Gray, and Bette Midler. These folks are not conservative. They're not fascist. They dislike being intimidated. They dislike being silenced. They dislike the infection of our culture with ridiculous nonsense like pronoun declarations. They dislike the anti-intellectual, anti-factual, anti-scientific claims that "sex is a spectrum," "sex is a social construct," "sex is a label assigned at birth," "trans women ARE women," and "women are whoever identifies as a woman."
The groups opposing gender ideology are diverse. The gender critical is a broad movement that's supported beyond just conservatives. The supporters from the left shouldn't be mislabeled as coming from the right. That is factually incorrect.