It's pretty hard for me to take either Phantom or Raoul being man-whores seriously, it's just not their style.
There is definitely a double standard where Erik's suffering or vices make him oh-so-interesting and attractive while Raoul's suffering or vices are just not important even when exploring this subject further would have made a good story. That was my reaction to the infamous twist in The Phantom of Manhattan - who cares about this bullshit with Erik's lovechild, Raoul got his balls shot off, I want a novel-length angst fic about this!