I loved X-Factor until they did Secret Invasion crap. Then I dropped it. And thankfully I did so I didn't get dragged into another X-Men mega event. But up until that point, pure gold
You missed out. I think it was 2 issues, maybe 3? After that point I'd say they kicked right back into pure gold territory again.
Although I will say that the really disappointing part of that tie in was the not so triumphant return of Larry Stroman. I used to love his stuff, but the years away, they weren't kind.