I admit that I too was mesmerized by Sheets' potential. Signing him, however, would have definitely been the wrong move.
We already signed an injury question mark (AJ Burnett) this year. We signed an ace who lead both the AL and NL in complete games last year, and leads the universe in innings pitched the last 3 years (CC). We have a young pitcher who has never thrown more than 100 innings before in the majors already slotted into our rotation (Joba). We are counting on another starter who made only 15 starts last year (Wang).
We NEED someone who we can count on to throw 200 innings, and who knows what pitching and winning for the Yankees is all about. Pettitte will be the guy that ALL of these pitchers can go to and that Joe G (and we as fans) can count on to take the ball and compete every start.
While Sheets was sexier, signing Pettitte was the right move.