Should Brady be worried?

Now most of New England gave a collective gasp when they saw Tom Brady go down in game one of the season. "We're doomed, we'll never make it to the Super Bowl now". Many thought. But in walks this Cassel kid. Now I know we've only played 2 games, but we won both times, without Brady. I for one would LOVE to see the Patriots go all the way with Matt Cassel. Even if we win say 70% of the games with him, I say that's reason for Brady to worry. Maybe he's not the only golden boy of the Pats.
Good Luck Matt!