In a word: maybe.
In a lot more words: I worry a lot more about what death will be like than the fact that I am going to die. It doesn't make a lot of sense to 'worry' about a known fact that you can't do anything what-so-ever to change or even really appreciably delay. On the other hand... the unknown of "what is death, what am I after death?" is a huge worry - when I think about it, which I try not to because I realize I am not going to get an answer of any concrete nature by thinking it through rationally or even emotionally.
As I see it either A) there's nothing after death, you cease to exist - and what will non-existence be like?! I can't fathom and it is pointless to try to imagine what it would be like not to exist, since you won't even have an imagination when you don't exist so... futile there. Or, B) there is something after death - in which case you have a myriad of options to consider, reject, accept, or invent. Some of them are more appealing to me than others - like a nice luxurious limbo land of "wait here while we assign you a next life" before doing this again or a cloudy city full of what I can only imagine are very bored but content people adoring their ultimate creator for all eternity as their highest reward for spinning the wheel of religion and landing on the right peg... this, however, has an upsetting side due to the fact that, should we fail to win the saving throw vs. being born into the wrong region/family/situation to have lucked into the right religion we're going somewhere considerably less pleasant as an alternative. That, and the fact that even if we did 'win' some of our friends won't have been so lucky.
So, yeah, there's a good bit to be worried about regarding death in my opinion... but I agree with many of the above posters who said we can't spend life worrying about death, because that would be to waste what we have being worried about the day we won't have it anymore - a day we can't do anything to prevent happening anyway.