No, it doesn't sound stupid - I felt the same way. It really hit me when I finally had all three of my girls in the same room and the were just sitting around the table coloring with the youngest (they are 23,14 & 3) I couldn't shake the "this is the way it should have been" feeling...it was quite depressing.
I think we need to allow our selves to grieve, if we don't we will forever be consumed with it. For me, for the most part, the grieving has subsided - I still have my moments...but it is important and honor thoses things and moments that we can not go back in time and recapture.
You're not alone, it's perfectly normal, hang on to what you do have and build on that.
