You are not alone. DH and I have been married for 2 1/2 years but we are both 42. Neither one of us has ever had children. We are pretty set in our ways and used to being a little selfish. So yes we are concerned about the same things you are. On the other hand we have talked about the fact that we have had lots of time to ourselves and even if we enjoy that down time in the evening or weekends..... we are missing out on a huge amount of joy.
It may have taken us longer than others to get to this point but we are now ready to share our lives and move to another chapter of our lives. I think it is important to know how you feel (such as you do) and not fool yourself. You will be more prepared I think than the person that does not even think about it.
We don't have a child yet so I don't know how it will be but I will say we know how you are feeling
