I know where you are coming from! My father also was really not thrilled with the idea of my husband and I adopting.
My DH and I have one biological child, and were advised to not have more due to some complications, ongoing health stuff, etc. ANYway...we had one child, age 7, when we started into the adoption arena. My father was very concerned because not only were we adopting, but we were adopting from the state foster system, and the child would not be a baby, and the child would, in all likelihood be special needs.
His fears were motivated out of care for my Dh and I and our Dd. I know that he was concerned that we not be hurt. But, as I'm sure you've realized by now, parenting can be very painful in many ways! I am able to speak from experience here, whether the child is biological or not!
This situation was difficult for me because I wanted to honor and respect my father, but I didn't agree with him. My Dh listened respectfully to my father's concerns, but then also made it clear that this is where we believed God was leading us.
I believe that while we are to give our earthly parents due honor, we are to give obedience to our heavenly Father above all others. I was also abiding by my husband's decision as the head of my family. My husband and I were making decisions that involved "leaving and cleaving" unto each other, ("...a man shall leave his father and mother and cleave...") so that my husband making these decisions with my cooperation and support was not being disrespectful or dishonoring to my father. It wasn't always easy or stress free, but it was not wrong to pursue adoption.
When looking at having children you also must remember that all children come from the Lord, whether through adoption or birth. God is the one who controls the womb (look it up in your concordence and you will see that this is often referenced throughout the Bible). God also controls where children are placed adoptively. I've known couples who have the desire to adopt, have completed homestudies, and are still waiting for a child nine years later. God is in control. Just because you make yourself available doesn't mean that you will have a child placed with you.
Our story has continued on and my husband and I have one biological child (aged 10 1/2), an adopted child (age 5) and another adopted child (age 2). Both of our adoptions are "special needs" involving neglect, and prenatal exposure to drugs and alcohol. My father has accepted and loves without reservation both of these children that God brought into our family through adoption.
Has this been hard on our biological child? Well, yes...but I think that having siblings requires all children to learn to not be self centered, to learn patience, to grow in grace, to practise mercy, and on and on. As we all know, learning patience and mercy and showing love can be downright painful at times. It isn't easy to learn to die to self, is it? Do any of us have an easy time giving up what we want right now and allowing someone else to take what we don't want to give? However, this is what all Believers are called to do in Christ, so while at times this has been painful for our biological child to go through the fruits have been good and blessed.
Unless the Lord builds the house, they labor in vain that build it. Adoption is not an easy path to take, but if this is the path that God is leading you to take it is the best path for you to be on.
We are in the process of working toward a possible third adoption. It still isn't easy, but I'm learning (a little at a time!) to rest in God's promises and to wait upon Him, and to lean NOT on my own understanding.
Hope this is of some encouragement. I'm not sure if I was clear in all that I wrote, but not only do we have 3 children we also now do foster care (yes, God leads us in places we don't always see ourselves going!) and I have 2 extra children right now. My brain doesn't always put things together on paper as well as it used to!