I probably wouldn't go. If your neigbor said that to me, I would feel like my child was not totally welcome and based on circumstances, I would either get a baby sitter or decline the invitation because I don't have baby sitter.
From the other perspective, if I invited someone over to my house and I knew they had a young child, if I know the child is coming, I would just prepare my house best I could to accomodate the child, not
expect my guests to bring a playpen. If the child got unruly while they were there, I would certainly say something, but IF I know the child is coming, I would make reasonable accomodations for my guests child. I don't think it's polite to invite over people that have children (unless it's clearly an adults-only gathering) and make them feel their child is not welcome--in that case, maybe you shouldn't be inviting them over!
Sorry, this got longer than I intended...
