Atheist parents don't really teach atheism, they just don't take their child to church and they don't tell them to pray. It's more of an absence of God rather than teaching against God. (I'm not atheist, but a very good friend is)
For the first few years, the baby isn't going to have any concept of God or religion, anyway - so don't sweat it! This is an example of just living what you believe and letting that be the teacher. If your sister feels you're forcing your believes on her child, that might cause problems. Live what you believe, love the baby, and everything will be okay!