And you won't have as much endurance and stamina as you would at low altitudes.
A short vacation won't affect the baby.
But over the long term it can. Women who live at a high altitude tend to have slightly smaller babies.
In fact, places like Denver actually have their own growth charts.