Anyone have swine flu or regular flu in very early pregnancy?

I am approximately 5 weeks pregnant (from last menstruation). I am happy but worried as this is a terrible time to be pregnant with H1N1 flu going around. My doctor tells me to avoid the vaccine until after first trimester. Meanwhile, I am so scared of getting it and driving myself crazy worrying. I know high fevers early on can cause miscarriages and birth defects. I would love to ear from someone that had it in early pregnancy and what the outcome was. Specifically H1N1, but the regular flu as well. Thanks!