How could I have been away from her all day? And worse, how could I have actually enjoyed part of it?
But I also liked being back in the newsroom, having conversations uninterrupted by diapers or spit-up and getting back to work.
The guilt was overwhelming.
That was more than a year and a half ago and now, as I prepare to go on leave with my second baby, I don't expect my return in a few months to be any easier.
Every working mother is intimately familiar with the darkest depths of mommy guilt.
Women and their families have made a lot of progress in the workplace during the past 30 years. Paid maternity and paternity leave are more common. More companies allow telecommuting or flex time. And this year women outnumbered men at the office for the first time.
Yet the guilt endures.
"Honestly, the one thing that has not changed is this persistence of mommy guilt," said Suzanne Riss, editor-in-chief of Working Mother magazine. "I would like to help more women have more realistic expectations."
A recent survey by the magazine found that 57 percent of working moms feel guilty every day. Another 31 percent feel guilty at least once a week.
Women pile on expectations. They want to excel as wives and mothers, as nightly chef and chief bedtime storyteller. They want to achieve a meaningful career and provide financially.
Balancing all of those things often means that on any given day at least one isn't getting the attention it deserves. Open the guilt floodgates.
"Women are just feeling the pressure heaped up on them," Riss said. "Men do not feel that kind of guilt."
Mommy guilt bruises the mommy ego. I know I won't be there for every skinned knee, may get home too late for bath time or miss little joys like that first solo plunge down the playground slide.
Working mothers must also focus on what we do well. We set good, solid examples that show our children the importance of family as well as the value of contributing to our community through our work. A mother's career, says Riss, who has a 5-year-old son, often becomes a source of pride for children.
But attitudes that women with families can't cut it persist. Pennsylvania Gov. Ed Rendell demonstrated that when Janet Napolitano was nominated as Homeland Security chief.
"Janet's perfect for that job," Rendell said. "Because for that job you have to have no life. Janet has no family. Perfect."
How often do you hear a man evaluated for a job based on whether he has a family?
For most moms the ultimate question is what is best for my family? Every woman has to make that decision for herself.
I'm grateful for a supportive husband and know that it's our teamwork that helps us both to be good parents and manage careers.
When I return to work, it may not be any easier than the first time, but I intend to try to keep this in mind: Guilt isn't all bad. It's a sign of how much I love my family, care about my job and how fortunate I am to do both.
Beth Kassab can be reached at email@example.com or 407-420-5448. Read her blog at OrlandoSentinel.com/thebottomline.