The Orsay Museum can be a surprisingly fun place to visit with kids. Have a child who loves ballerinas? Hightail it to the Degas exhibit. Toddlers tend to love sculptures, and the second floor sculpture exhibits are no exception (they're also less hushed than the rest of the museum). Kids also tend to enjoy the miniature stage sets that are part of the Opera Garnier models on the bottom floor.
Need some space to move without the crowds? Head to the old Grand Ballroom that remains from the elegant hotel that used to be in this building - toddlers will love hiding behind the various pillars and while this isn't a place to run with wild abandon, it is a great place for young children to stretch and explore.
Tip: If you have a stroller-aged child you can skip to the front of the line - just ask a guard for help.
Warning: Neither of the museum's cafes are particularly child friendly. One is quite elegant and the other has only bar-height seating. Consider eating before or after your visit.