
Quotation marks are designed to fit against the letters within them.
This problem came up in a book I was working on recently. The author italicized a lot of words, but didn’t include any of his punctuation in the italicization. Put another way, his words were italic, but his punctuation was roman. It left a lot of visual crashes. As a tall person who’s constantly bumping his head, perhaps I’m extra-sensitive. But it hurts nonetheless. It’s more obvious with some letter pairs than others, but letters either crowd each other or leave gaps big enough to look like a letterspace. It’s ugly. And it’s fixable.
It’s like seating a left-hander next to a righty at Thanksgiving dinner. There will be jostling. Or jousting. If you’re a lefty, you already know to fix that mess before the elbow wars begin: pair up correctly.
Good type designers go to great lengths to make their letters fit together gracefully, and that includes the punctuation. But all that design effort presumes the letters are in the same style. Opening quotation marks that are in italics ootch over to the right a bit because they presume the next letter will be leaning to the right as well, and they want to nestle up close. Closing quotation marks do the same, because the letter preceding them is leaning into their personal space, so they lean away from it. With quality fonts, the fit is a beautiful thing to see.
So the short answer is “Yes!” As opposed to “Yes!” (See what I did there? See that italic exclamation point topple into the unitalicized quotation mark?) Any punctuation attached to a word should be of the same style.
Seems easy enough. When I have a usage question, I head straight to the experts, like the Chicago Manual of Style. But wait a minute! Even these exalted sources disagree. Some even disagree with themselves: do it one way in this case, another way in that.
No. No no no no. Easy rule: when setting italics, select everything the word touches. Personally, I even select the space following the word before setting the style to italics. Like this. Because sometimes it matters. Get in the habit.
Parenthetically speaking
It gets worse with parentheses, because they’re so tall. They start below the baseline and extend above the capitals, so the effect of all that leaning is exaggerated, like this:

Parentheses get really crashy if they don’t match the style of text within them
Use that rule for all punctuation. Except maybe one exception. There’s always an exception.
Dash away all
I tested dashes in a lot of fonts to see whether the dash itself moved to the right when italicized. Turns out it doesn’t much, if at all. Dashes fall at the waist of a leaning letter, and the gaps tend to appear only at the head and foot. The waist is in the middle, like the fulcrum of a teeter-totter. So no worries, right?

The first dash has square ends, and the italic dash has angled ends. Rats.
Wrong. You have to look really close, but in the example above, the italicized dash (second one in the bottom example) has ends that are slightly angled to match the italicization. Subtle, but handsome. Since the spacing between letters doesn’t really change with italicization, it’s better that your pairs of dashes match each other. Kinda weird to have a square dash followed by an angled one.
I suppose you could argue that this isn’t really an exception, because a dash isn’t attached to the words next to it—it is an independent character. But if you get in a drunken brawl over that point, I’m not going to step in and back you up.
Actually, if I’m typesetting a book and someone notices that one of my dashes is italic and the other isn’t, I’ll buy them a beer and thank them for caring.
Oh, this was great! But it wasn’t the answer I was looking for. As a perfectionist, now I have to go back through my manuscript and fix everything! Arghh.
(But thanks nonetheless!)