Placing the full-height quotation mark on the outside protected the smaller cast metal sort from damage if the word needed to be moved around within or between lines.
It has the same shape as an apostrophe or single closing quotation mark in many typefaces, but it differs from them in being placed on the baseline of the text.