Judge Pyncheon is certainly a sinister figure in The House of the Seven Gables, but in this encounter with Phoebe he moves from threatening to more ambiguous to even perhaps benign. Jaffrey is most threatening when he attempts to appear friendly, for it is here where he lays most bare his threatening character and seemingly malevolent intentions. When he smiles at Phoebe to soften his imposing appearance, this smile appears insincere, the attempt of a man to produce an appearance of cordiality where none exists. Phoebe instinctually draws away from the Judge when he approaches to kiss her. This kiss should appear as the most offensive action that the Judge undertakes toward Phoebe, presumptuous and inappropriate, yet it is here that Hawthorne presents the Judge at his most sympathetic. He explicitly states that this was an action of "acknowledged kindred and natural affection," essentially excusing the Judge for this action. The proud man even appears absurd; it is this embarrassment that makes him for the first time a recognizable human. In response to the kiss, the Judge subverts both Phoebe’s and the reader’s expectations. He becomes stern once more, but soon becomes amiable.

When Jaffrey first appears offended by Phoebe’s refusal to kiss him, he manifests those qualities of Colonel Pyncheon. Phoebe recognizes that the daguerreotype that she mistook for Colonel Pyncheon in modern dress was actually Judge Pyncheon, creating another link between the two generations. This connection between Judge Pyncheon and the Colonel leads Hawthorne to develop the idea of recurring familial qualities. He finds that the connection between the two men implies that weaknesses and moral diseases can be passed from one generation to another. Judge Pyncheon therefore represents the sins of his ancestor, a claim that Hawthorne bolsters with his extended list of qualities that Judge Pyncheon and Colonel Pyncheon share.

The suspicion that Phoebe shows of Judge Pyncheon when she refuses to kiss him soon becomes justifiable when he demands to see Clifford. Although he claims to have an affection for his cousin, his insistence that he must see Clifford becomes threatening. Although he does not yet explain the reason for this aversion, Hawthorne establishes that Hepzibah and Clifford fear the Judge.

Hawthorne often refers to Jaffrey as an "honorable" or "excellent man," bestowing positive characteristics on the Judge. However, these qualities do not refer to the Judge’s personal qualities, but rather the perception that the public has of Judge Pyncheon. The praise that Hawthorne lavishes on Judge Pyncheon relates only to external perceptions and reputation, rather than to the actual qualities of the man.