Robert pardoned Jaime for murdering Aerys II. Brienne knows the circumstances behind Aerys' death. And like Brienne, we respect Jaime's decision and feel sympathetic towards him. His quick thinking and decisive action mark Aerys' death as one of the most heroic moments of the ages. And yet, if Dany gets her hands upon Jaime, he's done.. along with anyone who approved of his royal pardon, i.e. Pycelle and Varys.
Jaime's loss of his hand has been frustrating for him, to say the least. Maiming him was cruel. But compare Jaime at the end of AFFC to Jaime up until he met Hoat.
In ACOK, Jaime confessed to Catelyn that he loved Cersei... that he'd only ever been with her... that her children were his... that he'd have murdered Robert without a second thought... and that he defenestrated Bran to conceal all of this. He rarely stopped to think because he could always rely upon his sword and his father's power to bail him out of all situations.
By the end of ADWD, Jaime had lost his hand and his father. He had to think and plan for himself. He negotiated the surrender of Riverrun, kept his oaths to Catelyn, arranged protection for Sansa, sorted out Harrenhal, ended the Blackwood-Bracken feud, ordered the Freys to hand over their hostages, and presumably ended his relationship with Cersei.
With his hand, he was a prisoner in Riverrun. Without his hand, he is the conqueror of Riverrun.
Though physically agonizing, socially shameful, psychologically scarring, and potentially career ending, the loss of his right hand has forced Jaime to grow up. Without his hand, he's more of a whole person.
He's taken on more responsibility at work. He's become more accountable in his love life. He's striven to keep his oaths.
Chopping off his hand was meant to inhibit Jaime, but it has really spurred along his mental maturity.
Granted, he has not changed over night. He recently threatened to kill an unborn baby. And he's still supporting an illegal regime founded upon his own lies and treachery. But he now has a glimmer of hope for redemption... and that is only because Zollo, a fat Dothraki, chopped off his sword hand.