Possible Clues:

• Really bother
• Anger
• Bother the heck out of
• Make mad
• Really get to
• Pique
• Get under the skin of
• Chafe
• Bug a lot
• Try the patience of