Problem for Defense is when you provocate an encounter you could give up your right to self defense. I'm afraid they could say following him and brandishing a weapon could be seen as that.