What do I do about this jerk of a guy?

Ok so there's this guy "Greg." and we used to be best friends. We hung out all the time and eventually ended up dating for three weeks. But the entire relationship was a complete and utter failure because he was the kiss on the first date type and things progressed from there, but we mutually ended it before anything serious happened. So I did my best to try and be friends but he kept telling people the secrets I had told him while we were dating. I told him to stop and I did my best to avoid him for the next few months. But he wouldn't leave me alone. He would always work his way into my group of friends and when we went on a school trip over summer break he kept being mean to me and following my group around and we always "happened" to be in the same place. Then I really started to like one of his best friends and his friend asked me out. But "Greg" told him he had to dump me because me dating him broke the bro code. and whenever we tried to have a date "Greg" would come along with his new girlfriend. So his friend ended our relationship. Then when greg and his girlfriend broke up he came to me and said "so which one of your best friends is single?" On top of that he's just really hurtful all the time and he makes me do physical punishment like pushups (he's my section leader in band) and if i try to get up he's actually pushed me to the ground before. He always goes out of his way to make fun of me and make my life miserable, but I can't seem to get away. I've had friends try to tell him to be nicer to me but all he does is blow it off and I've tried talking to him and what he said was "I don't hate you, it's just fun being mean to you, I'm only messing around because I know you can take it." But the thing is I can't take it and I've tried telling him but he won't listen! How can I get him off my back?