I completely understand. A friend of mine would do the same thing... even in the middle of telling a story. He would casually place his hand on my arm or back but I never really told him it was ok. I eventually brought it up to him and it turns out he was doing it unintentionally and apologized but I wanted to make sure he wasn't getting the wrong idea! He never would have known if I didn't say anything!! You should definitely bring it up!
YESSS! This happens to me all the time. I'm glad someone finally mentioned it!! How do I bring it up without making it awkward? And I'm sure he doesn't mean anything by it....
How can you be sure though? I know we want to think the best of our friends and maybe he really doesn't mean anything by it but what if you don't say anything... what kind of message does that send? That "it's okay to touch me without asking", that "you can put your hand on me because I'm not going to stop you". Even if there is an awkward conversation, it could very well save you in the future.
LovingAllLife2Love makes a good point... and what if some of his friends see him and notice that you don't do anything and just let it happen. That could send the seriously wrong message to them. Then it's not your friend you have to worry about but all the people that look at you as just an easy target who won't speak up.