A bum experience in the park?

Most days that it's not raining or freezing, I like to go to a park downtown. There are fountains, waterfalls, and a reflection pool. Now that it's cooler, there aren't quite as many people around. I like to go and reflect and write. I went to do that twice this week, and was interrupted both times by deaf panhandlers. The first time was no biggie. I had a little change on me, and felt as if I should give, so I did. The second time I went, however, this next deaf panhandler was very drunk. I could not understand what he was trying to tell me. I thought he wanted to sleep on the bench in front of me, and I nodded okay. Next he layed down on the ground and proceeded to slide under the bench at my feet. I found this a bit unnerving, but tried to keep going with my time. But, I couldn't. I kept peering down at him, because I didn't feel safe. I felt like he might grab my foot or something, I was trying to see if somebody was looking for him. I tried to show that it wasn't bothering me, though. After about 3 minutes, he started to slide out from under the bench... on my side of the bench. He started trying to signal something to me again. I'm not sure that he even knew sign language. I was feeling pretty uncomfortable about it all. When it was clear I had no idea what he wanted, after a couple of minutes he walked away slowly, but kept looking back at me. I saw him gesturing to someone else in the park, that he may have known, and kept looking at me. I prompty left. I know, big, tough, rocker. Well, I don't know if I did the right thing or not. Should I have tried to communicate with him, tried to listen to him, and be a friend, rather than shaking my head, shrugging my shoulders, and hitting the road? I guess I still don't know...