As Russell pointed out it’s all about leverage. Some people are concerned about what their nieghbors might think. Others my be more concerned with what you might do to them or property.
You also have to determine what it’s worth to you. How are you willing to push the issue?
I’d try to reason with the guy atleast once more. He may come around after having sometime to think it over. Then escalate as needed.
Also, the guy just might not have any money to return.
Have fun. Do whatever makes you feel better about the situation without getting yourelf into too much trouble. Anything that could lead to life sentence over this would be going waaaaaaaay too far!
P.S.
Try to minimize how much info about yourself you give to him.