Hi - I managed to get a PayPal button on my WordPress site. Don't ask me how I did it though! It took a while but I think I just trawled through the help pages on WordPress. Here's my site: http://thewanderermagazine.wordpress.com/
I used Wordpress because it has advanced features and lets me customize. I knew I was willing to put a small budget into it and needing something I could have multiples levels of things going on.
There is a difference between Wordpress - dot - COM and Wordpress - dot - ORG. Wordpress.com is free and will have restrictions but .org is the site to go do if you have your own hosted website. That's where the advantages of Wordpress really come in.
Check out my site at www.blogityourself.org - I was able to find a theme I liked online and then modify it to suit my tastes. I bought hosting through Dreamhost and they have what is called a "one click install" where they install the Wordpress software for you into your hosted site (so you don't have to).
Blogspot / Blogger is easier if you just want something simple. But didn't folks used to use MySpace for that?