Once again, it comes down to diversity of link types. If all you have is forum links then it will not hurt you but they value is discounted. For blogs, it depends on the blog. If it looks spammy or all you have is blog comments then those would be discounted. It is not that you are penalized more than it is a matter of diminished value of the links.