I've done a bit of reading up on this, and I don't see that having E-junkie buttons or links on your site would make any difference.
Basically, rather than a "penalty" on EMDs per-se, Google has simply reduced the significance they formerly granted to an EMD in calculating their Page Rank for a given search term. Previously, having a domain that contained keywords would improve your ranking in search results for those same keywords -- e.g., in searches for, say, "foo bar zot", a Web site with an exact-match domain like foobarzot.com would get a boost in those search results. However, now with this recent change, that's not as much the case anymore, which makes other factors more significant in your site's search ranking.
I think "content quality" has more to do with how many other sites link to your site -- i.e., how many links you have to your site from other "quality" sites. A key aspect of Google's ascent to dominance was their realization that that links between different sites could indicate the popular "quality" of the sites being linked to -- e.g., if Site A links to Site B, Google regards that as a "vote" for the quality of Site B, indicating that Site A thinks Site B is worth linking to, and the keywords in that linked text will give Site B a boost in searches for those same keywords. Moreover, links from a "quality" site (one which many other sites link to) carry more weight than links from a "low-quality" site (one which few if any other sites link to) -- e.g. Site A's link to Site B not only boosts Site B's search rankings, it also boosts the rankings of any other sites that Site B links to.
To improve your site's Google-perceived "quality", try to get more links to your site posted on more other, popular sites ("popular" meaning other sites also link to it, which a high search ranking would indicate). You might look for businesses that are complementary to your own without being direct competitors and contact them to propose a mutually-beneficial link exchange (they link to your site, and you link to theirs), or you might look for blogs in your industry and contact them to propose writing a "guest blog" entry on their site which would link back to your own site.
