Hi Tod--An interesting question. Let me first state that I had actually not meant to imply that iambic pentameter was necessarily the one great meter that was universally pleasing to all ears. That would be a silly argument indeed, and I have also experienced great pleasure in hearing tetrameter, hexameter, and any number of other poetic lines both measured and free. I was trying to explain to someone who has little knowledge of any of these why English meter is of any interest at all, and what I should have said was that iambic pentameter is among those meters which are pleasing to the ear, or better yet that it is an example of one kind of meter and that meter in general is employed as a tool with which the poet controls the sound of his poetry and, through sound, the effect it will have on the reader/listener.
That said, let me try to address your question--I think an incredibly good one-- as to why this meter has been so hugely influential in English. This is something I've pondered a fair amount without any claim to having a definitive answer, since I'm not sure there's ever an absolute single right answer for explaining cultural tastes. Here, however are a few takes on the issue, all of which I think are contributing factors:
1.Obviously one could similarly ask why so many song lyrics employ tetrameter. On a certain level pentameter became popular, just as I stated above, because the sound of it was appealing to enough people that it caught on.
2.Most art forms, poetry being no exception, seem to develop some sort of forms and guidelines somewhere in their evolution, which artists either employ in an attempt to make their work more effective, or (after such forms have become too standard) rebel against in order to make their work stand out. For English poetry one of the major formal features that poets have followed/rebelled against is the iambic pentameter line.
3. But of course you are asking why this particular meter was chosen to function in this way. As a scholar of the late Medieval and Renaissance periods—the era when iambic pentameter was really established as the meter of choice for English poetry—I can try to give an answer from the historical perspective. There was in fact a fair amount of debate about how poets should go about writing during this time period. In England this occurred especially in the sixteenth century (just before and during the age of Shakespeare). The poets of this period went about trying to define what would make their poetry work best and experimented with different approaches to using meter, rhyme, etc. One idea was to try to apply the classical method of scansion (the method of the Greek and Roman poets) which depends not on stressed and unstressed syllables but long and short length syllables. This works pretty well with Latin, but (I can tell you from attempts I’ve seen) did not work well at all with English. What emerged was an adaptation of the classical system reliant on stresses to suit the modern languages. Anyway, having read a fair number of early experiments in English verse, I can tell you that iambic pentameter did really stand out as one of the better options, and the best adapted to the English language (much better than the often rather clumsy “fourteener” that was popular for some time—though it has its defenders). Iambic pentameter was elected the meter of choice for some of the best poets of this period, who in turn had a profound influence on the works of subsequent generations of poets.
4.There have actually been some studies done that iambic pentameter is the poetic meter which occurs most often naturally in day to day speech in English. I think Shakespeare demonstrated the potential of this meter to sound “natural” beautifully in his plays. His use of blank verse flows so easily, that most students first coming to Shakespeare don’t even realize to what extent he is sticking to the meter.
5. One final note on Tetra vs. Penta : I agree with you that tetrameter is very well suited to songs (and pentameter not so much), but I'm with Blue that it doesn't flow as easily as pentameter in poetry. Tetrameter tends to really make the rhythm of the poem stand out. This can be a good thing in providing a poem with a real driving rhythm, but it can also come across as a bit “sing song” if not handled right. I don’t claim one is better than the other, just that they suit different types of poetry.
Well, you can tell I could discuss prosody all day, so I’d better stop before I bore you to death (though I may already have acheived this)! There’s still lots to think on in your question. It's intriguing to contemplate what draws certain cultures to certain forms. The above discussion makes me wish I knew more about Chinese poetry (not to mention Chinese language) so I could compare a very different system to the European forms I'm familiar with.