FWIW it's not just Americans - we're very much the same here in Australia.
Speaking for myself, the main reason I don't like soccer is because it's interminably boring. Long, long periods of tedium punctuated by someone scoring the occasional goal.
There's likely a few other issues too: for example, in a lot of ways the game has an image problem. Many people who don't know the game associate it with hooliganism and riots at matches, more so than pretty much any other sport.
Time differences make it hard for international TV audiences to watch European matches live too, which will hurt its popularity in places like the USA.
And then yes, there's the diving. Personally I find it to be the best thing about the game, but that's just because it injects a bit of amusement into the experience. It makes a joke of the game if you're trying to take it seriously, and I can't think of any other sport where diving is embraced so widely.