Apparently, some sites have already been seeing a move away from Flash in the past year or two, even if it's marginal (and anecdotal, obviously).
Daring Fireball, a blog about Apple, had 32% of its visitors not have flash-enabled browsers last week.
kottke.org had <4% of their visits coming from non-flash enabled browsers a year ago, 15% in January 2010.
On Lifehacker, who has a much more Windows heavy audience, it was 2.3% four years ago (the lowest point), 6% last month. Lifehacker has a separate mobile site, though, so those numbers don't include smartphone visits, which would raise the non-flash number significantly.
If, in a year, you won't be able to rely on Flash being enabled on 10% or 20% of all browsers, I do think web designers will have to take a look at how they can reach the rest of that audience. Along with Youtube and Vimeo beginning to move away from Flash for video content, I think it's in trouble.
Also, it runs like crap on OS X. Fuck Flash.