I think the thing that killed it for me with the first game was that you were confined to one dungeon and a town.
Now, that was okay with Diablo because it was an older game and was still experimenting quite a bit to where a one-area dungeon type of game seemed like a good idea to them.
Fate, however, came when Diablo 2 was already out, so them copying the idea but changing it into a "cave" makes it seem like a
complete Diablo clone with some added features, to where it seemed they were like "Oh, you liked the first Diablo? Well here's our version, and it's much better."
And granted it did have some good ideas such as having the companion pet and, as you said, fishing. But really?
And that's probably why they opted for a more cartoonish look for D3 rather than the gritty-hardcore realistic type of looks that they were building up in 2. Torchlight became pretty successful to where it was considered almost a spiritual successor to the series despite them being completely different games. So I imagine the makers of Diablo saw and capitalized on the fact that people loved the graphics especially, and thus Diablo III was made in that style.
Personally, I didn't mind the graphics of D3 in the slightest, but I know some people were pretty disappointed in the fact that it was more cartoonish than realistic, which they considered a "downgrade" compared to 2.