The one thing I dont see in the OPs post is the ecological effects that removing whales causes.
There is a balance that needs to be maintained in the ocean and whales are apart of that, whales eat krill and plankton and a number of other marine species.
I dont think there has been any studies as to the effect taking whales out of the equation would do to the oceans balance, while whales aren the only species to consume such things they do consume alot of it, so its hard to say what an overabundance of krill and plankton would do to the oceans, especially considering all the other damaging effects humans have had on them already.
So long answer would be no, let the whales be, its not worth fucking up yet another ecosystem just because a few thousand people might suffer. Besides people suffering is a long human tradition, if whalers are too wussy to fight through it and adapt then they dont belong in the gene pool.