A couple of things I've heard !Smalltalk people say killed their market:
1. Smalltalk's image-based system didn't mesh well with OS filesystems;
2. #Smalltalk vendor turmoil and their high pricing.
(I wasn't even aware of the language's existence until the late #1990s. I took numerous programming courses over the years, but I never heard ST mentioned until another student mentioned it in a #Java course. )
CC: @clacke@libranet.de