Lidar was expensive back then and would've added huge costs to the vehicles. Not to mention, it looked ugly on consumer cars. Elon conveniently used "humans use only vision" as an excuse and promised every Tesla has "sufficient" hardware for full autonomy. It's that premature promise that doesn't let them add sensors even now (and perhaps Elon's ego) without breaking trust and/or eating big costs for retrofitting.
In short, Elon made a high risk bet that vision-only would be enough and so far has been proven horribly wrong. But I've got to say, it was brilliantly executed because it gave Tesla mindshare as a tech company, drove sales and contributed massively to their insane valuation.
Their radar removal was baffling. There were rumors (perhaps Musk's tweets?) of Tesla investigating 4D imaging radars to replace their really old Continental radars, but they suddenly decided to remove it altogether. Many have attributed it to chip shortage and yet again Musk using vision-only as an excuse for removal.
My money is on chip shortage. There were some real problems with the radar (like reflections from overpasses) though. I think not being able to ship cars because not enough radars is what forced this decision though (vs. trying to fix the radar and/or the way it's fused with the vision).
In short, Elon made a high risk bet that vision-only would be enough and so far has been proven horribly wrong. But I've got to say, it was brilliantly executed because it gave Tesla mindshare as a tech company, drove sales and contributed massively to their insane valuation.