Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I am about 90% the way ridding a larger company of Kinesis. You might not have needed a Kafka expert but instead you had to create (likely poorly implemented) frameworks for consuming from Kinesis (when you could have just used Consumer Groups). Also Kinesis is insanely expensive, to the point that if your throughput is anything but trivial (in which case it doesn't matter which option you use, both will -just work-) that is just becomes untennable.

I actually did miss something but it's not complexity, you can easily use hosted Kafka (AWS even provides it with MSK now). It's actually authentication. Kafka does have authentication but it doesn't easily tie into AWS or GCP authentication mechanisms without the help of a tool like Vault.

That said.. I don't think that minor benefit is enough to ever justify Kinesis over Kafka.



> Kinesis is insanely expensive

No, it is not. Also, an equivalent cluster of EC2 instances running zookeeper/Kafka/etc will outprice Kinesis.


At tiny throughput maybe? Pretty sure you could run Kafka on small ec2 instances and still get great throughput, especially if you don't need much retention. (reducing retention reduces need for disk I/O as consumers can't fall far behind causing disks to seek).

I think for any reasonable workload, i.e 10k/s+ and/or throughput over 100mb/s Kinesis gets dumpstered for price every time.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: