Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Kagi works by scraping Google and other engines, so this should still worry you.

I still support the use of Kagi though, as a market signal to Google that we'd even pay them for their product if it wasn't dogshit.

 help



For Google, the business side of you paying them doesn't work out as obvious as it might seem.

Advertisers are sold the idea that 1000 clicks/impression is X dollars. The average impression might be priced at 1000/X$.

But you are worth much more than that for two reasons.

- you are a person willing to pay 5$

- Google can sell a “collectivized” product. The same way that eg collective farmer crop insurance creates actual value.

and the third reason (as consequence of the second) they dont want people to think about: it gives them more opportunity to fudge the value by making it all very complicated.


The advert paradox, anyone willing to spend money to avoid ads is inherently worth more to advertisers, since they're willing to spend money.

The more money you're willing to pay to avoid ads the more you're worth to advertisers.

That's why tools like uBlock and YouTube morphe are the only answer, negotiating with terrorists never works.


I'm willing to pay time or money to avoid ads, not to be spied and harassed by companies making me offers to buy something that almost certainly I don't want to buy. Those companies should pay me a percent of what I make them save by not buying ads to serve me.

And eventually when I want to buy something I research it and buy it and nobody had to pay anything to Google.


> That's why tools like uBlock and YouTube morphe are the only answer, negotiating with terrorists never works.

This is silly. The economy doesn't work if no one is willing to pay for goods. The content you're consuming won't be made if theres no market value to create it in the first place. Pirating is not a solution to our data privacy problem.


The internet had a plenty of content before advertising.

I'm not making an argument for advertising.

The capitalist economy is based on the explicit idea that everyone will do everything they can to extract value from everyone else, and it'll come to some kind of equilibrium. If you do not do this but you allow everyone else to do this, you receive worse quality products for ridiculous prices and do nothing about it.

In other words, you owe me $1000 as a condition of reading this comment. Your options are to unread the comment, pay me $1000, or pay me nothing to incentivize me to charge a price you'll actually pay (likely $0.00 for comments but not for everything)


Free riders send a market signal of failure or they cause friction. In the former, the service or creator just shuts down and moves on to something else. In the latter the service must make a choice with very little knowledge.. what should they do? Bake ads into the content? Ad block wall? Subscription? Pay per view?

Basically it doesn't make any meaningful progress towards a better outcome. It just temporarily makes the content more enjoyable to you at the expense of the person who created it and maybe to you if you enjoyed it and wanted more.


I don't understand your logic. Why would someone that never sees my ads be valuable to me? Why would I care? Everyone spends money on something, the fact you spend it on adblockers doesn't make you automatically more valuable to advertisers. I have never heard of this "advert paradox" concept you mention and I have worked in industry for a while now. All this is just conjecture on your part and doesn't really correspond to how it works.

If you have $20 to spend avoiding ads, you have $30 to spend on cool products so the advertiser is willing to spend up to $24 to make you not block ads about a $30 product you'll buy (that costs $5 to make)

> If you have $20 to spend avoiding ads, you have $30 to spend on cool products

Yes, but if you are so against ads that you spend $20 to not see ads (I am definitely in this group) and then see an ad for a "cool product" anyway, then that product is no longer cool. Any time something like this happens, I legitimately develop a grudge against that company and their products.

I might have bought that product before seeing the ad, but there is no chance of that after seeing the ad.


No you don't. Studies always show that you don't. You think you do, but you don't.

Before the ad, you didn't even know the product existed. Now you know it exists, making you infinitely more likely to buy it.


> Studies always show that you don't. You think you do, but you don't.

Did those studies have any outliers that DID have a negative reaction to ads? I am 100% certain I'd react EXTREMELY negatively in a situation like that.

The same way I wanted to buy a Tesla and rented one a couple of times in the past. Now I would NEVER AGAIN even go into a Tesla. Fuck that company.


That does not make sense at all. Why would you care about people who want to avoid ads? The real data shows very little people actually do that. How much you spend on adblocker is irrelevant to your profile for a business. They will get you some other way if you ad block on digital.

Sad but true. I paid for Prime to be ad free, easily worth 3$/month but sure enough now part of their catalog has become "only available with ads" or ads in their FireTV is before you hit the prime app (and then again when you do). So now their whole product is crap and I'm looking for a new streaming stick solution [anyone?] Sounds like it's time for a newcomer/disruptor like Angel Studios to release one.

Natural next step of the ad lobby is obviously to pay the lobby to make it illegal to bypass the money making machine if they haven't done so yet. I don't doubt that capitalism still works, it just takes time to "boil the frogs" dead [enough] to stop them from buying such crappy products.


If they don't want to sell me a deshittified product, I'll continue paying someone else to extract the deshittified product from them. Their choice.

Kagi don't scrape, they pay other engines for API access: https://help.kagi.com/kagi/search-details/search-sources.htm...

>Our search results also include anonymized API calls to all major search result providers worldwide


The only “other engine” that matters is really a thin paid SERP façade in front of a scraped Google result. In other words, Kagi pays for access to an API whose implementation is basically live scraping Google and throwing away the ads.

There are a few other indexes that Kagi’s aggregator mixes in (such as Bing Search) but those haven’t been contributing much in terms of meaningful results. The sheer size of Google’s index still dwarfs all the others, including Bing Search.

So yes, Kagi doesn’t scrape but they pay SERP providers who do.


Marginalia likely returns better results than Google, when it returns useful results. DDG/Bing likely returns average quality results which can be better than Google's slop results. It's not true that Google is the only useful one.

They pay scraping sites for results, including SERP API. When I pointed this out, last time Kagi was discussed on Hacker News, an employee of Kagi said that they're trying to build their own internal index, but he didn't provide details.

They use a ton of different search index's from what I've seen. Kagi Small Web is also awesome

I believe they are building their own index but targeted towards useful results not included in the others


Is Marginalia the only real competing search engine?

AFAIK Brave Search is also using its own index.

I’m pretty sure Kagi’s entire pitch is that they’re an alternate and original dataset. Theirs is the only search engine whose data isn’t based upon google or bing’s databases.

To anyone who didn’t find Kagi useful in the past, I strongly recommend you try it again, as they have made huge strides in usability and quality of surfaced data in just the past year…. Huge strides since even a few months ago too, with more adaptive curation, data labeling, and UI/UX


> I’m pretty sure Kagi’s entire pitch is that they’re an alternate and original dataset. Theirs is the only search engine whose data isn’t based upon google or bing’s databases.

It's not. In addition to their own indexes the results include calls to "all major search result providers worldwide"[1].

There are independent search engines that only use their own index, like Mojeek[1]. Kagi is explicitly something like a search engine aggregator.

[1]: https://help.kagi.com/kagi/search-details/search-sources.htm... [2]: https://www.mojeek.com/


Kagi is mostly based on Bing and Google results, officially by API and not scraped, and ranked by there own mechanism I think.

Kagi was using Yandex, it comes up on every Kagi post here. Seems that they've removed that info from their site now though...

Officially Kagi pays SerpAPI to scrape Google for them, since Google doesn't have an API they'll let Kagi use.

Interesting. So this means the new google.com/goto will exclude Kagi from the results?

Not necessarily, as evidenced by TFA, whose scraper still seems to work even after it visits each individual SERP entry to harvest the URLs.

But Kagi’s SERP API provider is going to have to adapt, I guess.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: