For those not familiar with Rust or the debate around this subject, "namespace" here is not like a C++ namespace (Rust's modules are analogous to C++'s namespaces). "namespace" here is about the crates.io package registry, not the Rust language.
Let's say you have a crate (Rust's terminology for a package) named "foo". Inside "foo" you have a few subcrates (named "foo_bar", "foo_baz", "foo_qux"). There are a number of situations where a crate will have subcrates. Sometimes these subcrates are intended to be used publicly (e.g., you want people to be able to use just the "foo_bar" subcrate if they don't need the whole "foo" crate) Sometimes these subcrates aren't intended to be used publicly (e.g., Cargo forces you to put your proc macro in a separate crate, but that's an implementation detail of "foo" and isn't something other people should use directly).
So now you have "foo" on crates.io, as well as a bunch of "foo_*" subcrates on crates.io. Currently there is nothing stopping some random developer from publishing their own "foo_haha" crate. This new crate isn't related to your "foo" crates and might even be squatting on a name you would have liked to use for a subcrate.
Or multiple people might want to create different crates for similar things. e.g., there are multiple regular expression crates. But only one of these crates got the "regex" name. Everyone else has to use a different name, and sometimes people use creative (unintuitive) crate names.
So for various reasons, some people want namespacing on crates.io that would allow someone to reserve an entire namespace. This would allow you to prevent others from using a name that might be confusingly similar to yours. It also allows you to use conflicting crate names if multiple people want to publish different crates with the same name.
TL;DR some people want something like npm's scopes for crates.io. And other people don't want it. This has led to much debate and arguing and whatnot.
It's funny, though, because the team that owns crates.io almost never weighs in on the discussion, and on the rare occasion that one of the team members does say something they always do it in a personal capacity and not in a way that represents the team. This has created its own set of frustrations.
> the team that owns crates.io almost never weighs in on the discussion
There's two big things here:
1. "The discussion" has often happened outside of the contexts that the teams are supposed to evaluate; that is, RFCs are the way in which policies are changed, and the vast majority of the discussions here haven't been on RFCs, but on forum threads. Team members cannot be everywhere, and comments outside of the decision making process are kinda inherently non-normative.
2. Team members used to comment more in these threads but they've gone back and forth and around and around for years, and people are frustrated by it, so they've just stopped commenting in these threads. See #1. When it's optional, and it isn't pleasant, it shouldn't be a surprise that people don't participate.
(I am not on the relevant team anymore, to be clear.)
"Make an RFC" means coming up with a comprehensive plan for implementing the feature. If the team says "hey we're not sure we've ever seen a plan that addresses our problems," then it's not really their job to figure out how to address those problems, it's the people who actually want the feature. That may be the team, of course, if they think that it is important enough to pursue. There are people who actively do not want it too, and you can't forget about them. The way you've written this sorta suggests that everyone but the team wants this thing, but the team is ignoring them, but that is very much not the case. It's just a contentious issue with tons of people on every possible side. There are more than two!
Also, characterizing it as "no response" is not accurate, as I said, people have not come up with proposals that addressed concerns. It hasn't been ignored. Just not every person has the ability to comment on every thread in every forum.
And there have been some people who have put forward proposals. There's an open RFC I linked downthread about a related problem, "crate name transfer," and there was an RFC that was opened two weeks ago, but the person closed it rather than respond to the team's feedback. That there haven't been successful proposals is different than "there have been no proposals."
Personally, I think that we haven't had movement because it is an incredibly complicated space, with a ton of overlapping desires, and a lot of complexity, and a lot of angry, low-signal discourse. That nobody wants to wade through all of it isn't surprising. I'm glad this blog post was created; it's the exact kind of work that needs to be done to get things moving in a positive direction.
> "Make an RFC" means coming up with a comprehensive plan for implementing the feature.
That directly contradicts your previous comment (that RFCs are the contexts that the teams are supposed to evaluate). Either:
1: non-comprehensive discussions are appropriate for a RFC, and random forum threads are the wrong place for that, in which case your previous comment is absolutely correct that it's unnecessary and likely counterproductive for the crates.io team to weigh in on said threads, but that still leaves the question of why they [edit: the crates.io team] haven't made such a RFC [edit: and told people commenting on the forum threads to instead comment on the RFC, where the crates.io team can evaluate] yet.
Or:
2: non-comprehensive discussions are not appropriate for a RFC, and should take place elsewhere (ie random forum threads), in which case it's completely reasonable to expect normative (but obviously not binding) feedback on what would be necessary to produce a RFC-quality proposal in the first place.
Personally I weakly favor the former view, on the basis that RFC literally stands for "Requests For Comment", but either one would be a sensible position, as long as one doesn't try to assert the negation simultaneously.
1 is correct, sorta. Comments on an RFC are absolutely fine. That is their purpose. Random forum threads aren't the wrong place for that, like, you can discuss ideas for RFCs on forum threads, but you can't expect the team to always weigh in on threads. Their only responsibility is to the RFC process itself. If you're trying to get a proposal accepted, it's the wrong place. If you're trying to work on a proposal and get early feedback from other people, even if they aren't team members, then it's the right place.
> that still leaves the question of why they haven't made such a RFC yet.
> that still leaves the question of why they [edit: the crates.io team] haven't made such a RFC
Because as I said before, coming up with an RFC means putting together a viable plan. The team has not come up with a viable plan yet. Therefore, there's no RFC from them.
Anyone who feels that they have a viable plan is free to submit an RFC, and then the team will discuss it and give them feedback.
> coming up with an RFC means [I assume: requires, first] putting together a viable plan.
Okay, so it's actually number 2[2], then? Which is a prefectly resonable position, except that it's the opposite of what you said before.
> "The discussion" has often happened outside of the contexts that the teams are supposed to evaluate;
(Also:
> 1 is correct, sorta. Comments on an RFC are absolutely fine. That is their purpose. Random forum threads aren't the wrong place for that, like, you can discuss ideas for RFCs on forum threads, but you can't expect the team to always weigh in on threads.
but I'm not clear on how much of that was attributable to my poor phrasing of 1.)
1. You write up a proposal. We have a template so that you get the structure right and some general prompts to help you ask the right questions.
2. You can post it on the forums, as a "pre-RFC." This is like a vague temperature check from anyone and everyone. Nobody is required to pay attention to these, but it can be a valuable step if they do. This step is optional for everyone involved, you can go straight to step 3.
3. You take your proposal and submit it as an actual RFC. The teams are required to pay attention to these, and they either accept the proposal, reject the proposal, postpone the proposal, or give feedback designed to drive the consensus of the team towards one of these states.
When people say "the team is ignoring threads," they are referring to threads that are in step 2. Most discussions aren't full proposals yet. And they generate a lot of discussion. And then people never move to step 3. Those that do are guaranteed feedback from the team in some fashion.
Your initial edited comment:
> that still leaves the question of why they [edit: the crates.io team] haven't made such a RFC
Nobody is required to make an RFC for anything. If the team doesn't want to write one up, they don't have to. It's on the advocates of the feature to start the process, no matter if they're on the team or not.
> [edit: and told people commenting on the forum threads to instead comment on the RFC, where the crates.io team can evaluate] yet.
They didn't say "comment on the RFC," because there is no RFC at that stage. They said "please make an RFC." This is the same process every single major change to Rust goes through. It's pretty standard. The vast majority of people who desire this feature never actually do it though, because they tend to post on the forums, get a ton of comments from people, and then burn out, it seems.
I hope this helps. To be honest this thread is so old I'm likely to lose track of it and not reply further.
is there a TL;DR for why some people don't want namespaces for crates? Is it just because NPM does and javascript is bad or is there something else I'm missing? I couldn't imagine github or npm working well without them.
> Is it just because NPM does and javascript is bad
Not at all! In fact, npm's implementation is why some are pro, but also why some say that it's not worth it; see https://news.ycombinator.com/item?id=24444852 for some npm specific sentiment.
(Incidentally, npm added scoped packages in 2015, after these decisions were made for crates.io. However, crates.io is still smaller than npm was then...)
There isn't really a TL;DR because this space is complex, and there's a lot of details, and they matter, and different people have different reasons for not wanting them. I think it's good that github has namespaces, but that doesn't mean I think that Cargo should have them, because they're fundamentally different services.
The short reason we gave back in 2014 was:
> Because namespaces are strictly more complicated in a number of ways,and because they can be added compatibly in the future should they become necessary, we’re going to stick with a single shared namespace.
But unlike traditional namespace, that you automatically got alias to things inside the same namespace.
The npm's namespace is basically just a preserved prefix, mention scoped package inside another scoped package still requires you to write full name (scope + package) or it didn't work.
I don't really see a reason that you can't implement it in a flat name package manager? Something like "you can't publish a package named as 'USERNAME_private_WHATEVER' unless you are USERNAME"
Nobody is saying that it's impossible to implement. However, someone would have to write that up as a plan, including all the details, and put it through the RFC process, and get folks to agree that that plan is the one that should be implemented, rather than other possible plans.
The article is more or less already the TL;DR. My perception is that there isn't a strong aversion to adding namespaces, it's just that
1. the crates.io team is understaffed (unlike NPM it's not a VC funded entity), and it has not been a priority for them (or for anyone deeply involved in Rust/Cargo/crates.io development, AFAIK)
2. there are some core design decisions around the feature for which no consensus has been reached yet
If there was a sports-like betting site for future decisions that major open source projects are going to make, I would put a thousand dollars on "Cargo/Crates will implement namespaces."
The developers have convinced themselves that they're right on this, having, I suppose, not learned from the hundreds of package managers that have come before them. They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.
But, this is the nature of open source (and even startups). New projects start under the (oftentimes correct) belief that they can do something better, which experiences linear extrapolation into believing everything you do is better through virtue of being unlike what everyone else is doing. We see the end-state of Maven, NPM, etc; we don't see the war that those projects went through to reach this state. History is elided, forgotten, then repeated.
> The developers have convinced themselves that they're right on this, having, I suppose, not learned from the hundreds of package managers that have come before them. They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.
Comments like this are why this discussion has devolved over the years. It's "all heat and no light." It does not help convince anyone that you're right, and if you want this to change, I would suggest that you change your approach.
The team has stated that they are open to implementing namespaces, but that the people who advocate for them need to address the concerns. This hasn't been done successfully yet, and is part of what this post is trying to help out with, by cataloging some of the large amounts of previous discussion.
What concerns? Most people don't care because they feel that the ship has sailed, the mistake is set in the concrete. Transitioning thus becomes the biggest conceivable concern for me.
What are the other possible concerns? The only comments I've seen on this is "It's not any different, people will just squat namespaces" to which I reply: reserve namespaces up front for Rust nursery and bespoke crates.
That has always been the biggest benefit of namespaces, when i type "std.rand" I never have to worry about an attacker squatting "std.rnd". But I could just mistype the namespace! Fine, I could, but all the obvious mistypings could _also_ be reserved!
The article has some discussion on all of the various bits of the problems in this space, and has links to previous discussions.
It seems from your comment (and I may be wrong!) that you're purely talking about the "use namespace to solve squatting" issue, which is only one dimension here. Some people want namespaces and don't care about squatting. Some people want to solve squatting through other means. Some people do see namespaces as a solution to squatting, while others disagree.
I think the point of the post is that whatever the reason, every other package manager that has started out without namespaces has eventually implemented them.
The reasons may be unclear, but it seems likely that eventually the crates.io team will receive the same reasons that the other package managers got and implement namespaces.
There's been some discussion around it for PyPI, but it doesn't seem like there's a consensus that it is necessary, or even on what the set of problems that should be addressed by it is.
> The developers have convinced themselves that they're right on this
This is a complete misrepresentation of the issue. Adding namespaces to crates.io is a social problem, not a technical one, because adding them means you now must do the work of arbitrating disputes over an identity layer, which is a job unfit for part-time volunteers, which is all that crates.io has. The only alternative to arbitrating your own identity layer is to tie yourself to some other preexisting identity layer (e.g. GitHub, DNS), which raises questions about the notion of project ownership, the potential transfer of ownership, the immutability of the package registry, and all the while just pushes the "race to register" to a different sphere.
If crates.io had full-time employees rather than only volunteers, I'd be all for paying someone to handle identity problems in exchange for the existence of namespacing. But who's going to pay for it?
I agree with the main thrust of your post, but there is some level of technical problem because the Rust language doesn't support namespaces in crate names themeselves, and so you have to deal with that (or not...) in some capacity too.
Obviously we're all talking hypotheticals here, but Rust-the-language doesn't need to support crate-level namespaces (which isn't to say that some first-class support couldn't be imagined, but again: hypotheticals). Rust links to libraries (via explicit or implicit `extern crate`) that it finds on the system via the ordinary OS-level library search path mechanism; it's up to the package manager to put those libraries on the system in a place (and with a name!) where Rust can find them. As long as the package manager can turn the registry namespaces into something usable from Rust, then there's no need to add anything to the language itself. A package manager could decide to put a namespaced package "foo/bar" on the system as merely "bar", and require the user to manually disambiguate when colliding (using ordinary rename rules, such as is already supported by Cargo). Alternatively, a registry could forbid uploading packages whose names internally contain underscores, and then the package manager could install "foo/bar" as a crate "foo_bar", which has no risk of collision (well, with packages from that registry).
This is why the conflation in the title of the OP here is especially annoying; package management is a concern that the language doesn't necessarily benefit from being aware of, hence the separation of the compiler and the package manager. :)
> Rust links to libraries (via explicit or implicit `extern crate`) that it finds on the system via the ordinary OS-level library search path mechanism
Nit: rustc gets passed each crate available to be used or extern crate via an --extern flag that specifies the name of the crate as well as the full path. The path is optional and if not specified rustc performs a search, but the name is mandatory. Cargo only uses the mode with both path and name specified. This also isn't an edition 2018 thing, it exists on older cargo versions too. It's possible to specify the same file under multiple names, then it's available to Rust programs under multiple names. It's also possible to specify multiple paths for the same name, but then only one will be taken, the first one which is available.
Also, note that Cargo works in terms of packages, and rustc in terms of crates. A package can contain up to one library crate, and the names of the two don't have to match.
I'm also under the impression that the crates.io team was somewhat reluctant to implement this because it was viewed as a potential security risk from a social engineering point-of-view, even if there were people dedicated to it full time.
That said, I'm having a hard time finding the Rust Internals discussion thread, so I can't tell if I'm misremembering somehow.
Edit: Actually, I think this was in context of re-assigning crate names that may have been "squatted", and how making decisions around what amounted to team member's arbitrary decisions around re-assignment could cause issues. Scratch this, please.
> The only alternative to arbitrating your own identity layer is to tie yourself to some other preexisting identity layer (e.g. GitHub, DNS)
Devolving identity down to DNS has been a really good decision for Go. I'm not sure why everyone doesn't do it. It's like XML namespaces, but actually useful.
> Devolving identity down to DNS has been a really good decision for Go
If it works for go, the because go is not old enough. Domain registrations lapse quickly and then someone else can buy them. That's very close to the worst that can happen for trust in packages.
Ok, then how about Maven Central? They give you publishing rights to a particular group ID by verifying your ownership of the corresponding DNS names. They've been around long enough, I think, even if Go hasn't.
Maven Central uses that as a first-time verification (actually, it's OSSRH that requires this, technically there are other ways to publish to Maven Central).
Golang's package system depends on the current DNS records.
With maven, there are plenty of old projects that do not own the domain names for their group ID, and that doesn't cause any problems - they are well-known, established projects, and there's no reason to introduce breaking changes to everyone just because they want to go through some DNS naming changes.
Because of this, there's little reason to fear that if you publish your project today as "io.github.my-project:my-module" you may find yourself having to break all your users if github blocks your account tomorrow (or just changes their URL schema) and you move to gitlab.
If you lose your domain, spreading bad software is the least of your concerns. They can reset all email passwords and access every service you’ve ever used.
Either the package manager site allows email resets and the same issue applies, or it doesn’t and you can be locked out forever by forgetting a password. Both failure modes suck.
> If you lose your domain, spreading bad software is the least of your concerns.
OTOH, it's probably higher on the list of downstream code users’ lost of concerns.
> They can reset all email passwords and access every service you’ve ever used
Not if you dealt with all that before letting the domain lapse. And if you don't, more to the point, that's your problem, but not the package manager user communities problem. And that it is a bigger concern of yours than the impacts on the package manager user community is exactly why that community might want to protect themselves from you neglecting their concerns..
Organizations live shorter than packages, just look at all the com.sun stuff in Java. And even in the absence of corporate changes, you may well want to transfer a package to a new home.
> The google.golang.org/protobuf module is APIv2. We have taken advantage of the need to change the import path to switch to one that is not tied to a specific hosting provider.
Basically, Google realized that it's not a good idea to tie your package's identity to
GitHub's URL scheme. This is easy for Google to fix, but the majority of Go packages suffer from the same problem, and it is much harder for a small project to start maintaining a DNS record that they control to point that to GitHub.
Not to mention, if GitHub decided to change its URL scheme for some reason, almost all Go builds on the planet would have to change.
And of course, the whole problem stems from relying on DNS to name your packages. No one wants to bother with DNS, so they go for the low effort option and let some hosting provider handle it for them, without caring what happens in 10 years time.
> They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.
This sentence takes the place of a reason. You could have offered a reason why you were right or a rational argument about the matter. Instead, you basically yelled "I'm right, they're wrong! I'm right, they're wrong!".
I am not arguing that this should happen; I am asserting that this will happen. The reasons why it should or shouldn't happen don't matter when it will happen.
If you'd like to: Go back and re-read my comment, but re-read it in the mindframe that I'm against the change. It still holds. I didn't provide reasons why they should make the change, because I don't have a stance on whether they should or shouldn't. I am not qualified to say whether its the "Right" decision; just that Rust is on the wrong side of history with this one.
Just because they will do it eventually doesn't mean they are wrong now. It's entirely possible that a flat namespace is reasonable decision for a small registry with few resources, but a bad decision for a well funded registry of an ecosystem that had a few more years to develop.
No, their first paragraph was a prediction. Their second was a assertion of fact without any evidence provided. They could have hedged that statement with "I think", or "in my opinion", but instead they went with "They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this". Strong assertions require strong evidence, and not providing that evidence deserves to be called out. At least that's generally the consensus of the community here, as I see it.
It's not that evidence doesn't exist (it may or may not, and people may or may not agree that something is evidence), it's that none was provided. Without that, all you get is people yelling that they're right and someone else is wrong. Evidence lets people move the argument to useful territory.
This seems like human nature. We continue to do the same thing with client side / Server side rendering. When it comes to computing, we oscillate a lot. In all these cases there is the desired system (lack of namespaces in a way they doesn't absolutely kill you later), we pursue it, we inevitably fail, get burned, revert, forget, repeat.
In the case of namespaces, this is a tale told a thousand times. To the point where I think people don't even want to try to fight for them. What's the point? They shall forge forward, namespace free, until it hurts enough decision makers. Then we get namespaces.
Now, back to Ruby for me. Where anything anywhere can just yolo reach into your namespace and mess your whole life up.
They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.
To be honest, that's an impressive oratory monstrosity you've produced there. No argument, no nothing, and nevertheless it sounds completely brain-shutting-down conclusive.
I don't think the rust people are deserving of it. They've done impressive work with minimal resources, and they have shown the ability to create consensus that's better than any of the individual parts that contribute to it. So I'm going to trust them in this for now, and maybe, they'll evolve in the future.
Nevertheless, I have to admit respect for that sentence, in the same sense I'll admit respect for an earth-shattering nuclear explosion. You might want to run for US president if you produce more of them, though I'm not sure if that's praise or damnation.
> The developers have convinced themselves that they're right on this, having, I suppose, not learned from the hundreds of package managers that have come before them. They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.
A thousand times this.
Why are so many things in software three-steps-forward-two-steps-back?
> If there was a sports-like betting site for future decisions that major open source projects are going to make, I would put a thousand dollars on "Cargo/Crates will implement namespaces."
There is, sort of! Assuming you choose a semi-reasonable timeframe (5? 10 years?), I’d be happy to take the other side on Longbets.
I believe there’s even a reasonable argument to be made that your bet is societally important, as the rules [1] require, based on your final paragraph. Roughly your take is that over time, efforts realize that the naïveté of youth misread the battle scars of what came before. crates.io having namespaces or not is just the mechanism for this debate :)
You really should look to other ecosystems and see what lessons they've learned. In java, packages are normally "namespaced" by the author's reverse domain name, like `org.lwjgl/lwjgl`.
Since clojure uses maven as well, the same applies, but clojure tools like leiningen decided to create a shortcut: if the group and artifact name are the same, like `iglu/iglu`, they can be collapsed into one name: `iglu`
Well, that just encouraged everyone to choose collapsible names. In retrospect, this didn't buy us much. Who cares about saving a few characters of typing? Most now seem to agree it wasn't a good idea.
When the "collapsed" name falls out of maintenance, the forks will all seem somehow less "official", even if they are much higher quality. Forks are inevitable; why would you want to discourage them?
I finally decided to start using the reverse of my personal domain for my future libraries. The java folks were right all along.
Agree, Go went a similar route as Java and I think that's good as well.
The new tools.deps in Clojure actually is moving to disallow collapsed names for similar reason and will force iglu/iglu.
Here's a rationale from them:
> The groupId exists to disambiguate library names so our ecosystem is not just a race to grab unqualified names. The naming convention in Maven for groupId is to follow Java’s package naming rules and start with a reversed domain name or a trademark, something you control. Maven itself does not enforce this but the Maven Central repository does all new projects.
> In cases where you have a lib with no domain name or trademark, you can use a third party source of identity (like github) in combination with an id you control on that site, so your lib id would be github-yourname/yourlib. Using a dashed name is preferred over a dotted name as that could imply a library owned by github.
Can I get a link to the quoted document? The quote raises more questions than it answers. Who determines that someone applying for a qualified name is the owner of that trademark (presumably this is a full-time employee; who pays their salary?), and what is the process? Trademarks are not a universal namespace--even within a single legal jurisdiction you can have the same name legally owned by different people due to different contexts--so who decides who wins?
That said, ya this is best intentions unfortunately. I'm guessing if you own a real trademark, you could actually sue people using your trademark as their group-id.
Otherwise in general they recommend using a registered web domain name. Someone else could take over your domain name as theirs, but I think the registry owner, like maven-central, if you contacted them and could show you own that domain, they might be able to take actions against the impersonator. Same for a github user.
Actually, thinking about this, I feel it be great if the repository owner like maven-central required a form of proof of ownership of the domain or the github id. That could add a lot of trust to the whole process.
The quote was a guideline, not a requirement. Cognitect (who makes the clojure CLI tool) doesn't even control clojars, the main clojure maven repo, so they wouldn't be able to enforce that even if they wanted to.
>Forks are inevitable; why would you want to discourage them?
The epitome of this mindset I think is "hostile fork" -- the entire notion is nonsensical. The whole point of being FOSS/OSS is the freedom to fork -- by all damn rights, you should fork as you please, and be pleased to fork!
The actual problem is not forking.. its community fragmentation, and more importantly loss of a "source of truth". Of course, maintaining that source of truth is otherwise known as centralization, with all the problems that brings, but there's nothing inherently wrong with forking.. that's just the natural specialization and evolutionary processes at work.
The solution is to make it easier to find those top-tier libraries, and this is orthogonal to forking; mainly handled by blog posts and "official" library listings/recommendations, and things like This Week in Rust.... namespacing or not doesn't really get you anything there.
> "hostile fork" -- the entire notion is nonsensical.
It is not. It is based in experience. See the xMule/aMule fork. A hostile fork is when the fork project starts bad-mouthing the original project and its maintainers.
The notion that forking is by itself hostile is non-sense.
If I'm remembering right, something similar kind of happened with uBlock and uBlock Origin, but it was the original maintainer who came back and forked after the new maintainer became hostile, or something like that.
This was discussed in detail in Homesteading the Noosphere of Eric S Raymond.
I think it's in there somewhere that he compares the right to fork with the right to bear arms: Good to have, but the situation must have gotten really shitty if a fork is a good solution.
I have less experience with CPAN and RubyGems but npm's namespacing system has two very serious problems:
1. It was introduced very late, meaning the community had already formed patterns of contribution around a flawed flat system. This is a problem of the flat system, not of the namespaced one.
2. It is still to this day entirely optional (for understandable backward compat. reasons). This gives namespaceless packages a misplaced position of authority over namespaced ones, which erodes the value of namespacing.
These are tough problems to get around if you start with a flat structure, but they really just outline the urgency of switching to namespaces for a relatively young project.
I agree with a lot of this perspective. It's also directly relevant to our situation, because we are basically in exactly that place now, and dealing with these problems is something that proponents of adding namespaces need to navigate.
This is the only good argument I've heard yet for not adding namespace. And maybe it's a defeating argument, maybe Crates is doomed to not have namespace due to the cost of putting them in after the fact.
I'm not sure you followed the above 2 points, or perhaps read them through tinted glasses.
I wasn't arguing that npm's namespacing system is worse than their initial system, nor that their switch to namespacing was a mistake.
The current npm namespaced system, with flaws, is head-and-shoulders better than the previous flat system.
You're saying you did "look and learn". If by that you mean you looked at the end product (npm's is seriously flawed) without looking at the journey to that product (npm's is still a huge improvement over what they started with), then you're not going to learn much from that kind of "looking".
I highlighted Composer/Packagist in a sibling comment as a system you should look and learn from (w.r.t. namespaces).
Choosing to only look at flawed systems that started flat seems like you're just being selective to support your own thesis.
PHP and it's ecosystem has a lot of problems, but I think Composer/Packagist is as surprisingly exemplary example of how to go about structuring package management.
Add a "legacy" namespace and move all existing packages there. Allow for a transition period where tooling will add "legacy" to instances where no namespace is given. Add a mechanism for legacy packages to indicate their new namespace so that transitioning could be mostly automatic for package users.
Not effortless, but not necessarily very costly either.
This is not the first time typosquatting attacks of this kind have been uncovered.
Popular repository platforms such as Python Package Index (PyPi) and GitHub-owned Node.js package manager npm have emerged as effective attack vectors to distribute malware.
"Orthogonal" suggests no connection but what I see above is a list of package managers that don't have namespacing.
They didn't make the claim that no namespaces had anything to do with this, that's an inference you're making from the specific list, when it could be for any number of reasons. For example, these are some of the largest package management ecosystems in the world, so they're more likely to be attacked than smaller ones. (You can of course come back and say that there are other massive ecosystems too, but that's kind of my point: there's more to a discussion than a random article listing a few ecosystems.)
I stated my reasoning in my comment: you can typo squat a namespace, just as easily you can any identifier. I don't see any inherent difference between the two.
Maybe. Regardless of what my parent meant, a lot of people in these discussions imply that we never looked at prior art because we did not make the choices around the tradeoff that they wanted us to make. And we did look at many, many approaches. We just decided to not go in those directions.
It is not a comprehensive list of things considered, it is a list of successful ecosystems that we decided to pick something closer to, than others that we decided not to.
And even grouping those together in terms of downsides is not really doing justice to the individual problems that each of those systems deal with.
Java was my introduction to namespacing, so I only suspected but didn't know for a long time that Java overdid namespacing.
Companies change names, they merge. Sometimes they go out of business but stick around as a foundation stewarding their old projects, and you might be going to example.org for years for documentation on a com.example module.
And the namespaces weren't enforced (who is going to stop me from publishing a com.example.foo module?), so it expected much and delivered little.
No namespaces is bad. Five level namespaces are better, but still bad for different reasons. Two might be good. Some might prefer three. But zero is right out.
I agree, the Java namespace system isn’t that good. In fact I hate it. First because it uses reverse DNS while common use of URL are in the opposite order. Second because the package sbu-namespace is enforced with the file system structure, which makes for crazy long names.
On the other hand I really like how C# and dotnet in general handle the matter. Package namespace are separated from logical (in-code) namespace. Package namespace are usually two/three dotted term, making ownership clear while not bloating the names.
> First because it uses reverse DNS while common use of URL are in the opposite order.
Well, that's more a bad thing about the DNS though.
"toplevel.domainname.subdomain/path" should really be how it should be structured. SUN improved this and made the hierarchy proper.
> Forks are inevitable; why would you want to discourage them?
The conclusion that we don't want to discourage forks may be valid, but this doesn't seem to be good reasoning. Lots of things are inevitable that we want to discourage or delay.
> Well, that just encouraged everyone to choose collapsible names. In retrospect, this didn't buy us much. Who cares about saving a few characters of typing?
This appears to be using evidence to prove the opposite conclusion; if everyone voluntarily chose to use shorter names, then it means that everyone cares about having shorter names. If there is a more substantial argument for why people have decided that the collapsing was mistake, I'd like to read it.
I didn't choose the shorter names because i "care[d] about having shorter names", i did so defensively, because i figured if i chose `net.sekao/iglu`, someone else would choose `iglu/iglu` which would imply that theirs was the original or official version.
Another point i didn't mention is that maven was designed from the start to be decentralized; many companies run their own private maven repos, but also pull artifacts from maven central. Having group names reduces the chances of collisions between their private servers and a public maven server.
That's a better rationale, although I don't think that really solves your stated problem; as an uninformed user I am still more likely to think that iglu/iglu is the more authoritative source there. Given this, any project that wants to authoritatively own its identifier should probably also register its own top-level namespace... which unfortunately brings us back around to where we started.
It would at least be far less of an issue. I don't see anyone being confused that https://github.com/facebook/react is the official repo, and not https://github.com/react/react. It's the fact that a collapsed name is a shortcut that imbues it with this special stature. And i believe maven central doesn't even allow one-segment group names for new libraries, though clojars obviously does.
This article is poorly titled; it's about the lack of namespaces on crates.io. Rust does have namespaces. Cargo is capable of supporting registries that use namespaces. Using Rust doesn't even require one to use Cargo, and using Cargo doesn't require one to use crates.io.
(For the record, I do think that namespaces on crates.io would be a marginal improvement, but the benefits are often overstated and it would add organizational costs that the crates.io maintainers can't currently bear. The amount of ire over this topic is disproportionate to its impact.)
> This is a strike against any namespace system that allows namespace ownership to unexpectedly change. Discontinuous identity has a couple of issues.
Hard agree but this has nothing to do with namespaces. This is equally applicable to names in a flat registry. I don't see the point of bringing this up in this article.
> there are also real benefits. [...] discourages forks
What. How? Why?
I guess there's possibly some valid argument for disliking forks (though it's certainly not put forward in this article), but discouraging them via package registry structure is... delusional.
Forks are a reality of software and they neither happen as a result of registry structures, nor can the be discouraged by their designs. They happen for entirely external, human reasons, and a well-designed registry structure (i.e. a non-flat one) will be set up to be resilient to forking.
None of these are features, which are incompatible with namespaces. I can't find a compelling reason that Crate works this way in this entire article. The article reads like a list of "why Crate is poorly implemented". Of course, it works when the community is rather young and small. Give it time.
That person is definitely not helping the perception that a significant fraction of the cryptofinance community is scammers, and that their presence has a negative effect on the community.
That account is weirder though, they have a bunch of real (and real popular) projects like zeroise, the filled-in projects are rather complex, and even the placeholder packages have a somewhat proper description, though some of them are very very odd.
They're not out there hawking popular names for sale and some of their placeholders are even "namespaced" (the abscissa or signatory subcomponents).
Interestingly, you can't name squat on GitHub. I just asked them for the guy squatting my first name and they gave it to me. Ah, the power of non-community-run product. You can do the right thing without having to worry about the slippery slope.
Sure, and that's why I said I was assuming, but my (poorly communicated) intent was to question how unique your first name is, and how someone is squatting on it if it's a real account and maybe they have that name too?
I mean, it's probably not a big deal in this case. But if I had a github account that someone wanted and github just gave out my (personal?) info, I might feel a bit betrayed by them. I know, I shouldn't give them any info I really want private, but that doesn't mean I can't expect them to act responsibly with what I have given them. I just didn't want to state all that up front, because it assumes a lot about what happened when there wasn't a lot of detail provided.
It is nice when people can just make simple stuff happen though, I agree. It's just a hard line to walk, because you allow that and then you get stuff like the recent Twitter hack. It's hard for us to all have nice things and nice people that can help us because there really are a bunch of people out there to get you (or someone in a similar situation, at least). :/
They don't give you access to the account. They tombstone the old one and assign its handle to yours. So you get the other GitHub handle but your account retains its repos, its keys, etc and doesn't get mixed in with the other account.
Of course, if they just let you login to the other one that would be crazy business. You'd know the other guy's email for starters. But they don't.
It's all perfectly kosher. You just get the namespace.
Ah, they gave the account name. When you said "asked them for the guy" and "they just gave it to me", I was interpreting it as the guy's contact info and them just giving it to you, which is what was raising red flags for me. Yeah, it's much less problematic if they just let you use the name of a discontinued account (as long as the github single-sign-on stuff keys off an ID and not a name, which I assume it does).
Yeah sorry about that. Though it's funny. You can just scrape folks' repos to get their email anyway. They all include it in commits. Emails are better made public than kept private imho.
This also happens with PyPI. I've been lucky to reclaim a package name I wanted and had been squatted, but I've also seen some squatters observant of requests to reclaim squatted names, and then they start adding some (usually useless) functionality to the package. See [0] for one example.
So then they just change their script to only create 3 projects per generated username?
There's nothing you can do to prevent this except devise a system where it's inconsequential to register all of these names: namespaces.
If it bothers you that one person can do this with many packages, then presumably it bothers you that 100,000 people can do it benignly with a few package names which is what goes on in npm/ruby/cpan. The solution is to simply making them worthless.
Even before, while Mozilla occasionally paid some people to do some development work, including creating it in the first place, they never paid for full-time support staff, which is what this would require.
Well we now have a number of companies who are paying for all sorts of things, Amazon and Microsoft being two of them. With hopefully more in the future once we get the foundation going.
It's actually terrible, justs look at NPM and RubyGems.
Instead of just coming up with a good name for your project, you have to come up with a unique one even though the project with that name isn't even active. And since most names are taken, you have to come up with a clever one for no real reason. Or go down the route of naming strats you see on npm/gems: better-lib, lib2, lib2-better, lib3, better-lib3, etc.
It really does nobody good except for the person who was able to clutch the original name, and that's a weird incentive structure. Subreddits have this same problem where you're instantly gimped when named anything other than r/bitcoin, r/politics, r/news, r/atlanta, etc.
Instead, there should be no preciousness to names nor dictionary words. That's the correct incentive structure: none.
People keep linking https://crates.io/users/swmon and it probably seems like an easy fix to just revoke those names. But swmon isn't the problem, the system is. It's no bigger problem when swmon does it overtly vs thousands of people doing it incidentally when they register a package they don't do anything with.
The problem isn't to calibrate a system that can detect swmon but build a system where you don't care what swmon does.
We really need a colorful term for the practice of ‘splaining to people how a frustrating part of your code is not only not a bug but you are either doing it (to them) on purpose, or pretending to have done so in order to save face.
That's the 'gold standard' but it leaves it open to interpretation. Some people mean it with very little sense of irony.
Do you think Sam is being ironic with that title, or sincere? His intro is telling us how he's going to calmly explain to us dumb hicks and newbies that we wouldn't know a feature if it bit us on the ass (which it most definitely will, at some point).
The rest of the article doesn't (quite) follow up on that warning, but if you start out by putting everyone on guard nobody is going to listen to you.
The intro says that Sam is laying out the advantages of no-namespacing because typically only the disadvantages are discussed. Sam also makes clear that they're "summarizing points raised across the various proposals I’ve read".
The rest of the article follows through on the intro's promise.
I more meant a word for every situation in which someone explains their thoughts of something in a way that you find distasteful, so that you can use the word in a dismissive way. We don’t really need more of that, do we?
Well, what's wrong with namespacing, exactly? Github, java, etc, show us that namespaces are perfectly fine. Not perfect, but fine.
The rest of Rust is extremely pleasant to work with; a lot of thought has been put into ergonomics and I love it as a result.
But with crates.io it feels like the opposite. I have to deal with name-squatting and a much more difficult naming process for crates as pretty much every short, meaningful name in English is taken at this point. For what gains? Less forks, less fracturing -in theory-?
I feel like a lot of the arguments for no namespaces are strawmans or make big stretches. For example, figuring out which fork of a project to use is really not a problem. And a lack of namespaces does not fix that anyways, because it's not a code problem... Importing two crates with the same name would be fairly rare, and rust easily solves that with `use crate::... as ...`. Etc.
Anyway, this discussion has been trod to death. There's no point having it, as I'll just be told why I'm wrong and why namespaces are just -impossible-.
As mentioned elsewhere in here, this is a misrepresentation of the issue. The problem is that crates.io is a free, volunteer-run project with zero full-time employees who could be tasked with the drudgery of intervening in naming disputes or managing an identity layer. This is unlike GitHub and unlike "Java" (by which you probably mean Maven Central, which is operated by Sonatype). Solve the funding issue first, and then you can start solving the rest.
"could be tasked with the drudgery of intervening in naming disputes or managing an identity layer"
What do these have to do with the issue of namespaces?
Having namespaces would lessen naming disputes since users would be able to name their crate whatever. Any system will have disputes, so I'm not sure what the argument is here.
I understand why crates uses github for identity management, but it's not ideal that people are locked into using one specific git platform. Github is already an unavoidable entity, much like facebook, but this is a bit of a harder issue to solve without a lot more resources.
So? It's still less disputes. Nobody should be taking namespaces such as "http" for example. The namespaces represent who people are, not what the package does.
What's right with package namespaces? How do they help at all? Where's the benifit? The cost is that package paths are longer and hard to remember. Also confusing because multiple packages can have the same name
> I have to deal with name-squatting and a much more difficult naming process for crates as pretty much every short, meaningful name in English is taken at this point
Squatting is not a major problem just name your package yourname-xml or randomname instead of yourname.xml
"How do they help at all? Where's the benifit? [sic]"
The obvious one is people can put packages under their own namespace and not worry about naming collisions. If this were really such an issue, things like github,maven, or just about any other library manager would be impossible to work with. But that isn't the case.
"The cost is that package paths are longer and hard to remember. Also confusing because multiple packages can have the same name"
In 5+ years of programming I have not once had an issue due to namespaces. In both systems you still need to remember the author + repo name. Both systems would have multiple competing projects for a problem space. Etc. It's up to you (the engineer) to do your research before adding a library to your project. Namespaces or the lack of them will never fix that.
Name length? Really? Is ~5-15 chars vs ~20-25 so important? It's not like we're memorizing these things. This is not worth removing namespaces for.
Both systems have issues due to things being named the same. One example in rust is `http` and its related types, which get referred to by multiple crates. I ran into that recently when I pulled in various networking crates into the same project. Removing namespaces doesn't fix this.
"Squatting is not a major problem just name your package yourname-xml or randomname instead of yourname.xml"
I'm sorry but I cannot agree with this. Naming should not be a compromise. There's a reason we don't use random names or just random numbers.
> So what’s a bad actor? I consider someone who squats a bunch of crates to make or point or prevent their names from being used to be a bad actor.
In my experience, the thorny problem with namespaces isn't bad actors because you can fix that with content moderation. Yes, that takes time and effort which can be scarce in an open-source community, but it's doable.
No, the thorny problem is good actors with bad execution: people who publish a package with a nice name that does what it says it does, more or less, but does it poorly. It doesn't cover all the edges case or it is poorly optimized or it has limited features or it has some fatal design flaw in its architecture... Maybe it was fine at some point but just hasn't kept up with the times.
So what do you do if a package like that grabs the name "json" or "http" or something? You don't want to kick packages off just for not being great, but you also don't want it to seem like the de facto package for a task in your language. I do a lot of Haskell, and it's not the best developer experience when you have to tell people "don't use json, use aeson instead".
Namespaces would solve this problem. You could have a "blessed" namespace (like rust/) and, when functionality becomes important enough that the quality of the "default" package matters use the namespace to provide that package (rust/json, rust/http... etc). You don't have to kick off the existing package (even if it's squatting a "nice" namespace like json/json) but you also don't get into the point where packages with nice names look fine but are actually flawed and a bad choice for real work.
This is why I like the Go approach: just use an existing global namespacing system and delegate to it all ownership disputes etc: DNS + actual web servers serving traffic (not necessarily just got repos, since the "vanity imports" offer an indirection based in html meta tags)
Yes, there are benefits to that approach. We rejected it because we want builds to be reproducible, and not to have that property depend on external entities.
That's orthogonal with the naming scheme. In fact the Go team eventually implemented a way to decouple the storage of a copy of the package from the name of the package, namely the goproxy. How they got there was admittedly a bit clumsy, but it proves that it's possible
That's an implementarion choice of the proxy. You can run a private proxy that inplementa whatever policy you want.
If crates wants a policy that stores crates forever then just store crates forever. What I'm talking about just affects how new crates and new versions of existing crates get "uploaded".
Imagine you call your crate foo.com/bar. In order to push your crate the crates server needs to verify that you actually own the foo.com/bar URL.
There are many ways to do that. A simple one is to just have the crates server pull code from that URL instead of pushing it. If you want to push (e.g. to work around reachability issues such as firewalls) it's just the matter of devising one or more ways to prove that you "own" that handle. For example by serving some challenge response in a html meta tag
Right, but then you're back to the exact same issues we're talking about. The choices made by that proxy is the same discussion we're having here, and also means you can't totally rely on the external authority for identity, because you have to manage the mutability in their service. There are many ways to do it, and that in fact is the exact issue at stake here.
All of the existing services it would make sense to delegate to are more mutable than we wanted for our system.
1. I like a package repository that has "namespaces". In fact that's what I like about Go approach to package repository from day zero, despite other rough edges.
2. Classic approach to "namespacing" like maven's is susceptible to name squatting or just "split authority": who can claim the "org.apache" org? First comes, first served? If that doesn't match who owns the apache.org "brand" on the wider internet, it's not technically a problem of the package repository but surely it's a practical one.
3. Name squatting problems exist for DNS too; it's a know problem, with an existing administration and dispute resolution body, a marketplace and whatnot. If you cannot get hold of foo.com you'll settle on getfoo.com or (some abomination like that) and that will be your brand as far as online presence is concerned. It's only natural for artifacts you want to associate to that owner to just piggy back on that more prominent identity rather than have to fight it independently for getfoo.com and perhaps land on gotfoo.com because getfoo.com has been taken by somebody else.
4. Now, if you decided that you want namespacing and given the observed consequences of allowing your users to compete for namespace names independently of their main online presence, I claim there is a technical approach that can achieve your stated goal of reproducible builds. The goproxy is almost what you want. The main different is that the go team aims at a best effort service and thus want to call it a caching mirror. They could have chosen to offer a more strict retention guarantee. Go packages are indeed immutable. The sumdb transparent log is there to enforce that invariant. It's really just about somebody willing to take the burden of storing the package bits forever, which is what I understood you choose to do with crates.
I'd like to understand more about the arguments for and against a URL-based module naming / resolution approach. Could you point me to any past discussions on this for further reading?
I am not 100% sure exactly what you're asking for, so if you could say a teeny bit more than that, that would help me understand what part of the space you're trying to explore. It would also be handy to know what problem you are trying to solve, there are several here.
I should be clear my interest is theoretical. I haven't run into any practical issues with "crates.io" and it's been working great for me.
I think the way Go and ES Modules (Deno / browsers) represent module naming and resolution by URLs has some nice benefits:
* decentralized module hosting
* could be extended beyond HTTP / Git (ie. import modules via "ipfs://")
* re-uses existing web infrastructure for namespace ownership (ie. via DNS)
* possibility of hosting backend and frontend modules for multiple languages in a single HTTP registry
However, I do see some important tradeoffs to that approach:
* modules may become unresolvable from their original URL (link rot)
* cannot guarantee immutability at the original URL
* semver support becomes a concern of the registry, and some registries may not support it (ie. the logic to resolve "https://example.com/my-module/1.x.x/lib.rs")
* more difficult to discover (browse and search) all available modules
I see some proposed solutions to the tradeoffs:
* proxies and dep lockfiles of file hashes to help prevent link rot and immutability
* possibility for open source registry implementations with built-in semver support
* central index of modules which can be searched / browsed, if modules are submitted to it
However, those solutions don't guarantee fixes for every tradeoff.
I'd be very interested in learning more about the costs / benefits of the various approaches and the Rust team's discussions on modules. I really appreciate the team's thorough approach to design decisions, as it's manifested into a really cleanly-designed language.
Cool, thanks! So yeah, these decisions were made during the initial implementation, and most of the discussion has only been on one part of this, and so I would say it's an under-explored area. Here's what I've got for you:
1. the only part of this that's been in recent discussions is using the URL as a namespace. This has led to questions around how those are represented in code, given that URLs are not identifiers. Also, using DNS for namespaces is mutable and costs money, this is a huge barrier to adoption that we did not want. Publishing should be free and easy.
2. I think you've already identified some of the tradeoffs that made this be rejected: for reproducible build reasons, referring to external services is considered unacceptable. This is due to stuff like downtime of services we don't control, but also the mutability of those services. In theory, you could proxy them, but it's not as clear what this actually buys you, because to get those advantages back, you'd want to always refer to the proxy, and now you've re-centralized everything.
Another way to look at this is that all of the solutions you've proposed are significantly more complicated than what we've built. That complexity is for good reasons, but given 2, it's not clear that they're good enough to justify the costs. The current situation is significantly simpler for all parties, and the end experience ends up the same anyway.
If anything, namespacing doesn't go far enough. We have a wonderful namespacing mechanism already for the Internet - it's called DNS. We use DNS validation as a security foundation for issuing TLS certificates. Package registries should allow me to "own a domain" - backed up by DNS validation - and upload packages under that name. "Domain ownership" should be paired to a timestamp issued at verification time, so that a) package uploading can be blocked entirely if the DNS verification is lost, b) consumers can continue to download packages under the old domain timestamp c) domain ownership can still transfer as needed d) changes in ownership are tracked, and consumers are notified.
Why does everybody always have to reinvent the wheel? DNS has been around for decades. Using DNS records to verify ownership has been around since, what, the early days of the ACME protocol in 2014/2015? Earlier?
DNS records have a financial and technical barrier to account and package creation that we found unacceptable; and those costs are ongoing, leading to an unacceptable level of mutability.
Well, you don't have to only accept DNS records! It's like arguing that independent email providers shouldn't provide general addresses like @gmail.com or @fastmail.com if they're going to provide any support for custom domains. You could do both. In fact, offering support for custom domains is usually an opportunity to up-sell, particularly to the kind of customers who have already shown that they're willing to spend money on an on-going basis (by paying for the DNS name).
I’m still not sure how that solves the problem, but regardless, the point is that there are many possibilities and tons of complexity. It is not a simple, unambiguous problem with a clear correct answer.
The problem is caused over people squatting on desirable package names, right? Only solves the issue up to the point where people start to squat on desirable namespaces.
If you only develop one or two packages, you don't really care about the squatting issue. Especially if a non-DNS-namespace is free. Light users are generally unaffected by the problem, so they continue to fit within a "free" offering.
Squatting is a non-issue with DNS. Nobody can take your domain from you; effectively, using DNS outsources the control issue to DNS control. The only issue is to guarantee consumers that their builds won't break just because their packages went through forced deprecation when DNS control transferred. The people who suffer from the issue badly enough to need DNS namespacing are either heavy users (who are therefore much more likely to pay) or corporate entities that actually desire a paid option.
For what it's worth, and I don't have a ton of numbers here, the loudest people who want namespaces are people who haven't published a lot of packages, in my estimation. I don't have numbers on that, just gut feels.
I hate namespaces like everyone else, because they're inelegant and kind of uncool.
dependencies: regex
Vs
dependencies: com.cooladtech/regex
But the world doesn't let us have it elegantly, because the library with the name "regex" will almost never be the regex library you want to use. It will almost never happen to be the best, most maintained, best documented, highest performing, with all the regex features you need library.
So instead, the regex library you want will be called "potato". Like a hot potato, potato doesn't let you down when parsing regex.
And then there's the issue of groups of packages that generally make sense together, and no good way to define those groups. So you might have: regex_core regex_engine and regex_api, but it turns out regex_api is for another package and not really part of the group.
Anyhow, back to the article...
Identity? So I read and re-read the section on identity, and I'm not sure what they're saying, it seems they're saying that not having namespaces makes identity worse for discoverability? So basically it argues for namespaces? Now I'm confused...
> A flat registry makes identity management (naming a crate) harder
Than for continuity, it seems to say that if you have a namespace it's bad, because you could confuse two packages of the same name, but in different namespaces?
Okay, I did see this sometimes, mostly in tutorials where they don't list their dependencies, but only their import statements (and sometimes they don't even list the imports). So the reader is kind of left figuring which "regex" lib they are supposed to choose since it exists in many places.
But then the post also says that the lib name can differ from the package.name already? Which means you can already have the same lib name for two different packages in Crates? So I'm confused again?
> Updating to a new package version with different content under different ownership is a real security risk
I have no idea how this would happen unless I manually changed the dependency to point to the one from somewhere else? So I really don't get the risk here.
> Doubly so if you don’t ban new minor versions on the last major version after an unintentional ownership change
Okay, hold on a second. I don't think this article is debating what I think having namespaces mean. Here it seems like the article imagined a world where you depend on the package name and do not specify a namespace, and then in crates.io someone could transfer the ownership of the package to someone else. Or something like that? This makes zero sense to me, I need someone to explain it to me.
> This is a strike against schemes that encourage multiple distinct crates to have the same default name in code. I don’t think any proposal that encourages this could be approved.
Okay, now we're talking. This is a GREAT point. Obviously, the import statement must include the namespace as well. Otherwise you can depend on two packages and have import conflicts where who knows which module you're getting? But is this about namespaces? Maybe the argument here is that the way Rust work, this would not be the case, thus risky? If true, I think this is a defeating argument honestly, something has to be done about this. Also it sounds like this is already a problem, because lib name can differ from package name?
Anyhow, I'd direct people to this amazing post from the Go world, and I think they got everything right when it comes to their view of package management: https://research.swtch.com/vgo-principles
I'm sorry but a language that has data entropy and link rot built into its compiler is not appealing to me.
I prefer the Go approach which lets you host or import packages via a HTTP or git URI. If you have to have packaging built-in to the standard toolchain these days, then please use already well-known and existing protocols like DNS and git. Java also figured this out years ago (com.sun... etc).
crates.io is package repository, cargo is a build tool, rustc is the compiler, and they all three are separate entities. rustc as far as I know operates solely on local files, and has no built-in logic or knowledge about crates.io. And cargo can be configured to use different repository than crates.io, which can have different naming policy
You can also make an alternative registry and add dependencies from that registry.
That said, an alternative registry that enforces namespacing would not work, just because of cargo's limitations on package names that prevent characters like /
It could be done by making every "namespace" a different registry, and for those registries to allow packages with dependencies from other registries (something that crates.io doesn't allow, for a reasonably good reason). Would be quite verbose for the user though.
"bad faith stuff" is not dealt with directly. You could absolutely publish that crate right now, and I would have no inherent recourse about it.
For the most part, unless a lawyer says that it needs to be taken down, things aren't taken down. There is a small amount of wiggle room around registering a large number of packages in an automated fashion, but generally speaking, the principle is non-intervention.
What qualifies as "a large number of packages"? As was posted in another thread, it seems like the packages of user https://crates.io/users/swmon would qualify?
Alternative to just removing them, would the crates team be more open to transferring ownership of these packages?
Yeah, it's not obvious what the person I linked's intent is. Resale? I'm pessimistic that the above mechanism would work but I'm glad that rust is thinking about it.
fwiw I also don't think namespacing necessarily solves this problem, it just shoves the squatting problem up a level.
I don’t know Rust but have learned enough about it to know that lack of namespaces in crates.io and anything at all Rust related Is inarguably a feature
It sounds like both flat hierarchies and namespaces won't work cause both sides have fundamental problems that can't be solved in a way that everyone agrees upon. So what about combining them and having the best of both worlds? I think the docker namespace schema provides a great way to have a 2 stage lookup where the first part of the id (up until the last '/') is a full url to the registry, and the last name in the id is the flat hierachy of names for that registry.
In fact this scheme is already supported with cargo since you can specify the registry per dep. This means the default registry, if nothing is added is 'crates.io/' but you can specify any other registry, eg 'gitlab.local/looks/like/namespace' for personal CI, or even 'crates.io/{username}'
Let's give each user their own registry at 'crates.io/{username}'. This works in a backwards compatible way cause registry:'crates.io/' + crate:'mypackage' is different than registry:'crates.io/mypackage' + crate:'mypackage', so nothing changes with the current crates and it's all an incremental implementation detail for crates.io
Now we have the npm user style namespace ids that lessens the namesquatting problem, but increases the security issues if the registry changes hands. Honestly, I've never really understood the security argument here cause there's no reason an owner can't hand over a single package. As well, there's no reason a currently trusted owner can't patch update a package with bad code. This is a social problem that has no solution and is the same for both naming systems, so we need to come up with another way of handling things. Personally I think all published crates should be immutable and all deps should be pegged, not even allowing patches. But this leads to the argument that sometimes publishers really do want to take down a crate. I'm on the fence with this one, so let's try to solve it in another way.
So the problem is we need to potentially remap any (transitive) dependency to something else, and since we still have a flat hierarchy we really just need to remap the registry for that dependency. So let's add a registry remapping section for the cargo deps ... wait cargo already has something for that (https://doc.rust-lang.org/cargo/reference/source-replacement...). (Reading through the docs here, it does seems like it might be a little strict on whether we can use it for this case, but it's pretty close.)
OK wrapping up, it looks like we could have backwards compatible namespaces right now if crates.io would just implement user specific registries, ie it's an implementation detail. And the security issues are really no different than before. Thus I've got to ask, what am I missing?
Let's say you have a crate (Rust's terminology for a package) named "foo". Inside "foo" you have a few subcrates (named "foo_bar", "foo_baz", "foo_qux"). There are a number of situations where a crate will have subcrates. Sometimes these subcrates are intended to be used publicly (e.g., you want people to be able to use just the "foo_bar" subcrate if they don't need the whole "foo" crate) Sometimes these subcrates aren't intended to be used publicly (e.g., Cargo forces you to put your proc macro in a separate crate, but that's an implementation detail of "foo" and isn't something other people should use directly).
So now you have "foo" on crates.io, as well as a bunch of "foo_*" subcrates on crates.io. Currently there is nothing stopping some random developer from publishing their own "foo_haha" crate. This new crate isn't related to your "foo" crates and might even be squatting on a name you would have liked to use for a subcrate.
Or multiple people might want to create different crates for similar things. e.g., there are multiple regular expression crates. But only one of these crates got the "regex" name. Everyone else has to use a different name, and sometimes people use creative (unintuitive) crate names.
So for various reasons, some people want namespacing on crates.io that would allow someone to reserve an entire namespace. This would allow you to prevent others from using a name that might be confusingly similar to yours. It also allows you to use conflicting crate names if multiple people want to publish different crates with the same name.
TL;DR some people want something like npm's scopes for crates.io. And other people don't want it. This has led to much debate and arguing and whatnot.
It's funny, though, because the team that owns crates.io almost never weighs in on the discussion, and on the rare occasion that one of the team members does say something they always do it in a personal capacity and not in a way that represents the team. This has created its own set of frustrations.