Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
USB Mass Storage and USB-Attached SCSI Are Both SCSI (devever.net)
77 points by hlandau on Sept 11, 2020 | hide | past | favorite | 39 comments


SCSI rules the world I guess.

ATAPI tunneled SCSI commands over PATA for CD drives and other such things.

AFAIK SATA uses the SCSI command set too (as does Serial Attached SCSI obviously). To the point that most SAS controllers can also run SATA drives without issue.

Apparently USB mass storage also uses SCSI in both BOT and UAS forms.

Sometimes a protocol is designed so well to fit its purpose yet just flexible enough to support future use cases that there is no need to replace it. Hats off to the original designers of the SCSI command set.


SATA doesn't use the SCSI command set. It's all still ATA, right down to the inability to abort tasks. (But it is common for operating systems, such as FreeBSD, to model all commands internally as SCSI commands and just translate them to ATA commands if necessary at the last minute.)

SAS HBAs are backwards compatible with SATA drives because they're specifically designed to be. So I guess they're capable of passing either SCSI or, when SATA drives are used, ATA commands.

Where it gets amusing is when you use SAS expanders, since then the link from the HBA to the expander is SAS, but the link from the expander to the drive is SATA. Thus SAS has a special protocol called Serial ATA Tunneling Protocol (STP), which allows ATA to be tunneled over SAS.

What's really comical is that this means if you attached a SATA optical drive to a SAS expander, it follows that you'd be talking SCSI tunneled inside ATA/ATAPI tunneled inside STP over SAS.


AFAIK the SATA just takes the set of registers an IDE drive would have, and puts them in a different physical/transport protocol.


The ATA bus is a cut down ISA bus, with some address lines cut. All protocols on it operate in terms of ISA bus transfers, with SATA replacing the physical layer but nothing else.


just to complete the story, SCSI does have ATA command pass through. That is ATA commands over SCSI interface. In Linux, the libata library provides this implementation.

There are SATA drives that understand SCSI over ATAPI albeit in a limited way just to be clear.


>just to complete the story, SCSI does have ATA command pass through.

Very interesting! I just looked it up - it's the ATA PASS-THROUGH command in the SCSI-ATA Translation (SAT) specification. Someone just told me this is apparently a common way to read SMART data from hard drives in USB enclosures.

>There are SATA drives that understand SCSI over ATAPI albeit in a limited way just to be clear.

I assume you mean hard drives. This is extremely interesting. Can you provide more information? Examples of drives, any links?


Some protocols have so much persistence, where others seem to be tied to one vendor's fate.

I wonder if its worth spending some time trying to understand what makes some protocols so robust and widely adopted. It's not just being open vs closed.

A few other examples:

  - PostScript (probably) / PDF (iffy)
  - CAN (adaptable)
  - G-code
  - Serial
  - HTTP
  - HTML (sort of, badly)
  - VNC (maybe)
  - JSON (for now)


Hayes command set, as expanded from land-line modems to GSM cell phones:

https://en.wikipedia.org/wiki/Hayes_command_set


- FAT32 became the only filsystem people use for external drives due to cross compatibility

- The QuickTime container became the container for AAC audio, ISO MPEG4/h264/h265 video, HEIC, AVIF

- Zip files have become a generic container used in all kinds of formats (Java jar files, Android apk, Microsoft office documents, usdz, etc )


SCSI itself came from a single vendor as "Shugart Associates Standard Interface", otherwise know as SASI which went on to become SCSI. It was a simple interface, easy to program and made it possible to hook computers to disk drives with a handful of TTL chips. Good, simple design that did the job so no surprise that it's survived so well.


On the other hand, I'm really glad I can plug in USB flash drives without worrying whether to plug in a SCSI terminator between the drive and the computer, or on the other end of the drive…


Virtio initially offers virtio-blk for storage virtualization but now virtio-scsi is recommended to support additional features.


And in the Linux kernel, a large part of the SCSI codebase is also used for many non-SCSI storage devices for its convenience, hence /dev/sdX.


Years ago, I think when I was still using a Commodore Amiga system, I went into a computer shop in St. Louis to see if they had any SCSI drives.

The clerk at the store went off on this surprisingly lengthy and heated diatribe about how SCSI was mediocre technology, and how nobody in their right mind should ever want anything SCSI ever again.

I kind of miss that sort of passion in computer store clerks.


Passion related to computers as a hobby is the thing I miss. Once upon a time if your were "into computers" that meant something, now everybody thinks that they are into computers. "Like facebook right?" <eye-twitch>

My first x86 PC had a 425MB SCSI-II hard-drive and a "triple Speed" (3x) CD-ROM. Having a SCSI drive with a seek time of UNDER 11ms was enough to blow the doors off of every other computer I came across (IIRC 4ms was the actual time vs 11ms for most other 120MB HD's at the time.)

Everything about SCSI was cool. Over powered, over engineered, and willing to take abuse better then anything else.

It was an AMD 386DX-40, 8MB of RAM, a VGA-Wonder XL video card, a Logitech bus mouse, soundblaster, and 2 modems; 14.4 & a 56k USR.

AND I ran linux on it. Well I could install linux, and couldn't really figure out what to do with it after that. But Even back then Linux detected EVERYTHING, and it was FAST.


The main thing I remember (I didn't worry as much about performance back then) was when I was grabbing a replacement from a pile of ribbon cables we had in a box, the wider ones were for SCSI, the narrower ones for IDE.

Then SCSI-II and SCSI-III and ATAPI/UATA/PATA cables came along and started making the ease-of-identification based on just visual traits a lot more difficult.


Now ask people if they have Internet.

"What, no, I have Facebook."


> The clerk at the store went off on this surprisingly lengthy and heated diatribe about how SCSI was mediocre technology, and how nobody in their right mind should ever want anything SCSI ever again.

vs What in 1990? Pray tell. Parallel IDE? I'd have laughed in his face if he said that.

The biggest problem with SCSI was always the fact that you had to understand termination or the bus you built wasn't going to work.

The second biggest problem with SCSI was always the fact that it was more expensive. Cables were especially expensive, but controllers weren't cheap either.


> vs What in 1990? Pray tell. Parallel IDE? I'd have laughed in his face if he said that.

Yeah, back then Macs were bad, but at least they had SCSI, so they had something going for them. Of course, then they went to IDE, and had nothing going for them anymore.


> The biggest problem with SCSI was always the fact that you had to understand termination or the bus you built wasn't going to work.

This is just a fundamental limitation of a high-speed parallel shared bus for 1990s' technology, not really specific to SCSI. Termination is needed because of physics, there was no way to workaround it. Moving termination to the drives and backplane is a solution (e.g. PCI uses it), but I guess it was too inflexible for SCSI when cables were often chained together. Ultimately the problem has been solved by Moore's Law - when chips are powerful enough, it makes sense to just stop using parallel buses entirely, and move to point-to-point serial lines. Termination is implemented at end-points, and the bus is never shared, eliminating this problem.

Just like how Ethernet began as a low-cost bus network (and yes, you need to add terminations on 10BASE-2) and ultimately evolved to point-to-point serial lines.


> Moving termination to the ... backplane is a solution (e.g. PCI uses it)

While PCI does solve the reflection problem in the backplane, the PCI bus is not terminated. When a signal first passes by the receiving device it only sees a weak signal that is only a fraction of the expected amplitude. When the signal reflects off the end of the bus, it passes by the receiving device again. The receiving device sees the initial signal's amplitude added to the reflecting wave's amplitude as the expected full strength signal.

https://en.wikipedia.org/wiki/Reflected-wave_switching


>> Moving termination to the drives and backplane is a solution.

I should have used "and/or".

> the PCI bus is not terminated.

I'd say PCI, arguably, is still terminated. It just uses series-termination instead of parallel-termination. In a parallel-terminated line, when the signal hits the receiver, the termination resistor to ground absorbs the energy, an ideal textbook scenario. In a series-terminated line, there's a resistor in series of the transmitter, which is the termination of the line. When the signal hits the receiver, it reflects back to the transmitter via the series resistor again, and the energy is absorbed.

Quote High Speed Digital Design by digital guru Howard Johnson,

> 1. The driving waveform is cut in half by the series-termination resistor before it begins propagating down the line.

> 2. The driving signal propagates at half-intensity to the end of the line.

> 3. At the far end, the signal reflection coefficient is +1. The reflected signal is half-intensity. The half-sized reflection plus the original incoming half-sized signal together bring the signal at the receiving end to a full level.

> 4. The reflected signal (half-sized) propagates back alone the line toward the source, where it damps out at the source termination.

Note the step (1) and step (4), when omitted, gives an impression that Reflected Wave Switching has no termination at all, which is not how it actually works. The termination resistor is chosen in a way that, when the resistor is combined with the driver's internal impedance, they approximately match the characteristic impedance of the transmission line, thus ensuring the termination is optimal.

> Reflected Wave Switching

It's the most common method to terminate a point-to-point digital interconnect, usually people just call it "series termination". I just finished designing a circuit board yesterday and the data interface was terminated using this technique. Reflected Wave Switching was just another confusing term for that ("confusing" because, while it's a fairly descriptive name, but in many books with a voluminous discussion on serial/parallel termination and whether the signal is reflected, the term Reflected Wave Switching is often not explicitly mentioned. On the other hand, we have computer books that mention Reflected Wave Switching in PCI, but without a discussion of how series termination really works).

The problem of series-termination on a shared bus is, first, there's no clearly-defined transmitter or receiver, a signal can come from either end from multiple devices. Also, the impedance of the bus is affected by the number of cards and the physical locations of the slots on a backplane. It's simply impossible to select an optimal series termination resistance to achieve an ideal series termination. Using series-termination in a shared bus is a hack, but PCI managed to successfully engineer a "good enough" compromise solution for PCI's data rate.

One can say PCI doesn't have termination since many drivers feature on-chip series termination. But it's true for other interfaces as well, so I still like to imagine PCI's "Reflected-Wave Switching" as a compromised form of series termination, as described by Howard Johnson [0].

[0] http://www.sigcon.com/Pubs/news/3_3.htm

---

Ultimately, it was the transition to point-to-point serial lines that ended all the termination headaches.


"Years ago" I was always told by my computer geekier friends that SCSI was better than IDE. And that you could put it in RAID arrays etc..


Looks down at Ryzen PC with three SCSI CD-ROMs:

"Hear that fellas? You're just mediocre."


> Ryzen PC with three SCSI CD-ROMs

Out of curiosity, what's that for? I'm struggling to think of a case where reading that many optical discs is actually useful


My first ever CD burner was a external SCSI drive. So that computer ended up having 3 drives, but realistically we only ever used 2: one for music, one for a program usually. We didn't rip the music because a) it took forever back then to encode to mp3, and b) we didn't have enough HDD space!

I remember when we were trying to burn CDs, about 1/4 of the time it failed. I pointed out that we didn't have a SCSI terminator on the other slot, but my dad insisted what difference could that possibly make? And to be honest, at the time (I must have been between 8-10 at this point), I couldn't really disagree with him. It was just this chunk of plastic that plugged into the port, it looked more like a protector than anything actually useful. Anyway, we tried it out and it did resolve our issues, but I still had no idea why until I got to college and learned about different buses in some computer engineering class.


Can't speak for the OP, but I used a rig with three CDROMS on it to rip my wife's CD collection to the first iPod.

It took months, but would have taken months longer if I only had one drive.


Sounds like a hassle. Didn't you need to buy a PCIe-SCSI HBA somewhere along the line, along with some kind of terminator for the upper half of a modern SCSI bus? Aren't very good SATA optical drives $20?


Some B350 boards still had legacy PCI slots. I have another PCI card I need to use so there was a spare slot left over. My use case was ripping CDs and for that you are better off using a longer wavelength CD-ROM laser and Plextor drives in particular for their better firmware capabilities.


My first CD burner came with a PCMCIA ("People Can't Memorize Computer Industry Acronyms") SCSI card. That only had Windows drivers. It might be supported by the Linux kernel by now, but it's long gone. And I don't have a computer with PCMCIA slots anymore.


ATAPI and SATA are also SCSI at their core.

https://en.wikipedia.org/wiki/ATA_Packet_Interface


You can tunnel SCSI over PATA/SATA using ATAPI, used for optical drives. That's a bit different from SATA being "SCSI at their core"; the entire point of ATAPI is that ATA isn't SCSI.


In the early days of USB, I sat through a presentation from a portable storage vendor at Comdex. They specifically made a point to mention that their external USB storage was seen by the operating system as a SCSI device because SCSI support was widely implemented by all operating systems at the time without having to install additional software.

Perhaps ubiquitous OS support was one of the main driving forces behind the SCSI protocol being adopted by so many technologies since.


I wonder what would change if we tried to nvme-ify.

In the meantime I hope USB4 with pcie takes off. AMD needs to make it happen. Not enough mobile chip makers about for them to try to be competitive & throw in support on cell phones, not till 2026l5 or so I'd guess, even though it's be so neat & thoroughly bridge the chasm between pc & mobile.


Why would anybody ever use ACA with either BOT (that doesn't support it) or UAS?

It's basically a solution in search of a problem, since it is needed when commands are delivered asynchronously with respect to responses but you don't have autosense. BOT only has a single command in flight so it didn't need ACA, but UAS has autosense so it doesn't need it either...


Author here. I'm no expert on the problems ACA was intended to solve, but if I understand it correctly, I could see some remaining applications for it.

Suppose I schedule 10 commands all at once, with the Ordered task attribute. If the third command fails, do I necessarily want the remaining commands to be executed? Possibly these are a logical sequence (e.g. for a tape drive, rewind, write, etc.)

I have no idea if anything actually exists which can make constructive use of ACA in this way, but it occurs to me that there is still potentially some use there even when autosense is enabled.

That is, if I'm understanding ACA correctly.


True, the problem is that no one has ever implemented ACA in the target. :-) So it does not really matter if the transport supports it.


The article seems slightly pedantic in the argument that Mass Storage is "SCSI"—it implements a subset of the protocol, so yes, it has some bits of SCSI, but I don't think it follows to say "USB Mass Storage is SCSI".


Every SCSI device implements a subset. The difference is the transport; higher layers of the SCSI stack, like the part that handles SCSI disks (as opposed to eg optical drives), don’t really care. They talk SCSI.

As a sidenote, if you configure FreeBSD on Raspberry Pi 0 to serve USB Mass Storage (like the gadget functionality in Linux), the USB-tunneled SCSI requests will be handled by the same target that serves Fibre Channel or iSCSI, with persistent reservations, VAAI and other features that might feel like a bit of an overkill for a virtual flash drive.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: