Sounds like you have a solution that works with today's medium-sized drives, so how is this even relevant to a discussion of 60-120TB drives? Do you think you'd be well served by putting that 96TB on one drive? I doubt it. If your active set (for any useful or relevant definition of "active") is X and the number of drives you need for IOPS or MB/s is Y, then the ceiling for how much you can effectively use per drive is X/Y. That's quite likely less than current drive capacities, and almost certainly within what can be achieved by current PMR technology. How would HAMR or BPM or any other more fundamentally new disk technology actually help you other than to store cold data on that "spare" capacity?

Yes? I mean, the sequential write speeds are fine for many video-heavy applications. You don’t want it in cold storage. Tape drives are not cheap. And neither is retrieving from tape terribly practical, either. There’s a large overhead for tape storage if you want it automatically accessible (ie a big tape jukebox). A big hard drive can store large volumes of data just a few milliseconds away, not minutes as you have to wind the tape to the right spot in a big tape jukebox... or hours or days if you have to ask someone to retrieve some tapes by hand. And can do hard drive storage for an order of magnitude lower cost than SSD.

Spinning disk is still really useful for video. Which is not shrinking any time soon.

That's a lot of time spent trashing tape, which I never suggested as a solution. The central question remains whether that single huge disk will provide the I/O rates that you need, and it won't. If video sizes increase and frames/second doesn't, then I/O need goes up so you'll need multiple drives even more. That means the sizes we already have, without fundamental shifts in technology, are even more likely to be sufficient. You'll just need more of them. What part of this math is escaping you?

No math is escaping me. Video is sequential I/O, which hard drives do fine at. Multiple drives is needed under many workloads, but not video.

If you want to do video stuff on a laptop you have room for maybe one extra hard drive. And if you are willing to have an external drive (which isn’t too bad), it’s not going to work to plug several hard drives in and expect to RAID them together.

Secondarily, there are only so many drives you can stuff in a workstation. A lot of compact ones only have a couple slots in them, and not everyone wants a RAID or JBOD controller with a bunch of ports on it just to do a video workflow. And if you have a video surveillance small server or appliance, you might only have a few slots in it (4 is fairly common).

So again, even one or two hard drives is fine for most uses. Not everyone is gonna put a 24 slot JBOD/RAID chassis in to just provide enough storage space for their video surveillance system or whatever.

> even one or two hard drives is fine for most uses.

The scenario you describe hardly sounds like "most uses" to me.

"Video" is ambiguous use case. It would be fine for watching video on home server by single person or hoarding video files, but obviously not fine for video editing, video CDN edge, etc.

You asked, "What market is there for big hard drives outside of FAANG [etc.]?"

I answered.

The home prosumer market. And if anyone ever bothered to develop a multi-terabyte solution for "normal people", whereby they could easily have their DVD / Blu-ray / Compact Disc collection converted from disc to a media server, probably a lot more people.

100 TB hard drives will end up in consumer PCs by 2035, I'm certain of it, even if we won't really need it. 8K is just plain stupid for home use, I honestly don't know why its being pushed, since you'd need a 120" screen or more, but I'm sure there'll be new media types beyond 4K... 4K60 FPS for instance, or high-resolution VR movies. I could easily see a world where 4K60 FPS movies take up 200 GB space each, and high-res VR movies are that large or even larger.

> You asked, "What market is there for big hard drives outside of FAANG [etc.]?"

No, I didn't. PaulHoule did.

> 100 TB hard drives will end up in consumer PCs by 2035, I'm certain of it, even if we won't really need it.

I think you're confusing the need for more capacity with the need for more capacity within a single drive. This is the same distinction you'll find in the original RAID papers. With a big single drive ("SLED" in the papers) your performance per megabyte stored and your reliability per megabyte stored keep going down. At some point this will make your system unusable. I contend that we're already on the edge and these newer technologies that only increase capacity will push a lot of people over. Increasing video resolutions only make the problem worse, not better. How many 4K60 streams are you going to get over a single interface?

Now consider an alternative: same amount of storage, across multiple drives. In the same physical space, because that higher areal density can be used to make drives physically smaller just as well as it helps make them logically bigger. In a similar power/heat envelope, because smaller also means lighter. But with better performance across more heads and more external interfaces. And with better reliability because with multiple drives you can add some redundancy (and even if you don't at least a single failure will only cost you some of your data).

Why would you want a single big disk instead? Complexity? It's not actually that big a deal. Non-specialists build such systems every day. The only thing the drive vendors need to do is use those improvements in areal density to make drives physically smaller instead of logically larger. Stop making the capacity/bandwidth gap bigger. Let people build balanced systems that actually work, instead of on systems with a striking resemblance to those used for "virtual tape" cold storage in the bad old days.

> Increasing video resolutions only make the problem worse, not better. How many 4K60 streams are you going to get over a single interface?

How many does a prosumer need? One, maybe two. Maybe 5 in a NAS. A single drive today can already handle that in 8K.

When it comes to media files, we hit storage limits all the time but we're nowhere near the bandwidth limits of a hard drive. We're not on the edge when it comes to performance per megabyte. Program files and game files are miles away on one side, and videos and photos are miles away on the other side.

There's some point where increasing the density of hard drive platters is too slow for a prosumer media library, but I'm confident it's far out there, past a petabyte.

> we're nowhere near the bandwidth limits of a hard drive

According to my calculations, SATA-3 could support almost four 4K60 streams, but only if the data for those videos was very carefully interleaved (never happens). Also literally nothing else happening on the drive, no bad-block relocation, no bottlenecks elsewhere in the system, etc. Closer to reality, those files would be laid out on different parts of the disk and you'd lucky to get even two concurrent streams without seeks between them ruining your throughput. So yes, you are at the real-world bandwidth limits of a hard drive.

By contrast, two physically smaller drives adding up to the same capacity in the same space could reliably deliver one stream each, plus probably a third with data stored on both if you had decent buffering (because now you have enough MB/s headroom to buffer) to cover the seeks that remain. Just as when I was building storage systems for video professionals in 1994-95, if I had to deliver such a system and it had to work before I got paid I know which way I'd go.

You're talking about raw very-lightly-compressed footage I think? But cbozeman and I are talking about final products. The high end is around 10 megabytes per second per stream.

If you have raw 4K footage, and you're editing with it, and you need multiple streams and the ability to scrub around, I would simply say not to use any hard drive.

What is the purpose of "prosumer media library" for? I suspect it's not for editing because of HDD is too slow for such use case.

The stuff you're actively editing goes on your SSD, everything else can live on a hard drive.

But the media library would be photos you take, videos you take, DVDs you rip, etc.

So what percentage of the market would you say that is? 1% would be generous. Should that drive the technology?

I would say that the vast majority of the consumer hard drive market is in the bracket where size matters much more than performance.

I have no idea what fraction of the server market.

But there's also an important thing to note about product families. You talk about using density improvements to make drives smaller, and then install more of them to keep performance up. I think that's reasonable, but I also think that one of the best ways to do that is to reduce the platter count and drive height. In that world, where the main product is thin drives, it takes only a small amount of engineering effort to keep making an XL model that has lower performance but is significantly cheaper per TB.

I said I'm not an expert on the HDD market, but I have seen some stuff so here's a bit of perspective. The hyperscalers are more than 50% of the market. They're each big enough that price distortion from their own buys is a real concern that they plan for. (Note BTW that these are often left out of analysts' charts, because leaving them in makes it harder to see patterns among the rest.) More than half of what's left is sold to businesses, with most of that going to companies big enough to build and run their own data centers (including supercomputer facilities). And then, finally, all consumer drive sales account for something less than 20% of the total.

Yes, size does matter a lot for those markets. If you want performance use flash. However, again, size per drive is a red herring. What matters is the capacity you can fit into a system, whether it's a laptop or a server. Having that much capacity present as a single volume through a single interface is simply not ideal either for performance (which might not be the same goal but still has a lower limit) or for reliability. That's all I've been saying. You're better off combining multiple lower-capacity drives, even if you can get by (at least for a while) with a single larger drive. Serious video folks and even gamers have known the advantages of dual drive RAID-0 or RAID-1 for years.

> where the main product is thin drives, it takes only a small amount of engineering effort

What you're now suggesting is no more than what I suggested nearly a day and several posts ago (look for "drive manufacturers could help"), which you and "others" took issue with. Yes, drive manufacturers can and should make those thin drives, and then sell multiples packed into a single enclosure like we already have today. The fact that it's multiple physical drives could be more or less transparent. The transparent version would be cheaper and offer the system designer more flexibility. The non-transparent version, akin to existing HW RAID or even multiple platters today, would be a bit easier to conceptualize for people not used to thinking of enclosures and spindles and platters and heads as separate things, but it would be a bit more expensive (controller plus memory as part of the package) and not necessarily better.

In short, using "drive" to mean both the package with connectors on the side and the piece(s) of oxide-coated metal inside it is sloppy, and leads to wrong conclusions. Once you realize that higher density creates more options than "every limit the same except for higher capacity" then it quickly becomes clear that 120TB on a single spindle isn't the best use of that technology.

> However, again, size per drive is a red herring. What matters is the capacity you can fit into a system, whether it's a laptop or a server.

I agree, but the argument I'm making is about cost per terabyte. I'm not inappropriately clinging to terabytes per drive.

> What you're now suggesting is no more than what I suggested nearly a day and several posts ago (look for "drive manufacturers could help"), which you and "others" took issue with.

I didn't take issue with smaller or multi-component drives existing, I just don't think they are necessary for all use cases. I was referring to what you said before on purpose, but disagreeing with the conclusion that "mostly we'd all better get used to higher drive counts". I got the impression you were treating it as a temporary transition measure.