Revenge of the Hard Drive: Storage in an Age of Content Creation, AI, and RAM Shortage
When the story of the modern computing and mobile era is told, we usually start with microprocessors, personal computers, smartphones, and networking. But the importance of storage is vastly underestimated, and not only in apocryphal Bill Gates quotes that “640K ought to be enough for anyone” (he never said that). The Apple II computer only sold in volume after Steve Wozniak designed the Apple Drive II – the first floppy disk drive for a personal computer. Dan Bricklin wrote Visicalc for the Apple II because it was the only PC that had enough storage for the spreadsheet to be useful.
Even in the modern age of cloud-based storage subscriptions, local storage is as important as ever. We’re creating more content, we need immediate access to it, and crucial personal and business data should never be fully entrusted to a third party without a local backup.
Everyone is a content creator now.
No, we aren’t all YouTubers – though that is the #1 career choice for over half of U.S. teens in recent surveys, so the aspiration is there.
Chase Jarvis famously said that the best camera is the one you have with you. That used to mean that even a low resolution cameraphone could capture meaningful images. Today’s consumers actually carry around extraordinary imaging systems all the time. Even mid-tier smartphones can shoot 4K HDR for both still images and video. Once the friction and cost of film was removed, it became the norm to take multiple shots rather than just one, and no, we’re not deleting the “bad ones.” We’re keeping them all and editing them …later. Photography hobbyists might have taken the occasional still life, but it took Instagram and Facebook to instill the habit of taking photos of your daily latte.
Video is worse from a storage space perspective, but even though we know that we may never properly edit the video, we still might watch it later. When everyone is recording everything, there is an expectation that if you want to go back and rewatch your child’s fifth birthday – or just a random day at the park – you’ll be able to.
More people are actually professional (or semi-pro) content creators.
Teenagers who want to be Instagram influencers aren’t the only ones who are creating content. There are full-time content creators and even micro-influencers, but also people working on a side hustle, and small businesses. All of these people are essentially building out professional content workflows.
One consequence of the democratization of digital tools and AI is that it has raised the bar for quality and presentation. Even a simple short form video today might have production values and digital effects that would have required a Hollywood studio and big budgets a decade ago. Aspiring content creators want to look like the pros, but small businesses using these platforms to market and sell their goods and services need to look good, too.
There are entirely new types of data being generated to fill some of these niches. For example, gamers who record their sessions so that they can pull out clips to share on the social media channels later need a lot of storage. Drone inspection generates enormous volumes of video
The tools for doing so are getting better, including the use of AI to actually go back and make new things out of all that footage you took. However, if you use the tools, now you have more data: all the original files you started with, plus the new edited content – and you’re not going to delete the original files. AI speeds ideation: it can generate new variations of your idea until you find a version that works. Of course, that also means that you end up with multiple branches of an idea for content, and you’re likely to save more than one throughout the process. AI doesn’t cut down the amount of work humans do, it multiplies it. You’re going to have to store all that data somewhere.
That data tends to proliferate as prosumers and professionals target multiple platforms to stay relevant and build audiences. For a single piece of content, you might be spinning off as many as a dozen new versions in different file formats, aspect ratios, and even HDR types.
Revenge of the hard drive
The run on components to build out datacenters started with GPUs and then moved to memory. Even consumer storage and memory is affected because the manufacturers are allocating the wafers for more profitable HBM (High Bandwidth Memory) for training LLMs or doing inference in the data center. Rapidly escalating memory and storage prices are making it uneconomical to buy phones and laptops with maximal amounts of onboard storage. Instead, it can make more sense to buy enough for regular usage, then offload the content to fast hard drives and RAID arrays.
Projects are getting bigger, and unless you’re doing all your creation, editing, and version management in the cloud, you’ll need local access to the files for the most efficient workflow. Fortunately, Thunderbolt 4 and 5 make connected storage act more like internal storage, and modern caching techniques that use smaller amounts of higher speed memory can make hard drive transfer speeds approach what was only possible with fully solid state solutions just a few years ago.
Managing all the demands on storage in a modern workflow is going to require a mix of solutions, whether you’re backing up all the video you took on vacation, recording gaming sessions, or storing project files for a social media campaign for your business. Some content you will want to store in the cloud, but there are strong security reasons to at least keep your most private personal content and critical business data available locally as well. That includes financial records, medical history and scans, contracts, sales records, inventory data, and anything you might need if your Internet provider or cloud service has an outage.
For Techsponential clients, a report is a springboard to personalized discussions and strategic advice. To discuss the implications of this report on your business, product, or investment strategies, contact Techsponential at avi@techsponential.com.