Twitter Probably Isn't Going Anywhere

(I do not work for Twitter and I have no experience or visibility into the internals of their tech stack. This is an opinion based on my recent experience supporting and maintaining complex softwares. The opinions expressed here are my own and it's important that everyone do their own research and base their decisions on their own due diligence)

(Yours truly on the TikToks)

I have been on Twitter since 2006. I originally joined because I had a crush on Neil Gaiman. And then he married Amanda Palmer. Then I had a crush on Amanda Palmer. Since then I've organically grown my following to about 850 real people, mostly writers and readers and lovers of Folklore. It's been a space where, unlike Facebook or Instagram, I have been comparatively encouraged to be myself. I have heard it said that Twitter is where people go to become brands, and brands go to become people, but there is a side of Twitter that encourages authenticity and values the personal journey. I have always found Instagram to be a place of inspiration and creativity, much like TikTok, but also like TikTok, Instagram is a place where complexity and reality is hidden cleverly beneath very shiny and polished exterior, like a beautiful car with a crap engine.

Facebook is just where I go to see my friends, real people that I've met and developed relationship with, though even a relatively intimate relationship online is always going to be clouded by the persona people wish others to see.

Twitter has become a place of support and community, especially for the #WritingCommunity, where we lift each other up for exposure and often become consumers of the hobby we ourselves practice. I've been happy to meet and promote the works of many Indie authors in that space.

The idea that Twitter could be snuffed out due to one man's ignorance, greed, disconnectedness, and disregard for human life is frightening. If the idea that Twitter could disappear as early as Monday were to bear fruit, hundreds if not thousands of voices would be lost: the victims of Iranian tyranny, displaced Ukrainians, people who are not able to earn their own living for one reason or another and who rely on crowd sourcing to pay their bills.

Fortunately for us, I don't think that's going to happen, and here's why.

Twitter On The Edge

This was a decent run-down of the latest issues at Twitter, though it does lack a little of the finer details. Here's what is being reported.

The above gives a decent-ish run down of the concerns users are having about how well the software can run on so few people with offices closed.

It is true that large teams of people are needed to keep big platform apps like Twitter running, but there's a lot that people don't seem to understand about how large distributed systems work that makes me think the idea of Twitter slowing down, or shutting off as a result of no one being in the office, is a load of bupkiss.

Twitter Is Software

Like most apps, and anything we use that is a human-interactive interface--called a user interface--there is a complex software underneath the smooth and hopefully enjoyable set of images, text, links, buttons, and however many dozens of interactions that make up the exterior of an application. This complex system requires dozens of people working either together or independently to function under normal circumstances: the addition of new features, the maintenance of old code, the upkeep of hardware, the scalability of architecture, proactive--and sometimes reactive--defenses against external attack. Taking this logic into account, with so many layoffs, strikes, and lockouts, it stands to reason that without everyone having their eyes on the software all the time that it's only a matter of hours before the entire thing falls to pieces.

Yeah...... no.

You see, in the wide world of scalable distributed systems, much of these processes needed to ensure there is enough bandwidth to handle intense load, space to handle increases of data, and protections against cyber attack are largely automated. Case in point, an organization might employ a series of load balancers hosted in the cloud (should they choose to do so) that can be templated to spin up and spin down "instances" needed to distribute intense load. Once load drops off, the "instances" shut themselves down. That is all wiiiiiiildly over simplified. And dubiously accurate (I'm an SRE, not an architect--and not even a really good SRE, just one with some visibility and experience with cloud hosted apps). Databases and hard drives can be backed up on snapshots or redundantly backed up physically (if you insist on doing it the hard way). Software patches for most operating systems are monthly updates if we're not talking bug hotfixes, and monitoring, either third-party or built internally--or a combination of the two--helps folks like DevOps/SRE to see where something is going wrong: "some process is eating up a lot of memory, better reboot that server," or "that machine doesn't usually run that hot. Looks like something is eating up a lot of CPU. Better start checking the logs."

Even some of the processes by which teams maintain a complex software can be automated. You can set up monitors in cool systems like Dynatrace or Pingdom to make API calls to an Ansible tower or Jenkins setup to run automation under certain conditions. Response times are too high on that host? Restart something, clear cache, drop connections, do something to bring those response times back down (responses are as myriad as the issues). Is that data older than 160 days? Purge it. It almost doesn't matter what the issue is; automation can be written to take care of it--most of the time.

In a system the size of Twitter, I find it incredibly hard to believe that human eyeballs are needed to monitor the infrastructure and scalability to the point where the total absence of humans for five days (if their offices open back up on the 21st) means the total destruction of the platform.

In reality, it only takes a few people from each team to maintain a software, even a software the size of Twitter. And of course that number is relative. If our teams are made of up of six people each, it might only require the work of two people to maintain the functionality of the software, and this involves anything that a monitor or automation can't fix, and in a good system with good monitors and lots of automation, those people don't even need to be on-site.

So as someone who has at least two seconds of experience supporting complex software from both the frontend and the back end, I can reassure some folks to a certain degree within an acceptable margin for error that I am myself not worried that Twitter could be dead by tonight. I'm not even sure Twitter would be dead by Monday. In fact, I'm pretty sure that if he screws up bad enough, He Who Shall Not Be Named needs to only hire as many scabs as it would take to maintain the software and protect it from attack.

The Difference Between Development and Maintenance

There are a lot of people, some people in the tech industry even, throwing around the word "dev". The devs have been locked out of Twitter offices. The devs are being worked to death or potentially could be if they stay on at Twitter. The devs this. The devs that.

Do you have any idea how little work devs do when they are not developing? Development is involved in maintenance, but really developers work on producing a stable and reliable piece of software with a reasonable number of new features and is often involved in speeding things up on the back end to make the use of the app overall more enjoyable. No one wants an app that's got 100% uptime but takes forever to load anything. No one wants an app that's got 100% uptime but the front-end design hasn't changed in 20 years. Something is always shifting and changing, and that requires developers.

But when things are stable, i.e. there is a 99.99% uptime (Google suggested service level objective), and no new features or improvements are being made, devs do relatively little work on a software. When no new development is taking place within a software's infrastructure, responsiveness, user interface, data engineering, or communication methods (i.e. how it speaks to exterior and interior systems), devs are largely not needed.

(dude above reminds me of Mauru from Little Spanish Farmstead on Youtube)

When a system is stable and being well-monitored, development falls to those whose job it is to maintain that stability: DevOps, Site Reliability Engineering or System Admins (same function, different paradigms), and Support. DevOps are subject matter experts in how it's put together. SREs know that much, but also monitor the infrastructure and hardware to ensure it's functioning properly. Support is there to answer questions, triage bugs, and escalate issues of functionality.

DevOps, SRE (or combination of the two) and Support work together to ensure that the latest, most stable version of the software has minimal downtime. This doesn't mean no downtime. That's not possible. There is always going to be downtime. Patching requires reboots; hotfixes require services to be stopped; full-scale deployments can drop users, shut-down services, interrupt dataflow, and introduce bugs. It is the job of those housed under Architecture and Support (those are arbitrary terms) to ensure that downtime (regardless of the source) is minimal.

Some Notable Concerns

It's unclear to me--and it's better this way--how Twitter houses its data and hosts its processes. So if there is not a lot of cloud architecture, there may very well be the need for human intervention in the tech stack to make sure hardware is working correctly. This could include replacing cables, triggering firmware updates, replacing hard drives, etc. If that is the case, we could very well see a downturn in performance if not service interruptions over time.

Musk has made it clear that folks can't work from home. SREs take things in shifts, and prior to a lot of more flexible WFH models, some teams did, and do, have to be onsite to handle things. If they can't get in, they can't address alerts.

If there are current issues such as known memory leaks, performance enhancements that can't be made automatically, things that can't scale, or technologies that are vulnerable, we could see interruptions in service if devs are required to be working on fixes for those.

There are still a lot of softwares out there that are not currently in development that are only being maintained. As someone who has some experience with that, I can reassure folks to a certain degree within a reasonable margin of error, that it takes very few people to maintain a stable piece of software.

Conclusions

So when I hear stories of devs and teammates at Twitter being locked out, layed off, severed, or any other of the human horrors He Who Shall Not Be Named seems capable of doing, I hear human voices. I hear human anger. Who do I not hear screaming? Software. Because it's not a person. It's a complex system with probably hundreds of fail-safes built into it (unless they're reeeeeeeally dumb) to keep it going in the events humans stop touching it. Humans stop touching Twitter every night, over holidays, and during code freezes (like we might be experiencing now). It doesn't wind itself down, run out of steam, require someone to go down into the server banks and wind them back up, or slowly grind to a halt in a matter of hours. Something has to happen, and if that something does happen, automation can handle it, or if it can't, very few people are needed to respond.

So no, I don't think Twitter will be dead by Monday. I think a lot of people are coming to grips with how megalomaniacal Elon Musk is, and how disconnected he is from his peers. I think the real tragedy is not that Twitter could be severely impacted by his decisions, but that we have become so ingrained and dependent on our social media that this might actually be a problem for us. The problems facing the world today requiring true intervention would require us all to put down our phones and pick up a weapon to defend freedom as we know it.

Twitter dying isn't one of those.

Find me on Twitter @SquealingNerd EST. 2006