The sad thing is that it’s a joke that many people don’t seem to be in on.
You get all these defenders coming out of the woodwork when people wonder what the hell all the programmers at say, Twitter, are doing. Supposedly you puny mind can’t comprehend the challenges of developing a website “at scale”
The real answer is, almost nothing. People are being paid very high salaries to do nothing.
With an addendum: it's not that they do "nothing", and that's the "problem". The end result is minimal, but the amount of work is still significant.
In any saner place it would amount to almost nothing, but for certain companies it is an Herculean task. And developers know it.
However due to the fact that the company hired thousand of coders, their communication overhead is now too massive. Their architecture also suffers due to the needs of handling those thousands of developers, so there's also a lot of work to be done in coding.
But the pressure is still there. Why hasn't your team changed the color of the button yet? "Why is it so hard?", "Why is it taking so long", etc etc
It is funny because people who are unemployed are shamed for being "unproductive members of society", yet when you get out into the real world, you see all of these people doing nothing and getting paid lots of money, which is even worse than being unemployed in that context.
Anyway, it seems there is just a lot of money to go around and political and social favours dominate a lot of the decisions.
Free market communism as in? Free market economy with communist political system and allocation?
Trying to parse your definition of the term, as it seems like any definition for communism disrupts the feedback mechanism that is a free market's primary (only?) advantage.
It's a free market but with the financial incentive system removed, but maybe social/status/religious incentive retained. Everyone does what they want but at 100% tax rate and the product is shared. Of course this only works in IP because it has zero marginal cost of consumption (and taxation!). In meat space you have the political problem of how the fruits of labor are distributed.
Who pays the developers of said software so they don't starve? The state? Then the state will decide what projects gets traction, such centralized planning is the main reason communism fails. Software engineers would then be more like scientists having to spend most of their time writing grant requests to motivate why they should get any pay, I really doubt this would give society better software compared to the current free market solution.
you could have a ubi system and then add incentives for devs on open source products that are used. Just a thought, I'm sure there would be corruption of this system too
Not all communism is authoritarian/statist. "Libertarian communism" (sometimes known as anarchy) is socialist while allowing freedom of association and economic self-determinism. Historically, it has been strongly linked to the concept of republicanism and unions.
What part of it is most socialist? (Honestly asking and curious, I'm terribly ignorant in the more niche -isms of potential political-economic organization)
It's socialist in the sense of promoting mutual aid and communalism. The union movement was originally very influenced by these ideas. Unions, co-ops, FOSS projects, and community organizations are good examples of these type of political/economic structures.
If you're interested, Peter Kropotkin wrote a lot of books and pamphlets that are freely available.
That's what I am thinking. E.g. I am looking at lichess which has a massive user base and is developed by one full time developer and a few dozens volunteers. Twitter is obviously bigger, but I simply don't buy that you need more than 7000 people to run that site.
I have been an independent consultant for almost my entire career - working with clients large and small, mostly on solo projects.
Took a FT job at a megacorp a few years ago and assigned a new project - I read the specs/requirements and in my IC mind thought to myself - if I was hired as contractor to develop this tool, how much would I charge? and how long would it take me? - and came up with 1 person, for 2 months, and would have been delighted to be able to bill $40K for it, but probably would have went it at $60K on a fixed bid (I always preferred fix bids).
Of course this wasn't a project I was bidding on, I was just the lead dev - the company pulled together a team of almost a dozen people and a budget of just over a million dollars with a 8 month timeframe to get it done - they also budgeted another $120K/year to host it when it was done on multiple big servers in the internal data center.
We spent the full million dollars and got it done - I did however have one small win and pushed them to micro-services solution in S3 and Aws Lambda and got the whole thing to run within the free-tier of an AWS account.
If you ever start a small software company, don't ever hire people that have only worked in large enterprises - they just don't know how to get things done cost or time efficiently.
Maybe you don't need 7000, but software is even worse than icebergs (Only 30% visible). How many people at Twitter are there to fulfill functions we aren't even aware of? Compliance, privacy, legal, internal tooling, etc and god knows what else.
My experience at a tech company with around the same number of employees is that it actually feels understaffed in some of the areas I've seen. It's surprising how much responsibility some teams have.
The whole reason it feels understaffed is because there's too many people.
I know it sounds counter-intuitive, but software doesn't really scale linearly. Unless they're 100% independent, the communication overhead will always be there. Fred Brooks and all that.
With 7000 developers working in a connected system, it probably feels slower and less things get done than if there were less people.
It's not simple to go back to having less people, however. The architecture is most certainly tied to the organizational structure (Conway's Law), and firing people will make it worse in the meantime.
Please elaborate. It doesn't feel understaffed because of communication overhead or being in a connected system, but because of the level of responsibility (I.e. critical system) or product surface area a team is responsible for.
Sure. The things you mention (Compliance, privacy, legal, internal tooling) also have to grow together with the number of developers to scale. Not only those, but also support, QA, management, UX/UI. All those ares have to increase when the dev headcount increases. But since most of those need some degree of centralization and coordination, they can't escape communication overhead and orgs slow to a crawl.
Also, software development teams have to assume more responsibilities just to escape this communication/interconnection overhead: devops is a good example, you no longer have to depend on another team, but now you gotta do operations yourself. An example of avoiding interconnection overhead is having isolated services: now you don't run queries on team Y's database, but you still have to maintain that data on your own DB. Isolation is a good alternative, but it also has its costs.
Unless we're talking about 7000 software developers working on 400 completely different and completely isolated products and answering to different people, there's gonna be a lot of invisible communication overhead, or overhead because of the need to work together with the other 6999. We developers tend to abstract it as being "part of the job", even though it is definitely non-essential complexity.
Even consulting companies doing different projects can fall prey to that: I know lots of cases of internal frameworks/libraries/UI-toolkits being adopted to accelerate development, but they ended up being a bottleneck in development because the team that owned had a full backlog.
Honestly why I'm considering not starting my own business. Getting the product to work is the first 80% of the work. The remaining 96% is billing, authentication, and marketing.
Companies like twitter operate at a different scale. Lichess can off load hiring devs by using out of box solutions like paying for AWS, paying for cdn, using adsense to serve ads, utilizing thirdparty libraries, etc. For large companies, a lot of these are in house solutions, because at that scale, it could be cheaper, they wont as dependent on other companies for reliability, they have more control over where the data is being used or stored. etc.
> People are being paid very high salaries to do nothing.
I'm patiently waiting for some kind of deep-dive historical monograph to be written in the 2050s. What I suspect may have happened is that these companies overhire top talent simply so their competitors can't have them. For the actual work they do, it's massively overpaying. As a strategic investment to prevent said talent from working elsewhere or starting their own business, perhaps it pays off.
I think it is like brand name vs off brand. I just bought off brand oatmeal that was $1.98 vs the brand name was $3.50.
Someone buying the brand name is just operating on an unthinking heuristic that they only buy "the best". I am sure they would justify it after the fact that the brand name taste better even though it is just oatmeal.
So many decision points in society run on that heuristic IMO.
How do I get in on this? For the 10 years I've been in this industry I've never been in a job where I felt like I had it easy. It's always been a rush to get one project done then there's always something else in the pipeline. Whether those projects are actually worth doing is a different question...
Careful what you wish for. It's not "easy" at places like that. It's just different. Instead of your work being keeping a system up and running putting out fires or implementing features or coming up with algorithms on a tight schedule... it's, instead: figuring out who is doing what, what you're supposed to be doing, what permissions list you need to be on in order to access X system, asking to be on that list and waiting, submitting a minor feature to code review that takes a week to go through because someone didn't check their email, rewriting a major part of the code in that review because someone came in at the last minute and didn't like some thing about it that was actually decided on weeks ago but they weren't looped in.
The amount of actual lines of code written by an average SWE at a BigCorp is miniscule. The amount of churn and spinning and hand wringing that goes into that code is staggering. But it's like that for a reason. There's just a lot of people.
It's torture, and in order to maintain productivity you need to have about 5-6 of these processes running concurrently, and it's why you get paid a lot of money to do it.
Do poorly at juggling that and making "progress"? Find yourself on the receiving end of a performance review where the label "meets expectations" actually means "you better start kicking ass at this, or you'll find yourself with no stock refresh grants for next year."
All the money and free food and so on is just a salve to try to make the whole thing less unpleasant. Some people get a high out of the process, while it drives other people slowly crazy.
(In my last months at Google the only part of the job I "enjoyed" was the week I was on-call for the first time on my new team. Because it was the only time in over a year where I felt useful even though it was quite stressful. That's how I knew it was time to pack it in)
Manager pay correlates with how many reports they have. So all of them are incentivised to making it look like their team is spread thin, so even if you don't have to work hard he will make you work hard so that he can tell his manager that he needs more headcount.
Become a contractor or even more so a consultant. You'll often find yourself on dysfunctional teams where management barely knows what they want and things often never get done.
I wouldn't recommend it though since it's soul crushing.
They don't have it easy either. They have to demonstrate at least once a year that they've "accomplished" something in spite of an entire organization of gates that make accomplishing anything nearly impossible. But they still have to figure out a way to do it, and it still has to be somehow represent enough to justify their pay.
It might sound exciting but its actually super depressing. You get paid mad $$ and in ur head ur think "what the ef are these idiots paying us for?". I ended up leaving just because it made me feel so useless i couldn't handle it anymore.
As I commented elsewhere. They're not doing nothing. They're maintaining the organization. In a large company with a service that needs to be reliable, doing things quickly can result in things... breaking quickly and causing headaches when there's no consensus, etc. So everything needs to be done slowly and carefully and looping in dozens of people even for small trivial things. It sucks working this way, so they have to pay a lot for it.
> People are being paid very high salaries to do nothing.
People are being paid very high salaries to be frustrated by explicit prohibitions on doing anything. Nobody gets to being good enough to be hired by a company like Twitter if they don't actually want to do real software development - management exists to crush that desire and ensure that nothing ever gets done.
Nah, most people feel some guilt at doing nothing when they're paid well, so my guess is lots of internal projects that don't go anywhere and that we never hear about. That's quite different to nothing at all.
There's scale, and then there's _scale_. And I can happily believe that Twitter is working at the latter.
It's like something like AWS or Azure. You can provision and deploy a K8s cluster? Awesome! Now, can you do that consistently for the next 100 thousand K8s clusters you're selling to your clients?
I think regardless of scale, spending over a year working on edit functionality is endemic of something being horribly wrong organizationally. Maybe it doesn't even involve the software engineering groups (though it probably does). But that's simply not something that takes even a fraction of that time in a healthy organization.
Yup i recently worked at a company that had a lot of their base technology already built out. They hired a lot of new expensive talent and the talent was pretty bored so they made a big project to re-do all the existing technology that already worked pretty well.
The results are less than good..
This ongoing debate on productivity at small startups vs large companies will never be resolved by argumentation. But as we can see from this thread, corporate engineers overwhelmingly outnumber early-stage startup engineers. Therefore, they tend to "win" the debate by being louder.
The other thing about big corps is that they work on stuff that actually matters to customers for a long time, not "Our Incredible Journey" demo software intended to be rug pulled as soon as the founders get acquihired.
> The sad thing is that it’s a joke that many people don’t seem to be in on.
On each of my job in my early career, I spent considerable effort weeding out people who got too used to such mode of work.
There are software developers with vastly different prior work experience, and expectations. People who do 30 minutes of actual coding a day, and ones who do 11 hours in coding sweatshops.
People who spent their entire career on 30 minutes of coding a day jobs, and think of themselves too much once they rise in seniority are the worst, and are the biggest source of dumpster fire/death march projects.
My third serious job in Canada: work in a consumer electronics trading company, upgrading their shipping system to get shipment status from DHL API, and submitting new ones. My predecessor, and ex-unit head wasted close to $1m, and more than a year, got SAP involved, invited many consultants, change managers, a project management contractor, and just been in the process of "fixing deficiencies" of his work by buying "enterprise SOA solution."
23 year old me called this bullshit on the second week, fought to have a serious talk with the company owner, rewrote a 10 years old steaming pile of PERL, and BASH scripts to Python, Postgres, and JS frontend in under one week, showed it working, and countered "You don't understand SOA!" protests of the unit head in front of the boss. He gets fired, I get a bonus the size of my annual salary.
Since then, my bosses always threw me as a "firefighter" on projects in crisis.
------------
We now have 2 major trends in the industry:
1. More, and more non-IT companies are trying to do software
2. A giant amount of newcomers to the industry who would've normally went working for banks, and management consulting companies eroding big IT companies as later grew with little regard to quality of hires.
You get all these defenders coming out of the woodwork when people wonder what the hell all the programmers at say, Twitter, are doing. Supposedly you puny mind can’t comprehend the challenges of developing a website “at scale”
The real answer is, almost nothing. People are being paid very high salaries to do nothing.