This comment reminds me of a joke where the punchline is that a person is so poor that burglars break in to their house and leave money.
Similarly, I could see ransomware groups hacking in and feeling bad for GH so they improve a few things to help them get to at leave nine fives of uptime.
Many years ago there was an attack that went around that used the server’s BMC as an entry point. Thing is, BMCs are universally shit, so as part of the attack, the attackers also fixed a bunch of bugs so their connection could persist. I was working in hardware management at the time, and when we heard about that, we all gave that one a hard think…
On hn, a joke increases its fun factor by being over-explained in excruciating detail with several digressions into related jokes and the history and philosophy of joking, and someone ends up showing a site they made with all the possible variations of that joke and something about the scrolljacking css annoys one of the commenters enough that they break in and fix it.
A variation of that joke is used in Zen Buddhism as a teaching story. A famous monk, who lived in voluntary poverty in a mountain hut, wakes up in the middle of the night because a robber had broken in - except the robber couldn't find anything of value. So the monk listened to the rummaging sound for a while, and feeling bad for the robber's family, offers his blanket. The robber is so surprised by the kindness of the monk that he gives up his stealing ways and decides to become a good guy.
The availibilty problems are caused by incapable managers overloading Azure boxes, code fixes will not help much. Maybe they get into HR and help get them fired. And help rehire the ones who could fix it. But that needs a nation state actor, not just your best hacker group.
It's certainly not the right platform. It'd be one thing if they had any official communication on the matter anywhere else. Maybe they're ashamed and are trying to limit the visibility while only technically issuing an announcement.
They announced this exclusively on X.com, which ranks barely above Pinterest in terms of usage. That's below Reddit, Snapchat, WeChat, and Instagram, and requires a user account to view profiles and posts. And that's ignoring all the reasons X is a divisive platform with an extreme political bent.
GitHub chose not to announce this on any other social media either (BlueSky, Facebook, TikTok, YouTube, LinkedIn, or Mastodon, as of this posting, and with no emails sent on the matter.)
I’m not on X, so it’s good to know I don’t matter in tech. I always suspected. Since I’m a paying GitHub customer, though, I should probably matter to them. The right forum for GitHub to post this is with their status page, their blog, their website, or an email to all their customers. Using any sort of social media for this kind of thing is either incredibly sloppy or very intentionally quiet. Given that my tiny employer has a better incident communication plan than this, my guess is this an attempt to downplay things.
Maybe we need a cultural shift then, because if one needs to use a platform like X, nowadays owned and operated by fascists, then there's something deeply wrong with the tech world. It'd probably take a lot of effort to do so, but it'd be absolutely worth it.
Besides, even if that wasn't a consideration, only posting the announcement to X is just crazy. As others have said, you'd expect for GitHub to make the announcement on their official website. Any paying client would then just follow that for their announcements.
I just spent a few minutes trying to think of a better place, I can't think of one, there is no professional social network, and linkedin doesn't qualify.
You don't need a professional network. This is a company informing customers about a security issue. It should be on their website. Anyone can subscribe to the RSS feed if they are a customer.
Remember RSS?
There is no need to add a social network element.
It's been pretty common in the past for tech companies to announce outages and quick updates about them on twitter for decades. I'm sure their status page etc will be updated soon, but it's historically been the fastest way to get things out to the wider audience whilst bypassing the "official mail out" review by marketing etc.
I think that was a lot more justifiable when Twitter reliably let logged out users read tweets. X seem to tweak it all the time, or maybe it’s just broken a lot, but sometimes I can’t even load a tweet in a browser that isn’t logged in.
It doesn't show live profile pages to logged out users since a while ago. You get cached summary pages, an age gate error, or sometimes a straight up 404.
Most individual permalinks (.com/username/1234...) don't work without logging in, either, and the official client now uses `/i/` in place of usernames for permalinks(bogus usernames always worked; pkey was the timestamp).
This means an organizationally shared Twitter account for announcements is not a viable concept, at least until Twitter is to be transferred again to whoever would be a better keeper of it.
Even if it's a wingnut dense place, there's good arguments for using a channel independent of your infra in a case like this. You (or Github themselves) don't know if their status page is pwned.
They should send messages directly to their customers as a first step in addition to posting an official article on their site. That’s the minimum. If they haven’t done that then it is hard to defend.
Beyond that, Twitter is the de facto default dissemination vehicle, due to its reach. Even if people are not on Twitter, they are likely to see things from people that are on Twitter.
So? Is this where your corporate paying clients should find out about an issue of this severity?
Not to mention Twitter is not an open platform anymore! (A) I'm an employee in an organization paying for Github. (B) I don't have a Twitter account. I already have a Github account because of (A). Why should (B) stop/delay me from getting official comms about this?
> I can't imagine they'd spam every account with an email address
It's not "spam" if it is relevant to me, such as security incident disclosures.
Also, as tiffanyh pointed out, what's wrong with Github blog or is that exclusively for marketing fluff now? That would've been appropriate enough, without having to spend Sendgrid credits.
Probably the best option after sending a mass email when customers need to take action.
The status page is for reliability issues impacting end users & the blog is for in-depth analysis.
I mean if you are going to use AI which was trained on code of statistically mediocre average at the best, have outages and major incidents every few days, why not go wild and start publishing incidents to twitter too? It checks out with the rest of the stuff.
GitHub: " Our current assessment is that the activity involved exfiltration of GitHub-internal repositories only. The attacker’s current claims of ~3,800 repositories are directionally consistent with our investigation so far."
Pre-AI, having access to code (e.g. if it leaked or even just open source) could allow hackers to more easily discover exploits. I wonder if that threat is now much more severe in the age of AI. Thankfully GitHub have probably themselves run their code through many AI security tools so any vulnerabilities would have already been found and patched. Hopefully.
The security issue aside, seeing more companies push announcements like these on X as the only official source is a trend I'm not sure I like.
I can understand the rationale, this feels lighter and not something that belongs on status.github.com or the blog. Maybe what's actually missing is an official channel for ephemeral stuff on a domain they own, somewhere between a status page and a tweet? Just sharing an observation.
Are you from 2015? Companies have been announcing stuff on Twitter for a decade, and the rest of social media has been regurgitating Twitter posts for almost as long. Newspapers routinely quote Twitter. All that happened before they even renamed it to X.
I’m not saying it’s a good idea. I am saying it somehow became the single source of truth for the Internet with all that entails.
Which is why I wouldn't want to normalize it being the kind of place where company announcements are made. IMO anyone who sees it as worrying is right, and I'm glad they're not desensitized.
Just because it's been going on for a decade doesn't make it any less crazy that Twitter has become a primary source of news.
As a stock listed company is GitHub or Microsoft not required to disclose such security breaches to their shareholders? As in a stock market communication?
GitHub: "We are investigating unauthorized access to GitHub’s internal repositories. While we currently have no evidence of impact to customer information stored outside of GitHub’s internal repositories (such as our customers’ enterprises, organizations, and repositories), we are closely monitoring our infrastructure for follow-on activity."
This reminds me of George Carlin standup routine about PTSD. If you want to make any bad news sound less bad, just wrap the concept around complicated jargon to sterilize it.
How hard would it be to have one installation step to be to have Claude read through all the code to the extension and strip out anything that looks risky (ie. Calls out to external servers).?
Do that automatically for all code downloaded from the web and run outside a sandbox.
Maybe won't catch everything, but should catch most evil stuff, especially if a variety of models and prompts are used.
I installed Zed on a work machine at a well-known software company and a week later they forced me to reimage my machine because they got some alert that the app was attempting to access browser credentials :(
No shade on Zed, sometimes in-house security tools just don't like new software.
I really need to find the time to properly test Zed. I'm mainly using PHP Storm and I love what it can do, especially when it comes to code discovery and auto-completion. I'm not a huge fan of having a bloated toolbox, I never use PHP Storm's included terminal or database browser.
Zed was super impressive when I first started it, but I don't know yet how it compares with PHP Storm.
The extension capability is much less powerful than VSCode (no embedded web view) so it's a lot harder to pull off crazy stuff. All of the language support is done via language servers.
In this day and age, and extensión is the thing is ask my local AI to do for me. They are very simple, self contained code that can be crappy as I'll run it locally.
Browser extensions have been a great playground for me.
Can you cite this? It's not YAML execution syntax, surely Github doesn't do it, the only vector I can see is if you put it unquoted into a shell script inside of a GHA yaml.
All you need is user content containing `backticked`, and a github action referencing that via eg "github.event.issue.title" where the shell would normally execute `backticked` as a command (like echo, cat, etc).
This is bad. If they came out announcing this, without a long winded explanation and further details, it's because they're staring at a bottomless pit and they haven't put the lid on it yet.
For a Fortune 100, to go out of your way to spook investors is the least desirable approach.
Letting people know promptly is also the right thing to do and probably mandated by (at least some) customer contracts. You can't tell just some people; it would leak anyway.
Part of this is likely driven by regulations. Github has plenty of clients that fall under DORA, NIS2 or both.
I don't remember the exact wording about what qualifies as "incident" or "major incident" but the TL;DR is that the regulated entities are required to notify their regulators of impactful supplier incidents within 24h with initial information and within 72h with more complete details.
Which in turn means that Github will have signed contracts that bind them to accommodating timelines.
Sympathy to engineers and everyone at github, it's good that they're being open even if findings are limited. I'm sure they will figure out the root cause and will publish results to be a learning experience for everyone else
I would say, first and foremost, the era where a developer machine with source code access also has access to meaningful security systems should be over. Internal repository access should mean nothing. It's just text files. It does look like this is the case here, where there aren't actually meaningful outcomes from this, but this should be the case everywhere. Isolate these systems from each other. GitHub compromise could happen at any time, even from GitHub themselves.
Do they publish these things on a platform other than Twitter too? Or is their policy that you ought to need a Twitter account to follow their security statements?
Microsoft’s GitHub was compromised when a Microsoft developer using Microsoft VSCode installed a rogue extension from Microsoft’s VSCode extension library, which is moderated and hosted by Microsoft.
Most large companies won’t allow direct access to Docker hub or PyPI, and now they’ll have to restrict access to VSCode extensions. How did the extension get poisoned?
counter intuitively criminal ransomware gangs operate on trust. They have to ensure that we believe they really will shred it, otherwise no victim will ever pay a ransom ever again.
Therefore one way to weaken these criminals would be to weaken this trust factor. In a way therefore comments like "can we actually believe they will really shred it" goes towards this aim.
I have to wonder what criminal hacking gangs that do not operate on trust would do. Would it be like the replacement of organized crime (mafia) with the arguably wider damaging unorganized violent drug gangs?
Join the club! I did as soon as the Microsoft acquisition realizing this would be only a matter of time… with more projects (finally) leaving that ecosystem, I might finally be able to delete my last account with Microsoft.
As some of us stated in the last weeks: Microsoft is working hard to get people to reconsider GitHub. All those small issues keep on adding up. Something is seriously flawed at Microsoft here - those problems did not exist in that way 2 or 3 years ago. It coincides with the rise of AI.
Self hosted gitea for many years with ~25 devs.
Yes, it's essentially a FOSS carbon copy of GitHub. CI/CD is also intercompatible, uses the same syntax and pulls the original GitHub Actions packages.
Now with the Forgejo split, I would prefer Forgejo, as it has way more steam behind it with Codeberg and Blender as the big use-cases.
I'm not OP but probably the licensing drama. Gitea is now open core if I remember correctly. Some details are available here[1]. I also used to run Gitea, but I don't any more. The open-source churn is getting tedious and difficult to keep up with.
I think AI has helped to a degree. I think a lot of people have known about massive gaps in security, but it's been a sort of "why would I?" and a gap that didn't feel worth hopping for attackers.
The gap is smaller now.
I've been talking about package worms for... fuck, a decade. Insane. I've even thought about publishing one to prove a point but, well, it's illegal obviously. And ethically questionable.
Someone just vibecoded up what we've all known was possible for a long, long time. Just like a lot of other vibe coded projects.
I remember talking to a malware author a long time ago and I think this would have been exactly what he would have loved. He liked building custom C2 protocols, tiny malware, etc, but when we discussed a particular idea for owning massive amounts of infrastructure his response was basically "that's a lot of effort to get a krebs article and FBI attention". Now it's not so much effort!
Also coincides with the time I started seeing Juniors installing "recommended extensions" into GitHub-hosted Visual Studio environments.. because there was a popup that helpfully suggested doing so, based on the programming languages used in the checked out repository.
It's more likely that it isn't coincidental at all: software development-oriented LLMs became a lot better towards the end of 2025, and so there's a non-zero chance that people are using them to find new security exploits.
(People are not sleeping on this and it is not something people have failed to notice. I don't use LLMs at all and even I have noticed it - largely because there is approximately nobody that isn't talking about it.)
I think the other side is much more important. With company mandates to use AI as much as possible, there has been a deluge of low-quality PRs. Everybody is feeling tired from reviewing those, and quite possibly numerous security issues have been introduced since.
Ahh, that's a good point, and I actually hadn't thought of that angle! I was thinking of it purely from the point of view of the attackers using LLMs to generate interesting new exploits, with a side helping of letting myself get mildly annoyed, possibly incorrectly, by the writing style.
But yes, it's also possible the defenders have been kind of forced into having the slop machine shit out a huge pile of shit-ass changes, one way or another, that end up making the attackers' job even easier. (Even assuming no mechanisation at their end! Which is of course in nearly-June of 2026, probably unrealistic. And LLMs do appear to be really quite good at that side of the equation...)
The most dangerous is where the new feature works well and is using safe APIs, but integration is quietly broken somewhere. The risk of incoherent state is way higher because you no longer have a small set of people that knows the complete theory of the software and can find discrepancies.
There is a 100% chance that people are using LLMs to find vulnerabilities and build exploits. If it was possible for something to be a 101% chance, that's what it would be.
Apologies to all - I am British. The phrase "non-zero" does cover every case other than zero, but the intent is that it covers some cases more than others. What I'm trying to say is: yes. My intent was just to push back on this specific (and slightly bizarre to me) instance of kind-of-vagueposting, to my eyes written to imply that it might be some sort of unnoticed conspiracy, detectable only by the most enlightened of observers, attuned to the subtle signals that most people miss: that people are using LLMs to find security exploits.
Right, no, what I'm snarkily saying is that basically everybody who has ever looked for a vulnerability before is now using LLMs to do it. It's a huge thing in exploit development right now.
Do you mean because more people are vibe coding, trusting the models' output, and putting code directly into production, so there are more security vulnerabilities created?
Or because there are more source code scanners which end up finding more vulnerabilities?
I heard an engineer at Anthropic was submitting 150 PRs per day. That's one PR every 5 to 10 minutes, so you can guess the level of review and quality control involved.
I think it's more about the popularity than the capability. The chances you might accidentally put a Github access token into an undesired security context goes up dramatically when you actually create and use one on a regular basis. The developers at GH are certainly using these tools just like the rest of us.
Seems like disgruntled tech bros who lost their jobs to AI are now wrecking havoc on tech platforms.
This is going to create so much work and job security for software developers.
Large companies are going to have to adopt all kinds of policies and bureaucratic processes to protect themselves from supply chain attacks. It's going to increase the amount of engineering work, create new blockers, increase the on-boarding time for new tech talent. I suspect that software devs are going to get their jobs back with a thick, cushiony layer of bureaucracy on top.
Software developers are a bit like lawyers. As an aggregate, they have the capacity to create problems which translate directly into billable hours for themselves.
between all the Linux LPEs and Claude's known security flaws, alone, I'd be shocked if Github and Microsoft hadnt gotten hacked by now. reasonable bet we mainly hear it when big shops get bit
Before 2026 I hosted client code on GitHub, now it feels suboptimal, code is both an intellectual property asset and security risk. Especially if the company is software based, self-hosting your code just has a much better risk profile for almost no cost.
It's also one of those things that warms your team up and gets them ready for actual work, a team that has to self host their git and other infra, like self-hosting DNS servers with bind, will have a much better work ethic than engineers who click buttons on a SaaS and conflate their role as users of a system instead of admins of one.
Additionally, using github actions, and relying on Pull Requests (Tm) (R) (C) has always been (useful) vendor lock in (and a security risk in case of GH Actions). It wasn't enough to lock down a choice, but it tilts the balance in favour of less dependencies, which with the increase of CVEs and supply chain vulns, seems to be the name of the game for this new era. Build it in house, ignore the dogma.
Read-only access to all non-sensitive code is how things should be. Huge engineering culture and productivity booster. It’s also very useful to keep each other honest (I’ve found so many “interesting” things hidden away in organizations with tight read access restrictions).
Devs not having read access to all code seems like a massive org smell. What’s worse, in many cases not having access doesn’t just prevent you from seeing it it also prevents you from knowing it exists. Now you don’t know what to ask for, who to ask, or what to not implement again.
There is no security risk that you could use to convince me
that ”devs should only have access to code they need to modify”.
I think this might be more aimed at ensuring that if an attacker gains access to cloud login credentials via a compromised dev machine, those credentials can't then be used to access customer data.
Not saying it’s good but I think it’s quite common for devs to have read only access to everything. I suspect that with all the recent news, including this, the needle might start to shift a bit.
I think it’s actually non-trivial to determine how many repos you should have read-only access to. I frequently hop through multiple repos that I don’t contribute to, just to understand how the system is architected and what it does at different stages. We even have an internal Claude skill for finding relevant repo for a given problem which relies on personal gh access (via CLI). It _can_ be done more securely but those defaults built over many years will take time to change.
Shoot dude, the engineering organization I mentor/teach at a high school has ~75 internal repos.
Robot source code; satellite ground station hardware; satellite ground station software; visualization; satellite hardware; satellite software; nuttx + its submodules for 2 different projects; linux kernel fork; circuitpython fork; raspberry pico tools fork; embedded programming/debugging tools; my lecture notes; my automated grading tooling; etc etc etc. That's just me + ~35 students in classes.
Pretty easy to see how when you have scale you can get to a few thousand.
It's normal that a dev has *access* to all the code.
But did he clone all the repos into his machine? I doubt it. So, the hacker extracted all the 3800 repos using the employee's machine as a gateway? I doubt it as well, I'm sure they would have detected this huge amount of data much earlier than transferring all of it?
> The real question is why github has 3800 internal repos.
They’ve been developing git and GitHub for over a decade. It really isn’t surprising they have made thousands of internally available repos. They probably have hundreds just for running automated tests alone.
Depends how it's set up. Many companies add an IP address check so if you don't come via their VPN (or are not in the office) the connection will be rejected before any auth is asked.
So you'd need to authenticate for the VPN, which often has 2nd factor.
Security is often overlooked internally and seen as source of friction.
I worked at a popular US social media firm and it wasn't hard to get a permission that allows me to delete the entire company's dataset. Often arguments around "I'm working on org-level initiative and I need to get permission to get it done" would easily get me the permission.
I can think of _one_ product that allows you to set up low-friction access management, and AFAIK most users of that product don't set it up that way.
Software engineers _should_ be able to request access to dev resources JIT during their day-to-day work, have that access auto-approve in >99% of cases, have it auto-expire if they don't actually use the resources, and have all of that be subject to anomaly detection/approval escalations and other auditing.
Instead in most orgs it's like fill out a form, get your manager (who's always in meetings) to approve and then wait some number of days for a human to click-ops your request. At best you can open a PR and have the changes applied in an hour or two.
You _should_ be able to get access to things pretty much immediately if you need them and they're not sensitive. Then we could deny by default without cratering productivity.
It’s the big advantage that small companies have over big ones.
I’ve ridden startups through the phase where they transition to “responsible adults”, and start putting in policies and locking things down and generally behaving like the giant corporations they expect to be one day (and that the locker downers came from and are used to).
You can feel the deceleration, like taking your foot off the gas on the freeway. I’ve sat through all hands meetings where the ceo asked why we don’t ship as fast anymore, and since by that time most of the fast moving folk have moved on, nobody has an explanation.
Security is often an excuse to block other teams to do legitimate work and so often it's fairly braindead. Security IMO needs to get it's act together, passkeys is a great example of security gone wrong from a UX design perspective because you can't hold them to the same standards as product or infra teams, they have the special privilege of breaking things and it increasing their metrics.
Tell them to make a better UX and they lose their minds in a huffy puff of fake crisis mode or get avoidant with stonewalling 'secret security stuff' that you can't hold them to account for. Or eat 50% of developer machine performance for "endpoint security" and the carnival of sadness goes on and on.
Signal is an example of security as a product that was actually designed for user UX in mind to give one example.
Sounds like a great way to have outages because you can’t tell what legacy features are still in use or not. Or even worse, not being able to ever refactor or clean up because you have no means to discover your dependencies.
If you want to move fast, you need access. Unfortunately and obviously this allows threat actors to move fast, too. The tradeoff had a different risk profile a year ago, heck a couple weeks ago.
I'm not sure if this is related or not. But a few days ago, I saw commits from the "future tense" in some repositories. When you read "committed tomorrow" after a commit, it's not funny at all. I posted a screenshot in the announcement on GitHub.
That's probably unrelated. The date of a commit in git can be modified to whatever you want. I once backdated commits because my timezone was off, and I wanted the timestamps to match the ticketing system. Github displays the date stored in the commit, since there is not really a way to verify it.
I think the commit timestamp is just passed through from timestamps in the git repo, not the time at which the commits were pushed to the server. You can probably set your system time to the future, make some commits and push them.