rHXN - Just the rumour of a bug is enough to find an exploit these days

nickcw - 16 hours ago

This describes my life as an open source maintainer at the moment!

In the first 10 years of the rclone project we received about 20 security disclosures through GitHub. We had to deal with over 40 in the last month! That has taken a huge amount of my time, even using AI tools to triage and come up with fixes for review.

The hit rate for those security disclosures is pretty good - about 75% of them have a nugget of something which needs looking at. The configurations for rclone have got increasingly unlikely so I'm hoping they will dry up eventually.

I was considering just merging the fixes straight to master just to make my life easier rather than holding a dozen independent security fixes on branches and merging them at the point release and hoping not to have too many conflicts to fix up. I've decided to stick with the process for the moment.

GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal.

Not sure what the solution is, but it is definitely a problem for us.

dannyw - 15 hours ago

Thank you for making and maintaining rclone. It is truly a blessing.

jiggawatts - 10 hours ago

I came across rclone at work because "Copy or move data to Azure Storage by using AzCopy v10" is a lie, it literally can't `move` files, only `copy` them.

I can't express in polite words how pathetic it is to see the only official blob storage bulk transfer CLI tool from a multi-trillion-dollar company fail to do the simplest, most essential functionality after ten major revisions.

Meanwhile, rclone Just Works(tm).

Thank you from me too!

stavros - 9 hours ago

Do not try and move the files; that's impossible. Instead, only try to realize the truth... Delete them after you copy.

TomatoCo - 8 hours ago

Right? If you move between file system or device I would expect "move" to be impossible. At best you could copy a logical block and then delete a logical block. If you share a filesystem then perhaps a logical block is a few hundred kilobytes. If you don't then I'd expect the smallest logical block to be the file itself.

inigyou - 3 hours ago

Moving on the same filesystem moves the pointer to the file.

rq1 - 9 hours ago

Bend the spoon*

* for people who don’t know the ref

dingdongditchme - 5 hours ago

The star only confuses me more...?

thedonncha - 1 hour ago

Same with some of the projects I maintain. The reports came in for one project over a few months and then once those dried up, and they did become increasingly unlikely or appear to be edge cases, the reports started to accumulate for another project. Great to get these issues addressed but it's exhausting.

x______________ - 13 hours ago

> Not sure what the solution is

Possibly some sort of ai agent code review system that churns through code looking for these vulns before the code is published. It feels like it's all about who has the resources to find bugs at the moment but that it should be a standard to catch issues before prod moving forward..

pixl97 - 13 hours ago

Yea, AI bug finding over entire projects, at least on SOTA models is super expensive, hence those with the resources setup to find the most bugs in an automated way.

That said there are a number of people and companies working on more focused means of driving the LLM to look were bugs would be the most dangerous and in doing so reduce the token spend of each bug found.

In some ways the better you are at security stuff the more you can reduce your spend by better driving the LLM to problem spots.

ftchd - 13 hours ago

> I was considering just merging the fixes straight to master just to make my life easier

I felt that. The problem with doing that is you hate yourself afterwards so not a good solution either, gotta do it properly.

Thanks for working on rclone, Nick!

caminanteblanco - 14 hours ago

Thank you so much for maintaining rclone! That project is literally the only thing that makes Google Drive an acceptable experience on Linux.

Kubuxu - 16 hours ago

As long as you are not running a paid bounty program. Otherwise now you are getting 40 per day.

yjftsjthsd-h - 14 hours ago

> GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal.

A strange bottleneck; anyone know why that would be so slow?

bobmcnamara - 7 hours ago

Trying to keep the rest of GH online?

htrp - 14 hours ago

human triage in review

dingdongditchme - 5 hours ago

Thank you for the insight! Are you going back to the changelog and editing the CVE-PENDING? I think this would be a justifiable edit of the release history that I'm not sure is possible on github.

nickcw - 40 minutes ago

Yes I update the CVE-PENDING once the CVEs are in to keep the record straight.

Groxx - 11 hours ago

You can add CVE info later in a git note, if you want something git-managed to point to it. Might take some habit-tweaking to include notes in log output though, to see them later.

cyanydeez - 14 hours ago

I feel like someone somewhere is working on an opensource GPU compute resource pool you guys could pull from as part of a donation type framework. I got a bunch of local gpu resources just relaxing, and if I could load up a a binary to provide compute for X hours a day overnight or whatever, that'd be cool.

nickcw - 13 hours ago

I got a 6 month free subscription of Claude Code Max under Anthropic's open source program which has been very helpful.

gazarsgo - 13 hours ago

http://github.com/buzz/block is quickly becoming a reasonable approach to shared agent compute, though it's still a bit more hands-on than the older 'compute donation' projects like SETI or Folding@Home.

cpach - 10 hours ago

404?

majorchord - 7 hours ago

https://github.com/block/buzz

zmgsabst - 16 hours ago

I can’t comment on if it applies to your workflow, but one process I’ve used is to aggregate and land ~10 security patches at a time. Eg,

- grab a group of (related) bugs/defects/vulns

- fix them on a branch like bug-batch-XXX

- run that group through the verification, landing in main, CI/CD flow to amortize process cost

- repeat as needed to process backlog

My experience is that process often has irreducible time (eg, two days due to reviews by various parties); but that time slot can be shared between several bugs in a single PR — especially if you have several related to the same feature.

dataviz1000 - 15 hours ago

> even using AI tools to triage

Can you discuss this? I might be able to help.

darig - 9 hours ago

[dead]

hypfer - 7 minutes ago

> The big engineering shops (like Google) have been building microupdates directly into their software to ensure that fixes directly reach users as a priority over (e.g.) being fixed in the Chrome code repository. We don't really have that kind of luxury in Docker or OCaml, as we don't control the endpoints our software is used in. Aside from Docker Desktop 3, downstream distributions quite rightly repackage OSS on their own timescales and terms.

Yes, no. Do not even think about doing that. You cannot, should not, and must not "microupdate" code running on users machines without their consent.

Also, the whole idea is completely insane. "We might have unpatched 0days in the field, so to mitigate that, we shall add arbitrary and remote code execution via the cloud"

___

And even if you were to think that taking a page out of googles book - you know, the company known for being laser focused on having the wrongest opinions - is a good idea, you shouldn't do so for free.

Live patching is something enterprises pay a lot of money for. Making it a Foss expectation is insane.

godelski - 16 hours ago

It's easier to find bugs, fix them, yet there's less will than ever. My bosses just want speed and will give me a 30 minute lecture on why I don't need to solve a bug that Claude solved in 5 minutes, I've verified, and it's already in an open PR. All the while we're pushing out bugs faster and faster.

No matter how good AI gets at fixing bugs we'll never fix them when there's no will to fix things. Software will never be good if there's no will to make good software. The problem has always been about will. To many better products. It's insane that in a time where we can do better on speed and quality we still choose speed and tell ourselves it's velocity

ozim - 9 hours ago

It is not "will" it always is money.

There was an article some days ago where someone was claiming "with AI only you decide how many bugs you have" — well no the same forces apply because tokens are not free and business wants to do stuff that directly earns money.

Well we have to make cases and measure where the bug costs money and how. Lots of bugs are irrelevant and not blocking people from using the software. If bug doesn't drop database but is "dropdown doesn't exactly align" or "given precodnitions A,B and C something bad will happen" while A, B and C have very small possibility of occuring.

godelski - 5 hours ago

  > It is not "will" it always is money.

Bullshit. There are so many ways to make money. And let's be honest, are the levels of wealth these people have money is entirely meaningless. There is nothing Elon can obtain, through money, that Alex Karp can't. That is despite more than an order of magnitude difference in wealth.

So it isn't money. You can argue that it is power, that the money is the proxy, but this still wouldn't answer the question.

The reason I'm pushing back hard here is these simplistic answers are just thought terminating cliches. They dismiss the problem, calling it inevitable and unsolvable. It only helps to preserve the status quo. It only helps empower those who seek to take our own. So I call bullshit

kulahan - 10 hours ago

I spent about a decade coding, then half a decade in QA. QA has always been a thorn in everyone's side. The amount of money some companies spend on these departments just to ignore them is wild.

Now we've got an ability to push code out faster than ever, and absolutely zero innovation for QA. You simply cannot trust AI to verify your code is working. You can't have Quality Assurance without some form of assurance. So you're either hiring twice as many QA guys for the 10x output, or you're mostly ignoring the idea.

Economics has bubbles. Does computer science? Anyways, screw this. I'm moving to nursing.

icedchai - 9 hours ago

Unfortunately, in many smaller companies, QA is no longer even a separate entity. It is foisted upon developers (and sometimes PMs.) It went the way of agile, similar to sysadmins and DBAs.

elmer2 - 13 hours ago

Something that might come out of all of this is that companies that previously slow rolled security fixes will now be forced to fix them quickly, because the speed of AI and the liability of being insecure.

pixl97 - 13 hours ago

Yea, when their cyber insurance policies start biting them because AI found a bug months ago and it was never fixed I figure we'll start seeing a change.

With that said there are numerous companies that are very concerned about the situation. They know AI is finding bugs in their software at an accelerated rate, one they are having difficult times keeping up with because they want human understanding and review of the fixes to avoid introducing new bugs.

godelski - 11 hours ago

Did CrowdStrike actually suffer? That was a huge outage and their stock is today at an all time high. Seems like no one actually cares. Neither the companies providing the software, the companies purchasing the software, nor the markets.

To me, this is the craziest part about all of it. Why doesn't anyone seem to care?

SoftTalker - 5 hours ago

Because the purpose of Crowdstrike is not to prevent exploits or make computing safer, its purpose is to allow corporate CTOs to tick a checkbox.

lbreakjai - 11 hours ago

Why would they suffer? They showcased that they were big enough to disrupt the global economy. Why _wouldn't_ you invest in something so big and powerful?

This is the same as when junkies specifically seek batches of drugs on which others overdosed.

godelski - 10 hours ago

I know you're being sarcastic but what worries me is it seems people really do think that way. AI didn't create the addiction, but it's like switching from opium to fentanyl

Lvl999Noob - 4 hours ago

USA has that digital misuse law that makes it a crime to use software in unintended ways. So claude or codex finding bugs in their software is actually illegal... I am not sure if really no one tried this yet or if they did and I just missed the news.

illiac786 - 13 hours ago

Depends on the jurisdiction. Where I live, companies are not liable for insecure software, if sold to a private person for example. Enter huge botnets of crappy home routers. Hopefully this is going to change.

BrenBarn - 10 hours ago

The better outcome would be for everyone to slow roll everything rather than speeding up the rate of bug propagation.

9cb14c1ec0 - 10 hours ago

I guess I often don't know how good I have it as the owner of my own tech company. We've been busy fixing bugs and tackling tech debt, and I couldn't be happier.

ryandrake - 15 hours ago

Most places I've worked have been infuriatingly uninterested in fixing bugs, and would release software with major known bugs and a vague plan to fix them later. Of course, when "later" came, there were more features to cram. No time to fix those bugs.

I'm hoping one of the unintended side effect of it being essentially free to find and exploit (and fix) software bugs is that companies become less cavalier about shipping bugs in their software. Unlike most of the industry I don't believe "bugs are inevitable." Bugs are a choice developers make when they're rushing and careless and when all of their incentives are to ship quickly. You can ship bug-free software but it takes (or used to take) a really long time and a lot of care, care that commercial software developers just don't ever seem to muster.

Maybe when their software is getting 0wned over and over and 30 security issues are published a day, they'll start caring and taking their time.

denkmoon - 10 hours ago

Bugs are a choice by managers, not developers.

godelski - 10 hours ago

I disagree, a bit. There should always be some contention between manager and developer. It's healthy. But I think many developers just remain silent and are afraid to speak up. They are afraid of losing their jobs, or worse, getting kicked out of the country. It's reasonable, but it is one of many factors that contribute to the negative feedback loop of myopic management.

We've all seen developers who game the system. They hide the bugs just enough so checked out management doesn't see them. They convince themselves that those bugs don't matter, even when they trip over them later. They get rewarded because they appear to move fast, eventually become management, and the whole thing gets worse as time goes on.

It's a structural problem. Yes, the level of influence is higher the higher up in the org chart you go, but there is still "power" at every level. Even the most junior developer has power. The worst thing we can do is become apathetic, shrugging it off, saying "well what can we do?" That attitude is one of many factors that got us to this point. Importantly, it is a factor we actually can influence.

That's why I object to it. Not because I think it is going to solve the problem overnight, but because it is a thing we have some power over. And it is a very different situation when one engineer in a team is vocalizing "our software has issues" while most of the team silently agrees vs several members of the team simply vocalizing agreement. There's no magic single variable fix to problems like these, but we got here because a bunch of little problems added up. Unfortunately, or fortunately, the way to solve it is through solving a bunch of little problems. Each seems insignificant in isolation, but they accumulate

denkmoon - 1 minute ago

[delayed]

brabel - 1 hour ago

You probably never had developers working for you. First, if you don’t give them a deadline they will work on it for months trying to get it perfect, but it will still be buggy and do the wrong thing unless you keep very close attention. Second, because you must give deadlines due to the previous point, the result will be buggier still, as they will spend most of their times on what to name variables and how to structure the packages and modules. Their testing will cover their mocks and when put together in the real thing most functionality will just not work at all. I’ve seen this happen even with senior developers. With juniors it’s guaranteed to happen without a very good tutor. Take some fucking responsibility. If you deliver software, make sure it’s in a working state. Don’t blame your manager for everything, they did not force you to write shitty code, if you think they can do that you are not ready to work as a professional developer, you are just a code monkey.

denkmoon - 7 minutes ago

>as they will spend most of their times on what to name variables and how to structure the packages and modules.

Incredible strawman of dev behaviour, I can only assume you are a manager yourself. I assure you, that's not what we're doing.

dingdongditchme - 5 hours ago

My take is that when the bubble bursts good software will remain. Right now all the big players are hiding the cost. Anthropic (in my mind) is especially egregious at tokenmaxxing without telling you whats going on. A simple request to edit a text file launches multiple agents which takes ages, and burns tokens. All the while it calles it "Sautéeing" or whatever random verb they spew out as you wait for your answer.

NateEag - 3 hours ago

> All the while it calles it "Sautéeing" or whatever random verb they spew out as you wait for your answer.

Today, while bitterly staring at Claude per my current employer's "use genAI or else" mandate, I got "bloviating."

yieldcrv - 16 hours ago

My boss is a big yapper too

Very low signal information, preemptively trying to cover every rebuttal despite nobody ever planning on making one, in the few times someone does he plays devils advocate endlessly

Like bro just let us babysit these agents, everything’s going to happen

vasco - 15 hours ago

If your boss asks for X feature and you split it into 4 PRs, 3 that implement X and a 4th that fixes a security bug, how would they know? Someone that uses those 30mins like that isn't reviewing your PRs themselves. Just fix it and be done with it. You only needed to argue with the boss when you needed to make time for it which would delay something else. If it just appears done they'll just react with clap emoji later when you announced you also fixed this extra thing.

supriyo-biswas - 15 hours ago

You haven't worked at conservative software shops I guess. The last place I worked at had this huge song and dance with you trying to raise a PR that looked irrelevant to the task at hand would lead to meeting with your lead dev, your manager and your skip level as you tried to explain why the bug needs fixing...

The worst one was where I fixed a datetime bug and although it had been sending out false alerts, I was asked to dry run the 5 lines of code I changed, like a coding interview. In all this pressure I forgot what the code was meant to do, and was dismissed and asked to set up another meeting with an explanation of all the various cases that could happen...

talon8635 - 11 hours ago

But, isn’t it our job to impress upon the managers the importance, in a certain regard?

Alternatively, what about just doing the right thing? Either you convince them to take this stuff seriously or you find alternate employment. How can you subject yourself to the moral degradation and conflict of principles? I could understand for someone with no other financial options, or in some sort of oppressive culture/economy. But most in our field probably don’t fall into those

godelski - 9 hours ago

  > But, isn’t it our job to impress upon the managers the importance, in a certain regard?

I think it is. But this appears to be an unpopular opinion and I'm not sure why. The question I'm still unsure about is if managers realize they are surrounding themselves with "yes men". The other question I'm still unsure about is if people realizing that not saying "no" (or "yes, but") is not meaningfully different from being a "yes man".

Our job is to engineer. Our job is to build (good) products. Information can't just flow top down, it has to go the other way too.

What seems weird to me is that during "the good times" in our field, that happened more frequently. A strong employee market (as opposed to an /employeer/ market) seemed to be good for employee, employeer, and the people buying everything. But myopia is quick to set in.

  > Alternatively, what about just doing the right thing?

That's the main motivation of why I speak up. There are consequences to our actions. Our choices may have small or little influence, but unfortunately the problem is that the world is complicated. The problems we face are mainly composed of many small issues that add up. I am surprised this is more contentious in places like HN as we deal with this every day. The way we solve big problems is we break them down into many different small and manageable problems. The only difference here is that we're viewing things bottom up rather than explicitly breaking them down. Though that is harder to figure out which small problems add up to the big problem. But we deal with this type of problem solving in programming all the time too.

If you haven't heard it before, allow me to introduce you to Pournelle's Iron Law of Bureaucracy[0]. I think one of the important things it states is that the second group is actually bad for business. I think there's a common misconception. People are often saying "well it is good for business", pointing to all kinds of messed up shit. I don't buy that. It is myopically good for business, but not in any meaningful length of time. Though then again, as Buffet says "The market can stay irrational longer than you can stay solvent." Michael Burry famous learned this first hand.

[0] https://www.jerrypournelle.com/reports/jerryp/iron.html

  In any bureaucratic organization there will be two kinds of people:
    -  First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration.
    - Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc.
  The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization.

dingdongditchme - 5 hours ago

Michael Burry managed just fine, I believe. Secondly, I think the "in every case" is too strongly worded. I am working in an org right now where it has not happened yet. Perhaps it is only a matter of time, but let me have my optimism.

godelski - 4 hours ago

  > Michael Burry managed just fine

True, but selection bias makes it hard to famously learn this lesson and not come out on top. Dropping a name no one knows that was right but was never vindicated would serve no purpose

godelski - 13 hours ago

Let's spend $5m to have a meeting about why we can't spend $50.

Penny wise, pound foolish

The best is the explanations are in the PRs. They even have Claude these days to answer their questions... I swear, these people are more addicted to spending money than they are about making money.

tough - 14 hours ago

holy shit that sounds like a nightmar-ish work environment

godelski - 13 hours ago

That's mostly how I'm shifting to doing things but let's admit, this is metric hacking. And let's also admit, it takes longer to go through that song and dance than it actually takes to do the work.

johnbarron - 15 hours ago

You will see, they will change both laws and expectations, to say its normal for software to always have terrible bugs. You can always solve a problem by lowering your expectations. :-)

godelski - 13 hours ago

No laws are needed. We already have oligopolies. Don't like the buggy software? Go to the only other competitor who also has buggy software.

FFS, we're living in a world where Linux has more than doubled in popularity mainly due to Microsoft actively fleecing their customers. It's crazy

/I use arch btw (and have been on it for over a decade)

flanked-evergl - 15 hours ago

What law says software may not have bugs?!

odo1242 - 15 hours ago

Not explicitly laws, but like contracts, regulatory requirements, SLAs all indirectly enforce that

swiftcoder - 13 hours ago

Well, they at least say that one is obligated to address bugs in a timely manner (where "not widely exploitable; won't fix" is a perfectly valid resolution).

novia - 14 hours ago

!!! You described my experience in the workforce to a t! So frustrating that tech managers look at tech debt as a thing to be maintained at a certain level instead of allowing us to achieve perfection

godelski - 13 hours ago

  > tech managers look at tech debt as a thing to be maintained at a certain level

What they don't seem to understand is all debts have interest. In my current project our entire team is tripping over that interest created by one person. Though my manager is frustrated with me because I'm "spending too much time trying to understand". Btw, that is a few hours here and there, maybe a day for a rabbit hole which results in me creating a dozen or more issues. How is that "slow"?

  > allowing us to achieve perfection

Impossible

Look, I'll defend high quality code all day long. Good code allows you to move fast just like taking 30s to tie your shoe laces allows you to run faster.

But we can't write high quality if we pretend that perfection exists. Globally optional solutions are the exception, not the norm. There's almost always tradeoffs we need to make. The high quality code, the high quality engineering, is understating, tracking, triaging, and minimizing those tradeoffs. It is writing code that allows you to change those decisions as quick and effectively as possible. But perfection doesn't exist. It's a good thing to chase, like Utopia, but ultimately unobtainable. A good engineer knows how to triage.

Advocating for perfect code will be a losing battle. But we should not, even for a second, let that be interpreted as meaning quality doesn't matter. I'm still frustrated with people who say "don't let perfection be the enemy of good enough" as most of the people that say that believe "good enough" is "it looks like it is working" and call a "demo" a MVP

evilduck - 13 hours ago

Your tech manager is likely very aware of the imperfections and is making prioritizations and compromises to keep you employed.

novia - 1 hour ago

Nah, I quit at the end of 2023, so much for those compromises

I have pride in my work, and having to watch it be destroyed repeatedly is not a way to keep me around.

bonsai_bar - 13 hours ago

That's exactly what managers should be doing.

VladVladikoff - 13 hours ago

Perfection is the enemy of progress. There will always be a balancing act.

lazyasciiart - 10 hours ago

Well, that goes with the metaphor. Debt is not an intrinsically bad thing, and most people agree on that: countries run deficits, people get mortgages and millionaires trade on leverage.

tokioyoyo - 14 hours ago

> instead of allowing us to achieve perfection

Huh?

bri3d - 16 hours ago

I don't think this is new with LLMs (finding an exploit based on a few words offhand has always been a fun part of exploit development), but it's scaled and democratized to mass exploitation of low value targets. Backing exploit PoCs out of patches, commit messages, and random overheard or over-read sentences is a practice as old as vulnerability research. The difference with LLMs is that an explosion in actors "skilled enough" (human or not) has enabled sloppy / low-skill "exploit the whole Internet" actors in a way they weren't previously enabled.

I do agree with the author's ideas, though; most of these are things that should have been done much sooner, and I suppose it's good in a sense that there is a forcing factor now.

happyopossum - 15 hours ago

> Backing exploit PoCs out of patches, commit messages, and random overheard or over-read sentences is a practice as old as vulnerability research

True, but it used to take days or weeks of research, testing, and RE to get those PoCs.

Today the entire chain - reading commits, RE patch binaries, building exploit, scripting exploit scan, $profit - can be fully automated and happen in minutes or hours.

stephbook - 14 hours ago

I think roll-out and deployment are even bigger issues. Who updates their software stack within 10 minutes? Most CI runs take longer to verify the business logic is still working.

Add to that the danger of supply-chain attacks where you don't even want automatic updates.

talon8635 - 11 hours ago

Your last point is the conundrum I keep running up against

It’s a trap regardless:

A) run a known vuln B) accept and run any and all updates immediately… which could be compromised

Maybe A is worse because it’s a known vuln?

Leherenn - 41 minutes ago

AI auditing of dependency updates will mostly likely come soon (if it's not already there), and should mitigate the most obvious cases at least.

(By this I mean integrated in something like dependabot, not just some security companies doing it and publishing reports.)

xboxnolifes - 11 hours ago

I prefer the devil I know over the one I dont. At least I can make judgement calls with vulnerabilities im aware of. Automatic updates have unbounded risk.

rndhouse - 11 hours ago

I built a tool that monitors commits and tries to detect silent bug fixes. With GPT-5.5-class models, it can identify fixes hidden within otherwise routine commits quite reliably. Obfuscating the code changes enough to avoid detection is difficult.

I have heard of at least one project (c-lightning?) temporarily releasing a closed-source binary as a workaround until users could update safely.

9cb14c1ec0 - 10 hours ago

Have you heard of these things called LLMs are that are quite good at reversing changes in closed source binaries?

teravor - 9 hours ago

you can diff binaries in a disassembler, the technique has been used for a long time to discover patched Microsoft vulnerabilities, create exploits and then race the world.

now you can instruct an LLM to do this with MCP...

rndhouse - 8 hours ago

There are many techniques for making a closed-source binary harder to reverse engineer: polymorphism, symbol stripping, code encryption, packing, and so on. Closed-source binaries at least stand a chance of being effectively obfuscated, unlike open-source code.

teravor - 8 hours ago

and those measures have always been merely annoyances. except for the VM methods which impact performance significantly.

now none of those methods matter as you can just instruct an LLM to bang its head against the wall until the wall breaks.

rndhouse - 7 hours ago

They still matter in the sense that they impose a cost. In other words, they buy time, which is all the users need before updating.

I'm not sure if the cat and mouse game resolves clearly one way or another. Since the obfuscation can be hardened against LLMs during development.

lazyasciiart - 10 hours ago

People are trying to secretly fix bugs?

dspillett - 9 hours ago

Yes. The problem with openly releasing a fix for a security bug is that people are not always quick to update their instances. It only takes one black-hat monitoring your releases by some means to dissect the changes, find the hole, and make an exploit that can be immediately used against unpatched systems, and the sudden availability of better automated tools for that makes it a much bigger problem than it was a year or more ago.

Information about the problem then gets released later, once everyone has had appropriate time to get up-to-date. Or not, and we are non the wiser.

stavros - 9 hours ago

Yes, they don't want the vulnerabilities exposed so they ship the fix in secret, contained in unrelated features.

lazyasciiart - 9 hours ago

Ah, I guess that would advertise that users of the old version are vulnerable. I was imagining some altruistic contributor trying to sneak in a bug fix past the maintainer.

stavros - 9 hours ago

Yep, this is for maintainers releasing fixes to secretly-reported 0days without tipping off attackers who are looking to exploit a (sometimes very large) install base.

ChrisMarshallNY - 12 hours ago

Sadly, it seems that the lesson from this, is to keep your repos private.

I am not a fan of that, but I think many folks will take that away from this.

jameshart - 14 hours ago

I do wonder what the hit rate is in general for Claude finding a successful exploit when prompted with a rumor that leads it to assume the bug is there.

“I’m told there’s a path traversal exploit in this package. Can you find it?” - probably a reasonably high chance of it finding one, even if you just made that rumor up.

a2ff6eeb0 - 14 hours ago

That's basically the approach Anthropic takes.

saghm - 15 hours ago

Sounds like the next step is to start rumors about vulnerabilities so that someone else will find them and you have plausible deniability. "Hey, did you hear? Someone told me <competitor company> might be leaking their logs from a publicly accessible endpoint..."

cyanydeez - 14 hours ago

[flagged]

janpeuker - 8 hours ago

I do get the anxiety about new bugs but to be honest I fear more that in a few years it's so cheap to fix medium to high security bugs that lowkey hacking and finding worrying privacy violations becomes prohibitively expensive for citizens.

29349175 - 11 hours ago

"This particular report arrived privately on a Slack channel via Jane Street last week, and was itself found via Claude Fable. That compresses all timelines considerably..."

So Fable knew about it. Maybe someone is running experiments again like in OpenAI's Huggingface hack.

Cute to see that the Glasswing apparatchiks still protect their income stream and hand out no accesse.

xbar - 14 hours ago

Hm. I wonder if you can force an exploit into existence with a false rumor of a bug?

jamilton - 11 hours ago

One way you could do it would be to find something that 1. would sound like a bug to a human and an LLM, 2. would be "confirmed" as a bug by a LLM, and 3. would be consistently solved in the same exploitable but reasonable way by an LLM. That's assuming a codebase that's largely AI-written with human review that you're able to open an issue for (possibly indirectly).

Another way is maybe something like saying there's a bug at some endpoint and thus manipulating a bunch of bots to DDOS that endpoint without having to pay for it?

metrofun - 8 hours ago

proves you only need conviction to break smth

Tomte - 15 hours ago

So we have automated Thomas Pornin?

westoque - 12 hours ago

with agent loops, the cost of exposing bugs should be cheap now. every pen tester should be using agent loops and on the flip side, an agent could easily see this issue once filed and a fix immediately done. it's incredible how cheap code is now.

zingababba - 15 hours ago

Transferring ideas in their abstract form is entirely possible now. What this blog post describes I believe even undersells the current reality. I've had success with finding crits by pasting in extremely high level or abstract concepts. I recently went from a talk abstract (3-4 sentences) to crits lol (finding that vuln type).

petesergeant - 15 hours ago

Even that sounds too involved, just tell the agent that one exists, even if it doesn’t…

https://sgnt.ai/p/terrible-mistake/

logical_people - 1 hour ago

[flagged]

vladigtr - 15 hours ago

[flagged]

throwaway613746 - 14 hours ago

[dead]

zb3 - 16 hours ago

[flagged]

skybrian - 16 hours ago

They were not happy about it and loudly warned everyone it was coming, but instead of listening a lot of people said LoL MaRkEtInG.

ozozozd - 14 hours ago

Well, maybe we would say different things if they shared the bounties of being the chosen one.

Even dictators have to share.

loeg - 15 hours ago

No mention of memory safe languages? Sure, it does not help existing projects, and sure, of course you can still have logic bugs (or memory bugs using escape hatches like unsafe). But they do help significantly in reducing the number of exploitable bugs.

avsm - 15 hours ago

OCaml (the language I use in the post) is an extremely memory safe language, but there are plenty of higher level logic issues that spring up.

LLMs are extremely good at finding corner-case vulnerabilities in C bindings even within a memory safe language; see for example the fixes in an OCaml crypto library here: https://discuss.ocaml.org/t/the-series-of-mirage-crypto-rele...

aseipp - 14 hours ago

Yeah, but there's already plenty written on that topic elsewhere; the OP is making a distinctly different point and everything else in it still applies, I think. Even if you are using a memory safe language -- if you are developing security sensitive software, you need to understand that exploits can now be researched, developed, and deployed in hours (or for stupid bugs, minutes!) instead of days or weeks or months. And this process can now just be massively scaled up with money and compute. It was not like this even 24 months ago.

loeg - 10 hours ago

Everything's been discussed to death, including the topics OP chose to focus on. Whatever.