r/opensource • u/mynamesleon • 1d ago
Discussion Open Source, AI, and Trust
AI usage is everywhere in dev, and it's not going away. But AI makes trusting open source... complicated... (arguably in a good way though).
A big part of open source projects (particularly solo projects) was that they were typically someone's passion project. They cared about what they made, and that extended to the code too. with attention to detail to make it easily maintainable (i.e. legible to other humans), performant, and free of anything malicious. A big part of the trust associated with open source is trust in the maintainers. But when those maintainers aren't actually writing the code themselves, that trust is shaken.
I have no specific objection to open source maintainers using AI though, as long as they know what they're doing. AI is a tool after all. But it's a tool that should be used responsibly, and with appropriate oversight. In the same way as I wouldn't trust a random 12-year-old to responsibly use a chainsaw, my trust in open source projects disappears when I see that it's been written by a non-dev relying entirely on AI.
I've been a dev for over a decade (always in commercial settings, so my contribution to open source has always been financial, rather than through contributing code). I'm fortunate that my team's attitude is that even if AI wrote the code, it's still that person's code, so they better be damn sure that it's written well, and performs well. Code quality is extremely important to us, and we know that even with the most detailed prompts and surrounding context (instructions, skills, etc.), that AI sometimes spits out absolute crap. So if AI writes our code, then the time we would have spent writing it is expected to be spent on planning, reviewing, and confirming its output.
The situation is muddied by more and more people using AI to churn out code without really understanding what that code is doing. Those sorts of projects can be genuinely dangerous too. Especially in a world of automatic updates, I'm not going to trust an app from a maintainer that won't understand when an LLM has just put ads and trackers (or more malicious things, like external executables) into their project. I'm still perfectly happy to use and support projects that were made with AI help, but I have less (or usually no) trust in them by default, and need to spend the time to review what I can of the code myself first.
I guess what I'm trying to say is: for all the AI slop out there, there are also a lot of devs out there who know what they're doing. Devs who still care about what they work on, and who may be (and probably are) using AI tools to help them do it, whether that's personal choice, or their company expecting/requiring them to. AI has really highlighted the importance of human critical thinking, which is a good thing. Please don't immediately disregard a project because AI was used. Instead, approach it critically, and sceptically, as you always should have been doing. Continue to call out the slop, but also praise what's been well-reviewed and made with care and attention (regardless of the tooling that was used in its creation).
2
u/QstnMrkShpdBrn 1d ago
The inherent nature of trust in open source is that the source is open and available.
2
u/mynamesleon 1d ago
Yes and no. The inherent nature of trust in open source has always extended beyond simply the fact that it's open and available. And that's my point. You used to be able to trust that it was someone's passion project. The fact that they put the effort in, and cared (either about the actual code output, or just cared enough to figure out how to do it) created a sort of implicit trust contract through proof-of-work.
That's the part that has been eroded by AI. The code is still open and available, so devs like me can still review and verify it. But even after verifying it in its current state, AI erodes the ability to trust automatic updates too, because the AI tools they've used to build it are most likely privately operated. And we've already seen cases where AI will inject product suggestions into things.
That's why I'm highlighting this. There's a huge spectrum of AI use in software: from people with no programming experience at all who are relying entirely on AI, to people with decades of software experience that don't use it at all, and then every possibility in between. And it's important that people are mindful of that. A lot of the debates around AI boil down to AI vs no-AI, as if the only ways to use it are to 100% vibe-code something, or not use AI at all. But there are so many steps in between.
2
u/Lyraele 1d ago
Nah, this stuff is not remotely worth the costs and externalities it imposes on the world. Take your pro-LLM nonsense somewhere else.
1
u/mynamesleon 1d ago
I'm trying to highlight that when someone has used AI in a project, there are different extents to which they may have used it, and that it doesn't automatically mean that the project is slop. And that usage of AI highlights how important it is to think critically about its usage.
I'm against AI myself, for many reasons. But I'm also a realist. This stuff is out there now, and it's not going away. But alongside that, there's a tremendous lack of education around it, and a lot of sensationalist headlines.
As a dev, my company also expects me to use AI. So I've made sure that I know how to, and I've setup workflows that intentionally require a lot of human intervention (plus we require multiple human reviewers of any committed code as part of our processes). I primarily use it for chore/highly repetitive tasks, like writing tests. I prefer to write the code for the actual functionality myself. I'm a dev because I like writing code after all. And I have to use at least some of my token allowance, because the company monitors that.
3
u/Lyraele 1d ago
That's just classic defeatism right there. Don't use it, won't use it, have straight up informed management it's unethical to use in addition to not being particularly good at anything. And since I'm usually the one that figures out what actually went wrong and can suggest a fix that actually works without a giant unreadable PR, the line is holding. And you can already see cracks in the narrative as the RoI just isn't materializing. This bubble can't last, and what remains after is not what everyone is going to be so gung-ho over. But it's also going to take years to undo the damage.
→ More replies (3)
10
u/UpSheep10 1d ago
This is a Tragedy of the Commons problem. Open Source has always existed with a fundamental assumption that even the most devoted power user can only produce a limited amount of code.
AI is industrialization reaching the waterways: it will pollute more than any single person could. Just like the Industrial Revolution our rules are now outdated because we cannot comprehend how much slop can be produced by one determined person with industrial tools.
English common law used to say everyone who lived on the river could partake and dump as needed because agriculture and craft were entire economy. The first mills exploited this to quickly overwhelm society and dominate the resource (waterways). By the time laws changed to limit industrial use, the common use rivers functionally belonged to the mills.
Our open source 'rivers' do need to be protected from the slop producers and our past tools may not be enough anymore.
That being said, "I don't know any other language other than English, so all other languages are banned," is not the right solution.
14
u/do-sieg 1d ago edited 1d ago
Yeah, AI made output so fast you can have something working and polished rather quickly, then dropped because of the low personal investment from the author. And since anybody can reverse engineer the thing in days, I don't even know how people will not just make their own clones instead of contributing.
I think the only projects that can survive long term are the ones where the devs are also users and will have an interest in maintaining the thing.
3
u/Substantial_Swan_144 1d ago
The problem with that reasoning is that even with 100% fully human code, nothing stops you from creating a project and abandoning it; in fact, there are more abandoned projects than completed projects (unsurprisingly). It's really more a matter of scale than anything.
2
u/blackkettle 1d ago
I think you are right and what were in the middle of Both in open and closed source is a huge ecosystem change. There are still a lot of things that are difficult to build, maintain and manage even with the best AI use by experienced devs. Realtime systems and systems with serious compliance requirements are not IMO “easier” to build. Faster to document? For sure! Easier and quicker to architect? Yes, absolutely. But trying to one shot them still leads to an enormous mess while careful curation like what Torvalds is doing - leads to huge measurable improvements - albeit still requiring stewardship.
But little personal projects? Fun games? These are one offs. Several of my friends who have zero programming experience built and shared a sweet html game with our group over the weekend. They built it with Claude over a couple of beers, just for our friends. That kind of thing is here to stay imo. And while it might technically be slop, it’s dope-ass slop. And I can take that - something dropped as an html file - and turn it into a live multiplayer game over webrtc in an another couple afternoons!
4
u/Houdinii1984 1d ago
I think this is the part that's being skipped over. There's a new genre of app right now that has no home. Right now, those apps are what's flooding the subs and it's a real shit situation if you look at it from a different angle.
The reason the apps are slop is because the folks behind them aren't devs. We're collectively treating them like they'll never be devs when we should be fostering values in them as newbies instead. We shouldn't be influencing them to quit. We should be influencing them to stop, look, and listen.
That means we, ourselves, need to learn a bit, understand where those apps need to live, understand the pain points these folks have, and we can take this entire group of newbie programmers and set them on a course for success instead of shunning them. Only one solution removes the slop from the situation, and it's not the one where we build a wall and act like it doesn't exist.
The only thing that works on a newbie is education, and if you take away the tool that got them started, they won't be receiving that education from you or your group, and they'll just go and continue to produce awful code. You don't need to embrace AI to pass on good coding information or tell someone they're doing something dangerous.
Instead, we're gonna let all the newest in the industry get taught by the very AI apparently no one wants them to use. You shun the AI users, you lose all control and ability to do anything whatsoever about anyone learning that way, and more and more are going to be getting involved daily.
→ More replies (1)
0
u/Affectionate_Use_164 1d ago
Good point about trust, still most of open source projects before AI were slop too, abandoned after 40 commits, or author ignores any issues and pull requests.
Have 3 projects that were fully vibe coded with AI, but in language that I'm proficient in, so at least I can judge it. Putting strict e2e tests for all main functionality was the biggest difference, same as humans AI assumptions often end up wrong, and it's doesn't remember high level vision behind the projects, have to pin it.
Second thing is to run code review in background agents. Or even personas testing when agents imitate user behavior.
3
u/mynamesleon 1d ago
Absolutely - the slop has always been there. And weirdly, it sort of should be (or maybe shouldn't be published). A lot of my early learning experience was coding things that there were already established libraries for, because it was important for me to understand how to get something like that working, and why certain workarounds were necessary.
Using background agents is really handy. They have completely separate context, so you can use specific models for different purposes, and get them to verify each others work from a blank slate. E.g. Sonnet for a comprehensive plan, than Haiku to implement (because from a detailed plan, it's actually okay at producing minimum viable code), then Opus to do code review.
→ More replies (3)
1
u/Fabulous-Put8401 1d ago
I'm an outsider following a post from r/antiai
I understand your point. Ai can be a tool. I would consider generating slop more akin to doom scrolling on social media, but also can understand that more refined usage would be different. It's a spectrum of usage.
If I'm understanding correctly, ai writing code can be a good time save if it wrights a block and is reviewed by someone who understands it, leading to less manual typing out the lines themselves. This is the tool usage of ai.
That is contrasted against asking AI to write a whole app and it going through little to no review and receiving updates also made by AI. No review, no understanding of the code, just some guy who had some vibes. This is more the doom scrolling analogy I give.
How does a platform, like this subreddit, filter those two apart without it being cost prohibitive?
0
u/mynamesleon 1d ago
It's definitely a spectrum of usage, yeah. From a non-dev relying entirely on AI, to a dev with decades of experience not using it at all. But there are a lot more phases in between than I think people tend to realise. Like experienced devs using it for chore-work, or using it for writing languages that they can competently read and review but not write. Or junior devs using it to help them learn, specifically by questioning its output (which is largely how I originally learned to code, by seeing how other libraries did things and questioning why they did it that way, and by writing code that there were already established solutions for so that I could discover the nuances for myself).
That's a tough question. And any answer that applies right now may not be applicable in a year's time, because of how the AI landscape is changing so rapidly. In the long-run though, filtering them apart is a losing battle. Tools eventually become ubiquitous, or at the very least, the output produced using them eventually becomes largely indistinguishable from the output produced without them. E.g. I wouldn't be able to filter out code that had been written in a fully capable IDE with the help of Intellisense compared against code that had been pain-stakingly written in basic Notepad without any Intellisense or Type analysis. So I'd argue that filtering out code by how it's written is the wrong approach. As much as that pains me to say (I'm a dev because I enjoy writing code after all).
1
u/Fabulous-Put8401 1d ago
Do you see a problem with volume?
From what I could tell, there's a real potential that AI content made autonomously without human input can be produced at a much higher rate than human content, and that most of the interactions would or could be from ai bot farms as well.
How does a company like reddit, steam, YouTube, Instagram or otherwise afford to keep hosting that content? Also, how do advertisers manage having the ads they payed for being pushed out to bot views?
Thank you for your well grounded replies to my ignorant questions
→ More replies (2)
3
u/Shoddy-Childhood-511 1d ago
Don't forget the surveillance plagiarism problem.:
All the hosted LLM companies like Anthropic and OpenAI train upon user sessions, so anytime you do work using their tools you are paying them for the privilege of improving their LLM at doing your job.
Anthropic and OpenAI exploit surveillance plagiarism to claim their models have some intellectual autonomy, but it's an existential risk for their business customers.
An open source company like Collabora or GitLab would still have some "secret sauce" in their internal hosting tooling, but if Anthropic and OpenAI learn this then they'll teach many small companies in India, china, etc, who'll sell the same services cheaper, and now just as well.
Also, the "we won't train on your sessions" promise to enterprise accounts is worthless, because there is a rich "privacy preserving machine learning" literature they could apply. Around this, non-cryptographic flavors seemingly get broken eventually, while the cryptographic flavors cost way too much. It's worse though since nobody knows what different customers wish to protect.
It's ultimately far safer for companies to rent cloud GPUs or even buy their own hardware, on which they run open weights models for their employees. As companies figure this out, then Anthropic and OpenAI would've no business model. lol
→ More replies (1)
4
u/zenmatrix83 1d ago
people either hate ai or embrace it, unfortunately we are spitting reddit and the internet into for ai or against ai, instead of looking for actual problems for the low effort spam that ai accelerates. Thats the biggest issue, if people stopped cloning existing systems, actually though for a second, and got past the amazement that got something working they didn't really have to work for and created something of us these issues would less of a problem for some. I'm all for using AI, but like any tool, people abuse it and it ruins it for others.
→ More replies (1)
2
u/pr0m1th3as 1d ago
AI is just another tool. We've had stack-overflow before that, and google search, and further back in time we had 600-pages "How to program with C++ Borland" textbooks, where we would search for "adivce" and "solutions" on how to make our code work. AI-coding assistants are as sloppy as any human coder out there. What really makes the difference between good and bad software out there is the determination of devs and maintainers to make it as good as possible. I've been coding since the 90's and I've been regular FOSS contributor for more than a decade already. Does my code contained (and probably still contains) bugs, sure it did and most likely it still does. The only secure way to 100% avoid bugs in code is to avoid writing any code. Plain and simple. Once you get into the business of developing software, what matters is your determination to produce good quality software.
Of course I use AI nowadays, and of course I used to visit stack-overflow with almost every google search for the best part of decade or so. And I use them because they allow me to improve my software development skills. The argument that AI writes sloppy code is exactly the same as you can find bad advice on stack-overflow. I don't recall this massive panic attack about devs using stackoverflow and other forums, nor do I remember moronic guidelines of FOSS projects about "reporting what you google searched and what discussions you participate in x-y-z forums before contributing this piece of code. If certain devs feel threatened by the lower entry level into programming, that's something to discuss in a therapy session, not an excuse for dooming/calling out a tool which further democratizes software development.
There has always been sloppy code out there pretending to be valuable way before LLMs emerged. Trust me, I've spent almost a decade as a researcher in biological departments across Europe watching people claiming credits for their 30-50 lines spagetti code in python or R as something so important that unlocks the mysteries of the universe. GitHub and Zenodo have literally millions of unmaintained crapware that no longer run in any platform and all this way before the current AI trend "I wanted to do X, so I built Y" that reddit is cluttered nowadays.
The biggest thread in FOSS is our capacity to build and maintain healthy communities around software projects and it has nothing to do with devs or non-devs generating AI-slop. We really need to get serious about FOSS because AI is here and it's not going anywhere. Our FOSS communities on the other hand may well be harmed if we continue along this bipartisan slippery road of pro-AI vs anti-AI cultural and political debate.
1
u/debackerl 1d ago
It's true that we used to joke about script kiddies, and people copy pasting code from the internet. Lot of people don't know algorithms anymore, and formal verification of programs, or memory management without GC. At least, LLMs can do those things...
→ More replies (1)
1
u/bezdazen 14h ago
what happens when the code made by AI is more secure and trustworthy than human code? At what point do we say, "oh don't worry, this [cutting edge model] coded it." And at what point do we give credit to design and thought put into what the outcome and final result should be? Or innovative app ideas, or really useful tools? Suppose someone vibecodes a tool that helps people, works really well, and is really useful?
I think there are 2 camps. Those who look at AI like it is an intelligence, a being, a person to attribute credit to. And those who see AI as a multi-purpose tool that is a super compiler with the ability to compile human speech, context, other resources and output code written in one or more high-level languages. It's clear AI companies are pushing people to look at their products and treat their products like the former rather than the latter. Language and culture is a big contributor to certain framings.
And another point. Look at the software clones being made. It still takes months to debug and solidify them and make them actually viable as an alternative.
1
u/debackerl 1d ago
I think that you would be a great mod.
I see a lot of thoughts in your essay, you would surely be much more competent to write good (non slop) rules for the community.
4
u/Made-In-Slovakia 1d ago
I disregard project that is clearly done with AI agent but author refuses to add clear AI statement. It may sound sad but in this era it is required to communicate it clearly and directly.