r/opensource • u/nicholashairs • 28d ago

Promotional Feedback on AI Policy

Hey r/opensource, I'm the solo maintainer of `python-json-logger`.

I've recently started to receive a lot of AI generated contributions so it's about time I set a policy.

For those of you who are also maintainers of projects which allow AI contributions, does anyone have any feedback or recommendations?

My planned policy is in this PR on GitHub:

AI Policy

The use of AI tools is allowed with the following conditions:

  • The use of AI tools must be declared. For example in the PR description, or through dedicated users / commiters.
  • The specific AI tool / provider should be declared.
  • AI tools must be under supervision of a real human who is responsible for all contributions made.
  • The use of AI tools does not lower the expectations of any contribution.
  • Don't be a meat-proxy.
  • Respect the time of the maintainers by ensuring that generated contributions are genuinely valuable.

Thanks in advance!

43 Upvotes

88% Upvoted

21 comments sorted by

9

u/cyb3rofficial 28d ago

Set up an action to fire on PR to auto comment to ask the User if they genuinely looked at the code, the code is verified to work, fixes applied were not just one shotted so on. Multiple failures to comply/understand /read will result in being blocked from future contributions.

https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow

It would be best to set ground rules and have it auto comment on every new pr.

This could help ease the submissions.

5

u/nicholashairs 28d ago

Using a workflow to comment and gtet a response is a good idea, it's probably a bit more effective than a PR description with checkboxes since the AI would accept them anyway.

I probably won't implement this at the moment as the volume is not so great that I need automations to help, I can manually enforce the rules once I have them.

But a good idea to keep in mind if it does become a problem 🙏

3

u/David_AnkiDroid 27d ago

A checkbox in the PR template is less overhead

18

u/omniuni 28d ago

That's basically reasonable. At the end of the day, if the PR is clean, specific, readable, and the author demonstrates that they, personally, have complete understanding of it, that's what matters. Just because Gemini found the faulty code, GLM fixed it, and Claude wrote the unit tests, that doesn't negate it inherently.

1

u/martinus 27d ago

I am unsure an AI attribution is useful. You can't trust this anyways, and it's safe to assume in near future basically all contributions will use AI in some form. I think it just adds noise.

What bothers me more is the the difficulty to detect malicious contributions. It's getting far too easy to make contributions that include some malicious stuff. I think nowadays it would be safer to only allow issues and no code contributions, unless you really trust the contributor. The issues can then be analyzed by an AI for prompt injection and other safety issues, and then implemented by a trusted AI setup.

1

u/nicholashairs 9d ago

I agree that the AI attribution isn't trustworth data, however my main reason for keeping it is that in the future if there is an issue with having AI generated code I know where to go looking in order to remediate it.

Also a valid point of malicious contributions. I do review all code before it's added so as long as the GitHub UI displays all the changes accurately I should be relatively safe from prompt injections and other malicious behaviour.

1

u/NXTler 26d ago

I saw on some repositories that they ban mentioning the specifc provider/model/tool so it cannot serve as an advertisement, which I like.

1

u/nicholashairs 9d ago

That's a fair approach, it's definitely part of their marketing strategy (like all the email providers that used to add "Sent using X the Y email provider").

That said, I'd rather actually keep it so if in the future there is issues with having AI generated code in the codebase I know where to go looking.

1

u/Fresh-Daikon-9408 22d ago

For bug fixes, ask for a small reproduction and the test they ran. Makes 'I checked it' easier for you to verify.

1

u/earonesty 1d ago

I've been getting a lot of ai prs on my oss stuff and it's all been decent so far.  generally well tested, and with perf benchmarks no human would have had the patience to do on an edge case.  6 months ago, it was slop prs that I wasted my time closing.  a lot has changed 

1

u/stealthagents 8d ago

Totally agree with you on that. As long as the human behind the AI can explain their choices and the code is solid, who cares how it got there? It's all about the end result and keeping the quality up, right?

0

u/[deleted] 28d ago

3

u/SheriffRoscoe 27d ago

Are there any models that aren't trained from copyrighted material?