SustainOSS Media as a Product

In invisible world of digital infrastructure, the survival of people and projects depends on the visibility of meta issues that happen with people and projects. Visibility allows these issues to be fixed. The journalist approach would be to write about it constantly, and the product approach would be setting relevant metrics and tracking them.

My personal story this month would be I personally have is the lack of reviews/merges for https://pypi.org for more than half of a year. The reason I guess because people are people, they want to be in control, they don’t like to be forced, and they can not and do not feel like solving conflicts. That’s why we have forks in Open Source. There is no single truth, single source, single repo. Forking is a way to solve conflicts.

Now we are talking about infrastructure. We have https://pypi.org and due to the reason how economics work, it is probably the only reason why Python Software Foundation is sponsored. Much like NPM and others. I can’t validate this assumption. I just guess. To validate I need to compare $300000 that were donated just for PyPI with total donations to PSF. Some donations are probably funds to develop specific features, some maybe only for maintenance, but who am I to taks to foundations? I can’t even find the right mood to mince my words to speak with people so that they won’t break. And that’s why I came here. To see if SustainOSS can help. And it could help if it could be an Open Source Media Product, that also resolves conflicts in addition to be publishing platform.

You know, in traditional media business you have these actors, TV personalities and the ability to choose channels if you don’t like it, but even if you put them in the open, ask questions about areas they have no experience with, they would probably also show their other human sides, which is - being opinionated, offensive in their views, awkwards, religions, whatever. You can exploit their human qualities to cancel them. But in our world we don’t have other people except those who are already open. Code of conducts, rules, bans, they all about desire to ignore this nature. But if there is a thing that can help bring up conflicts and solve them, then we could bring open source collaboration to a new level.

I’m not sure I understand your concern. Are you saying that you submitted patches to pypi and these are not being reviewed?

When I managed the OpenStack community, I dealt with multiple cases where the interests and priorities of maintainers and contributors clashed for disparate reasons. In my experience, there is no way to solve these issues without looking at the details.

Well, it all started with this PR - Extract and host wheel METADATA on upload by abitrolly · Pull Request #9972 · pypa/warehouse · GitHub - which is the 3rd top voted feature among issues. I was trying to solve dependency resolution problem for Python and found that pip downloads huge archives from PyPI just to parse the metadata with dependencies and versions from there. If packages in metadata doesn’t meet currents constrains, pip downloads other versions until a match is found. In case of Tensorflow each version is 400Mb+ archive. Considering how many people are trying Python solely for learning AI, that’s a huge waste of traffic, so I expected $300k sponsors to the project and PyPI maintainers to be interested in solving this problem. But after some initial comments nothing is happening. In the end I got completely ignored even for PRs that Fix broken development setup by abitrolly · Pull Request #10469 · pypa/warehouse · GitHub

I see… Waiting over 4 months for a patch to merge is frustrating. I’m not very familiar with the way PyPi organizes its development. With OpenStack, each project held a weekly meeting online on IRC, at different timezones to accommodate participation from the east and west of the world. Also, much of the tech leaders hang out on IRC and can be asked to provide feedback on merge requests. During the IRC meetings it was possible also to ask for a slot to further discuss features. And ultimately, OpenStack held quarterly in-person meetings among developers. Additionally, a mailing list was available to further discuss a PR.

All these would have been great opportunities to iron out differences of opinion, gauge the importance of the patch proposed, rally support from other community members.

Usually, the problems I saw were caused by reviewers being busy with other priorities. It’s as simple as that: it’s not your patch the problem nor lack of money. The problem most likely is the fact that your patch is not included in a roadmap yet. This is not a problem that money can fix: my experience tells me it’s a maintainer bandwidth issue, a release management and project management issue.

When problems like this increased too much in OpenStack, we introduced processes to debate features before they came in as proposed patches. That way a release manager would know to allocate precious review time to the features that were planned for the release.

Is there a way for you to engage with other PyPi developers and users that is not on GitHub? IRC or voice chat? Can you think of the root cause that is preventing the tech leads of PyPi to take the time to fully review your patch?