Dependency Mapping Working Group

Invite sent out. Everyone who is interested in talking about Libraries.io, thatā€™s the topic for Wednesday based on last weekā€™s conversation, along with weight mapping for deps.

Agenda is here.

Matt Germonprez from CHAOSS said heā€™d be able to attend the next meeting to talk about what theyā€™re doing there involving metrics. @GeorgLink You might be interested, too. Iā€™ve sent him an invite.

Yes, please send me an invite as well :slight_smile:

1 Like

I contacted ClearlyDefined, and they donā€™t keep trees of project dependencies. However, they pointed me to this list: Compliance Process for Developers - Open Compliance Program

Might be some projects in there that do this work, too?

Two years ago I tried to use CHAOSS to see my digital trace and because the ticket was closed Iā€™ve got an impression that the project is more about collecting data for companies rather than serving users. It is still interesting to know about what metrics they gather.

I couldnā€™t tell you on the meeting, but just want to comment that FOSS Backstage was a very useful event. Need to find a time to wrap it up though.

1 Like

Hi @abitrolly. Thanks for your interest in the CHAOSS project. A lot has changed in two years :slight_smile:

We have several initiatives that are meant to draw out the developed metrics so that they are more meaningful to people. Happy to chat more and help in any way.

Hey Everyone,

I created another topic for weight maps specifically, it can be found here.

1 Like

Thanks, Joel.

An update on today: Our regularly scheduled Dependency WG is today, but I canā€™t make it, as Iā€™m attending the Mozfest session on open source governance run by Georg, Javi, and Greg (here: mozfest-2021-website).

As well, I think attending a few more calls with the Risk WG would be great to assess whether or not we overlap in efforts. Alyssa and I were at the last call; would be great to see others there. GitHub - chaoss/wg-risk: Risk Working Group Repository

Given that, I think it makes sense to cancel todayā€™s session. Let me know if youā€™d like to meet, anyway!

1 Like

CHAOSS risk assessment focus may be too broad for discussing building dependency trees. But I donā€™t mind. There are still things I need to write down since the last meeting, so I will dedicate that hour to the writing.

  1. deps.clouds and fetching the graph of dependencies for my Python projects. Parts are there, but interface is not. Evaluated pipgrip and a few other tools. Need some kind of awesome list with reviews, roadmaps and progress updates on these tools for my user story.

  2. CHAOSS digital trace. I planned to create a separate topic on this forum, but could not find place and time. Basically I need a public timeline of my activities for each day that I can mark. This tool should help people to take a glimpse into what the life of maintainer entails and ā€œbillā€ activities to certain aspects. Like calculate the maintenance debt for my open source libraries by multiplying my hourly rate to all the time Iā€™ve spent answering and troubleshooting issues, and then diffing that to my income and cost of life.

1 Like

Looks like Gunner and Tobias may be able to make it today, @abitrolly and others. Iā€™ll make sure you can get into the zoom before hopping off to help with the Mozfest session.

just seen this. thanks for being on hand to keep things together!

Slight snafu in the calendar invite, but hopefully others were able to make it. Thank you for coming and taking over the host, @abitrolly!

Hey all! Our regularly scheduled WG is starting in an hour. Hope you can all make it - let me know if anyone is missing the invite. Here is the WG calendar for people who want to see the event on a calendar.

1 Like

After talking to @benjam, I think it makes sense to free up the Dep WG hour for now by not continuing with the active meetings.

Over time, the WG has solidified around helping out Joel from Flossbank with their work on dependency graphs, and @benjam is going to keep helping them out as needed. By this point, all of the WG members know each other well enough to reach out and ask for help as needed, I believe. As well, the Risk WG from CHAOSS is also doing similar work in this area.

If anyone has other topics that you think would be good to discuss, drop them here, or open a topic like this one: Dependency Tree Node Weights - #7 by sgoggins. Also, if Iā€™ve radically misunderstood the status of this WG, let me know.

It may be a good outcome to make a write up as there are people who may find this thread and I imagine they would be left with an impression that this meeting stuff is useless. While we are synchronized and are definitely way ahead of where we were before we started, the outside world is not synced. Thatā€™s the consequence of not being able to stream in the public, where you may be cancelled for expressing critics or arguments against things that people love or even build.

For me the write up is basically the discovery of few initiatives that do dependency mapping - some older ones, some newer (with CHAOSS in the end). I still not that good organized to keep track of who came from which project, though. It was nice to hear about funded opportunity for dealing with Python dependencies on https://deps.cloud/ (which I havenā€™t used), the discovery of closed conferences about open source (still nice), the way how people see the dependency mapping from different aspects (marketing, design, management, legal, etc.) that I have no experience with.

In the end it was nice that I switched to testing practical solutions, which now requires significantly more time than just 1 hour every 2 weeks. Like getting all user stories in order and each two weeks iterating over them stating current status. Stories like ā€œget a list of all Python dependencies for my package without rebuilding the packageā€ are still not solved in 2021 and there is no place where you can place spotlight on them. Or coding a visual weight mapping in Godot or Observable - thatā€™s like a full time job, and that job is not sustainable. I feel like there is a need in capacity building to continue digging. It was fun so far.

1 Like

:+1:

@abitrolly thanks for the notes. For me I think the conversation went something along the lines of

resolving dependency trees is something we all need to do and a job we can all share

from there we discussed the merits of what we could do with that dependency map, which we decided to ignore as thatā€™s where we all diverged, but we share this core.

Libraries.io was built intentionally to be a central resource that others could build from and extend

was my pitch to the group. Librareis.io is currently under the stewardship of Tidelift who took it on when Andrew and myself joined, but they have done little with it. Some are limited by the rates onthe API, some wondered whether we should fork the project. This is an undertaking and one that has costs associated with it (databases are large and growing) so it is only worth considering this at the point we would see considerable uplift in value. Some did not thing that crossover was yet upon us.

Letā€™s continue to work on our propositions and come together when we feel this pain more accutely.

Was the conclusion that I took from this. @joelwass is happy to continue resolving dependency trees on the fly and taking the hit to do so if needed. FairOSS didnā€™t appear to be far enough along yet for this to be hurting them either. Iā€™m not sure about LibreCelery as Tobias wasnā€™t able to make it, but I beleive the LibreCelery is on pause for the moment.

If anyone took a different reading from the above please do share it. Everyone brings their own interpretation and no one is right or wrong.

Thanks

2 Likes

Iā€™m going to revive this conversation because I few things happened last week:

I noted a request from a colleague re. some dependency mapping projects, and I recalled a bunch of analysis we did at Libraries.ioā€¦ but the pages we made feat

  • ā€˜digital infrastructureā€™ projects with >100k dependent repos
  • ā€˜unseenā€™ infrastructures with 100k+ dependencies and <100 stars
  • bus factor dependencies with >100k dependents and 1 maintianers

I quickly checked and, last month, Tidelift started methodically removing these routes,:

then, later that week I got a note from a Universirty in Berlin who are looking at Libraries.io to help with some of their studentsā€™ research, so I tried to point them at the data downloads that we documented at libraries.io/data

again gone.

Then I read the report from Plaintext group: Securing Open Source Software at the Source

Recommendation 1: Identify and catalog critical software in need of support

and I canā€™t help but think thatā€™s what we built Libraries.io for, and Tidelift are paring it down to be there own little personal library. And I think nope.

So, my question is. Does anyone in this group actually need a resource like this? Because I am pretty sure I can find the :dollar: and the :innocent: to support itā€¦

Iā€™m not sure if Flossbank has a direct need for this (maybe we will in the future) but we build things that the community needs - not just us - so I think weā€™re very interested in reviving libraries.io if the money to support the infrastructure isnā€™t too costly (or can be covered)

We could definitely take care of the engineering aspect. Happy to chat more

For advocacy I am interested to have a single link to such dashboards, and I am interested in mapping all OSS, not just libraries.

Right now Iā€™ve got only GitHub - epam/OSCI: Open Source Contributor Index - which is a dashboard for corporations to brag about how much they commit to open source projects. Which can also be the stats how much open source projects do they own,