Improving the release announcement editing/reviewing process of major releases

[joyeecheung]

In the release announcement curation process of recent major releases there have been a lot of room for improvement, so opening an issue to discuss how we can make it edited better. Specifically, what goes out in the blog post on nodejs.org, and in the summary of the change logs:

1. We can make the release announcements reviewed more precisely by contributors using a regular PR adding a document somewhere (the website repo?), and we can do reviews on GitHub
2. We can make some changes to the release process so that we have sufficient time to edit the release announcement, and notify authors of notable changes/semver-major changes in time for the editing process
3. We can do some format changes to the release announcements on the blog posts to improve readability and help users grasp what's new in the release e.g. maybe remove/fold the list of commits in the posts, since people probably won't read them in a blog post and they belong more in changelogs.

[joyeecheung]

cc @nodejs/tsc @nodejs/releasers

[marco-ippolito]

Major release announcements already follow the process you described.
Example: nodejs/nodejs.org#6661

[joyeecheung]

I think they may be true to some extent for 1, but 2 and 3 are still not there yet.

For 1, since the release announcement is copied multiple ways, it's unclear which copy is the source of truth. I feel that the website repo maybe the easiest one, but I don't think we have confirmed that once you update the blog post everything else will be updated to be in sync before.

For 2 the more important part is notifying and involving authors of the notable changes being released in the editing of the announcement, and the current process IIUC is that releasers may or may not ask the author personally (depending on how much time they have) to get some text somehow, and then put together a summary based on the texts somehow, and then the final text may or may not get reviewed by authors of the changes before going out.

In terms of the _somehow_s mentioned above, from personal experience sometimes I get asked to edit OP of the PR, sometimes I was asked to edit the bot comment, sometimes there was no asking; I tried to stay on top of things and edit the announcements either before or soon after the release but it can be easily missed if I am not paying attention to the release schedule, and I have seen other contributors complaining the major release announcements picking the wrong things to highlight for a component, so I assumed they were not pinged before the announcement was published.

I remember that we had some discussions before about automating these, but as a first step maybe we should come up with a process that can be followed more precisely before trying to put together something to automate it.

For 3, it doesn't seem to be part of a process and the format isn't settled AFAICT. e.g. https://nodejs.org/en/blog/release/v23.0.0 still contains list of commits

[targos]

I agree that there are many issues with the process, but why should we only improve it for major releases? Those aren't usually the most interesting to the end user anyway.

[ruyadorno]

1. We can make the release announcements reviewed more precisely by contributors using a regular PR adding a document somewhere (the website repo?), and we can do reviews on GitHub

I haven't been brave enough to volunteer to a major release yet but I'd love to hear from folks with a lot of experience on it what are their thoughts, given that this proposes adding even more process to it. cc @RafaelGSS @BethGriggs

2. We can make some changes to the release process so that we have sufficient time to edit the release announcement, and notify authors of notable changes/semver-major changes in time for the editing process

When the collaborators follow up with the current bot automation, it works pretty well and it makes the life of a releaser 100x easier, here's a good example: PR in which the author followed up from the bot comment by adding release notes and the final result. I think we might want to explore some education / awareness raising among the collaborator base but maybe there's also things that we can tweak to make the process work better. Anyways, open to hear more on this one but I wanted to highlight what is the intended process from the point of view of Releasers today.

3. We can do some format changes to the release announcements on the blog posts to improve readability and help users grasp what's new in the release e.g. maybe remove/fold the list of commits in the posts, since people probably won't read them in a blog post and they belong more in changelogs.

My initial reaction is that I'm -1 on removing the list of commits since historically the blog post has been "the changelog" that the community refers to - on top of that it also doubles as extra recognition for folks that contributed something to that release and I don't want that to be hidden / folded.

[ruyadorno]

To add to that: of course I'm always open to change my mind if there are better alternatives to be explored but in the case of historically consistent formats like the blog post we should also consider that any time we change something there's a big friction on the side of consumers. I don't think we should take it lightly.

[BethGriggs]

I don't have a lot of time on my hands to provide input, but, just to share my experience.

When preparing major releases I used to try and prepare the blog post a couple of weeks in advance and then share it with Releasers/TSC for review. I got minimal responses from the TSC - I recall on one occasion only ~2-3 of the 20+ TSC members provided any feedback (no shade intended, we're all busy with other things). The process in nodejs/admin#675 worked a little better - the only difference was using GitHub rather than a Google Doc (original approach from working with the foundation).

Whatever the improvements are, they need to be crowdsourced from contributors (and ideally automated). It needs to be push-based: add notable-change label, provide text as the bot requests. This would also mitigate the critique directed at the releaser when a feature is missed or miscommunicated.

Releasers really cannot afford any more cat herding process on their shoulders.

My initial reaction is that I'm -1 on removing the list of commits since historically the blog post has been "the changelog"

Also -1 on removing, I know people use those as a reference for each release. It's also useful to mitigate GitHub failing to render some of our longer changelog files.

[RafaelGSS]

I've been doing Node.js major releases since v19 (I suppose) and I can share some of my perspective here. I feel that people underestimate the amount of work a release (a major release specifically) takes, and even though the release proposal is opened 7 days before the target date, people review it just a few days or hours before the release. If you look at previous releases, you will see that we always had to rush because some changes were requested very close to the release date, such as:

• Update the CHANGELOG
• Include a new PR
• Remove a PR

That's the main reason I wrote nodejs/node#55732. I believe that with a fair amount of time (1 month) and a strict policy of not including commits less than a month prior to the release, it should make testing and collaboration much easier.

My suggestion is to see how it will work for Node.js 24, document the failures and improve for Node.js v25.

[joyeecheung]

When the collaborators follow up with the current bot automation, it works pretty well and it makes the life of a releaser 100x easier

AFAIK, this part isn't actually automated? The bot says:

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

Which according to my interpretation, means you should just put that text in the PR description and it'll get copied, but I think most releases I've seen don't exactly copy the PR description into the release announcement. Maybe the magic sauce is that one has to add a line "For release notes:" but that format isn't clear from what the bot says. I remember I asked in the node-releases channel a while back and the answer I've got was also that it wasn't automated. I think this should use some explicit format that's actually practical to automate (I discussed about it with @aduh95 a bit on slack, will open a different issue about the format)

[joyeecheung]

Whatever the improvements are, they need to be crowdsourced from contributors (and ideally automated).

That's also what I was trying to emphasize in the OP, that it should really be collaborated with the authors of the notable changes - I think releasers and the TSC are not necessarily the right people to consult for these release notes, there's no guarantee that a notable change must be implemented or reviewed by a releaser or TSC member, or that TSC/releaser has the right expertise to write a proper text description for it. Some past complaints I've seen about the release announcements actually came from non TSC/releaser contributors.

[RafaelGSS]

but I think most releases I've seen don't exactly copy the PR description into the release announcement. Maybe the magic sauce is that one has to add a line "For release notes:" but that format isn't clear from what the bot says. I remember I asked in the node-releases channel a while back and the answer I've got was also that it wasn't automated. I think this should use some explicit format that's actually practical to automate (I discussed about it with @aduh95 a bit on slack, will open a different issue about the format)

We are working on it. This process isn't automated yet, but it will be.

[joyeecheung]

I opened #1068 to discuss the notable change text format - I am pretty sure without an explicitly standardized format it's almost impossible to automate it, what the bot currently says is way too open to interpretation. Even before any automation is done, having a proper format alone could eliminate most of the micommunication.