Episode Transcript
[00:00:00] If you looked at the PG Backrest repo anytime on or after April 27, you might have gotten a big surprise because this repository was archived by the owner on that date and it's now read only. So in case you aren't familiar with it, it is a reliable PostgreSQL backup and restore tool, and some people would argue it's the best option in the ecosystem right now.
[00:00:28] Or maybe it was, but I hope you, your friends, family and co workers continue to do well. Our first piece of content is actually the GitHub page itself for PG Backrest, where the main maintainer posted a notice of obsolescence.
[00:00:46] So PG Backrest is no longer being maintained and he has been working on it for 13 years through a corporate sponsorship of who he was employed by, which was Crunchy Data, but then Crunchy Data was sold and him being able to continue working on this project, he didn't have a clear avenue to make that happen.
[00:01:07] But as a result of this announcement, there's been a ton of blog posts done and I've ordered them in. Kind of my preference in terms of reading through them. The first one is PG backrest is dead now. What this is from mydbanotebook.org and she has been recommending PG Backrest as the best backup tool for years for Postgres.
[00:01:28] And David Steele was the sole maintainer of PG Backrest. But also she mentions Stephen Frost and Steven Furcott also contributed significantly as well. So she basically expresses her disappointment that this happened. You know, she says, David is a brilliant engineer and the product is really good, but spending resources on this product is not something that the acquiring company of crunchy data wanted to do. And David did try to find additional sponsors for the project, but it never really worked out. And to talk about what makes PG Backrest a little bit different is that the tool that Postgres provides, PGbase Backup, just basically takes your backup. Notably, There is no PGbase restore backup, so there's no tooling in Postgres itself to maintain a repository of what backups have been done for ensuring all the wall files are archived regularly. Basically, you're up to doing that yourself through scripts or get a tool that does it like PG Backrest or the other one is also Barman.
[00:02:35] So in terms of what to do about this, you know, there's discussion that maybe it will be forked. There's also discussion that maybe things aren't quite done yet.
[00:02:46] We'll take a look at a blog post about that and of course mentions Barman as another Alternative what's interesting about PG Backrest is that it came out and started coming into prominence right about when a lot of the cloud computing started taking off. So in terms of clients that I work with, they already had developed scripts or helped them develop scripts for maintaining their backup and restore process.
[00:03:13] So we didn't look for a tool like PG Backrest or use Barman or especially in later years, they were in a cloud environment where all of the backups and restores are handled for you with the tooling provided by the cloud vendors. So part of me wonders if this is a little bit of unfortunate timing too, where things set for PG Backrest Next article I wanted to cover is Notice of Obsolescence. This is from thebuild.com he recounts the story as well, and he emphasizes PG Backrest was the feature complete open source backup tool. It had block level incremental backups, parallel restore page checksum validations, S3 Azure, Google Cloud support encryption and a custom protocol. And it was maintained primarily by one person.
[00:04:03] And of course this hearkens back to the image you see here that demonstrates all modern digital infrastructure within one little precarious piece down here, a project some random person in Nebraska has been thanklessly maintaining since 2003.
[00:04:18] Now he recounts the story of course as well, and he says, well, what do we do about it? Well, one alternative again is the custom script that you can do. And in this case, as of Postgres 17, if you're on that version, you could use PGbase Backup along with PG Combine Backup to at least get the incremental backup features that you may need. You could switch to Barman or Wallg is another alternative he mentions.
[00:04:44] Or maybe you use Wallg in conjunction with PGbase Backup and PG Combine Backup. Next piece of content PG Backrest is archived. What now this is from percona.community and he says, all right, what's going to happen now? Are we going to have the fork wars or different people try to fork it? But basically this probably won't help in the long term future. Now what's interesting about Percona is because Percona includes PG Backrest in the percona distribution for PostgreSQL. So they are definitely relying on it for, you know, their product offering.
[00:05:22] So it looks like they're not going to be moving to or recommending Walgie or Barman, but saying what can they do working with others in the community to continue supporting PG Backrest. There's a quote here. Percona will continue supporting PG Backrest, but apparently discussions are ongoing. So in terms of immediate things to do, their recommendation is, you know, keep using PG Backrest as you have in the past and see what transpires over the next number of weeks. But my thinking is that if this project PG Backrest is going to survive the then the people who are actively dependent on it should step forward to take over the project. But if nobody does that, it'll just become abandoned. Next piece of content, also from percona.community is open source doesn't die, it gets unfunded. And he took a particular offense with someone saying PG Backrest was end of life.
[00:06:17] And he says no, no, no, this is open source, nothing is end of life. But again, someone or multiple organizations is going to have to step up to support the project moving forward.
[00:06:29] And he brings up a point that he would really love to find some sort of organization, maybe a foundation that can shepherd some of these projects, maybe that's providing governance, helping hand to the ecosystem, maybe funding, providing guarantees of healthiness, etc.
[00:06:49] Now related to that next blog post, PostgreSQL ecosystem problems this is from PropenSource.it she's calling out the Postgres ecosystem problems where projects like this can just die and it would be beneficial to have some sort of perhaps non profit organization in the Postgres ecosystem that could shepherd these projects, something she calls an umbrella for the ecosystem. And she mentions organizations like the Apache Software Foundation, Codeberg, but it would be interesting to see what that looks like.
[00:07:22] Next post why the cycle of Open Source sustainability needs to be virtuous this is from gabrielebartolini.it again talking about PG backrest, but he has a unique perspective on it because he actually is the creator of Barman. Now he hasn't worked on that in quite some time and now of course he's working on Cloud Native pg.
[00:07:46] But he gives a little bit of a history of the two tools, PG Backrest, Barman, and how they kind of approach things in slightly different ways as to why the two projects exist as opposed to just one. But it gets to the heart of an open source project protecting itself. And what he's done with regard to Cloud Native PG in that it has been accepted into the CNCF sandbox, which means its intellectual property and its continuity are protected by a vendor neutral, transparently governed foundation independent of any single company, employer or sponsor.
[00:08:25] Now this is just the first rung of the ladder, the sandbox level. You can move into the incubation stage which requires demonstration across multiple organizations and then graduation where no single organization can ever hold the project's future hostage. And unfortunately, he says, PG Backrest, for all its technical merit, was a solo maintained project with no equivalent of this type of structural protection.
[00:08:52] And he talks a little bit about this virtuous cycle where the community works with end users and those end users pay for potentially maintenance contracts to supporting organizations for support and the tooling potentially to be built for what they need, and that pours back into the community again. And this is the type of structure needed to maintain projects. Now the next piece of content is related, but not directly related to PG backrest on why sell the idea of contributing to PostgreSQL to your employer? This is from Valeria Kaplan and talking about the right ways to sell working on Postgres to your current employer. Because as an example, according to a Stack Overflow survey of half a million developers, over half of them have done extensive development work in Postgres, but yet only around 2,000 people actively contributed to it.
[00:09:50] So as he says here, only 1 in 8,500 users contribute to the project.
[00:09:55] Now I'm sure there's other people that contribute different ways to it. I mean personally I think my show is a little bit of a way to contribute to the project, communicating what's happening recently in Postgres.
[00:10:07] Because personally I don't think you want me writing C code for the project.
[00:10:11] But basically this post is his call to get more people involved in contributing to Postgres.
[00:10:18] Next piece of content My Queries to Monitor Autovacuum this is from cybertech PostgreSQL and this post is great because he just walks through the different queries he uses to keep track of his Autovacuum processes on his system.
[00:10:34] The first covers monitoring how it's doing deleting dead tuples.
[00:10:40] The next one is monitoring table bloat.
[00:10:43] Next is monitoring the status of auto analyze.
[00:10:49] Next is a query to monitor how the visibility map is being maintained as well as watching out for a transaction ID and multi exact transaction ID wrap around.
[00:11:00] So definitely encourage you to check out this blog post and get those queries to use if you don't have any of your own.
[00:11:06] Next piece of content There was another episode of Postgres fm. This one was on PG Weighttracer.
[00:11:13] So Nick and Michael were joined by Dmitry Foman because he developed this new tool called PG Weight Tracer. So this traces weight events. So I was looking at this and thinking wait, don't we already have that with PG weight sampling? But this is actually different. It uses a technique to read the back end memory state to in near real time determine what the current weight events are. And I think he proposes on a nanosecond level. So if we look at the project here, it captures every wait event transition across all backends with nanosecond precision.
[00:11:53] And because it's just reading the memory of the system, it doesn't require any Postgres patches, extensions and no restarts. He shows a demo of a graph analyzing the different wait events as they're happening here and it even has a text dump of the results as well.
[00:12:10] So this looks super interesting to me. And in the show itself they definitely talk a lot about the importance of wait events as that's definitely something Nick finds super important as well. So if you want to learn more, definitely encourage you to listen to the episode or watch the YouTube video down here.
[00:12:28] Next Piece of Content Parallel Auto Vacuum it's not about the CPU. This is from thebuild.com and apparently in Postgres 19 we're getting a new configuration option, Autovacuum Max Parallel workers.
[00:12:42] So he says since Postgres 13 we've been able to do vacuums in parallel, but you could never auto vacuum in parallel. Well, apparently you can do that in Postgres 19, or at least there's a patch available to do it. But he says hold on now, because a lot of times people hear parallelism and they think, oh, this will make everything faster.
[00:13:06] But the issue with vacuum is that he says the CPU isn't your problem in terms of vacuum. He says it's overwhelmingly I O bound. So if you're I O bound, splitting things up into workers across multiple CPUs is not going to solve your I O problem.
[00:13:23] It's the disk subsystem that needs to be sped up. So he thinks for the vast majority of databases, you probably don't need parallel autovacuum, he said, but there may be certain tables where it makes sense. Maybe you have a gen index, or if you have four or five plus indexes on that table, then maybe it makes sense. And in those cases, maybe just set the autovacuum parallel workers at the table level as opposed to doing it globally. But if you want to learn more, definitely check out this blog post. Next piece of content RLS sounds great. Until it isn't. This is from planetscale.com and RLS of course he's talking about row level security, which is definitely known to have performance issues, and he covers a lot of those issues here.
[00:14:11] Basically every query is essentially adding a where statement to it. He does show a way to try and avoid some of the performance issues, but still it's a workaround you have to deal with.
[00:14:22] So even though he does give some general guidance on how you could potentially do it, he still says at PlanetScale, we typically recommend against relying on Postgres RLS due to a higher overhead not only to performance, but also developer experience complexity. Because now all of these rules exist in the database, it's not in your code base. You have to make sure to check the database configuration to understand what's happening.
[00:14:47] But if you want to learn more, check out this blog post Next Piece of Content hot updates in Postgres this is from boringsql.com and this talks all about heap only tuple updates and how they work in Postgres. So definitely check this out if you want to learn more about that.
[00:15:04] And the last piece of content permissive by choice, permanent by accident. This is from thebuild.com and he's actually talking about the current permanent Postgres license or state of it, and how no other project is basically like Postgres.
[00:15:23] He says so many other projects have copyright sign off or they assign copyright to a foundation or some sort of organization like Free Software foundation here, Apache Software foundation here.
[00:15:38] But Postgres doesn't do any of this.
[00:15:41] There's no assignment, there's no foundation that owns the code. All the contributors independently own the code. That's also what makes it so hard to try and take it over, because you would have to get copyright approval from thousands of people, which is pretty crazy. But this is an interesting blog post and if you're want to learn more about it, definitely encourage you to check it out. I hope you enjoyed this episode. Be sure to check out scalingpostgres.com where you can find links to all the content mentioned, as well as sign up to receive weekly notifications of each episode. There you can find an audio version of the show as well as a full transcript. Thanks and I'll see you next week.
