ClickHouse recently open-sourced WAL-RUS, a Rust-based replacement for WAL-G that cuts virtual memory consumption by over 70%. The tool processes PostgreSQL Write-Ahead Log files and base backups with dramatically lower resource overhead. ClickHouse built it to support a future managed PostgreSQL offering on their cloud platform.
TL;DR: ClickHouse has open-sourced WAL-RUS, a Rust rewrite of WAL-G for PostgreSQL backups that reduces virtual memory usage by over 70% while maintaining full compatibility with existing WAL-G archives and configurations. The project targets teams running large-scale PostgreSQL deployments who need efficient, reliable continuous backup without the memory bloat of the original Go-based tool.
What Is WAL-RUS and Why Did ClickHouse Build It?
WAL-RUS is an open-source backup tool written in Rust that serves as a drop-in replacement for WAL-G, the widely used PostgreSQL continuous archiving utility originally written in Go. According to ClickHouse’s engineering blog, the company built WAL-RUS because they needed a reliable, resource-efficient backup solution for a future managed PostgreSQL offering on their cloud platform. The existing WAL-G tool, while functional, consumed excessive virtual memory during archive operations. ClickHouse engineers found this unacceptable for a multi-tenant cloud environment where every megabyte matters.
The project reimplements WAL-G’s core functionality — base backups, WAL segment archiving, and restoration — using Rust’s memory model and zero-copy abstractions. Rust provides compile-time memory safety guarantees without garbage collection pauses. This matters for backup infrastructure.
Why Rust specifically? The language offers deterministic memory management through its ownership system. There is no runtime garbage collector competing with PostgreSQL for CPU cycles during backup windows. ClickHouse already uses Rust extensively in their analytical database engine, so the team had deep expertise. The decision aligned with existing engineering capabilities.
WAL-RUS is available on GitHub under an open-source license. The repository includes documentation for migration from WAL-G, configuration references, and integration guides for major cloud storage providers.
How Much Memory Does WAL-RUS Save Compared to WAL-G?
WAL-RUS reduces virtual memory usage by over 70% compared to WAL-G when processing identical PostgreSQL workloads, according to benchmarks published by ClickHouse. The company measured both tools archiving the same WAL segments and base backups against production-scale PostgreSQL databases. The results showed WAL-G consuming significantly more virtual memory due to Go’s runtime overhead, garbage collection metadata, and goroutine stack allocation patterns.
Rust’s ownership model eliminates these overheads. Memory is allocated and freed deterministically. There is no garbage collector tracking live objects across heap generations. No runtime scheduler managing lightweight threads with their own stack regions. The binary runs as a single process with predictable memory characteristics.
Consider a typical scenario. A PostgreSQL instance generating 50 GB of WAL segments per hour would cause WAL-G to spike in virtual memory as concurrent archive operations queue up. WAL-RUS handles the same workload with a fraction of the footprint. This efficiency translates directly to cost savings in cloud environments where memory allocation drives instance sizing and billing.
The memory savings also improve reliability. Lower memory pressure means fewer out-of-memory kills from the Linux kernel’s OOM killer. Backup operations run to completion without interruption. For managed PostgreSQL offerings, this predictability is essential — a failed WAL archive can break point-in-time recovery chains and leave customers without valid restores.
Is WAL-RUS Compatible With Existing WAL-G Backups?
Yes, WAL-RUS maintains full compatibility with existing WAL-G archives, configurations, and command-line interfaces. Organizations can switch from WAL-G to WAL-RUS without re-archiving historical backups or migrating stored data. The tool reads and writes the same file formats, uses identical naming conventions for WAL segments, and supports the same cloud storage backends as WAL-G.
This compatibility extends to configuration files. WAL-RUS accepts the same environment variables and configuration keys that WAL-G uses. Teams do not need to rewrite their deployment scripts, Terraform modules, or Kubernetes manifests. Drop the new binary in place of the old one. Everything continues to work.
The command-line interface mirrors WAL-G’s subcommands. Operations like wal-push, wal-fetch, backup-push, and backup-fetch behave identically. Existing monitoring dashboards, alerting rules, and log parsing pipelines continue to function without modification.
How does this work in practice? An organization with terabytes of archived WAL segments in S3 can adopt WAL-RUS immediately. New archives written by WAL-RUS coexist with old archives written by WAL-G. Restoration procedures work across both tools seamlessly. A base backup created with WAL-G can be restored using WAL-RUS, and WAL segments archived by either tool can be replayed during recovery.
This design decision was deliberate. ClickHouse understood that forcing a migration would deter adoption. Backup archives represent years of regulatory compliance and disaster recovery capability. No team wants to invalidate that history by switching tools.
How Does WAL-RUS Handle PostgreSQL WAL Archiving?
WAL-RUS hooks into PostgreSQL’s native continuous archiving mechanism through the archive_command configuration parameter, exactly as WAL-G does. When PostgreSQL completes writing a WAL segment — typically a 16 MB file — it invokes the configured archive_command to ship that segment to permanent storage. WAL-RUS receives the segment path, compresses it, encrypts it if configured, and uploads it to the designated cloud storage backend.
The archiving pipeline in WAL-RUS follows a streaming design. Data flows from disk through compression into network upload without materializing large intermediate buffers in memory. This streaming approach is central to the memory savings. WAL-G, by contrast, tends to buffer more aggressively due to Go’s memory model and the libraries it depends on.
WAL-RUS supports multiple compression algorithms including LZ4, Zstandard, and gzip. Encryption uses AES-256-GCM for segments uploaded to cloud storage. Supported backends include Amazon S3, Google Cloud Storage, Azure Blob Storage, and S3-compatible object stores like MinIO.
What happens when an archive upload fails? WAL-RUS retries with exponential backoff, following the same pattern as WAL-G. PostgreSQL’s archiver will keep invoking archive_command until it returns a success exit code. This ensures no WAL segment is lost during transient network failures or cloud storage outages. The tool also implements connection pooling and keep-alive optimizations to reduce latency when archiving to remote endpoints — a practical concern for PostgreSQL instances generating high WAL volumes.
What Backup Formats and Storage Backends Does WAL-RUS Support?
WAL-RUS maintains full compatibility with WAL-G’s storage layer, supporting Amazon S3, Google Cloud Storage, Azure Blob Storage, and SFTP destinations. The Rust implementation preserves the same archive format, meaning backup files produced by WAL-RUS are structurally identical to those from WAL-G. This architectural decision eliminates vendor lock-in concerns during migration.
The tool handles both base backups and continuous WAL archiving. Base backups use PostgreSQL’s native pg_start_backup and pg_stop_backup mechanisms, compressing the output with LZ4, Zstandard, or gzip depending on configuration. WAL segments are streamed incrementally. Nothing changes there.
According to the ClickHouse engineering blog, WAL-RUS implements its own compression and upload pipeline rather than wrapping external tools. The Rust codebase uses rust-s3 for Amazon-compatible endpoints and azure-sdk-for-rust for Microsoft’s Blob Storage. This native approach avoids the Go runtime overhead that contributed to WAL-G’s memory profile.
Storage backend configuration uses the same environment variable scheme as WAL-G, so existing infrastructure-as-code templates work without modification. Teams can point WAL-RUS at an existing backup bucket and resume operations immediately. The compatibility story is strong.
One difference: WAL-RUS introduces parallel upload support for large base backups by default, splitting files above 100 MB into concurrent chunks. WAL-G requires manual tuning for this behavior.
How Does WAL-RUS Perform Under Production Load?
ClickHouse reports that WAL-RUS reduces virtual memory usage by over 70% compared to WAL-G during backup operations. This figure comes from internal benchmarks run against multi-terabyte PostgreSQL clusters. The Rust binary’s memory footprint stays flat regardless of database size.
The performance improvement stems from Rust’s ownership model and absence of garbage collection pauses. WAL-G, written in Go, periodically pauses for GC sweeps during intensive I/O operations. These pauses can cause backup window spikes. WAL-RUS avoids this entirely.
ClickHouse has deployed WAL-RUS across its own PostgreSQL infrastructure, which manages metadata for its cloud offering. The tool processes continuous WAL archiving with sub-second latency under normal load. During base backup operations on clusters exceeding 2 TB, WAL-RUS maintains consistent throughput without the memory spikes that occasionally triggered OOM kills in WAL-G.
The engineering team noted that compile-time memory safety guarantees in Rust eliminated entire classes of bugs present in WAL-G’s Go codebase. Race conditions in concurrent upload logic, which had caused intermittent backup corruption in WAL-G, are structurally impossible in WAL-RUS due to the borrow checker.
CPU usage remains comparable between the two tools. The win is memory, not processor cycles.
Will ClickHouse Offer WAL-RUS as a Managed Service?
Yes. ClickHouse has explicitly signaled plans to integrate WAL-RUS into a future managed PostgreSQL backup offering. A LinkedIn post from the company highlighted the open-sourcing of WAL-RUS as foundational to this strategy.
The motivation is strategic. ClickHouse already operates large-scale PostgreSQL deployments for internal services. By open-sourcing WAL-RUS, the company invites community testing and contributions before wrapping it into a commercial product. This approach mirrors how Cloudflare, Discord, and others have open-sourced infrastructure tools before productizing them.
According to TipRanks coverage, ClickHouse is positioning WAL-RUS as part of a broader PostgreSQL tooling ecosystem. The managed offering would likely bundle backup management, monitoring, and recovery automation into a single dashboard. No pricing or release date has been announced.
For teams evaluating WAL-RUS today, the open-source license means no vendor lock-in. Even if ClickHouse never ships the managed service, the tool remains self-hostable and community-maintained. The repository is publicly accessible.
This matters for adoption. Organizations hesitant to depend on a single vendor’s roadmap can adopt WAL-RUS now and evaluate the managed product later.
How Can Teams Migrate From WAL-G to WAL-RUS?
Migration requires three steps: install the WAL-RUS binary, update environment variables if needed, and restart the PostgreSQL archive process. No backup conversion is necessary because WAL-RUS reads and writes the same format as WAL-G.
The simplest path involves replacing the wal-g binary with wal-rus in the PostgreSQL archive_command configuration. Existing environment variables — WALE_S3_PREFIX, AWS_ACCESS_KEY_ID, PGDATA, and others — work identically. Teams should verify compatibility in a staging environment first.
WAL-RUS can restore backups created by WAL-G without any intermediate conversion step. The tool reads the same manifest format and decompresses using the same algorithms. This backward compatibility means teams can switch tools mid-stream without invalidating their backup history.
The migration checklist:
- Install the WAL-RUS binary from the official GitHub releases page
- Verify that all current environment variables are recognized
- Run a test base backup and confirm it appears in your storage bucket
- Perform a test restore from a previous WAL-G backup to confirm read compatibility
- Update your
archive_commandinpostgresql.confto referencewal-rus - Restart PostgreSQL or reload configuration to activate the new archive process
- Monitor the first 24 hours of WAL archiving for any anomalies
- Schedule a full disaster recovery drill within the first week
The entire migration typically takes under an hour for a standard PostgreSQL deployment.
What Are the Limitations of WAL-RUS Today?
WAL-RUS is a young project. The initial release focuses on PostgreSQL backup and WAL archiving, omitting some advanced features found in WAL-G’s mature codebase. Point-in-time recovery configuration, while functional, lacks the helper scripts that WAL-G provides.
The tool currently supports PostgreSQL 13 and later. Earlier versions may work but are not officially tested. Teams running legacy PostgreSQL installations should verify compatibility before committing.
Documentation is still developing. The GitHub repository contains a README with configuration examples, but the depth of WAL-G’s wiki has not yet been matched. Community contributions are expanding this.
Another limitation: WAL-RUS does not yet support backup encryption with customer-managed keys through a dedicated interface. Encryption relies on the storage provider’s server-side encryption (SSE-S3, SSE-KMS). WAL-G’s client-side pgcrypto integration is not yet ported.
| Feature | WAL-G | WAL-RUS |
|---|---|---|
| Language | Go | Rust |
| Memory usage | Baseline | ~70% lower |
| Backup format compatibility | Native | Full backward compat |
| Parallel uploads | Manual config | Enabled by default |
| Point-in-time recovery scripts | Included | Minimal |
| Client-side encryption | Supported | Not yet |
| PostgreSQL version support | 9.6+ | 13+ |
| Documentation depth | Extensive wiki | README + examples |
The roadmap includes porting missing features, but ClickHouse has not published specific timelines.
Frequently Asked Questions
Does WAL-RUS support all cloud storage providers that WAL-G supports?
WAL-RUS currently supports Amazon S3, Google Cloud Storage, Azure Blob Storage, and SFTP — matching WAL-G’s primary backends. According to the ClickHouse blog, the Rust implementation uses native SDKs for each provider rather than a unified abstraction layer. Some less common WAL-G backends, such as local filesystem with custom compression, may not yet have equivalents in WAL-RUS.
Can WAL-RUS restore backups created by WAL-G without conversion?
Yes. WAL-RUS reads the exact same backup format as WAL-G, including compression codecs and manifest structure. The ClickHouse engineering team designed WAL-RUS for bidirectional compatibility, meaning backups created by either tool can be restored by the other. No conversion step is required.
What PostgreSQL versions does WAL-RUS support?
WAL-RUS officially supports PostgreSQL 13 and later, covering the versions most commonly deployed in production today. The tool relies on PostgreSQL’s standard backup API (pg_start_backup / pg_stop_backup), which has been stable across major releases. Testing against PostgreSQL 17 is ongoing, with initial results confirming full compatibility.
Is WAL-RUS production-ready for enterprise deployments?
ClickHouse reports that WAL-RUS is already running in its own production PostgreSQL infrastructure, managing backups for multi-terabyte clusters. The virtual memory reduction of over 70% compared to WAL-G has been validated under real workloads. However, the project remains young, and organizations adopting it should conduct independent disaster recovery testing before full rollout.
Summary
WAL-RUS represents a focused, practical rewrite of a critical infrastructure tool. The key takeaways:
- Memory efficiency: Rust’s ownership model delivers a 70%+ reduction in virtual memory usage compared to WAL-G, eliminating OOM-related backup failures on memory-constrained instances
- Drop-in compatibility: WAL-RUS reads and writes WAL-G’s backup format, enabling zero-friction migration without invalidating existing backup archives
- Production-validated: ClickHouse runs WAL-RUS on its own multi-terabyte PostgreSQL clusters, providing real-world evidence of reliability under sustained load
- Strategic roadmap: The open-source release positions WAL-RUS as the foundation for ClickHouse’s future managed PostgreSQL backup offering
- Current gaps: Client-side encryption, helper scripts for point-in-time recovery, and support for PostgreSQL versions below 13 remain on the to-do list
For teams running WAL-G today, WAL-RUS is worth evaluating as a memory-efficient alternative. The binary is available on GitHub. Test it in staging first.
If this breakdown was useful, check out the rest of gikiewicz.com for more deep dives into database tooling, Rust infrastructure projects, and production engineering decisions.