pglogrepl

pglogrepl gives Go programs direct access to PostgreSQL’s replication protocol, letting them act as a replication client without shelling out to pg_recvlogical or embedding a full CDC engine. It wraps the low-level command set — IDENTIFY_SYSTEM, CREATE_REPLICATION_SLOT, START_REPLICATION, BASE_BACKUP — in typed Go functions and structs, and decodes the binary messages a server streams back over the replication connection (keepalives, XLogData, and the full set of logical decoding messages) into usable Go values.

It’s built directly on top of github.com/jackc/pgx/v5’s pgconn and pgproto3 packages, so it doesn’t open its own connection — it hands a *pgconn.PgConn that’s already been put into replication mode. That makes it a thin, composable layer rather than a standalone client: teams building change-data-capture pipelines, WAL-based backup tooling, or custom logical decoding output plugins use it as the piece that talks the wire protocol, while the surrounding application owns connection setup, retries, and business logic.

Both the original (V1) and streaming-transaction (V2) logical decoding message formats are supported, along with base backup helpers covering incremental backups and manifest upload introduced in newer PostgreSQL versions. The project is tested against a live logical-replication-enabled PostgreSQL instance across Postgres 13 through 18 in CI, which matters for a library whose entire value is faithfully following a binary protocol that changes across server versions.

What You Get

• Typed wrappers for the replication command set: IDENTIFY_SYSTEM, TIMELINE_HISTORY, CREATE_REPLICATION_SLOT, DROP_REPLICATION_SLOT, and START_REPLICATION
• Parsers for the messages a replication stream sends back: PrimaryKeepaliveMessage and XLogData, including LSN and timestamp decoding
• Full logical decoding message support for both the V1 protocol (Begin/Commit/Relation/Insert/Update/Delete/Truncate) and the V2 streaming-transaction protocol (StreamStart/StreamStop/StreamCommit/StreamAbort)
• A LSN type implementing sql.Scanner and driver.Valuer so log sequence numbers can round-trip through database/sql directly
• BASE_BACKUP support, including PostgreSQL 17’s incremental backups and manifest upload/checksum options
• Two runnable example programs (pglogrepl_demo, pgphysrepl_demo) showing a full logical and physical replication client end to end

Common Use Cases

• Building a custom change-data-capture (CDC) pipeline that reads a PostgreSQL logical replication slot and forwards row changes to a message queue or downstream store
• Writing WAL-based backup or point-in-time-recovery tooling that needs to drive BASE_BACKUP and follow a physical replication stream
• Implementing a custom logical decoding output plugin consumer for a specific downstream format instead of using wal2json or pgoutput indirectly
• Building monitoring or replication-lag tooling that needs to parse keepalive and XLogData messages directly from a replication connection

Under The Hood

Architecture The package is organized as a flat set of files at the repository root (pglogrepl.go, message.go, messageV2.go) built directly on github.com/jackc/pgx/v5’s pgconn and pgproto3 packages rather than opening its own connections. Each replication command follows a consistent exec/parse split — for example CreateReplicationSlot calls conn.Exec and hands the result to a standalone ParseCreateReplicationSlot, which lets the wire-format parsing be tested independently of a live server. Message decoding uses a small polymorphic Message/MessageDecoder interface with a shared baseMessage embedding a msgType field, dispatched through type switches in Parse/ParseV2, which keeps the original (V1) and streaming (V2) logical decoding formats cleanly separated between message.go and messageV2.go. The one abstraction everything else depends on is the LSN/byte-order decoding logic, since StartReplication, SendStandbyStatusUpdate, and the base-backup helpers all rely on it being correct.

Tech Stack The module targets Go 1.25 and leans on the standard library (encoding/binary, database/sql/driver, context, strconv, time) for its core logic, with github.com/jackc/pgx/v5 (pgconn, pgproto3) as the only non-test dependency alongside github.com/jackc/pgio for binary encoding helpers. Test-only dependencies are github.com/stretchr/testify plus its transitive packages. There’s no ORM or web framework involved, since this operates at the wire-protocol layer beneath them; CI runs a GitHub Actions matrix across Go 1.25/1.26 and PostgreSQL 13 through 18, standing up a real Postgres instance via docker-compose with wal_level=logical for genuine integration coverage rather than mocks.

Code Quality All three source files have matching test files with a combined 32 test functions, using testify’s assert/require for assertions and, via the PGLOGREPL_TEST_CONN_STRING environment variable, real integration tests against a live logical-replication-enabled PostgreSQL server. Errors are handled explicitly and wrapped with fmt.Errorf("...: %w", err) throughout with nothing silently swallowed, naming follows standard Go conventions, and the custom LSN type implementing sql.Scanner/driver.Valuer adds meaningful type safety around SQL interop. CI enforces go test -race across the full Go/Postgres version matrix, giving real regression coverage on every change; the one gap is that no dedicated linter configuration is checked into the repository.

What Makes It Unique pglogrepl doesn’t introduce a new protocol — it’s a careful, complete Go implementation of PostgreSQL’s own documented replication wire protocol, covering IDENTIFY_SYSTEM, CREATE_REPLICATION_SLOT, START_REPLICATION, and BASE_BACKUP alongside XLogData and keepalive message parsing. Its most distinguishing feature is supporting both the original and the newer streaming-transaction (V2) logical decoding message formats in the same library, plus base-backup features like incremental backups and manifest upload tied to newer PostgreSQL releases — coverage broader than most alternative Go replication clients, even though the underlying approach is a direct mapping of Postgres’s protocol documentation rather than a novel technique.