An example output plugin can be found in the
contrib/test_decoding
subdirectory of the PostgreSQL source tree.
An output plugin is loaded by dynamically loading a shared library with
the output plugin's name as the library base name. The normal library
search path is used to locate the library. To provide the required output
plugin callbacks and to indicate that the library is actually an output
plugin it needs to provide a function named
_PG_output_plugin_init
. This function is passed a
struct that needs to be filled with the callback function pointers for
individual actions.
typedef struct OutputPluginCallbacks { LogicalDecodeStartupCB startup_cb; LogicalDecodeBeginCB begin_cb; LogicalDecodeChangeCB change_cb; LogicalDecodeCommitCB commit_cb; LogicalDecodeShutdownCB shutdown_cb; } OutputPluginCallbacks; typedef void (*LogicalOutputPluginInit)(struct OutputPluginCallbacks *cb);
The begin_cb
, change_cb
and commit_cb
callbacks are required,
while startup_cb
and shutdown_cb
are optional.
To decode, format and output changes, output plugins can use most of the
backend's normal infrastructure, including calling output functions. Read
only access to relations is permitted as long as only relations are
accessed that either have been created by initdb in
the pg_catalog
schema, or have been marked as user
provided catalog tables using
ALTER TABLE user_catalog_table SET (user_catalog_table = true); CREATE TABLE another_catalog_table(data text) WITH (user_catalog_table = true);
Any actions leading to transaction ID assignment are prohibited. That, among others,
includes writing to tables, performing DDL changes, and
calling txid_current()
.
Output plugin callbacks can pass data to the consumer in nearly arbitrary
formats. For some use cases, like viewing the changes via SQL, returning
data in a data type that can contain arbitrary data (e.g., bytea) is
cumbersome. If the output plugin only outputs textual data in the
server's encoding, it can declare that by
setting OutputPluginOptions.output_mode
to OUTPUT_PLUGIN_TEXTUAL_OUTPUT
instead
of OUTPUT_PLUGIN_BINARY_OUTPUT
in
the startup
callback. In that case, all the data has to be in the server's encoding
so that a text datum can contain it. This is checked in assertion-enabled
builds.
An output plugin gets notified about changes that are happening via various callbacks it needs to provide.
Concurrent transactions are decoded in commit order, and only changes
belonging to a specific transaction are decoded between
the begin
and commit
callbacks. Transactions that were rolled back explicitly or implicitly
never get
decoded. Successful savepoints are
folded into the transaction containing them in the order they were
executed within that transaction.
Only transactions that have already safely been flushed to disk will be
decoded. That can lead to a COMMIT not immediately being decoded in a
directly following pg_logical_slot_get_changes()
when synchronous_commit
is set
to off
.
The optional startup_cb
callback is called whenever
a replication slot is created or asked to stream changes, independent
of the number of changes that are ready to be put out.
typedef void (*LogicalDecodeStartupCB) ( struct LogicalDecodingContext *ctx, OutputPluginOptions *options, bool is_init );
The is_init
parameter will be true when the
replication slot is being created and false
otherwise. options
points to a struct of options
that output plugins can set:
typedef struct OutputPluginOptions { OutputPluginOutputType output_type; } OutputPluginOptions;
output_type
has to either be set to
OUTPUT_PLUGIN_TEXTUAL_OUTPUT
or OUTPUT_PLUGIN_BINARY_OUTPUT
. See also
the section called “Output Modes”>.
The startup callback should validate the options present in
ctx->output_plugin_options
. If the output plugin
needs to have a state, it can
use ctx->output_plugin_private
to store it.
The optional shutdown_cb
callback is called
whenever a formerly active replication slot is not used anymore and can
be used to deallocate resources private to the output plugin. The slot
isn't necessarily being dropped, streaming is just being stopped.
typedef void (*LogicalDecodeShutdownCB) ( struct LogicalDecodingContext *ctx );
The required begin_cb
callback is called whenever a
start of a committed transaction has been decoded. Aborted transactions
and their contents never get decoded.
typedef void (*LogicalDecodeBeginCB) ( struct LogicalDecodingContext *, ReorderBufferTXN *txn );
The txn
parameter contains meta information about
the transaction, like the time stamp at which it has been committed and
its XID.
The required commit_cb
callback is called whenever
a transaction commit has been
decoded. The change_cb
callbacks for all modified
rows will have been called before this, if there have been any modified
rows.
typedef void (*LogicalDecodeCommitCB) ( struct LogicalDecodingContext *, ReorderBufferTXN *txn );
The required change_cb
callback is called for every
individual row modification inside a transaction, may it be
an INSERT, UPDATE,
or DELETE. Even if the original command modified
several rows at once the callback will be called individually for each
row.
typedef void (*LogicalDecodeChangeCB) ( struct LogicalDecodingContext *ctx, ReorderBufferTXN *txn, Relation relation, ReorderBufferChange *change );
The ctx
and txn
parameters
have the same contents as for the begin_cb
and commit_cb
callbacks, but additionally the
relation descriptor relation
points to the
relation the row belongs to and a struct
change
describing the row modification are passed
in.
Only changes in user defined tables that are not unlogged
(see UNLOGGED
) and not temporary
(see TEMPORARY
or TEMP
) can be extracted using
logical decoding.
To actually produce output, output plugins can write data to
the StringInfo
output buffer
in ctx->out
when inside
the begin_cb
, commit_cb
,
or change_cb
callbacks. Before writing to the output
buffer, OutputPluginPrepareWrite(ctx, last_write)
has
to be called, and after finishing writing to the
buffer, OutputPluginWrite(ctx, last_write)
has to be
called to perform the write. The last_write
indicates whether a particular write was the callback's last write.
The following example shows how to output data to the consumer of an output plugin:
OutputPluginPrepareWrite(ctx, true); appendStringInfo(ctx->out, "BEGIN %u", txn->xid); OutputPluginWrite(ctx, true);