head	1.3;
access;
symbols
	REL9_0_0:1.3
	REL9_1_ALPHA1:1.3
	REL9_0_RC1:1.3
	REL9_0_BETA4:1.3
	REL9_0_STABLE:1.3.0.4
	REL9_0_BETA3:1.3
	REL9_0_BETA2:1.3
	REL9_0_BETA1:1.3
	REL9_0_ALPHA5_BRANCH:1.3.0.2
	REL9_0_ALPHA5:1.3
	REL9_0_ALPHA4:1.2
	REL9_0_ALPHA4_BRANCH:1.2.0.2;
locks; strict;
comment	@# @;


1.3
date	2010.03.24.06.25.39;	author heikki;	state Exp;
branches;
next	1.2;

1.2
date	2010.01.20.09.16.24;	author heikki;	state Exp;
branches;
next	1.1;

1.1
date	2010.01.15.09.19.03;	author heikki;	state Exp;
branches;
next	;


desc
@@


1.3
log
@Update description of walrcv_receive() function to match reality.
@
text
@$PostgreSQL: pgsql/src/backend/replication/README,v 1.2 2010/01/20 09:16:24 heikki Exp $

Walreceiver - libpqwalreceiver API
----------------------------------

The transport-specific part of walreceiver, responsible for connecting to
the primary server and receiving WAL files, is loaded dynamically to avoid
having to link the main server binary with libpq. The dynamically loaded
module is in libpqwalreceiver subdirectory.

The dynamically loaded module implements three functions:


bool walrcv_connect(char *conninfo, XLogRecPtr startpoint)

Establish connection to the primary, and starts streaming from 'startpoint'.
Returns true on success.


bool walrcv_receive(int timeout, unsigned char *type, char **buffer, int *len)

Retrieve any message available through the connection, blocking for
maximum of 'timeout' ms. If a message was successfully read, returns true,
otherwise false. On success, a pointer to the message payload is stored in
*buffer, length in *len, and the type of message received in *type. The
returned buffer is valid until the next call to walrcv_* functions, the
caller should not attempt freeing it.

void walrcv_disconnect(void);

Disconnect.


This API should be considered internal at the moment, but we could open it
up for 3rd party replacements of libpqwalreceiver in the future, allowing
pluggable methods for receiveing WAL.

Walreceiver IPC
---------------

When the WAL replay in startup process has reached the end of archived WAL,
recoverable using recovery_command, it starts up the walreceiver process
to fetch more WAL (if streaming replication is configured).

Walreceiver is a postmaster subprocess, so the startup process can't fork it
directly. Instead, it sends a signal to postmaster, asking postmaster to launch
it. Before that, however, startup process fills in WalRcvData->conninfo,
and initializes the starting point in WalRcvData->receivedUpTo.

As walreceiver receives WAL from the master server, and writes and flushes
it to disk (in pg_xlog), it updates WalRcvData->receivedUpTo. Startup process
polls that to know how far it can proceed with WAL replay.

Walsender IPC
-------------

At shutdown, postmaster handles walsender processes differently from regular
backends. It waits for regular backends to die before writing the
shutdown checkpoint and terminating pgarch and other auxiliary processes, but
that's not desirable for walsenders, because we want the standby servers to
receive all the WAL, including the shutdown checkpoint, before the master
is shut down. Therefore postmaster treats walsenders like the pgarch process,
and instructs them to terminate at PM_SHUTDOWN_2 phase, after all regular
backends have died and bgwriter has written the shutdown checkpoint.

When postmaster accepts a connection, it immediately forks a new process
to handle the handshake and authentication, and the process initializes to
become a backend. Postmaster doesn't know if the process becomes a regular
backend or a walsender process at that time - that's indicated in the
connection handshake - so we need some extra signaling to let postmaster
identify walsender processes.

When walsender process starts up, it marks itself as a walsender process in
the PMSignal array. That way postmaster can tell it apart from regular
backends.

Note that no big harm is done if postmaster thinks that a walsender is a
regular backend; it will just terminate the walsender earlier in the shutdown
phase. A walsenders will look like a regular backends until it's done with the
initialization and has marked itself in PMSignal array, and at process
termination, after unmarking the PMSignal slot.

Each walsender allocates an entry from the WalSndCtl array, and advertises
there how far it has streamed WAL already. This is used at checkpoints, to
avoid recycling WAL that hasn't been streamed to a slave yet. However,
that doesn't stop such WAL from being recycled when the connection is not
established.


Walsender - walreceiver protocol
--------------------------------

See manual.
@


1.2
log
@Rethink the way walreceiver is linked into the backend.	Instead than shoving
walreceiver as whole into a dynamically loaded module, split the
libpq-specific parts of it into dynamically loaded module and keep the rest
in the main backend binary.

Although Tom fixed the Windows compilation problems with the old walreceiver
module already, this is a cleaner division of labour and makes the code
more readable. There's also the prospect of adding new transport methods
as pluggable modules in the future, which this patch makes easier, though for
now the API between libpqwalreceiver and walreceiver process should be
considered private.

The libpq-specific module is now in src/backend/replication/libpqwalreceiver,
and the part linked with postgres binary is in
src/backend/replication/walreceiver.c.
@
text
@d1 1
a1 1
$PostgreSQL: pgsql/src/backend/replication/README,v 1.1 2010/01/15 09:19:03 heikki Exp $
d20 1
a20 4
bool walrcv_receive(int timeout, XLogRecPtr *recptr, char **buffer, int *len)

Retrieve any WAL record available through the connection, blocking for
maximum of 'timeout' ms.
d22 6
@


1.1
log
@Introduce Streaming Replication.

This includes two new kinds of postmaster processes, walsenders and
walreceiver. Walreceiver is responsible for connecting to the primary server
and streaming WAL to disk, while walsender runs in the primary server and
streams WAL from disk to the client.

Documentation still needs work, but the basics are there. We will probably
pull the replication section to a new chapter later on, as well as the
sections describing file-based replication. But let's do that as a separate
patch, so that it's easier to see what has been added/changed. This patch
also adds a new section to the chapter about FE/BE protocol, documenting the
protocol used by walsender/walreceivxer.

Bump catalog version because of two new functions,
pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for
monitoring the progress of replication.

Fujii Masao, with additional hacking by me
@
text
@d1 33
a33 1
$PostgreSQL$
@

