head	1.7;
access;
symbols
	REL7_4_29:1.6
	REL7_4_28:1.6
	REL7_4_27:1.6
	REL7_4_26:1.6
	REL7_4_25:1.6
	REL7_4_24:1.6
	REL7_4_23:1.6
	REL7_4_22:1.6
	REL7_4_21:1.6
	REL7_4_20:1.6
	REL7_3_21:1.6
	REL7_4_19:1.6
	REL7_3_20:1.6
	REL7_4_18:1.6
	REL7_3_19:1.6
	REL7_4_17:1.6
	REL7_3_18:1.6
	REL7_4_16:1.6
	REL7_4_15:1.6
	REL7_3_17:1.6
	REL7_4_14:1.6
	REL7_3_16:1.6
	REL7_3_15:1.6
	REL7_4_13:1.6
	REL7_3_14:1.6
	REL7_4_12:1.6
	REL7_3_13:1.6
	REL7_4_11:1.6
	REL7_3_12:1.6
	REL7_4_10:1.6
	REL7_3_11:1.6
	REL7_4_9:1.6
	REL7_2_8:1.5
	REL7_3_10:1.6
	REL7_4_8:1.6
	REL7_2_7:1.5
	REL7_3_9:1.6
	REL7_4_7:1.6
	REL7_4_6:1.6
	REL7_3_8:1.6
	REL7_2_6:1.5
	REL7_2_5:1.5
	REL7_4_5:1.6
	REL7_3_7:1.6
	REL7_4_4:1.6
	REL8_0_0BETA1:1.6
	REL7_4_3:1.6
	REL7_4_2:1.6
	REL7_3_6:1.6
	REL7_4_1:1.6
	REL7_3_5:1.6
	REL7_4:1.6
	REL7_4_RC2:1.6
	REL7_4_STABLE:1.6.0.6
	REL7_4_RC1:1.6
	REL7_4_BETA5:1.6
	REL7_4_BETA4:1.6
	REL7_4_BETA3:1.6
	REL7_4_BETA2:1.6
	WIN32_DEV:1.6.0.4
	REL7_4_BETA1:1.6
	REL7_3_4:1.6
	REL7_3_2:1.6
	REL7_2_4:1.5
	REL7_3_STABLE:1.6.0.2
	REL7_2_3:1.5
	REL7_2_STABLE:1.5.0.2
	REL7_2:1.5
	REL7_2_RC2:1.5
	REL7_2_RC1:1.3
	REL7_2_BETA5:1.3
	REL7_2_BETA4:1.3
	REL7_2_BETA3:1.3
	REL7_2_BETA2:1.3
	REL7_2_BETA1:1.3
	REL7_1_2:1.2
	REL7_1_STABLE:1.2.0.2
	REL7_1:1.2;
locks; strict;
comment	@# @;


1.7
date	2004.08.17.17.11.44;	author momjian;	state dead;
branches;
next	1.6;

1.6
date	2002.02.20.00.44.30;	author momjian;	state Exp;
branches;
next	1.5;

1.5
date	2002.01.23.19.22.08;	author momjian;	state Exp;
branches;
next	1.4;

1.4
date	2002.01.23.18.40.22;	author momjian;	state Exp;
branches;
next	1.3;

1.3
date	2001.09.06.20.41.30;	author momjian;	state Exp;
branches;
next	1.2;

1.2
date	2001.01.20.18.55.35;	author momjian;	state Exp;
branches;
next	1.1;

1.1
date	2001.01.20.05.00.03;	author momjian;	state Exp;
branches;
next	;


desc
@@


1.7
log
@Remove transactions TODO.detail.
@
text
@From pgsql-hackers-owner+M11649@@postgresql.org Wed Aug  1 15:22:46 2001
Return-path: <pgsql-hackers-owner+M11649@@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f71JMjN09768
	for <pgman@@candle.pha.pa.us>; Wed, 1 Aug 2001 15:22:45 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.1) with SMTP id f71JMUf62338;
	Wed, 1 Aug 2001 15:22:30 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M11649@@postgresql.org)
Received: from sectorbase2.sectorbase.com (sectorbase2.sectorbase.com [63.88.121.62] (may be forged))
	by postgresql.org (8.11.3/8.11.1) with SMTP id f71J4df57086
	for <pgsql-hackers@@postgresql.org>; Wed, 1 Aug 2001 15:04:40 -0400 (EDT)
	(envelope-from vmikheev@@SECTORBASE.COM)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2653.19)
	id <PG1LSSPZ>; Wed, 1 Aug 2001 12:04:31 -0700
Message-ID: <3705826352029646A3E91C53F7189E32016705@@sectorbase2.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
To: "'pgsql-hackers@@postgresql.org'" <pgsql-hackers@@postgresql.org>
Subject: [HACKERS] Using POSIX mutex-es
Date: Wed, 1 Aug 2001 12:04:24 -0700 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="koi8-r"
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: RO

1. Just changed
	TAS(lock) to pthread_mutex_trylock(lock)
	S_LOCK(lock) to pthread_mutex_lock(lock)
	S_UNLOCK(lock) to pthread_mutex_unlock(lock)
(and S_INIT_LOCK to share mutex-es between processes).

2. pgbench was initialized with scale 10.
   SUN WS 10 (512Mb), Solaris 2.6 (I'm unable to test on E4500 -:()
   -B 16384, wal_files 8, wal_buffers 256,
   checkpoint_segments 64, checkpoint_timeout 3600
   50 clients x 100 transactions
   (after initialization DB dir was saved and before each test
    copyed back and vacuum-ed).

3. No difference.
   Mutex version maybe 0.5-1 % faster (eg: 37.264238 tps vs 37.083339 tps).

So - no gain, but no performance loss "from using pthread library"
(I've also run tests with 1 client), at least on Solaris.

And so - looks like we can use POSIX mutex-es and conditional variables
(not semaphores; man pthread_cond_wait) and should implement light lmgr,
probably with priority locking.

Vadim

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@@postgresql.org)

From pgsql-hackers-owner+M18052=candle.pha.pa.us=pgman@@postgresql.org Wed Jan 23 13:39:19 2002
Return-path: <pgsql-hackers-owner+M18052=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0NIdIU26480
	for <pgman@@candle.pha.pa.us>; Wed, 23 Jan 2002 13:39:18 -0500 (EST)
Received: (qmail 59371 invoked by alias); 23 Jan 2002 18:39:18 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 23 Jan 2002 18:39:18 -0000
Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g0NIJ8l47400
	for <pgsql-hackers@@postgreSQL.org>; Wed, 23 Jan 2002 13:19:08 -0500 (EST)
	(envelope-from pgman@@candle.pha.pa.us)
Received: (from pgman@@localhost)
	by candle.pha.pa.us (8.11.6/8.10.1) id g0NIJ5i24508
	for pgsql-hackers@@postgreSQL.org; Wed, 23 Jan 2002 13:19:05 -0500 (EST)
From: Bruce Momjian <pgman@@candle.pha.pa.us>
Message-ID: <200201231819.g0NIJ5i24508@@candle.pha.pa.us>
Subject: [HACKERS] Savepoints
To: PostgreSQL-development <pgsql-hackers@@postgresql.org>
Date: Wed, 23 Jan 2002 13:19:05 -0500 (EST)
X-Mailer: ELM [version 2.4ME+ PL96 (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: RO

I have talked in the past about a possible implementation of
savepoints/nested transactions.  I would like to more formally outline
my ideas below.

We have talked about using WAL for such a purpose, but that requires WAL
files to remain for the life of a transaction, which seems unacceptable.
Other database systems do that, and it is a pain for administrators.  I
realized we could do some sort of WAL compaction, but that seems quite
complex too.

Basically, under my plan, WAL would be unchanged.  WAL's function is
crash recovery, and it would retain that.  There would also be no
on-disk changes.  I would use the command counter in certain cases to
identify savepoints.

My idea is to keep savepoint undo information in a private area per
backend, either in memory or on disk.  We can either save the
relid/tids of modified rows, or if there are too many, discard the
saved ones and just remember the modified relids.  On rollback to save
point, either clear up the modified relid/tids, or sequential scan
through the relid and clear up all the tuples that have our transaction
id and have command counters that are part of the undo savepoint.

It seems marking undo savepoint rows with a fixed aborted transaction id
would be the easiest solution.

Of course, we only remember modified rows when we are in savepoints, and
only undo them when we rollback to a savepoint.  Transaction processing
remains the same.

There is no reason for other backend to be able to see savepoint undo
information, and keeping it private greatly simplifies the
implementation.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

From hstenger@@adinet.com.uy Wed Jan 23 14:13:33 2002
Return-path: <hstenger@@adinet.com.uy>
Received: from correo.adinet.com.uy (fecorreo01.adinet.com.uy [206.99.44.217])
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g0NJDWU29832
	for <pgman@@candle.pha.pa.us>; Wed, 23 Jan 2002 14:13:33 -0500 (EST)
Received: from adinet.com.uy (200.61.76.155) by correo.adinet.com.uy (5.5.052) (authenticated as hstenger@@adinet.com.uy)
        id 3C4DBC5C00017E9F; Wed, 23 Jan 2002 16:13:25 -0300
Message-ID: <3C4F0BC0.5CFBB919@@adinet.com.uy>
Date: Wed, 23 Jan 2002 16:15:12 -0300
From: Haroldo Stenger <hstenger@@adinet.com.uy>
X-Mailer: Mozilla 4.78 [en] (Win98; U)
X-Accept-Language: en
MIME-Version: 1.0
To: Bruce Momjian <pgman@@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@@postgreSQL.org>
Subject: Re: [HACKERS] Savepoints
References: <200201231819.g0NIJ5i24508@@candle.pha.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Status: OR

Bruce Momjian wrote:
> 
> Basically, under my plan, WAL would be unchanged.  WAL's function is
> crash recovery, and it would retain that.  There would also be no
> on-disk changes.  I would use the command counter in certain cases to
> identify savepoints.

This is a pointer to the previous August thread, where your original proposal
was posted, and some WAL/not WAL discussion took place. Just not to repeat the
already mentioned points. Oh, it's google archive just for fun, and to not
overload hub.org ;-)

http://groups.google.com/groups?hl=en&threadm=200108050432.f754Wdo11696%40candle.pha.pa.us&rnum=1&prev=/groups%3Fhl%3Den%26selm%3D200108050432.f754Wdo11696%2540candle.pha.pa.us

Regards,
Haroldo.

From vmikheev@@SECTORBASE.COM Wed Jan 23 18:23:04 2002
Return-path: <vmikheev@@SECTORBASE.COM>
Received: from sectorbase2.sectorbase.com ([66.106.163.120])
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g0NNN3U21442
	for <pgman@@candle.pha.pa.us>; Wed, 23 Jan 2002 18:23:04 -0500 (EST)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2653.19)
	id <DKXVZ14S>; Wed, 23 Jan 2002 15:22:52 -0800
Message-ID: <3705826352029646A3E91C53F7189E32518483@@sectorbase2.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
To: "'Bruce Momjian'" <pgman@@candle.pha.pa.us>,
   PostgreSQL-development
  <pgsql-hackers@@postgreSQL.org>
Subject: RE: [HACKERS] Savepoints
Date: Wed, 23 Jan 2002 15:22:42 -0800
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Status: ORr

> I have talked in the past about a possible implementation of
> savepoints/nested transactions.  I would like to more formally outline
> my ideas below.

Well, I would like to do the same -:)

> ...
> There is no reason for other backend to be able to see savepoint undo
> information, and keeping it private greatly simplifies the
> implementation.

Yes... and requires additional memory/disk space: we keep old records
in data files and we'll store them again...

How about: use overwriting smgr + put old records into rollback
segments - RS - (you have to keep them somewhere till TX's running
anyway) + use WAL only as REDO log (RS will be used to rollback TX'
changes and WAL will be used for RS/data files recovery).
Something like what Oracle does.

Vadim

From pgsql-hackers-owner+M18085=candle.pha.pa.us=pgman@@postgresql.org Wed Jan 23 20:15:02 2002
Return-path: <pgsql-hackers-owner+M18085=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0O1F1U26461
	for <pgman@@candle.pha.pa.us>; Wed, 23 Jan 2002 20:15:02 -0500 (EST)
Received: (qmail 92866 invoked by alias); 24 Jan 2002 01:14:59 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 24 Jan 2002 01:14:59 -0000
Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g0O18ml91949
	for <pgsql-hackers@@postgresql.org>; Wed, 23 Jan 2002 20:08:50 -0500 (EST)
	(envelope-from pgman@@candle.pha.pa.us)
Received: (from pgman@@localhost)
	by candle.pha.pa.us (8.11.6/8.10.1) id g0O18jV26044;
	Wed, 23 Jan 2002 20:08:45 -0500 (EST)
From: Bruce Momjian <pgman@@candle.pha.pa.us>
Message-ID: <200201240108.g0O18jV26044@@candle.pha.pa.us>
Subject: Re: [HACKERS] Savepoints
In-Reply-To: <3705826352029646A3E91C53F7189E32518483@@sectorbase2.sectorbase.com>
To: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
Date: Wed, 23 Jan 2002 20:08:45 -0500 (EST)
cc: PostgreSQL-development <pgsql-hackers@@postgresql.org>
X-Mailer: ELM [version 2.4ME+ PL96 (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

Mikheev, Vadim wrote:
> > I have talked in the past about a possible implementation of
> > savepoints/nested transactions.  I would like to more formally outline
> > my ideas below.
> 
> Well, I would like to do the same -:)

Good.

> > ...
> > There is no reason for other backend to be able to see savepoint undo
> > information, and keeping it private greatly simplifies the
> > implementation.
> 
> Yes... and requires additional memory/disk space: we keep old records
> in data files and we'll store them again...

I was suggesting keeping only relid/tid or in some cases only relid. 
Seems like one or the other will fit all needs:  relid/tid for update of
a few rows, relid for many rows updated in the same table.  I saw no
need to store the actual data.

> How about: use overwriting smgr + put old records into rollback
> segments - RS - (you have to keep them somewhere till TX's running
> anyway) + use WAL only as REDO log (RS will be used to rollback TX'
> changes and WAL will be used for RS/data files recovery).
> Something like what Oracle does.

Why record the old data rows rather than the tids?  While the
transaction is running, the rows can't be moved anyway.  Also, why store
them in a shared area.  That has additional requirements because one old
transaction can require all transactions to keep their stuff around. 
Why not just make it a private data file for each backend?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@@postgresql.org

From pgsql-hackers-owner+M18086=candle.pha.pa.us=pgman@@postgresql.org Wed Jan 23 20:25:47 2002
Return-path: <pgsql-hackers-owner+M18086=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0O1PkU26964
	for <pgman@@candle.pha.pa.us>; Wed, 23 Jan 2002 20:25:47 -0500 (EST)
Received: (qmail 94878 invoked by alias); 24 Jan 2002 01:25:44 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 24 Jan 2002 01:25:44 -0000
Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g0O1L1l94075
	for <pgsql-hackers@@postgreSQL.org>; Wed, 23 Jan 2002 20:21:01 -0500 (EST)
	(envelope-from pgman@@candle.pha.pa.us)
Received: (from pgman@@localhost)
	by candle.pha.pa.us (8.11.6/8.10.1) id g0O1Kwm26748;
	Wed, 23 Jan 2002 20:20:58 -0500 (EST)
From: Bruce Momjian <pgman@@candle.pha.pa.us>
Message-ID: <200201240120.g0O1Kwm26748@@candle.pha.pa.us>
Subject: Re: [HACKERS] Savepoints
In-Reply-To: <3705826352029646A3E91C53F7189E32518483@@sectorbase2.sectorbase.com>
To: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
Date: Wed, 23 Jan 2002 20:20:58 -0500 (EST)
cc: PostgreSQL-development <pgsql-hackers@@postgresql.org>
X-Mailer: ELM [version 2.4ME+ PL96 (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

> > There is no reason for other backend to be able to see savepoint undo
> > information, and keeping it private greatly simplifies the
> > implementation.
> 
> Yes... and requires additional memory/disk space: we keep old records
> in data files and we'll store them again...
> 
> How about: use overwriting smgr + put old records into rollback
> segments - RS - (you have to keep them somewhere till TX's running
> anyway) + use WAL only as REDO log (RS will be used to rollback TX'
> changes and WAL will be used for RS/data files recovery).
> Something like what Oracle does.

I am sorry.  I see what you are saying now.  I missed the words
"overwriting smgr".  You are suggesting going to an overwriting storage
manager.  Is this to be done only because of savepoints.  Doesn't seem
worth it when I have a possible solution without such a drastic change.
Also, overwriting storage manager will require MVCC to read through
there to get accurate MVCC visibility, right?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@@postgresql.org

From vmikheev@@SECTORBASE.COM Wed Jan 23 21:03:29 2002
Return-path: <vmikheev@@SECTORBASE.COM>
Received: from sectorbase2.sectorbase.com ([66.106.163.120])
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g0O23TU28813
	for <pgman@@candle.pha.pa.us>; Wed, 23 Jan 2002 21:03:29 -0500 (EST)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2653.19)
	id <DKXVZFBY>; Wed, 23 Jan 2002 18:03:18 -0800
Message-ID: <3705826352029646A3E91C53F7189E32518487@@sectorbase2.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
To: "'Bruce Momjian'" <pgman@@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@@postgreSQL.org>
Subject: RE: [HACKERS] Savepoints
Date: Wed, 23 Jan 2002 18:03:11 -0800
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Status: ORr

> > How about: use overwriting smgr + put old records into rollback
> > segments - RS - (you have to keep them somewhere till TX's running
> > anyway) + use WAL only as REDO log (RS will be used to rollback TX'
> > changes and WAL will be used for RS/data files recovery).
> > Something like what Oracle does.
> 
> I am sorry. I see what you are saying now. I missed the words

And I'm sorry for missing your notes about storing relid+tid only.

> "overwriting smgr". You are suggesting going to an overwriting
> storage manager. Is this to be done only because of savepoints.

No. One point I made a few monthes ago (and never got objections)
is - why to keep old data in data files sooooo long?
Imagine long running TX (eg pg_dump). Why other TX-s must read
again and again completely useless (for them) old data we keep
for pg_dump?

> Doesn't seem worth it when I have a possible solution without
> such a drastic change.
> Also, overwriting storage manager will require MVCC to read
> through there to get accurate MVCC visibility, right?

Right... just like now non-overwriting smgr requires *ALL*
TX-s to read old data in data files. But with overwriting smgr
TX will read RS only when it is required and as far (much) as
it is required.

Simple solutions are not always the best ones.
Compare Oracle and InterBase. Both have MVCC.
Smgr-s are different. What RDBMS is more cool?
Why doesn't Oracle use more simple non-overwriting smgr
(as InterBase... and we do)?

Vadim

From dhogaza@@pacifier.com Wed Jan 23 21:05:37 2002
Return-path: <dhogaza@@pacifier.com>
Received: from comet.pacifier.com ([199.2.117.155])
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g0O25bU28962
	for <pgman@@candle.pha.pa.us>; Wed, 23 Jan 2002 21:05:37 -0500 (EST)
Received: from pacifier.com (dsl-dhogaza.pacifier.net [207.202.226.68])
	by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id g0O24qX29917;
	Wed, 23 Jan 2002 18:04:52 -0800 (PST)
Message-ID: <3C4F6BF0.2010406@@pacifier.com>
Date: Wed, 23 Jan 2002 18:05:36 -0800
From: Don Baccus <dhogaza@@pacifier.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.7) Gecko/20011221
X-Accept-Language: en-us
MIME-Version: 1.0
To: Bruce Momjian <pgman@@candle.pha.pa.us>
cc: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>,
   PostgreSQL-development <pgsql-hackers@@postgresql.org>
Subject: Re: [HACKERS] Savepoints
References: <200201240120.g0O1Kwm26748@@candle.pha.pa.us>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Status: OR

Bruce Momjian wrote:


> I am sorry.  I see what you are saying now.  I missed the words
> "overwriting smgr".  You are suggesting going to an overwriting storage
> manager.


Overwriting storage managers don't suffer from unbounded growth of 
datafiles until garbage collection (vacuum) is performed.  In fact, 
there's no need for a vacuum-style utility.  The rollback segments only 
need to keep around enough past history to rollback transactions that 
are executing.

Of course, then the size of your transactions are limited by the size of 
your rollback segments, which in Oracle are fixed in length when you 
build your database (there are ways to change this when you figure out 
that you didn't pick a good number when creating it).

 >Is this to be done only because of savepoints.

Not in traditional storage managers such as Oracle uses.  The complexity 
of managing visibility and the like are traded off against the fact that 
you're not stuck ever needing to garbage collect a database that 
occupies a roomful of disks.

It's a trade-off.  PG's current storage manager seems to work awfully 
well in a lot of common database scenarios, and Tom's new vacuum is 
meant to help mitigate against the drawbacks.   But overwriting storage 
managers certainly have their advantages, too.

 >  Doesn't seem

> worth it when I have a possible solution without such a drastic change.
> Also, overwriting storage manager will require MVCC to read through
> there to get accurate MVCC visibility, right?


Yep...

-- 
Don Baccus
Portland, OR
http://donb.photo.net, http://birdnotes.net, http://openacs.org


From Inoue@@tpf.co.jp Thu Jan 24 11:34:48 2002
Return-path: <Inoue@@tpf.co.jp>
Received: from p2272.nsk.ne.jp (p2272.nsk.ne.jp [210.145.18.145])
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g0OGYjU23980
	for <pgman@@candle.pha.pa.us>; Thu, 24 Jan 2002 11:34:47 -0500 (EST)
Received: from mcadnote1 (ppm132.noc.fukui.nsk.ne.jp [61.198.95.32])
	by p2272.nsk.ne.jp (8.9.3/3.7W-20000722) with SMTP id BAA12147;
	Fri, 25 Jan 2002 01:34:24 +0900 (JST)
From: "Hiroshi Inoue" <Inoue@@tpf.co.jp>
To: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
cc: "PostgreSQL-development" <pgsql-hackers@@postgreSQL.org>,
   "'Bruce Momjian'" <pgman@@candle.pha.pa.us>
Subject: RE: [HACKERS] Savepoints
Date: Fri, 25 Jan 2002 01:34:29 +0900
Message-ID: <EKEJJICOHDIEMGPNIFIJKEFBGJAA.Inoue@@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
In-Reply-To: <3705826352029646A3E91C53F7189E32518483@@sectorbase2.sectorbase.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200
Importance: Normal
Status: OR

> -----Original Message-----
> From: Mikheev, Vadim
> 
> How about: use overwriting smgr + put old records into rollback
> segments - RS - (you have to keep them somewhere till TX's running
> anyway) + use WAL only as REDO log (RS will be used to rollback TX'
> changes and WAL will be used for RS/data files recovery).
> Something like what Oracle does.

As long as we use no overwriting manager
1) Rollback(data) isn't needed in case of a db crash.
2) Rollback(data) isn't needed to cancal a transaction entirely.
3) We don't need to mind the transaction size so much.

We can't use the db any longer if a REDO recovery fails now.
Under overwriting smgr we can't use the db any longer either
if rollback fails. How could PG be not less reliable than now ?

regards,
Hiroshi Inoue

From pgsql-hackers-owner+M18123=candle.pha.pa.us=pgman@@postgresql.org Thu Jan 24 14:15:11 2002
Return-path: <pgsql-hackers-owner+M18123=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0OJFAU12547
	for <pgman@@candle.pha.pa.us>; Thu, 24 Jan 2002 14:15:10 -0500 (EST)
Received: (qmail 43413 invoked by alias); 24 Jan 2002 19:13:48 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 24 Jan 2002 19:13:48 -0000
Received: from sectorbase2.sectorbase.com ([66.106.163.120])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g0OJC4l42011
	for <pgsql-hackers@@postgreSQL.org>; Thu, 24 Jan 2002 14:12:04 -0500 (EST)
	(envelope-from vmikheev@@SECTORBASE.COM)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2653.19)
	id <DKXVZF9P>; Thu, 24 Jan 2002 11:11:54 -0800
Message-ID: <3705826352029646A3E91C53F7189E3251848B@@sectorbase2.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
To: "'Hiroshi Inoue'" <Inoue@@tpf.co.jp>
cc: PostgreSQL-development <pgsql-hackers@@postgresql.org>,
   "'Bruce Momjian'"
  <pgman@@candle.pha.pa.us>
Subject: Re: [HACKERS] Savepoints
Date: Thu, 24 Jan 2002 11:11:52 -0800
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

> > How about: use overwriting smgr + put old records into rollback
> > segments - RS - (you have to keep them somewhere till TX's running
> > anyway) + use WAL only as REDO log (RS will be used to rollback TX'
> > changes and WAL will be used for RS/data files recovery).
> > Something like what Oracle does.
> 
> As long as we use no overwriting manager
> 1) Rollback(data) isn't needed in case of a db crash.
> 2) Rollback(data) isn't needed to cancal a transaction entirely.

-1) But vacuum must read a huge amount of data to remove dirt.
-2) But TX-s must read data they are not interested at all.

> 3) We don't need to mind the transaction size so much.

-3) The same with overwriting smgr and WAL used *only as REDO log*:
we are not required to keep WAL files for duration of transaction
- as soon as server knows that changes logged in some WAL file
applied to data files and RS on disk (and archived, for WAL-based
BAR) that file may be reused/removed. Old data will still occupy
space in RS but their space in data files will be available
for reuse.

> We can't use the db any longer if a REDO recovery fails now.

Reset WAL and use/dump it. Annoying? Agreed. Fix bugs and/or
use good RAM - whatever caused problem with restart.

> Under overwriting smgr we can't use the db any longer either
> if rollback fails.

Why should it fail? Bugs? Fix them.

> How could PG be not less reliable than now ?

Is today' RG more reliable than Oracle, Informix, DB2?

Vadim

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@@postgresql.org so that your
message can get through to the mailing list cleanly

From pgsql-hackers-owner+M18125=candle.pha.pa.us=pgman@@postgresql.org Thu Jan 24 14:23:42 2002
Return-path: <pgsql-hackers-owner+M18125=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0OJNfU13481
	for <pgman@@candle.pha.pa.us>; Thu, 24 Jan 2002 14:23:42 -0500 (EST)
Received: (qmail 49604 invoked by alias); 24 Jan 2002 19:23:40 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 24 Jan 2002 19:23:40 -0000
Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g0OJMTl48885
	for <pgsql-hackers@@postgreSQL.org>; Thu, 24 Jan 2002 14:22:29 -0500 (EST)
	(envelope-from pgman@@candle.pha.pa.us)
Received: (from pgman@@localhost)
	by candle.pha.pa.us (8.11.6/8.10.1) id g0OJMJf13378;
	Thu, 24 Jan 2002 14:22:19 -0500 (EST)
From: Bruce Momjian <pgman@@candle.pha.pa.us>
Message-ID: <200201241922.g0OJMJf13378@@candle.pha.pa.us>
Subject: Re: [HACKERS] Savepoints
In-Reply-To: <3705826352029646A3E91C53F7189E32518487@@sectorbase2.sectorbase.com>
To: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
Date: Thu, 24 Jan 2002 14:22:19 -0500 (EST)
cc: PostgreSQL-development <pgsql-hackers@@postgresql.org>
X-Mailer: ELM [version 2.4ME+ PL96 (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR


OK, I have had time to think about this, and I think I can put the two
proposals into perspective.  I will use Vadim's terminology.

In our current setup, rollback/undo data is kept in the same file as our
live data.  This data is used for two purposes, one, for rollback of
transactions, and perhaps subtransactions in the future, and second, for
MVCC visibility for backends making changes.

So, it seems the real question is whether a database modification should
write the old data into a separate rollback segment and modify the heap
data, or just create a new row and require the old row to be removed
later by vacuum.

Let's look at this behavior without MVCC.  In such cases, if someone
tries to read a modified row, it will block and wait for the modifying
backend to commit or rollback, when it will then continue.  In such
cases, there is no reason for the waiting transaction to read the old
data in the redo segment because it can't continue anyway.

Now, with MVCC, the backend has to read through the redo segment to get
the original data value for that row.

Now, while rollback segments do help with cleaning out old UPDATE rows,
how does it improve DELETE performance?  Seems it would just mark it as
expired like we do now.

One objection I always had to redo segments was that if I start a
transaction in the morning and walk away, none of the redo segments can
be recycled.  I was going to ask if we can force some type of redo
segment compaction to keep old active rows and delete rows no longer
visible to any transaction.  However, I now realize that our VACUUM has
the same problem.  Tuples with XID >= GetOldestXmin() are not recycled,
meaning we have this problem in our current implementation too.  (I
wonder if our vacuum could be smarter about knowing which rows are
visible, perhaps by creating a sorted list of xid's and doing a binary
search on the list to determine visibility.)

So, I guess the issue is, do we want to keep redo information in the
main table, or split it out into redo segments.  Certainly we have to
eliminate the Oracle restrictions that redo segment size is fixed at
install time.

The advantages of a redo segment is that hopefully we don't have
transactions reading through irrelevant undo information.  The
disadvantage is that we now have redo information grouped into table
files where a sequential scan can be performed.  (Index scans of redo
info are a performance problem currently.)  We would have to somehow
efficiently access redo information grouped into the redo segments. 
Perhaps a hash based in relid would help here.  Another disadvantage is
concurrency.  When we start modifying heap data in place, we have to
prevent other backends from seeing that modification while we move the
old data to the redo segment.

I guess my feeling is that if we can get vacuum to happen automatically,
how is our current non-overwriting storage manager different from redo
segments?

One big advantage of redo segments would be that right now, if someone
updates a row repeatedly, there are lots of heap versions of the row
that are difficult to shrink in the table, while if they are in the redo
segments, we can more efficiently remove them, and there is only on heap
row.

How is recovery handled with rollback segments?  Do we write old and new
data to WAL?   We just write new data to WAL now, right?  Do we fsync
rollback segments?

Have I outlined this accurately?

---------------------------------------------------------------------------

Mikheev, Vadim wrote:
> > > How about: use overwriting smgr + put old records into rollback
> > > segments - RS - (you have to keep them somewhere till TX's running
> > > anyway) + use WAL only as REDO log (RS will be used to rollback TX'
> > > changes and WAL will be used for RS/data files recovery).
> > > Something like what Oracle does.
> > 
> > I am sorry. I see what you are saying now. I missed the words
> 
> And I'm sorry for missing your notes about storing relid+tid only.
> 
> > "overwriting smgr". You are suggesting going to an overwriting
> > storage manager. Is this to be done only because of savepoints.
> 
> No. One point I made a few monthes ago (and never got objections)
> is - why to keep old data in data files sooooo long?
> Imagine long running TX (eg pg_dump). Why other TX-s must read
> again and again completely useless (for them) old data we keep
> for pg_dump?
> 
> > Doesn't seem worth it when I have a possible solution without
> > such a drastic change.
> > Also, overwriting storage manager will require MVCC to read
> > through there to get accurate MVCC visibility, right?
> 
> Right... just like now non-overwriting smgr requires *ALL*
> TX-s to read old data in data files. But with overwriting smgr
> TX will read RS only when it is required and as far (much) as
> it is required.
> 
> Simple solutions are not always the best ones.
> Compare Oracle and InterBase. Both have MVCC.
> Smgr-s are different. What RDBMS is more cool?
> Why doesn't Oracle use more simple non-overwriting smgr
> (as InterBase... and we do)?
> 
> Vadim
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@@postgresql.org)

From pgsql-hackers-owner+M18141=candle.pha.pa.us=pgman@@postgresql.org Thu Jan 24 19:43:38 2002
Return-path: <pgsql-hackers-owner+M18141=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0P0hbU15026
	for <pgman@@candle.pha.pa.us>; Thu, 24 Jan 2002 19:43:38 -0500 (EST)
Received: (qmail 28642 invoked by alias); 25 Jan 2002 00:43:24 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 25 Jan 2002 00:43:24 -0000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
	by postgresql.org (8.11.3/8.11.4) with SMTP id g0P0YIl27208
	for <pgsql-hackers@@postgreSQL.org>; Thu, 24 Jan 2002 19:34:18 -0500 (EST)
	(envelope-from Inoue@@tpf.co.jp)
Received: (qmail 3661 invoked from network); 25 Jan 2002 00:34:19 -0000
Received: from unknown (HELO viscomail.tpf.co.jp) (100.0.0.108)
  by sd2.tpf-fw-c.co.jp with SMTP; 25 Jan 2002 00:34:19 -0000
Received: from tpf.co.jp (3dgateway1 [126.0.1.60])
	by viscomail.tpf.co.jp (8.8.8+Sun/8.8.8) with ESMTP id JAA00756;
	Fri, 25 Jan 2002 09:34:18 +0900 (JST)
Message-ID: <3C50A807.32A29E09@@tpf.co.jp>
Date: Fri, 25 Jan 2002 09:34:15 +0900
From: Hiroshi Inoue <Inoue@@tpf.co.jp>
X-Mailer: Mozilla 4.73 [ja] (Windows NT 5.0; U)
X-Accept-Language: ja
MIME-Version: 1.0
To: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
cc: PostgreSQL-development <pgsql-hackers@@postgresql.org>,
   "'Bruce Momjian'" <pgman@@candle.pha.pa.us>
Subject: Re: [HACKERS] Savepoints
References: <3705826352029646A3E91C53F7189E3251848B@@sectorbase2.sectorbase.com>
Content-Type: text/plain; charset=iso-2022-jp
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

"Mikheev, Vadim" wrote:
> 
> > > How about: use overwriting smgr + put old records into rollback
> > > segments - RS - (you have to keep them somewhere till TX's running
> > > anyway) + use WAL only as REDO log (RS will be used to rollback TX'
> > > changes and WAL will be used for RS/data files recovery).
> > > Something like what Oracle does.
> >
> > As long as we use no overwriting manager
> > 1) Rollback(data) isn't needed in case of a db crash.
> > 2) Rollback(data) isn't needed to cancal a transaction entirely.
> 
> -1) But vacuum must read a huge amount of data to remove dirt.
> -2) But TX-s must read data they are not interested at all.
> 
> > 3) We don't need to mind the transaction size so much.
> 
> -3) The same with overwriting smgr and WAL used *only as REDO log*:

The larger RS becomes the longer it would take time to cancel
the transaction whereas it is executed in a momemnt under no
overwriting smgr and for example if RS exhausted all disk space
is PG really safe ? Other backends would also fail because they
couldn't write RS any mode. Many transactions would execute
UNDO operations simultaneously but there's no space to write
WALs (UNDO operations must be written to WAL also) and PG
system would abort. And could PG restart under such situations ?
Even though there's a way to recover from the situation, I
think we should avoid such dangerous situations from the
first. Basically recovery operations should never fail.
 
> 
> > We can't use the db any longer if a REDO recovery fails now.
> 
> Reset WAL and use/dump it. Annoying? Agreed. Fix bugs and/or
> use good RAM - whatever caused problem with restart.

As I already mentioned recovery operations should never fail.
> 
> > Under overwriting smgr we can't use the db any longer either
> > if rollback fails.
> 
> Why should it fail? Bugs? Fix them.

Rollback operations are executed much more often than
REDO recovery and it is hard to fix such bugs once PG
was released. Most people in such troubles have no
time to persue the cause. In reality I replied to the
PG restart troubles twice (with --wal-debug and pg_resetxlog
suggestions ) in Japan but got no further replies.

> 
> > How could PG be not less reliable than now ?
> 
> Is today' RG more reliable than Oracle, Informix, DB2?

I have never been and would never be optiomistic
about recovery. Is 7.1 more reliable than 7.0 from the
recovery POV ? I see no reason why overwriting smgr is
more relaible than no overwriting smgr as for recovery.

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

From ZeugswetterA@@spardat.at Fri Jan 25 09:21:40 2002
Return-path: <ZeugswetterA@@spardat.at>
Received: from smxsat1.smxs.net (smxsat1.smxs.net [213.150.10.1])
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id g0PELde10640
	for <pgman@@candle.pha.pa.us>; Fri, 25 Jan 2002 09:21:39 -0500 (EST)
Received: from m01x1.s-mxs.net [10.3.55.201]
	by smxsat1.smxs.net
	with XWall v3.18f ;
	Fri, 25 Jan 2002 15:22:51 +0100
Received: from m0103.s-mxs.net [10.3.55.3]
	by m01x1.s-mxs.net
	with XWall v3.18a ;
	Fri, 25 Jan 2002 15:21:23 +0100
Received: from m0114.s-mxs.net ([10.3.55.14]) by m0103.s-mxs.net with Microsoft SMTPSVC(5.0.2195.2966);
  Fri, 25 Jan 2002 15:21:22 +0100
X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [HACKERS] Savepoints
Date: Fri, 25 Jan 2002 15:21:22 +0100
Message-ID: <46C15C39FEB2C44BA555E356FBCD6FA42128DE@@m0114.s-mxs.net>
Thread-Topic: [HACKERS] Savepoints
Thread-Index: AcGkZ8SMKn//UUTjS3mi+qC7+gZAwwBQ4YMA
From: "Zeugswetter Andreas SB SD" <ZeugswetterA@@spardat.at>
To: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>,
   "Bruce Momjian" <pgman@@candle.pha.pa.us>,
   "PostgreSQL-development" <pgsql-hackers@@postgresql.org>
X-OriginalArrivalTime: 25 Jan 2002 14:21:22.0648 (UTC) FILETIME=[9090BD80:01C1A5AB]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id g0PELde10640
Status: OR

Vadim wrote:
> How about: use overwriting smgr + put old records into rollback
> segments - RS - (you have to keep them somewhere till TX's running
> anyway) + use WAL only as REDO log (RS will be used to rollback TX'
> changes and WAL will be used for RS/data files recovery).
> Something like what Oracle does.

We have all the info we need in WAL and in the old rows,
why would you want to write them to RS ?
You only need RS for overwriting smgr.

Andreas

From pgsql-hackers-owner+M18209=candle.pha.pa.us=pgman@@postgresql.org Fri Jan 25 16:14:02 2002
Return-path: <pgsql-hackers-owner+M18209=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0PLE1e19182
	for <pgman@@candle.pha.pa.us>; Fri, 25 Jan 2002 16:14:01 -0500 (EST)
Received: (qmail 85111 invoked by alias); 25 Jan 2002 21:13:59 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 25 Jan 2002 21:13:59 -0000
Received: from smxsat1.smxs.net (smxsat1.smxs.net [213.150.10.1])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g0PL48l79366
	for <pgsql-hackers@@postgresql.org>; Fri, 25 Jan 2002 16:04:09 -0500 (EST)
	(envelope-from ZeugswetterA@@spardat.at)
Received: from m01x1.s-mxs.net [10.3.55.201]
	by smxsat1.smxs.net
	with XWall v3.18f ;
	Fri, 25 Jan 2002 22:05:21 +0100
Received: from m0102.s-mxs.net [10.3.55.2]
	by m01x1.s-mxs.net
	with XWall v3.18a ;
	Fri, 25 Jan 2002 22:03:54 +0100
Received: from m0114.s-mxs.net ([10.3.55.14]) by m0102.s-mxs.net with Microsoft SMTPSVC(5.0.2195.2966);
  Fri, 25 Jan 2002 22:03:53 +0100
X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: Re: [HACKERS] Savepoints
Date: Fri, 25 Jan 2002 22:03:53 +0100
Message-ID: <46C15C39FEB2C44BA555E356FBCD6FA41EB4C4@@m0114.s-mxs.net>
Thread-Topic: [HACKERS] Savepoints
Thread-Index: AcGlDMGVwSWndt4kT1C7QhclLvQPWgA1arbw
From: "Zeugswetter Andreas SB SD" <ZeugswetterA@@spardat.at>
To: "Bruce Momjian" <pgman@@candle.pha.pa.us>,
   "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
cc: "PostgreSQL-development" <pgsql-hackers@@postgresql.org>
X-OriginalArrivalTime: 25 Jan 2002 21:03:53.0685 (UTC) FILETIME=[CBB48850:01C1A5E3]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id g0PLDAm83732
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: ORr


> Now, with MVCC, the backend has to read through the redo segment to get

You mean rollback segment, but ...

> the original data value for that row.

Will only need to be looked up if the row is currently beeing modified by 
a not yet comitted txn (at least in the default read committed mode)  

> 
> Now, while rollback segments do help with cleaning out old UPDATE rows,
> how does it improve DELETE performance?  Seems it would just mark it as
> expired like we do now.

delete would probably be: 
1. mark original deleted and write whole row to RS

I don't think you would like to mix looking up deleted rows in heap
but updated rows in RS

Andreas

PS: not that I like overwrite with MVCC now
If you think of VACUUM as garbage collection PG is highly trendy with
the non-overwriting smgr.

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

From pgsql-hackers-owner+M18211=candle.pha.pa.us=pgman@@postgresql.org Fri Jan 25 16:53:45 2002
Return-path: <pgsql-hackers-owner+M18211=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0PLrie22174
	for <pgman@@candle.pha.pa.us>; Fri, 25 Jan 2002 16:53:44 -0500 (EST)
Received: (qmail 96831 invoked by alias); 25 Jan 2002 21:53:43 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 25 Jan 2002 21:53:43 -0000
Received: from smxsat1.smxs.net (smxsat1.smxs.net [213.150.10.1])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g0PLpRl96298
	for <pgsql-hackers@@postgresql.org>; Fri, 25 Jan 2002 16:51:27 -0500 (EST)
	(envelope-from ZeugswetterA@@spardat.at)
Received: from m01x1.s-mxs.net [10.3.55.201]
	by smxsat1.smxs.net
	with XWall v3.18f ;
	Fri, 25 Jan 2002 22:52:54 +0100
Received: from m0103.s-mxs.net [10.3.55.3]
	by m01x1.s-mxs.net
	with XWall v3.18a ;
	Fri, 25 Jan 2002 22:51:25 +0100
Received: from m0114.s-mxs.net ([10.3.55.14]) by m0103.s-mxs.net with Microsoft SMTPSVC(5.0.2195.2966);
  Fri, 25 Jan 2002 22:51:25 +0100
X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: Re: [HACKERS] Savepoints
Date: Fri, 25 Jan 2002 22:51:24 +0100
Message-ID: <46C15C39FEB2C44BA555E356FBCD6FA41EB4C5@@m0114.s-mxs.net>
Thread-Topic: [HACKERS] Savepoints
Thread-Index: AcGlznYKFcqoYpMnSlGQHhQuEf6LuAAGpxnQ
From: "Zeugswetter Andreas SB SD" <ZeugswetterA@@spardat.at>
To: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
cc: <pgsql-hackers@@postgresql.org>
X-OriginalArrivalTime: 25 Jan 2002 21:51:25.0008 (UTC) FILETIME=[6F39E500:01C1A5EA]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by postgresql.org id g0PLrP196418
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR


> > > How about: use overwriting smgr + put old records into rollback
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > segments - RS - (you have to keep them somewhere till TX's running
> > > anyway) + use WAL only as REDO log (RS will be used to 
> rollback TX'
> > > changes and WAL will be used for RS/data files recovery).
> > > Something like what Oracle does.
> > 
> > We have all the info we need in WAL and in the old rows,
> > why would you want to write them to RS ?
> > You only need RS for overwriting smgr.
> 
> This is what I'm saying - implement Overwriting smgr...

Yes I am sorry, I am catching up on email and had not read Bruce's 
comment (nor yours correctly) :-(

I was also long in the pro overwriting camp, because I am used to 
non MVCC dbs like DB/2 and Informix. (which I like very much) 
But I am starting to doubt that overwriting is really so good for
an MVCC db. And I don't think PG wants to switch to non MVCC :-)

Imho it would only need a much more aggressive VACUUM backend.
(aka garbage collector :-) Maybe It could be designed to sniff the 
redo log (buffer) to get a hint at what to actually clean out next.

Andreas

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

From pgsql-hackers-owner+M18218=candle.pha.pa.us=pgman@@postgresql.org Fri Jan 25 19:14:24 2002
Return-path: <pgsql-hackers-owner+M18218=candle.pha.pa.us=pgman@@postgresql.org>
Received: from server1.pgsql.org (www.postgresql.org [64.49.215.9])
	by candle.pha.pa.us (8.11.6/8.10.1) with SMTP id g0Q0ENe03543
	for <pgman@@candle.pha.pa.us>; Fri, 25 Jan 2002 19:14:23 -0500 (EST)
Received: (qmail 22482 invoked by alias); 26 Jan 2002 00:13:55 -0000
Received: from unknown (HELO postgresql.org) (64.49.215.8)
  by www.postgresql.org with SMTP; 26 Jan 2002 00:13:55 -0000
Received: from candle.pha.pa.us (216-55-132-35.dsl.san-diego.abac.net [216.55.132.35])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id g0PNw1l20714
	for <pgsql-hackers@@postgresql.org>; Fri, 25 Jan 2002 18:58:01 -0500 (EST)
	(envelope-from pgman@@candle.pha.pa.us)
Received: (from pgman@@localhost)
	by candle.pha.pa.us (8.11.6/8.10.1) id g0PNvoL02515;
	Fri, 25 Jan 2002 18:57:50 -0500 (EST)
From: Bruce Momjian <pgman@@candle.pha.pa.us>
Message-ID: <200201252357.g0PNvoL02515@@candle.pha.pa.us>
Subject: Re: [HACKERS] Savepoints
In-Reply-To: <46C15C39FEB2C44BA555E356FBCD6FA41EB4C4@@m0114.s-mxs.net>
To: Zeugswetter Andreas SB SD <ZeugswetterA@@spardat.at>
Date: Fri, 25 Jan 2002 18:57:50 -0500 (EST)
cc: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>,
   PostgreSQL-development <pgsql-hackers@@postgresql.org>
X-Mailer: ELM [version 2.4ME+ PL96 (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

Zeugswetter Andreas SB SD wrote:
> 
> > Now, with MVCC, the backend has to read through the redo segment to get
> 
> You mean rollback segment, but ...


Sorry, yes.  I get redo/undo/rollback mixed up sometimes. :-)

> > the original data value for that row.
> 
> Will only need to be looked up if the row is currently beeing modified by 
> a not yet comitted txn (at least in the default read committed mode)  

Uh, not really.  The transaction may have completed after my transaction
started, meaning even though it looks like it is committed, to me, it is
not visible.  Most MVCC visibility will require undo lookup.

> 
> > 
> > Now, while rollback segments do help with cleaning out old UPDATE rows,
> > how does it improve DELETE performance?  Seems it would just mark it as
> > expired like we do now.
> 
> delete would probably be: 
> 1. mark original deleted and write whole row to RS
> 
> I don't think you would like to mix looking up deleted rows in heap
> but updated rows in RS

Yes, so really the overwriting is only a big win for UPDATE.  Right now,
UPDATE is DELETE/INSERT, and that DELETE makes MVCC happy. :-)

My whole goal was to simplify this so we can see the differences.


> PS: not that I like overwrite with MVCC now
> If you think of VACUUM as garbage collection PG is highly trendy with
> the non-overwriting smgr.

Yes, that is basically what it is now, a garbage collector that collects
in heap rather than in undo.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@@postgresql.org so that your
message can get through to the mailing list cleanly

From pgman Wed Jan 23 10:36:13 2002
Subject: Savepoints
To: PostgreSQL-development <pgsql-hackers@@postgreSQL.org>
Date: Wed, 23 Jan 2002 13:19:05 -0500 (EST)
X-Mailer: ELM [version 2.4ME+ PL96 (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
Content-Length:  1829
Status: OR

I have talked in the past about a possible implementation of
savepoints/nested transactions.  I would like to more formally outline
my ideas below.

We have talked about using WAL for such a purpose, but that requires WAL
files to remain for the life of a transaction, which seems unacceptable.
Other database systems do that, and it is a pain for administrators.  I
realized we could do some sort of WAL compaction, but that seems quite
complex too.

Basically, under my plan, WAL would be unchanged.  WAL's function is
crash recovery, and it would retain that.  There would also be no
on-disk changes.  I would use the command counter in certain cases to
identify savepoints.

My idea is to keep savepoint undo information in a private area per
backend, either in memory or on disk.  We can either save the
relid/tids of modified rows, or if there are too many, discard the
saved ones and just remember the modified relids.  On rollback to save
point, either clear up the modified relid/tids, or sequential scan
through the relid and clear up all the tuples that have our transaction
id and have command counters that are part of the undo savepoint.

It seems marking undo savepoint rows with a fixed aborted transaction id
would be the easiest solution.

Of course, we only remember modified rows when we are in savepoints, and
only undo them when we rollback to a savepoint.  Transaction processing
remains the same.

There is no reason for other backend to be able to see savepoint undo
information, and keeping it private greatly simplifies the
implementation.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

@


1.6
log
@Add savepoints discussion.
@
text
@@


1.5
log
@Add Goggle mention of previous discussion.
@
text
@d170 1008
@


1.4
log
@Add new email;remove old entries.
@
text
@d132 38
@


1.3
log
@Update transactions for nested idea.
@
text
@a0 1055
From pgsql-hackers-owner+M215@@postgresql.org Fri Nov  3 17:50:40 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id RAA05273
	for <pgman@@candle.pha.pa.us>; Fri, 3 Nov 2000 17:50:39 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA3Mm1s26018;
	Fri, 3 Nov 2000 17:48:01 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M215@@postgresql.org)
Received: from sss.pgh.pa.us (sss.pgh.pa.us [209.114.132.154])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA3Mles25919
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 17:47:40 -0500 (EST)
	(envelope-from tgl@@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.11.1/8.11.1) with ESMTP id eA3Mle508385
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 17:47:40 -0500 (EST)
To: pgsql-hackers@@postgresql.org
Subject: [HACKERS] Transaction ID wraparound: problem and proposed solution
Date: Fri, 03 Nov 2000 17:47:40 -0500
Message-ID: <8382.973291660@@sss.pgh.pa.us>
From: Tom Lane <tgl@@sss.pgh.pa.us>
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: ORr

We've expended a lot of worry and discussion in the past about what
happens if the OID generator wraps around.  However, there is another
4-byte counter in the system: the transaction ID (XID) generator.
While OID wraparound is survivable, if XIDs wrap around then we really
do have a Ragnarok scenario.  The tuple validity checks do ordered
comparisons on XIDs, and will consider tuples with xmin > current xact
to be invalid.  Result: after wraparound, your whole database would
instantly vanish from view.

The first thought that comes to mind is that XIDs should be promoted to
eight bytes.  However there are several practical problems with this:
* portability --- I don't believe long long int exists on all the
platforms we support.
* performance --- except on true 64-bit platforms, widening Datum to
eight bytes would be a system-wide performance hit, which is a tad
unpleasant to fix a scenario that's not yet been reported from the
field.
* disk space --- letting pg_log grow without bound isn't a pleasant
prospect either.

I believe it is possible to fix these problems without widening XID,
by redefining XIDs in a way that allows for wraparound.  Here's my
plan:

1. Allow XIDs to range from 0 to WRAPLIMIT-1 (WRAPLIMIT is not
necessarily 4G, see discussion below).  Ordered comparisons on XIDs
are no longer simply "x < y", but need to be expressed as a macro.
We consider x < y if (y - x) % WRAPLIMIT < WRAPLIMIT/2.
This comparison will work as long as the range of interesting XIDs
never exceeds WRAPLIMIT/2.  Essentially, we envision the actual value
of XID as being the low-order bits of a logical XID that always
increases, and we assume that no extant XID is more than WRAPLIMIT/2
transactions old, so we needn't keep track of the high-order bits.

2. To keep the system from having to deal with XIDs that are more than
WRAPLIMIT/2 transactions old, VACUUM should "freeze" known-good old
tuples.  To do this, we'll reserve a special XID, say 1, that is always
considered committed and is always less than any ordinary XID.  (So the
ordered-comparison macro is really a little more complicated than I said
above.  Note that there is already a reserved XID just like this in the
system, the "bootstrap" XID.  We could simply use the bootstrap XID, but
it seems better to make another one.)  When VACUUM finds a tuple that
is committed good and has xmin < XmaxRecent (the oldest XID that might
be considered uncommitted by any open transaction), it will replace that
tuple's xmin by the special always-good XID.  Therefore, as long as
VACUUM is run on all tables in the installation more often than once per
WRAPLIMIT/2 transactions, there will be no tuples with ordinary XIDs
older than WRAPLIMIT/2.

3. At wraparound, the XID counter has to be advanced to skip over the
InvalidXID value (zero) and the reserved XIDs, so that no real transaction
is generated with those XIDs.  No biggie here.

4. With the wraparound behavior, pg_log will have a bounded size: it
will never exceed WRAPLIMIT*2 bits = WRAPLIMIT/4 bytes.  Since we will
recycle pg_log entries every WRAPLIMIT xacts, during transaction start
the xact manager will have to take care to actively clear its pg_log
entry to zeroes (I'm not sure if it does that already, or just assumes
that new pg_log entries will start out zero).  As long as that happens
before the xact makes any data changes, it's OK to recycle the entry.
Note we are assuming that no tuples will remain in the database with
xmin or xmax equal to that XID from a prior cycle of the universe.

This scheme allows us to survive XID wraparound at the cost of slight
additional complexity in ordered comparisons of XIDs (which is not a
really performance-critical task AFAIK), and at the cost that the
original insertion XIDs of all but recent tuples will be lost by
VACUUM.  The system doesn't particularly care about that, but old XIDs
do sometimes come in handy for debugging purposes.  A possible
compromise is to overwrite only XIDs that are older than, say,
WRAPLIMIT/4 instead of doing so as soon as possible.  This would mean
the required VACUUM frequency is every WRAPLIMIT/4 xacts instead of
every WRAPLIMIT/2 xacts.

We have a straightforward tradeoff between the maximum size of pg_log
(WRAPLIMIT/4 bytes) and the required frequency of VACUUM (at least
every WRAPLIMIT/2 or WRAPLIMIT/4 transactions).  This could be made
configurable in config.h for those who're intent on customization,
but I'd be inclined to set the default value at WRAPLIMIT = 1G.

Comments?  Vadim, is any of this about to be superseded by WAL?
If not, I'd like to fix it for 7.1.

			regards, tom lane

From pgsql-hackers-owner+M232@@postgresql.org Fri Nov  3 20:20:32 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id UAA08863
	for <pgman@@candle.pha.pa.us>; Fri, 3 Nov 2000 20:20:31 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA41Jgs31567;
	Fri, 3 Nov 2000 20:19:42 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M232@@postgresql.org)
Received: from thor.tht.net (thor.tht.net [209.47.145.4])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA41CMs31023
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 20:12:22 -0500 (EST)
	(envelope-from tgl@@sss.pgh.pa.us)
Received: from sss.pgh.pa.us (sss.pgh.pa.us [209.114.132.154])
	by thor.tht.net (8.9.3/8.9.3) with ESMTP id VAA14928
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 21:13:08 GMT
	(envelope-from tgl@@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.11.1/8.11.1) with ESMTP id eA41CK508777;
	Fri, 3 Nov 2000 20:12:21 -0500 (EST)
To: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
cc: pgsql-hackers@@postgresql.org
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
In-reply-to: <8F4C99C66D04D4118F580090272A7A234D3146@@sectorbase1.sectorbase.com> 
References: <8F4C99C66D04D4118F580090272A7A234D3146@@sectorbase1.sectorbase.com>
Comments: In-reply-to "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
	message dated "Fri, 03 Nov 2000 16:24:38 -0800"
Date: Fri, 03 Nov 2000 20:12:20 -0500
Message-ID: <8774.973300340@@sss.pgh.pa.us>
From: Tom Lane <tgl@@sss.pgh.pa.us>
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

"Mikheev, Vadim" <vmikheev@@SECTORBASE.COM> writes:
> So, we'll have to abort some long running transaction.

Well, yes, some transaction that continues running while ~ 500 million
other transactions come and go might give us trouble.  I wasn't really
planning to worry about that case ;-)

> Required frequency of *successful* vacuum over *all* tables.
> We would have to remember something in pg_class/pg_database
> and somehow force vacuum over "too-long-unvacuumed-tables"
> *automatically*.

I don't think this is a problem now; in practice you couldn't possibly
go for half a billion transactions without vacuuming, I'd think.

If your plans to eliminate regular vacuuming become reality, then this
scheme might become less reliable, but at present I think there's plenty
of safety margin.

> If undo would be implemented then we could delete pg_log between
> postmaster startups - startup counter is remembered in pages, so
> seeing old startup id in a page we would know that there are only
> long ago committed xactions (ie only visible changes) there
> and avoid xid comparison. But ... there will be no undo in 7.1.
> And I foresee problems with WAL based BAR implementation if we'll
> follow proposed solution: redo restores original xmin/xmax - how
> to "freeze" xids while restoring DB?

So, we might eventually have a better answer from WAL, but not for 7.1.

I think my idea is reasonably non-invasive and could be removed without
much trouble once WAL offers a better way.  I'd really like to have some
answer for 7.1, though.  The sort of numbers John Scott was quoting to
me for Verizon's paging network throughput make it clear that we aren't
going to survive at that level with a limit of 4G transactions per
database reload.  Having to vacuum everything on at least a
1G-transaction cycle is salable, dump/initdb/reload is not ...

			regards, tom lane

From pgsql-hackers-owner+M238@@postgresql.org Fri Nov  3 21:30:14 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id VAA12038
	for <pgman@@candle.pha.pa.us>; Fri, 3 Nov 2000 21:30:13 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA42TQs33780;
	Fri, 3 Nov 2000 21:29:26 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M238@@postgresql.org)
Received: from sss.pgh.pa.us (sss.pgh.pa.us [209.114.132.154])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA42TCs33632
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 21:29:12 -0500 (EST)
	(envelope-from tgl@@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.11.1/8.11.1) with ESMTP id eA42T5509042;
	Fri, 3 Nov 2000 21:29:05 -0500 (EST)
To: Philip Warner <pjw@@rhyme.com.au>
cc: pgsql-hackers@@postgresql.org
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution 
In-reply-to: <3.0.5.32.20001104130922.045c3410@@mail.rhyme.com.au> 
References: <3.0.5.32.20001104130922.045c3410@@mail.rhyme.com.au>
Comments: In-reply-to Philip Warner <pjw@@rhyme.com.au>
	message dated "Sat, 04 Nov 2000 13:09:22 +1100"
Date: Fri, 03 Nov 2000 21:29:04 -0500
Message-ID: <9039.973304944@@sss.pgh.pa.us>
From: Tom Lane <tgl@@sss.pgh.pa.us>
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

Philip Warner <pjw@@rhyme.com.au> writes:
>> * disk space --- letting pg_log grow without bound isn't a pleasant
>> prospect either.

> Maybe this can be achieved by wrapping XID for the log file only.

How's that going to improve matters?  pg_log is ground truth for XIDs;
if you can't distinguish two XIDs in pg_log, there's no point in
distinguishing them elsewhere.

> Maybe I'm really missing the amount of XID manipulation, but I'd be
> surprised if 16-byte XIDs would slow things down much.

It's not so much XIDs themselves, as that I think we'd need to widen
typedef Datum too, and that affects manipulations of *all* data types.

In any case, the prospect of a multi-gigabyte, ever-growing pg_log file,
with no way to recover the space short of dump/initdb/reload, is
awfully unappetizing for a high-traffic installation...

			regards, tom lane

From pgsql-hackers-owner+M240@@postgresql.org Fri Nov  3 21:42:30 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id VAA13035
	for <pgman@@candle.pha.pa.us>; Fri, 3 Nov 2000 21:42:29 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA42fjs40619;
	Fri, 3 Nov 2000 21:41:45 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M240@@postgresql.org)
Received: from hse-toronto-ppp119263.sympatico.ca (HSE-Toronto-ppp85465.sympatico.ca [216.209.18.18])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA42fXs40530
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 21:41:33 -0500 (EST)
	(envelope-from rbt@@zort.on.ca)
Received: (qmail 66996 invoked by uid 0); 4 Nov 2000 02:46:34 -0000
Received: from unknown (HELO zort.on.ca) (rbt@@10.0.0.100)
  by hse-toronto-ppp85465.sympatico.ca with SMTP; 4 Nov 2000 02:46:34 -0000
Message-ID: <3A037759.2D6A67E4@@zort.on.ca>
Date: Fri, 03 Nov 2000 21:41:29 -0500
From: Rod Taylor <rbt@@zort.on.ca>
Organization: Zort
X-Mailer: Mozilla 4.75 [en] (X11; U; FreeBSD 4.1.1-STABLE i386)
X-Accept-Language: en
MIME-Version: 1.0
To: Tom Lane <tgl@@sss.pgh.pa.us>
CC: Philip Warner <pjw@@rhyme.com.au>, pgsql-hackers@@postgresql.org
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
References: <3.0.5.32.20001104130922.045c3410@@mail.rhyme.com.au> <9039.973304944@@sss.pgh.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

Tom Lane wrote:
> 
> Philip Warner <pjw@@rhyme.com.au> writes:
> >> * disk space --- letting pg_log grow without bound isn't a pleasant
> >> prospect either.
> 
> > Maybe this can be achieved by wrapping XID for the log file only.
> 
> How's that going to improve matters?  pg_log is ground truth for XIDs;
> if you can't distinguish two XIDs in pg_log, there's no point in
> distinguishing them elsewhere.
> 
> > Maybe I'm really missing the amount of XID manipulation, but I'd be
> > surprised if 16-byte XIDs would slow things down much.
> 
> It's not so much XIDs themselves, as that I think we'd need to widen
> typedef Datum too, and that affects manipulations of *all* data types.
> 
> In any case, the prospect of a multi-gigabyte, ever-growing pg_log file,
> with no way to recover the space short of dump/initdb/reload, is
> awfully unappetizing for a high-traffic installation...

Agreed completely.  I'd like to think I could have such an installation
in the next year or so :)  

To prevent a performance hit to those who don't want, is there a
possibility of either a compile time option or 'auto-expanding' the
width of the XID's and other items when it becomes appropriate?  Start
with int4, when that limit is hit goto int8, and should -- quite
unbelievibly so but there are multi-TB databases -- it be necessary jump
to int12 or int16?   Be the first to support Exa-objects in an RDBMS. 
Testing not necessary ;)

Compiletime option would be appropriate however if theres a significant
performance hit.

I'm not much of a c coder (obviously), so I don't know of the
limitations.  plpgsql is my friend that can do nearly anything :)

Hmm... After reading the above I should have stuck with lurking.

From pgsql-hackers-owner+M264@@postgresql.org Sun Nov  5 01:07:08 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id BAA29566
	for <pgman@@candle.pha.pa.us>; Sun, 5 Nov 2000 01:07:07 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA564Ks60463;
	Sun, 5 Nov 2000 01:04:20 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M264@@postgresql.org)
Received: from gate1.sectorbase.com ([208.48.122.134])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA55sas57106
	for <pgsql-hackers@@postgreSQL.org>; Sun, 5 Nov 2000 00:54:36 -0500 (EST)
	(envelope-from vmikheev@@sectorbase.com)
Received: from dune (unknown [208.48.122.182])
	by gate1.sectorbase.com (Postfix) with SMTP
	id 170DB2E806; Sat,  4 Nov 2000 21:53:56 -0800 (PST)
Message-ID: <016601c046ed$db6819c0$b87a30d0@@sectorbase.com>
From: "Vadim Mikheev" <vmikheev@@sectorbase.com>
To: "Tom Lane" <tgl@@sss.pgh.pa.us>
Cc: <pgsql-hackers@@postgresql.org>
References: <8F4C99C66D04D4118F580090272A7A234D3146@@sectorbase1.sectorbase.com> <8774.973300340@@sss.pgh.pa.us>
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
Date: Sat, 4 Nov 2000 21:59:00 -0800
MIME-Version: 1.0
Content-Type: text/plain;
	charset="windows-1251"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

> > So, we'll have to abort some long running transaction.
> 
> Well, yes, some transaction that continues running while ~ 500 million
> other transactions come and go might give us trouble.  I wasn't really
> planning to worry about that case ;-)

Agreed, I just don't like to rely on assumptions -:)

> > Required frequency of *successful* vacuum over *all* tables.
> > We would have to remember something in pg_class/pg_database
> > and somehow force vacuum over "too-long-unvacuumed-tables"
> > *automatically*.
> 
> I don't think this is a problem now; in practice you couldn't possibly
> go for half a billion transactions without vacuuming, I'd think.

Why not?
And once again - assumptions are not good for transaction area.

> If your plans to eliminate regular vacuuming become reality, then this
> scheme might become less reliable, but at present I think there's plenty
> of safety margin.
>
> > If undo would be implemented then we could delete pg_log between
> > postmaster startups - startup counter is remembered in pages, so
> > seeing old startup id in a page we would know that there are only
> > long ago committed xactions (ie only visible changes) there
> > and avoid xid comparison. But ... there will be no undo in 7.1.
> > And I foresee problems with WAL based BAR implementation if we'll
> > follow proposed solution: redo restores original xmin/xmax - how
> > to "freeze" xids while restoring DB?
> 
> So, we might eventually have a better answer from WAL, but not for 7.1.
> I think my idea is reasonably non-invasive and could be removed without
> much trouble once WAL offers a better way.  I'd really like to have some
> answer for 7.1, though.  The sort of numbers John Scott was quoting to
> me for Verizon's paging network throughput make it clear that we aren't
> going to survive at that level with a limit of 4G transactions per
> database reload.  Having to vacuum everything on at least a
> 1G-transaction cycle is salable, dump/initdb/reload is not ...

Understandable. And probably we can get BAR too but require full
backup every WRAPLIMIT/2 (or better /4) transactions.

Vadim



From vmikheev@@sectorbase.com Sun Nov  5 03:55:31 2000
Received: from gate1.sectorbase.com ([208.48.122.134])
	by candle.pha.pa.us (8.9.0/8.9.0) with SMTP id DAA10570
	for <pgman@@candle.pha.pa.us>; Sun, 5 Nov 2000 03:55:30 -0500 (EST)
Received: from dune (unknown [208.48.122.185])
	by gate1.sectorbase.com (Postfix) with SMTP
	id 5033D2E806; Sun,  5 Nov 2000 00:54:22 -0800 (PST)
Message-ID: <01cf01c04707$10085aa0$b87a30d0@@sectorbase.com>
From: "Vadim Mikheev" <vmikheev@@sectorbase.com>
To: "Bruce Momjian" <pgman@@candle.pha.pa.us>, "Tom Lane" <tgl@@sss.pgh.pa.us>
Cc: <pgsql-hackers@@postgresql.org>
References: <200011041843.NAA28411@@candle.pha.pa.us>
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
Date: Sun, 5 Nov 2000 01:02:01 -0800
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Status: OR

> One idea I had from this is actually truncating pg_log at some point if
> we know all the tuples have the special committed xid.  It would prevent
> the file from growing without bounds.

Not truncating, but implementing pg_log as set of files - we could remove
files for old xids.

> Vadim, can you explain how WAL will make pg_log unnecessary someday?

First, I mentioned only that having undo we could remove old pg_log after
postmaster startup because of only committed changes would be in data
files and they would be visible to new transactions (small changes in tqual
will be required to take page' startup id into account) which would reuse xids.
While changing a page first time in current startup, server would do exactly
what Tom is going to do at vacuuming - just update xmin/xmax to "1" in all items
(or setting some flag in t_infomask), - and change page' startup id to current.

I understand that this is not complete solution for xids problem, I just wasn't
going to solve it that time. Now after Tom' proposal I see how to reuse xids
without vacuuming (but having undo): we will add XidWrapId (XWI) - xid wrap
counter - to pages and set it when we change page. First time we do this for
page with old XWI we'll mark old items (to know later that they were changed
by xids with old XWI). Each time we change page we can mark old xmin/xmax
with xid <= current xid as committed long ago (basing on xact TTL restrinctions).

All above assumes that there will be no xids from aborted transactions in pages,
so we need not lookup in pg_log to know is a xid committed/aborted, - there will
be only xids from running or committed xactions there.

And we need in undo for this.

Vadim



From pgsql-hackers-owner+M396@@postgresql.org Tue Nov  7 20:57:16 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id UAA17110
	for <pgman@@candle.pha.pa.us>; Tue, 7 Nov 2000 20:57:16 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA81vcs17073;
	Tue, 7 Nov 2000 20:57:38 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M396@@postgresql.org)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA81kos15436
	for <pgsql-hackers@@postgresql.org>; Tue, 7 Nov 2000 20:46:50 -0500 (EST)
	(envelope-from pgsql-hackers-owner@@postgresql.org)
Received: from me.tm.ee (adsl895.estpak.ee [213.168.23.133])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA5Esds15479
	for <pgsql-hackers@@postgresql.org>; Sun, 5 Nov 2000 09:54:40 -0500 (EST)
	(envelope-from hannu@@tm.ee)
Received: from tm.ee (IDENT:hannu@@localhost.localdomain [127.0.0.1])
	by me.tm.ee (8.9.3/8.9.3) with ESMTP id PAA01401;
	Sun, 5 Nov 2000 15:48:14 +0200
Message-ID: <3A05651D.47B18E2F@@tm.ee>
Date: Sun, 05 Nov 2000 15:48:13 +0200
From: Hannu Krosing <hannu@@tm.ee>
X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.17 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Tom Lane <tgl@@sss.pgh.pa.us>
CC: Philip Warner <pjw@@rhyme.com.au>, pgsql-hackers@@postgresql.org
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
References: <3.0.5.32.20001104130922.045c3410@@mail.rhyme.com.au> <9039.973304944@@sss.pgh.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

Tom Lane wrote:
> 
> Philip Warner <pjw@@rhyme.com.au> writes:
> >> * disk space --- letting pg_log grow without bound isn't a pleasant
> >> prospect either.
> 
> > Maybe this can be achieved by wrapping XID for the log file only.
> 
> How's that going to improve matters?  pg_log is ground truth for XIDs;
> if you can't distinguish two XIDs in pg_log, there's no point in
> distinguishing them elsewhere.

One simple way - start a new pg_log file at each wraparound and encode 
the high 4 bytes in the filename (or in first four bytes of file)

> > Maybe I'm really missing the amount of XID manipulation, but I'd be
> > surprised if 16-byte XIDs would slow things down much.
> 
> It's not so much XIDs themselves, as that I think we'd need to widen
> typedef Datum too, and that affects manipulations of *all* data types.

Do you mean that each _field_ will take more space, not each _record_ ?

> In any case, the prospect of a multi-gigabyte, ever-growing pg_log file,
> with no way to recover the space short of dump/initdb/reload, is
> awfully unappetizing for a high-traffic installation...

The pg_log should be rotated anyway either with long xids or long-long
xids.

-----------
Hannu

From pgsql-hackers-owner+M284@@postgresql.org Sun Nov  5 16:19:47 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id QAA03570
	for <pgman@@candle.pha.pa.us>; Sun, 5 Nov 2000 16:19:46 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA5LKbs64176;
	Sun, 5 Nov 2000 16:20:37 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M284@@postgresql.org)
Received: from me.tm.ee (adsl895.estpak.ee [213.168.23.133])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA5LKCs64044
	for <pgsql-hackers@@postgresql.org>; Sun, 5 Nov 2000 16:20:12 -0500 (EST)
	(envelope-from hannu@@tm.ee)
Received: from tm.ee (IDENT:hannu@@localhost.localdomain [127.0.0.1])
	by me.tm.ee (8.9.3/8.9.3) with ESMTP id WAA00997;
	Sun, 5 Nov 2000 22:14:24 +0200
Message-ID: <3A05BFA0.5187B713@@tm.ee>
Date: Sun, 05 Nov 2000 22:14:24 +0200
From: Hannu Krosing <hannu@@tm.ee>
X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.17 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Peter Eisentraut <peter_e@@gmx.net>
CC: Tom Lane <tgl@@sss.pgh.pa.us>, pgsql-hackers@@postgresql.org
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
References: <Pine.LNX.4.21.0011051638470.780-100000@@peter.localdomain>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

Peter Eisentraut wrote:
> 
> Hannu Krosing writes:
> 
> > > The first thought that comes to mind is that XIDs should be promoted to
> > > eight bytes.  However there are several practical problems with this:
> > > * portability --- I don't believe long long int exists on all the
> > > platforms we support.
> >
> > I suspect that gcc at least supports long long on all OS-s we support
> 
> Uh, we don't want to depend on gcc, do we?

I suspect that we do on many platforms (like *BSD, Linux and Win32).

What platforms we currently support don't have functional gcc ?

> But we could make the XID a struct of two 4-byte integers, at the obvious
> increase in storage size.

And a (hopefully) small performance hit on operations when defined as
macros,
and some more for less data fitting in cache.

what operations do we need to be defined ?

will >, <, ==, !=, >=, <== and ++ be enough ?

-------------
Hannu

From pgsql-hackers-owner+M325@@postgresql.org Mon Nov  6 12:36:49 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id MAA24746
	for <pgman@@candle.pha.pa.us>; Mon, 6 Nov 2000 12:36:49 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA6HWqs14206;
	Mon, 6 Nov 2000 12:32:52 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M325@@postgresql.org)
Received: from granger.mail.mindspring.net (granger.mail.mindspring.net [207.69.200.148])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA6HT2s13718
	for <pgsql-hackers@@postgresql.org>; Mon, 6 Nov 2000 12:29:02 -0500 (EST)
	(envelope-from mhh@@mindspring.com)
Received: from jupiter (user-2inikn4.dialup.mindspring.com [165.121.82.228])
	by granger.mail.mindspring.net (8.9.3/8.8.5) with SMTP id MAA07826;
	Mon, 6 Nov 2000 12:28:37 -0500 (EST)
From: Mark Hollomon <mhh@@mindspring.com>
Reply-To: mhh@@mindspring.com
Date: Mon, 6 Nov 2000 13:09:19 -0500
X-Mailer: KMail [version 1.1.99]
Content-Type: text/plain;
  charset="iso-8859-1"
Cc: pgsql-hackers@@postgresql.org
To: Tom Lane <tgl@@sss.pgh.pa.us>
References: <8382.973291660@@sss.pgh.pa.us> <3A0567FF.37876138@@tm.ee> <788.973447357@@sss.pgh.pa.us>
In-Reply-To: <788.973447357@@sss.pgh.pa.us>
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
MIME-Version: 1.0
Message-Id: <00110613091900.00324@@jupiter>
Content-Transfer-Encoding: 8bit
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

On Sunday 05 November 2000 13:02, Tom Lane wrote:
> OK, 2^64 isn't mathematically unbounded, but let's see you buy a disk
> that will hold it ;-).  My point is that if we want to think about
> allowing >4G transactions, part of the answer has to be a way to recycle
> pg_log space.  Otherwise it's still not really practical.

I kind of like vadim's idea of segmenting pg_log. 

Segments in which all the xacts have been commited could be deleted.

-- 
Mark Hollomon

From pgsql-hackers-owner+M531@@postgresql.org Fri Nov 10 15:06:07 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id PAA23678
	for <pgman@@candle.pha.pa.us>; Fri, 10 Nov 2000 15:06:06 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eAAK5fs44672;
	Fri, 10 Nov 2000 15:05:41 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M531@@postgresql.org)
Received: from charybdis.zembu.com (charybdis.zembu.com [209.157.144.99])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eAAK30s44361
	for <pgsql-hackers@@postgresql.org>; Fri, 10 Nov 2000 15:03:01 -0500 (EST)
	(envelope-from ncm@@zembu.com)
Received: (qmail 15640 invoked from network); 10 Nov 2000 20:02:12 -0000
Received: from store.z.zembu.com (192.168.1.142)
  by charybdis.z.zembu.com with SMTP; 10 Nov 2000 20:02:12 -0000
Received: from ncm by store.z.zembu.com with local (Exim 3.12 #1 (Debian))
	id 13uKMX-0003rZ-00; Fri, 10 Nov 2000 12:01:25 -0800
Date: Fri, 10 Nov 2000 12:01:25 -0800
From: Nathan Myers <ncm@@zembu.com>
To: pgsql-hackers@@postgresql.org
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
Message-ID: <20001110120125.Q8881@@store.zembu.com>
Reply-To: pgsql-hackers@@postgresql.org
References: <3.0.5.32.20001104130922.045c3410@@mail.rhyme.com.au> <9039.973304944@@sss.pgh.pa.us> <3A05651D.47B18E2F@@tm.ee>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Mutt/1.0.1i
In-Reply-To: <3A05651D.47B18E2F@@tm.ee>; from hannu@@tm.ee on Sun, Nov 05, 2000 at 03:48:13PM +0200
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

On Sun, Nov 05, 2000 at 03:48:13PM +0200, Hannu Krosing wrote:
> Tom Lane wrote:
> > 
> > Philip Warner <pjw@@rhyme.com.au> writes:
> > >> * disk space --- letting pg_log grow without bound isn't a pleasant
> > >> prospect either.
> > 
> > > Maybe this can be achieved by wrapping XID for the log file only.
> > 
> > How's that going to improve matters?  pg_log is ground truth for XIDs;
> > if you can't distinguish two XIDs in pg_log, there's no point in
> > distinguishing them elsewhere.
> 
> One simple way - start a new pg_log file at each wraparound and encode 
> the high 4 bytes in the filename (or in first four bytes of file)

Proposal:

Annotate each log file with the current XID value at the time the file 
is created.  Before comparing any two XIDs, subtract that value from 
each operand, using unsigned arithmetic. 

At a sustained rate of 10,000 transactions/second, any pair of 32-bit 
XIDs less than 2.5 days apart compare properly.

Nathan Myers
ncm@@zembu.com


From pgsql-hackers-owner+M229@@postgresql.org Fri Nov  3 20:17:35 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id UAA08743
	for <pgman@@candle.pha.pa.us>; Fri, 3 Nov 2000 20:17:35 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA415Hs30899;
	Fri, 3 Nov 2000 20:05:22 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M229@@postgresql.org)
Received: from thor.tht.net (thor.tht.net [209.47.145.4])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA40dns30224
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 19:39:49 -0500 (EST)
	(envelope-from vmikheev@@SECTORBASE.COM)
Received: from sectorbase2.sectorbase.com ([208.48.122.131])
	by thor.tht.net (8.9.3/8.9.3) with SMTP id UAA14292
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 20:40:31 GMT
	(envelope-from vmikheev@@SECTORBASE.COM)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2650.21)
	id <V8XQBFBG>; Fri, 3 Nov 2000 16:20:43 -0800
Message-ID: <8F4C99C66D04D4118F580090272A7A234D3146@@sectorbase1.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
To: "'Tom Lane'" <tgl@@sss.pgh.pa.us>, pgsql-hackers@@postgresql.org
Subject: RE: [HACKERS] Transaction ID wraparound: problem and proposed sol
	ution
Date: Fri, 3 Nov 2000 16:24:38 -0800 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain;
	charset="iso-8859-1"
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

> This comparison will work as long as the range of interesting XIDs
> never exceeds WRAPLIMIT/2.  Essentially, we envision the actual value
> of XID as being the low-order bits of a logical XID that always
> increases, and we assume that no extant XID is more than WRAPLIMIT/2
> transactions old, so we needn't keep track of the high-order bits.

So, we'll have to abort some long running transaction.
And before after-wrap XIDs will be close to aborted xid you'd better
ensure that vacuum *successfully* run over all tables in database
(and shared tables) aborted transaction could touch.

> This scheme allows us to survive XID wraparound at the cost of slight
> additional complexity in ordered comparisons of XIDs (which is not a
> really performance-critical task AFAIK), and at the cost that the
> original insertion XIDs of all but recent tuples will be lost by
> VACUUM.  The system doesn't particularly care about that, but old XIDs
> do sometimes come in handy for debugging purposes.  A possible

I wouldn't care about this.

> compromise is to overwrite only XIDs that are older than, say,
> WRAPLIMIT/4 instead of doing so as soon as possible.  This would mean
> the required VACUUM frequency is every WRAPLIMIT/4 xacts instead of
> every WRAPLIMIT/2 xacts.
> 
> We have a straightforward tradeoff between the maximum size of pg_log
> (WRAPLIMIT/4 bytes) and the required frequency of VACUUM (at least

Required frequency of *successful* vacuum over *all* tables.
We would have to remember something in pg_class/pg_database
and somehow force vacuum over "too-long-unvacuumed-tables"
*automatically*.

> every WRAPLIMIT/2 or WRAPLIMIT/4 transactions).  This could be made
> configurable in config.h for those who're intent on customization,
> but I'd be inclined to set the default value at WRAPLIMIT = 1G.
> 
> Comments?  Vadim, is any of this about to be superseded by WAL?
> If not, I'd like to fix it for 7.1.

If undo would be implemented then we could delete pg_log between
postmaster startups - startup counter is remembered in pages, so
seeing old startup id in a page we would know that there are only
long ago committed xactions (ie only visible changes) there
and avoid xid comparison. But ... there will be no undo in 7.1.
And I foresee problems with WAL based BAR implementation if we'll
follow proposed solution: redo restores original xmin/xmax - how
to "freeze" xids while restoring DB?

(Sorry, I have to run away now... and have to think more about issue).

Vadim

From pgsql-hackers-owner+M335@@postgresql.org Mon Nov  6 17:29:50 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id RAA06780
	for <pgman@@candle.pha.pa.us>; Mon, 6 Nov 2000 17:29:49 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA6MSus41571;
	Mon, 6 Nov 2000 17:28:56 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M335@@postgresql.org)
Received: from sectorbase2.sectorbase.com ([208.48.122.131])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA6MPUs41171
	for <pgsql-hackers@@postgresql.org>; Mon, 6 Nov 2000 17:25:30 -0500 (EST)
	(envelope-from vmikheev@@SECTORBASE.COM)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2650.21)
	id <V8XQBHD5>; Mon, 6 Nov 2000 14:08:12 -0800
Message-ID: <8F4C99C66D04D4118F580090272A7A234D314A@@sectorbase1.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@@SECTORBASE.COM>
To: "'mhh@@mindspring.com'" <mhh@@mindspring.com>,
        Tom Lane
  <tgl@@sss.pgh.pa.us>
Cc: pgsql-hackers@@postgresql.org
Subject: RE: [HACKERS] Transaction ID wraparound: problem and proposed sol
	ution
Date: Mon, 6 Nov 2000 14:12:07 -0800 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain;
	charset="iso-8859-1"
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

> > OK, 2^64 isn't mathematically unbounded, but let's see you 
> > buy a disk that will hold it ;-).  My point is that if we want
> > to think about allowing >4G transactions, part of the answer
> > has to be a way to recycle pg_log space. Otherwise it's still
> > not really practical.
> 
> I kind of like vadim's idea of segmenting pg_log. 
> 
> Segments in which all the xacts have been commited could be deleted.

Without undo we have to ensure that all tables are vacuumed after
all transactions related to a segment were committed/aborted.

Vadim

From pgsql-hackers-owner+M235@@postgresql.org Fri Nov  3 21:11:00 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id VAA10173
	for <pgman@@candle.pha.pa.us>; Fri, 3 Nov 2000 21:10:59 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA42A7s33061;
	Fri, 3 Nov 2000 21:10:07 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M235@@postgresql.org)
Received: from acheron.rime.com.au (albatr.lnk.telstra.net [139.130.54.222])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA429Ss32948
	for <pgsql-hackers@@postgreSQL.org>; Fri, 3 Nov 2000 21:09:28 -0500 (EST)
	(envelope-from pjw@@rhyme.com.au)
Received: from oberon (Oberon.rime.com.au [203.8.195.100])
	by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id NAA13631;
	Sat, 4 Nov 2000 13:08:54 +1100
Message-Id: <3.0.5.32.20001104130922.045c3410@@mail.rhyme.com.au>
X-Sender: pjw@@mail.rhyme.com.au
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
Date: Sat, 04 Nov 2000 13:09:22 +1100
To: Tom Lane <tgl@@sss.pgh.pa.us>, pgsql-hackers@@postgresql.org
From: Philip Warner <pjw@@rhyme.com.au>
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed
  solution
In-Reply-To: <8382.973291660@@sss.pgh.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

At 17:47 3/11/00 -0500, Tom Lane wrote:
>* portability --- I don't believe long long int exists on all the
>platforms we support.

Are you sure of this, or is it just a 'last time I looked' statement. If
the latter, it might be worth verifying.


>* performance --- except on true 64-bit platforms, widening Datum to
>eight bytes would be a system-wide performance hit, 

Yes, OIDs are used a lot, but it's not that bad, is it? Are there many
tight loops with thousands of OID-only operations? I'd guess it's only one
more instruction & memory fetch.


>* disk space --- letting pg_log grow without bound isn't a pleasant
>prospect either.

Maybe this can be achieved by wrapping XID for the log file only.


>I believe it is possible to fix these problems without widening XID,
>by redefining XIDs in a way that allows for wraparound.  Here's my
>plan:

It's a cute idea (elegant, even), but maybe we'd be running through hoops
just for a minor performance gain (which may not exist, since we're adding
extra comparisons via the macro) and for possible unsupported OSs. Perhaps
OS's without 8 byte ints have to suffer a performance hit (ie. we declare a
struct with appropriate macros).


>are no longer simply "x < y", but need to be expressed as a macro.
>We consider x < y if (y - x) % WRAPLIMIT < WRAPLIMIT/2.

You mean you plan to limit PGSQL to only 1G concurrent transactions. Isn't
that a bit short sighted? ;-}


>2. To keep the system from having to deal with XIDs that are more than
>WRAPLIMIT/2 transactions old, VACUUM should "freeze" known-good old
>tuples. 

This is a problem for me; it seems to enshrine VACUUM in perpetuity.


>4. With the wraparound behavior, pg_log will have a bounded size: it
>will never exceed WRAPLIMIT*2 bits = WRAPLIMIT/4 bytes.  Since we will
>recycle pg_log entries every WRAPLIMIT xacts, during transaction start

Is there any was we can use this recycling technique with 8-byte XIDs?

Also, will there be a problem with backup programs that use XID to
determine newer records and apply/reapply changes?


>This scheme allows us to survive XID wraparound at the cost of slight
>additional complexity in ordered comparisons of XIDs (which is not a
>really performance-critical task AFAIK)

Maybe I'm really missing the amount of XID manipulation, but I'd be
surprised if 16-byte XIDs would slow things down much.


----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/

From pgsql-hackers-owner+M3501@@postgresql.org Sat Jan 20 03:42:19 2001
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA12652
	for <pgman@@candle.pha.pa.us>; Sat, 20 Jan 2001 03:42:18 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f0K8ZG020426;
	Sat, 20 Jan 2001 03:35:16 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M3501@@postgresql.org)
Received: from store.z.zembu.com (nat.zembu.com [209.128.96.253])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f0K8TU016385
	for <pgsql-hackers@@postgresql.org>; Sat, 20 Jan 2001 03:29:30 -0500 (EST)
	(envelope-from ncm@@zembu.com)
Received: by store.z.zembu.com (Postfix, from userid 509)
	id B33D9A782; Sat, 20 Jan 2001 00:29:24 -0800 (PST)
Date: Sat, 20 Jan 2001 00:29:24 -0800
To: pgsql-hackers@@postgresql.org
Subject: Re: [HACKERS] Transaction ID wraparound: problem and proposed solution
Message-ID: <20010120002924.A2797@@store.zembu.com>
Reply-To: pgsql-hackers@@postgresql.org
References: <8382.973291660@@sss.pgh.pa.us> <200101200500.AAA05265@@candle.pha.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <200101200500.AAA05265@@candle.pha.pa.us>; from pgman@@candle.pha.pa.us on Sat, Jan 20, 2001 at 12:00:09AM -0500
From: ncm@@zembu.com (Nathan Myers)
Precedence: bulk
Sender: pgsql-hackers-owner@@postgresql.org
Status: OR

I think the XID wraparound matter might be handled a bit more simply.

Given a global variable X which is the earliest XID value in use at 
some event (e.g. startup) you can compare two XIDs x and y, using
unsigned arithmetic, with just (x-X < y-X).  This has the further 
advantage that old transaction IDs need be "frozen" only every 4G 
transactions, rather than Tom's suggested 256M or 512M transactions.  
"Freezing", in this scheme, means to set all older XIDs to equal the 
chosen X, rather than setting them to some constant reserved value.  
No special cases are required for the comparison, even for folded 
values; it is (x-X < y-X) for all valid x and y.

I don't know the role of the "bootstrap" XID, or how it must be
fitted into the above.

Nathan Myers
ncm@@zembu.com

------------------------------------------------------------
> We've expended a lot of worry and discussion in the past about what
> happens if the OID generator wraps around.  However, there is another
> 4-byte counter in the system: the transaction ID (XID) generator.
> While OID wraparound is survivable, if XIDs wrap around then we really
> do have a Ragnarok scenario.  The tuple validity checks do ordered
> comparisons on XIDs, and will consider tuples with xmin > current xact
> to be invalid.  Result: after wraparound, your whole database would
> instantly vanish from view.
> 
> The first thought that comes to mind is that XIDs should be promoted to
> eight bytes.  However there are several practical problems with this:
> * portability --- I don't believe long long int exists on all the
> platforms we support.
> * performance --- except on true 64-bit platforms, widening Datum to
> eight bytes would be a system-wide performance hit, which is a tad
> unpleasant to fix a scenario that's not yet been reported from the
> field.
> * disk space --- letting pg_log grow without bound isn't a pleasant
> prospect either.
> 
> I believe it is possible to fix these problems without widening XID,
> by redefining XIDs in a way that allows for wraparound.  Here's my
> plan:
> 
> 1. Allow XIDs to range from 0 to WRAPLIMIT-1 (WRAPLIMIT is not
> necessarily 4G, see discussion below).  Ordered comparisons on XIDs
> are no longer simply "x < y", but need to be expressed as a macro.
> We consider x < y if (y - x) % WRAPLIMIT < WRAPLIMIT/2.
> This comparison will work as long as the range of interesting XIDs
> never exceeds WRAPLIMIT/2.  Essentially, we envision the actual value
> of XID as being the low-order bits of a logical XID that always
> increases, and we assume that no extant XID is more than WRAPLIMIT/2
> transactions old, so we needn't keep track of the high-order bits.
> 
> 2. To keep the system from having to deal with XIDs that are more than
> WRAPLIMIT/2 transactions old, VACUUM should "freeze" known-good old
> tuples.  To do this, we'll reserve a special XID, say 1, that is always
> considered committed and is always less than any ordinary XID.  (So the
> ordered-comparison macro is really a little more complicated than I said
> above.  Note that there is already a reserved XID just like this in the
> system, the "bootstrap" XID.  We could simply use the bootstrap XID, but
> it seems better to make another one.)  When VACUUM finds a tuple that
> is committed good and has xmin < XmaxRecent (the oldest XID that might
> be considered uncommitted by any open transaction), it will replace that
> tuple's xmin by the special always-good XID.  Therefore, as long as
> VACUUM is run on all tables in the installation more often than once per
> WRAPLIMIT/2 transactions, there will be no tuples with ordinary XIDs
> older than WRAPLIMIT/2.
> 
> 3. At wraparound, the XID counter has to be advanced to skip over the
> InvalidXID value (zero) and the reserved XIDs, so that no real transaction
> is generated with those XIDs.  No biggie here.
> 
> 4. With the wraparound behavior, pg_log will have a bounded size: it
> will never exceed WRAPLIMIT*2 bits = WRAPLIMIT/4 bytes.  Since we will
> recycle pg_log entries every WRAPLIMIT xacts, during transaction start
> the xact manager will have to take care to actively clear its pg_log
> entry to zeroes (I'm not sure if it does that already, or just assumes
> that new pg_log entries will start out zero).  As long as that happens
> before the xact makes any data changes, it's OK to recycle the entry.
> Note we are assuming that no tuples will remain in the database with
> xmin or xmax equal to that XID from a prior cycle of the universe.
> 
> This scheme allows us to survive XID wraparound at the cost of slight
> additional complexity in ordered comparisons of XIDs (which is not a
> really performance-critical task AFAIK), and at the cost that the
> original insertion XIDs of all but recent tuples will be lost by
> VACUUM.  The system doesn't particularly care about that, but old XIDs
> do sometimes come in handy for debugging purposes.  A possible
> compromise is to overwrite only XIDs that are older than, say,
> WRAPLIMIT/4 instead of doing so as soon as possible.  This would mean
> the required VACUUM frequency is every WRAPLIMIT/4 xacts instead of
> every WRAPLIMIT/2 xacts.
> 
> We have a straightforward tradeoff between the maximum size of pg_log
> (WRAPLIMIT/4 bytes) and the required frequency of VACUUM (at least
> every WRAPLIMIT/2 or WRAPLIMIT/4 transactions).  This could be made
> configurable in config.h for those who're intent on customization,
> but I'd be inclined to set the default value at WRAPLIMIT = 1G.
> 
> Comments?  Vadim, is any of this about to be superseded by WAL?
> If not, I'd like to fix it for 7.1.
> 
> 			regards, tom lane

d27 1
a27 1
Status: OR
d59 11
a69 12
From pgsql-hackers-owner+M11790@@postgresql.org Sun Aug  5 14:41:34 2001
Return-path: <pgsql-hackers-owner+M11790@@postgresql.org>
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f75IfXh25356
	for <pgman@@candle.pha.pa.us>; Sun, 5 Aug 2001 14:41:33 -0400 (EDT)
Received: from postgresql.org.org (webmail.postgresql.org [216.126.85.28])
	by postgresql.org (8.11.3/8.11.4) with SMTP id f75IfY644815;
	Sun, 5 Aug 2001 14:41:34 -0400 (EDT)
	(envelope-from pgsql-hackers-owner+M11790@@postgresql.org)
Received: from candle.pha.pa.us (candle.navpoint.com [162.33.245.46])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f75IUs641174
	for <pgsql-hackers@@postgresql.org>; Sun, 5 Aug 2001 14:30:54 -0400 (EDT)
d72 2
a73 2
	by candle.pha.pa.us (8.10.1/8.10.1) id f75IUhM25071;
	Sun, 5 Aug 2001 14:30:43 -0400 (EDT)
d75 5
a79 8
Message-ID: <200108051830.f75IUhM25071@@candle.pha.pa.us>
Subject: Re: [HACKERS] Idea for nested transactions / savepoints
In-Reply-To: <8173.997022088@@sss.pgh.pa.us> "from Tom Lane at Aug 5, 2001 10:34:48
	am"
To: Tom Lane <tgl@@sss.pgh.pa.us>
Date: Sun, 5 Aug 2001 14:30:43 -0400 (EDT)
cc: PostgreSQL-development <pgsql-hackers@@postgresql.org>
X-Mailer: ELM [version 2.4ME+ PL90 (25)]
d85 1
a85 10
Status: OR

> Bruce Momjian <pgman@@candle.pha.pa.us> writes:
> > My idea is that we not put UNDO information into WAL but keep a List of
> > rel ids / tuple ids in the memory of each backend and do the undo inside
> > the backend.
> 
> The complaints about WAL size amount to "we don't have the disk space
> to keep track of this, for long-running transactions".  If it doesn't
> fit on disk, how likely is it that it will fit in memory?
d87 33
a119 10
Sure, we can put on the disk if that is better.  I thought the problem
with WAL undo is that you have to keep UNDO info around for all
transactions that are older than the earliest transaction.  So, if I
start a nested transaction, and then sit at a prompt for 8 hours, all
WAL logs are kept for 8 hours.

We can create a WAL file for every backend, and record just the nested
transaction information.  In fact, once a nested transaction finishes,
we don't need the info anymore.  Certainly we don't need to flush these
to disk.
d128 1
a128 1
TIP 1: subscribe and unsubscribe commands go to majordomo@@postgresql.org
d130 1
a130 67
From pgman Sun Aug  5 21:16:32 2001
Return-path: <pgman>
Received: (from pgman@@localhost)
	by candle.pha.pa.us (8.10.1/8.10.1) id f761GWH11356;
	Sun, 5 Aug 2001 21:16:32 -0400 (EDT)
From: Bruce Momjian <pgman>
Message-ID: <200108060116.f761GWH11356@@candle.pha.pa.us>
Subject: Re: [HACKERS] Idea for nested transactions / savepoints
In-Reply-To: <200108051938.f75Jchi27522@@candle.pha.pa.us> "from Bruce Momjian
	at Aug 5, 2001 03:38:43 pm"
To: Bruce Momjian <pgman@@candle.pha.pa.us>
Date: Sun, 5 Aug 2001 21:16:32 -0400 (EDT)
cc: Tom Lane <tgl@@sss.pgh.pa.us>,
   PostgreSQL-development <pgsql-hackers@@postgresql.org>
X-Mailer: ELM [version 2.4ME+ PL90 (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
Status: OR

> > Bruce Momjian <pgman@@candle.pha.pa.us> writes:
> > >> The complaints about WAL size amount to "we don't have the disk space
> > >> to keep track of this, for long-running transactions".  If it doesn't
> > >> fit on disk, how likely is it that it will fit in memory?
> > 
> > > Sure, we can put on the disk if that is better.
> > 
> > I think you missed my point.  Unless something can be done to make the
> > log info a lot smaller than it is now, keeping it all around until
> > transaction end is just not pleasant.  Waving your hands and saying
> > that we'll keep it in a different place doesn't affect the fundamental
> > problem: if the transaction runs a long time, the log is too darn big.
> 
> When you said long running, I thought you were concerned about long
> running in duration, not large transaction.  Long duration in one-WAL
> setup would cause all transaction logs to be kept.  Large transactions
> are another issue.
> 
> One solution may be to store just the relid if many tuples are modified
> in the same table.  If you stored the command counter for start/end of
> the nested transaction, it would be possible to sequential scan the
> table and undo all the affected tuples.  Does that help?  Again, I am
> just throwing out ideas here, hoping something will catch.

Actually, we need to keep around nested transaction UNDO information
only until the nested transaction exits to the main transaction:

	BEGIN WORK;
		BEGIN WORK;
		COMMIT;
		-- we can throw away the UNDO here
		BEGIN WORK;
			BEGIN WORK;
			...
			COMMIT
		COMMIT;
		-- we can throw away the UNDO here
	COMMIT;

We are using the outside transaction for our ACID capabilities, and just
using UNDO for nested transaction capability.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
@


1.2
log
@Add.
@
text
@d1056 187
@


1.1
log
@Add emails about transaction rollover.
@
text
@d922 134
@
