Обсуждение: postgresql-v6.5beta2.tar.gz ...
I am just creating this now...this is to be considered the first *releaes*
candidate for v6.5 ...
I expect there to be *no more* commits to the source tree until after a
formal release is done...
Baring any bugs, this will become the formal release as of 15:30EST on
Wednesday, giving two days for ppl to run her through and make sure
nothing obvious is outstanding.
Consider the cvs repository *frozen* until I announce otherwise, as I will
once more tag things so that we have a -CURRENT vs -STABLE source tree to
work with.
There are to be *no* code changes between now and then, except by having
such changes reviewed by one other committer...about the only thing I'd
expect to see is something port specific...
At this time, I hope everyone will take a second to download the above
.tar.gz file and run things through to make sure there aren't any
outstanding problems ...
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
The Hermit Hacker wrote: > > I am just creating this now...this is to be considered the first *releaes* > candidate for v6.5 ... > > I expect there to be *no more* commits to the source tree until after a > formal release is done... Ah. I wanted to implement proper handling of case when tuple in update-chain is updated by "too old" xaction (part of chain should be deleted) but ok - it's not bug, just unimplemented feature. (BTW, Mark, I have to update lock.sgml) Hiroshi, please run your test. You'll get NOTICEs instead of ERRORs. Unfortunately, I can't get your test finished locally - I always get core and gdb says too little: #0 0x2823452f in strcpy () (gdb) bt #0 0x2823452f in strcpy () #1 0xefbf8944 in ?? () #2 0x812d19d in hash_search (hashp=0x1c518bfc, keyPtr=0x8901428d <Address 0x8901428d outof bounds>, action=1233853505, foundPtr=0x4d89410c <Address 0x4d89410c out of bounds>) at dynahash.c:651 #3 0x4d8b0000 in ?? () Cannot access memory at address 0x3c7. (gdb) f 2 #2 0x812cfbd in hash_search (hashp=0x1c518bfc, keyPtr=0x8901428d <Address 0x8901428d outof bounds>, action=1233853505, foundPtr=0x4d89410c <Address 0x4d89410c out of bounds>) at dynahash.c:651 651 memmove(destAddr, keyPtr, hctl->keysize); ^^^^^^^ ??? memmove shouldn't call strcpy. Something is broken, but maybe due to FreeBSD 3.0 on my box... I'll run test on 2.2.6 soon. Vadim
On Mon, 7 Jun 1999, Vadim Mikheev wrote:
> The Hermit Hacker wrote:
> >
> > I am just creating this now...this is to be considered the first *releaes*
> > candidate for v6.5 ...
> >
> > I expect there to be *no more* commits to the source tree until after a
> > formal release is done...
>
> Ah. I wanted to implement proper handling of case when tuple in
> update-chain is updated by "too old" xaction (part of chain should
> be deleted) but ok - it's not bug, just unimplemented feature.
Can that wait until *after* 6.5 is released, for inclusion on v6.5.1?
> (BTW, Mark, I have to update lock.sgml)
I have no major qualms about docs, if thomas doesn't ...
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
> -----Original Message----- > From: owner-pgsql-hackers@postgreSQL.org > [mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Vadim Mikheev > Sent: Monday, June 07, 1999 1:12 PM > To: The Hermit Hacker > Cc: pgsql-hackers@postgreSQL.org > Subject: Re: [HACKERS] postgresql-v6.5beta2.tar.gz ... > > > The Hermit Hacker wrote: > > > > I am just creating this now...this is to be considered the > first *releaes* > > candidate for v6.5 ... > > > > I expect there to be *no more* commits to the source tree until after a > > formal release is done... > > Ah. I wanted to implement proper handling of case when tuple in > update-chain is updated by "too old" xaction (part of chain should > be deleted) but ok - it's not bug, just unimplemented feature. > > (BTW, Mark, I have to update lock.sgml) > > Hiroshi, please run your test. You'll get NOTICEs instead of ERRORs. I have run my test and got the following NOTICEs. NOTICE: Child itemid in update-chain marked as unused - can't continue vc_rpfheap Thanks. Hiroshi Inoue Inoue@tpf.co.jp
Hiroshi Inoue wrote: > > > > > Hiroshi, please run your test. You'll get NOTICEs instead of ERRORs. > > I have run my test and got the following NOTICEs. > > NOTICE: Child itemid in update-chain marked as unused - can't > continue vc_rpfheap I'm still getting troubles when running your test due to 1. bug in cache invalidation code: when we invalidate relcache we forget to free MdfdVec in md.c! Vacuum invalidates a relation tuple in pg_class and concurrent xactions invalidate corresponding relcache entry, but don't free MdfdVec and so allocate new one for the same relation more and more. Each MdfdVed requires own fd.c:Vfd entry-> below 2. fd.c:pg_nofile()->sysconf(_SC_OPEN_MAX) returns in FreeBSD near total number of files that can be opened in system (by _all_ users/procs). With total number of opened files ~ 2000 I can run your test with 10-20 simultaneous xactions forvery short time, -:) Should we limit fd.c:no_files to ~ 256? This is port-specific, of course... Vadim P.S. Hiroshi test I run: 10-20 psql < file with 10000 xactions: BEGIN; UPDATE single row in table; END; A script run "vacuum table" each rand(10) + 1 sec.
> > Ah. I wanted to implement proper handling of case when tuple in > > update-chain is updated by "too old" xaction (part of chain should > > be deleted) but ok - it's not bug, just unimplemented feature. > > Can that wait until *after* 6.5 is released, for inclusion on v6.5.1? > > > (BTW, Mark, I have to update lock.sgml) > > I have no major qualms about docs, if thomas doesn't ... No. If you want it, do it now, not in 6.5.1. Adding features in minor releases is worse than adding stuff now. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Mon, 7 Jun 1999, Bruce Momjian wrote:
> > > Ah. I wanted to implement proper handling of case when tuple in
> > > update-chain is updated by "too old" xaction (part of chain should
> > > be deleted) but ok - it's not bug, just unimplemented feature.
> >
> > Can that wait until *after* 6.5 is released, for inclusion on v6.5.1?
> >
> > > (BTW, Mark, I have to update lock.sgml)
> >
> > I have no major qualms about docs, if thomas doesn't ...
>
> No. If you want it, do it now, not in 6.5.1. Adding features in
> minor releases is worse than adding stuff now.
With Vadim himself uncertain as to the effect of doing the patch, then it
should bve left for 6.6 then...
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
> -----Original Message----- > From: root@sunpine.krs.ru [mailto:root@sunpine.krs.ru]On Behalf Of Vadim > Mikheev > Sent: Monday, June 07, 1999 7:49 PM > To: Hiroshi Inoue > Cc: The Hermit Hacker; pgsql-hackers@postgreSQL.org > Subject: Re: [HACKERS] postgresql-v6.5beta2.tar.gz ... > > > Hiroshi Inoue wrote: > > > > > > > > Hiroshi, please run your test. You'll get NOTICEs instead of ERRORs. > > > > I have run my test and got the following NOTICEs. > > > > NOTICE: Child itemid in update-chain marked as unused - can't > > continue vc_rpfheap > > I'm still getting troubles when running your test due to > > 1. bug in cache invalidation code: when we invalidate relcache > we forget to free MdfdVec in md.c! > > Vacuum invalidates a relation tuple in pg_class and concurrent > xactions invalidate corresponding relcache entry, but don't > free MdfdVec and so allocate new one for the same relation I have noticed the same thing But at that point my concern was truncation of segmented relations. AFAIC concurrent xactions invalidate relcache entry when StartTransaction() or Command- CounterIncrement() is called. Unfortunately vacuum couldn't stop other sessions from executing StartTransaction() nor Command- CounterIncrement(). > more and more. Each MdfdVed requires own fd.c:Vfd entry -> below > > 2. fd.c:pg_nofile()->sysconf(_SC_OPEN_MAX) returns in FreeBSD > near total number of files that can be opened in system > (by _all_ users/procs). With total number of opened files > ~ 2000 I can run your test with 10-20 simultaneous > xactions for very short time, -:) > I have remembered another thing. Someone has already reported to ML in Japan and I was able to find the cause easily thanks to him. vacuum calls open() directly to create pg_vlock file. Sometimes I was not able to open the file because of the lack of file descriptors(). FileNameOpenFile()/FileClose() etc should be called instead of open()/close() etc ? Regards. Hiroshi Inoue Inoue@tpf.co.jp
>vacuum calls open() directly to create pg_vlock file. >Sometimes I was not able to open the file because of the lack of >file descriptors(). >FileNameOpenFile()/FileClose() etc should be called instead of >open()/close() etc ? Thanks Hiroshi! I forgot to forward his message regarding pg_vlock issue. Also note that there are some codes that directly open files in createdb/destroydb stuffs... -- Tatsuo Ishii
"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> vacuum calls open() directly to create pg_vlock file.
> Sometimes I was not able to open the file because of the lack of
> file descriptors().
> FileNameOpenFile()/FileClose() etc should be called instead of
> open()/close() etc ?
That is a good point, but actually I think it should not be a problem.
vacuum.c just does open() and immediately close() on pg_vlock; it
doesn't hold the file descriptor open.
When I fixed psort.c a few weeks ago, I looked through all the other
direct calls of open() and fopen() in the backend. There are still half
a dozen or so, but none of them open more than one file or will hold the
file descriptor for longer than the execution of the function they're
in. So I felt it was OK to leave them alone.
The reason it's OK is that fd.c doesn't use up all the available
file descriptors --- it tries to leave ten or so unused. That's
necessary to ensure that library functions like dlopen() will work,
because they don't know anything about using fd.c's routines.
So, the occasional short-term file opening in vacuum.c and similar
places should not matter. If those do fail for lack of FDs, then the
*real* problem is that fd.c is not estimating correctly how many file
descriptors it can safely use; that's what we need to fix.
But what I want to know right now is whether this behavior has been
seen with code from the last week or two. Maybe the report is just
a side-effect of the FD leaks that used to exist in several places...
regards, tom lane
>That is a good point, but actually I think it should not be a problem. >vacuum.c just does open() and immediately close() on pg_vlock; it >doesn't hold the file descriptor open. > >When I fixed psort.c a few weeks ago, I looked through all the other >direct calls of open() and fopen() in the backend. There are still half >a dozen or so, but none of them open more than one file or will hold the >file descriptor for longer than the execution of the function they're >in. So I felt it was OK to leave them alone. > >The reason it's OK is that fd.c doesn't use up all the available >file descriptors --- it tries to leave ten or so unused. That's >necessary to ensure that library functions like dlopen() will work, >because they don't know anything about using fd.c's routines. > >So, the occasional short-term file opening in vacuum.c and similar >places should not matter. If those do fail for lack of FDs, then the >*real* problem is that fd.c is not estimating correctly how many file >descriptors it can safely use; that's what we need to fix. > >But what I want to know right now is whether this behavior has been >seen with code from the last week or two. Maybe the report is just >a side-effect of the FD leaks that used to exist in several places... If I correctly remember, the report was regarding 6.4.2. I will check if it happens with current, and report back soon. --- Tatsuo Ishii
> -----Original Message----- > From: owner-pgsql-hackers@postgreSQL.org > [mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Tom Lane > Sent: Tuesday, June 08, 1999 11:34 AM > To: pgsql-hackers@postgreSQL.org > Subject: Re: [HACKERS] postgresql-v6.5beta2.tar.gz ... > > > "Hiroshi Inoue" <Inoue@tpf.co.jp> writes: > > vacuum calls open() directly to create pg_vlock file. > > Sometimes I was not able to open the file because of the lack of > > file descriptors(). > > FileNameOpenFile()/FileClose() etc should be called instead of > > open()/close() etc ? > > That is a good point, but actually I think it should not be a problem. > vacuum.c just does open() and immediately close() on pg_vlock; it > doesn't hold the file descriptor open. > > When I fixed psort.c a few weeks ago, I looked through all the other > direct calls of open() and fopen() in the backend. There are still half > a dozen or so, but none of them open more than one file or will hold the > file descriptor for longer than the execution of the function they're > in. So I felt it was OK to leave them alone. > > The reason it's OK is that fd.c doesn't use up all the available > file descriptors --- it tries to leave ten or so unused. That's > necessary to ensure that library functions like dlopen() will work, > because they don't know anything about using fd.c's routines. > > So, the occasional short-term file opening in vacuum.c and similar > places should not matter. If those do fail for lack of FDs, then the > *real* problem is that fd.c is not estimating correctly how many file > descriptors it can safely use; that's what we need to fix. > > But what I want to know right now is whether this behavior has been > seen with code from the last week or two. Maybe the report is just > a side-effect of the FD leaks that used to exist in several places... > I don't remember correctly how the phenomenon occured,sorry. I couldn't reproduce it now. Regards. Hiroshi Inoue Inoue@tpf.co.jp
>>That is a good point, but actually I think it should not be a problem. >>vacuum.c just does open() and immediately close() on pg_vlock; it >>doesn't hold the file descriptor open. >> >>When I fixed psort.c a few weeks ago, I looked through all the other >>direct calls of open() and fopen() in the backend. There are still half >>a dozen or so, but none of them open more than one file or will hold the >>file descriptor for longer than the execution of the function they're >>in. So I felt it was OK to leave them alone. >> >>The reason it's OK is that fd.c doesn't use up all the available >>file descriptors --- it tries to leave ten or so unused. That's >>necessary to ensure that library functions like dlopen() will work, >>because they don't know anything about using fd.c's routines. >> >>So, the occasional short-term file opening in vacuum.c and similar >>places should not matter. If those do fail for lack of FDs, then the >>*real* problem is that fd.c is not estimating correctly how many file >>descriptors it can safely use; that's what we need to fix. >> >>But what I want to know right now is whether this behavior has been >>seen with code from the last week or two. Maybe the report is just >>a side-effect of the FD leaks that used to exist in several places... > >If I correctly remember, the report was regarding 6.4.2. I will check >if it happens with current, and report back soon. Done. With 32/64 users and 100 transactions each, I see no problem so far. -- Tatsuo Ishii