Обсуждение: Beta4 Tag'd and Bundled ...

Поиск
Список
Период
Сортировка

Beta4 Tag'd and Bundled ...

От
"Marc G. Fournier"
Дата:
Check her over and let me know if there are any problems ... will do a
full general announce tomorrow for it ...


Re: Beta4 Tag'd and Bundled ...

От
Adam Witney
Дата:
Hi,

Many of the regression tests are failing on my OSX 10.2.6 machine. I have
put the regression.diffs file here

http://bugs.sghms.ac.uk/downloads/regression.diffs

Has this been seen before?

Thanks

adam

> Check her over and let me know if there are any problems ... will do a
> full general announce tomorrow for it ...
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



Re: Beta4 Tag'd and Bundled ...

От
Tom Lane
Дата:
Adam Witney <awitney@sghms.ac.uk> writes:
> Many of the regression tests are failing on my OSX 10.2.6 machine. I have
> put the regression.diffs file here
> http://bugs.sghms.ac.uk/downloads/regression.diffs

Seems to be all

! psql: FATAL:  sorry, too many clients already

What did initdb set your max_connections to?
        regards, tom lane


Re: Beta4 Tag'd and Bundled ...

От
Adam Witney
Дата:
On 4/10/03 5:16 pm, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:

> Adam Witney <awitney@sghms.ac.uk> writes:
>> Many of the regression tests are failing on my OSX 10.2.6 machine. I have
>> put the regression.diffs file here
>> http://bugs.sghms.ac.uk/downloads/regression.diffs
> 
> Seems to be all
> 
> ! psql: FATAL:  sorry, too many clients already
> 
> What did initdb set your max_connections to?
> 
> regards, tom lane

From src/test/regress/log/initdb.log

selecting default max_connections... 10

Is this the info you are asking for?

adam



Re: Beta4 Tag'd and Bundled ...

От
Tom Lane
Дата:
Adam Witney <awitney@sghms.ac.uk> writes:
> On 4/10/03 5:16 pm, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
>> What did initdb set your max_connections to?

> From src/test/regress/log/initdb.log
> selecting default max_connections... 10

Hm.  The parallel regression tests require at least 20.  I deliberately
allowed initdb to select values as small as 10 on the theory that
installing and not being able to run the parallel regression tests is
better than not installing at all.  Does anyone want to argue the
opposite?  Perhaps we just need a note in the documentation --- but if
so, where?
        regards, tom lane


Re: Beta4 Tag'd and Bundled ...

От
Rod Taylor
Дата:
> Hm.  The parallel regression tests require at least 20.  I deliberately
> allowed initdb to select values as small as 10 on the theory that
> installing and not being able to run the parallel regression tests is
> better than not installing at all.  Does anyone want to argue the
> opposite?  Perhaps we just need a note in the documentation --- but if
> so, where?

Another alternative is to have the regression suite discover the max
connections, and defer tests when there are (max_connections - 1)
connections already.

Re: Beta4 Tag'd and Bundled ...

От
Tom Lane
Дата:
Rod Taylor <rbt@rbt.ca> writes:
>> Hm.  The parallel regression tests require at least 20.  I deliberately
>> allowed initdb to select values as small as 10 on the theory that
>> installing and not being able to run the parallel regression tests is
>> better than not installing at all.

> Another alternative is to have the regression suite discover the max
> connections, and defer tests when there are (max_connections - 1)
> connections already.

Maybe.  After thinking a bit more, I seem to recall one of the reasons
for having wide parallel sets in the regression tests is that we
*wanted* to consider inability to support a dozen or two connections as
a serious problem.  If we still believe that old logic, then indeed the
right thing to do is for initdb to insist on setting max_connections no
smaller than 20.  (Pre-7.4, the default setting was generally 32, so
this is still more flexible than before from a portability standpoint.)
        regards, tom lane


max_connections/shared_buffers (was Re: Beta4 Tag'd and Bundled ...)

От
Tom Lane
Дата:
I said:
> Hm.  The parallel regression tests require at least 20.  I deliberately
> allowed initdb to select values as small as 10 on the theory that
> installing and not being able to run the parallel regression tests is
> better than not installing at all.

Actually, after trying to reproduce the problem on my own OS X machine,
I realize that it's a little more subtle than I thought.  The Darwin
port does not use SysV semaphores at all (it uses Posix semaphores) and
so the resource restriction you're hitting must actually be the
max-shared-memory limit, rather than number-of-semaphores as I first
assumed.  max_connections does have an impact on shared memory size,
though not as large as shared_buffers has.

Therefore, the real problem is that initdb initially probes for a
workable number of shared_buffers while using max_connections=5, and
then it selects max_connections while holding shared_buffers constant.
I was thinking that a small max_connections would prevent us from
mistakenly choosing tiny shared_buffers when the system's real
restriction is on number of semaphores.  But what we seem to have here
is that the selected number of buffers was just a little under the
system's max-shared-memory limit, so that max_connections could be
raised to 10 but not to 20.

(BTW, on my OS X machine, with out-of-the-box configuration, initdb
selects shared_buffers 400 and max_connections 20.  I'm guessing that
you had either a nondefault shared memory limit, or some other process
using shared memory.)

Perhaps we should avoid all attempts at cuteness and just run the
initial probes for workable shared_buffers with max_connections=20,
as well as making that be the minimum max_connections value probed for.

Anyone see a better way?
        regards, tom lane


Re: max_connections/shared_buffers (was Re: Beta4 Tag'd and Bundled ...)

От
Greg Stark
Дата:
Tom Lane <tgl@sss.pgh.pa.us> writes:

> (BTW, on my OS X machine, with out-of-the-box configuration, initdb
> selects shared_buffers 400 and max_connections 20.  I'm guessing that
> you had either a nondefault shared memory limit, or some other process
> using shared memory.)

This points out another issue with this approach of probing for the maximum
shared memory. There might be another program using shared memory when the
probe is done -- or worse when the database is started but *not* when the
probe is run.

Perhaps the shared_buffers should only be set to 50% of the maximum size
probed? That would a) give postgres a decent chance of starting even if some
other program uses some amount of the shared memory between initdb and
database starting. and b) leave enough memory for a reasonable
max_connections?

> Anyone see a better way?

Switch everything to mmap and pthreads and dump all this antiquated SysV IPC
and semaphore junk? *DUCK*

-- 
greg



Re: max_connections/shared_buffers (was Re: Beta4 Tag'd and Bundled ...)

От
Tom Lane
Дата:
Greg Stark <gsstark@mit.edu> writes:
> Perhaps the shared_buffers should only be set to 50% of the maximum size
> probed?

I think it's reasonable to expect the DBA to make any adjustments needed
for changes in environment.  Partly this is because I don't see any
defensible way to do otherwise --- your 50% slop figure is without
foundation in terms of what might really be going on --- and partly
because we'd be handicapping ourselves unnecessarily if there *aren't*
any subsequent changes in environment.

On machines where shared memory actually gets used for anything by
default, I think that the default limits are likely to be fairly sane.
If shared memory is tight, then very likely Postgres is the only thing
on the machine that's going to want it.  We might as well use what we
can get.
        regards, tom lane


Re: max_connections/shared_buffers (was Re: Beta4 Tag'd

От
"Joshua D. Drake"
Дата:

>>Anyone see a better way?
>>    
>>
>
>Switch everything to mmap and pthreads and dump all this antiquated SysV IPC
>and semaphore junk? *DUCK*
>  
>

You are a brave soult. I salute you.

Sincerely,

Joshua D. Drake



-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC - S/JDBC
Postgresql support, programming, shared hosting and dedicated hosting.
+1-503-222-2783 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL.Org - Editor-N-Chief - http://www.postgresql.org




Re: max_connections/shared_buffers (was Re: Beta4 Tag'd and

От
Adam Witney
Дата:
On 4/10/03 8:10 pm, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:

> I said:
>> Hm.  The parallel regression tests require at least 20.  I deliberately
>> allowed initdb to select values as small as 10 on the theory that
>> installing and not being able to run the parallel regression tests is
>> better than not installing at all.
> 
> Actually, after trying to reproduce the problem on my own OS X machine,
> I realize that it's a little more subtle than I thought.  The Darwin
> port does not use SysV semaphores at all (it uses Posix semaphores) and
> so the resource restriction you're hitting must actually be the
> max-shared-memory limit, rather than number-of-semaphores as I first
> assumed.  max_connections does have an impact on shared memory size,
> though not as large as shared_buffers has.
> 
> Therefore, the real problem is that initdb initially probes for a
> workable number of shared_buffers while using max_connections=5, and
> then it selects max_connections while holding shared_buffers constant.
> I was thinking that a small max_connections would prevent us from
> mistakenly choosing tiny shared_buffers when the system's real
> restriction is on number of semaphores.  But what we seem to have here
> is that the selected number of buffers was just a little under the
> system's max-shared-memory limit, so that max_connections could be
> raised to 10 but not to 20.
> 
> (BTW, on my OS X machine, with out-of-the-box configuration, initdb
> selects shared_buffers 400 and max_connections 20.  I'm guessing that
> you had either a nondefault shared memory limit, or some other process
> using shared memory.)

These are my current settings
   sysctl -w kern.sysv.shmmax=4194304   sysctl -w kern.sysv.shmmin=1   sysctl -w kern.sysv.shmmni=32   sysctl -w
kern.sysv.shmseg=8  sysctl -w kern.sysv.shmall=1024
 

This is a laptop so I have quite a few other apps running, including my
current PostgreSQL installation (7.2.3), the settings of which are

#max_connections = 32
shared_buffers = 100


Let me know if you need any more info?

Thanks

Adam


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



Re: max_connections/shared_buffers (was Re: Beta4 Tag'd

От
Peter Eisentraut
Дата:
Tom Lane writes:

> Perhaps we should avoid all attempts at cuteness and just run the
> initial probes for workable shared_buffers with max_connections=20,
> as well as making that be the minimum max_connections value probed for.
>
> Anyone see a better way?

Maybe just run one loop and try pairs of (shared_buffers, max_connections):

(1000, 100) (800, 50) (600, 40) (400, 30) (200, 20) (50, 10)

-- 
Peter Eisentraut   peter_e@gmx.net



Re: max_connections/shared_buffers (was Re: Beta4 Tag'd and Bundled ...)

От
Tom Lane
Дата:
Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane writes:
>> Anyone see a better way?

> Maybe just run one loop and try pairs of (shared_buffers, max_connections):
> (1000, 100) (800, 50) (600, 40) (400, 30) (200, 20) (50, 10)

Hmm ... that wouldn't work real well as-is, because if max_connections
is being constrained by a limit on SysV semaphores, we could find
ourselves restricting shared_buffers to no purpose.

Would it work to loop as above, and then try a second loop in which we
use the selected max_connections and see if we can make shared_buffers
bigger?
        regards, tom lane


Re: Beta4 Tag'd and Bundled ...

От
Adam Witney
Дата:
Hi,

Many of the regression tests are failing on my OSX 10.2.6 machine. I have
put the regression.diffs file here

http://bugs.sghms.ac.uk/downloads/regression.diffs

Has this been seen before?

Thanks

adam

> Check her over and let me know if there are any problems ... will do a
> full general announce tomorrow for it ...
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.