Обсуждение: Asynchronous connect using libpq
Hello all, I have just today re-subscribed to this list after a few months away. I hope that I am not repeating a recent question; I have searched the archives and found nothing appropriate. I need non-blocking versions of PQconnectdb and PQreset in libpq, analogous to the asynchronous query processing functions (PQsendQuery et al). I am willing to write these, if no one else has time to do so, but would appreciate some input before I get started. I suspect it can be split in such a way that the caller will be able to use select, PQconsumeInput, PQisBusy, etc, like the async query stuff. I am in quite a rush for these, and a prompt response from someone in the know would be greatly appreciated. TIA, Ewan Mellor.
eem21@cam.ac.uk writes:
> I need non-blocking versions of PQconnectdb and PQreset in libpq,
> analogous to the asynchronous query processing functions (PQsendQuery
> et al).
Gonna have to write them yourself, I'm afraid. PQconnect is actually
a rather long sequence, involving completing an authentication protocol
with the server and then sending a few standard setup commands.
See libpq/fe-connect.c.
PQreset just closes the connection and then invokes PQconnect.
> I am willing to write these, if no one else has time to do so, but would
> appreciate some input before I get started.
For maintainability, there'd need to be just one copy of the setup code
not two, so you'd want to do something similar to the way I implemented
the async query processing: write the async code and then replace the
synchronous routines with simple loops that call the async version.
Turning the bottom part of connectDB() plus PQsetenv() into a state
machine would be a fairly straightforward way of making this happen.
connectDB is nearly there already, you'd just need to move the loop
up to its caller.
Probably the most serious problem you'd have to consider before diving
into this is that on most platforms, both gethostbyname() and connect()
can involve nontrivial delays, and I know of no portable way to replace
them with asynchronous operations. If you can live with a DNS lookup
delay then you can probably live with the other delays, and if you can't
then it's not clear you would have a workable solution when you got done
anyway :-(.
regards, tom lane
On 18 Jul, Tom Lane wrote: > eem21@cam.ac.uk writes: >> I need non-blocking versions of PQconnectdb and PQreset in libpq, >> analogous to the asynchronous query processing functions (PQsendQuery >> et al). > > Gonna have to write them yourself, I'm afraid. PQconnect is actually > a rather long sequence, involving completing an authentication protocol > with the server and then sending a few standard setup commands. > See libpq/fe-connect.c. > > PQreset just closes the connection and then invokes PQconnect. > >> I am willing to write these, if no one else has time to do so, but would >> appreciate some input before I get started. > > For maintainability, there'd need to be just one copy of the setup code > not two, so you'd want to do something similar to the way I implemented > the async query processing: write the async code and then replace the > synchronous routines with simple loops that call the async version. > > Turning the bottom part of connectDB() plus PQsetenv() into a state > machine would be a fairly straightforward way of making this happen. > connectDB is nearly there already, you'd just need to move the loop > up to its caller. That all sounds fine. > Probably the most serious problem you'd have to consider before diving > into this is that on most platforms, both gethostbyname() and connect() > can involve nontrivial delays, and I know of no portable way to replace > them with asynchronous operations. If you can live with a DNS lookup > delay then you can probably live with the other delays, and if you can't > then it's not clear you would have a workable solution when you got done > anyway :-(. I realised that some time after I sent my first message, and have been thinking about it since. The delay for gethostbyname is not acceptable, and we are going to have to get around that problem elsewhere in the application anyway. Therefore, I was thinking that I could arrange to pass an IP address into libpq. This would eliminate the host look-up, and we can worry about the portability of our own software, leaving libpq free from such trouble. With regards to the connect() problem, I was hoping that by setting the socket to non-blocking mode before connecting (exactly as it is currently done after the connect call) I could solve that problem. The behaviour of a non-blocking socket on connection is documented in my glibc 2.0 docs, and is supposed to be portable to SVr4 and 4.4BSD. I hope to be able to avoid breaking systems that don't handle such a setup appropriately, by continuing to handle the case that the connection is completed on completion of the initial connect() call. Of course, the caveat there is that the function loses its non-blocking properties, but at least nothing is broken. I don't know anything about the behaviour of WinSock, but I should be able simply not to break anything ;-) Ewan Mellor.
eem21@cam.ac.uk writes:
> I realised that some time after I sent my first message, and have been
> thinking about it since. The delay for gethostbyname is not
> acceptable, and we are going to have to get around that problem
> elsewhere in the application anyway. Therefore, I was thinking that I
> could arrange to pass an IP address into libpq.
A dotted-decimal address, you mean? Yeah, that sounds like it should
work.
> With regards to the connect() problem, I was hoping that by setting the
> socket to non-blocking mode before connecting (exactly as it is
> currently done after the connect call) I could solve that problem.
By golly, there is something about non-blocking connect in the man page:
If the socket is of type SOCK_STREAM, connect() attempts to contact the remote host in order to make a
connectionbetween the remote socket (peer) and the local socket specified by s. The call normally blocks until
theconnection completes. If non-blocking mode has been enabled using the O_NONBLOCK or O_NDELAY fcntl() flags or
theFIOSNBIO ioctl() request and the connection cannot be completed immediately, connect() returns an error as
describedbelow. In these cases, select() can be used on this socket to determine when the connection has
completedby selecting it for writing.
Maybe you can make this work after all, with the limitation that the
user must supply a dotted-decimal IP address (or use a Unix-domain
connection) if he doesn't want to risk DNS lookup delays.
Good luck!
regards, tom lane
On 18 Jul, Tom Lane wrote: > eem21@cam.ac.uk writes: >> I realised that some time after I sent my first message, and have been >> thinking about it since. The delay for gethostbyname is not >> acceptable, and we are going to have to get around that problem >> elsewhere in the application anyway. Therefore, I was thinking that I >> could arrange to pass an IP address into libpq. > > A dotted-decimal address, you mean? Yeah, that sounds like it should > work. Or a struct in_addr *. I'll have to see how IPv6 addresses are being shoehorned into the socket schemes. Perhaps we could support them too. >> With regards to the connect() problem, I was hoping that by setting the >> socket to non-blocking mode before connecting (exactly as it is >> currently done after the connect call) I could solve that problem. > > By golly, there is something about non-blocking connect in the man page: > > If the socket is of type SOCK_STREAM, connect() attempts to contact > the remote host in order to make a connection between the remote > socket (peer) and the local socket specified by s. The call normally > blocks until the connection completes. If non-blocking mode has been > enabled using the O_NONBLOCK or O_NDELAY fcntl() flags or the FIOSNBIO > ioctl() request and the connection cannot be completed immediately, > connect() returns an error as described below. In these cases, > select() can be used on this socket to determine when the connection > has completed by selecting it for writing. > > Maybe you can make this work after all, with the limitation that the > user must supply a dotted-decimal IP address (or use a Unix-domain > connection) if he doesn't want to risk DNS lookup delays. > > Good luck! Thanks! Ewan.