Обсуждение: SocketException on connect, busy server
Hi!
On a busy server, serving web pages using tomcat and apache, I
get this error sometimes:
java.net.SocketException: errno: 48, error: Address already in
use for fd: 168
at java.net.PlainSocketImpl.socketConnect(Native
Method)
at
java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java,
Compiled Code)
at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java,
Compiled Code)
at
java.net.PlainSocketImpl.connect(PlainSocketImpl.java, Compiled
Code)
at java.net.Socket.<init>(Socket.java, Compiled Code)
at java.net.Socket.<init>(Socket.java, Compiled Code)
at org.postgresql.PG_Stream.<init>(PG_Stream.java,
Compiled Code)
at
org.postgresql.Connection.openConnection(Connection.java,
Compiled Code)
at org.postgresql.Driver.connect(Driver.java, Compiled
Code)
at
java.sql.DriverManager.getConnection(DriverManager.java,
Compiled Code)
at
java.sql.DriverManager.getConnection(DriverManager.java,
Compiled Code)
at
net.pingpong.util.core.PPDbBroker.getConnection(PPDbBroker.java,
Compiled Code)
at
net.pingpong.util.core.PPGlobalDbBroker.getConnection(PPGlobalDbBroker.java,
Compiled Code)
at
net.pingpong.core.PPPerson.getDepartmentId(PPPerson.java,
Compiled Code)
at
pp.system.ppentrance._0002fpp_0002fsystem_0002fppentrance_0002fevents_0002dcatalog_0002ejspevents_0002dcatalog_jsp_0._jspService(_0002fpp_0002fsystem_0
002fppentrance_0002fevents_0002dcatalog_0002ejspevents_0002dcatalog_jsp_0.java,
Compiled Code)
at
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java,
Compiled Code)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java,
Compiled Code)
at
org.apache.jasper.servlet.JspServlet$JspServletWrapper.service(JspServlet.java,
Compiled Code)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java,
Compiled Code)
at
org.apache.jasper.servlet.JspServlet.service(JspServlet.java,
Compiled Code)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java,
Compiled Code)
at
org.apache.tomcat.core.ServletWrapper.doService(ServletWrapper.java,
Compiled Code)
at org.apache.tomcat.core.Handler.service(Handler.java,
Compiled Code)
at
org.apache.tomcat.core.ServletWrapper.service(ServletWrapper.java,
Compiled Code)
at
org.apache.tomcat.core.ContextManager.internalService(ContextManager.java,
Compiled Code)
at
org.apache.tomcat.core.ContextManager.service(ContextManager.java,
Compiled Code)
at
org.apache.tomcat.service.connector.Ajp12ConnectionHandler.processConnection(Ajp12ConnectionHandler.java,
Compiled Code)
at
org.apache.tomcat.service.TcpWorkerThread.runIt(PoolTcpEndpoint.java,
Compiled Code)
at
org.apache.tomcat.util.ThreadPool$ControlRunnable.run(ThreadPool.java,
Compiled Code)
at java.lang.Thread.run(Thread.java, Compiled Code)
End of Stack Trace
It happens around here in our app (in the database connection
broker):
synchronized (connections) {
if (connections.size() < maxConnections) {
// All connections used but we can open a new one.
try {
con = DriverManager.getConnection(dbURL, DEFAULT_USER,
DEFAULT_PASSWD);
connections.put(con, new PPDbConnData(false, new Date()));
(connections is a Hash containing a number of db connections).
Hence, it seems to happen only when all connections are busy
and opening a new one is necessary. This seems to me like some
kind of synchronization problem, but I can't find it. Any
ideas?
/Palle
--
Partitur Informationsteknik AB
Wenner-Gren Center +46 8 566 280 02
113 46 Stockholm +46 70 785 86 02
Sweden girgen@partitur.se
On Sun, 02 Sep 2001 23:41:49 +0200, you wrote: >On a busy server, serving web pages using tomcat and apache, I >get this error sometimes: > >java.net.SocketException: errno: 48, error: Address already in >use for fd: 168 > at java.net.PlainSocketImpl.socketConnect(Native >Method) This means the underlying connect(2) socket call returns errno EADDRINUSE. On a connect(), this means that the 5-tuple <protocol,local address,local port,remote address,remote port> would not be unique if the operation had been allowed. Since protocol, local address, remote address and remote port are fixed, this occurs if the local socket implementation cannot provide a local port number that is still available with respect to the 4 fixed parts. The port number is a 16 bit number, ports 0-1024 are usually not available as dynamically allocated, leaving 64511 ports available. TCP defines a 4 minute TIME_WAIT state, so there can be no more than 64511 / 4 * 60 = 268 connections per second from a single client IP address to one particular service. This is usually only a problem with load balancing HTTP servers and other stateless protocols, not with database servers, since database connections are typically pooled. Is your application opening and closing PostgreSQL connections at such a high rate? In any case, you should probably attack this problem at the TCP implementation level of your OS. It is most likely not PostgreSQL specific. Google gives some interesting hits: http://www.google.com/search?q=time%5Fwait Regards, René Pijlman <rene@lab.applinet.nl>
On Fri, 07 Sep 2001 02:37:36 +0200, I wrote: >Is your application opening and closing PostgreSQL >connections at such a high rate? Or, I should add, is it opening that many connections? Regards, René Pijlman <rene@lab.applinet.nl>
Hi! Thanks for your reply, and sorry for my late rereply! I should of course have mentioned that we use FreeBSD-4.3. I doubt we open postgres connections at that rate, but quite possibly HTTP and other protocols, but that is different ports. Postgres connections are pooled, and we don't even allow 268 connections, so... I don't really think there is problem with the TCP implementation in FreeBSD, but maybe I can tweak som parameters. I'll read about the stuff and see what I can come up with. Any ideas are welcome. Thanks! /Palle Rene Pijlman wrote: > > On Sun, 02 Sep 2001 23:41:49 +0200, you wrote: > >On a busy server, serving web pages using tomcat and apache, I > >get this error sometimes: > > > >java.net.SocketException: errno: 48, error: Address already in > >use for fd: 168 > > at java.net.PlainSocketImpl.socketConnect(Native > >Method) > > This means the underlying connect(2) socket call returns errno > EADDRINUSE. > > On a connect(), this means that the 5-tuple <protocol,local > address,local port,remote address,remote port> > would not be unique if the operation had been allowed. Since > protocol, local address, remote address and remote port are > fixed, this occurs if the local socket implementation cannot > provide a local port number that is still available with respect > to the 4 fixed parts. The port number is a 16 bit number, ports > 0-1024 are usually not available as dynamically allocated, > leaving 64511 ports available. TCP defines a 4 minute TIME_WAIT > state, so there can be no more than 64511 / 4 * 60 = 268 > connections per second from a single client IP address to one > particular service. > > This is usually only a problem with load balancing HTTP servers > and other stateless protocols, not with database servers, since > database connections are typically pooled. Is your application > opening and closing PostgreSQL connections at such a high rate? > > In any case, you should probably attack this problem at the TCP > implementation level of your OS. It is most likely not > PostgreSQL specific. > > Google gives some interesting hits: > http://www.google.com/search?q=time%5Fwait > > Regards, > René Pijlman <rene@lab.applinet.nl> -- Partitur Informationsteknik AB Wenner-Gren Center +46 8 566 280 02 113 46 Stockholm +46 70 785 86 02 Sweden girgen@partitur.se