Обсуждение: Random UnknownHostExceptions

Not exactly sure how to help you here as clearly this is not a postgresql issue.

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


On Thu, Jun 28, 2012 at 11:31 AM, George Woodring
<george.woodring@iglass.net> wrote:
> We are seeing random UnknownHostExceptions and are looking for suggestions
> on trying to find out the root cause.  We are seeing one error per 100K
> connections as my best guesstimation.
>
> We are running on CentOS 5.2, tomcat 6.0.29, jdbc 9.0-801.jdbc4 and using
> tomcat-dbcp connection pooling.
>
> An example of the failure senario is as follows.
>
> At 2012-06-28 08:46:43,886 we got the exception at the bottom.  According to
> the tcpdump, there was not a DNS lookup for the server at this time and we
> have active DB connections to the server.
>
> According to the db log, we had successful connections before and after the
> failure:
> Jun 28 08:46:02 iprobe113 postgres[12386]: [2-1] LOG:  connection received:
> host=192.168.175.21 port=53592
> Jun 28 08:46:02 iprobe113 postgres[12387]: [2-1] LOG:  connection received:
> host=192.168.175.21 port=53593
> Jun 28 08:47:58 iprobe113 postgres[12461]: [2-1] LOG:  connection received:
> host=192.168.175.21 port=53614
> Jun 28 08:47:58 iprobe113 postgres[12462]: [2-1] LOG:  connection received:
> host=192.168.175.21 port=53615
> There are no DNS attempts for any of these connections.  According to the
> tcpdump, the next DNS attempt is not until 8:50.
>
> Any suggestions would be appreciated.
> George Woodring
>
>
>
> The following is the exception we are seeing
>
> org.postgresql.util.PSQLException: The connection attempt failed.
>        at org.postgresql.Driver$ConnectThread.getResult(Driver.java:348)
>        at org.postgresql.Driver.connect(Driver.java:271)
>        at
> org.apache.tomcat.dbcp.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
>        at
> org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
>        at
> org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)
>        at
> org.apache.tomcat.dbcp.dbcp.AbandonedObjectPool.borrowObject(AbandonedObjectPool.java:79)
>        at
> org.apache.tomcat.dbcp.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
>        at
> org.apache.tomcat.dbcp.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
>        at
> org.apache.commons.dbutils.QueryRunner.prepareConnection(QueryRunner.java:195)
>        at net.iglass.db.QueryRunner.prepareConnection(QueryRunner.java:211)
>        at net.iglass.db.QueryRunner.query(QueryRunner.java:287)
>        at
> net.iglass.jglass.core.dao.StaticGraphDAO.fetchCpeHistHealthDataForDataSources(StaticGraphDAO.java:1172)
>        at
> net.iglass.jglass.core.dao.StaticGraphDAO.fetchCpeHistHealthDataSources(StaticGraphDAO.java:591)
>        at
> net.iglass.jglass.jrobin.builder.CpeHistHealthBuilder.addLines(CpeHistHealthBuilder.java:131)
>        at
> net.iglass.jglass.jrobin.builder.AbstractBuilder.buildMiniGraph(AbstractBuilder.java:266)
>        at
> net.iglass.jglass.jrobin.builder.AbstractBuilder.buildGraph(AbstractBuilder.java:179)
>        at
> net.iglass.jglass.jrobin.action.GraphAction.getGraph(GraphAction.java:145)
>        at
> net.iglass.jglass.jrobin.action.GraphAction.doExecute(GraphAction.java:75)
>
>        at
> com.cc.framework.adapter.struts.ActionUtil.execute(ActionUtil.java:751)
>        at
> com.cc.framework.adapter.struts.FWAction.execute(FWAction.java:279)
>        at net.iglass.jglass.BaseAction.execute(BaseAction.java:93)
>        at
> com.cc.framework.adapter.struts.FWAction.execute(FWAction.java:314)
>        at
> com.cc.framework.adapter.struts.RequestProcessorUtil.processAction(RequestProcessorUtil.java:76)
>        at
> com.cc.framework.adapter.struts.FWRequestProcessor.processActionPerform(FWRequestProcessor.java:90)
>        at
> org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
>        at
> org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
>        at
> org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>        at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>        at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>        at
> net.iglass.jglass.servlets.SessionListener.doFilter(SessionListener.java:529)
>        at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>        at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>        at
> net.iglass.jglass.servlets.AccessControlFilter.doFilter(AccessControlFilter.java:303)
>        at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>        at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>        at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
>        at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>        at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>        at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>        at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>        at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>        at
> org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:427)
>        at
> org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:384)
>        at
> org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1584)
>        at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.UnknownHostException: iprobe113
>
>        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
>        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>        at java.net.Socket.connect(Socket.java:529)
>        at java.net.Socket.connect(Socket.java:478)
>        at java.net.Socket.<init>(Socket.java:375)
>        at java.net.Socket.<init>(Socket.java:189)
>        at org.postgresql.core.PGStream.<init>(PGStream.java:62)
>        at
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76)
>        at
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
>        at
> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
>        at
> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
>        at
> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
>        at
> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
>        at
> org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
>        at org.postgresql.Driver.makeConnection(Driver.java:393)
>        at org.postgresql.Driver.access$100(Driver.java:47)
>        at org.postgresql.Driver$ConnectThread.run(Driver.java:308)
>        ... 1 more
>
> --
> iGLASS Networks
> www.iglass.net

George Woodring <george.woodring 'at' iglass.net> writes:

> Caused by: java.net.UnknownHostException: iprobe113
>
>        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
>        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>        at java.net.Socket.connect(Socket.java:529)
>        at java.net.Socket.connect(Socket.java:478)
>        at java.net.Socket.<init>(Socket.java:375)
>        at java.net.Socket.<init>(Socket.java:189)
>        at org.postgresql.core.PGStream.<init>(PGStream.java:62)

The PG driver seems to be using the standard Socket java
interface, so there's little chance it has anything to do with
it. I would try to run a local Java program using a Socket object
to connect to a fake port on the same machine, for example you
could base on network connection code from:


http://code.google.com/p/androidnetworktester/source/browse/trunk/src/org/gc/networktester/tester/TcpConnectionTester.java

Btw, one per 100K connections means you keep on creating new
connections. This usually has very poor performance. You should
use a connection pool.

--
Guillaume Cottenceau