Обсуждение: ODBC Developers
Hi All, As many of you will know, I recently placed a news item on www.postgresql.org to try to get some new developers working on psqlODBC. I was pleseantly surprised to find that after just a few days I had received eight responses from people interested in joining the project. This message has been BCC'd to them. First of all, I'd like to say thank you to those that responded - input from all of you would be most welcome as there are a number of end users keen to see psqlODBC move forwards. If you have not done so already, please subscribe to the psql-odbc@postgresql.org mailing lists (just send a message containing the word 'subscribe' (wiithout the quotes) to psql-odbc-request@postgresql.org). In the remainder of this message, I'll try to outline some history and what I see as the future for the driver. This is all open for discussion on-list so feel free to chip in with comments. History ------- I first started using psqlODBC back around 1997, when it was maintained by solely Byron Nikolaidis from Insight Distribution Systems. When Byron stopped working on the driver, after a period with no maintainer, Hiroshi Inoue took over, and I started looking after the website and managing the releases of the driver. It should be noted that whilst I have done some work on the internals, I am by no means an expert on how the driver works. After some years, Hiroshi was also forced to quit working on the driver, thus leaving it in it's current state. At the same time, my time has also been refocussed elsewhere as I rarely use the driver nowadays (some of you will know that my primary project, pgAdmin, now uses libpq), thus I only really have time to chip in now and again. The upshot is, that we have a community of users, and a project that needs to move forwards, but currently isn't. The future ---------- Over the last few releases, we have been introducing new versions of the driver, one with ODBC 3 compliance, and one with ODBC 3 compliance, and Unicode support. We have gradually been promoting the driver to make the ODBC 3 Unicode driver the default. Changes to CVS have already been committed to make this the only version of the driver shipped in future releases, though this has not been tested under *nix at present. There are a number of tasks that need addressing - following any discussion, I'd be happy if any of our new volunteers could co-ordinate efforts to undertake this work. This list is not necessarily complete - corrections/comments welcome :-) 1) ODBC 3 compliance should be checked. Are all the required ODBC functions present. Do they work as expected. 2) Ditto, Unicode versions of all the functions. 3) An audit of the code for possible buffer overrun problems should be undertaken. 4) An update to the v3 fe-be protocol is required. 5) SSL support should be added. 6) The GUI needs a cleanup (move to use of tabs rather than separate dialogues, and a 'test connection' button?). 7) Documentation and the website needs updating (as always :-) ) (note; would 4 & 5 be better solved by replacing the current code with libpq?) In addition, there are always bugs with new versions, and the latest snapshot is no exception. These need to be fixed as reported. Initially, any work that is undertaken should be supplied to the list as patches for review following any appropriate discussion. If the general concencus is that a patch is good, then I will apply it to CVS. I will also continue to produce releases for now. In the future, we can look at giving more people CVS commit privileges once they have proven the quality of their work (as is the norm for any good open source project). So, I'll finish up now. I hope I haven't put off any one and welcome any comments (or a flurry of patches :-) ). Regards, Dave.
"Dave Page" <dpage@vale-housing.co.uk> writes:
> As many of you will know, I recently placed a news item on
> www.postgresql.org to try to get some new developers working on
> psqlODBC. I was pleseantly surprised to find that after just a few days
> I had received eight responses from people interested in joining the
> project. This message has been BCC'd to them.
This is really, really good news. With you and Hiroshi both pretty much
withdrawn from development, I was afraid psqlODBC would die on the vine.
(I have no time to contribute to it either :-()
One comment on your todo list:
> 3) An audit of the code for possible buffer overrun problems should be
> undertaken.
I think this is really critical and should be done ASAP. We already
have at least one known issue of this kind. The bad news of course
is that it is boring, tedious work ... but on the other hand it's a
great way to learn one's way around the code. I hope that several of
the new developers will perform such reviews.
regards, tom lane
Dave Page wrote: > 3) An audit of the code for possible buffer overrun problems should > be undertaken. I'm planning to rip out all uses of the libc string functions and replace them by pqexpbuffer that is also used successfully in libpq. Look for a patch soon. > Initially, any work that is undertaken should be supplied to the list > as patches for review following any appropriate discussion. If the > general concencus is that a patch is good, then I will apply it to > CVS. Maybe you can start by applying the 64-bit fix patch of the other day?
Hi there, I can tackle 1 and 2. They will also serve as a good introduction to both ODBC, the driver and unicode. Where can I get details/specs for all these? Also another thing that I have come across when using psqlODBC with OTL (http://otl.sourceforge.net/home.htm) is that the driver will not work when inserting data with a buffer size other than 1 (this is the size of the otl_stream buffer). Regards, Sterge -----Original Message----- From: pgsql-odbc-owner@postgresql.org [mailto:pgsql-odbc-owner@postgresql.org]On Behalf Of Dave Page Sent: Wednesday, June 30, 2004 12:45 PM To: pgsql-odbc@postgresql.org Subject: [ODBC] ODBC Developers Hi All, As many of you will know, I recently placed a news item on www.postgresql.org to try to get some new developers working on psqlODBC. I was pleseantly surprised to find that after just a few days I had received eight responses from people interested in joining the project. This message has been BCC'd to them. First of all, I'd like to say thank you to those that responded - input from all of you would be most welcome as there are a number of end users keen to see psqlODBC move forwards. If you have not done so already, please subscribe to the psql-odbc@postgresql.org mailing lists (just send a message containing the word 'subscribe' (wiithout the quotes) to psql-odbc-request@postgresql.org). In the remainder of this message, I'll try to outline some history and what I see as the future for the driver. This is all open for discussion on-list so feel free to chip in with comments. History ------- I first started using psqlODBC back around 1997, when it was maintained by solely Byron Nikolaidis from Insight Distribution Systems. When Byron stopped working on the driver, after a period with no maintainer, Hiroshi Inoue took over, and I started looking after the website and managing the releases of the driver. It should be noted that whilst I have done some work on the internals, I am by no means an expert on how the driver works. After some years, Hiroshi was also forced to quit working on the driver, thus leaving it in it's current state. At the same time, my time has also been refocussed elsewhere as I rarely use the driver nowadays (some of you will know that my primary project, pgAdmin, now uses libpq), thus I only really have time to chip in now and again. The upshot is, that we have a community of users, and a project that needs to move forwards, but currently isn't. The future ---------- Over the last few releases, we have been introducing new versions of the driver, one with ODBC 3 compliance, and one with ODBC 3 compliance, and Unicode support. We have gradually been promoting the driver to make the ODBC 3 Unicode driver the default. Changes to CVS have already been committed to make this the only version of the driver shipped in future releases, though this has not been tested under *nix at present. There are a number of tasks that need addressing - following any discussion, I'd be happy if any of our new volunteers could co-ordinate efforts to undertake this work. This list is not necessarily complete - corrections/comments welcome :-) 1) ODBC 3 compliance should be checked. Are all the required ODBC functions present. Do they work as expected. 2) Ditto, Unicode versions of all the functions. 3) An audit of the code for possible buffer overrun problems should be undertaken. 4) An update to the v3 fe-be protocol is required. 5) SSL support should be added. 6) The GUI needs a cleanup (move to use of tabs rather than separate dialogues, and a 'test connection' button?). 7) Documentation and the website needs updating (as always :-) ) (note; would 4 & 5 be better solved by replacing the current code with libpq?) In addition, there are always bugs with new versions, and the latest snapshot is no exception. These need to be fixed as reported. Initially, any work that is undertaken should be supplied to the list as patches for review following any appropriate discussion. If the general concencus is that a patch is good, then I will apply it to CVS. I will also continue to produce releases for now. In the future, we can look at giving more people CVS commit privileges once they have proven the quality of their work (as is the norm for any good open source project). So, I'll finish up now. I hope I haven't put off any one and welcome any comments (or a flurry of patches :-) ). Regards, Dave. ---------------------------(end of broadcast)--------------------------- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
> -----Original Message----- > From: Stergios Zissakis [mailto:szis@intranet.gr] > Sent: 01 July 2004 07:52 > To: Dave Page; pgsql-odbc@postgresql.org > Subject: RE: [ODBC] ODBC Developers > > Hi there, Hi, > I can tackle 1 and 2. They will also serve as a good > introduction to both ODBC, the driver and unicode. > Where can I get details/specs for all these? Excellent :-). ODBC is a Microsoft thing, so they have the definitive reference. It's part of the MDAC SDK - http://www.microsoft.com/downloads/details.aspx?familyid=5067faf8-0db4-4 29a-b502-de4329c8c850&languageid=f49e8428-7071-4979-8a67-3cffcb0c2524&di splaylang=en There used to be some validation and test tools included which may be of use. > Also another thing that I have come across when using > psqlODBC with OTL > (http://otl.sourceforge.net/home.htm) is that the driver will > not work when inserting data with a buffer size other than 1 > (this is the size of the otl_stream buffer). Can you log this on Gborg please? http://gborg.postgresql.org/project/psqlodbc/projdisplay.php Regards, Dave
> -----Original Message----- > From: Peter Eisentraut [mailto:peter_e@gmx.net] > Sent: 30 June 2004 18:11 > To: Dave Page; pgsql-odbc@postgresql.org > Subject: Re: [ODBC] ODBC Developers > > Dave Page wrote: > > 3) An audit of the code for possible buffer overrun > problems should be > > undertaken. > > I'm planning to rip out all uses of the libc string functions > and replace them by pqexpbuffer that is also used > successfully in libpq. > Look for a patch soon. OK, thanks. Are you planning on spending much time on the project? If so, do you want commit access? > > Initially, any work that is undertaken should be supplied > to the list > > as patches for review following any appropriate discussion. If the > > general concencus is that a patch is good, then I will apply it to > > CVS. > > Maybe you can start by applying the 64-bit fix patch of the other day? It only arrived in my inbox this morning for some reason - I'll be looking at it in a minute. Regards, Dave
[CC'd to the ODBC List] > -----Original Message----- > From: Stergios Zissakis [mailto:szis@intranet.gr] > Sent: 02 July 2004 14:58 > To: Dave Page > Subject: RE: ODBC Developers > > Hi Dave, Hi Stergios, Please CC any future emails to the list so other know what's going on so we don't get more than one person doing the same thing (which can be *really* annoying when someone else submits a patch the same as one that you spent hours working on!) > Regarding the work I undertook on ODBC driver, I have a few questions: > 1) Where can I get a spec for ODBC 3.0? The ODBC spec is included with the Microsoft MDAC 2.8 SDK, along with various test utilities if I remember correctly. Download from http://www.microsoft.com/downloads/search.aspx?displaylang=en (search for 'mdac sdk'). > 2) Where can I get information on UNICODE? Google is your friend :-). Alternatively, try http://www.unicode.org/ > 3) Are there any specific test cases I have to execute? No. We have no test plan or test cases. Feel free to look into developing a plan, perhaps using the utilities in the SDK. > 4) Is there a standard document for test results? Nope. See above :-) > > I need the first things to get me started. 3 and 4 will help > with testing the conformance of driver with respect to the > 3.0 standard and unicode. > I hope you understand that all this is new to me (can't say > Greek to me because I am a Greek:) and I need a starting > point. I can work on this and produce a document that can be > used to initiate other people to the driver > developemnt/testing environment but I will need to be pointed > to the right direction. I'll help whereever I can - as always though, it's best to email the list when you need help as there are other people (with better brains and more time) who may also be able to help! > I want to take slowly and don't jump straight to the code > before having a understanding of the environment. A good plan. > BTW I've logged the OTL bug in GBorg and also had a look at > the M$ link you mailed me ther other day. I don't really get > this; this is a development platform for ODBC... will I be > working on Windows or Linux? I prefer Linux to be honest once > I've managed to get everything up and running (psqlODBC, > PG7.4.2 and unixODBC2.2.8). Also I haven't got a clue about > installing PostgreSQL on Windows. The Microsoft link I sent is to the SDK I mentioned above. The docs in the SDK are in compiled HTML Help format. You can work on either Windows or Linux, however you will probably need Windows somewhere to get the docs out of the SDK, and certainly to run the test utilities (you might have some success running them under Wine, but it might skew the results). Perhaps Qemu or Vmware would be of help if you wish to run both in the long term. Alternatively, once you've extracted the docs, you should be able to read them on Linux using something like: http://xchm.sourceforge.net/. Other test tools can be written using gcc of course. > How about producing some > regression tests for conformance? Yes, if you can develop an automated test suite, that would be excellent. Regards, Dave.
Dave Page wrote: > 4) An update to the v3 fe-be protocol is required. > > 5) SSL support should be added. > > 6) The GUI needs a cleanup (move to use of tabs rather than separate > dialogues, and a 'test connection' button?). > > 7) Documentation and the website needs updating (as always :-) ) > > (note; would 4 & 5 be better solved by replacing the current code with > libpq?) Using libpq seems like it would be a great solution. Why didn't the original driver use libpq? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
> -----Original Message----- > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us] > Sent: 06 July 2004 15:10 > To: Dave Page > Cc: pgsql-odbc@postgresql.org > Subject: Re: [ODBC] ODBC Developers > > > Using libpq seems like it would be a great solution. Why > didn't the original driver use libpq? No idea - that was back in the Postgres95 days (and possibly earlier). Certainly long before most of us were here... Regards, Dave.
Dave Page wrote: > > > > -----Original Message----- > > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us] > > Sent: 06 July 2004 15:10 > > To: Dave Page > > Cc: pgsql-odbc@postgresql.org > > Subject: Re: [ODBC] ODBC Developers > > > > > > Using libpq seems like it would be a great solution. Why > > didn't the original driver use libpq? > > No idea - that was back in the Postgres95 days (and possibly earlier). > Certainly long before most of us were here... My basic question was why _not_ convert it to use libpq? Seems it would be smaller and easier to maintain. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
> > >> -----Original Message----- >> From: Bruce Momjian [mailto:pgman@candle.pha.pa.us] >> Sent: 06 July 2004 15:10 >> To: Dave Page >> Cc: pgsql-odbc@postgresql.org >> Subject: Re: [ODBC] ODBC Developers >> >> >> Using libpq seems like it would be a great solution. Why >> didn't the original driver use libpq? > > No idea - that was back in the Postgres95 days (and possibly earlier). > Certainly long before most of us were here... My bet would be that, at the time, lib PQ didn't give the functionality needed for an ODBC interface. > > Regards, Dave.
Dave Page wrote: > 1) ODBC 3 compliance should be checked. Are all the required ODBC > functions present. Do they work as expected. A brief status on this: We have all ODBC 3 functions in the code, but the following functions claim various forms of "not (fully) implemented": BrowseConnect ColumnPrivileges DescribeParam GetDescRec SetDescRec Whether any function works as expected is of course undecidable. :-) > 2) Ditto, Unicode versions of all the functions. We have almost all the required Unicode functions, but some functions are in the file odbcapi25w.c, which is not linked in anywhere. What's up with that? The function GetDescRecW is completely missing. The additional required SQL functions are also mostly present, but there seems to be some duplication/overlap between convert.c and odbc.sql, that isn't quite clear to me yet. Overall it looks quite good from the conformance point of view. -- Peter Eisentraut http://developer.postgresql.org/~petere/
> -----Original Message----- > From: Peter Eisentraut [mailto:peter_e@gmx.net] > Sent: 16 July 2004 07:23 > To: Dave Page; pgsql-odbc@postgresql.org > Subject: Re: [ODBC] ODBC Developers > > Dave Page wrote: > > 1) ODBC 3 compliance should be checked. Are all the required ODBC > > functions present. Do they work as expected. > > A brief status on this: We have all ODBC 3 functions in the > code, but the following functions claim various forms of "not (fully) > implemented": > > BrowseConnect > ColumnPrivileges > DescribeParam > GetDescRec > SetDescRec > > Whether any function works as expected is of course undecidable. :-) Hmm, OK. > > 2) Ditto, Unicode versions of all the functions. > > We have almost all the required Unicode functions, but some > functions are in the file odbcapi25w.c, which is not linked > in anywhere. What's up with that? All the functions in there are deprecated in ODBC3 (as well as the ansi equivalents). The driver manager will re-map any calls to them to the ODBC3 replacements: SQLColAttributes(W) -> SQLColAttribute(W) SQLError(W) -> SQLGetDiagRec(W) SQLGetConnectOption(W) -> SQLGetConnectAttr(W) SQLSetConnectOption(W) -> SQLSetConnectAttr(W) > The function GetDescRecW is completely missing. Yup, it's missing it's placeholder. The driver does correctly report that it is not implemented so there is no real problem, just an incomplete api. I'm guessing not many apps use it... I have removed the comment that says that SQLGetDescRecW is in odbcapi30w.c Which makes me think - should we add or remove placeholders for unimplemented functions? The important thing is that SQLGetFunctions is correct of course, however we should be consistent. I'm kinda on the fence about which way to go on that one. On one had the placeholders are useful reminders that work needs to be done - on the other, they do add to the code needlessly. > The additional required SQL functions are also mostly > present, but there seems to be some duplication/overlap > between convert.c and odbc.sql, that isn't quite clear to me yet. convert.c should (imho) make odbc.sql obsolete as much as possible. > Overall it looks quite good from the conformance point of view. :-) Thanks Peter.
Dave Page wrote: > Which makes me think - should we add or remove placeholders for > unimplemented functions? The important thing is that SQLGetFunctions > is correct of course, however we should be consistent. I'm kinda on > the fence about which way to go on that one. On one had the > placeholders are useful reminders that work needs to be done - on the > other, they do add to the code needlessly. I would leave it as is. As long as the fraction of placeholders compared to the implemented functions is low, it's not really a problem. What could be useful is a TODO file listing missing functions and other issues. > convert.c should (imho) make odbc.sql obsolete as much as possible. I've fixed a few bugs in convert.c and verified all the functions. odbc.sql is obsolete and removed. -- Peter Eisentraut http://developer.postgresql.org/~petere/
> -----Original Message----- > From: Peter Eisentraut [mailto:peter_e@gmx.net] > Sent: 17 July 2004 22:05 > To: Dave Page; pgsql-odbc@postgresql.org > Subject: Re: [ODBC] ODBC Developers > > Dave Page wrote: > > Which makes me think - should we add or remove placeholders for > > unimplemented functions? The important thing is that > SQLGetFunctions > > is correct of course, however we should be consistent. I'm kinda on > > the fence about which way to go on that one. On one had the > > placeholders are useful reminders that work needs to be > done - on the > > other, they do add to the code needlessly. > > I would leave it as is. As long as the fraction of > placeholders compared to the implemented functions is low, > it's not really a problem. OK. > What could be useful is a TODO file listing missing functions > and other issues. Gborg todo/task list do? > > convert.c should (imho) make odbc.sql obsolete as much as possible. > > I've fixed a few bugs in convert.c and verified all the functions. > odbc.sql is obsolete and removed. :-) Regard,s Dave.