Re: TwoPO: experimental join order algorithm

On Fri, Jul 30, 2010 at 7:02 AM, Jan Urbański <wulczer@wulczer.org> wrote:
> On 24/07/10 15:20, Adriano Lange wrote:
>>
>> Hi,
>
> Hi!
>
>>
>> I'd like to release the last version of my experimental join order
>> algorithm (TwoPO - Two Phase Optimization [1]):
>>
>> http://git.c3sl.ufpr.br/gitweb?p=lbd/ljqo.git;a=summary
>>
>> This algorithm is not production-ready, but an experimental set of
>> ideas, which need to be refined and evaluated. As the join order
>> optimization is a hard problem, the evaluation of a search strategy is
>> also a hard task. Therefore, I think the most important TODO item
>> related to replacement of GEQO algorithm is to define a set of
>> evaluation criteria considered as relevant.
>
> Good to hear from you --  I don't know if you are aware about a simulated
> annealing join search module that I'm working on. When I first started, I
> planned to base is on the patch that you sent to -hackers some time ago (and
> actually, if you look at the repo for my module, twopo.c is still in there)
> but ended up doing it from scratch. However, reading your code was a big
> help in the beginning.
>
> I gave a talk at this year's PGCon
> (http://www.pgcon.org/2010/schedule/events/211.en.html) and you will find
> your name in the acknowledgments section of the presentation :)
>
> I'll make sure to read your new code and compare the approaches, my results
> so far are not perfect, but also not very pessimistic. I think there is
> actually a chance to replace GEQO with SA in the future, or at least to ship
> more join search modules with the standard distribution and gather field
> reports.
>
> Cheers,
> Jan
>

Hi Jan,

It's good to know that you are also interested about this issue!

I saw the slides of your presentation at PGCon and I noted excellent
ideas. I think that as more as implementations and ideas will be
raised, more easily will be to converge them in a reliable and
competitive solution.

The current version of TwoPO is suitable for the classic
select-project-join problem. Therefore, there are some issues not
covered yet, as I have observed in a test schema presented by Andres
Freund (http://archives.postgresql.org/pgsql-hackers/2009-07/msg00546.php).
I think we will need to build a robust benchmark to deal with it.

At this moment I'm really busy with a "homework", but I hope to
dedicate more time to this issue next month.

--
Adriano Lange