Re: Does people favor to have matrix data type?

On 31/05/16 12:01, Kouhei Kaigai wrote:
>> On 05/29/2016 04:55 PM, Kouhei Kaigai wrote:
>>> For the closer integration, it may be valuable if PL/R and PL/CUDA can exchange
>>> the data structure with no serialization/de-serialization when PL/R code tries
>>> to call SQL functions.
>> I had been thinking about something similar. Maybe PL/R can create an
>> extension within the R environment that wraps PL/CUDA directly or at the
>> least provides a way to use a fast-path call. We should probably try to
>> start out with one common use case to see how it might work and how much
>> benefit there might be.
>>
> My thought is the second option above. If SPI interface supports fast-path
> like 'F' protocol, it may become a natural way for other PLs also to
> integrate SQL functions in other languages.
>
>>> IIUC, pg.spi.exec("SELECT my_function(...)") is the only way to call SQL functions
>> inside PL/R scripts.
>>
>> Correct (currently).
>>
>> BTW, this is starting to drift off topic I think -- perhaps we should
>> continue off list?
>>
> Some elements are common for PostgreSQL (matrix data type and fastpath SPI
> interface). I like to keep the discussion on the list.
> Regarding to the PoC on a particular use case, it might be an off-list
> discussion.
>
> Thanks,
> --
> NEC Business Creation Division / PG-Strom Project
> KaiGai Kohei <kaigai@ak.jp.nec.com>
>
Possibly there should be two matrix types in Postgres: the first would 
be the default and optimized for small dense matrices, the second would 
store large sparse matrices efficiently in memory at the expensive of 
speed (possibly with one or more parameters relating to how sparse it is 
likely to be?) - for appropriate definitions 'small' & 'large', though 
memory savings for the latter type might not kick in unless the matrices 
are big enough (so a small sparse matrix might consume more memory than 
a nominally larger dense matrix type & a sparse matrix might have to be 
sufficiently sparse to make real memory savings at any size).

Probably good to think of 2 types at the start, even if the only the 
first is implemented initially.


Cheers,
Gavin