Обсуждение: [pgadmin-hackers] SlickGrid

Hi

On Mon, Feb 13, 2017 at 10:47 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
> Hi Hackers,
>
> We saw a story about transferring query results to SlickGrid on Redmine
> (https://redmine.postgresql.org/issues/2036). We are considering more
> selection options (in addition to by-row) within the results display, which
> got us wondering about the use of SlickGrid.
>
> It appears that SlickGrid hasn't been maintained in roughly 3 years. What is
> the plan going forward with this library? There does seem to be a fork
> https://github.com/6pac/SlickGrid/tree/2.3.4 that is still maintained.

That's the version we use.

> For this and perhaps other js dependencies, would pulling it in via npm be
> an option?

No - we never rely on the availability of third party sites,
especially not commercial ones. More importantly, we a) want to know
we have stable code in our tree (we don't want users running with
random versions that we may not have tested yet), and b) on very rare
occasions we may modify our copies of the code - which is always a
last resort that is documented, and where appropriate, with a patch
sent upstream.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Hi

On Tue, Feb 14, 2017 at 3:45 PM, Atira Odhner <aodhner@pivotal.io> wrote:
> Hi Dave,
>
>>
>> we never rely on the availability of third party sites,
>> especially not commercial ones.
>
>
> npm is commercial and pip is supported by a noncommercial foundation, but
> npm is still open source.

The difference with PIP is that a) the packages can be specific to the
version of Python in use, and b) there are multiple other packaged
sources of many of the packages available, e.g. RPMs.

> If you are concerned about relying on a 3rd party commercial site, perhaps
> we could use npm to install and manage dependencies but also check them in
> or set up pgadmin's own npm registry.
>
>>
>>  More importantly, we a) want to know
>> we have stable code in our tree (we don't want users running with
>> random versions that we may not have tested yet)
>
>
> npm allows version locking, the same way we handle our python dependencies.
>
>> and b) on very rare
>> occasions we may modify our copies of the code - which is always a
>> last resort that is documented, and where appropriate, with a patch
>> sent upstream.
>
>
> We agree that there might be cases where we need to vendorize assets but
> that shouldn't dictate our default approach to managing assets.

Neither should a trend towards using NPM like all the cool kids
(which, as relational database people, we're not) :-).

Before we go any further on this:

1) What would the workflow look like for developers, packagers and
from-source users?

2) What benefit would it give us?

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Hi

On Wed, Feb 15, 2017 at 3:05 PM, Sarah McAlear <smcalear@pivotal.io> wrote:
> Hi Dave
>
>>
>> The difference with PIP is that a) the packages can be specific to the
>> version of Python in use, and b) there are multiple other packaged
>> sources of many of the packages available, e.g. RPMs.
>
>
> There are multiple packaged sources of many js packages available as well.
> For example, bootstrap can be installed by npm, bower, or composer.
>
>> 1) What would the workflow look like for developers, packagers and
>> from-source users?
>
>
> - Developers would add any new js dependencies to a package.json, which
> would be checked into the pgadmin4 repo
> - For setup and packagers, the workflow would introduce the step npm
> install, which would pick up js dependencies from the package.json
> - For development or installation, the package manager would need to be
> installed on the machine

Additional complication to an already non-trivial build environment.

>> 2) What benefit would it give us?
>
>
> - clearer separation of vendor from application code. To fresh eyes, it is
> not immediately apparent what js is actual pgadmin code
> - transitive dependencies don't need to be manually managed
> - keeping up to date with bugfixes, etc. in our dependencies. One way of
> addressing the compatibility concern (besides running tests) is to
> major-version-lock the dependencies (e.g. "~1.2.0" would allow 1.2.x but not
> 1.3)
> - protects the codebase from changes to vendorized code

The problem is that we won't get to that point, as a number of our
dependencies are not available in npm from I can see, so we'll have a
mix n match of in-tree components and out of tree ones, which arguably
means that at least the first couple of points you give above become
even more complex.

Another couple of random questions:

- How would our modularisation be affected? At present, if we only
need a JS module in a particular module, then it's confined to there.
e.g. the grant wizard may include a JS library in the python module
that implements it.

- One of the TODO items on my list is to ensure we only keep the
source to each JS module in the GIT repo, and that as part of the
various build processes we create a minimised version of every JS and
CSS we have, with pgAdmin picking the one to used based on whether or
not debugging is enabled. Could we still do something like this with
npm, or are we at the mercy of each upstream project and whether or
not they wish to provide minimised files (and with what filenames)?

Don't get me wrong, I'm not against improving things here - but I'm
as-yet unconvinced that this proposal adds more than it would
inconvenience.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Hi

On Wed, Feb 22, 2017 at 9:23 PM, Atira Odhner <aodhner@pivotal.io> wrote:
>> The difference with PIP is that a) the packages can be specific to the
>> version of Python in use
>
>
> The javascript is going to run in the end-user's browser regardless of what
> is installed on the machine where the app is packaged. Varying what versions
> of the libraries are in use in that way for javascript dependencies would be
> unnecessary build complication.

That's my point. We don't need or want a varying set of packages at
the javascript end.

>> The problem is that we won't get to that point, as a number of our
>> dependencies are not available in npm from I can see, so we'll have a
>> mix n match of in-tree components and out of tree ones, which arguably
>> means that at least the first couple of points you give above become
>> even more complex.
>
>
> I took a quick peek at the dependencies that we are using and didn't see any
> not available in npm. Additionally, I would like to at least quarantine the
> vendorized dependencies away from the pgadmin code itself into a separate
> directory. e.g static/js/vendor/

piranha:pgadmin4 dpage$ npm search wcdocker
No matches found for "wcdocker"
piranha:pgadmin4 dpage$ npm search acitree
No matches found for "acitree"

I have no problem with moving packages into a vendor directory. Feel
free to send a patch - in fact, that would be a good time to
re-structure such that all vendor code follows it's original
structure, rather than using the strict separation of js/css that
we've (short-sightedly) moved it into. That alone would make future
upgrades much easier.

>> - How would our modularisation be affected? At present, if we only
>> need a JS module in a particular module, then it's confined to there.
>> e.g. the grant wizard may include a JS library in the python module
>> that implements it.
>
>
> Other than changing some file paths in define/require, there's no technical
> reason that would force us to change anything about the modularity of the
> code simply by moving dependencies to npm.

So we could define dependencies on a per-module basis?

>> - One of the TODO items on my list is to ensure we only keep the
>> source to each JS module in the GIT repo, and that as part of the
>> various build processes we create a minimised version of every JS and
>> CSS we have, with pgAdmin picking the one to used based on whether or
>> not debugging is enabled. Could we still do something like this with
>> npm, or are we at the mercy of each upstream project and whether or
>> not they wish to provide minimised files (and with what filenames)?
>
>
>  There are lots of tools like Grunt/Gulp babelify/browserify/uglify which
> could automate minification/non-minification(DEBUG mode) for us at build
> time. That would also allow us to minify our own code, not just our
> dependencies.

Right. What I have been playing with in the past is the idea of having
only un-minimised code in the repo, with a build step that will
minimise all JS/CSS and potentially HTML, creating versions of each
file with a name.ext.min name format, then modifying pgAdmin to look
for a .min version of each file if running in release mode.

What concerns me with using npm and trying to automate this is that
package A might install:

js/pkgA.js
js/pkgA.js.min

Package B might install:

js/pkgB.js

Package C might install:

js/pkgC.js
js/pkgC.min.js

And package D might install:

js/pkgD-1.js
js/pkgD-2.js
js/pkgD-3.js
js/pkgD.min.js

and so on.

By having the dependencies in the tree, we can fix and control those
differences, and then have a standardised, automated and global way of
minimising.

> I think moving to npm would make it easier to pull in some of that tooling,
> but of course we could just keep checking all that code into our git repo.
> Either way, moving the DEBUG configuration out of the code and into the
> build configuration would tidy up our js dependency inclusion.
>
> I'm still not sure I see the benefit of keeping the js checked into the git
> repo as compared to PIP dependencies, but we could do so if we wished.  One
> thing I've done on certain projects is check in the *built* javascript so
> that developers who are not concerned with making js changes do not have to
> set up any js tooling.

That goes against what many consider to be a basic rule of version
control - never check in compiled/built files, only the source to
build them.

>> > We agree that there might be cases where we need to vendorize assets but
>> > that shouldn't dictate our default approach to managing assets.
>> Neither should a trend towards using NPM like all the cool kids
>> (which, as relational database people, we're not) :-).
>
>
> I realize that npm is a spritely young 7 year old tool, but if we want to
> build a community of contributors, might it behoove us to consider what the
> cool kids are doing? :-p

Lol! I'm happy to consider - I just want to be sure we're making
changes for the right reasons; i.e. they will improve the
code/product, or make it easier for us to get work done. I don't want
to do things *just* because they're what the cool kids are doing :-)

> All that said, I understand the allure of not adding new tools as it is one
> more thing to worry about.
>
> Cheers,
> Tira
>
> On Wed, Feb 15, 2017 at 10:59 AM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> Hi
>>
>> On Wed, Feb 15, 2017 at 3:05 PM, Sarah McAlear <smcalear@pivotal.io>
>> wrote:
>> > Hi Dave
>> >
>> >>
>> >> The difference with PIP is that a) the packages can be specific to the
>> >> version of Python in use, and b) there are multiple other packaged
>> >> sources of many of the packages available, e.g. RPMs.
>> >
>> >
>> > There are multiple packaged sources of many js packages available as
>> > well.
>> > For example, bootstrap can be installed by npm, bower, or composer.
>> >
>> >> 1) What would the workflow look like for developers, packagers and
>> >> from-source users?
>> >
>> >
>> > - Developers would add any new js dependencies to a package.json, which
>> > would be checked into the pgadmin4 repo
>> > - For setup and packagers, the workflow would introduce the step npm
>> > install, which would pick up js dependencies from the package.json
>> > - For development or installation, the package manager would need to be
>> > installed on the machine
>>
>> Additional complication to an already non-trivial build environment.
>>
>> >> 2) What benefit would it give us?
>> >
>> >
>> > - clearer separation of vendor from application code. To fresh eyes, it
>> > is
>> > not immediately apparent what js is actual pgadmin code
>> > - transitive dependencies don't need to be manually managed
>> > - keeping up to date with bugfixes, etc. in our dependencies. One way of
>> > addressing the compatibility concern (besides running tests) is to
>> > major-version-lock the dependencies (e.g. "~1.2.0" would allow 1.2.x but
>> > not
>> > 1.3)
>> > - protects the codebase from changes to vendorized code
>>
>> The problem is that we won't get to that point, as a number of our
>> dependencies are not available in npm from I can see, so we'll have a
>> mix n match of in-tree components and out of tree ones, which arguably
>> means that at least the first couple of points you give above become
>> even more complex.
>>
>> Another couple of random questions:
>>
>> - How would our modularisation be affected? At present, if we only
>> need a JS module in a particular module, then it's confined to there.
>> e.g. the grant wizard may include a JS library in the python module
>> that implements it.
>>
>> - One of the TODO items on my list is to ensure we only keep the
>> source to each JS module in the GIT repo, and that as part of the
>> various build processes we create a minimised version of every JS and
>> CSS we have, with pgAdmin picking the one to used based on whether or
>> not debugging is enabled. Could we still do something like this with
>> npm, or are we at the mercy of each upstream project and whether or
>> not they wish to provide minimised files (and with what filenames)?
>>
>> Don't get me wrong, I'm not against improving things here - but I'm
>> as-yet unconvinced that this proposal adds more than it would
>> inconvenience.
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>>
>> --
>> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgadmin-hackers
>
>



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company