Обсуждение: [pgAdmin4][Debugger]: Initial Patch

Hi

On Tue, Apr 5, 2016 at 12:40 PM, Neel Patel <neel.patel@enterprisedb.com> wrote:
> Hi,
>
> Please find attached debugger v1 patch. To test this patch we need to apply
> the "function" module patch submitted by Khushboo.
>
> For the debugger functionality, direct and indirect debugging is working as
> per pgadmin3 except below TODOs.
>
> Below are the TODOs which will be submitted in next patch along with the
> fixed review comments.
>
> For the direct debugging, 'NULL' and 'Expression' column value should be
> validated before pressing the debug button.
> User can deposit the value of input arguments and local variable but need to
> be
> validated against respective data types.
> Currently we have tested for plpgsql functions only, We need to test against
> EDB SPL functions and trigger functions.
> For the direct debugging, once the execution is completed, only
> "Continue/Restart" button will be enabled and user should allow to restart
> the debugging with same function and previous values.
> Values entered by the user in input dialog during direct debugging should be
> saved.
>
>
> Currently we have tested this patch with PostgreSQL v9.5.
>
> Do review it and let us know for the comments.

I just get a blank tab when I try to debug something. I do see the
params dialogue, but then nothing. Note:

- I have Akshay's Query Tool patch applied, and am running from git master

- PostgreSQL 9.4, with the debugger plugin in the public schema.

- Various functions in the pem schema tested, both with and without parameters.

- Both direct and indirect debugging have the same issue.

- No errors seen in the console, except when closing the debugger tab:

2016-04-05 17:27:37,720: INFO werkzeug: 127.0.0.1 - - [05/Apr/2016
17:27:37] "GET /debugger/close/8569717 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/dpage/git/pgadmin4-demo/web/pgadmin/tools/debugger/__init__.py",
line 397, in close
    session_id=obj['session_id'])
KeyError: 'session_id'


Thanks.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Hi Dave,

We will fix the error "KeyError: 'session_id'" but regarding the blank tab, we have tried with PostgreSQL 9.4 with both the version of python (2.7 & 3.4) but we are not able to reproduce the issue.

Below are some of the questions to debug the issue.

• Is it only reproduce on PostgreSQL 9.4 ? Is it working on PostgreSQL 9.5 ?
• Is it possible to know the debugger version ?
• The issue is reproduce on specific functions ? If yes, can you please give definition of the function ?
• Are you able to see toolbar buttons ? Or nothing on the screen ? Only blank panel ?
• Can you please share request/response of last request URL sent to server in browser ?

Thanks,

Neel Patel

On Tue, Apr 5, 2016 at 9:59 PM, Dave Page <dpage@pgadmin.org> wrote:

Hi

On Tue, Apr 5, 2016 at 12:40 PM, Neel Patel <neel.patel@enterprisedb.com> wrote:
> Hi,
>
> Please find attached debugger v1 patch. To test this patch we need to apply
> the "function" module patch submitted by Khushboo.
>
> For the debugger functionality, direct and indirect debugging is working as
> per pgadmin3 except below TODOs.
>
> Below are the TODOs which will be submitted in next patch along with the
> fixed review comments.
>
> For the direct debugging, 'NULL' and 'Expression' column value should be
> validated before pressing the debug button.
> User can deposit the value of input arguments and local variable but need to
> be
> validated against respective data types.
> Currently we have tested for plpgsql functions only, We need to test against
> EDB SPL functions and trigger functions.
> For the direct debugging, once the execution is completed, only
> "Continue/Restart" button will be enabled and user should allow to restart
> the debugging with same function and previous values.
> Values entered by the user in input dialog during direct debugging should be
> saved.
>
>
> Currently we have tested this patch with PostgreSQL v9.5.
>
> Do review it and let us know for the comments.

I just get a blank tab when I try to debug something. I do see the
params dialogue, but then nothing. Note:

- I have Akshay's Query Tool patch applied, and am running from git master

- PostgreSQL 9.4, with the debugger plugin in the public schema.

- Various functions in the pem schema tested, both with and without parameters.

- Both direct and indirect debugging have the same issue.

- No errors seen in the console, except when closing the debugger tab:

2016-04-05 17:27:37,720: INFO werkzeug: 127.0.0.1 - - [05/Apr/2016
17:27:37] "GET /debugger/close/8569717 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/dpage/git/pgadmin4-demo/web/pgadmin/tools/debugger/__init__.py",
line 397, in close
    session_id=obj['session_id'])
KeyError: 'session_id'


Thanks.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Hi Dave,

Updated above information.

We want this pgadmin4 log information to debug the debugger blank tab open issue.

Thanks,

Neel Patel

On Wed, Apr 6, 2016 at 11:50 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:

One more additional information.

Follow the below steps and send us pgAdmin4.log file.

• Browse till function which we want to debug. Do not click on the "Debug" context menu.
• Stop the server

            Stop the web server. 

• Enable the "Debug" option in the config file under "web" folder.

             Set the Log setting to "Debug" and save the log to files and send the updated debug log information.

• Start the server.

               Start the Web Server. 

• Now click on the "Debug" context menu and add requirement input. After clicking on the debug button on the input dialog send us the pgAdmin4.log file.

Thanks,

Neel Patel

On Wed, Apr 6, 2016 at 11:38 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:

Hi Dave,

We will fix the error "KeyError: 'session_id'" but regarding the blank tab, we have tried with PostgreSQL 9.4 with both the version of python (2.7 & 3.4) but we are not able to reproduce the issue.

Below are some of the questions to debug the issue.

• Is it only reproduce on PostgreSQL 9.4 ? Is it working on PostgreSQL 9.5 ?
• Is it possible to know the debugger version ?
• The issue is reproduce on specific functions ? If yes, can you please give definition of the function ?
• Are you able to see toolbar buttons ? Or nothing on the screen ? Only blank panel ?
• Can you please share request/response of last request URL sent to server in browser ?

Thanks,

Neel Patel

On Tue, Apr 5, 2016 at 9:59 PM, Dave Page <dpage@pgadmin.org> wrote:

Hi

On Tue, Apr 5, 2016 at 12:40 PM, Neel Patel <neel.patel@enterprisedb.com> wrote:
> Hi,
>
> Please find attached debugger v1 patch. To test this patch we need to apply
> the "function" module patch submitted by Khushboo.
>
> For the debugger functionality, direct and indirect debugging is working as
> per pgadmin3 except below TODOs.
>
> Below are the TODOs which will be submitted in next patch along with the
> fixed review comments.
>
> For the direct debugging, 'NULL' and 'Expression' column value should be
> validated before pressing the debug button.
> User can deposit the value of input arguments and local variable but need to
> be
> validated against respective data types.
> Currently we have tested for plpgsql functions only, We need to test against
> EDB SPL functions and trigger functions.
> For the direct debugging, once the execution is completed, only
> "Continue/Restart" button will be enabled and user should allow to restart
> the debugging with same function and previous values.
> Values entered by the user in input dialog during direct debugging should be
> saved.
>
>
> Currently we have tested this patch with PostgreSQL v9.5.
>
> Do review it and let us know for the comments.

I just get a blank tab when I try to debug something. I do see the
params dialogue, but then nothing. Note:

- I have Akshay's Query Tool patch applied, and am running from git master

- PostgreSQL 9.4, with the debugger plugin in the public schema.

- Various functions in the pem schema tested, both with and without parameters.

- Both direct and indirect debugging have the same issue.

- No errors seen in the console, except when closing the debugger tab:

2016-04-05 17:27:37,720: INFO werkzeug: 127.0.0.1 - - [05/Apr/2016
17:27:37] "GET /debugger/close/8569717 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/dpage/git/pgadmin4-demo/web/pgadmin/tools/debugger/__init__.py",
line 397, in close
    session_id=obj['session_id'])
KeyError: 'session_id'


Thanks.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Apr 6, 2016 at 8:57 AM, Neel Patel <neel.patel@enterprisedb.com> wrote:
> Hi Dave,
>
> I got the root cause. Issue will reproduce when we applied Akshay's query
> tool patch.
> Issue was once the panel was initialized, HTML file was not getting loaded
> in to the panel due to the change in frame.js.
>
> Both query tool and debugger patch file contains the change in "frame.js"
> file. Once you applied the query tool patch and after that debugger then
> debugger patch should fail but i don't know why you do not get any error.
>
> Solution:-
>
> Open "frame.js"  file and add the below line in "setTimeout" function which
> is required by debugger.
>
>                  pgBrowser.Events.trigger('pgadmin-browser:frame:urlloaded:'
> + that.name, frame, that.url, self);
>
> Do let us know if you still face the same issue.

Ahh, good catch - that got it :-)

Thanks!

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Hi

On Monday, April 18, 2016, Neel Patel <neel.patel@enterprisedb.com> wrote:

Hi Dave,

Please find inline comments with all the fixes.

Attached is the updated patch file. Do review it and let me know for any comments.

If you find any issues, let me know .Now, Working on the remaining TODOs as mentioned in below email.

Thanks - committed with some minor tweaks. One problem partly still remains though - you've partially copied the toolbar styling. Please use the actual classes used by the Properties panel. I've already updated the query tool in that way. Whilst your version looks much closer, it's missing the minimum button widths, and duplicates CSS unnecessarily.

Thanks.

 

On Fri, Apr 15, 2016 at 2:09 AM, Dave Page <dpage@pgadmin.org> wrote:

Hi

On Thu, Apr 14, 2016 at 1:52 PM, Neel Patel <neel.patel@enterprisedb.com> wrote:
> Hi,
>
> Please find attached v2 patch file of the debugger which fixes the below
> issues which was not present in the first patch.
> In this patch, we have added new table in sqlite database to store the
> functions arguments value user has given during debugging.
> After applying this patch, user needs to execute "setup.py" to create the
> new table in pgadmin4.db file.
>
> In direct debugging, when user debug the function then arguments values will
> be stored in the sqlite database so when user debug the same function again
> then previous values will be filled in the user input dialog.
> Once the execution is completed then user will be able to do the debug of
> the same function again by pressing the "Continue/Restart" button.
> User can debug the "procedure" which is supported in PPAS database.
> Replaced the "Glyphicon" with the "font-awesome" icons.

Very cool! Committed, understanding that there are still improvements
to be made.

> Below are the TODOs
>
> Validate the input arguments values changed by user while depositing the
> value during debugging.
> Need to implement the code folding in the codemirror editor area.
> As per the Ashesh's suggestion, need to add debug logs information so that
> we can get the state of the debug function. Also need to add "arrow" next to
> breakpoint in the gutters as per the pgadmin3.
> Need to add "Debug package initializer" in the user input dialog for the
> direct debugging.
> Last but not least "Review comments" :)

Here you go :-)

- Ensure all messages are gettext enabled.

 

Fixed. 

- Constructs like the following won't work, because Jinja will
evaluate the string " + err.errormsg + " before it ever gets evaluated
as JS by the browser.

Alertify.alert("{{ _('" + err.errormsg + "') }}");

 

Fixed.

 

- Please adjust the button bar to use the same styling as the button
bar on the Properties tab.

 

Fixed 

- Let's make the stack pane tab part of the tabset at the bottom of
the debugger, and ensure docking etc. works so tabs can be split off
and arranged around the main source window.

 

Fixed. Now stack pane will be displayed along with another panel at bottom and also docking has been introduced for all the panels so tabs will be arranged around main debugger panel.

 

- Stepping is quite slow. What's causing that? I wonder if we really
need to have all the one line SQL templates - they're probably quite
expensive to process.

Fixed. This is due to polling timeout was high (1 second) and we are getting delay in getting the results. Now polling timeout has reduced to to 200ms.

- Is backend_running.sql required? I've removed both versions as I
can't find any references to them. Are any other templates not
required?

Ok. No other templates.

Will log any other issues that come up in further work.

> Below functionalities are implemented but testing are pending.
>
> Trigger functions need to test with the debugger.
> Functions are tested with data types (like text, integer etc.)  but it needs
> to be tested with all the data types for direct debugging.
> Functions/Procedures need to test with PPAS 9.2 and earlier version where
> debugger version is different.

Thanks!

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Thanks - applied!

On Tuesday, April 19, 2016, Neel Patel <neel.patel@enterprisedb.com> wrote:

Hi Dave,

Please find the attached patch file with below fix.

• Remove the duplicate CSS and used the actual class for the debugger button toolbar.
• As per the Ashesh's comment, previously we used 2 wcDocker instance to arrange the multiple tabs to main debugger panel. Now with this patch file, we have used only 1 wcDocker instance.

Do review it and let us know for comments.

Thanks,

Neel  Patel

On Mon, Apr 18, 2016 at 6:07 PM, Dave Page <dpage@pgadmin.org> wrote:

Hi

On Monday, April 18, 2016, Neel Patel <neel.patel@enterprisedb.com> wrote:

Hi Dave,

Please find inline comments with all the fixes.

Attached is the updated patch file. Do review it and let me know for any comments.

If you find any issues, let me know .Now, Working on the remaining TODOs as mentioned in below email.

Thanks - committed with some minor tweaks. One problem partly still remains though - you've partially copied the toolbar styling. Please use the actual classes used by the Properties panel. I've already updated the query tool in that way. Whilst your version looks much closer, it's missing the minimum button widths, and duplicates CSS unnecessarily.

Thanks.

 

On Fri, Apr 15, 2016 at 2:09 AM, Dave Page <dpage@pgadmin.org> wrote:

Hi

On Thu, Apr 14, 2016 at 1:52 PM, Neel Patel <neel.patel@enterprisedb.com> wrote:
> Hi,
>
> Please find attached v2 patch file of the debugger which fixes the below
> issues which was not present in the first patch.
> In this patch, we have added new table in sqlite database to store the
> functions arguments value user has given during debugging.
> After applying this patch, user needs to execute "setup.py" to create the
> new table in pgadmin4.db file.
>
> In direct debugging, when user debug the function then arguments values will
> be stored in the sqlite database so when user debug the same function again
> then previous values will be filled in the user input dialog.
> Once the execution is completed then user will be able to do the debug of
> the same function again by pressing the "Continue/Restart" button.
> User can debug the "procedure" which is supported in PPAS database.
> Replaced the "Glyphicon" with the "font-awesome" icons.

Very cool! Committed, understanding that there are still improvements
to be made.

> Below are the TODOs
>
> Validate the input arguments values changed by user while depositing the
> value during debugging.
> Need to implement the code folding in the codemirror editor area.
> As per the Ashesh's suggestion, need to add debug logs information so that
> we can get the state of the debug function. Also need to add "arrow" next to
> breakpoint in the gutters as per the pgadmin3.
> Need to add "Debug package initializer" in the user input dialog for the
> direct debugging.
> Last but not least "Review comments" :)

Here you go :-)

- Ensure all messages are gettext enabled.

 

Fixed. 

- Constructs like the following won't work, because Jinja will
evaluate the string " + err.errormsg + " before it ever gets evaluated
as JS by the browser.

Alertify.alert("{{ _('" + err.errormsg + "') }}");

 

Fixed.

 

- Please adjust the button bar to use the same styling as the button
bar on the Properties tab.

 

Fixed 

- Let's make the stack pane tab part of the tabset at the bottom of
the debugger, and ensure docking etc. works so tabs can be split off
and arranged around the main source window.

 

Fixed. Now stack pane will be displayed along with another panel at bottom and also docking has been introduced for all the panels so tabs will be arranged around main debugger panel.

 

- Stepping is quite slow. What's causing that? I wonder if we really
need to have all the one line SQL templates - they're probably quite
expensive to process.

Fixed. This is due to polling timeout was high (1 second) and we are getting delay in getting the results. Now polling timeout has reduced to to 200ms.

- Is backend_running.sql required? I've removed both versions as I
can't find any references to them. Are any other templates not
required?

Ok. No other templates.

Will log any other issues that come up in further work.

> Below functionalities are implemented but testing are pending.
>
> Trigger functions need to test with the debugger.
> Functions are tested with data types (like text, integer etc.)  but it needs
> to be tested with all the data types for direct debugging.
> Functions/Procedures need to test with PPAS 9.2 and earlier version where
> debugger version is different.

Thanks!

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company