Creating a development environment for MO plugins?

[trongus]

Concerning the half-dead status of SASTEP, I've been motivated to write some plugins for MO to contribute on the topic. However I'm having troubles on just setting a decent environment for developing plugins via PythonProxy.
 
Currently my goal is to establish a development environment. To do that, I need MO connected with a decent Python debugger, or at least a responsive REPL for interactive introspection on plugin APIs. What would you suggest?
 
[spoiler=What had been tried so far...]

I've been trying in two ways to achieve that: one is to remote debugging via existing Python IDE, and the other is to directly attaching a REPL module onto the MO. But neither have been successful so far.

 

1. Remote debugger

I've tried with PTVS, my favorite IDE. There are two ways to initiate connections from MO to PTVS: via "Attach to Process" menu or via ptvsd module. However the former failed to pause execution of MO which renders it useless. It works perfectly normal if you attach to an ordinary python process, so I suspect that PTVS failed to hook into the debuggee for some reason. The latter worked with a bit of trick (replacing python27.dll to stock 2.7.5 binary, adding _socket.pyd onto PYTHONPATH, etc...) but it still did not work correctly; I was able to attach into the process and see the code, but within a short time the debugger REPL becomes disconnected along with the rest of PTVS functionalities.

 

2. Launching REPL from MO

I've given a try on Spyder. However it seems to require full-featured PyQt5, which is not available in MO (it contains only three PYD modules and SIP). I did not dare to replace PyQt5 to full package for three reasons:

 

 - MO itself is built upon Qt. It would be dangerous to attach a full package of PyQt5 which ships its own Qt binaries on the same process.

 - I have no idea which version of PyQt5 is deployed with MO, so I can't find a compatible package of PyQt5. Github repo of MO does not tell me anything as they are shipped with binaries only.

 - Weirdly enough, googling PyQt5 for Python 2.7 32 bit does not show anything. PyPI refuses to show older version of PyQt5 which makes hard to find a compatible version to work with MO.

 

There are other Qt-based IDEs (such as Jupyter and so on), but it would show similar result. So I'm stuck here.

 

 

Edited September 12, 2016 by trongus

[Ganda]

First of all, if you're trying to access the VFS with the plugin, forget about it since MO doesn't expose that to python plugins (so C++ it the only option here).

 

Second, you can also forget about using debuggers or REPL's with MO, in fact any attempt to create any decent environment will fail. MO is embedding python, not extending it, this means that the C++ side can "import" python but not the other way around.

 

About the plugin API, I recommend reading this and extensive use of dir().

 

From your search of PyQt5 on PyPi I'm assuming you've never worked with it before so I also recommend reading this and this. Also the 3 binaries are enough for about 90% of plugins you can/want to make. (if you want to add more then yes, you'll have to build PyQt5 from source on Windows using a 32bit Python. The version used wasn't too hard to figure out.)

 

 

So, my final recommendations are Notepad++ or equivalent and lots and lots of patience :)

[trongus]

First of all, if you're trying to access the VFS with the plugin, forget about it since MO doesn't expose that to python plugins (so C++ it the only option here).

Manipulating VFS via MO interface won't be necessary.

 

Second, you can also forget about using debuggers or REPL's with MO, in fact any attempt to create any decent environment will fail. MO is embedding python, not extending it, this means that the C++ side can "import" python but not the other way around.

I don't plan to use C++ side code other than what is exposed already. If you're referring to a codebase something other than MO, it is surely possible to load pyd modules inside the embedded python so it can "import" them. In fact, it is quite trivial to install most of pip modules into embedded python, whether it be distributed in pure python or not.

 

It seems that you're correct on that I should forget about debuggers since they seem to be mostly incompatible with embedded python. However I believe that REPL can be initiated if a decent GUI is provided as a terminal. (Technically, it should be done by just popping win32console.AllocConsole() then launching code.interact() within the console. It is just not as useful as other good REPLs.)

 

About the plugin API, I recommend reading this and extensive use of dir().

 

So, my final recommendations are Notepad++ or equivalent and lots and lots of patience :)

Yes, I have read that doc before. The first thing I've decided upon reading it is that the first plugin I should write is to bootstrap the dev environment so that I can do some experiments in it. I don't want to do cumbersome dir-or-pprint-and-run-check routines every time when trying to inspect an object. That's where REPL should kick in.

 

From your search of PyQt5 on PyPi I'm assuming you've never worked with it before so I also recommend reading this and this. Also the 3 binaries are enough for about 90% of plugins you can/want to make. (if you want to add more then yes, you'll have to build PyQt5 from source on Windows using a 32bit Python. The version used wasn't too hard to figure out.)

Yes, I've never worked with PyQt5. I have avoided that since I believe it is a special case: Qt is linked against MO, which embeds python, which again refers to Qt from original MO via PyQt5 interface. Considering that standard distribution of PyQt5 ships Qt binaries by itself, I suspected that it should be used as standalone mode for default build.

 

Anyway, you're probably right on that it must be built manually. It would be great to clarify things before getting hand into it.

 

1) The screenshot says about the version of Qt, not PyQt5. Are you suggesting that the version string of PyQt5 matches to that of Qt?

2) Would PyQt5 integrate with already Qt application via embedded python?

 

Edited September 12, 2016 by trongus

[Ganda]

1) The screenshot says about the version of Qt, not PyQt5. Are you suggesting that the version string of PyQt5 matches to that of Qt?

2) Would PyQt5 integrate with already Qt application via embedded python?

1) Yup, each PyQt version is built against a specific Qt version.

2) As far as I could tell, all the python proxy/runner is doing is adding that data/ folder to sys.path so you can import from there, so you should be able to add anything. Not a C++ veteran though, so try and see I guess :p

 

 

EDIT: PyQt5 actually has wheels on PyPi o.0 Wow, the time I spent 3 months ago building that thing by hand

Edited September 12, 2016 by Ganda

[Tannin]

Sorry for the late reply. What Ganda said so far is true:

- Debugging the plugin is pretty nasty. What I did when I wrote the included plugins was write a mock for the mobase functionality I was using in MO, then run/debug the plugin as a regular python application for testing.

- The Python code includes PyQt which is a complete separate build of Qt so you can add your own PyQt libraries if you need. The only connection between C++Qt and PyQt is the parent widget you get passed in "setParentWidget(self, widget)". This is a widget handle created in C++Qt but you need it in PyQt to display modal dialogs for example. I honestly have no clue what would happen if the PyQt version and the C++Qt version were too different because then the python part might be using that widget "wrong".