[[PageOutline]] = PyRunner PyRunner is a tool to run Python code from Java. The PyRunner can take a ready-to-use python pip-installable tar.gz file that was generated using the setuptool in python. This file contains all dependencies. The PyRunner then creates a virtual environment, and installs your tar.gz file and all dependencies in the virtual environment. At this moment PyRunner is compatible with python 3.8, 3.9 and 3.10. === Network PyRunner in many cases is dependent on the network, because most systems will use pip to install 3rd party libraries, even for fundamental things like wheel for installation support or multipledispatch for implementing multiple same-name methods with different arity. Python does not have a caching mechanism nor a build-with-dependencies. === Preparing Python This is a tool to run python code from Java. The tool requires FULL python to be installed, including pip and venv. Some python installations require manual post-installation of these. To do this you need something like {{{ sudo apt-get update sudo apt install python3-pip sudo apt install python3.-venv }}} where is 8,9 etc depending on your python version. To check whether your python is ready for use, run this command {{{ python -m venv venv }}} If you get any errors, check the error and do further installations. Unfortunately this seems the python way of getting a working installation. === PYTHON3EXE After installing python, you have to set the {{{PYTHON3EXE}}} environment variable to point to the python3 executable. On ubuntu, you can find the location with {{{which python3}}}. How you set this varies depending on how you want to use the PythonVenv (from Eclipse, from command line, double clicking etc; and which OS). If you want to use a venv, PYTHON3EXE usually points to YOURVENV/bin/python3. That version of python automatically uses the libraries that were pip-installed there. === Usage The PythonVenv usage is roughly like this: * create a PythonVenv E, possibly already with the targz code you want to run. You can use the usual {{{python3 setup.py sdist}}}, or another mechanism such as the java2python compiler. * Pass your targz directly into the PythonVenv constructor * or if you dont have a targz, copy the code into E. * call the code you want to run eg from [source:pyrunner/src/test/java/tudelft/utilities/pyrunner/PythonVenvTest.java PythonVenvTest]: {{{ PythonVenv env = new PythonVenv( new File("src/test/resources/helloworld-1.0.0.tar.gz"), logger); String res = env.call(Arrays.asList("-c","import helloworld; helloworld.helloworld()"), 40000); }}} * call E.remove to remove the venv The PythonVenv will run completely in its own venv. The zip file is installed with {{{pip}}} and thus respects all the setup including pip installing dependencies (which may be downloaded from the web). ==== results To determine the result(s), stdin and stderr are collected and checked. * If the deadline is exceeded, an InterruptedException is thrown * If there is content in stderr, the result is a PythonError containing the stderr message. * otherwise, the call was a success and the value in stdin is returned. This will normally contain all the text that was printed to stdout in python. Your program can decide then what to do with that. === Run all tests programatically PythonVenv also has support to run unit tests with discovery. Your java code is assumed to have ...Test.java files that are compiled into your zip file, If you use java2python, you can keep the src and test java code separate and use something like {{{ PyProgram tests = PyProgram.fromDirectories(Arrays.asList(src, test)); }}} The src and test dir can have overlapping modules as usual in java (eg, you have {{{geniuswebsrc/issuevalue/Bid.java}}} and a matching {{{geniuswebtest/issuevalue/BidTest.java}}}. This results in the tests in the same directory in the compiled code, while being separate in the sources. Having it all in a single zip file allows for all dependencies, both for the code and for the testing, to be installed in one go. Then you run the test using {{{ PythonVenv py = new PythonVenv(tests.getZip(), logger); TestResult res = py.test("geniusweb", 60000); }}} {{{#!td style="background: #eef" When using a zipped python resource you can not use python unittest directly (using {{{python -m unittest discover}}}) because python's {{{pip install}}} procedure will remove all the {{{__init__.py}}} files that are essential for discover. }}} ==== results When a test is run, the stdin and stderr are used to collect the test results. Therefore any message to stderr will be interpreted as a test failure. When the python program finishes, it wraps the collected python TestResult in a json-stringified object which is then sent to stdout, which is then collected on the java side and converted into a java Testresult. {{{#!td style="background: #fee" Your tests should NOT write to stdout, as this will mess up the TestResult }}} === Running/Debugging unitpy tests If the unittests are using [wiki:unitpy], the test class does not need to extend TestCase and is by itself not discoverable. If the test class was compiled with the j2p and the junit-t translator, then a bit of glue code has been added to your module. This glue code is an additional class like this: {{{ class Test(TestCase): def test(self): runTests(YOURTESTCLASS) }}} where YOURTESTCLASS is the name of the class containing your unityp tests. If this glue code is there, or if you can add this glue code manually, you can use the normal procedure with {{{python -m unittest path.to.YOURTESTCLASS}}}. This only works for single classes because pip install removes all {{{__init__.py}}} files, breaking auto discovery of the tests. Either with or without glue code, you can use runTests directly, using {{{python -c "from unitpy.Runner import runTests; from path.to.YOURTESTCLASS import YOURTESTCLASS; runTests(YOURTESTCLASS)" }}}. For debug use {{{ python -c "import pdb; from unitpy.Runner import runTests; from path.to.YOURTESTCLASS import YOURTESTCLASS; pdb.set_trace(); runTests(YOURTESTCLASS)" -m pdb }}}. We recommend then to set a breakpoint using {{{b /ABSOLUTE/PATH/TO/YOURMODULE.py:LINENR}}} instead of debugging the entire runTest code. And then use 'c' to continue to the breakpoint. Theoretically, for debugging you would need to **install** pytest ({{{sudo apt install python3-pytest}}}) after which you can do {{{pytest-3 --trace path/to/YOURTESTCLASS.py}}}. Unfortunately there seems a bug in pytest-3 and it does not debug at the first test. This makes this approach mostly useless.