= Java to Python (j2p)

This tool can translate java programs into python programs.
This is a complex mechanism and still experimental/in development. This documentation is very incomplete. This code is not yet published on the artifactory.

Some features:
* both single-file as multi-file projects
* translates calls to external libraries
* pluggable translators supporting calls to external liraries
* can generate pip-installable zip file that automatically (pip) installs all required dependencies.
* [https://tracinsy.ewi.tudelft.nl/pubtrac/Utilities/wiki/pyrunner Another tool  provided by us] supports running these python zip files from java.
* Handles java method overloading using the @dispatch annotation from multipledispatch. If your java code uses overloading, your python code will require multipledispatch installed 

The repo contains the translator in the module named "core". The other modules, ending with "-t", are translator plug-ins described below.

== Internal mechanism
For normal use, you do not need to know exactly how the translation is done. But the way the translator works becomes highly relevant if you want to inject your own python translations into the code (see the following section). 

Translation is done on a per-java-file basis. 

There are two main translation components in the core:
1. The translator that parses java file with [https://github.com/javaparser/javaparser Javaparser] and creates equivalent python code.
2. per-java-class translators that know how to translate all calls to any java class method or field into equivalent python code. This code is using introspection and assumes all used java classes are available, either through libraries or from the java compiler.

(1) generic and used for all java programs. It is currently pretty complete but details will be filled in over time as the need arises to support more java syntax.

(2) is currently very partial. The reason is that there are a huge number of java classes and even more 3rd party libraries, and almost every field and function in it will need a specialized translator. This will grow slowly over time as needed.


The mechanism assumes a fixed mapping from java to python objects, as in the table below. This fixed mapping is needed to ensure the translator can be sure which objects will appear on the python side of the translation, so that proper actions can be taken to further the translation.

==== constructs
The {{{private}}} keyword is reflected by the python convention of adding "__" to private fields function names and method names

The {{{static}}} keyword results in fields being set in the class definition object. 

Fields in Java have to be initialized in the __init__ function in the python translation.

Overloaded methods can not be handled by default python and need to be handled using an external library {{{multidispatch==0.6.0}}} and result in additional annotations {{{@dispatch(...)}}} in the translated code.

==== built-in "primitives"
||=java=||=python=||=remarks=||
||String||str||
||int||int||
||float,Float,double,Double||float||in Java, float and double overflow at different places. Translation may fail if code depends on overflow behaviour||
||BigDecimal||Decimal||
||Map, HashMap||dict||
||Set||set||

==== Classes
Classes are by default translated using the "Stub" translator which assumes
assumes the class names and modules are identical in Python and Java,
and that all functions have the same functions with the same arguments and types.

References to classes in the code (SomeClass.class in java) translates to SomeClass in Python. It is assumed the proper imports are done on the python side to enable this.

This default translation can be overridden using custom translators. 

==== Translated classes
The number of translated classes is too big to even list here. Please check the  [source:/Utilities/browser/java2python/core/src/main/java/tudelft/utilities/j2p/t/java source code]


If function calls in python deliver a wrong object (not matching the above mapping), the translator has to inject additional code to convert it to the proper object. For example, {{{Map.getKeys()}}} in python would be {{{set(map.keys())}}} where the extra {{{set}}} converts the dict_keys into a proper set.

=== Stream
Stream classes are currently not translated. It might be possible to translate them automatically but it looks pretty complex because the python way to deal with streams is list comprehension and the translation is quite far from straightforward.

==== Inner classes
Inner classes (classes inside classes) are **not supported**. You can not import, call or use inner classes. Inner classes can not be translated. This is for many reasons but the brief summary:
* The useful variant of inner class is the non-static inner class. This class can access fields in the enclosing class.
* Python does not support accessing fields of the enclosing class unless some tricks are applied. But these tricks would break uniformity of constructors.
* Many deep technical reasons #175 that largely complicate dealing with these.

We recommend now:
* manually override translation when you really need to call an inner class
* use package-private classes to replace your own inner classes.
* For the non-static inner class: pass the enclosing class as argument to the child class constructor. You may have to use more tricks to get around cyclic dependencies.

===== Exceptions
There are a very few exceptions. If some particular inner classes, particularly constants, are used in very specific translators, these translators may opt to attempt to recognise these and process them separately. One example is the JsonDeserializer (jackson-t package) that recognises use of "com.fasterxml.jackson.databind.JsonDeserializer.None". Check the documentation of the translators for details. 

==== More notes:
* some classes like {{{Arrays}}} or {{{Override}}} do not have a direct python equivalent. calls to static functions in these classes can still be translated. 
* Different java classes, eg {{{Map}}} {{{AbstractMap}}} and {{{HashMap}}}, may translate to the same python class ({{{map}}}). This can be done because  translation is only 1 way.

== Usage
For usage you have the choice between translating a single file or an entire package. We generally recommend translating the entire package.

=== Single File
If you want to translate a single java file or even just a string containing your program, that does not need any specialized libraries, you can do the approach as many tests in core do. Check the examples. The heart of the code will look like this, where most boilerplate is about setting up the javaparser:

{{{
		ParserConfiguration conf = new ParserConfiguration();
		CombinedTypeSolver typeSolver = new CombinedTypeSolver();
		typeSolver.add(new ReflectionTypeSolver(false));
		JavaSymbolSolver symbolSolver = new JavaSymbolSolver(typeSolver);
		conf.setSymbolResolver(symbolSolver);
		JavaParser parser = new JavaParser(conf);
		ParseResult<CompilationUnit> res = parser.parse(new File(javaFile));

		Block translation = Translator.translate(res.getResult().get());

}}}

After you get the translation, you can print it or put it in a {{{.py}}} file.

=== Entire Package
If you want to translate an entire package, first create a separate source directory containing all the java code that you want to translate. Also make sure that your code is compiled (eg, using the build-helper-maven-plugin , check the pom files). Then you just do
{{{
PyProgram program = PyProgram.fromDirectory(Paths.get("src/test/myprogram"));
}}}

This gives you a fully translated program that you can print or get a zip file from. To get a zip file, call
{{{program.getZip() }}}

The zip file is ready for a pip install (eg in your own virtual environment) or running from java through our PythonVenv.


== Overriding the translation
Comments can contain python code to override the automatic code, if the block starts with {{{#PY}}}. This code replaces the entire object (if/case/while block; statement) that follows the comment.

Place the #PY block directly before the object you want to override. For example if you place a #PY block before an annotation, you override only the annotation, not the actual method/class after the annotation.  This unfortunately will look a bit non-java but it seems the best way to accurately target the  override.

Python has strict requirements regarding indentation. To make this possible, we need to be strict about indentation as well. In a single line python comment, the code must look exactly like

{{{
 //#PY single_python_line 
}}}

  Note the single whitespace after the #PY. Your code starts after this single whitespace.

 In a multi line comment the code must look exactly like
{{{
/*#PY
 * codeline1
 * codeline2
 * ...
 */
}}}

Your code lines each start with "* ", note the whitespace after the star. You are free to indent before the "*".

Your code is automatically indented to the level needed at the insertion place in the code.

Code must be placed in either a standard block comment or a single line
comment. Starting a javadoc with {{{#PY}}} is not
allowed. This is to encourage proper use of javadoc.
 
  A comment block overrides also annotations.

 If the code block contains no code at all, it is translated as
 {{{pass}}}, to ensure that the code is a proper statement.


== Translation modules
J2P has a modular translation mechanism. A translation module can be created separately and plugged in as needed, to add support for translating libraries. Also this allows to change the translation process, for instance to use another python library for the translation.

A number of translation modules are already available

||=module=||=what it translates=||=details=||=limitations=||
||jackson-t||jackson serialization annotations||translates jackson to pyson annotations that look very similar to jackson||Covers what we need for translation of GeniusWeb||
||junit-t||junit calls assertTrue,assertEquals|| ||Very limited, no support yet for junit annotations||
||tudutils-t||translates calls to [https://tracinsy.ewi.tudelft.nl/pubtrac/Utilities/wiki/utilities utilities] package|| ||limited, currently mainly to support parts of immutablelist||



== Resulting file


== FAQs
||=Question=||=Explanation=||
||I'm getting "No translator found for X". But X is a class that I'm trying to translate||The java files you are trying to translate are probably not compiled by the java compiler. When the translator finds a method call, it needs the compiled java to determine the proper signature (function name and arguments). ||