Java to Python (j2p)

This tool can translate java programs into python programs. This is a complex mechanism and still experimental/in development. This documentation is very incomplete. This code is not yet published on the artifactory.

Some features:

• both single-file as multi-file projects
• translates calls to external libraries
• pluggable translators supporting calls to external liraries
• can generate pip-installable zip file that automatically (pip) installs all required dependencies.
• Another tool provided by us supports running these python zip files from java.
• Handles java method overloading using the @dispatch annotation from multipledispatch. If your java code uses overloading, your python code will require multipledispatch installed

The repo contains the translator in the module named "core". The other modules, ending with "-t", are translator plug-ins described below.

Internal mechanism

For normal use, you do not need to know exactly how the translation is done. But the way the translator works becomes highly relevant if you want to inject your own python translations into the code (see the following section).

Translation is done on a per-java-file basis.

There are two main translation components in the core:

1. The translator that parses java file with ​Javaparser and creates equivalent python code.
2. per-java-class translators that know how to translate all calls to any java class method or field into equivalent python code. This code is using introspection and assumes all used java classes are available, either through libraries or from the java compiler.

(1) generic and used for all java programs. It is currently pretty complete but details will be filled in over time as the need arises to support more java syntax.

(2) is currently very partial. The reason is that there are a huge number of java classes and even more 3rd party libraries, and almost every field and function in it will need a specialized translator. This will grow slowly over time as needed.

The mechanism assumes a fixed mapping from java to python objects, as in the table below. This fixed mapping is needed to ensure the translator can be sure which objects will appear on the python side of the translation, so that proper actions can be taken to further the translation.

constructs

The private keyword is reflected by the python convention of adding "" to private fields function names and method names

The static keyword results in fields being set in the class definition object.

Fields in Java have to be initialized in the init function in the python translation.

Overloaded methods can not be handled by default python and need to be handled using an external library multidispatch==0.6.0 and result in additional annotations @dispatch(...) in the translated code.

built-in "primitives"

java	python	remarks
String	str
int	int
float,Float,double,Double	float	in Java, float and double overflow at different places. Translation may fail if code depends on overflow behaviour
BigDecimal	Decimal
Map, HashMap	dict
Set	set

Classes

Classes are by default translated using the "Stub" translator which assumes assumes the class names and modules are identical in Python and Java, and that all functions have the same functions with the same arguments and types.

References to classes in the code (SomeClass.class in java) translates to SomeClass in Python. It is assumed the proper imports are done on the python side to enable this.

This default translation can be overridden using custom translators.

Translated classes

The number of translated classes is too big to even list here. Please check the source code

If function calls in python deliver a wrong object (not matching the above mapping), the translator has to inject additional code to convert it to the proper object. For example, Map.getKeys() in python would be set(map.keys()) where the extra set converts the dict_keys into a proper set.

Stream

Stream classes are currently not translated. It might be possible to translate them automatically but it looks pretty complex because the python way to deal with streams is list comprehension and the translation is quite far from straightforward.

Inner classes

Inner classes (classes inside classes) are not supported. You can not import, call or use inner classes. Inner classes can not be translated. This is for many reasons but the brief summary:

• The useful variant of inner class is the non-static inner class. This class can access fields in the enclosing class.
• Python does not support accessing fields of the enclosing class unless some tricks are applied. But these tricks would break uniformity of constructors.
• Many deep technical reasons #175 that largely complicate dealing with these.

We recommend now:

• manually override translation when you really need to call an inner class
• use package-private classes to replace your own inner classes.
• For the non-static inner class: pass the enclosing class as argument to the child class constructor. You may have to use more tricks to get around cyclic dependencies.

Exceptions

There are a very few exceptions. If some particular inner classes, particularly constants, are used in very specific translators, these translators may opt to attempt to recognise these and process them separately. One example is the JsonDeserializer (jackson-t package) that recognises use of "com.fasterxml.jackson.databind.JsonDeserializer.None". Check the documentation of the translators for details.

More notes:

• some classes like Arrays or Override do not have a direct python equivalent. calls to static functions in these classes can still be translated.
• Different java classes, eg Map AbstractMap and HashMap, may translate to the same python class (map). This can be done because translation is only 1 way.

Usage

For usage you have the choice between translating a single file or an entire package. We generally recommend translating the entire package.

Single File

If you want to translate a single java file or even just a string containing your program, that does not need any specialized libraries, you can do the approach as many tests in core do. Check the examples. The heart of the code will look like this, where most boilerplate is about setting up the javaparser:

		ParserConfiguration conf = new ParserConfiguration();
		CombinedTypeSolver typeSolver = new CombinedTypeSolver();
		typeSolver.add(new ReflectionTypeSolver(false));
		JavaSymbolSolver symbolSolver = new JavaSymbolSolver(typeSolver);
		conf.setSymbolResolver(symbolSolver);
		JavaParser parser = new JavaParser(conf);
		ParseResult<CompilationUnit> res = parser.parse(new File(javaFile));

		Block translation = Translator.translate(res.getResult().get());

After you get the translation, you can print it or put it in a .py file.

Entire Package

If you want to translate an entire package, first create a separate source directory containing all the java code that you want to translate. Also make sure that your code is compiled (eg, using the build-helper-maven-plugin , check the pom files). Then you just do

PyProgram program = PyProgram.fromDirectory(Paths.get("src/test/myprogram"));

This gives you a fully translated program that you can print or get a zip file from. To get a zip file, call program.getZip()

The zip file is ready for a pip install (eg in your own virtual environment) or running from java through our PythonVenv.

Overriding the translation

Comments can contain python code to override the automatic code, if the block starts with #PY. This code replaces the entire object (if/case/while block; statement) that follows the comment.

Place the #PY block directly before the object you want to override. For example if you place a #PY block before an annotation, you override only the annotation, not the actual method/class after the annotation. This unfortunately will look a bit non-java but it seems the best way to accurately target the override.

Python has strict requirements regarding indentation. To make this possible, we need to be strict about indentation as well. In a single line python comment, the code must look exactly like

 //#PY single_python_line 

Note the single whitespace after the #PY. Your code starts after this single whitespace.

In a multi line comment the code must look exactly like

/*#PY
 * codeline1
 * codeline2
 * ...
 */

Your code lines each start with "* ", note the whitespace after the star. You are free to indent before the "*".

Your code is automatically indented to the level needed at the insertion place in the code.

Code must be placed in either a standard block comment or a single line comment. Starting a javadoc with #PY is not allowed. This is to encourage proper use of javadoc.

A comment block overrides also annotations.

If the code block contains no code at all, it is translated as pass, to ensure that the code is a proper statement.

Translation modules

J2P has a modular translation mechanism. A translation module can be created separately and plugged in as needed, to add support for translating libraries. Also this allows to change the translation process, for instance to use another python library for the translation.

A number of translation modules are already available

module	what it translates	details	limitations
jackson-t	jackson serialization annotations	translates jackson to pyson annotations that look very similar to jackson	Covers what we need for translation of GeniusWeb
junit-t	junit calls assertTrue,assertEquals		Very limited, no support yet for junit annotations
tudutils-t	translates calls to utilities package		limited, currently mainly to support parts of immutablelist

Resulting file

FAQs

Question	Explanation
I'm getting "No translator found for X". But X is a class that I'm trying to translate	The java files you are trying to translate are probably not compiled by the java compiler. When the translator finds a method call, it needs the compiled java to determine the proper signature (function name and arguments).