[[PageOutline]]


= GeniusWebPython =

''GeniusWeb'' is an open architecture for negotiation via the internet. The core of the architecture is a JSON based communication protocol that standardizes the communication messages comprising a negotiation. 
General info on GeniusWeb, including tutorials, can be found on the [https://ii.tudelft.nl/GeniusWeb/ GeniusWeb main page].

This project contains a [https://www.python.org/downloads/ c-python-3]-based implementation of GeniusWeb core objects. It allows easy (de)serialization of many data structures needed for negotiation in GeniusWeb. 

Some data structures have not yet been implemented in GeniusWebPython, this will be done as needed. Please contact us if you need some missing classes. A specification and full reference implementation in java of the GeniusWeb core is available [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb here]



== Installation
{{{#!td style="background: #eef"
You only need python for just writing and compiling your parties, and for debugging them with the simplerunner. You can also just use Eclipse or another IDE of your choice. No further installation is needed.
}}}

{{{#!td style="background: #fee"
We require a complete python installation, including pip and venv support.
Some python installations do not include pip and venv support. You may have
to install these separately. 
}}}

To compile and test parties you need [https://www.python.org/downloads/ python (3.8, 3.9)]. We recommend using an IDE for development and debugging, such as Eclipse with PyDev or PyCharm.

If you want a GeniusWebPythonPartiesServer, please check the  [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb#Installation GeniusWebServers installation instructions] 

== Security considerations
GeniusWebPython assumes python parties are packaged using the standard tar.gz distribution format used by Python. This implies that the package is not fully self-contained: it requires dependent libraries to be installed separately, according to setup.py included in the package. This dependency on later installation steps poses an additional security risk compared to the fully self-contained java jar packaging.

= Overview of the available core objects
Most classes in the python core are explained in the java reference implementation:
||= class =||= wiki reference =||
||issuevalue||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#issuevalue wiki issuevalue]||
||actions||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#Actions wiki actions]||
||bidspace||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb#BidSpace wiki bidspace]||
||inform||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#Inform wiki inform]||
||events||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#Events wiki events]||
||deadline||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#Timeline wiki deadline]||
||progress||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#Timeline wiki progress]||
||protocol||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb#Protocol wiki protocol]||
||connection||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#Connection wiki connection]||
||party||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#Party wiki party] and see below for details on writing a party||
||partystdio||a module to support running a party using pythons stdin and stdout||
||profile||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#profile wiki profile]||
||profileconnection||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#ProfileConnectionFactory wiki profileconnection]||
||references||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb/wiki/WikiStart#References wiki references]||
||voting||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb#Voting wiki voting]||
||opponentmodel||see [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb#OpponentModel wiki opponentmodel]||
||simplerunner||see the section on standalone running below. You can also have a look at the very similar [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb#Stand-aloneRunning wiki java simplerunner] but there are small differences||


= Example Parties
Example parties can be found [source:/exampleparties here]. Many of these are also included with the [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPythonPartiesServer parties server] to provide basic functionality.

Below is a table showing the currently available example parties and their properties. Notice, "Supported profile types" gives the class names of all supported types, so for instance "PartialOrdering" means all subclasses of PartialOrdering which also includes all UtilitySpaces.

||= Party =||= Protocol =||= Supported profile types =||= Parameters (default value) =||
||RandomParty||SAOP, Learn||LinearAdditive||minPower (2), maxPower (infinity). Used to control voting behaviour||
||stupidparty||SAOP||LinearAdditive||-||
||TimeDependentParty||SAOP, MOPAC, Learn||LinearAdditive||e (1.2) controls concession speed, minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response. ||
||Boulware||SAOP, MOPAC, Learn||LinearAdditive||minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response. ||
||Conceder||SAOP, MOPAC, Learn||LinearAdditive||minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response. ||
||Hardliner||SAOP, MOPAC, Learn||LinearAdditive||minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response. ||
||Linear||SAOP, MOPAC, Learn||LinearAdditive||minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response. ||



== Writing a party
Example parties can be found [source:/exampleparties here]. You can easily clone a party with SVN using {{{svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWebPython/exampleparties/randomparty/}}} (this clones randomparty, use a different name to fetch another example).

After cloning, create your venv and install the requirements, eg
{{{
cd randomparty
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
}}}

You can now run the junit tests and debug your party.

You may have to take different or additional steps if you use an IDE like eclipse with pydev, or pycharm. 



The basic structure of a party that supports the SAOP behaviour looks like this

{{{#!python
class RandomParty (DefaultParty):
    """
    Offers random bids until a bid with sufficient utility is offered
    """
    def __init__(self):
        super().__init__()
        self.getReporter().log(logging.INFO,"party is initialized")
        self._profile = None
        self._lastReceivedBid:Bid = None

    def notifyChange(self, info: Inform):
        #self.getReporter().log(logging.INFO,"received info:"+str(info))
        if isinstance(info,Settings) :
            settings:Settings=cast(Settings,info)    
            self._me = settings.getID()
            self._protocol:str = str(settings.getProtocol().getURI())
            self._progress = settings.getProgress()
            if "Learn" ==  self._protocol:
                self.getConnection().send(LearningDone(self._me))
            else:
                self._profile = ProfileConnectionFactory.create(info.getProfile().getURI(), self.getReporter())
        elif isinstance(info, ActionDone):
            action:Action=cast( ActionDone,info).getAction()
            if isinstance(action, Offer):
                self._lastReceivedBid = cast(Offer, action).getBid()
        elif isinstance(info, YourTurn):
            self._myTurn()
            if isinstance(self._progress, ProgressRounds) :
                self._progress = self._progress.advance()
        elif isinstance(info, Finished):
            self.terminate()
        else:
            self.getReporter().log(logging.WARNING, "Ignoring unknown info "+str(info))


    def getCapabilities(self) -> Capabilities:
        return Capabilities( set([ "SAOP", "Learn"]), set(['geniusweb.profile.utilityspace.LinearAdditive']))

    def getDescription(self) -> str:
        return "Offers random bids until a bid with sufficient utility is offered"

    def _myTurn(self):
        if self._isGood(self._lastReceivedBid) :
            action = Accept(self._me, self._lastReceivedBid)
        else:
            for _attempt in range(20):
                bid = self._getRandomBid(self._profile.getProfile().getDomain())
                if self._isGood(bid):
                    break
            action = Offer(self._me, bid);
        self.getConnection().send(action)

    def _isGood(self, bid:Bid)->bool:
        profile = self._profile.getProfile()
        return profile.getUtility(bid) > 0.6

...
}}}
Notice, above is slightly simplified code, for fully working code check the [source:exampleparties/randomparty/randomparty/RandomParty.py source code].

A party can directly use the Inform and Actions. This allows proper type checking of all the Party's code, and saves the programmer from interpreting JSON structures.

The party must follow the behaviours that it promises. The example above folows the SAOP behaviour.  

Use the logger for outputting messages from your party. 

{{{#!td style="background: #fee"
Do not use stdin and stdout/print in your party. These are used for communicating with the partyserver. Using them will disturb that communication and most likely result in your party requesting illegal actions. Executing an illegal action will result in your party being killed by the protocol.
}}}

== Compilation
If you want to deploy your party on your [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPythonPartiesServer PythonPartiesServer] you need to compile your party. Compile with {{{python3 setup.py sdist}}}. This gives you a  you get a {{{dist/yourparty-X.Y.Z.tar.gz}}} that can be copied into the pypartiesserver for deployment.

Make sure that you update the setup.py to do this: the name, packages, install_requires etc all may have to be updated, depending on what you changed in your project. Also notice that compilation takes all dependencies from the setup.py. It ignores your requirements.txt file.


=== Logging
The log file name can be set by the party in its constructor. If it uses the default (None), then the party main class name is used as log filename, and {{{.log}}} as extension.

So assuming your party main class is RandomParty, Logs are written to a file named RandomParty.log in the current working directory of the JVM (normally the directory from where you start up the tomcat server). 

=== Throwing exceptions
A special warning is in place regarding throwing exceptions from your implementation, particularly from notifyChange. In line with the general Listener interface, exceptions are only logged and your party usually is immediately disconnected if it throws any exception. Make sure that you free up any used resources to avoid memory leaks if you really need to throw an exception. 

=== Preparing the tar.gz file
Your party has to be packaged as a standard tar.gz file for deployment on the [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPythonPartiesServer pypartiesserver]. 

In contrast with java, python tar.gz files do not contain their dependent libraries. References to the dependent libraries are stored in the setup.py program inside the tar.gz file. The pypartiesserver runs a standard {{{pip install <your.tar.gz>}}} to install your party for use. 

Your tar.gz file must contain a {{{party.py}}} program in the root of the project, defining a function party() that returns your main class, like this:

{{{
from randomparty.RandomParty import RandomParty
def party():
    return RandomParty
}}}

Of course you should replace RandomParty with your party's main class name but the file name must stay {{{party.py}}}.

Refer to [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPythonPartiesServer#AddingorchangingaParty install new party] to deploy your party.tar.gz on the pythonpartiesserver. 

== Using PyDev in Eclipse
We recommend you use an IDE, eg Eclipse or PyCharm, to develop your python parties. Configuring Eclipse is quite elaborate, here is an outline but check other tutorials and videos (eg our own [https://ii.tudelft.nl/GeniusWeb/students.html tutorial video]) for more details.

* Start Eclipse and select your workspace
* Install PyDev. Go to Eclipse MarketPlace and search for PyDev. Click on Install and follow the instructions.
* Download an example project as usual in Eclipse:
  * Install an SVN client if you don't have one installed. Go to Eclipse MarketPlace and search for subclipse. Click on Install and follow the instructions. Also follow the javahl installation instructions. This will involve separate software as well.
  * Import the project: 
    * click File/Import/SVN/Checkout Projects from SVN. Click Next, 
    * select Create a new repository location and Next. Enter  e.g. https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWebPython/exampleparties/randomparty. 
    * Select the root folder and Finish. 
    * Select General/Project, Next, enter some project name "demo", Finish
* Create a virtual env for python
  * open terminal
  * Go the project workspace
  * {{{python3 -m venv venv; source venv/bin/activate; pip install -r requirements.txt}}}
* Set up Eclipse env
  * Window/Preferences/PyDev/Interpreters/Python Interpreter
  * New/Browse for python/pypy.exe
  * Browse to the venv/bin/python3 file you created
  * enter interpreter name, eg your project name "demo"
  * check also the site-packages and click OK
  * PyDev/Editor/Code Analysis/Mypy and check "Use Mypy?"
  * Apply and close
* Configure your project
  * Right click on demo project in navigator or project explorer
  * PyDev/Set as PyDev Project
  * Right click on demo project in navigator or project explorer and select "Properties"
  * Select PyDev/PyDev Interpreter Grammar
  * Select the Interpreter "demo".
  * Click on PyDev-PYTHONPATH
  * Click Add source folder and select the demo folder (the root of the demo project)
  * Go to PyDev/Editor/

Your party is now ready for programming. 
For instance you can run the unit tests by right-clicking the test folder and selecting Run As/Python unit test. Place a breakpoint and use debug instead of run to debug your party.
  

= Stand-alone running
The standalone runner is available in geniuswebcore 2. Stand-alone running works similar as the [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWeb#Stand-aloneRunning java stand-alone runner]. . 
* The parties you want to run must be python-based (using GeniusWebPython, not GeniusWeb) and in your pythonpath. Typically by adding them to your PYTHONPATH environment variable.
* The profiles and settings you want. To get them, you can do this:
 * do {{{svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWebPython/geniuswebcore/test/resources}}}
 * Copy the profiles (eg the entire jobs directory) you want to use to your project
 * Copy a settings.json to your project.
* Edit the settings.json file:
 * Change the parties to your local party as needed. Party references are URIs, starting with pythonpath: followed by the full.class.name. All parties must be in your python path. 
 * Change the profiles as needed. Profiles are URIs, starting with file: followed by the full/file/path (either absolute or relative).
 * parameters contains a json dict with additional initialization parameters for each party. They are a json dict, with strings as the key and any json object (but usually a string or number) as value.
* Run {{{python -m geniusweb.simplerunner.NegoRunner settings.json}}}. 
 * To do this from Eclipse, ensure your python3 venv is visible in the package explorer and browse to venv/lib/python3.8/site-packages/geniusweb/simplerunner/NegoRunner.py
* The log will be written to stdout by default. 
* Extract the data from the line {{{INFO:protocol ended..:}}}. It's a json structure with the final state of the session. You can easily parse it to a python dict using pyson.

== With the SimpleRunner GUI
If you do not provide settings and just run {{{python -m geniusweb.simplerunner.NegoRunner}}}, you get the GUI to enter your settings. It looks like this

[[Image(simplerunner.png )]]

It works just like the SimpleGUI in java:
* Enter the class path of your party without the leading {{{pythonpath:}}}, for example {{{test.testparty.TestParty.TestParty}}}
* Enter the parameters as json dictionary without the enclosing curly's, eg  "persistentstate": "a48b8bc9-4275-4a4e-bee9-6c0a734dc99a",        "negotiationdata": [] 
* Enter the profile without the leading {{{file:}}}, eg {{{src/test/resources/jobs/jobs2.json}}}


== Limitations
 SimpleRunner has a number of restrictions, compared to a run using a runserver and partyserver

* With stand-alone runner, your parties are run together in a single python interpreter. This implies
 * there may arise library version conflicts between parties. You will have to somehow resolve these to get the parties working together properly. If you can not resolve the conflicts, your only option is to use the server based runner.
 * There may be multithreading effects due to the python GIL.
* You can only use GeniusWebPython-based parties, not GeniusWeb-based parties
* Not all protocols are translated to python yet. You can only use protocols that have been translated.
* These issues are resolved by using the partiesserver. Therefore SimpleRunner in python might behave slightly different from running on the [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPythonPartiesServer python-partiesserver]. For this reason the partiesserver will be used for competitions and tests. Always make sure that your party runs properly also on the pythonpartiesserver.



= Debugging
For debugging you can use the normal python debugging facilities from your IDE (eg Eclipse PyDev). E.g. place a breakpoint in your code, and then run the unit test that reproduces the issue. The IDE switches to your program and you can step, inspect variables, etc. 

You can run your party best from a unit test, so that the fault situation is easily reproduced. But you can also use the simplerunner to run a session against some other party.


== GeniusWebPython sources
 You can browse the GeniusWeb core sources directly using the browse button at the right top of this page.

You can download the source code of this component using 

{{{
svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWebPython
}}}

Normal developers that write new parties do not need to install the GeniusWeb source code. Even if you want to debug/trace into the GeniusWeb code -- for instance for debugging or understanding the inner workings of geniusWeb--, IDEs like Eclipse automatically allow this as the pip-installed libraries contain sources.