GeniusWebPython

GeniusWeb is an open architecture for negotiation via the internet. The core of the architecture is a JSON based communication protocol that standardizes the communication messages comprising a negotiation. General info on GeniusWeb, including tutorials, can be found on the ​GeniusWeb main page.

This project contains a ​c-python-3-based implementation of GeniusWeb core objects. It allows easy (de)serialization of many data structures needed for negotiation in GeniusWeb.

Some data structures have not yet been implemented in GeniusWebPython, this will be done as needed. Please contact us if you need some missing classes. A specification and full reference implementation in java of the GeniusWeb core is available ​here

Python in general is about 100 times slower than java. For small domains, up to 10 million bids, this is usually no problem. But for larger domains we recommend using the java version of GeniusWeb.

Installation

You only need python for just writing and compiling your parties, and for debugging them with the simplerunner. You can also just use Eclipse or another IDE of your choice. No further installation is needed.

We require a complete python installation, including pip and venv support. Some python installations do not include pip and venv support. You may have to install these separately.

To compile and test parties you need ​python (3.8, 3.9). We recommend using an IDE for development and debugging, such as Eclipse with PyDev or PyCharm.

If you want a GeniusWebPythonPartiesServer, please check the ​GeniusWebServers installation instructions

Security considerations

GeniusWebPython assumes python parties are packaged using the standard tar.gz distribution format used by Python. This implies that the package is not fully self-contained: it requires dependent libraries to be installed separately, according to setup.py included in the package. This dependency on later installation steps poses an additional security risk compared to the fully self-contained java jar packaging.

Overview of the available core objects

Most classes in the python core are explained in the java reference implementation:

class	wiki reference
issuevalue	see ​wiki issuevalue
actions	see ​wiki actions
bidspace	see ​wiki bidspace
inform	see ​wiki inform
events	see ​wiki events
deadline	see ​wiki deadline
progress	see ​wiki progress
protocol	see ​wiki protocol
connection	see ​wiki connection
party	see ​wiki party and see below for details on writing a party
partystdio	a module to support running a party using pythons stdin and stdout
profile	see ​wiki profile
profileconnection	see ​wiki profileconnection
references	see ​wiki references
voting	see ​wiki voting
opponentmodel	see ​wiki opponentmodel
simplerunner	see the section on standalone running below. You can also have a look at the very similar ​wiki java simplerunner but there are small differences

Example Parties

Example parties can be found here. Many of these are also included with the ​parties server to provide basic functionality.

Below is a table showing the currently available example parties and their properties. Notice, "Supported profile types" gives the class names of all supported types, so for instance "PartialOrdering" means all subclasses of PartialOrdering which also includes all UtilitySpaces.

Party	Protocol	Supported profile types	Parameters (default value)
RandomParty	SAOP, Learn	LinearAdditive	minPower (2), maxPower (infinity). Used to control voting behaviour
stupidparty	SAOP	LinearAdditive	-
TimeDependentParty	SAOP, MOPAC, Learn	LinearAdditive	e (1.2) controls concession speed, minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response.
Boulware	SAOP, MOPAC, Learn	LinearAdditive	minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response.
Conceder	SAOP, MOPAC, Learn	LinearAdditive	minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response.
Hardliner	SAOP, MOPAC, Learn	LinearAdditive	minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response.
Linear	SAOP, MOPAC, Learn	LinearAdditive	minPower (1), maxPower (infinity). Used to control voting behaviour. delay (0) delays response.

Writing a party

Example parties can be found here. You can easily clone a party with SVN using svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWebPython/exampleparties/randomparty/ (this clones randomparty, use a different name to fetch another example).

After cloning, create your venv and install the requirements, eg

cd randomparty
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

You can now run the junit tests and debug your party.

You may have to take different or additional steps if you use an IDE like eclipse with pydev, or pycharm.

The basic structure of a party that supports the SAOP behaviour looks like this

class RandomParty (DefaultParty):
    """
    Offers random bids until a bid with sufficient utility is offered
    """
    def __init__(self):
        super().__init__()
        self.getReporter().log(logging.INFO,"party is initialized")
        self._profile = None
        self._lastReceivedBid:Bid = None

    def notifyChange(self, info: Inform):
        #self.getReporter().log(logging.INFO,"received info:"+str(info))
        if isinstance(info,Settings) :
            settings:Settings=cast(Settings,info)    
            self._me = settings.getID()
            self._protocol:str = str(settings.getProtocol().getURI())
            self._progress = settings.getProgress()
            if "Learn" ==  self._protocol:
                self.getConnection().send(LearningDone(self._me))
            else:
                self._profile = ProfileConnectionFactory.create(info.getProfile().getURI(), self.getReporter())
        elif isinstance(info, ActionDone):
            action:Action=cast( ActionDone,info).getAction()
            if isinstance(action, Offer):
                self._lastReceivedBid = cast(Offer, action).getBid()
        elif isinstance(info, YourTurn):
            self._myTurn()
            if isinstance(self._progress, ProgressRounds) :
                self._progress = self._progress.advance()
        elif isinstance(info, Finished):
            self.terminate()
        else:
            self.getReporter().log(logging.WARNING, "Ignoring unknown info "+str(info))


    def getCapabilities(self) -> Capabilities:
        return Capabilities( set([ "SAOP", "Learn"]), set(['geniusweb.profile.utilityspace.LinearAdditive']))

    def getDescription(self) -> str:
        return "Offers random bids until a bid with sufficient utility is offered"

    def _myTurn(self):
        if self._isGood(self._lastReceivedBid) :
            action = Accept(self._me, self._lastReceivedBid)
        else:
            for _attempt in range(20):
                bid = self._getRandomBid(self._profile.getProfile().getDomain())
                if self._isGood(bid):
                    break
            action = Offer(self._me, bid);
        self.getConnection().send(action)

    def _isGood(self, bid:Bid)->bool:
        profile = self._profile.getProfile()
        return profile.getUtility(bid) > 0.6

...

Notice, above is slightly simplified code, for fully working code check the source code​.

A party can directly use the Inform and Actions. This allows proper type checking of all the Party's code, and saves the programmer from interpreting JSON structures.

The party must follow the behaviours that it promises. The example above folows the SAOP behaviour.

Use the logger for outputting messages from your party.

Do not use stdin and stdout/print in your party. These are used for communicating with the partyserver. Using them will disturb that communication and most likely result in your party requesting illegal actions. Executing an illegal action will result in your party being killed by the protocol.

Compilation

If you want to deploy your party on your ​PythonPartiesServer you need to compile your party. Compile with python3 setup.py sdist. This gives you a you get a dist/yourparty-X.Y.Z.tar.gz that can be copied into the pypartiesserver for deployment.

Make sure that you update the setup.py to do this: the name, packages, install_requires etc all may have to be updated, depending on what you changed in your project. Also notice that compilation takes all dependencies from the setup.py. It ignores your requirements.txt file.

Logging

The log file name can be set by the party in its constructor. If it uses the default (None), then the party main class name is used as log filename, and .log as extension.

So assuming your party main class is RandomParty, Logs are written to a file named RandomParty.log in the current working directory of the JVM (normally the directory from where you start up the tomcat server).

Throwing exceptions

A special warning is in place regarding throwing exceptions from your implementation, particularly from notifyChange. In line with the general Listener interface, exceptions are only logged and your party usually is immediately disconnected if it throws any exception. Make sure that you free up any used resources to avoid memory leaks if you really need to throw an exception.

Preparing the tar.gz file

Your party has to be packaged as a standard tar.gz file for deployment on the ​pypartiesserver.

In contrast with java, python tar.gz files do not contain their dependent libraries. References to the dependent libraries are stored in the setup.py program inside the tar.gz file. The pypartiesserver runs a standard pip install <your.tar.gz> to install your party for use.

Your tar.gz file must contain a party.py program in the root of the project, defining a function party() that returns your main class, like this:

from randomparty.RandomParty import RandomParty
def party():
    return RandomParty

Of course you should replace RandomParty with your party's main class name but the file name must stay party.py.

Refer to ​install new party to deploy your party.tar.gz on the pythonpartiesserver.

Using PyDev in Eclipse

We recommend you use an IDE, eg Eclipse or PyCharm, to develop your python parties. Configuring Eclipse is quite elaborate, here is an outline but check other tutorials and videos (eg our own ​tutorial video) for more details.

• Start Eclipse and select your workspace
• Install PyDev. Go to Eclipse MarketPlace and search for PyDev. Click on Install and follow the instructions.
• Download an example project as usual in Eclipse:
  • Install an SVN client if you don't have one installed. Go to Eclipse MarketPlace and search for subclipse. Click on Install and follow the instructions. Also follow the javahl installation instructions. This will involve separate software as well.
  • Import the project:
    • click File/Import/SVN/Checkout Projects from SVN. Click Next,
    • select Create a new repository location and Next. Enter e.g. ​https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWebPython/exampleparties/randomparty.
    • Select the root folder and Finish.
    • Select General/Project, Next, enter some project name "demo", Finish
• Create a virtual env for python
  • open terminal
  • Go the project workspace
  • python3 -m venv venv; source venv/bin/activate; pip install -r requirements.txt
• Set up Eclipse env
  • Window/Preferences/PyDev/Interpreters/Python Interpreter
  • New/Browse for python/pypy.exe
  • Browse to the venv/bin/python3 file you created
  • enter interpreter name, eg your project name "demo"
  • check also the site-packages and click OK
  • PyDev/Editor/Code Analysis/Mypy and check "Use Mypy?"
  • Apply and close
• Configure your project
  • Right click on demo project in navigator or project explorer
  • PyDev/Set as PyDev Project
  • Right click on demo project in navigator or project explorer and select "Properties"
  • Select PyDev/PyDev Interpreter Grammar
  • Select the Interpreter "demo".
  • Click on PyDev-PYTHONPATH
  • Click Add source folder and select the demo folder (the root of the demo project)
  • Go to PyDev/Editor/

Your party is now ready for programming. For instance you can run the unit tests by right-clicking the test folder and selecting Run As/Python unit test. Place a breakpoint and use debug instead of run to debug your party.

Stand-alone running

The standalone runner is available in geniuswebcore 2. Stand-alone running works similar as the ​java stand-alone runner. .

• The parties you want to run must be python-based (using GeniusWebPython, not GeniusWeb) and in your pythonpath. Typically by adding them to your PYTHONPATH environment variable.
• The profiles and settings you want. To get them, you can do this:
  • do svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWebPython/geniuswebcore/test/resources
  • Copy the profiles (eg the entire jobs directory) you want to use to your project
  • Copy a settings.json to your project.
• Edit the settings.json file:
  • Change the parties to your local party as needed. Party references are URIs, starting with pythonpath: followed by the full.class.name. All parties must be in your python path.
  • Change the profiles as needed. Profiles are URIs, starting with file: followed by the full/file/path (either absolute or relative).
  • parameters contains a json dict with additional initialization parameters for each party. They are a json dict, with strings as the key and any json object (but usually a string or number) as value.
• Run python -m geniusweb.simplerunner.NegoRunner settings.json.
  • To do this from Eclipse, ensure your python3 venv is visible in the package explorer and browse to venv/lib/python3.8/site-packages/geniusweb/simplerunner/NegoRunner.py
• The log will be written to stdout by default.
• Extract the data from the line INFO:protocol ended..:. It's a json structure with the final state of the session. You can easily parse it to a python dict using pyson.

With the SimpleRunner GUI

If you do not provide settings and just run python -m geniusweb.simplerunner.NegoRunner, you get the GUI to enter your settings. It looks like this

It works just like the SimpleGUI in java:

• Enter the class path of your party without the leading pythonpath:, for example test.testparty.TestParty.TestParty
• Enter the parameters as json dictionary without the enclosing curly's, eg "persistentstate": "a48b8bc9-4275-4a4e-bee9-6c0a734dc99a", "negotiationdata": []
• Enter the profile without the leading file:, eg src/test/resources/jobs/jobs2.json

Limitations

SimpleRunner has a number of restrictions, compared to a run using a runserver and partyserver

• With stand-alone runner, your parties are run together in a single python interpreter. This implies
  • there may arise library version conflicts between parties. You will have to somehow resolve these to get the parties working together properly. If you can not resolve the conflicts, your only option is to use the server based runner.
  • There may be multithreading effects due to the python GIL.
• You can only use GeniusWebPython-based parties, not GeniusWeb-based parties
• Not all protocols are translated to python yet. You can only use protocols that have been translated.
• These issues are resolved by using the partiesserver. Therefore SimpleRunner in python might behave slightly different from running on the ​python-partiesserver. For this reason the partiesserver will be used for competitions and tests. Always make sure that your party runs properly also on the pythonpartiesserver.

Debugging

For debugging you can use the normal python debugging facilities from your IDE (eg Eclipse PyDev). E.g. place a breakpoint in your code, and then run the unit test that reproduces the issue. The IDE switches to your program and you can step, inspect variables, etc.

You can run your party best from a unit test, so that the fault situation is easily reproduced. But you can also use the simplerunner to run a session against some other party.

GeniusWebPython sources

You can browse the GeniusWeb core sources directly using the browse button at the right top of this page.

You can download the source code of this component using

svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWebPython

Normal developers that write new parties do not need to install the GeniusWeb source code. Even if you want to debug/trace into the GeniusWeb code -- for instance for debugging or understanding the inner workings of geniusWeb--, IDEs like Eclipse automatically allow this as the pip-installed libraries contain sources.