[[PageOutline]] = GeniusWeb = ''GeniusWeb'' is an open architecture for negotiation via the internet. The core of the architecture is a JSON based communication protocol that standardizes the communication messages comprising a negotiation. This project also contains a java-based reference implementation. It provides protocols, data structures and many components needed in negotiation such as isues, values, domain descriptions, preference profiles, timelines, negotiation protocols, json serialization of these items, automatic negotiation software, etc. This page contains technical documentation describing the core json structures for describing issues, values, bids, offers, negotiation events etc. General info on GeniusWeb, including tutorials, can be found on the [https://ii.tudelft.nl/GeniusWeb/ GeniusWeb main page]. On this page, we take a hands-on, implementation focused approach rather than specifying json structures. The servers (see table below) discuss the structures in more detail. Also for the exact json structures please refer to the examples and the json annotations in the code. == GeniusWeb overview GeniusWeb contains a number of components ||= name =||= description =||= more information =|| ||the core ||the data structures for issues, values, bids, profiles, events and actions, parties etc.||here|| ||profilesserver||a web server that provides profiles and domain descriptions||[https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebProfilesServer profiles server]|| ||partiesserver||A web server that provides instances of running parties to use for negotiation||[https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPartiesServer parties server]|| ||runserver||A web server that can run sessions and tournaments||[https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebRunServer run server]|| You can either run these servers locally on either your computer or on your web server, or use a public server somewhere else. For creating your own profile or party, you only need the core. If you want to get a quick top-down idea how GeniusWeb works, we suggest to install the servers and contact the runserver to run a session. == Installation The GeniusWeb core code does not need to be installed. Check the profilesserver, partiesserver and runserver wiki pages (see above) for installation details. == Server choice, privacy and security You can choose to either run services privately, on your intranet, or completely open on the internet. ||= privacy need =||= configuration =|| ||completely private, no sharing with others ||All services run on tomcat server(s) on your computer or intranet. Outside network access is blocked. You need copies of the profiles and parties (jar files) you want to run.|| ||dedicated business negotiation, competition : share your party with the world, but not its code ||You host your own partiesserver that is connected to the web*. || ||shared profile ||profilesserver on own server or on an existing profilesserver on the web* || ||public party with built-in private profile||Hard-code the profile in the party. Party runs as in competition. || ||semi-private sessions/tournaments||Run your own runserver, possibly even from behind a firewall or on your local computer. Existing parties- and profilesservers are used in your runs. Negotiation results are handled only on your computer and not visible outside. Others might be able to see parties act on your behalf, but it's not trivial to determine that they are negotiating against each other, especially if these parties are on different partiesservers.|| ||run your own protocols||as with running semi-private sessions/tournaments|| * All firewalls above the server have to be configured appropriately, most corporations have firewalls between your computer and inside the company there is only connection between company computers. = Core overview This section gives an overview of the core functionalties. The image below gives an overview class diagram with the GeniusWeb core modules and their functionalities. The figure also shows the 3 servers and the functions inside those. Click on the figure to zoom in. [[Image(source:design/classdiagram.svg, 80%)]] All classes are documented in detail in the javadoc with the code. Also the serialization is annotated with the classes. Here we just give a brief overview. Later in this document we explain the serialization of a domain and profile. Also check the examples to see how this works and what typical values look like. == issuevalue This module contains the basic objects that make up a bid: issues, values, domains, and bids. === Issue The issues are just String objects. Issue names must be unique === Value There are two type of values: discrete values and number values. Discrete values are strings eg "low", "yes" or "1". Number values are numbers, eg -1 or 12.423. In GeniusWeb, all number values are processed as BigDecimal to avoid rounding errors. === ValueSet A ValueSet indicates the possible values for an issue. There are 2 types of valuesets: numeric and discrete. DiscreteValueSet contains the list of DiscreteValue which is a basically a possible (string) values for the issue. NumberValueSet contains a Range which is a set of numbers defined by the minimum, maximum and stepsize (upwards from the minimum). * a DiscreteValueSet contains a set of possible discrete values. It looks like "values", a column and then a list of discrete values (all strings) Form example, {{{ "values":["yes","no"] }}}. * a number valueset contains a range of numbers and looks like {{{ {"range":[12.2,12.6,0.3]} }}} so "range:" followed by a list of 3 values. The first value is the minimum value in the range, the second the maximum value in the range, and the third the step value. This example range thus contains 12.2 and 12.5 (the next one would be 12.8 but that is already outside the range). Numbers are without quotes. === Domain A domain is a description of the allowed issues and values for each issue. To avoid confusion in a negotiation, it's important that all participants are dealing with the same domain. A Domain contains a map, with each key the issue (string) and the value a ValueSet. When describing the domain, the set of allowed values has to be given. A domain looks like this {{{ {"name":"jobs", "issuesValues":{ "lease car":{"values":["yes","no"]}, "permanent contract":{"values":["yes","no"]}, "career development opportunities":{"values":["low","medium","high"]}, "fte":{"values":["0.6","0.8","1.0"]}, "salary":{"values":["2000","2500","3000","3500","4000"]}, "work from home":{"values":["0","1","2"]} } } }}} * The name is just a string. Our reference implementation of the profiles server additionally requires that the name must match the filename and directory name when placed on the profiles server. So your directory must be domainsrepo/jobs and the filename must be jobs.json. * The issueValues contains a dictionary with issues. Each issue is indicated by a name (a string), followed by a column (:) and then a ValueSet as discussed above. === Bid Finally, a Bid is a Map where the key is the issue and the value either a DiscreteValue or NumberValue that is valid for that issue. DiscreteValues are serialized to json inside double quotes, NumberValues are just the numbers (no double quotes). Here is an example of a bid in JSON format. issue 2 and issue 3 are number values, issue 1 is a discrete issue. All values in quotes are discrete values, while values without quotes are number values. {{{ {"issuevalues":{ "issue3":9012345678901234567.89, "issue2":1, "issue1":"b" }} }}} Your party can create any bid that it likes but the protocol will check if your bid fits in the current negotiation and may kick you out of the negotiation if you don't behave properly. If you write a java-based party, all these objects may already have been converted to Java objects. Please check [source:issuevalue/src/main/java/geniusweb/issuevalue] for conversion details. There is not yet a domain editor available so you have to create domains by manually editing a file with JSON code. == profile A profile is a function that can tell if a bid is preferred over another bid. In the profile module a number of different types of profiles have been defined: * [source:profile/src/main/java/geniusweb/profile/FullOrdering.java FullOrdering]: this provides a function {{{isPreferredOrEqual()}}} that can tell if a bid is preferred over another * [source:profile/src/main/java/geniusweb/profile/PartialOrdering.java PartialOrdering]: as {{{FullyOrderedSpace}}}, but may not know the answer for part of the bids * [source:profile/src/main/java/geniusweb/profile/utilityspace/UtilitySpace.java UtilitySpace]: as {{{FullOrdering}}}, but additionally this provides a function {{{getUtility(bid)}}} that maps the bid into a {{{BigDecimal}}} in [0,1]. The higher the value, the more preferred is that bid. The [source:profile/src/main/java/geniusweb/profile/utilityspace/LinearAdditiveUtilitySpace.java Linear Additive Space] is the most commonly used UtilitySpace. If the accuracy of {{{BigDecimal}}} is not needed, you can just call {{{BigDecimal#doubleValue()}}} to get a standard double. === Linear Additive The most common is the LinearAdditiveUtilitySpace. It contains * the domain description containing the possible issueValues. * the name of the space * the issueUtilities containing a set of ValueSetUtilities. * The issueWeightsL a dictionary assigning a number to each of the issues. These numbers must add up to 1 and determine how much each ValueSetUtility adds to the total utility. * a reservationBid that contains the bid that is the least preferred. Only bids equal or better than this bid should be accepted. To give a full profile example: {{{ {"LinearAdditiveUtilitySpace": {"name":"testprofile", "domain":{"name":"test", "issuesValues":{ "issue2":{"values":["issue2value1","issue2value2"]}, "issue1":{"values": ["issue1value1","issue1value2"]}}}, "issueUtilities":{ "issue2":{"numberutils":{"lowValue":12,"lowUtility":0.3,"highValue":18,"highUtility":0.6}}, "issue1":{"discreteutils":{"valueUtilities":{"issue1value1":0.2,"issue1value2":0.3}}}}, "issueWeights":{"issue2":0.4,"issue1":0.6}, "reservationBid":{"issuevalues":{"issue2":"issue2value1","issue1":"issue1value2"}} }} }}} A party receives a reference to a profile but has to fetch the actual profile. You can defer this task to the ProfileConnectionFactory and in that case you receive a ready-to-use parsed java object. Most likely you get a Linear Additive Utilityspace, please check [source:profile/src/main/java/geniusweb/profile/utilityspace] for conversion details. This is an "LinearAdditiveYtilitySpace", other utilityspaces would have different contents. It contains a number of components: * issueUtilities. Here, each issue value from the domain gets a utility assigned. The two types of issue values each having their matching utility function: ||= Issue Value class =||= Utility class =||= json serialization example =|| {{{#!td DiscreteValue }}} {{{#!td DiscreteValueSetUtilities }}} {{{#!td "discreteutils": { "valueUtilities": { "2000": 0, "2500": 0.25, "3000": 0.3, "3500": 0.75, "4000": 1.0 } } }}} |---------------------------- {{{#!td NumberValue }}} {{{#!td NumberValueSetUtilities }}} {{{#!td numberutils": { "lowValue":12, "lowUtility":0.3, "highValue":18, "highUtility":0.6 } }}} |---------------------------- * issueWeights: this is a map, for each issue there is a value in [0,1] and the sum of the weights in this map must be exactly 1. * domain: identical to above. We replaced some issueValues with "..." in this example . * name: a simple string with the name of the profile. The name must match the filename of the profile. == Party A party is a program that receives inform messages from the protocol, and can send the actions it wants to take regarding the negotiation back to the protocol. These actions are sent over a websocket that is created by the partiesserver that created the party, which in turn creates this according to a http GET request that comes from a running protocol. The party can also pro-actively search for information and take other actions as it likes, as long as it adheres to the requirements of the protocol (discussed later in this document). The party module contains the inform objects. There are several, and although they have quite specific meanings, their fine details can be tweaked by the protocol. ||inform object||meaning|| ||Settings||usually sent as first inform to a party, indicating start of the session and providing the party name, protocol, profile, deadline etc|| ||YourTurn||Indicating that the party receiving this inform now has the turn. || ||ActionDone||Informing that some party did an action|| ||Finished||Indicating that a session has been finished|| Please check the [source:party/src/main/java/geniusweb/party/inform source code of Inform objects] for all the details especially on the json serialization. The party module also contains basic interfaces for implementing your negotiation party. These interfaces are only relevant to create a party compatible with our reference implementation of the [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPartiesServer partiesserver]. The main class is Party, which defines the basic functionality required for a negotiation party on our server. The heart of the party is that your Party implements Connectable. Connectable contains 2 main functions: connect and disconnect. When connect is called, your party receives a Connection over which it receives Inform objects and can send Action objects. What exactly is sent and received is determined by the protocol (see the protocol section below). For example if the SAOP protocol is used, your party will receive a YourTurn message, after which it decides on its action and sends it into the Connection. See also the example party discussed below. == Timeline The timeline module describes the deadline and a progress objects. The deadline indicates how much time a negotiation session can take and is used to specify the settings for a session or tournament. Two examples: {{{ {"deadlinerounds":{"rounds":100,"durationms":999}} }}} and {{{ {"deadlinetime":{"durationms":2000}} }}} * durationms indicates the maximum run time in milliseconds for the party. The clock starts ticking at the moment the party receives its SessionSettings object. * rounds indicates the maximum number of rounds in the session. The session will end after this number of rounds, deal or no deal. The progress indicates where currently running session is towards the deadline. Progress is contained in the settings object. Round based progress must be updated by the party after each round. == References Parties, domains, profiles and protocols are stored and used on remote machines. We use IRI's (internationalized resource identifier, which looks similar to the well known URLs you use in your web browser) to refer to them. These IRI's are packed inside objects like the PartyRef, ProtocolRef, ProfileRef, DomainRef so that it is clear what type of object the IRI is referring to. For example, when your party is initialized, it usually receives a Settings object that contains a ProfileRef. The intention is that the party fetches the actual profile from the web, using the IRI in the ProtocolRef. See the example party below for an example. There are a number of schemes used for references: ||scheme||used with||example||comments|| ||http:||party||http://localhost:8080/partiesserver/run/randompyparty-1.0.0|| || ||ws:||profile||ws://localhost:8080/profilesserver/websocket/get/jobs/jobs1.json|| || ||file:||profile||file:src/test/settings.json ||gives file relative to local current working dir|| ||classpath:||party||classpath:geniusweb.exampleparties.randomparty.RandomParty||must be in classpath|| == BidSpace The bidspace module contains functionality to support building a negotiation party. We currently have * OpponentModel: this is a category of classes that can estimate the opponent's profile from the bids that he places. * Pareto: this is a category of classes that can compute the pareto frontier from a set of profiles. Pareto optimality is an important mechanism to place optimal bids. * AllBidsList: this can be used to craete a list containing all possible bids in a domain. This list is created in a lazy way, and competely avoids storing the whole list in memory (which might not even fit) == Protocol The protocol module contains the functionality to define and execute a negotiation protocol. There are session protocols and tournament protocols. The basic classes defining a protocol are: * The [source:/protocol/src/main/java/geniusweb/protocol/NegoSettings.java NegoSettings]: these define the settings for the protocol, such as the deadline, the participants and the profile * The [source:/protocol/src/main/java/geniusweb/protocol/NegoState.java NegoState]: this contains the current state of execution of the protocol. To give some example states: "waiting for bid from party 2", "ended with agreement", "ended because party 1 broke the protocol". A state also has various {{{with()}}} functions defining the new state from an old state and a party doing an action. * The [source:/protocol/src/main/java/geniusweb/protocol/NegoProtocol.java NegoProtocol]: this defines what happens when a negotiation starts, when a participant enters halfway the session, what are the allowed actions, what happens if a participant breaks the protocol, etc. = Writing a party in Java Example parties can be found [source:/exampleparties here]. You can easily clone a party with SVN using {{{svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWeb/exampleparties/randomparty/}}} (this clones randomparty, use a different name to fetch another example). A party is compiled with maven. After compilation ({{{mvn package}}}) you get a {{{target/yourparty-X.Y.Z-jar-with-dependencies.jar}}} that can be copied into the parties server for deployment. The basic structure of an party looks like this {{{ public class RandomParty extends DefaultParty { @Override public void notifyChange(Inform info) { if (info instanceof Settings) { Settings settings = (Settings) info; this.profileint = ProfileConnectionFactory .create(settings.getProfile().getURI(), getReporter()); this.me = settings.getID(); this.progress = settings.getProgress(); } else if (info instanceof ActionDone) { Action otheract = ((ActionDone) info).getAction(); if (otheract instanceof Offer) { lastReceivedBid = ((Offer) otheract).getBid(); } } else if (info instanceof YourTurn) { myTurn(); if (progress instanceof ProgressRounds) { progress = ((ProgressRounds) progress).advance(); } } else if (info instanceof Finished) { getReporter().log(Level.INFO, "Final ourcome:" + info); } } private void myTurn() { Action action; if (isGood(lastReceivedBid)) { action = new Accept(me, lastReceivedBid); } else { // for demo. Obviously full bids have higher util in general AllPartialBidsList bidspace = new AllPartialBidsList( profileint.getProfile().getDomain()); Bid bid = null; for (int attempt = 0; attempt < 20 && !isGood(bid); attempt++) { long i = random.nextInt(bidspace.size().intValue()); bid = bidspace.get(BigInteger.valueOf(i)); } action = new Offer(me, bid); } getConnection().send(action); } private boolean isGood(Bid bid) { return bid != null && ((LinearAdditiveUtilitySpace) profileint.getProfile()) .getUtility(bid).doubleValue() > 0.6; } @Override public Capabilities getCapabilities() { return new Capabilities(new HashSet<>( Arrays.asList(new ProtocolRef(new URI("SAOP"))))); } @Override public String getDescription() { return "places random bids until it can accept an offer with utility >0.6"; } } }}} == Preparing the jar file In order to put your party on the [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPartiesServer partiesserver] for running, you need a jar file of your party. Normally the party includes all dependencies. The jar files are loaded with an isolated jar class loader that should avoid collisions with possibly identically named but possibly different packages in other jar files. Party jar files must have a Main-Class set in the MANIFEST.MF file. This main-class must implement Party and have a no-arg constructor. The example randomparty does this from the maven build script. We recommend to do initialization of the party only in the init() and not in the constructor or static code. This because instances of your class can be made both for extracting general info as getDescription(), or to really run your class. = Writing a party in Python We provide a python-to-java adapter so that you can easily write your party in python instead of Java. You can check the full working example code [source:/exampleparties/randompartypy/src/main/resources/RandomParty.py here]. A python-based party looks like this: {{{ class RandomParty (DefaultParty): def notifyChange(self, info): if isinstance(info, Settings) : self.profile = ProfileConnectionFactory.create(info.getProfile().getURI(), self.getReporter()); self.me = info.getID() self.progress = info.getProgress() elif isinstance(info , ActionDone): self.lastActor = info.getAction().getActor() otheract = info.getAction() if isinstance(otheract, Offer): self.lastReceivedBid = otheract.getBid() elif isinstance(info , YourTurn): self._myTurn() if isinstance(self.progress, ProgressRounds) : self.progress = self.progress.advance(); def getCapabilities(self): # -> Capabilities return Capabilities(HashSet([ ProtocolRef(URI("SAOP"))])) def getDescription(self): return "places random bids until it can accept an offer with utility >0.6. Python version" def terminate(self): self.profile.disconnect() def _myTurn(self): if self.lastReceivedBid != None and self.profile.getProfile().getUtility(self.lastReceivedBid).doubleValue() > 0.6: action = Accept(self.me, self.lastReceivedBid) else: bidspace = AllPartialBidsList(self.profile.getProfile().getDomain()) bid = None for attempt in range(20): i = self.random.nextInt(bidspace.size()) # warning: jython implicitly converts BigInteger to long. bid = bidspace.get(BigInteger.valueOf(i)) if self._isGood(bid): break action = Offer(self.me, bid); self.getConnection().send(action) def _isGood(self, bid): return bid != None and self.profile.getProfile().getUtility(bid).doubleValue() > 0.6; }}} You need to wrap your python party into a jar wrapper to get it accepted by the parties server. To do this, check the javadoc with the [source:/pythonadapter/src/main/java/geniusweb/pythonadapter/PythonPartyAdapter.java PythonPartyAdapter]. = Writing a party in other languages If you want to use another language than java or python2 to write your parties, you have a number of options * Make your own adapter that runs your language from Java. Check [source:pythonadapter/src/main/java/geniusweb/pythonadapter/PythonPartyAdapter.java our pythonadapter] for an example how this can be done. * Write your own partiesserver that correctly implements the [https://tracinsy.ewi.tudelft.nl/pubtrac/GeniusWebPartiesServer partiesserver interface]. This boils down to creating a webserver that correctly can handle calls to a number of prescribed URLs and websockets. = Stand-alone Running = For stand-alone running you need to have the following available in your project space (this includes the maven dependencies you have set for your project) * The parties you want to run (in compiled form) must be in your classpath * The profiles you want to provide to the parties (alternatively you can refer to a profile on a running profile server) * A settings.json file containing the [source:/protocol/src/main/java/geniusweb/protocol/session SessionSettings] eg SAOP Settings. [source:/simplerunner/src/test/resources/settings.json view example file]. * a simple stand-alone runner, eg download from [http://artifactory.ewi.tudelft.nl/artifactory/libs-release/geniusweb/simplerunner/ geniusweb artifactory] select latest version simplerunner--jar-with-dependencies.jar. A complete example is available of [source:/simplerunner simplerunner here] == Running == Make sure your parties are in the java classpath. A stand-alone runner can now be started as follows (using the example, set your working directory to the root of the simplerunner project ) {{{ java -jar simplerunner...with-dependencies.jar src/test/resources/settings.json }}} When the protocol is completed, the runner prints out the final state which usually contains all actions that were done as well. = Debugging = Debugging can be done in several ways * From Eclipse EE: * Place your party source code in the Eclipse workspace * Place a breakpoint in your party's code * Run the partiesserver directly from Eclipse EE , in Debug mode. * Run a session or tournament in which you want to debug your party. * Eclipse will halt server execution and switch to debugging when it hits your breakpoint * Be aware of the automatic time-outs that will still be enforced while you are debugging. This includes tomcat session time-outs. * Using a stand-alone runner (normal Eclipse for Java developers, no EE needed) * Have your party's source code in the Eclipse workspace * Clone the simplerunner into your workspace {{{svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWeb/simplerunner}}} * Edit the src/test/resource/settings.json file in the simplerunner to match your party * Place a breakpoint in your party where you want to debug. * Create a debug configuration for NegoRunner with some customized settings: * Add your party to the user entries in the classpath. * Set {{{src/test/resources/settings.json}}} as program argument * Run the debug configuration * Eclipse will halt server execution and switch to debugging when it hits your breakpoint * time-outs may still be enforced while you are debugging but you avoid the Tomcat session time-out. * With stand-alone runner, your parties are run together in a single classloader. This is different from running in the partiesserver. = GeniusWeb sources = == downloading source code You can browse the GeniusWeb core sources directly using the browse button at the right top of this page. You can download the source code of this component using {{{svn co https://tracinsy.ewi.tudelft.nl/pub/svn/GeniusWeb/}}} Normal developers that write new parties do not need to install the GeniusWeb source code. This is only needed if you want to debug/trace into the GeniusWeb code for instance for debugging or understanding the inner workings of geniusWeb. == Import all sources in Eclipse Install Subclipse using "help/Eclipse MarketPlace" and search for subclipse. Disable the JavaHL native DLLs and install. NOTE: due to a bug in Eclipse Photon the marketplace may not work. We suggest to upgrade... You may get some errors on JavaHL library. To get rid of those, go to preferences/Team/SVN/ * disable General SVN settings / JavaHL * SVN interface/Client: select SVNKit instead of JavaHL. Right click in Package Explorer in Eclipse, select "Import/SVN/Checkout Projects from SVN". Select the root of the project, finish (selecting sub-projects will result in a stupid loop in the checkout procedure in Eclipse and won't lead anywhere...) Richt click on the checked-out project that you want eclipse to recognise as Maven project (the project are maven but Eclipse does not recognise this after check-out). Select import/maven/existing maven projects. Finish. Note. Eclipse does not automatically recognise this as a maven project because Eclipse supports this only with GIT repositories while we use SVN.