Literature Survey: Opponent Models

This page provides an overview of the literature survey of Mark Hendrikx. The first Section discusses the researched papers, including a summary
of the relevant details. The second Section provides an overview of all meetings. Finally, the third Section outlines a global planning.
For an overview of the current status, see attached PDF

Structure

1. Introduction (also limits scope of nego models)
   
2. Related Work
   
3. Components of Opponent Models (basic explanation of KDE, Bayesian, reinforcement learning, ...)
   
4. Opponent models (make a taxonomy of this! Order is still wrong)
   

4.a. Learning Reservation Values (sep sections matching structure in 3!)
4.b. Learning Opponent Strategies
4.c. Learning Utility curves
4.d. Learning Value Ordering
4.e. Learning Issue Ordering
4.f. Learning Deadlines

5. Using Opponent Models
   
6. Evaluating Opponent Models
   

6.a. Qualitative Measures
6.b. Quantitative Measures

7. Future Work
   
8. Conclusion
   

Papers

Title	A Framework for Building Intelligent SLA Negotiation Strategies under Time Constraints
Author(s)	G.C. Silaghi, L.D. Şerban and C.M. Litan
Cited	-
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	A Framework for Multi-agent Electronic Marketplaces: Analysis and Classification of Existing Systems
Author(s)	K. Kurbel and I. Loutchko
Cited	25
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	A Machine Learning Approach to Automated Negotiation and Prospects for Electronic Commerce
Author(s)	J.R. Oliver
Cited	198
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	AgentFSEGA - Time Constrained Reasoning Model for Bilateral Multi-Issue Negotiation
Author(s)	L.D. Serban, G.C. Silaghi, and C.M. Litan
Cited	-
Subject(s)	Learning issue utility curves by Bayesian learning; Learning issue ordering
Summary	The opponent model of FSEGA tries to approximate the ordering of the issues and the utility function of each issue by using Bayesian learning. Each value in an issue is imagined as approximating one of the three basic functions (downhill, uphill, triangular). Using the Bayesian formula, each hypothesis for a value is updated. Finally the hypothesis are combined based on their likelihood to determine the final form of the utility function for each value in the issue; combining these results in the utility function for an issue. Finally, the bidding strategy uses isocurves and the opponent model to increase acceptance.
Relevance	8
Bibtex	X
Cites seen	Yes

Title	Agents that Acquire Negotiation Strategies Using a Game Theoretic Learning Theory
Author(s)	N. Eiji Nawa
Cited	2
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	An Adaptive Bilateral Negotiation Model for E-Commerce Settings
Author(s)	V. Narayanan and N.R. Jennings
Cited	26
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	An Adaptive Learning Method in Automated Negotiation Based on Artificial Neural Network
Author(s)	Z. Zeng, B. Meng, Y. Zeng
Cited	4
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	An Architecture for Negotiating Agents that Learn
Author(s)	H.H. Bui, S. Venkatesh, and D. Kieronska
Cited	2
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	An Automated Agent for Bilateral Negotiation with Bounded Rational Agents with Incomplete Information
Author(s)	R. Lin, S. Kraus, J. Wilkenfeld, J. Barry
Cited	23
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	An Evolutionairy Learning Approach for Adaptive Negotiation Agents
Author(s)	R.Y.K. Lau, M. Tang, O. Wong, S.W. Milliner
Cited	19
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Analysis of Negotiation Dynamics
Author(s)	K. Hindriks, C.M. Jonker, D. Tykhonov
Cited	5
Subject(s)	Strategy evaluation
Summary	The process of the negotiation determines the outcome. This work presents an outline of a formal toolbox to analyze the dynamics of negotiation based on an analysis of move types. Boulware is a hard bargaining tactic, whereas conceder is soft. Besides evaluating the outcome one should also analyze the dance. This can be done by classifying the moves (nice, selfish, etc). A trace is a list of bids. The percentage of a type of move can be calculated. The sensitivy for opponent moves is based on this measure.
Relevance	7, interesting technique for evaluating strategies
Bibtex	​Link
Cites seen	Yes

Title	Anticipating Agent's Negotiation Strategies in an E-marketplace Using Belief Models
Author(s)	F. Teuteberg, K. Kurbel
Cited	11
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Automated Multi-Attribute Negotiation with Efficient Use of Incomplete Preference Information
Author(s)	C. Jonker and V. Robu
Cited	44
Subject(s)	Mechanism for taking learning and initial information into account in a standard bilateral negotiation model
Summary	The classic technique for negotiation with undisclosed preferences is to use a mediator, however can we be sure that he is impartial? The negotiation strategy discussed is for billeteral multi-issue negotiation. A decreasing util curve is considered. A bid is calculated to fit the current util. Each issue has a seperate parameter such that more or less concession can be made on certain issues. General tolerance determines the general speed of concession. For each issue for the opponent bid and new calculated bid it is considered how much concession is made towards the opponent bid based on the configuration tolerance for each issue. This full formula depends on the weights of the opponent, which have to be estimated. The weights for each attribute can be estimated by comparing the distance between attributes for an issue in sequential bids and using this distance to mark the importance of an attribute. This last step is domain dependent. Concluding, the technique works, but requires tuning for the domain and assumes that the other agent plays a more or less similiar concession based technique.
Relevance	4, domain dependent opponent modelling approach for learning ordering of attributes
Bibtex	​Link
Cites seen	Yes

Title	Bayesian Learning in Bilateral Multi-issue Negotiation and its Application in MAS-based Electronic Commerce
Author(s)	J. Li, Y. Cao
Cited	6
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Bayesian Learning in Negotiation
Author(s)	D. Zeng, K. Sycara
Cited	355
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Benefits of Learning in Negotiation
Author(s)	D. Zeng, K. Sycara
Cited	116
Subject(s)	Benefits of learning, Bayesian learning, reservation values
Summary	Growing interest in e-commerce motivates research in automated negotiation. Building intelligent negotiation agents is still emerging. In contrast to most negotiation models, sequential decision model allows for learning. Learning can help understand human behaviour, but can also result in better results for the learning party. Bayesian learning of reservation values can be used to determine the zone of agreement for an issue based on the domain knowledge and bidding interactions. Concluding for one-issue, learning positively influences bargaining quality, number of exchanged proposals, and leads to a better compromise if both learn. Learning works always works better in the proposed case.
Relevance	8. Strong example of Bayesian learning
Bibtex	​Link
Cites seen	Yes

Title	Bilateral Negotiation with Incomplete and Uncertain Information: A Decision-Theoretic Approach Using a Model of the Opponent
Author(s)	C. Mudgal, J. Vassileva
Cited	42
Subject(s)
Summary
Relevance
Bibtex
Cites seen	Yes

Title	Building Automated Negotiation Strategies Enhanced by MLP and GR Neural Networks for Opponent Agent Behaviour Prognosis
Author(s)	I. Roussaki, I. Papaioannou, and M. Anagostou
Cited	3
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Comparing Equilibria for Game-Theoretic and Evolutionary Bargaining Models
Author(s)	S. Fatima, M. Wooldridge, N.R. Jennings
Cited	21
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Compromising Strategy based on Estimated Maximum Utility for Automated Negotiating Agents
Author(s)	S. Kawaguchi, K. Fujita, T. Ito
Cited	-
Subject(s)
Summary
Relevance
Bibtex	X
Cites seen	Yes

Title	Determining Succesful Negotiation Strategies: An Evolutionary Approach
Author(s)	N. Matos, C. Sierra, N.R. Jennings
Cited	149
Subject(s)	Analysing strengths and weakness of tactics
Summary	This work uses an evolutionary approach to find on how agents using particular negotiation strategies fare against each other in a negotiation. A standard bileteral multi-lateral negotiation model is used. The issues are continious between a given range. Three types of techniques are considered: time-dependent, resource- dependent, and behaviour dependent. The results give an nice overview on which tactic is effective against which opponent. This is very interesting if the opponent is somehow able to determine the type of agent.
Relevance	8, motivation for learning of strategies
Bibtex	​Link
Cites seen	Yes
Processed	Yes

Title	Facing the Challenge of Human-Agent Negotiations via Effective General Opponent Modeling
Author(s)	Y. Oshrat, R. Lin, S. Kraus
Cited	19
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Genetic Algorithms for Automated Negotiations: A FSM-Based Application Approach
Author(s)	M.T. Tu, E. Wolff, W. Lamersdorf
Cited	37
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	IAMhaggler: A Negotiation Agent for Complex Environments
Author(s)	C.R. Williams, V. Robu, E.H. Gerding, and N.R. Jennings
Cited	-
Subject(s)	ANAC, Pareto search, Bayes' rule
Summary	IAMhaggler first determines the discount factor of the opponent by using non-linear regression. Next, the found curve is discounted to find the opponent bid curve. Next, the maximum is found on the opponent curve, and an appropriate curve is plotted for the own utility curve. For domains without unordered issues Pareto-search is used to determine all possible bids matching an utility. Next, it is determined which bid is the closest to the best received opponent bid by using the euclidean distance. For domains with unordered issues, each unorderded value is varied, after which the possible bids are determined which satisfy the utility. Finally, using Bayes' rule for opponent modelling, the best possible bid for the opponent is chosen.
Relevance	8, beautifull strategy
Bibtex	​Link
Cites seen	Yes

Title	Inferring implicit preferences from negotiation actions
Author(s)	A. Restificar and P. Haddawy
Cited	10
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Integration of Learning, Situational Power and Goal Constraints Into Time-Dependent Electronic Negotiation Agents
Author(s)	W.W.H. Mok
Cited	-
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Learning Algorithms for Single-instance Electronic Negotiations using the Time-dependent Behavioral Tactic
Author(s)	W.W.H Mok and R.P. Sundarraj
Cited	17
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Learning an Agent's Utility Function by Observing Behavior
Author(s)	U. Chajewska, D. Koller, D. Ormoneit
Cited	54
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Learning an Opponent's Preferences to Make Effective Multi-Issue Negotiation Trade-Offs
Author(s)	R.M. Coehoorn, N.R. Jennings
Cited	78
Subject(s)	KDE Learning, Negotiation model, Concession based strategy
Summary	Effective and efficient multi-issue negotiation requires an agent to have some indication of it's opponent's preferences over the issues in the domain. Kernel Density Estimation (KDE) is used to estimate the weight attached to different issues by different agents. It is assumed that if the value of an issue increases, that this is positive for one agent, and negative for the other. No assumptions about relation between time, negotiation history and issue-weight are required, in contrast to Bayesian learning. The difference between concessive (counter)offers is used to estimate the weights of the issues (assumption: stronger concessions are made later on in the negotiation). Faratin's hill climbing algorithm augmented with KDE is used to propose the next bid. KDE proved succesful on the used negotiation model. Future works entails testing the approach against different opponent strategies and extending the approach to other negotiation models (see assumption in summary).
Relevance	9. KDE learning described in detail. Strong related work section
Bibtex	​Link
Cites seen	Yes

Title	Learning Opponents' Preferences in Multi-Object Automated Negotiation
Author(s)	S. Buffett and B. Spencer
Cited	18
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Learning other Agents' Preferences in Multiagent Negotiation using the Bayesian Classifier.
Author(s)	H.H. Bui, D. Kieronska, S. Venkatesh
Cited	29
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Learning to Select Negotiation Strategies in Multi-Agent Meeting Scheduling
Author(s)	E. Crawford and M. Veleso
Cited	21
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Modelling Agents Behaviour in Automated Negotiation
Author(s)	C. Hou
Cited	10
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Modeling Opponent Decision in Repeated One-shot Negotiations
Author(s)	S.Saha, A. Biswas, S. Sen
Cited	26
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Negotiating agents that learn about others' preferences
Author(s)	H.H. Bui, D. Kieronska and S. Venkatesh
Cited	5
Subject(s)	Logic-like representation negotiation model, bin-based opponent model, one-issue continious
Summary	The used method assumes that domain knowledge is available to partition the search space. Each turns, the agents communicate the space where an agreement is possible. Each turn there is a negotiation between all agents to find a common space, which means that the agent recommunicate a refined space of agreement until an agreement is reached. The proces continues until a common decision is found (a decision is an element in the space of agreement). A learning algorithm can be used as follows: first the full domain space is split into zones, which are allocated a uniform chance. This chance is updated for each region for each agent based on the received space of agreement. When agents do not agree about the space, then a the space is chosen which has the maximum support based on the chances of each space for each agent. This leads to a higher chance of agreement.
Relevance	3, domain knowledge required and only considers one issue
Bibtex	​Link
Cites seen	Yes

Title	Negotiation Decision Functions for Autonomous Agent
Author(s)	P. Faratin, C. Sierra, N.R. Jennings
Cited	718
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Negotiation Dynamics: Analysis, Concession Tactics, and Outcomes
Author(s)	K. Hindriks, C.M. Jonker, D. Tykhonov
Cited	7
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	On-Line Incremental Learning in Bilateral Multi-Issue Negotiation
Author(s)	V. Soo, C. Hung
Cited	18
Subject(s)	Online incremental learning using neural networks
Summary	This paper discusses using neural networks for learning the opponent model, however it is not described how, and the results are not promosing. By limiting the amount exchanges, opponent models become more important, and lead to beter outcomes.
Relevance	2, since the paper is not specific enough
Bibtex	​Link
Cites seen	Yes

Title	On Learning Negotiation Strategies by Artificial Adaptive Agents in Environments of Incomplete Information
Author(s)	J.R. Oliver
Cited	6
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Opponent Model Estimation in Bilateral Multi-issue Negotiation
Author(s)	N. van Galen Last
Cited	-
Subject(s)
Summary
Relevance
Bibtex	X
Cites seen	Yes

Title	Opponent Modelling in Automated Multi-Issue Negotiation Using Bayesian Learning
Author(s)	K. Hindriks, D. Tykhonov
Cited	33
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Optimal negotiation strategies for agents with incomplete information
Author(s)	S.S. Fatima, M. Wooldridge and N.R. Jennings
Cited	88
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Predicting Agents Tactics in Automated Negotiation
Author(s)	C. Hou
Cited	12
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Predicting partner's behaviour in agent negotiation
Author(s)	J. Brzostowski, R. Kowalczyk
Cited	16
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	The Benefits of Opponent Models in Negotiation
Author(s)	K. Hindriks, C.M. Jonker, D. Tykhonov
Cited	-
Subject(s)	Nice Mirroring Strategy using Bayesian Learning
Summary	Opponent models can aid in preventing exploitation, by determining the type of move of the opponent (selfish, unfortunate, concession, nice, fortunate, silent), and by taking the opponents preferences into account to increase the chance of acceptation. The mirror strategy mirrors the behaviours of the opponent, based on a classification of the opponent move. Nice MS does the same, but adds a nice move, which is a move which only increases the opponents utility without decreasing ours. Overall the strategy is shown to be effective by comparing the result of first testing the strategy against a random agent, and then the other agents. Also, the distance to a Kalai-Smorodinsky solution and the distance to the Nash Point is used as a metric. For future work the exploitability of MS should be researched.
Relevance	8, interesting application of opponent modelling
Bibtex	​Link
Cites seen	Yes
Processed	Yes

Title	The First Automated Negotiating Agents Competition (ANAC 2010)
Author(s)	T. Baarslag, K. Hindriks, C. Jonker, S. Kraus, R. Lin
Cited	-
Subject(s)	ANAC, overview multiple agents, opponent models, acceptance conditions
Summary	The ANAC competition models bilateral multi-issue closed negotiations and provides a benchmark for negotiation agents. Opponent models can also be used to identify the type of strategy of the opponent. Interesting agents for further analysis are: IAM(crazy)Haggler, FSEGA (profile learning), and Agent Smith. Issues can be predicatable, which means that they have a logical order, or unpredicatable, such as colors. This paper also includes acceptance conditions.
Relevance	5, too global, however interesting citations
Bibtex	​Link
Cites seen	Yes

Title	Towards a Quality Assessment Method for Learning Preference Profiles in Negotiation
Author(s)	K.V. Hindriks and D. Tykhonov
Cited	6
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Using Similarity Criteria to Make Issue Trade-offs in Automated Negotiations
Author(s)	P. Faratin, C. Sierra, N.R. Jennings
Cited	367
Subject(s)
Summary
Relevance
Bibtex	​Link
Cites seen	Yes

Title	Yushu: a Heuristic-Based Agent for Automated Negotiating Competition
Author(s)	B. An and V. Lesser
Cited	-
Subject(s)	ANAC agent, Complexity learning
Summary	One of the interesting strategies of Yushu is that it tries to measure the competitiveness, which influences it's bidding strategy. Details can not be found in their paper, however this is not relevant for this survey. They also measures time by averaging over all bids. This is used to determine when to accept in panic.
Relevance	4, only learning the amount of rounds is interesting, but obvious
Bibtex	X
Cites seen	Yes

Meetings

Onaangekondigde Meeting (20/04/11)

In verband met de conferentie in Taiwan is in overleg met Tim besproken dat we waarschijnlijk meer tijd nodig gaan hebben voor ons paper.
Dit houdt in dat de deadline is verschoven naar begin september. Als persoonlijk doel heb ik gesteld om eind juni een kladversie te hebben.

Planning

26/04/11	Meeste interessante papers gevonden
28/04/11 - 09/05-11	Conferentie bijwonen en op laag tempo doorwerken
23/05/11	Alle papers samengevat
26/05/11	Opzet taxonomie
29/06/11	Kladversie paper af