Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning

Michael Schlechtinger; Damaris Kosack; Heiko Paulheim; Thomas Fetzer

doi:10.1007/978-3-030-85739-4_21

Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning

Michael Schlechtinger^*, Damaris Kosack, Heiko Paulheim, Thomas Fetzer

^*Corresponding author for this work

University of Mannheim

Research output: Contribution to Book/Report types › Conference contribution › peer-review

Abstract (may include machine translation)

Pricing decisions are increasingly made by AI. Thanks to their ability to train with live market data while making decisions on the fly, deep reinforcement learning algorithms are especially effective in taking such pricing decisions. In e-commerce scenarios, multiple reinforcement learning agents can set prices based on their competitor’s prices. Therefore, research states that agents might end up in a state of collusion in the long run. To further analyze this issue, we build a scenario that is based on a modified version of a prisoner’s dilemma where three agents play the game of rock paper scissors. Our results indicate that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems that are able to recognize situations which might lead to a collusion between competitors. We furthermore provide evidence for a situation where agents are capable of performing a tacit cooperation strategy without being explicitly trained to do so.

Original language	English
Title of host publication	Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection - 19th International Conference, PAAMS 2021, Proceedings
Editors	Frank Dignum, Juan Manuel Corchado, Fernando De La Prieta
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	255-266
Number of pages	12
ISBN (Print)	9783030857387
DOIs	https://doi.org/10.1007/978-3-030-85739-4_21
State	Published - 2021
Externally published	Yes
Event	19th International Conference on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2021 - Salamanca, Spain Duration: 6 Oct 2021 → 8 Oct 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12946 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	19th International Conference on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2021
Country/Territory	Spain
City	Salamanca
Period	6/10/21 → 8/10/21

Keywords

Algorithmic collusion
Multi agent reinforcement learning
Pricing agents

Access to Document

10.1007/978-3-030-85739-4_21

Cite this

Schlechtinger, M., Kosack, D., Paulheim, H., & Fetzer, T. (2021). Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning. In F. Dignum, J. M. Corchado, & F. De La Prieta (Eds.), Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection - 19th International Conference, PAAMS 2021, Proceedings (pp. 255-266). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12946 LNAI). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-85739-4_21

Schlechtinger, Michael ; Kosack, Damaris ; Paulheim, Heiko et al. / Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning. Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection - 19th International Conference, PAAMS 2021, Proceedings. editor / Frank Dignum ; Juan Manuel Corchado ; Fernando De La Prieta. Springer Science and Business Media Deutschland GmbH, 2021. pp. 255-266 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{17c635532872455896e6eefbe550bd7e,

title = "Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning",

abstract = "Pricing decisions are increasingly made by AI. Thanks to their ability to train with live market data while making decisions on the fly, deep reinforcement learning algorithms are especially effective in taking such pricing decisions. In e-commerce scenarios, multiple reinforcement learning agents can set prices based on their competitor{\textquoteright}s prices. Therefore, research states that agents might end up in a state of collusion in the long run. To further analyze this issue, we build a scenario that is based on a modified version of a prisoner{\textquoteright}s dilemma where three agents play the game of rock paper scissors. Our results indicate that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems that are able to recognize situations which might lead to a collusion between competitors. We furthermore provide evidence for a situation where agents are capable of performing a tacit cooperation strategy without being explicitly trained to do so.",

keywords = "Algorithmic collusion, Multi agent reinforcement learning, Pricing agents",

author = "Michael Schlechtinger and Damaris Kosack and Heiko Paulheim and Thomas Fetzer",

note = "Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 19th International Conference on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2021 ; Conference date: 06-10-2021 Through 08-10-2021",

year = "2021",

doi = "10.1007/978-3-030-85739-4_21",

language = "English",

isbn = "9783030857387",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "255--266",

editor = "Frank Dignum and Corchado, {Juan Manuel} and {De La Prieta}, Fernando",

booktitle = "Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection - 19th International Conference, PAAMS 2021, Proceedings",

}

Schlechtinger, M, Kosack, D, Paulheim, H & Fetzer, T 2021, Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning. in F Dignum, JM Corchado & F De La Prieta (eds), Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection - 19th International Conference, PAAMS 2021, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12946 LNAI, Springer Science and Business Media Deutschland GmbH, pp. 255-266, 19th International Conference on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2021, Salamanca, Spain, 6/10/21. https://doi.org/10.1007/978-3-030-85739-4_21

Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning. / Schlechtinger, Michael; Kosack, Damaris; Paulheim, Heiko et al.
Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection - 19th International Conference, PAAMS 2021, Proceedings. ed. / Frank Dignum; Juan Manuel Corchado; Fernando De La Prieta. Springer Science and Business Media Deutschland GmbH, 2021. p. 255-266 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12946 LNAI).

Research output: Contribution to Book/Report types › Conference contribution › peer-review

TY - GEN

T1 - Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning

AU - Schlechtinger, Michael

AU - Kosack, Damaris

AU - Paulheim, Heiko

AU - Fetzer, Thomas

PY - 2021

Y1 - 2021

N2 - Pricing decisions are increasingly made by AI. Thanks to their ability to train with live market data while making decisions on the fly, deep reinforcement learning algorithms are especially effective in taking such pricing decisions. In e-commerce scenarios, multiple reinforcement learning agents can set prices based on their competitor’s prices. Therefore, research states that agents might end up in a state of collusion in the long run. To further analyze this issue, we build a scenario that is based on a modified version of a prisoner’s dilemma where three agents play the game of rock paper scissors. Our results indicate that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems that are able to recognize situations which might lead to a collusion between competitors. We furthermore provide evidence for a situation where agents are capable of performing a tacit cooperation strategy without being explicitly trained to do so.

AB - Pricing decisions are increasingly made by AI. Thanks to their ability to train with live market data while making decisions on the fly, deep reinforcement learning algorithms are especially effective in taking such pricing decisions. In e-commerce scenarios, multiple reinforcement learning agents can set prices based on their competitor’s prices. Therefore, research states that agents might end up in a state of collusion in the long run. To further analyze this issue, we build a scenario that is based on a modified version of a prisoner’s dilemma where three agents play the game of rock paper scissors. Our results indicate that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems that are able to recognize situations which might lead to a collusion between competitors. We furthermore provide evidence for a situation where agents are capable of performing a tacit cooperation strategy without being explicitly trained to do so.

KW - Algorithmic collusion

KW - Multi agent reinforcement learning

KW - Pricing agents

UR - http://www.scopus.com/inward/record.url?scp=85116314928&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-85739-4_21

DO - 10.1007/978-3-030-85739-4_21

M3 - Conference contribution

AN - SCOPUS:85116314928

SN - 9783030857387

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 255

EP - 266

BT - Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection - 19th International Conference, PAAMS 2021, Proceedings

A2 - Dignum, Frank

A2 - Corchado, Juan Manuel

A2 - De La Prieta, Fernando

PB - Springer Science and Business Media Deutschland GmbH

T2 - 19th International Conference on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2021

Y2 - 6 October 2021 through 8 October 2021

ER -

Schlechtinger M, Kosack D, Paulheim H, Fetzer T. Winning at Any Cost - Infringing the Cartel Prohibition with Reinforcement Learning. In Dignum F, Corchado JM, De La Prieta F, editors, Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection - 19th International Conference, PAAMS 2021, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 255-266. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-85739-4_21