جستجو شبکه علمی ایران

جستجو

Computational Diplomacy in the Era of Large Language Models: A Research Protocol and Evaluation Framework for Assessing Capability, Reliability, Bias, and Human Oversight in Negotiation and Mediation Support

Niyaz Bigdeli¹

Abstract :
Large language models (LLMs) are increasingly capable of summarizing complex documents, extracting positions and interests, generating settlement options, reframing sensitive messages, and participating in multi-turn negotiation. However, evidence from bargaining games, mediation experiments, multilingual evaluations, and AI-facilitated deliberation does not establish that LLMs are sufficiently reliable, neutral, or governable for consequential international diplomacy. This article presents a preregistration-ready, multiphase mixed-method protocol for evaluating LLMs as decision-support tools—not autonomous diplomats—in international negotiation and mediation. The protocol is situated within computational diplomacy and integrates negotiation science, epistemic reliability, counterfactual bias auditing, and responsible-AI governance. The core design uses 24 public-document-grounded or carefully controlled synthetic scenarios spanning peace/security, trade/economic disputes, and multilateral governance. A frozen panel of five models is evaluated across English and Persian, three symmetric roles, grounded and ungrounded conditions, and four independent repetitions, yielding 5,760 planned model outputs before optional sensitivity extensions. Outcomes cover issue and interest extraction, BATNA/ZOPA detection, option quality, mediation quality, factuality and faithfulness, cross-run stability, uncertainty handling, geopolitical and cultural bias, hawkishness, and traceability. At least 25% of outputs are selected for stratified blind expert coding, complemented by 12–20 semi-structured expert interviews where feasible. Confirmatory analysis relies on multilevel models, preregistered interaction terms, multiple-comparison control, effect sizes, robustness checks, and inter-rater reliability; qualitative findings are integrated through joint displays. The protocol is designed to produce an LLM-CD Support Assessment profile and a task–risk–control matrix that distinguishes low-risk assistance, use requiring strict human verification, and tasks unsuitable for operational reliance. The framework shifts the research question from whether AI “can negotiate” to when. For these tasks, under what evidentiary conditions, and with what human controls, LLM support may be defensible in diplomacy.

Keywords : computational diplomacy large language models negotiation support mediation ai governance geopolitical bias

زبان مقاله : انگلیسی
محور مقاله : علوم انسانی
محل انتشار : پنجمین کنفرانس بین المللی پژوهش در حسابداری، مدیریت، اقتصاد و علوم انسانی
آدرس وب سایت کنفرانس : meaconf.com
کد IOI مقاله : ACEE-ACZZEZ
لینک مستقیم مقاله : https://isnac.ir/ACEE-ACZZEZ
نمایش چکیده فارسی : https://isnac.ir/SA/ACEE-ACZZEZ

مشاهده چکیده

(برای نویسندگان مقاله)

مشاهده چکیده

(برای نویسندگان مقاله)

سایر مقالات ارائه دهنده

Computational Diplomacy in the Era of Large Language Models: A Research Protocol and Evaluation Framework for Assessing Capability, Reliability, Bias, and Human Oversight in Negotiation and Mediation Support

جستجو

Computational Diplomacy in the Era of Large Language Models: A Research Protocol and Evaluation Framework for Assessing Capability, Reliability, Bias, and Human Oversight in Negotiation and Mediation Support

Niyaz Bigdeli1

سایر مقالات ارائه دهنده

سایر مقالات محل انتشار

Niyaz Bigdeli¹