Experiments¶
An experiment is a named, self-contained bundle of everything one study needs: its consent text, its scenarios, its questionnaires, its scenario order and its results. Experiments make it possible to run several studies from one HANI installation without their configuration or data touching each other.
Opt-in — nothing changes until you use it
When no experiment is selected, every path HANI uses is computed exactly
as it always was (settings under ~/negmas/hani/settings/, results under
~/negmas/hani/db/). The experiment layer only kicks in once
HANI_EXPERIMENT / --experiment is set.
Creating an experiment¶
This scaffolds ~/hani/experiments/my-study/ from your current defaults:
~/hani/experiments/my-study/
├── consent.md # copied from your settings (or package default)
├── scenario_order.txt # copied from your settings (or package default)
├── scenarios/ # seeded from the bundled sample scenarios
│ ├── Default/
│ └── ...
├── questionnaires/
│ └── per_negotiation.yaml # copied if one is found; otherwise drop yours here
└── results/ # this experiment's data is written here
Use --force (-f) to overwrite files that already exist. Existing files are
left alone by default, so re-running the command is safe.
Running an experiment¶
Select the experiment at startup — it is server-wide, not per session:
# any of these are equivalent
HANI_EXPERIMENT=my-study hani
hani-app --experiment my-study
hani-guest --experiment my-study
| Where | How |
|---|---|
| Environment | HANI_EXPERIMENT=my-study |
hani-app |
--experiment my-study |
hani-guest |
--experiment my-study |
Not on the hani sub-commands
hani main and hani guest do not (yet) take --experiment. Use the
HANI_EXPERIMENT environment variable, or call hani-app / hani-guest
directly.
What becomes experiment-scoped¶
With an experiment active, HANI resolves each artefact through this precedence chain (first hit wins):
~/hani/experiments/<name>/<file>— the experiment~/negmas/hani/settings/<file>— your shared user settings<package>/default_settings/<file>— the bundled defaults
| Artefact | Experiment-scoped location |
|---|---|
| Consent form | <exp>/consent.md |
| Scenario order | <exp>/scenario_order.txt |
| Scenarios | <exp>/scenarios/ — wins over HANI_PROLIFIC_SCENARIOS_BASE, so an experiment is fully self-contained |
| Per-negotiation questionnaire | <exp>/questionnaires/per_negotiation.yaml (an explicit PROLIFIC_PER_NEG_YAML still wins) |
| Results root | <exp>/results/ — instead of ~/negmas/hani/db/ |
Authentication, LLM settings, Support Agent settings and Autopilot settings stay
global (they live in ~/negmas/hani/settings/), since they describe the
deployment rather than the study.
Combining with Prolific¶
Experiments and the Prolific integration compose: run the guest app with both, and each participant's per-PID directory lands under the experiment's results root.
HANI_EXPERIMENT=my-study \
PROLIFIC_N_REQUIRED=5 \
PROLIFIC_PER_NEG_YAML=~/hani/experiments/my-study/questionnaires/per_negotiation.yaml \
hani-guest --port 5008
Per-participant data is then written to
~/hani/experiments/my-study/results/prolific_<PID>/ (prolific_session.json,
schedule.json, results.csv, last_scenario.txt — see
Prolific Integration).
Agent group files an external tool drops for you are also read from the active
results root: <results>/groups/<key>.json (see
Agents & Opponent Pools).
Data collected per negotiation¶
| Data | Where |
|---|---|
One row per negotiation (utilities, agreement, status, ended_by, practice, human_advantage, timing) |
results.csv in the participant's directory |
| Questionnaire answers | next to results.csv |
| Negotiation trace (what was offered) | the session's trace files |
| LLM prompts/responses of the opponent agent (why it offered) | <user_path>/llm_logs/<mechanism_id>.jsonl, one file per negotiation |
| Support Agent actions | event log, component="SupportAgent", value.actor="agent" |
Per-negotiation LLM call logging¶
Set HANI_LLM_LOG=1 to record every litellm.completion call the opponent
agent makes during a negotiation — prompt, full response, hidden reasoning if
the model emits any, model name, latency and token budget — appended to one
JSONL file per negotiation beside that negotiation's other data.
Where the trace tells you what was offered, this tells you why, which is what you need to diagnose an empty or degenerate agent turn after the fact. Logging is fully env-gated, per-session isolated (concurrent participants never cross-write), and best-effort: a logging error can never disturb a live negotiation.
Event tracking & the analytics dashboard¶
Independently of the named experiments above, HANI records UI and negotiation events through its event-tracking layer. To explore them:
This launches the Panel-based Event Analytics Dashboard in your browser.
The event database has its own, older notion of an "experiment" (a labelled window of event data), managed by a separate CLI:
hani-experiment create "Pilot A" --description "first pilot"
hani-experiment list --active-only
hani-experiment info <experiment_id>
hani-experiment end <experiment_id>
Two different things named 'experiment'
hani experiment create <name> (a sub-command of hani) creates the
directory-based experiment described on this page — consent,
scenarios, questionnaires, results.
hani-experiment (a separate executable) labels windows of event-tracking
data in the event database. They are unrelated; don't mix them up.