Ollama-Judge

The application of large language models to legal reasoning presents both opportunities and challenges. While LLMs demonstrate impressive capabilities in text understanding and generation, their use in high-stakes domains such as adjudication requires careful consideration of transparency, reproducibility, and structured reasoning [1]. Prior work has explored LLMs for legal document analysis [2], case outcome prediction [3], and argument mining [4], but few systems attempt to simulate the full adjudicative process with multiple interacting agents.

Related work

LEGAL-BERT [2] showed that domain-adaptive pretraining improves legal NLP, while Medvedeva et al. [3] demonstrated that case outcomes at the European Court of Human Rights can be predicted from textual features. Argument mining [4] recovers claim–premise structure from discourse. Foundation-model surveys [1], transformer architectures [7], few-shot prompting [8], chain-of-thought [9], and self-consistency [10] supply the reasoning substrate. Ollama-Judge differs in combining a multi-agent courtroom—ideologically diverse justices plus a separate jury—with an explicit, inspectable reasoning chain rather than a single predictive classifier.

Contributions

The remainder of this paper is organized as follows. Section II describes the system architecture. Section III details the agent design. Section IV presents the communication protocol. Section V discusses implementation. Section VI presents case studies. Section VII reports results and limitations. Section VIII concludes.

Ollama-Judge follows a phased pipeline architecture where each stage processes the case transcript through a different agentic lens. Figure 1 illustrates the overall flow from structured case input to a recorded verdict.

A. Case model

Cases are represented as JSON documents containing structured fields for parties, jurisdiction, opening testimony, cross-examination transcripts (with witness Q&A and objections), and closing testimony. The framework supports two case types—Criminal (beyond reasonable doubt) and Civil (preponderance of the evidence)—with appropriate burden-of-proof instructions automatically injected into agent prompts. Plain-text transcripts are also accepted; metadata such as caption and jurisdiction is inferred when present.

B. LLM backend

All LLM inference is performed through the Ollama REST API [5], which runs locally and manages model lifecycle. The default model is Llama 3.2 (8B parameters), though any Ollama-compatible model may be substituted—including a separate jury model. Metal GPU acceleration is provided transparently by Ollama on Apple Silicon hardware; no additional GPU compute crates are required in the Rust binary.

C. Throttling and resource management

To prevent resource exhaustion on consumer hardware, the system employs a tokio-based semaphore that limits concurrent LLM requests. Default settings restrict to three simultaneous calls with a 200 ms inter-request delay. These parameters are user-configurable; machines with roughly 8 GB of memory typically run more reliably at --throttle 2 --delay-ms 500.

A. Supreme Court justices

Nine justice agents are instantiated, each assigned a distinct judicial ideology drawn from a linear spectrum ranging from strict constructionist (Justices 1–3) through moderate pragmatist (4–6) to broad interpreter (7–9). Diversity is injected via system prompts that describe each justice’s interpretive framework, producing varied legal reasoning even when analyzing identical evidence.

Each justice follows a two-phase workflow that mirrors Supreme Court conference procedure:

B. Jury panel

Twelve juror agents operate independently, each receiving the full case transcript and producing a factual verdict. Jurors are instructed on the appropriate burden of proof—beyond a reasonable doubt for criminal cases, preponderance of the evidence for civil cases—and do not participate in the justice deliberation phase. This separation of legal and factual determination mirrors the distinction between judge and jury in common law systems. Optional odd-count jury re-simulations can compute a lossless majority consensus across seeded parameterizations.

C. Super Why reasoning engine

Super Why provides transparent, step-by-step reasoning by decomposing the decision process into two combined LLM calls that together cover four analytical levels. Earlier revisions used four separate calls; the current design preserves the four-level audit trail while reducing latency.

Each call builds on the previous output, producing a reasoning chain in which every step is independently verifiable—addressing the “black box” critique of LLM-based decision systems. Call 2 ends with a structured FINAL DECISION and a one-sentence SUPER WHY SAYS summary.

Agent communication is implemented using Rust’s async channel primitives from the tokio crate [6]. The protocol employs two channel types and is designed for deterministic collection without deadlocks: the orchestrator drops its sender endpoints after spawning all agents, allowing the receiver to terminate naturally once every agent has delivered a result. This pattern avoids explicit barrier synchronization.

procedure Deliberation(case, justices, client)
    prelim ← ∅
    tx_prelim ← mpsc(9)
    tx_broadcast ← broadcast(1)

    for each j ∈ justices
        spawn analyze(j, case, client, tx_prelim)

    for i ← 1 to 9
        prelim ← prelim ∪ recv(tx_prelim)

    send(tx_broadcast, prelim)

    for each j ∈ justices
        spawn deliberate(j, case, client, prelim, tx_final)

    return collect(tx_final)

Ollama-Judge is implemented in Rust (edition 2021) and compiles with Rust 1.75+. The implementation emphasizes minimal dependencies and small binary size. The release binary is 4 MB with zero runtime dependencies beyond the Ollama HTTP endpoint. Source is approximately 780 lines across 17 files.

A. Metal GPU acceleration

On Apple Silicon, Ollama automatically uses the Metal Performance Shaders framework for GPU-accelerated inference. The Rust application detects Metal availability at compile time with cfg!(target_os = "macos") and reports acceleration status at startup. No Metal shader code ships in the application binary.

B. Command-line interface

A clap-derived CLI exposes panel size, model selection, sampling, throttling, and output format. Justice temperature defaults to 0.3 (more conservative legal analysis); juror temperature defaults to 0.5 (slightly more diverse fact-finding).

ollama serve
ollama pull llama3.2
cargo run --release -- --case examples/criminal_case.json
cargo run --release -- --case examples/civil_case.json -o verdict.md

We evaluated Ollama-Judge on two synthetic cases designed to test different aspects of legal reasoning. Both include complete opening testimony, cross-examination (six witness sessions with Q&A and objections), and closing testimony.

Each trial generates a complete verdict document. The system consistently produces structured output with all required components. Deliberation typically yields some vote changes as justices respond to peer reasoning, though the majority opinion remains stable across rounds because initial analyses are independent.

A. Performance

With default throttle settings (3 concurrent requests, 200 ms delay), a full trial with 9 justices (2 rounds) and 12 jurors requires 30 LLM calls plus 2 Super Why calls—approximately 32 sequential batches. On an Apple M2 Max with 64 GB RAM, a complete trial with llama3.2 finishes in about 8–12 minutes depending on transcript length.

B. Limitations

We presented Ollama-Judge, a multi-agent framework for simulated courtroom adjudication that combines nine justice agents, twelve juror agents, and a transparent reasoning engine into a cohesive pipeline. The implementation demonstrates that complex multi-agent legal reasoning can be achieved with minimal dependencies and a small binary footprint.

Chandra, S. (2026). Ollama-Judge: An Agentic Multi-LLM Framework for Simulated Courtroom Adjudication with Transparent Reasoning (Technical Report No. 1). Sapana Micro Software, Pittsburg, KS.

@techreport{chandra2026ollamajudge,
  title       = {Ollama-Judge: An Agentic Multi-LLM Framework for Simulated Courtroom Adjudication with Transparent Reasoning},
  author      = {Chandra, Shyamal},
  institution = {Sapana Micro Software},
  year        = {2026},
  month       = jul,
  type        = {Technical Report},
  number      = {1},
  address     = {Pittsburg, KS}
}

References

1. R. Bommasani et al., “On the opportunities and risks of foundation models,” arXiv:2108.07258, 2021.
2. I. Chalkidis, M. Fergadiotis, P. Malakasiotis, N. Aletras, and I. Androutsopoulos, “LEGAL-BERT: The muppets straight out of law school,” in Proc. EMNLP, 2020.
3. M. Medvedeva, M. Vols, and M. Wieling, “Using machine learning to predict decisions of the European Court of Human Rights,” Artificial Intelligence and Law, vol. 28, no. 2, 2019.
4. I. Habernal and I. Gurevych, “Argumentation mining in user-generated web discourse,” Computational Linguistics, vol. 43, no. 1, 2017.
5. Ollama, “Ollama: Get up and running with large language models locally,” 2024. https://ollama.com
6. Tokio Contributors, “Tokio: An asynchronous runtime for the Rust programming language,” 2024. https://tokio.rs
7. A. Vaswani et al., “Attention is all you need,” in Proc. NeurIPS, 2017.
8. T. Brown et al., “Language models are few-shot learners,” in Proc. NeurIPS, 2020.
9. J. Wei et al., “Chain-of-thought prompting elicits reasoning in large language models,” in Proc. NeurIPS, 2022.
10. X. Wang et al., “Self-consistency improves chain of thought reasoning in language models,” in Proc. ICLR, 2023.
11. Rust Team, “The Rust programming language,” 2024. https://www.rust-lang.org
12. P. Sewell, S. Lindley, and K. Donnelly, “Asynchronous communication in Rust,” Journal of Functional Programming, 2010.
13. D. Silver et al., “Reward is enough,” Artificial Intelligence, vol. 299, 2016.
14. I. Goodfellow et al., “Generative adversarial nets,” in Proc. NeurIPS, 2014.
15. J. Achiam et al., “GPT-4 technical report,” arXiv:2303.08774, 2023.