---
title: "Sorena Research Copilot vs ChatGPT Benchmarks"
canonical_url: "https://www.sorena.io/solutions/benchmarks"
source_url: "https://www.sorena.io/solutions/benchmarks"
author: "Sorena AI"
description: "Independent January 2026 benchmark comparing Sorena Research Copilot to ChatGPT on real compliance research tasks, scored blind by two auditors on requirement coverage and factual accuracy."
---
**[SORENA](https://www.sorena.io/)** - AI-Powered GRC Platform

[Home](https://www.sorena.io/) | [Solutions](https://www.sorena.io/solutions) | [Artifacts](https://www.sorena.io/artifacts) | [About Us](https://www.sorena.io/about-us) | [Contact](https://www.sorena.io/contact) | [Portal](https://app.sorena.io)

---

# Sorena Research Copilot vs ChatGPT Benchmarks

*Benchmark Report*

## We kept the receipts. Here they are

Two auditors. 43 compliance tasks. One head-to-head against a leading general-purpose AI. Sorena covered every requirement and made zero things up. Read the numbers, not the slogans.

[Try Research Copilot](/solutions/research-copilot.md) | [Book a demo](/contact.md)

```text
HERO SECTION VISUAL (animated demo)

The lines below transcribe the animated product visual shown in this page hero. It illustrates how Sorena works; it is a demo walkthrough, not literal page copy.

The head-to-head
The benchmarks hero sets up a blind, auditor-scored head-to-head between Sorena and a general-purpose AI across 43 compliance tasks, then shows the coverage gap.

Head to head
  - Sorena Compliance AI vs general-purpose AI baseline
  - 43 tasks of real compliance work
  - 2 independent auditors, blind-scored on the same rubric
  - Compliance + regulatory: privacy, AI Act, ESG, NIS2, DORA

Graded, task by task
  - Privacy 12, AI Act 9, ESG 8, Security 14
  - Sorena: answers grounded in the source regulation text
  - General-purpose AI: same tasks, same rubric, no compliance grounding

Requirement coverage
  - Sorena 100% vs general-purpose AI 26%
  - 0 factual errors across all 43 tasks (baseline produced 183)
  - Swept 5 of 5 categories: Privacy +70, Technical +72, AI Act +72, Sustainability +79, Timelines +82

Key figures: 43 tasks tested | 100% requirement coverage | 0 factual errors | Sorena 100% vs baseline 26%
```

A line-by-line audit of 43 real compliance and document-analysis tasks in which Sorena covered 100% of requirements with zero factual errors versus a general-purpose baseline's 26% coverage and 183 unsourced statements.

## Same tasks. Same rules.

*The setup*

Two independent auditors scored both tools against the same requirements across 43 real compliance, regulatory, and document-analysis tasks. No home-field advantage.

## It answered all of it.

*Coverage gap*

Sorena covered 100% of evaluated requirements. The general-purpose baseline averaged 26%. That is a 74-point gap, scored line by line against source documentation.

```text
IN-PRODUCT VISUAL - It answered all of it.

Sorena covered 100% of evaluated requirements
The general-purpose baseline averaged 26%
74-point gap
```

## Zero errors. Not one.

*The receipts*

Across all 43 sessions Sorena produced zero factual errors, with every claim traceable to the exact source. The baseline produced 183 statements presented as fact.

```text
IN-PRODUCT VISUAL - Zero errors. Not one.

Zero factual errors across all 43 sessions
Every claim traceable to the exact source
Baseline produced 183 statements presented as fact
```

## Pick any category.

*No cherry-picking*

Privacy, AI Act, sustainability, technical review, timelines, employment law. Sorena hit 100% in every one. The gap holds wherever you look.

[Try Research Copilot](/solutions/research-copilot.md)

```text
IN-PRODUCT VISUAL - Pick any category.

Sorena hit 100% in every category
Categories: Privacy, AI Act, sustainability, technical review, timelines, employment law
```

## Related resource

ChatGPT in GRC: Helpful Assistant or False Confidence?

- [Read the article](/resources/chatgpt-in-grc-false-confidence.md)
- Category: Benchmarks

## Key Results

| Metric | Sorena Research Copilot | ChatGPT (baseline) |
| --- | --- | --- |
| Perfect sessions | 43/43 | 0/43 |
| Average coverage | 100% | 25% |
| Requirements evaluated | 4332/4332 | Avg of 2 passes |
| Factual errors | 0 | 183 |

## Coverage by task type

| Category | Sessions | Sorena Coverage | ChatGPT Coverage | Gap | ChatGPT Factual Errors |
| --- | ---: | ---: | ---: | ---: | ---: |
| Privacy Audits | 12 | 100% | 30% | 70pp | 43 |
| AI Act Audits | 6 | 100% | 28% | 72pp | 20 |
| Timelines | 3 | 100% | 18% | 82pp | 17 |
| Sustainability | 9 | 100% | 21% | 79pp | 53 |
| Employment Law | 2 | 100% | 18% | 82pp | 3 |
| Technical Review | 11 | 100% | 28% | 72pp | 47 |

## All sessions

| # | Date | Category | Scenario | Summary | Sorena | ChatGPT | Factual errors | Auditor 1 (S/B/T) | Auditor 2 (S/B/T) | Sorena did well | ChatGPT issue | Evaluation note |
| ---: | --- | --- | --- | --- | ---: | ---: | ---: | --- | --- | --- | --- | --- |
| 1 | 2026-01-06 | Privacy Audit | Privacy Notice Audit - Global e-commerce retailer | Audit of a global e-commerce privacy notice against GDPR and CPRA/CCPA, focusing on transparency, retention, cross-border transfers, and user rights. | 100% | 38% | 5 | 32/16/32 | 95/25/95 | Verified the current US and EU/UK notices, mapped GDPR + CPRA requirements to exact policy text, and produced a gap list with remediation steps. | Couldn't validate the live notices, relied on outdated sources, and missed core disclosures (rights, request methods, categories). | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 32/32, baseline 16/32. Pass 2: Sorena 95/95, baseline 25/95. |
| 2 | 2026-01-06 | AI Act Compliance | AI Terms & Privacy Audit - AI lab | Audit of an AI lab's consumer terms and privacy policy for EU AI Act and GDPR, focusing on provider duties, transparency, and operational compliance. | 100% | 19% | 3 | 42/13/42 | 156/12/156 | Mapped GDPR accountability and EU AI Act GPAI duties (copyright/TDM, watermarking, transparency) into an audit-ready checklist with citations. | Stayed high-level, omitted key GDPR accountability and AI Act obligations, and leaned on secondary sources. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 42/42, baseline 13/42. Pass 2: Sorena 156/156, baseline 12/156. |
| 3 | 2026-01-06 | Privacy Audit | Privacy Policy Audit - Consumer device manufacturer | Privacy policy audit for a consumer device ecosystem, assessing GDPR/CPRA disclosures, retention clarity, transfers, and rights transparency. | 100% | 21% | 5 | 40/14/40 | 247/19/247 | Pinpointed GDPR/CPRA gaps like right-to-object and recipient disclosures, backed by precise citations and service-level retention details. | Missed multiple mandatory disclosures and raised a few misleading compliance concerns without grounding in the policy. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 40/40, baseline 14/40. Pass 2: Sorena 247/247, baseline 19/247. |
| 4 | 2026-01-06 | AI Act Compliance | Cloud Service Terms Audit - Major cloud provider | Contract-focused audit of cloud service terms and privacy notices for EU AI Act and GDPR coverage, including transfers, processor terms, and AI restrictions. | 100% | 19% | 4 | 49/14/49 | 205/19/205 | Delivered clause-level GDPR processor analysis and AI governance review, including transfer safeguards, AI service restrictions, and concrete next steps. | Skipped contract-specific privacy and AI requirements and made unsupported claims instead of verifying the source terms. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 49/49, baseline 14/49. Pass 2: Sorena 205/205, baseline 19/205. |
| 5 | 2026-01-06 | Regulatory Timeline | EUDR Timeline - Office equipment manufacturer | EU Deforestation Regulation (EUDR) workback plan for a paper supply chain, with due diligence milestones, evidence expectations, and reporting deadlines. | 100% | 19% | 9 | 30/9/30 | 207/17/207 | Built an EUDR workback plan anchored to the amended deadlines, product scope, and technical evidence requirements (geolocation, reporting, customs). | Got key dates and scope wrong and missed multiple mandatory EUDR obligations, making the timeline unsafe to rely on. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 30/30, baseline 9/30. Pass 2: Sorena 207/207, baseline 17/207. |
| 6 | 2026-01-06 | Regulatory Timeline | EUDR Timeline - Beverage multinational | EUDR compliance timeline for a global beverage supply chain, mapping commodity sourcing to scope, due diligence steps, and declaration deadlines. | 100% | 13% | 7 | 46/8/46 | 204/16/204 | Mapped EUDR obligations to a beverage supply chain with commodity/CN code examples, correct deadlines, and a step-by-step evidence plan. | Misstated go-live dates and scope and omitted many legal requirements and operational steps needed for execution. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 46/46, baseline 8/46. Pass 2: Sorena 204/204, baseline 16/204. |
| 7 | 2026-01-06 | Regulatory Timeline | EU Data Act Timeline - Connected appliance manufacturer | EU Data Act compliance timeline for a connected-appliance manufacturer, covering data access, sharing, trade secrets, and cloud switching requirements. | 100% | 26% | 1 | 28/11/28 | 33/4/33 | Provided a date-driven roadmap covering user access/sharing, trade secret safeguards, and cloud switching duties, with a practical workstream plan. | Covered basics but omitted critical obligations like cloud switching rules, gatekeeper restrictions, and Commission guidance milestones. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 28/28, baseline 11/28. Pass 2: Sorena 33/33, baseline 4/33. |
| 8 | 2026-01-06 | Privacy Audit | Privacy Policy Audit - Gaming platform | Privacy policy audit for a gaming platform, focusing on GDPR transparency and CPRA/CCPA disclosures for California residents. | 100% | 43% | 3 | 28/16/28 | 47/14/47 | Performed a requirement-by-requirement GDPR + CPRA audit including retention-per-category, request methods, and opt-out signal expectations. | Left out several California and GDPR specifics (submission methods, statutory disclosures) and offered less operational guidance. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 28/28, baseline 16/28. Pass 2: Sorena 47/47, baseline 14/47. |
| 9 | 2026-01-06 | AI Act Compliance | AI Terms & Privacy Audit - AI platform | Audit of an AI platform's terms and privacy policy for EU AI Act and GDPR readiness, emphasizing transparency, training boundaries, and provider vs deployer responsibilities. | 100% | 48% | 6 | 43/23/43 | 94/40/94 | Separated what the documents prove vs what's missing, covering rights tooling, child handling, security posture, and EU AI Act GPAI obligations. | Missed practical rights pathways and several AI transparency and child-safety requirements, including a document-reference error. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 43/43, baseline 23/43. Pass 2: Sorena 94/94, baseline 40/94. |
| 10 | 2026-01-06 | AI Act Compliance | Cloud Terms + DPA Audit - Cloud provider | Audit of cloud service terms and a data processing addendum for GDPR Article 28 and EU AI Act readiness, including key contractual caveats and deployer obligations (e.g., FRIA). | 100% | 33% | 5 | 145/49/145 | 190/63/190 | Mapped processor contract clauses and highlighted high-impact caveats (like pre-release scope exclusions), plus deployer AI Act duties such as FRIA. | Focused on broad GDPR alignment but missed key contractual caveats and most deployer-focused AI Act obligations. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 145/145, baseline 49/145. Pass 2: Sorena 190/190, baseline 63/190. |
| 11 | 2026-01-06 | AI Act Compliance | AI API Terms + Privacy Audit - Model API provider | Audit of an AI model API's terms and privacy policy for GDPR and EU AI Act requirements, focusing on data-use boundaries, retention, and developer obligations. | 100% | 28% | 1 | 45/14/45 | 88/22/88 | Clarified paid vs unpaid data-use boundaries, retention windows, and both GDPR + AI Act transparency duties for developers and deployers. | Skipped controller/legal-basis details and several concrete requirements, and included an incorrect AI Act citation. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 45/45, baseline 14/45. Pass 2: Sorena 88/88, baseline 22/88. |
| 12 | 2026-01-06 | Privacy Audit | Privacy Policy Audit - Global search platform | Privacy policy audit for a global search platform, assessing data categories, purposes, rights, transfers, retention, and opt-out tooling under GDPR and CPRA. | 100% | 25% | 4 | 38/12/38 | 105/20/105 | Grounded findings in policy text, covering legal bases, controller identity, opt-out tooling (GPC, ad settings), and actionable fixes. | Missed major requirements (cookies, minors, sources) and made unsupported claims contradicted by the policy. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 38/38, baseline 12/38. Pass 2: Sorena 105/105, baseline 20/105. |
| 13 | 2026-01-06 | Privacy Audit | Privacy Policy Audit - Social platform | Privacy policy audit for a social platform, focusing on disclosure completeness, legal bases, retention clarity, and rights mechanisms under GDPR and CPRA. | 100% | 22% | 5 | 50/19/50 | 92/5/92 | Retrieved and analyzed the current geo-dynamic policy plus the US regional notice, verifying sale/share, GPC handling, and rights workflows with quotes. | Couldn't access the current policy, relied on an outdated version, and missed essential CPRA disclosures and request mechanisms. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 50/50, baseline 19/50. Pass 2: Sorena 92/92, baseline 5/92. |
| 14 | 2026-01-06 | Privacy Audit | Privacy Statement Audit - Enterprise software vendor | Enterprise privacy statement audit for GDPR and CPRA, focusing on transparency obligations, retention, DSAR mechanics, and user rights coverage. | 100% | 47% | 2 | 70/20/70 | 38/25/38 | Completed a comprehensive GDPR + CPRA audit with verified opt-out mechanisms, DSAR timelines, and cookie/ePrivacy considerations. | Covered headline items but omitted several statutory details (marketing objection, sources, timelines) needed for a compliance-grade assessment. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 70/70, baseline 20/70. Pass 2: Sorena 38/38, baseline 25/38. |
| 15 | 2026-01-06 | AI Act Compliance | Product Terms + Privacy Audit - Enterprise cloud/vendor | Audit of enterprise product terms and privacy statements for EU AI Act and GDPR, focused on contractual commitments and shared responsibilities across the AI value chain. | 100% | 31% | 1 | 72/26/72 | 47/12/47 | Connected product terms, DPA expectations, and AI governance obligations, calling out what must be confirmed contractually vs operationally. | Provided a higher-level review and missed several contract-specific protections and practical compliance actions (breach timing, training safeguards). | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 72/72, baseline 26/72. Pass 2: Sorena 47/47, baseline 12/47. |
| 16 | 2026-01-06 | Privacy Audit | Privacy Statement Audit - Streaming service | Privacy statement audit for a streaming service, evaluating GDPR transparency and CPRA disclosures such as sharing, preference signals, and required policy structure. | 100% | 33% | 5 | 41/19/41 | 74/14/74 | Verified EU/UK lawful bases and transfer safeguards, and pinpointed California disclosure gaps (GPC, 12-month lists, non-discrimination). | Made incorrect claims about lawful bases, transfers, and CPRA disclosures and conflated DNT vs GPC. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 41/41, baseline 19/41. Pass 2: Sorena 74/74, baseline 14/74. |
| 17 | 2026-01-06 | Privacy Audit | Terms + Privacy Audit - Secure messaging app | Audit of a secure messaging app's terms and privacy disclosures for GDPR and CPRA, focusing on lawful bases, retention, rights, and audit-ready gaps. | 100% | 32% | 1 | 38/18/38 | 67/11/67 | Reviewed multiple relevant sources (policy, shop opt-out page, support guidance) and flagged Art. 27 representative and CPRA signal requirements. | Missed several requirements and confused organizational structure, leading to a misleading compliance conclusion. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 38/38, baseline 18/38. Pass 2: Sorena 67/67, baseline 11/67. |
| 18 | 2026-01-06 | Privacy Audit | Privacy Policy Audit - Music streaming service | Privacy policy audit for a music streaming service, reviewing GDPR/CPRA disclosures around data categories, sharing, international transfers, and rights. | 100% | 36% | 1 | 40/21/40 | 84/17/84 | Found and cited specific policy statements (sale/share posture) and assessed children's protections, authorized agents, and request methods. | Missed key disclosures and incorrectly claimed important statements were absent. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 40/40, baseline 21/40. Pass 2: Sorena 84/84, baseline 17/84. |
| 19 | 2026-01-06 | Privacy Audit | Privacy Policy Audit - Messaging platform | Privacy policy audit for a messaging platform under GDPR and CPRA, including transfers, retention, rights workflows, and required disclosures. | 100% | 56% | 1 | 45/18/45 | 28/20/28 | Covered GDPR + CPRA specifics, including timelines, 12-month disclosures, and nuanced cross-regulation considerations (ePrivacy, case law). | Omitted several mandatory CPRA/GDPR elements (non-discrimination, SPI scope, minors) and provided less actionable remediation. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 45/45, baseline 18/45. Pass 2: Sorena 28/28, baseline 20/28. |
| 20 | 2026-01-06 | Privacy Audit | Privacy Policy Audit - Short-form video platform | Privacy policy audit for a short-form video platform under GDPR and CPRA, focusing on disclosures, rights, ad legal bases, and cross-border processing. | 100% | 24% | 6 | 38/14/38 | 103/12/103 | Validated EEA/UK disclosures (consent for ads, complaint routes) and pinpointed California statutory gaps like required link text and SPI handling. | Missed several policy-specific disclosures and lacked statutory precision on opt-out and automated decision-making requirements. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 38/38, baseline 14/38. Pass 2: Sorena 103/103, baseline 12/103. |
| 21 | 2026-01-06 | Privacy Audit | Privacy Policy Audit - Social network | Privacy policy audit for a social network, evaluating GDPR and CPRA transparency items, user rights coverage, and retention disclosures. | 100% | 30% | 5 | 120/47/120 | 66/14/66 | Mapped the policy to GDPR + CPRA with clear statutory checkpoints (toll-free methods, 12-month lists, SPI limit-use) and actionable remediation. | Skipped critical California format and request-method requirements and left gaps in indirect-source and necessity disclosures. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 120/120, baseline 47/120. Pass 2: Sorena 66/66, baseline 14/66. |
| 22 | 2026-01-07 | Employment Law | Union Comparison - Swedish software developer | Comparison of Swedish unions and collective agreements for a full-time software developer, covering benefits, tradeoffs, and agreement coverage. | 100% | 20% | 1 | 45/10/45 | 52/9/52 | Compared unions and collective agreements with the practical details that drive decisions (time bank, sick pay layers, pensions, notice periods). | Stayed at a high level, missed core CBA differences, and included an incorrect benefit-duration claim. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 45/45, baseline 10/45. Pass 2: Sorena 52/52, baseline 9/52. |
| 23 | 2026-01-07 | Employment Law | Employment Contract Review - Sweden | Employment contract compliance review under Swedish law, identifying risk areas, missing mandatory elements, and practical remediation guidance. | 100% | 17% | 2 | 33/8/33 | 53/5/53 | Applied the right Swedish-law framework (CBA context, working time limits, deductions, sick pay) and separated real risks from non-issues. | Flagged clauses as violations without CBA context and missed several statutory requirements needed for a defensible review. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 33/33, baseline 8/33. Pass 2: Sorena 53/53, baseline 5/53. |
| 24 | 2026-01-07 | Technical Review | Security Guidelines Review - Connected products | Technical review of connected product security guidelines, identifying inconsistencies and aligning requirements to real regulatory regimes and standards. | 100% | 25% | 12 | 26/10/26 | 141/16/141 | Turned internal security guidance into an audit-ready, regulator-aligned checklist (CRA/RED/EN 303 645) with dates, scope boundaries, and citations. | Missed most regulatory specifics and produced multiple incorrect or vague suggestions that weaken the guideline. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 26/26, baseline 10/26. Pass 2: Sorena 141/141, baseline 16/141. |
| 25 | 2026-01-10 | Technical Review | Cybersecurity Conformity Planning - CE/CRA readiness | Cybersecurity conformity assessment planning for CE/RED readiness, including evidence artifacts, assessment steps, test strategy, and documentation expectations. | 100% | 37% | 5 | 149/65/149 | 114/35/114 | Produced a CE conformity assessment plan with harmonized-standard traceability (OJ entries, clause IDs) and concrete pass/fail test criteria. | Gave a conceptual plan but missed traceability details auditors need (OJ numbers, provision IDs, acceptance criteria) and had version inconsistencies. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 149/149, baseline 65/149. Pass 2: Sorena 114/114, baseline 35/114. |
| 26 | 2026-01-10 | Technical Review | IoT Security Crosswalk + Test Plan - Consumer IoT | Consumer IoT security crosswalk and test plan, mapping ETSI and NIST requirements into testable procedures and evidence lists. | 100% | 30% | 4 | 118/45/118 | 90/20/90 | Delivered a full ETSI EN 303 645 <-> NIST 8259A crosswalk with quote-level traceability and assessor-grade test techniques. | Provided a general mapping but lacked verifiable quote anchors, missed key control extractions, and suggested risky version/citation approaches. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 118/118, baseline 45/118. Pass 2: Sorena 90/90, baseline 20/90. |
| 27 | 2026-01-10 | Technical Review | FIPS 140 Delta Analysis - Cryptographic modules | Delta analysis of FIPS 140-1 vs FIPS 140-2 for cryptographic modules, highlighting changed requirements and assessment implications. | 100% | 41% | 1 | 118/54/118 | 58/21/58 | Captured clause-by-clause deltas with testable requirements, including numeric thresholds and CMVP-ready artifacts. | Covered the basics but missed many validation-critical details (authentication thresholds, DTR/IG references, level-specific RNG rules). | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 118/118, baseline 54/118. Pass 2: Sorena 58/58, baseline 21/58. |
| 28 | 2026-01-10 | Technical Review | FIPS <-> ISO Crypto Module Mapping | Crosswalk between FIPS and ISO/IEC cryptographic module requirements, mapping controls and clarifying evidence expectations for audits. | 100% | 34% | 1 | 110/42/110 | 62/19/62 | Mapped FIPS 140-2/140-3 to ISO 19790 with deep links to the SP 800-140x series and validation evidence expectations. | Delivered a partial crosswalk but omitted key precision items (verbatim section quotes, interface taxonomy, program documents). | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 110/110, baseline 42/110. Pass 2: Sorena 62/62, baseline 19/62. |
| 29 | 2026-01-10 | Technical Review | ISO 27001/27002 Migration Package - ISMS update | ISO 27001/27002 migration package from 2013 to 2022, covering control changes, reorganization themes, and statement of applicability updates. | 100% | 34% | 4 | 130/54/130 | 88/24/88 | Created a practitioner-ready migration kit: machine-readable change matrix, filled SoA examples, and a timeboxed transition plan backed by authoritative sources. | Missed key migration artifacts (Annex B baseline, CSV/filled SoA) and made a couple of unverified claims about control groupings. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 130/130, baseline 54/130. Pass 2: Sorena 88/88, baseline 24/88. |
| 30 | 2026-01-10 | Technical Review | NIST 800-53 <-> ISO 27001/27002 Mapping | Control mapping between NIST SP 800-53 Rev. 5 and ISO/IEC 27001:2022 Annex A to support alignment, crosswalks, and audit preparation. | 100% | 12% | 5 | 175/40/175 | 68/1/68 | Explained rev4 to rev5 changes and produced an auditor-friendly crosswalk to ISO with gaps, tests, and source-grounded rationale. | Relied too much on workbook references and provided fewer verifiable anchors and test/evidence details. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 175/175, baseline 40/175. Pass 2: Sorena 68/68, baseline 1/68. |
| 31 | 2026-01-10 | Technical Review | NIST CSF 1.1 to 2.0 Crosswalk | Crosswalk from NIST Cybersecurity Framework 1.1 to 2.0, highlighting changes and mapping structure to support transition planning. | 100% | 29% | 5 | 180/72/180 | 39/7/39 | Mapped CSF changes to practical transition steps and linked crosswalks to authoritative artifacts for traceability and automation. | Delivered a reasonable summary but provided fewer pointers to official mapping exports and less detail on profile/tier migration. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 180/180, baseline 72/180. Pass 2: Sorena 39/39, baseline 7/39. |
| 32 | 2026-01-10 | Technical Review | NIST 800-171 Rev. 3 Delta + CMMC Mapping | Clause-level delta analysis of NIST SP 800-171 Rev. 2 vs Rev. 3 with CMMC 2.0 mapping, identifying added objectives and assessment impact. | 100% | 18% | 4 | 170/30/170 | 79/14/79 | Produced audit-defensible deltas with examples, ODP governance, and a clear Rev. 3 to Rev. 2 to CMMC mapping approach. | Listed new requirements but missed assessment-method impacts and source traceability, and added a nonessential news citation. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 170/170, baseline 30/170. Pass 2: Sorena 79/79, baseline 14/79. |
| 33 | 2026-01-10 | Technical Review | OT Security Framework Crosswalk + Gaps (IEC 62443/NIST) | OT security framework crosswalk between IEC 62443 requirements and NIST SP 800-82 guidance, identifying gaps plus example tests and evidence. | 100% | 20% | 3 | 99/20/99 | 99/20/99 | Built an OT-safe IEC 62443 <-> NIST 800-82 crosswalk with verification methods, evidence, and a realistic gap model. | Covered high-level mapping but missed several nuanced gaps and lacked the same level of audit-defensible sourcing. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 99/99, baseline 20/99. Pass 2: Sorena 99/99, baseline 20/99. |
| 34 | 2026-01-10 | Technical Review | PCI DSS v3.2.1 to v4.0 Delta + Crosswalk | PCI DSS v3.2.1 to v4.0 delta analysis with crosswalks to NIST SP 800-53 Rev. 5 and ISO/IEC 27001:2022, including key changes and timelines. | 100% | 32% | 3 | 155/52/155 | 155/47/155 | Delivered a PCI DSS migration package with authoritative citations, crosswalks, and evidence-ready remediation guidance. | Missed several audit-defensibility elements (official artifacts, full quotes) and included an incorrect timeline claim. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 155/155, baseline 52/155. Pass 2: Sorena 155/155, baseline 47/155. |
| 35 | 2026-01-14 | Sustainability Compliance | EU Energy Efficiency Directive Readiness - IoT appliances | Readiness assessment for an EU IoT home-appliance manufacturer under the EU Energy Efficiency Directive, including obligations, exemptions, and a practical implementation plan. | 100% | 26% | 8 | 81/21/81 | 83/22/83 | Separated what is mandatory vs optional, identified applicability triggers, and produced an evidence-driven readiness roadmap with governance and reporting steps. | Missed or diluted multiple explicit requirements and produced several overconfident obligations without sufficient grounding. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 81/81, baseline 21/81. Pass 2: Sorena 83/83, baseline 22/83. |
| 36 | 2026-01-14 | Sustainability Compliance | ESPR + Digital Product Passport Readiness - Appliances | Readiness assessment for ESPR and Digital Product Passport obligations for an EU smart-appliance manufacturer, covering applicability, data requirements, and execution plan. | 100% | 22% | 2 | 122/25/122 | 112/26/112 | Mapped the expected DPP/ESPR obligations to concrete product, data, and supply-chain controls with implementation sequencing. | Left key requirements vague, missed multiple Sorena-identified constraints, and under-specified required artifacts and scope conditions. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 122/122, baseline 25/122. Pass 2: Sorena 112/112, baseline 26/112. |
| 37 | 2026-01-14 | Sustainability Compliance | EU Batteries Regulation Readiness - Embedded batteries | Readiness plan for EU Batteries Regulation obligations relevant to consumer appliances with embedded or supplied batteries, including labeling, due diligence, and reporting. | 100% | 27% | 6 | 147/35/147 | 112/34/112 | Provided a compliance-ready breakdown of obligations by battery type and role (producer/importer), with evidence deliverables and timeline discipline. | Overlooked several explicit obligations and mis-prioritized workstreams, creating compliance gaps for key battery-related duties. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 147/147, baseline 35/147. Pass 2: Sorena 112/112, baseline 34/112. |
| 38 | 2026-01-14 | Sustainability Compliance | EU CSDDD Readiness - Supply chain due diligence | Readiness assessment for EU corporate sustainability due diligence obligations for an EU-listed appliance manufacturer, including governance, risk mapping, and remediation. | 100% | 34% | 4 | 98/28/98 | 98/38/98 | Turned due diligence requirements into implementable controls: governance, policy, risk mapping, supplier engagement, grievance handling, and reporting. | Missed several explicit duties and introduced ambiguous guidance that would leave audit-critical evidence and controls incomplete. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 98/98, baseline 28/98. Pass 2: Sorena 98/98, baseline 38/98. |
| 39 | 2026-01-14 | Sustainability Compliance | EU CSRD/ESRS Compliance Plan - Listed appliance manufacturer | CSRD/ESRS compliance applicability and readiness plan for an EU-listed smart-appliance manufacturer, including reporting scope, materiality, assurance, and data controls. | 100% | 20% | 11 | 119/19/119 | 104/25/104 | Clarified applicability boundaries and produced a compliance program plan spanning governance, materiality, ESRS datapoints, assurance, and disclosure logistics. | Had multiple scope/timeline inconsistencies and missed high-impact requirements around reporting mechanics and assurance-readiness artifacts. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 119/119, baseline 19/119. Pass 2: Sorena 104/104, baseline 25/104. |
| 40 | 2026-01-14 | Sustainability Compliance | EU CSRD/ESRS Compliance Plan - Listed automotive manufacturer | CSRD/ESRS applicability and compliance plan for an EU-listed automotive manufacturer, including ESRS scope, phased timelines, and operational reporting readiness. | 100% | 25% | 5 | 72/19/72 | 100/23/100 | Provided a structured, evidence-driven program plan with clear scoping, sequencing, and accountability to operationalize ESRS reporting. | Missed multiple explicit requirements and introduced misleading simplifications that would create gaps in CSRD reporting readiness. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 72/72, baseline 19/72. Pass 2: Sorena 100/100, baseline 23/100. |
| 41 | 2026-01-14 | Sustainability Compliance | EU Green Claims Readiness - IoT appliances | Readiness assessment for EU green-claims compliance in marketing and product communications for an EU IoT appliance manufacturer. | 100% | 12% | 6 | 89/11/89 | 99/12/99 | Converted green-claims obligations into a practical substantiation workflow: claims inventory, evidence standards, governance, and review gates. | Overlooked key compliance requirements and provided under-scoped guidance that could increase greenwashing risk. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 89/89, baseline 11/89. Pass 2: Sorena 99/99, baseline 12/99. |
| 42 | 2026-01-14 | Sustainability Compliance | EU Packaging Waste EPR Readiness - Appliances | Packaging waste and EPR compliance readiness plan for an EU home-appliance manufacturer, covering registration, reporting, labeling, and operational controls. | 100% | 14% | 6 | 103/14/103 | 119/17/119 | Outlined a compliance-ready EPR program with country-by-country obligations, operational ownership, and reporting/evidence requirements. | Missed several explicit obligations and under-specified evidence and process controls required for multi-country EPR compliance. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 103/103, baseline 14/103. Pass 2: Sorena 119/119, baseline 17/119. |
| 43 | 2026-01-14 | Sustainability Compliance | EU Water Sustainability Readiness - IoT appliances | EU water-sustainability and water-efficiency compliance readiness plan for IoT appliances, including product efficiency, disclosures, and governance. | 100% | 14% | 5 | 145/29/145 | 137/12/137 | Translated water-efficiency obligations and expectations into actionable controls, product requirements, and evidence-backed readiness steps. | Left multiple requirements uncovered and included misleading generalizations that would not hold up in an audit. | Two-pass scoring against the same requirements per pass. Pass 1: Sorena 145/145, baseline 29/145. Pass 2: Sorena 137/137, baseline 12/137. |

*Auditor columns show Sorena/Baseline/Total requirements met in each independent review pass.*

## What this means for GRC teams

- **Complete Coverage**: 100% coverage across 4,332 requirements, with no surprise gaps left for auditors to find.
- **Zero Factual Errors**: 0 factual errors flagged across 43 sessions, reducing the risk of acting on incorrect information.
- **Audit-Ready Citations**: Direct links to exact text passages in legal documents for full traceability.
- **Specialized Expertise**: Purpose-built for regulatory research, not a general-purpose tool stretched thin.

## Methodology

### Evaluation overview

| Field | Value |
| --- | --- |
| Period | Jan 2026 |
| Task Categories | 6 |
| Total Sessions | 43 |
| Requirements Evaluated | 4,332 |
| Internet Access | Enabled |
| Reasoning Effort | High |

### Scoring criteria

- Explicitly addressed the requirement
- Provided accurate information
- Cited verifiable sources where applicable

### Dual independent review

- Auditor 1: Independent review against compliance requirements
- Auditor 2: Independent review against compliance requirements

### Disclaimers

- Results based on internal evaluation conducted January 2026.
- ChatGPT (baseline) is OpenAI ChatGPT, used as a general-purpose AI comparison.
- All factual errors counted are from ChatGPT responses only.
- This evaluation focused on regulatory and compliance research tasks.
- Results may vary depending on specific use case and document types.
- Not a substitute for legal counsel or professional advice.

## The benchmark is done. The numbers are public. Now run yours.

[Book a demo](/contact.md) | [Explore the platform](/solutions.md)


---

[Privacy Policy](https://www.sorena.io/privacy.md) | [Terms of Use](https://www.sorena.io/terms-of-use.md) | [DMCA](https://www.sorena.io/dmca.md) | [About Us](https://www.sorena.io/about-us.md)

(c) 2026 Sorena AB (559573-7338). All rights reserved.

Source: https://www.sorena.io/solutions/benchmarks.md