Resolution
Did the shopper get the right outcome?
The correct answer, or the next step the store’s policy calls for, such as a handoff to a person. Each check is judged twice, blind.
- Shopping
- 40%
- Support
- 50%
Every agent gets the same customer conversations in the chat widget on live stores. We check each answer against the store’s own policies, time the wait, and publish the scores with the evidence behind them.
Operated by Alhena. Methods and criteria are public; the scored conversations open with a verified work email.
Each tool’s composite score out of 100 from its latest published study. Open a tool for its study, sample and capture dates.
Each bar splits the composite into the points each measure adds. Scores cover the on-site chat widget on the storefronts tested, not every product or deployment.
Shopping and support are scored separately. A great product guide and a great returns desk need different strengths, so each lane weighs the three measures differently.
Resolution
The correct answer, or the next step the store’s policy calls for, such as a handoff to a person. Each check is judged twice, blind.
Quality
Direct, specific answers grounded in the store’s catalog and policies, with useful product picks and clear next steps. Graded on 26 published criteria.
Speed
Time until the complete answer appears. Three seconds scores 100; 22 seconds or more scores zero.
The same four steps for every tool. The figures come from the Alhena vs. Gorgias study.
Read the full methodologyFive real stores for each tool. The vendor suggests three; we find and verify two more from its published customer stories.
10 storefront deployments
Every store gets the same scripted conversations in its on-site chat widget: five shopping, five support, and a guardrail test that tries to pull the agent off course.
Shopping
Support
Question pools from Gorgias’s public AI agent benchmark.
100 conversations, plus 10 guardrail tests
Each checkpoint gets an AI judgment and a separate blind audit, and only counts when both agree on the evidence. Scores come from fixed arithmetic, not from a judge’s opinion of the whole conversation.
836 checkpoints judged and audited
Scores, capture dates, coverage and limits are public. The conversations behind them open with a verified work email.
Published Sep 22, 2026
Switch the metric, compare shopping and support, and open any study for its full evidence. Scores describe each study’s storefront sample, not an overall vendor ranking.
Composite / 100. The study's weighted combination of policy-compliant resolution, answer quality and full-answer speed. Open the study for its weights and coverage.
| Tool | Shopping composite | Support composite | PCR evidence coverage | Capture dates | Study and evidence |
|---|---|---|---|---|---|
| Alhenaalhena.ai | 68.4 | 81.4 | Shopping: 5 / 5 storefronts Support: 5 / 5 storefronts 25 shopping / 25 support contexts included in PCR | Sep 21, 2026 – Sep 22, 2026 | Read the study policy-resolution-v1 Published Sep 22, 2026 |
| Gorgiaswww.gorgias.com | 55.8 | 75.8 | Shopping: 4 / 5 storefronts Support: 4 / 5 storefronts 20 shopping / 18 support contexts included in PCR | Sep 21, 2026 – Sep 22, 2026 | Read the study policy-resolution-v1 Published Sep 22, 2026 |
| Sierrasierra.ai | 59 | 61.6 | Shopping: 5 / 5 storefronts Support: 5 / 5 storefronts 25 shopping / 25 support contexts included in PCR | Sep 22, 2026 – Sep 23, 2026 | Read the study policy-resolution-v1 Published Sep 23, 2026 |
SHOPPING · COMPOSITE
Latest approved composite score per tool, out of 100.
SUPPORT · COMPOSITE
Latest approved composite score per tool, out of 100.
The table and charts use the same selected metric. Samples and merchant configurations differ, and excluded storefronts are not assigned zero scores. Capture dates describe the evidence; publication dates do not establish fresh testing.
Compare the research methodsPublished comparative research, with its versioned method and complete evidence.
Comparison assembled from published evaluations; no new conversations were run for this comparison.
Captures Sep 21, 2026 – Sep 23, 2026 · policy-resolution-v1
Selected public storefront sessions. Resolution includes a policy-prescribed next step; it does not establish downstream case completion.
COMPOSITE SCORES / 100
Comparison assembled from published evaluations; no new conversations were run for this comparison.
Captures Sep 21, 2026 – Sep 23, 2026 · policy-resolution-v1
Selected public storefront sessions. Resolution includes a policy-prescribed next step; it does not establish downstream case completion.
COMPOSITE SCORES / 100
Live storefront evaluation of policy-compliant resolution, answer quality and response speed.
Captures Sep 22, 2026 – Sep 23, 2026 · policy-resolution-v1
Selected public storefront sessions. Resolution includes a policy-prescribed next step; it does not establish downstream case completion.
COMPOSITE SCORES / 100
A study of shopping and support on ten storefront deployments, with conversation evidence, public merchant policies, fixed quality criteria and separate response timing.
Captures Sep 21, 2026 – Sep 22, 2026 · policy-resolution-v1
Selected public storefront sessions. Resolution includes a policy-prescribed next step; it does not establish downstream case completion.
COMPOSITE SCORES / 100
Earlier quality-only evaluations and comparison reports retain their original method and capture dates. They are separate from the latest research composites, PCR and speed results.
2 quality-pilot tools · 1 quality comparison reports. Pilot reuse eligibility is shown only for this archive.
| Tool | Shopping quality | Support quality | Pilot sample | Original capture dates | Pilot reuse eligibility |
|---|---|---|---|---|---|
| Alhenaalhena.ai | 96 | 100 | 3 storefronts 6 conversations | Sep 20, 2026 | Refresh needed |
| Gorgiaswww.gorgias.com | 76.7 | 82.7 | 3 storefronts 6 conversations | Sep 20, 2026 | Refresh needed |
Each tool has six ten-turn conversations across three customer storefronts: three shopping and three support conversations. Pilot comparisons reuse compatible source evaluations within 30 days; publication does not reset capture dates.
Read the methods and scopeShopping and support quality across six live storefronts, scored against Gorgias’s published criteria.
Original capture dates and limitations are retained. Read the evidence before interpreting the scores.
QUALITY SCORES / 100
Name your tool and three stores that use it, then verify your work email. We find two more, run the evaluation after review, and publish results that pass validation.
Considering Alhena? Book a demo to see how its shopping and support agents would handle your customers’ questions.
What the numbers cover, and what they don’t.
Latest published result for each tool:
No. Studies test one form factor, the AI agent in each store's on-site chat widget. Alhena lists AI visibility tools for answer engines (AEO/GEO), AI-written FAQs on product pages, and guided and visual shopping such as shade matching and virtual try-on, which gorgias.com does not list. Gorgias lists a helpdesk for human agents, a help center, and SMS, which alhena.ai does not sell as its own products. Alhena connects to a helpdesk for human agents and SMS through integrations instead. None of these were tested. The study's product scope table lists each vendor's offerings with sources. See the product scope table.
Each tool is tested on five live storefronts where it is deployed: the vendor suggests three and the Lab verifies two more. Every store gets the same scripted shopping and support conversations, plus a guardrail test. Each checkpoint gets an AI judgment and a separate blind audit, and scores come from fixed arithmetic. Read the methodology.
No. Each score describes the storefronts and capture dates in its study. Merchant setups differ, and storefronts that could not be scored are left out rather than counted as zero.
It combines resolution, answer quality and full-answer speed with published weights: 40/35/25 for shopping and 50/40/10 for support. Use the metric selector to inspect each part. Resolution can credit a documented next step, such as a required handoff; it does not confirm that a refund or case was completed afterwards.
Alhena operates and commissions these studies, and its own agent appears in the results. Separate AI judging and audits do not make the Lab an independent research institution, so the scoring record and limitations accompany every report.
Study scores, sample sizes, capture dates, summaries and methods are public. Detailed conversations, criterion decisions and evidence downloads require a verified work email.
Research verifies five customer deployments before an operator approves testing. AI then runs the conversations, judges responses and audits resolution decisions, and validated results publish automatically. Temporary failures can retry within fixed limits; unsupported widgets, uncertain response attribution and failed validation may need operator review. Incomplete runs stay unpublished.
Name the tool and three stores that use it, then verify your work email. The Lab researches two more storefronts, runs the evaluation after operator review, and publishes results that pass validation. Analyze your tool.
They remain in the historical quality pilot archive with their original scores and capture dates. The pilot used a separate scope and protocol, so its scores are not combined with the current results.