Ecommerce AI agents, tested on real storefronts.

Every agent gets the same customer conversations in the chat widget on live stores. We check each answer against the store’s own policies, time the wait, and publish the scores with the evidence behind them.

Operated by Alhena. Methods and criteria are public; the scored conversations open with a verified work email.

Latest results

Each tool’s composite score out of 100 from its latest published study. Open a tool for its study, sample and capture dates.

Browse the studies
  • Resolution
  • Quality
  • Speed

Each bar splits the composite into the points each measure adds. Scores cover the on-site chat widget on the storefronts tested, not every product or deployment.

What every score is made of

Shopping and support are scored separately. A great product guide and a great returns desk need different strengths, so each lane weighs the three measures differently.

Resolution

Did the shopper get the right outcome?

The correct answer, or the next step the store’s policy calls for, such as a handoff to a person. Each check is judged twice, blind.

Shopping
40%
Support
50%

Quality

Was the answer any good?

Direct, specific answers grounded in the store’s catalog and policies, with useful product picks and clear next steps. Graded on 26 published criteria.

Shopping
35%
Support
40%

Speed

How long did the shopper wait?

Time until the complete answer appears. Three seconds scores 100; 22 seconds or more scores zero.

Shopping
25%
Support
10%

How an evaluation works

The same four steps for every tool. The figures come from the Alhena vs. Gorgias study.

Read the full methodology
  1. Pick live storefronts

    Five real stores for each tool. The vendor suggests three; we find and verify two more from its published customer stories.

    10 storefront deployments

  2. Talk to the agent like a customer

    Every store gets the same scripted conversations in its on-site chat widget: five shopping, five support, and a guardrail test that tries to pull the agent off course.

    Shopping

    • Everyday value buyer
    • Gift shopper
    • Specific need / reassurance
    • Comparison, budget-tight
    • Total beginner

    Support

    • Order tracking / delay
    • Returns & exchanges
    • Damaged / faulty item
    • Modify / cancel order
    • Shipping & returns policy

    Question pools from Gorgias’s public AI agent benchmark.

    100 conversations, plus 10 guardrail tests

  3. Judge every checkpoint twice, blind

    Each checkpoint gets an AI judgment and a separate blind audit, and only counts when both agree on the evidence. Scores come from fixed arithmetic, not from a judge’s opinion of the whole conversation.

    836 checkpoints judged and audited

  4. Publish the evidence

    Scores, capture dates, coverage and limits are public. The conversations behind them open with a verified work email.

    Published Sep 22, 2026

Explore every result

Switch the metric, compare shopping and support, and open any study for its full evidence. Scores describe each study’s storefront sample, not an overall vendor ranking.

Composite / 100. The study's weighted combination of policy-compliant resolution, answer quality and full-answer speed. Open the study for its weights and coverage.

Latest approved study result per tool. Composite scores out of 100; capture dates and included coverage are retained.
ToolShopping compositeSupport compositePCR evidence coverageCapture datesStudy and evidence
Alhenaalhena.ai
68.4
81.4
Shopping: 5 / 5 storefronts
Support: 5 / 5 storefronts
25 shopping / 25 support contexts included in PCR
Sep 21, 2026 – Sep 22, 2026Read the study policy-resolution-v1
Published Sep 22, 2026
Gorgiaswww.gorgias.com
55.8
75.8
Shopping: 4 / 5 storefronts
Support: 4 / 5 storefronts
20 shopping / 18 support contexts included in PCR
Sep 21, 2026 – Sep 22, 2026Read the study policy-resolution-v1
Published Sep 22, 2026
Sierrasierra.ai
59
61.6
Shopping: 5 / 5 storefronts
Support: 5 / 5 storefronts
25 shopping / 25 support contexts included in PCR
Sep 22, 2026 – Sep 23, 2026Read the study policy-resolution-v1
Published Sep 23, 2026
One published study result per tool.

The table and charts use the same selected metric. Samples and merchant configurations differ, and excluded storefronts are not assigned zero scores. Capture dates describe the evidence; publication dates do not establish fresh testing.

Compare the research methods

Put your AI agent to the test.

Name your tool and three stores that use it, then verify your work email. We find two more, run the evaluation after review, and publish results that pass validation.

See Alhena on your store.

Considering Alhena? Book a demo to see how its shopping and support agents would handle your customers’ questions.

Read this before the scores

What the numbers cover, and what they don’t.

What are the latest results?

Latest published result for each tool:

  • In its latest study (captured Sep 21–22, 2026), Alhena scored 68.4 for shopping and 81.4 for support, as composite scores out of 100. Full answers arrived in 11.8 s for shopping and 8.7 s for support on average.
  • In its latest study (captured Sep 21–22, 2026), Gorgias scored 55.8 for shopping and 75.8 for support, as composite scores out of 100. Full answers arrived in 17.2 s for shopping and 14.1 s for support on average.
  • In its latest study (captured Sep 22–23, 2026), Sierra scored 59 for shopping and 61.6 for support, as composite scores out of 100. Full answers arrived in 8.5 s for shopping and 8.4 s for support on average.
  • Scores cover the AI agent in each store's on-site chat widget on the storefronts in each tool's study, not other products the vendors sell, every deployment, or an overall vendor ranking. Alhena operates Alhena Research Lab.

Browse the studies

Does this compare everything Alhena and Gorgias offer?

No. Studies test one form factor, the AI agent in each store's on-site chat widget. Alhena lists AI visibility tools for answer engines (AEO/GEO), AI-written FAQs on product pages, and guided and visual shopping such as shade matching and virtual try-on, which gorgias.com does not list. Gorgias lists a helpdesk for human agents, a help center, and SMS, which alhena.ai does not sell as its own products. Alhena connects to a helpdesk for human agents and SMS through integrations instead. None of these were tested. The study's product scope table lists each vendor's offerings with sources. See the product scope table.

How is each AI agent tested?

Each tool is tested on five live storefronts where it is deployed: the vendor suggests three and the Lab verifies two more. Every store gets the same scripted shopping and support conversations, plus a guardrail test. Each checkpoint gets an AI judgment and a separate blind audit, and scores come from fixed arithmetic. Read the methodology.

Is this a ranking of vendors?

No. Each score describes the storefronts and capture dates in its study. Merchant setups differ, and storefronts that could not be scored are left out rather than counted as zero.

What does the composite score measure?

It combines resolution, answer quality and full-answer speed with published weights: 40/35/25 for shopping and 50/40/10 for support. Use the metric selector to inspect each part. Resolution can credit a documented next step, such as a required handoff; it does not confirm that a refund or case was completed afterwards.

Who runs Alhena Research Lab?

Alhena operates and commissions these studies, and its own agent appears in the results. Separate AI judging and audits do not make the Lab an independent research institution, so the scoring record and limitations accompany every report.

What can I read without signing in?

Study scores, sample sizes, capture dates, summaries and methods are public. Detailed conversations, criterion decisions and evidence downloads require a verified work email.

Is every evaluation fully automatic?

Research verifies five customer deployments before an operator approves testing. AI then runs the conversations, judges responses and audits resolution decisions, and validated results publish automatically. Temporary failures can retry within fixed limits; unsupported widgets, uncertain response attribution and failed validation may need operator review. Incomplete runs stay unpublished.

How do I get my AI agent evaluated?

Name the tool and three stores that use it, then verify your work email. The Lab researches two more storefronts, runs the evaluation after operator review, and publishes results that pass validation. Analyze your tool.

What happened to the earlier quality scores?

They remain in the historical quality pilot archive with their original scores and capture dates. The pilot used a separate scope and protocol, so its scores are not combined with the current results.