Comparative study
Alhena vs. Gorgias: which AI agent gets shoppers the right answer?
We ran the same scripted shopping and support conversations with each AI agent on its own live storefronts, 10 in total, then scored every answer for resolution, quality and speed.
Key findings
- Shopping composite, out of 100: Alhena 68.4, Gorgias 55.8.
- Support composite, out of 100: Alhena 81.4, Gorgias 75.8.
- Overall composite, the mean of both lanes: Alhena 74.9, Gorgias 65.8.
- Average time to a full answer: shopping Alhena 11.8 s, Gorgias 17.2 s; support Alhena 8.7 s, Gorgias 14.1 s.
- Gorgias's results include 4 of 5 storefronts for shopping and 4 of 5 for support. Storefronts that could not be scored are left out, not counted as zero.
- Scores cover the AI agent in each store's on-site chat widget across the 10 storefront deployments captured Sep 21–22, 2026. They do not cover other products either vendor sells, every deployment, or an overall vendor ranking. Alhena operates Alhena Research Lab.
Results
Scores out of 100 for the storefronts in this study. Each bar splits a lane composite into the points resolution, quality and speed add.
ShoppingComposite / 100
- Alhena11.8 s to a full answer68.4
- Gorgias17.2 s to a full answer55.8
SupportComposite / 100
- Alhena8.7 s to a full answer81.4
- Gorgias14.1 s to a full answer75.8
- Resolution
- Quality
- Speed
Score breakdown
| Measure | Shopping | Support | ||
|---|---|---|---|---|
| Alhena | Gorgias | Alhena | Gorgias | |
| Composite | 68.4 | 55.8 | 81.4 | 75.8 |
| Resolution | 64.8 | 60.8 | 72.8 | 68 |
| Quality | 83 | 72 | 95 | 94 |
| Speed score | 53.7 | 25.3 | 70 | 41.6 |
| Average full answer | 11.8 s | 17.2 s | 8.7 s | 14.1 s |
| Storefronts included | 5 of 5 | 4 of 5 | 5 of 5 | 4 of 5 |
| Conversations included | 25 of 25 | 20 of 25 | 25 of 25 | 18 of 25 |
Full evidence coverage
| Company | Checkpoints: planned / attempted / observed | Recorded submitted / assessed / unassessable | Attained / unverified | Contexts: included / excluded | Stores: included / excluded | Quality eligible: contexts / stores | Captures: original / repaired |
|---|---|---|---|---|---|---|---|
| Alhena | 250 / 242 / 242 | 242 / 242 / 0 | 158 / 64 | 25 / 0 | 5 / 0 | 25 / 5 | 23 / 2 |
| Gorgias | 250 / 187 / 187 | 187 / 164 / 23 | 109 / 37 | 20 / 5 | 4 / 1 | 17 / 4 | 23 / 2 |
| Company | Checkpoints: planned / attempted / observed | Recorded submitted / assessed / unassessable | Attained / unverified | Contexts: included / excluded | Stores: included / excluded | Quality eligible: contexts / stores | Captures: original / repaired |
|---|---|---|---|---|---|---|---|
| Alhena | 250 / 250 / 250 | 250 / 250 / 0 | 182 / 63 | 25 / 0 | 5 / 0 | 25 / 5 | 19 / 6 |
| Gorgias | 250 / 205 / 205 | 205 / 180 / 25 | 122 / 45 | 18 / 7 | 4 / 1 | 18 / 4 | 23 / 2 |
- Policy-compliant resolution · public-session scope
- Verified correct answer or policy-prescribed next step, averaged equally by conversation and storefront. This does not measure downstream case completion.
- Quality
- The original fixed answer-quality rubric. Unchanged captures retain their original judgments; repaired captures receive a fresh judgment and audit.
- Full-answer speed
- Average time to the complete answer, scored 100 at 3 seconds, zero at 22 seconds, bounded between 0 and 100.
- Composite
- Shopping: 40% policy-compliant resolution, 35% quality, 25% speed. Support: 50% policy-compliant resolution, 40% quality, 10% speed.
- Overall composite
- Equal mean of the two unrounded lane composites; only available when both lanes meet coverage floors.
This study covers the chat widget only
Studies test one form factor: the AI agent in each store’s on-site chat widget. Both vendors sell more than that. The table lists what each one publicly offers, from its own website as of . It is not part of this study’s testing.
| Capability | Alhena | Gorgias | In this study |
|---|---|---|---|
| AI shopping assistant in the on-site chat widget | AI Shopping Assistant | AI Shopping Assistant | Tested |
| AI support agent in the on-site chat widget | AI Support Concierge | AI Agent | Tested |
| AI visibility in answer engines (AEO/GEO) | AI Visibility | Not listed on gorgias.com | Not tested |
| AI-written FAQs on product pages | Smart FAQs and the AEO FAQ Engine | Not listed on gorgias.com | Not tested |
| Conversational product search | Conversational Search | Search to revenue, in the AI Shopping Assistant | Only inside chat conversations |
| Guided and visual shopping (shade matching, similar products, virtual try-on) | Guided Discovery, Shade Matcher, Similar Product Finder, virtual try-ons | Not listed on gorgias.com | Not tested |
| Proactive nudges and offers | Conversion Nudges | Convert | Not tested |
| Helpdesk and ticketing for human agents | Via integrations: Zendesk, Gorgias, Salesforce Service Cloud, Freshdesk and more | Helpdesk | Not tested |
| Help center | Not listed on alhena.ai | Help Center | Not tested |
| Email support | Email integration | Helpdesk | Not tested |
| SMS | Via integrations: through a connected helpdesk | SMS | Not tested |
| Voice | Voice AI | Voice | Not tested |
| AI Social Commerce | Not tested | ||
| Instagram and Facebook | AI Social Commerce | Social media | Not tested |
| Commerce platforms listed | Shopify, WooCommerce, Salesforce Commerce Cloud | Shopify, BigCommerce, Magento, WooCommerce, PrestaShop | Not tested |
Alhena products this study doesn’t compare
- AI Visibility (AEO/GEO)Measures how your products appear inside AI-generated shopping answers and gives product-level actions to improve visibility.
- Smart FAQs and the AEO FAQ EngineAI-written answers embedded on every product page, plus citation-ready Q&A pairs built for answer engines.
- Conversational SearchNatural-language product search for layered queries. The study’s shopping conversations asked product questions but did not score search on its own.
- Guided and visual shoppingGuided Discovery, Shade Matcher, Similar Product Finder, and virtual try-ons for fit and color.
Read this before the scores
Policy-compliant resolution can include a verified policy-required next step. It does not establish actual refunds, completed account actions or human-resolved orders. This is a separately versioned Alhena Research Lab method, not an independent certification or the Gorgias leaderboard.
- Alhena commissioned and operates this study. Selected storefront deployments do not establish a universal provider ranking or independent certification.
- This methodology was developed after reviewing the original automation results, then frozen before the new policy-resolution judgments.
- The sessions were logged out. No real order, refund, account change or completed human resolution was independently verified.
- Unknown delivery or actor attribution remains unassessable. Unsent questions are not successes or failures. Conditional scores must be read with planned and assessed coverage.
- All ten JSHealth core contexts were excluded because a generative actor and delivery could not be confirmed. Two additional support contexts stopped at login gates.
- Two retained dryrobe shopping contexts have only six measured replies. Twelve cause-selected capture repairs contain all 120 submitted questions; every repaired outcome is retained.
- Merchant policy conflicts and gaps are retained. Product efficacy, review authenticity and downstream cart or refund completion were not independently established.
- The repaired captures have gaps in live resource telemetry. Terminal state and artifact hashes were verified; continuous resource observations are not claimed.
- Private cart, session or customer links are redacted from displayed evidence and quotations. Judgments used the unchanged original captures; these display redactions do not change scores.
How the scores were checked
Every assessed policy-resolution checkpoint receives a primary and a fresh blind audit. Both must award valid evidence-backed attainment. Attainment disagreements receive zero verified credit and remain visible.
Read the versioned methodologyWhat changed from the source study
- Replaces the source automation classifier with policy-compliant resolution.
- Every eligible checkpoint receives two fresh blind judgments; attainment requires agreement and valid evidence.
- Cause-selected capture repairs retained: 12. The source study remains unchanged.
Audit notes (7)
- Provider names were masked where practicable; full anonymity is not claimed.
- Retained original quality judgments have separate historical audit coverage. The repair-quality audit sees the primary quality verdicts and full captured replies; it is distinct from the blind policy-resolution audit.
- One initial scoring attempt failed in the response decoder with zero accepted decisions; upstream completion is unknown. The output was unavailable, and the documented infrastructure repair did not select among verdicts.
- The first blind policy-resolution audit exhausted its 8,000-token per-model-turn output allowance without a structured verdict. Its unusable output was preserved and the identical input was audited once more under the reviewed 16,384-token per-model-turn allowance.
- A later blind audit failed after about 602 seconds without a returned verdict. This is consistent with the original 600-second local timeout, but the inner error was not retained and provider completion is unknown. The unusable attempt was preserved and the identical input was retried once under a uniform 1,200-second future execution limit with a 1,250-second outer transport limit. All 36 previously successful stage outputs remain unchanged under their original timeout contracts.
- The exact successful initial primary judgment for 30 checkpoints is retained with an 8,000-token per-model-turn allowance. Every subsequent policy-resolution and repair-quality call uses 16,384 per model turn, with the same frozen prompts, model, effort and criteria. New calls use fresh independent sessions under reviewed pool releases with 2 and 4 active slots; 4 distinct isolated slots were used. A CLI session can contain multiple internal steps, so its total output can exceed the per-turn allowance.
- The original quality audit sampled only Alhena records. New policy-resolution audits cover both providers, but this does not retroactively expand historical quality-audit coverage.
Frozen method SHA-256 712b1e5753cd9b871881d328d79ecb20ccbb279c045fe9233891c6805441133b
Read every conversation behind these scores
Detailed decisions, source records and available study downloads require a verified work email. Your email and view or download activity are shared privately with Alhena. Privacy details.
Considering Alhena? See how its shopping and support agents would handle your customers’ questions.
Book a demo