Calendly: Methodology Appendix

This appendix supports the working note "Calendly lead routing: when does a booking link become a team purchase?". It is a proposal, not a report of a test that has been run. No sample size, observation window, uplift target or waiting interval is specified, because the sources give no baseline to set them from.

The question and what it covers

Does applying existing HubSpot ownership before calendar selection improve the rate of appropriate sales meetings actually held, compared with a pooled booking path followed by the team's usual manual reassignment? The test covers one routing configuration. It does not cover Calendly as a whole, the value of qualification questions, or the causal effect of buying a higher plan.

Segment and screen

• Segment: consenting B2B sales teams already using Calendly and HubSpot, with recurring inbound demo requests, an agreed ownership and delegation policy, a meaningful flow of known contacts and an existing segment-based shared pool with manual reassignment where necessary. The pool is an accepted existing process.
• Exclusions: teams already using the same owner-first setup. No team is asked to create deliberate ownership violations for the experiment.
• Audit before assignment: trace one representative request through the live system: form submission, CRM match, routing decision, calendar displayed, booking, record update and the meeting itself. Include a valid new contact, a known owner, an unavailable owner, a stale or duplicate record and someone who never books. These are proposed test cases, not reported Calendly defects.
• Questions the audit answers:
  • Authority: which source of ownership is approved, and what is the rule for delegation?
  • Timing: is the owner already present when routing reads it? A later assignment is not a booking-time decision.
  • Calendar: what can the buyer see, given useful slots, booking horizon, minimum notice and the assigned event? A rule mismatch differs from a calendar with no suitable time.
  • Off the happy path: what happens with unmatched records, invalid owner mapping, no booking and technical failure?
  • Evidence: can source requests be joined to routing responses, bookings, reschedules and attendance without counting duplicate events as new leads?
• Feasibility versus assigned-group analysis: teams that fail the screen are excluded before assignment and reported, with reasons, in a feasibility report. They are not outcome data. A team that is assigned and then fails to complete setup, turns routing off or loses a calendar connection stays in its assigned group. Its outcomes, effort and costs remain in the main analysis and the cost accounting, and the failure is also reported as an implementation finding.

Conditions held equal

• Shared setup: the same form, qualification rules, admission criteria, source traffic, event purpose and duration, underlying host calendars, reminders, human recovery, training and support. No enrichment, discounts, new qualification score, extra staff or no-show campaign.
• Control: the current form, qualification rules and segment-based pool, with the usual manual owner handoff and assistance available today.
• Treatment: the CRM ownership route is inserted before that same pool route, which is the order Calendly's HubSpot lookup guide recommends. Known owned contacts see the existing owner's Managed Event, and other requests keep their existing path.
• What is allowed to differ: the bookable options shown can differ because of the route. Those differences are part of the effect and are not adjusted away.

Cohort and outcome

• Primary cohort: all valid demo enquiries in the agreed window that qualify under frozen business rules and have a known owner under the approved policy at submission. A pre-routing CRM snapshot is captured in both groups. Membership is not defined using the treatment's successful matches or the final owner written after booking.
• Kept in the cohort: failed lookups and eligible people who never see or book a slot. New or unowned requests and the complete form population are reported separately.
• Primary outcome: the share of cohort enquiries that result in an attended, appropriate first sales meeting with the owner or a policy-approved delegate within a predeclared interval from submission. The attendance, qualification, delegation and repeat-enquiry rules are written before assignment. Each request is allowed its complete follow-up interval, including requests arriving near the end of the observation window. A booked slot or a correct lookup alone does not count.
• Attendance evidence: Calendly lets hosts mark no-shows by hand after the meeting's start time, and the status appears in the meetings export. The page does not say that an unmarked meeting was attended, so attendance is verified through consistently collected host or meeting evidence. Calendly's own reporting is used to join the evidence, and no new dashboard is built as the test.
• Rubric and assessors: written before assignment. Assessors are blind to group where feasible, and a second assessor rates a sample to check agreement.

Design and analysis

• Randomise independent teams or calendar pools. Shared hosts stay in one cluster. Two policies competing for the same scarce slots interfere with each other, so a visitor-level split cannot be read as the full rollout effect.
• Qualitative first. Observe a few teams and record a baseline. Use baseline variation and the available clusters to size a later controlled study for a worthwhile operational effect. No sample size, uplift target or universal waiting window is inferred from the sources.
• Analyse as assigned. Report actual exposure separately. Track repeated requests by account, use cluster-aware uncertainty, and predeclare whether the main analysis weights requests or teams.
• Missing evidence. Two cases are kept apart. A request confirmed to have had no meeting by its deadline fails the primary outcome. A request whose outcome cannot be assessed, for example because attendance evidence is missing, is missing and is neither a success nor a failure. Keep both in the assigned cohort, report missingness by group and apply predeclared sensitivity analyses. Assess each request only after its own deadline has passed. A team with no qualifying demand stays in the cohort report without an invented conversion rate.
• Whole-pool check. Improvements for known contacts that displace new prospects need whole-pool assessment before any rollout.
• Expansion rule. Recommend wider use only when appropriate held-meeting outcomes improve by a worthwhile amount, or a predeclared maintenance-saving case is supported without an unacceptable loss in those outcomes. Set acceptable quality, delay and workload limits before assignment. An equivalence conclusion needs a design capable of supporting it, and a small null result is not proof of equivalence. Conclusions must survive reasonable missing-data assumptions.
• Serious events. A serious misroute or disclosure pauses the affected implementation. Record the event and the exposure, and keep the original assignment for analysis. A small pilot cannot show the absence of rare serious errors.

Measures

Implementation gate

These are checks to run before real demand, not defects reported in Calendly.

• Test the awkward cases: an owner with no suitable slots, an inactive host, mismatched user email, a duplicate CRM record, a booking cancellation and a reschedule.
• Confirm the real fallback behaviour and recovery route. Calendly's HubSpot lookup guide does not say what happens when no match is found or the owner has no availability.
• Do not change other policies. If the account cannot implement the contrast without changing other policies, redesign the test or do not run it.
• What the gate limits. It limits execution. It does not prevent publication of a clearly labelled proposal.

Maintenance checks

Observe ordinary owner changes and an event-template update in both groups where feasible. These are supporting checks and not extra treatment features. Several good weeks can justify further evaluation. They do not establish durable retention or a causal revenue lift.

Measurement traps

• Form submissions or a CRM record created: shows demand arriving, not a meeting handled.
• Correct lookup: shows the route worked, not that anyone attended.
• Booking conversion: a booking is not attendance.
• Show rate among bookers: selected after treatment, so it cannot stand in for the enquiry-level outcome.
• Equal meeting counts: not equal deal value, preparation work or account complexity.
• Fewer reassignments: may reflect a smaller calendar and a longer wait.
• Seat growth: a cost and scope signal, not evidence of value.
• Vendor-reported figures: selected, undenominated and not a baseline for another team.