Join Product Garage
Join Product Garage

Methodology appendix 1 November 2026

Attio: Methodology Appendix

A proposal to test whether a task-backed handoff improves timely, appropriate responses.

This appendix supports the working note "Attio: which first workflow makes a flexible CRM feel essential?". It is a proposal, not a report of a test that has been run. No sample size, observation window, uplift target or deadline is specified, because the sources give no baseline to set them from.

The workflow under test: request, owner, verified next step

The segment is a small B2B team new to Attio for sales operations, with recurring warm enquiries from one existing source and a person responsible for intake. At least one handler is someone other than whoever builds the CRM. Customer support, cold prospecting and broad product sign-up streams are out of scope.

  1. Observe the request. Keep the source identity and original arrival time. Reconcile source events with CRM arrivals so a failed integration does not hide a request.
  2. Represent it once. Use the team's existing structure. Tell a genuine repeat from a duplicate event, and do not create an opportunity for every imported contact.
  3. Assign responsibility. Use an agreed rule and a deadline suited to service hours. Anything without an owner goes to a named intake operator.
  4. Show the context. Include the request and permitted account information. Unknown stays unknown, and the handler decides the response.
  5. Act and record the disposition. Keep evidence of the reply, a legitimate disqualification or an accepted transfer.
  6. Review the exceptions. Someone owns failures, stale tasks and changes to the rule.

This uses existing Attio building blocks. It needs no new inbox, no AI qualification engine and no automatic customer messages.

Conditions held equal

  • Teams: consenting teams that fit the segment above.
  • Eligibility screen: source, permissions, ownership and response policy already work, and the team does not yet use automatic task-backed handoffs for this job. Teams that fail the screen are excluded before assignment and reported, with reasons, as a feasibility finding. They are not outcome data.
  • Setup failures after assignment: a team that is assigned and then fails to complete setup stays in its assigned group. Its outcomes, effort and costs remain in the main analysis and the cost accounting, and the failure is also reported as an implementation finding. Poor integrations are not quietly fixed for treatment teams only.
  • Unit of work: one source request, not one imported record. Genuine repeat requests are separated from duplicate events, and the source identifier and original arrival time are kept. Source events are reconciled with CRM arrivals, so a failed integration cannot remove a request from the evidence.
  • Shared setup: the same source data, qualification policy, response standard, ownership rule, record views, permissions, training time, support and exception review. Missing ownership goes to a named intake operator in both groups.
  • Nothing bundled: no better enrichment, AI scoring, Slack campaign or automatic customer message. Notification settings are configured consistently across both groups. The extra assignment notification created by the treatment task is an intended part of the mechanism, so it is recorded and not erased. Actual notifications and manual tasks are logged.
  • The one difference: control shows the request, handler, deadline and next-action instruction in the existing inbound list or view. Treatment does the same and also creates one linked Attio task carrying that instruction, owner and deadline, with its ordinary task surface and assignment notification.

Outcome and scoring

  • Primary outcome: the share of all eligible source requests in the observation window that receive a verified, appropriate first disposition by a predeclared deadline. The clock starts at source arrival, and the rubric states how business hours, holidays and requests awaiting information are handled.
  • Appropriate disposition: a useful reply, a clarifying question, a justified disqualification or an accepted transfer, with evidence it happened. An empty acknowledgement, a ticked task or an unaccepted transfer does not count.
  • Rubric and assessors: written before assignment. Assessors are blind to group where feasible. A second assessor rates a sample to check agreement.
  • Limit of the inference: the test estimates the added value of a task-backed handoff for this segment and job. It does not isolate the notification from the task, show the value of the base CRM, compare inbound with other jobs, or show any effect on revenue.

Measures

MeasurePurposeQualification
Verified timely dispositions / eligible source requestsPrimary test of handling real demandA request confirmed unanswered by its deadline counts as a failure. Freeze exclusions before assignment
Time to appropriate dispositionShow where delay changesKeep unresolved requests with censoring. Do not average only completed ones
Wrong routing, duplicate contact, unsuitable responseCheck that speed does not displace judgementAssess rates and severity against predeclared limits, not counts alone
Setup, handling, support and correction effortExpose work moving to the builder or reviewerCount all assigned teams, including setups that fail after assignment. Screening failures go in the feasibility report. Time estimates are not audited savings
Workflow failures, task backlog and creditsCheck operational and economic feasibilityRun success and completed tasks are mechanism measures, not outcomes
Repeated handling after support endsAssess whether the process can continueSeparate no demand from ignored demand. Inspect real work by a second handler
Paid scope and retained useObserve the commercial consequence separatelyA trial conversion is not proof of causal revenue lift or dependence

Design notes

  • Randomise at workspace or independent intake-team level. Handlers who share a queue stay in the same cluster. Stratify or match on baseline request volume, response performance and team capacity.
  • Predeclare the estimand. Decide whether the main analysis weights requests or workspaces, since large queues otherwise dominate an apparently team-level claim. Report both where practical. A workspace with no eligible demand has no handling rate. Keep it in the assigned cohort and report the absence of demand, and do not quietly drop it from adoption reporting.
  • Qualitative first. Observe a few teams to check comprehension, permissions and the exception path. Use baseline variation and clustering to size a later controlled test. Do not invent a sample size or an uplift target. Choose the window from the team's real work cadence and the smallest worthwhile change from operational value.
  • Analyse as assigned. Teams that disable the task, change plan or stop using the process stay in their original group, and exposure is reported separately. Manual workarounds are recorded, not prohibited. Do not adjust away post-assignment workload or effort in the main policy analysis.
  • Missing evidence. Two cases are kept apart. A request confirmed unanswered by its deadline fails the primary outcome. A request whose outcome cannot be assessed, for example because the record or source evidence is unavailable, is missing and is neither a success nor a failure. Keep both in the assigned cohort, report missingness by group and apply the predeclared sensitivity analyses, such as best-case, worst-case and plausible assumptions about the missing requests. Assess each request only after its own deadline has passed. A result restricted to teams with complete records cannot support a claim about all assigned teams.
  • Success rule. Expand only if verified handling improves by a practically worthwhile amount, response quality stays within the agreed limits, and the extra notifications, effort and upkeep, counted across every assigned team, do not erase the gain.
  • Serious events. If a serious misrouting or disclosure occurs, pause that implementation, record the event and the exposure, and keep the original assignment for analysis. A small pilot cannot show the absence of rare serious errors.
  • Read the pattern. More completed tasks with unchanged verified work means no demonstrated operating benefit. Faster handling with worse responses fails the quality condition. Strong initial use that needs the facilitator to keep chasing people has not shown repeatability. Similar outcomes may mean the base queue is sufficient, a ceiling effect, poor implementation or too little evidence. None proves equivalence. If either group already handles requests almost perfectly, a task has little room to help.
  • Maintenance checks. Observe an ordinary update to the routing rule and a replacement handler in both groups if feasible. These are supporting checks and not extra treatment features. Several successful cycles are early evidence and do not establish long-term retention. Do not remove a working customer process to create a dependency signal.

Implementation checks before real demand

These are checks to run, not defects reported in Attio.

  • Configure deliberately. In the Create task block only the task text is required. Due date, linked record and assignee are optional, and an empty optional input produces a task without that value.
  • Test the awkward cases. Duplicate delivery, replay, an owner change and an already-resolved request.
  • Do not assume a retry undoes earlier effects. Attio's troubleshooting page describes retrying from the start, from the failed block or with the live workflow, and says cancelling leaves changes already in flight. It does not say earlier effects are reversed.
  • Permissions. Attio's Sharing and permissions page documents tasks as visible to and editable by everyone, and its email sync page says email bodies are private by default. Keep confidential email text out of task descriptions, and check who can edit a task before treating a ticked task as evidence.
  • Confirm the live behaviour of the trigger, identity mapping, task linkage, notifications and exception path on the team's actual plan before treating the setup as comparable across teams.

Measurement traps

  • Contact count or mailbox connected: shows data arriving, not work handled.
  • Number of objects or fields: shows modelling effort, not use.
  • Workflow run success: shows blocks executed. A run can succeed and a request still go unanswered.
  • Task completion: a tick is not a reply. Tasks are editable by everyone in the workspace, so it can be marked by someone other than the person who acted.
  • Average time to respond among completed requests: hides the requests nobody answered.
  • Daily logins: a team can rely on shared records without opening the app, as one customer account describes.
  • Credits consumed: a cost input, not value delivered.
  • Customer-reported savings: vendor-published and unaudited. The Granola figure is not a baseline for another team.
  • Trial conversion: if tracked, a secondary commercial outcome. This test cannot establish a causal effect on paid expansion.
← Back to the working noteJoin Product Garage →