Evaluate parallel retrieval against the same questions and evidence requirements used for the sequential run. Concurrency changes scheduling; it does not automatically make sources more independent or answers more accurate. Report coverage, duplicates, failures and supported answers separately.
Hold the research task fixed
| Controlled input | Reason |
|---|---|
| Question set | Keep the task comparable |
| Permitted source set and source versions | Avoid source changes explaining the result |
| Total query/read budget | Keep the opportunity to obtain evidence comparable |
| Selection and support rules | Use the same acceptance criterion |
| Model/configuration if answers are tested | Separate generation changes from scheduling |
Measure evidence before celebrating duration
Retain unique supporting sources, copied passages, unavailable reads and unresolved claims for each run. If one run obtains more independent supporting evidence, identify which selection or scheduling behavior caused that difference. Do not treat duplicated URLs as improved coverage.
A bounded parallel run can return partial output. Report that outcome even if its duration is shorter. Compare duration only alongside the work completed and the evidence accepted.
Keep the conclusions within the experiment
The three-worker partial-failure test validates local scheduling behavior. This page supplies a controlled evaluation design; it does not report an unperformed accuracy benchmark.
- Use dated run logs and fixed input manifests.
- Report incomplete and unsupported cases, not only successful answers.
- Treat a faster local fixture as scheduling evidence, not a whole-provider benchmark.
- Distinguish the general technique from the Parallel.ai provider brand and SearchCans account lanes.
Reference documentation: www.searchcans.com — /google-search-api; www.searchcans.com — /reader-api.
