FAQ
Questions, answered without hedging.
Every question and answer on the site, collected on one page: how we work in general, each of the ten services, and the five process steps.
General
Do we have to give you a test environment?
No, but we strongly recommend it. If we have to test in production we exclude every scenario that writes data and work only with read-only checks — which narrows the scope considerably.
Do you fix the bugs you find?
No. We don't ship fixes, because that's what keeps us independent. We give you the finding, the cause and the reproduction steps; your team fixes it and we verify it in the re-test round.
Can you tell us how many bugs you'll find?
We can’t, and you shouldn’t believe anyone who does. What we do give you up front, in writing, is how many cases we’ll test, which devices we’ll use and which services are in scope.
Will there be a surprise invoice?
No. The price is fixed together with the scope. If something outside it comes up, we ask you first and don't continue without approval.
Do you need to see our source code?
Not for functional, compatibility or usability testing — we work black box. For performance bottleneck analysis and the security review, code access makes the results noticeably better, but it isn't required.
Do you sign an NDA?
Yes, as standard. We can send ours or sign yours. All test data is deleted 90 days after the run ends.
Do you write automated tests too?
On the Fleet package we automate the critical flows of the regression suite with Playwright. You own the tests; when the contract ends they stay in your repository.
How soon can you start?
You join the queue the moment you sign up, pick your criteria and pay. Thanks to the network, testing usually starts within 2 business days; for urgent launches we sometimes start the same day.
Services
Functional testing
Do you write the scenarios?
Yes. You give us the list of critical flows and any existing test documents; we write the scenarios and put them to you for approval.
Who provides the test data?
Test accounts and payment sandbox keys come from you. We generate the rest; we never work with live customer data.
What happens to a finding that cannot be reproduced?
It doesn't enter the report. Nothing goes on the list that the core team could not reproduce independently.
Regression testing
Who owns the automated tests?
You do. They are written in your repository and stay there when the contract ends; you are not locked in.
How long does the suite take to build?
Usually within five business days, during the first test. Later releases need maintenance, not a rebuild.
Does the whole suite run every release?
On major releases, yes. On small patches we run the affected areas plus the smoke test, and the report says which was chosen.
Compatibility matrix
Which devices do you cover?
Everything above one per cent share in your own analytics. The matrix is not final until you approve the list.
Do you use emulators?
Only for triage. Every reported finding has been confirmed on a real device.
Are very old devices included?
If they appear in your analytics, yes. If they do not, we leave them out of scope and say so in the report.
Performance & load
Do you run load tests against production?
No. Load testing happens only on staging or a separate environment; in production we take measurements only.
How far do you push concurrency?
To twice your target, or until the system breaks — whichever comes first. You set the target when you choose your criteria.
Are the results repeatable?
Yes. The k6 scripts ship with the report so you can run the same test yourself.
Security review
Does this replace a penetration test?
No. It targets the common and expensive mistakes in the application layer. If regulation requires a pen test report, we will point you to an accredited firm.
Do you need the source code?
Not required — we can work black box. With code access the results improve noticeably, especially on permission checks.
Do you exploit what you find?
Only far enough to prove it exists. We do not exfiltrate data, do not leave changes behind, and log every attempt.
API testing
What if we have no schema?
That works. We build the inventory from a collection or from traffic; contract validation, however, needs a schema to exist.
Will rate limit testing affect production?
We work in a test environment. If production is unavoidable, rate limit and concurrency tests are left out of scope.
Do you support GraphQL?
Yes. Query depth, field-level permissions and N+1 behaviour are checked separately for GraphQL.
Payment testing
Is real money spent?
Yes, in small amounts. A sandbox cannot imitate a real bank decline, the 3-D Secure flow or how long a refund takes. Test charges are listed in the scope approval and refunded when the engagement ends.
Do you provide the cards?
We use the network’s test cards. If a method can only be opened at your own bank — a corporate card, a country-specific wallet — we ask you for that one.
Do you work against production?
If payments are genuinely to be tested, yes, against production. We place orders from flagged test accounts, log every transaction, and cancel or refund all of them at the end.
Accessibility
Which standard do you work to?
WCAG 2.2 AA. On request we also report AAA criteria in a separate section.
We already run an automated tool — what is different?
Automated tools catch measurable violations. Focus order, whether announcements make sense and keyboard traps are only found by hand, and most findings come from there.
Are mobile apps covered?
Yes. The same flows are walked with VoiceOver on iOS and TalkBack on Android.
Usability review
Is this user testing?
No. This is a heuristic review by experienced evaluators. If you want testing with real users we plan it separately and say so plainly.
Do you also design?
No. We say what the problem is and what could be done; your designer draws the screen.
Are the findings measurable?
Step counts and field counts are measurable, and we give them before and after. Conversion impact can only be measured once you ship.
Localization testing
How many languages are covered?
Two languages per package as standard; further languages are priced in the scope approval. We name the languages we have native speakers for in that approval — we do not promise one we cannot staff.
Do you do the translation?
No. We test how the translation you already have behaves inside the product. If we find a bad one we report it; your translation team writes the replacement.
Are right-to-left languages covered?
Yes. For Arabic and Hebrew, direction, alignment, icon mirroring and mixed-direction text form their own group of checks.
Process steps
Sign-up and scope
Do I have to pay to sign up?
No. Signing up is free; you can pick criteria, see the price and stop there. Payment is taken only when you confirm the scope.
What if I do not know what to choose?
Take one of the ready-made packages — each says what it covers. If you are unsure, Release is the right starting point for most products.
Payment and test plan
Can I change the scope after paying?
Shrinking it is always possible and the difference is refunded. To grow it you add line items in the panel; nothing proceeds until the price and date impact is approved.
What if I cannot provide a test environment?
If we must work in production, scenarios that write data drop out of scope — and you see that plainly at plan approval.
Execution
Can we watch the testing live?
Yes. The board link is shared on day one; you see scenario status and incoming findings as they happen.
What if the environment goes down?
We stop and tell you. The lost time is added to the delivery date and there is no extra charge.
Report
Can we dispute the ranking?
You can, and it happens often. Once we hear your context we change the level, and the reason for the change stays in the report.
How long does the report stay available?
Ninety days. After that the link closes and the data is deleted; if you want an archive, the PDF stays with you.
Re-test round
How many re-test rounds are there?
One on Scout, two on Release, unlimited on Fleet. There is no time limit between rounds.
Is there a charge if new findings appear?
No. Side-effect findings from a re-test go into the same report and do not count as scope growth.
If your question isn’t answered here, write to us and we’ll add it.
Don't test the next release alongside your users.
Sign up in the panel, pick what you want tested, pay. The first findings start landing in the same panel within hours.