Transparency
Sources register
Every source is recorded with its robots policy, its terms position and the decision taken, with a date and a name. Crawl-despite-disallow decisions are made here and in the PRD, never inside a pipeline.
Attributions
- Vendor names and domains include data from SaaSHub under CC BY-SA 4.0, fetched through the SaaSHub API.
- Trust-centre discovery includes the TrustLists dataset (Apache-2.0).
Registry data is reproduced for identification of the listed organisation and links to the official registry in every case.
| Source | Tier | Robots | Terms | Decision |
|---|---|---|---|---|
ANSSI qualified products and services catalogue cyber.gouv.fr/produits-services-qualifiesmonthly | 1 | public PDF | Public catalogue | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
CSA STAR registry cloudsecurityalliance.org/star/registryweekly | 1 | /download/ and /special/ disallowed | Public registry pages | Crawled despite robots.txt (recorded decision) Solomon Amos, 16 Sep 2026 Listing and entry pages are allowed; /download/ artefacts captured under the recorded owner decision (PRD 8.2, 11.4, open question 10). |
ESAs list of designated DORA critical ICT providers www.eiopa.europa.eu/monthly | 1 | public PDF | Public designation list | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
EU Cloud Code of Conduct adherents eucoc.cloud/monthly | 1 | public | Public list | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
EU-US Data Privacy Framework participants dpfapi.azurewebsites.net/api/Participants/weekly | 1 | robots allows all | US government public list | Allowed Solomon Amos, 16 Sep 2026 Per-ID enumeration 1..~10,400; filemodified endpoint gives the refresh timestamp. |
FedRAMP Marketplace data raw.githubusercontent.com/FedRAMP/marketplace-fedaily | 1 | public data file on GitHub | US government open data | Allowed Solomon Amos, 16 Sep 2026 Changelog fedramp-status-changelog.json is the freshness feed. |
France HDS certified hosts esante.gouv.fr/offres-services/hds/liste-des-hebmonthly | 1 | 403 to plain HTTP; browser session works | Public list | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
GovRAMP program participants govramp.org/program-participantsweekly | 1 | server HTML | Public list | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
Japan ISMAP cloud service list www.ismap.go.jp/csm?id=cloud_service_listmonthly | 1 | client-rendered | Public list | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
Mastercard SDP compliant service provider list www.mastercard.com/content/dam/mccom/shared/busimonthly | 1 | Akamai 403 to plain HTTP; browser session works | Public PDF | Allowed Solomon Amos, 16 Sep 2026 · not yet running Runs on the ETL VM with Playwright; snapshots diffed month to month. |
NHS DSPT organisation search www.dsptoolkit.nhs.uk/OrganisationSearch/ExportSmonthly | 1 | robots allows /OrganisationSearch | Public bulk CSV | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
Spain ENS certified systems gobernanza.ccn-cert.cni.es/certificadosmonthly | 1 | public | Public directory | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
TX-RAMP certified cloud products dir.texas.gov/weekly | 1 | Cloudflare 403 from Hetzner | Texas DIR public export | Allowed Solomon Amos, 16 Sep 2026 · not yet running Rediscover the monthly XLSX link from the landing page; dedupe on certification ID. |
Visa Global Registry of Service Providers www.visa.com/splisting/api/searchGrspweekly | 1 | SPA API, unauthenticated | Public registry; production use owner-approved 16 Sep 2026 (PRD 11.0) | Allowed Solomon Amos, 16 Sep 2026 Only validationType PCI DSS with a validationDate counts as PCI evidence. |
A-LIGN ISO certificate lookup a-lign.com/iso-certificateon_claim | 1b | allows AI crawlers | Public lookup | Lookup only Solomon Amos, 16 Sep 2026 · not yet running |
Coalfire ISO client directory www.coalfirecertification.com/client-directory.hmonthly | 1b | /data/ and /Certificates/ disallowed | Directory page is public | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
DNV certificate checker certificatechecker.dnv.com/never | 1b | Disallow: / | Not fetched | Excluded Solomon Amos, 16 Sep 2026 · not yet running |
IAF CertSearch www.iafcertsearch.org/on_claim | 1b | /certified-entity/ and /certification/ disallowed; search behind reCAPTCHA | Terms prohibit robots and offline tools; bulk via paid Certification Data API | Lookup only Solomon Amos, 16 Sep 2026 · not yet running Bulk ISO data is a Phase 1 budget line (open question 3). |
IASME Cyber Essentials certificate search iasme.co.uk/cyber-essentials/ncsc-certificate-seon_claim | 1b | reCAPTCHA Enterprise | Solely for checking certification; not for data research | Lookup only Solomon Amos, 16 Sep 2026 · not yet running Never enumerated. Expired certificates vanish, so lapses are detected across snapshots of our own lookups. |
Schellman certificate directory www.schellman.com/certificate-directoryon_claim | 1b | lookup by certificate number | Public lookup | Lookup only Solomon Amos, 16 Sep 2026 · not yet running |
UKAS CertCheck certcheck.ukas.com/on_claim | 1b | same platform and Disallow rules as IAF CertSearch | Bulk by arrangement | Lookup only Solomon Amos, 16 Sep 2026 · not yet running |
AlternativeTo alternativeto.net/never | 2 | terms prohibit scraping beyond robots.txt | Prohibits building a competing database | Excluded Solomon Amos, 16 Sep 2026 · not yet running |
SaaSHub www.saashub.com/site/apimonthly | 2 | documented API, key required | CC BY-SA 4.0 with on-site API disclosure; ShareAlike scope owner-approved (open question 11) | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
SEC EDGAR company submissions data.sec.gov/submissions/weekly | 2 | descriptive User-Agent, 10 requests a second | US government open data | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
TrustLists trust-centre dataset raw.githubusercontent.com/trustlists/trustlists-monthly | 2 | Apache-2.0 dataset on GitHub | Licence asks for a dofollow attribution link | Allowed Solomon Amos, 16 Sep 2026 |
UpGuard security reports www.upguard.com/security-report/monthly | 2 | open; only tracking parameters excluded | Public pages | Allowed Solomon Amos, 16 Sep 2026 · not yet running Grade used only as a third-party reported security signal, never as an attestation. |
Y Combinator companies (yc-oss mirror) yc-oss.github.io/api/companies/all.jsonmonthly | 2 | public JSON | Open mirror | Allowed Solomon Amos, 16 Sep 2026 Source of the V1 one-liner for YC vendors. |
Conveyor trust centres app.conveyor.com/profile/30d top, 90d tail | 3 | permissive | Public vendor pages | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
Drata / SafeBase trust centres trust.cname.drata.com30d top, 90d tail | 3 | allows /api/share/; server-rendered HTML | Public vendor pages | Allowed Solomon Amos, 16 Sep 2026 |
Oneleet trust centres trust.oneleet.com90d | 3 | unresearched | Public vendor pages | Pending review Solomon Amos, 16 Sep 2026 · not yet running |
Secureframe trust centres secureframetrust.com30d top, 90d tail | 3 | server-rendered HTML | Public vendor pages | Allowed Solomon Amos, 16 Sep 2026 |
Vanta trust centres (vendor subdomains) cname.vantatrust.com30d top, 90d tail | 3 | no robots.txt on vendor subdomains; app.vanta.com never fetched | Public GraphQL replayed exactly as the page uses it; analytics mutations never called (owner-approved, PRD 11.0) | Allowed Solomon Amos, 16 Sep 2026 |
Whistic customer-hosted trust centres whistic.com90d | 3 | directory gated; customer subdomains public | Public vendor pages | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
disclose.io bug bounty dataset raw.githubusercontent.com/disclose/diodb/master/monthly | 4 | CC0 | Public domain | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
Statuspage.io public status API /api/v2/status.jsonmonthly | 4 | public JSON | Public | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
Vendor first-party security and legal pages 30d top, 90d tail | 4 | CertReportsBot honours robots.txt on first-party pages | Public pages, single-digit requests per minute per host | Allowed Solomon Amos, 16 Sep 2026 |
Press releases and news RSS news.google.com/rss/searchdaily | 5 | RSS | Public feeds | Allowed Solomon Amos, 16 Sep 2026 · not yet running |
Vendor claim submissions on_submission | 6 | n/a | Submitter licence permits extraction and publication of facts only | Allowed Solomon Amos, 16 Sep 2026 |
Wayback Machine Save Page Now 2 web.archive.org/saveper fetch | provenance | authenticated 7/min, 30,000/day | Public archive | Allowed Solomon Amos, 16 Sep 2026 · not yet running |