k6 supports multiple execution models. The Hub Domains suite uses the The tool also collects standard HTTP metrics, including request duration, failed-request rate, and data transfer. Engineers can add tags to segment those metrics by stable request dimensions, such as the hostname and ID lookup paths used by the Hub Domains suite.<\/p>\n\n\n\n Because the test scripts, fixture configuration, thresholds, and output artifacts can be versioned together, k6 provides a practical foundation for repeatable performance testing in local development, CI pipelines, and approved non-production environments.<\/p>\n\n\n\n <\/p>\n\n\n\n A generic endpoint loop can provide an overall latency number, but it cannot explain which API behavior produced that number.<\/p>\n\n\n\n A repeated lookup for one popular hostname may follow a different cache or data-access path than lookups distributed across many records. A missing hostname has a valid 404 contract, not the same failure semantics as an unexpected server error. Collection requests also have a different query and response shape from individual-record lookups.<\/p>\n\n\n\n If these request types are mixed into one undifferentiated test, aggregate metrics can hide a regression in one path behind acceptable results in another. The Hub Domains k6 suite addresses that limitation by modeling each read contract independently and tagging hostname and ID traffic for comparison.<\/p>\n\n\n\n The goal is not to publish a single capacity benchmark. It is to create a repeatable harness that validates API contracts, highlights path-specific latency changes, and produces evidence engineers can compare across revisions and environments.<\/p>\n\n\n\n Each script encodes one request contract rather than looping over a generic endpoint.<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n This separation matters. A repeated lookup can exercise a different cache or data-access path than a varied lookup. An expected missing record also has a different correctness contract from an unexpected server error. <\/p>\n\n\n\n <\/p>\n\n\n\n The suite uses k6’s ramping-arrival-rate executor rather than a closed virtual-user loop.<\/p>\n\n\n\n A closed model measures how quickly a fixed set of virtual users can complete work. If the API slows down, that model naturally emits fewer requests. An arrival-rate model instead asks whether the service can sustain a defined incoming demand, even when response times change.<\/p>\n\n\n\n The shared profile currently uses these settings:<\/p>\n\n\n\n Each scenario ramps from zero to 20 requests per second over 30 seconds, holds that rate for two minutes, and ramps down over 30 seconds. Hostname and ID scenarios run concurrently, so lookup workloads target 20 request starts per second per lane, or 40 in total during the steady phase.<\/p>\n\n\n\n k6 preallocates virtual users for each scenario and can add more when required to maintain the arrival rate. <\/p>\n\n\n\n The framework assigns an identifier tag to each scenario and routes it to the appropriate execution function:<\/p>\n\n\n\n <\/p>\n\n\n\n This gives engineers separate hostname and ID metrics without adding a high-cardinality tag for every individual domain. The suite can therefore evaluate both global latency and tagged metrics such as:<\/p>\n\n\n\n The shared defaults evaluate a 95th percentile request duration below 1.5 seconds, a 99th percentile below 3 seconds, and a low failed-request rate. Individual workloads can override thresholds when they have more specific expectations.<\/p>\n\n\n\n Fast responses are not necessarily correct responses.<\/p>\n\n\n\n Known-record helpers validate HTTP 200 and confirm that the JSON response contains domain data. The missing-record workload explicitly validates HTTP 404. All shared requests use a consistent header set and a 60-second timeout.<\/p>\n\n\n\n Checks, thresholds, and HTTP failure metrics answer different questions:<\/p>\n\n\n\n A passing check does not, by itself, make an HTTP response \u201cexpected\u201d for k6’s built-in HTTP metrics, and a failed check does not necessarily fail the overall test unless a threshold enforces it.<\/p>\n\n\n\n That distinction is especially important for expected 404 responses. k6 normally counts a 404 as failed for http_req_failed, so the missing-record scenario relies on its explicit status is 404 check to validate the contract.<\/p>\n\n\n\n For teams using this approach as a strict CI gate, two useful hardening steps are to classify expected responses explicitly and threshold semantic checks:<\/p>\n\n\n\n A checks threshold ensures a contract violation fails the run. Monitoring dropped_iterations verifies that the load generator actually achieved the intended arrival profile.<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n The harness selects its configuration through an ENVIRONMENT variable and loads the matching environment data during k6 initialization. This keeps base URLs and fixture values out of the request logic.<\/p>\n\n\n\n A focused run looks like this:<\/p>\n\n\n\n The random workload selects a fixture value on each iteration. The repeated-known and missing-record workloads use designated values. The collection workload builds query parameters with the URL API, using a default limit and an optional page.<\/p>\n\n\n\n Environment-specific configuration makes the scripts reusable, but it does not eliminate the need for test-data stewardship. Fixtures should be safe to exercise, representative of the intended path, and reviewed as the service evolves.<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n Each run produces a terminal summary, an HTML report, and raw JSON data. The HTML report is useful for quick inspection, while the JSON artifact is better suited for CI retention and deeper comparison.<\/p>\n\n\n\n A useful comparison records the environment, application revision, fixture revision, k6 version, scenario, and arrival profile. Without that context, two results can look comparable while representing different conditions.<\/p>\n\n\n\n The actual load profile should come from the shared configuration and observed k6 metrics rather than hand-maintained summary labels. This keeps report descriptions aligned with the test that actually ran.<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n A meaningful performance signal must be reproducible by another engineer.<\/p>\n\n\n\n For automated use, pin the k6 version, retain reports as build artifacts, and version-pin or vendor reporting dependencies. The framework imports report utilities at runtime, so mutable external dependencies can change independently of the test code.<\/p>\n\n\n\n The same discipline applies to infrastructure. A non-production result is a comparative signal, not a production-capacity guarantee. Cache state, database size, network topology, competing workloads, and deployment configuration all affect the outcome.<\/p>\n\n\n\n Building the Hub Domains harness reinforced a few principles that apply to almost any API.<\/p>\n\n\n\n An endpoint can have several meaningful behaviors. For the Domains API, a known-record lookup, a repeated hot-record lookup, a missing-record lookup, and a collection query all exercise different contracts.<\/p>\n\n\n\n A useful load test begins by asking:<\/p>\n\n\n\n This produces scenarios that can explain a result. A generic request loop can report aggregate latency; contract-based scenarios can identify where behavior changed.<\/p>\n\n\n\n <\/p>\n\n\n\n A virtual-user loop is useful when modeling user workflows. An arrival-rate executor is more appropriate when the question is whether an API can sustain a defined incoming request rate.<\/p>\n\n\n\n The important lesson is to treat load configuration as part of the experiment. The requested rate, ramp profile, virtual-user allocation, and duration should be explicit, versioned, and recorded with the result.<\/p>\n\n\n\n Load generators also have limits. If k6 reports dropped iterations, the test client may not have generated the intended request rate. In that case, the run describes generator capacity as much as API behavior.<\/p>\n\n\n\n <\/p>\n\n\n\n Not every non-`200` response is a service failure. A `404` for a deliberately missing domain is the expected API contract; a timeout or unexpected `500` is not.<\/p>\n\n\n\n Tests should make that distinction explicit through response checks, expected-status classification, and thresholds. Otherwise, a valid negative-path test can appear unhealthy, or an incorrect response can be hidden in an aggregate success metric.<\/p>\n\n\n\n <\/p>\n\n\n\n A performance result without context is difficult to trust. Record the environment, application revision, fixture revision, k6 version, scenario, and load profile with the output.<\/p>\n\n\n\n Human-readable HTML reports support fast investigation, while raw JSON supports comparison and automation. Both are useful, but neither replaces a clear record of test conditions.<\/p>\n\n\n\n <\/p>\n\n\n\n Thresholds are not universal definitions of acceptable performance. They are explicit expectations for a known workload in a known environment.<\/p>\n\n\n\n They should be reviewed as the API, infrastructure, traffic profile, and customer expectations change. A threshold that is too loose fails to detect regressions; one that is too strict creates noise and teaches teams to ignore the signal.<\/p>\n\n\n\n The Hub Domains k6 suite is more than a collection of request loops. It models distinct API-read contracts, applies a consistent arrival-rate profile, separates hostname and ID metrics, validates response semantics, and produces artifacts that support engineering investigation.<\/p>\n\n\n\n The value of a load-testing harness is not the largest number it can produce. It is the confidence that a known workload can be reproduced, its API contract verified, and a change in behavior explained. By modeling Hub Domains read paths independently and preserving the context around every run, the k6 suite turns performance testing into repeatable engineering evidence.<\/p>\n\n\n\n The next step is to keep the suite aligned with real behavior: refresh fixtures, revisit traffic profiles, strengthen CI gates around semantic checks and dropped iterations, and preserve enough execution context to make every result explainable.<\/p>\n\n\n\n <\/p>\n","protected":false},"author":34,"featured_media":0,"parent":0,"menu_order":0,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[18,17],"tags":[118,90,116,119,120,117],"class_list":["post-2288","blog","type-blog","status-publish","format-standard","hentry","category-performance","category-testing-qa","tag-api-testing","tag-arcgis-hub","tag-ci-cd","tag-k6","tag-load-testing","tag-performance-testing"],"acf":[],"yoast_head":"\nramping-arrival-rate<\/code> executor, which starts iterations at a defined rate rather than asking a fixed number of virtual users to run as quickly as possible. This allows the suite to test whether the API can sustain a known incoming request profile.<\/p>\n\n\n\nWhy a single load test was not enough<\/h2>\n\n\n\n
Start with workload contracts<\/h2>\n\n\n\n
<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/10\/image-1024x328.png\")
<\/figure>\n\n\n\n
Each iteration performs exactly one HTTP GET request, so the configured iteration arrival rate directly represents the intended HTTP request-start rate for that scenario.<\/p>\n\n\n\nModel incoming demand with arrival rate<\/h2>\n\n\n\n
const TARGET_RPS = 20;\nconst RAMP_DURATION = '30s';\nconst STEADY_DURATION = '120s';\nconst RAMP_DOWN_DURATION = '30s';<\/code><\/pre><\/div>\n\n\n\n
Those virtual-user settings control the capacity of the k6 load generator. They should not be interpreted directly as the application’s concurrent-request count or as a service-capacity claim.<\/p>\n\n\n\nTag the paths you need to compare<\/h2>\n\n\n\n
tags: { identifier: tag },\n\nexec: tag === 'hostname' ? 'runByHostname' : 'runById'<\/code><\/pre><\/div>\n\n\n\nhttp_req_duration{identifier:hostname}\n\nhttp_req_duration{identifier:id}<\/code><\/pre><\/div>\n\n\n\nValidate response contracts as well as timing<\/h2>\n\n\n\n
\n
const expected404 = http.expectedStatuses(404);\n\nhttp.get(url, {\n ...requestParams(),\n responseCallback: expected404\n});\n\nthresholds: {\n checks: ['rate==1'],\n dropped_iterations: ['count==0']\n}<\/code><\/pre><\/div>\n\n\n\nKeep inputs environment-specific<\/h2>\n\n\n\n
cd load-testing\n\nENVIRONMENT=qa k6 run scripts\/random-domain.js\n<\/code><\/pre><\/div>\n\n\n\nGenerate evidence, not just console output<\/h2>\n\n\n\n
Make the harness reproducible in CI<\/h2>\n\n\n\n
Lessons Learned<\/h2>\n\n\n\n
Model contracts, not just endpoints<\/h3>\n\n\n\n
- Which request patterns matter to clients?\n- What does a correct response look like for each pattern?\n- Which paths are likely to have different cache, database, or payload characteristics?<\/code><\/pre><\/div>\n\n\n\nChoose a load model that matches the question<\/strong><\/h3>\n\n\n\n
Separate expected errors from failures<\/strong><\/h3>\n\n\n\n
Make every run reproducible<\/strong><\/h3>\n\n\n\n
Treat thresholds as engineering guardrails<\/strong><\/h3>\n\n\n\n
Conclusion<\/h2>\n\n\n\n