Overview
Upcake enables assertions to be performed against HTTP requests. Requests and their assertions are defined in a YAML file and can have dependencies on each other, provided that dependency requests are named. Assertions can be against request timing data and any response data including headers, content and response code.
The request URL, headers, headers_template and content support template rendering with Handlebars syntax.
Requests are run in parallel except where they have a dependency to another request in which case they will wait until their dependencies have completed before starting.
The response headers and content of a request is made available as part of the template context for dependent requests.
Installation
With cargo
From source
Usage
Command-line options
--env-var-prefix
- An optional prefix to filter environment variables injected into the template context-k
,--insecure
- Allow insecure server connections when using SSL-L
,--location
- Follow redirects-v
,--verbose
- Verbose output-s
,--max-response-size
- Maximum response size in bytes-e
,--extra-vars
Set additional variables as key=value or YAML. To use a file, prepend the value with "@". Available in the template context in theuser
property.--connect-timeout
- Maximum time allowed for connection in milliseconds-c
,--config-file
- Path to the request config file. Defaults to "Upcakefile.yaml". Required if--url
is not set.-u
,--url
- Run default assertions against a URL. Will use the default request config. Required if--config-file
is not set.-X
,--request-method
- Specify request method to use. Used in conjunction with--url
. Defaults to "GET".-H
,--header
- Pass custom headers to server. Used in conjunction with--url
.--status-code
- Verify the response code. Used in conjunction with--url
. Defaults to 200.
Note: Configuration set from the command line will override configuration for that property set in the Upcakefile.
Configuration
location
- Follow redirects. Overridden by--insecure
.insecure
- Allow insecure server connections when using SSL. Overridden by--insecure
.connect_timeout
- Maximum time allowed for connection. Overridden by--connect-timeout
.extra_vars
- Set additional variables from YAML mapping. Merged with vars set with--extra-vars
. Available in the template context in theuser
property.verbose
- Verbose output. Overridden by--verbose
.max_response_size
- Maximum response size in bytes. Overridden by--max-response-size
.requests
- List of request configurations.
Example
location: false
insecure: false
connect_timeout: 100
extra_vars:
my_var: Some Value
verbose: true
max_response_size: 32000
requests:
- url: "http://localhost:8888/post"
Request configuration
name
optional - Name of the request.requires
optional - List of named requests this request depends upon.request_method
optional - The HTTP method to use. Defaults to "GET".data
optional - Data to send with the request. Send the contents of a file by prefixing the value with an "@", for example "@path/to/body/template.hbs". Relative paths are relative to the directory of the loaded configuration file.headers
_optional_ - Mapping of headers to be sent.headers_template
optional - Render raw headers from a template.url
- The URL to make the request to.assertions
optional - A list of assertions to perform on the response. Defaults to a HTTP 200 assertion.
Example
- name: "Request B"
requires:
request_method: "POST"
data: "@data.hbs"
headers:
- name: Accept
value: application/json
- name: Content-Type
value: application/json
headers_template: |
{{#each requests.[A].headers}}
{{#if (eq name "Set-Cookie")}}
Cookie: {{value}}
{{/if}}
{{/each}}
url: "http://localhost:8888/post"
assertions:
- type: equal
path: ."response_code"
value: 200
Assertion configuration
type
- The type of assertion to use. See available assertions.path
- The jql path to the field the assertion should run against. Defaults to.
.path
is ignored on inner assertions, for example the length assertion.skip
optional - Whether to skip the assertion. If set, requires a string value for the reason.assertion
- The assertion to apply
Example
- type: length
path: ."content"."slideshow"."slides".[]
skip: Some reason for skipping the assertion
assertion:
type: equal
value: 2
Available assertions
In addition to the top-level assertion configuartion, each assertion has its own properties which are required to be set.
Between
Type: between
Assert that a value is within a range.
min
- Start of range.max
- End of range.inclusive
optional - Whether to includemin
andmax
in the assertion.
Example
- type: between
path: ."response_code"
min: 200
max: 399
inclusive: true
Equal
Type: equal
Assert that a value equals the given value.
value
- Value to assert.
Example
- type: equal
path: ."response_code"
value: 200
Not Equal
Type: not-equal
Assert that a value is not equal to the given value.
value
- Value to assert.
Example
- type: not-equal
path: ."response_code"
value: 204
Length
Type: length
Assert that the length of a value passes the given assertion
assertion
- Any assertion with the same configuration defined in assertion configuration.
Example
- type: length
path: ."headers".[]
assertion:
- type: equal
value: 5
Contains
Type: contains
Assert that a response value contains the given value.
-
For strings, it asserts that the substring is present in the value;
-
For arrays, it asserts that the value is present in the array;
-
For mappings (dictionary/object types), asserts that the input map is present in the response map.
-
value
- The value to assert is contained in the given value.
Examples
Assert mapping contains all key/value pairs
- type: contains
path: ."content".{}
value:
key: Value
another_property: Some other value
Assert array contains a value
- type: contains
path: ."content"."my_integer_array".[]
value: 10
or
- type: contains
path: ."content"."my_object_array".[]
value:
id: item_10
value: Item Value
Assert substring appears in response value
- type: contains
path: ."content"."my_string"
value: "value"
Exists
Type: exists
Assert that the given value exists as a key in the response value.
value
- The value to assert exists in the given mapping.
Example
- type: exists
path: ."content"."my_object".{}
value: id
Greater than
Type: greater-than
Assert that a value is greater than the given value.
value
- Value to assert.
Example
- type: greater-than
path: ."response_code"
value: 200
Greater than equal
Type: greater-than-equal
Assert that a value is greater or equal to the given value.
value
- Value to assert.
Example
- type: greater-than-equal
path: ."response_code"
value: 200
Less than
Type: less-than
Assert that a value is less than the given value
value
- Value to assert.
Example
- type: less-than
path: ."response_code"
value: 400
Less than equal
Type: less-than-equal
Assert that a value is less than or equal to the given value
value
- Value to assert.
Example
- type: less-than-equal
path: ."timing"."starttransfer"
value: 100
Response data
Request result
http_version
- The HTTP version used for the request.response_code
- The HTTP response code returned.response_message
- A HTTP response message returned from the host, if any.headers
- A list of response headers.timing
- Response timing data. See timing results.content
- Response content, either formatted as JSON, or raw content if it couldn't be parsed as JSON.
Timing results
namelookup
- Duration in milliseconds from the start of the request until name lookup resolved.connect
- Duration in milliseconds from the start of the request until a connection to the remote host is established.pretransfer
- Duration in milliseconds from the start of the request until file transfer was about to begin.starttransfer
- Duration in milliseconds from the start of the request until the first byte was received. AKA TTFB.total
- Duration in milliseconds from the start of the request until the request ended.dns_resolution
- Alias fornamelookup
.tcp_connection
- Difference ofconnect
andnamelookup
.tls_connection
- Difference ofpretransfer
andconnect
.server_processing
- Difference ofstarttransfer
andpretransfer
.content_transfer
- Difference oftotal
andstarttransfer
.
Examples
Examples are in examples.
They are configured to run against a local httpbin server.
httpbin Server
Start with docker
Start with docker-compose
Run the examples
Command-line examples
Inline POST request
Inline request with custom header
Inline request with environment variable
MY_TOKEN=token
Validate inline request status code
Dependencies example with AUTH_TOKEN
AUTH_TOKEN=my_token
Validation failure
Verbose output (libcurl)