# Scenario File Syntax This document defines the complete syntax for BerryCrush `.scenario` and `.fragment` files. ## File Types | Extension | Description | |-----------|-------------| | `.scenario` | Test scenario files | | `.fragment` | Reusable step fragments | ## File Encoding All files must be UTF-8 encoded. ## Complete Grammar (EBNF) ```ebnf (* Top-level structure *) scenario_file = [ parameters_block ] , { feature | scenario | fragment } ; (* Parameters block - file-level or feature-level configuration *) parameters_block = "parameters:" , NEWLINE , { parameter_entry | nested_param_block } ; parameter_entry = INDENT , parameter_name , ":" , parameter_value , NEWLINE ; nested_param_block = INDENT , identifier , ":" , NEWLINE , { INDENT , INDENT , parameter_entry } ; (* Tags *) tags = { tag } ; tag = "@" , identifier ; (* Feature block with optional parameters and background *) feature = tags , "feature:" , feature_name , NEWLINE , [ INDENT , parameters_block ] , [ background ] , { feature_scenario } ; background = INDENT , "background:" , NEWLINE , { step } ; feature_scenario = INDENT , tags , ( "scenario:" | "outline:" ) , scenario_name , NEWLINE , { step } , [ examples ] ; (* Scenario definition *) scenario = tags , "scenario:" , scenario_name , NEWLINE , { step } , [ examples ] ; (* Scenario outline with examples *) outline = tags , "outline:" , scenario_name , NEWLINE , { step } , examples ; (* Fragment definition *) fragment = "fragment:" , fragment_name , NEWLINE , { step } ; (* Step definition *) step = INDENT , step_keyword , step_description , NEWLINE , { step_directive } ; step_keyword = "given " | "when " | "then " | "and " | "but " ; (* Step directives *) step_directive = INDENT , INDENT , directive_type , [ directive_value ] , NEWLINE ; directive_type = "call" | "assert" | "extract" | "include" | "body:" | conditional ; (* Include directive with optional parameters *) include_directive = "include" , fragment_name , [ NEWLINE , include_params ] ; include_params = { INDENT , INDENT , INDENT , param_name , ":" , param_value , NEWLINE } ; param_value = quoted_string | number | boolean | variable | json_object | json_array ; (* Assertions - supports both built-in conditions and custom assertion patterns *) assertion = "assert" , [ "not" ] , ( condition | custom_assertion ) ; (* Custom assertions - any text pattern that doesn't match built-in conditions *) (* Used with custom assertion providers registered via AssertionRegistry *) custom_assertion = assertion_pattern ; assertion_pattern = { text_token | quoted_string | number | variable | json_path } ; (* Conditional branching - if/else if/else *) conditional = if_branch , { else_if_branch } , [ else_branch ] ; if_branch = "if" , condition , NEWLINE , { conditional_action } ; else_if_branch = "else if" , condition , NEWLINE , { conditional_action } ; else_branch = "else" , NEWLINE , { conditional_action } ; conditional_action = assertion | extraction | fail_action | conditional ; fail_action = "fail" , quoted_string ; (* Built-in conditions - for both 'if' and 'assert' *) condition = status_condition | jsonpath_condition | header_condition | body_contains | schema_condition | response_time | variable_condition ; status_condition = "status" , ( number | status_range ) ; status_range = digit , "xx" | number , "-" , number ; jsonpath_condition = jsonpath , [ "not" ] , operator , [ value ] ; header_condition = "header_" , header_name , [ "not" ] , operator , [ value ] ; body_contains = "contains" , quoted_string ; schema_condition = "schema" ; response_time = "responseTime" , number ; variable_condition = variable_path , [ "not" ] , operator , [ value ] ; (* Operators for conditions *) operator = "equals" | "=" | "contains" | "matches" | "exists" | "greaterThan" | ">" | "lessThan" | "<" | "hasSize" | "size" | "arraySize" | "notEmpty" | "in" ; (* Examples for parameterized scenarios *) examples = INDENT , "examples:" , NEWLINE , example_header , { example_row } ; example_header = INDENT , INDENT , "|" , { cell , "|" } , NEWLINE ; example_row = INDENT , INDENT , "|" , { cell , "|" } , NEWLINE ; (* Terminals *) feature_name = text ; scenario_name = text ; fragment_name = text ; step_description = text ; parameter_name = identifier , { "." , identifier } ; parameter_value = quoted_string | number | boolean ; cell = { character - "|" } ; text = { character - NEWLINE } ; text_token = identifier | keyword ; identifier = letter , { letter | digit | "_" | "-" } ; json_path = "$" , { "." , identifier | "[" , ( number | "*" ) , "]" } ; variable = "{{" , variable_path , "}}" ; variable_path = identifier , { "." , identifier } ; quoted_string = '"' , { character - '"' } , '"' ; number = [ "-" ] , digit , { digit } , [ "." , digit , { digit } ] ; boolean = "true" | "false" ; header_name = identifier , { "-" , identifier } ; INDENT = " " ; (* 2 spaces *) NEWLINE = "\n" | "\r\n" ; ``` ## Tags Tags are used to categorize and filter scenarios. They begin with `@` and must appear before the element they annotate. ```berrycrush # Tag a single scenario @smoke @critical scenario: Critical path test when I test call ^test # Tag a feature (inherited by all scenarios in the feature) @api @regression feature: API Tests scenario: list items when I list call ^list ``` ### Built-in Tags | Tag | Description | |-----|-------------| | `@ignore` | Skip this scenario during execution | | `@wip` | Work in progress (commonly filtered out) | | `@slow` | Marks slow-running tests | ### Tag Filtering with JUnit Use `@BerryCrushTags` annotation to filter scenarios: ```java // Exclude @ignore tagged scenarios @BerryCrushTags(exclude = {"ignore"}) // Only run @smoke tagged scenarios @BerryCrushTags(include = {"smoke"}) // Combine include and exclude @BerryCrushTags(include = {"api"}, exclude = {"slow", "wip"}) ``` ## Features and Background Features provide logical grouping for related scenarios with shared setup via background steps. ### Basic Feature ```berrycrush feature: Pet Store API scenario: list all pets when I list pets call ^listPets then I get results assert status 200 scenario: create a pet when I create a pet call ^createPet body: {"name": "Max"} then the pet is created assert status 201 ``` ### Feature with Background Background steps run before **each** scenario in the feature: ```berrycrush feature: Pet Operations background: given: setup test data call ^createPet body: {"name": "TestPet"} assert status 201 extract $.id => petId scenario: get pet by id when: retrieve the pet call ^getPetById petId: {{petId}} then: pet is returned assert status 200 scenario: update pet when: update the pet name call ^updatePet petId: {{petId}} body: {"name": "UpdatedPet"} then: pet is updated assert status 200 ``` ### Tagged Features Tags on a feature are inherited by all scenarios within: ```berrycrush @api @regression feature: Authentication Tests background: given: login call ^login extract $.token => authToken @smoke scenario: access profile when: get profile call ^getProfile header_Authorization: "Bearer {{authToken}}" then: profile is returned assert status 200 @ignore scenario: incomplete test when: TODO call ^incomplete ``` In this example: - "access profile" has tags: `api`, `regression`, `smoke` - "incomplete test" has tags: `api`, `regression`, `ignore` ## Quick Reference ### Basic Scenario ```berrycrush scenario: My test scenario given some precondition when I perform an action call ^operationId then I should see a result assert status 200 ``` ### With Parameters Block ```berrycrush parameters: baseUrl: "http://localhost:8080" timeout: 60 shareVariablesAcrossScenarios: true scenario: Test with custom configuration when I call the API call ^listPets then I get results assert status 200 ``` ## Step Keywords | Keyword | Description | |---------|-------------| | `given` | Precondition setup | | `when` | Action to perform | | `then` | Expected outcome | | `and` | Continuation of previous step type | | `but` | Exception/negative case | ## Step Directives ### API Call (`call`) Syntax: `call [using ] ^` ```berrycrush when I request pets call ^listPets ``` With named spec (multi-spec): ```berrycrush when I authenticate call using auth ^login ``` #### Call Parameters | Parameter | Description | Example | |-----------|-------------|---------| | Path parameter | Replace path variable | `petId: 123` | | Query parameter | Add query string | `status: "available"` | | Header | Add HTTP header | `header_Authorization: "Bearer token"` | | Body (inline) | Set JSON body directly | `body: {"name": "Fluffy"}` | | Body (structured) | Set body with properties | `body:` newline + indented properties | Example with inline body: ```berrycrush when I create a pet call ^createPet petId: 123 status: "available" header_Authorization: "Bearer {{token}}" body: {"name": "Fluffy", "category": "dog"} ``` Example with structured body: ```berrycrush when I create a pet call ^createPet body: name: Fluffy category: dog ``` ### Assertions (`assert`) > **Note:** Assertions and `if` conditions share the same condition evaluation logic. > The same operators and syntax work identically in both contexts. #### Status Code ```berrycrush assert status 200 assert status 2xx # Range: 200-299 assert status 201-204 # Range: 201-204 ``` #### Body-Level Assertions ```berrycrush assert contains "expected text" # Body contains substring assert not contains "unexpected text" # Body does not contain substring assert schema # Validate against OpenAPI schema ``` #### Response Time Assertions ```berrycrush assert responseTime 1000 # Response must complete within 1000ms ``` #### JSONPath Assertions ```berrycrush assert $.name equals "Fluffy" assert $.id exists assert $.pets notEmpty assert $.count greaterThan 0 assert $.tags contains "urgent" assert $.status in ["available", "pending"] assert $.items hasSize 5 ``` #### Header Assertions ```berrycrush assert header Content-Type equals "application/json" assert header X-Request-Id exists assert header Cache-Control contains "no-cache" ``` #### The `not` Keyword (Negation) The `not` keyword inverts any assertion, making it a negative check. It can be placed in two positions: **At the beginning (for body-level assertions):** ```berrycrush assert not contains "error" # Body does NOT contain "error" ``` **After the JSONPath (for JSONPath assertions):** ```berrycrush assert $.status not equals "sold" # Field does NOT equal "sold" assert $.items not hasSize 0 # Array does NOT have size 0 assert $.deleted not exists # Field does NOT exist ``` Both positions are supported to allow natural reading: "assert body does not contain" and "assert the name does not equal". #### Operators | Operator | Description | Example | |----------|-------------|---------| | `equals` | Exact equality | `assert $.name equals "Max"` | | `not` | Negation (see above) | `assert $.status not equals "sold"` | | `exists` | Field exists | `assert $.id exists` | | `notEmpty` | Array/string not empty | `assert $.pets notEmpty` | | `greaterThan` | Numeric comparison | `assert $.price greaterThan 0` | | `lessThan` | Numeric comparison | `assert $.age lessThan 10` | | `contains` | Body/array contains | `assert contains "text"`, `assert $.tags contains "dog"` | | `in` | Value in list | `assert $.status in ["x", "y"]` | | `hasSize` | Array/string length | `assert $.items hasSize 3` | | `matches` | Regex match | `assert $.email matches ".*@.*"` | | `schema` | Validate against schema | `assert schema` | ### Extraction (`extract`) Syntax: `extract => ` ```berrycrush then I capture the ID assert status 201 extract $.id => petId ``` Use extracted variables with `{{variableName}}`: ```berrycrush when I get the pet call ^getPetById petId: {{petId}} ``` ### Conditional Assertions (`if/else if/else/fail`) Conditional assertions allow branching logic based on response status, JSON path values, or headers. > **Note:** `if` conditions and `assert` statements share the same condition evaluation logic. > This means the same condition types (status, jsonpath, header) work identically in both contexts. #### Basic Syntax ```berrycrush when I make a request call ^operation if status 201 # Executed if status is 201 (created) assert $.id notEmpty extract $.id => createdId else if status 200 # Executed if status is 200 (already exists) assert $.id notEmpty else # Executed if none of the above matched fail "Expected status 200 or 201" ``` #### Condition Types Both `if` conditions and `assert` statements share a unified condition syntax: | Type | Syntax | Example | |------|--------|---------| | Status code | `status ` | `status 201` | | Status range | `status ` | `status 2xx` | | JSON path | `$.path ` | `$.count greaterThan 0` | | Header | `header ` | `header Content-Type equals "application/json"` | | Body contains | `contains ` | `contains "success"` | | Schema validation | `schema` | `schema` | | Response time | `responseTime ` | `responseTime 1000` | | Variable | ` ` | `test.type equals "invalid"` | > **Note:** While all condition types are syntactically valid in both contexts, some are more commonly used in assertions (schema, responseTime) and some in conditionals (variable). #### JSON Path Operators Same operators work in both `if` and `assert` contexts: `equals`, `exists`, `notEmpty`, `greaterThan`, `lessThan`, `contains`, `in`, `hasSize`, `matches`. ```berrycrush if $.items greaterThan 0 assert $.items[0].id notEmpty else # Empty list is acceptable assert $.items hasSize 0 ``` #### Nested Conditionals Conditionals can be nested for complex logic: ```berrycrush if status 200 if $.status equals "available" assert $.price notEmpty else assert $.status notEmpty else if status 201 assert $.id notEmpty else fail "Unexpected status" ``` #### The `fail` Action Use `fail` to explicitly fail a scenario with a custom message: ```berrycrush else fail "Expected status 200 or 201 but got something else" ``` ### Fragment Inclusion (`include`) Basic fragment inclusion: ```berrycrush scenario: Use authentication given I am authenticated include authenticate when I access protected resource call ^getProfile ``` #### Parameterized Fragments Fragments can accept parameters that become available as variables within the fragment's scope: ```berrycrush # Fragment definition (create_user.fragment) fragment: create_user when creating the user call ^createUser body: {"name": "{{name}}", "email": "{{email}}", "age": {{age}}} then user is created assert status 201 # Scenario using parameterized fragment scenario: Create specific user given I create a user include create_user name: "John Doe" email: "john@example.com" age: 30 then the user exists assert $.name equals "John Doe" ``` **Parameter Types:** | Type | Example | Notes | |------|---------|-------| | String | `name: "John Doe"` | Use quotes for values with spaces | | Number | `age: 30` | Integer or decimal | | Boolean | `active: true` | true or false | | Variable | `name: {{userName}}` | Reference existing variables | | JSON Object | `data: {"key": "value"}` | Inline JSON | | JSON Array | `tags: ["a", "b"]` | Inline arrays | **Variable References in Parameters:** Parameters can reference variables defined earlier in the scenario: ```berrycrush scenario: Create user from context given I have user data set userName => "Alice" set userEmail => "alice@example.com" when I create the user include create_user name: {{userName}} email: {{userEmail}} age: 25 ``` ### Request Body (`body:`) The `body:` keyword supports two modes: #### Inline JSON (Raw Mode) Specify JSON directly on the same line: ```berrycrush when I create a pet call ^createPet body: {"name": "Fluffy", "status": "available"} ``` #### Structured Properties Use indented properties to generate JSON with OpenAPI schema defaults: ```berrycrush when I create a pet call ^createPet body: name: Fluffy status: available ``` This generates a JSON body by: 1. Extracting default values from the OpenAPI requestBody schema 2. Merging with user-provided properties (user values take precedence) 3. Generating the final JSON **Example with partial data:** ```berrycrush # Only specify name, other fields use schema defaults when I create a pet call ^createPet body: name: MyPet ``` **Example with nested objects:** ```berrycrush when I create a pet call ^createPet body: name: NestedPet metadata: source: test version: 1.0 ``` #### Multi-line Body (Triple Quotes) Use triple quotes (`"""`) for multi-line raw body content. The common indentation is automatically removed: ```berrycrush when I create a pet call ^createPet body: """ { "name": "Fluffy", "status": "available", "tags": ["cute", "friendly"] } """ ``` This sends the JSON exactly as written (with common indentation stripped). This is useful for: - Large JSON payloads that are hard to read on one line - Copying JSON from other sources - Complex nested structures #### Multi-line JSON (Legacy) Use `>` for multi-line raw JSON: ```berrycrush when I create a pet call ^createPet body: > { "name": "Fluffy", "status": "available", "tags": ["cute", "friendly"] } ``` ## Auto-Generated Tests (`auto:`) The `auto:` directive enables automatic generation of invalid request, security, and idempotency tests based on OpenAPI schema constraints and common patterns. ### Syntax ```berrycrush call ^operationId auto: [] ``` Where `` is a space-separated list of: - `invalid` - Generate tests that violate OpenAPI schema constraints - `security` - Generate tests with common attack payloads - `multi` - Generate idempotency tests with sequential and concurrent requests ### Basic Example ```berrycrush scenario: Auto-generated tests for createPet when I create a pet with invalid request call ^createPet auto: [invalid security] body: name: "Fluffy" status: "available" if status 4xx and test.type equals invalid # Invalid tests should return 4xx - test passed else if status 4xx and test.type equals security # Security tests should return 4xx - attack blocked else fail "Expected 4xx for {{test.type}} test: {{test.description}}" ``` ### Test Types #### Invalid Tests (`invalid`) Generate tests that violate OpenAPI schema constraints: | Constraint | Test Generated | |------------|----------------| | `minLength` | String with length below minimum | | `maxLength` | String with length above maximum | | `minimum` | Number below minimum value | | `maximum` | Number above maximum value | | `pattern` | String that violates regex pattern | | `format` (email) | Invalid email format | | `format` (uuid) | Invalid UUID format | | `format` (date) | Invalid date format | | `required` | Missing required fields | | `enum` | Value not in allowed enum | | Type mismatch | Wrong type (e.g., string instead of number) | #### Security Tests (`security`) Generate tests with common attack payloads: | Category | Examples | |----------|----------| | SQL Injection | `' OR '1'='1`, `"; DROP TABLE users; --` | | XSS | ``, `javascript:alert(1)` | | Path Traversal | `../../etc/passwd`, `....//....//etc/passwd` | | Command Injection | `; ls -la`, `$(whoami)`, `\`id\`` | | LDAP Injection | `*)(uid=*))(|(uid=*`, `admin)(&)` | | XXE | `]>` | #### Multi Tests (`multi`) Generate idempotency tests that execute requests multiple times: | Mode | Description | |------|-------------| | Sequential | Execute requests one after another (default: 3 requests) | | Concurrent | Execute requests simultaneously (default: 5 requests) | Multi tests verify: - **Idempotency** - Same request returns consistent results - **Race conditions** - Concurrent requests don't cause data corruption - **State consistency** - API maintains correct state under load Example: ```berrycrush scenario: Idempotency test for getPet when I get a pet multiple times call ^getPetById auto: [multi] petId: 1 then all requests succeed consistently assert status 2xx ``` Configure request counts: ```berrycrush parameters: multiTestSequentialCount: 5 # Run 5 sequential requests multiTestConcurrentCount: 10 # Run 10 concurrent requests ``` ### Parameter Locations Auto-tests are generated for parameters in different locations: | Location | Description | Example | |----------|-------------|---------| | `request body` | JSON body fields | `name`, `status` | | `path variable` | URL path parameters | `petId` in `/pets/{petId}` | | `query parameter` | Query string params | `status` in `?status=available` | | `header` | HTTP headers | `Authorization`, `X-Api-Key` | ### Context Variables During auto-test execution, these variables are set: | Variable | Description | Example Values | |----------|-------------|----------------| | `test.type` | Test category | `"invalid"`, `"security"`, `"multi"` | | `test.field` | Field being tested | `"name"`, `"petId"` | | `test.description` | Test description | `"SQL Injection"`, `"minLength violation"` | | `test.value` | Attack/invalid value | `"' OR '1'='1"` | | `test.location` | Parameter location | `"request body"`, `"path variable"` | For multi-tests, additional variables are available: | Variable | Description | Example Values | |----------|-------------|----------------| | `multiTest.mode` | Execution mode | `"SEQUENTIAL"`, `"CONCURRENT"` | | `multiTest.count` | Number of requests | `3`, `5` | | `multiTest.passed` | Whether all passed | `true`, `false` | | `multiTest.duration` | Total time (ms) | `150` | ### Complete Example ```berrycrush scenario: Auto-generated path parameter tests for getPetById when I get a pet with invalid ID call ^getPetById auto: [invalid security] petId: 1 if status 4xx # Test passed - invalid request rejected else fail "Expected 4xx for [{{test.type}}] {{test.location}} test: {{test.description}}" scenario: Auto-generated create pet tests when I create a pet with invalid data call ^createPet auto: [invalid security] body: name: "TestPet" status: "available" if status 4xx and test.type equals invalid # Invalid input correctly rejected else if status 4xx and test.type equals security # Security attack blocked else if not status 2xx # Unexpected error fail "Unexpected error for {{test.type}}: {{test.description}}" ``` ### Test Display Names Auto-tests appear in test reports with descriptive names: **Invalid and Security Tests:** ``` [invalid - minLength] request body name with value [invalid - type] path variable petId with value not-a-number [security - SQL Injection] request body name with value ' OR '1'='1 [security - Path Traversal] path variable petId with value ../../etc/passwd ``` **Multi Tests (Idempotency):** ``` [multi:sequential] 3 requests [multi:concurrent] 5 requests ``` With custom counts: ``` [multi:sequential] 10 requests [multi:concurrent] 20 requests ``` ### Best Practices 1. **Provide valid base parameters** - Auto-tests modify one parameter at a time while keeping others valid 2. **Use conditional assertions** - Check `test.type` to handle invalid vs security tests differently 3. **Expect 4xx responses** - Both invalid and security tests should be rejected by a secure API 4. **Review generated tests** - The number of tests depends on schema constraints; complex schemas generate more tests ## Parameters Block Parameters can be specified at two levels: ### File-Level Parameters Place at the top of the file to configure all scenarios: ```berrycrush parameters: baseUrl: "http://localhost:8080" timeout: 60 environment: "staging" shareVariablesAcrossScenarios: true logRequests: true logResponses: true strictSchemaValidation: false followRedirects: true header.Authorization: "Bearer test-token" header.X-API-Key: "my-api-key" autoAssertions.enabled: true autoAssertions.schema: false ``` ### Feature-Level Parameters Place inside a feature block to configure only scenarios in that feature: ```berrycrush feature: Pet CRUD Operations parameters: shareVariablesAcrossScenarios: true scenario: Create pet when: I create a pet call ^createPet body: {"name": "SharedPet"} extract $.id => petId scenario: Use shared variable # Can access {{petId}} from previous scenario when: I get the pet call ^getPetById petId: {{petId}} then: I see the pet assert status 200 ``` Feature-level parameters override file-level parameters for scenarios in that feature. ### Supported Parameters | Parameter | Type | Description | |-----------|------|-------------| | `baseUrl` | String | Override API base URL | | `timeout` | Number | Request timeout in seconds | | `environment` | String | Environment name for reports | | `shareVariablesAcrossScenarios` | Boolean | Share extracted variables across scenarios | | `logRequests` | Boolean | Enable HTTP request logging | | `logResponses` | Boolean | Enable HTTP response logging | | `strictSchemaValidation` | Boolean | Fail on schema validation warnings | | `followRedirects` | Boolean | Follow HTTP redirects | | `header.` | String | Add/override default header | | `retry.