Metadata-Version: 2.1
Name: cz-testing-python
Version: 0.4.4rc1
Summary: Useful testing functions; remove all the boilerplate
Home-page: https://github.com/Cloudzero/cz-testing-python
Author: CloudZero
Author-email: support@cloudzero.com
License: UNLICENSED
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: boto3 (>=1.14.62)
Requires-Dist: botocore (>=1.17.62)
Requires-Dist: cachetools (>=4.1.1)
Requires-Dist: typing-extensions (>=3.7.4.3)
Requires-Dist: toolz (>=0.10.0)
Requires-Dist: voluptuous (>=0.12.0)
Requires-Dist: pyfaaster (>=0.2.0)
Requires-Dist: aws-lambda-powertools (>=1.22.0)
Requires-Dist: cz-common-python (>=0.10.53)
Requires-Dist: faker (>=13.3.4)
Requires-Dist: boto3-stubs[stepfunctions]
Requires-Dist: docker (>=6.0.0)
Requires-Dist: deprecated (>=1.2.10)


[![Build Status](https://cloudzero.semaphoreci.com/badges/cz-testing-python/branches/main.svg)](https://cloudzero.semaphoreci.com/projects/cz-testing-python)

# Overview

`cz-testing-python` provides the `czt` python package that contains useful feature modules like:

- `czt.datagenerator`: generates fake data
- `czt.smoke`:
  - `api_test_runner`: low level smoke test runners
  - `api_test_client`: high level smoke test client APIs
- `czt.unittest`:
  - `Context` context manager for use in context fixtures
  - `MockEx` class decorates mocks, adding assertable chainable methods, e.g. `MockEx(mock).was_called_once_with(*args, **kwargs)` does all of the nasty argument unpacking and comparison and compares call count.


# Scopes Testing

In order to easily add basic role security testing to for your Web API routes, you can use the ScopeTest fixture. You can create one test that will dynamically generate
tests for each role against each scope in your web API (a scope is the combination of web method (i.e. GET, POST, etc.) and route. By using this mechanism, you will not
need to keep adding new smoke tests to test out role access. Your smoke tests can focus on functionality.

This fixture works by discovering all routes/methods from the deployed APIGateway and compares them to the `setup-template.json` file. Additionally you can define test configurations
that detail what roles should be allowed to call the API (by default it assumes only organizers and super-users can call it).

These API calls for these test do not need to necessarily return a 2xx. If a role is allowed to call an API, then any 2xx status and some 4xx status codes (400, 404, 406, 409, 410, 411, 412, 415, 416, 417)
will count as a success (i.e. if they are expected to be allowed to call and a 401 or 403 is returned, then the scope is not setup correctly). Any 5xx codes would be considered a failure as those return codes
indicate internal failures.

To add these tests you must do the following:

1. Add the `PublicApiId` to the `Outputs` section of your CloudFormation deployment file (`template.yaml`) file:

```
Outputs:

  #### NEW OUTPUT ####
  PublicApiId:
    Description: Public Feature REST API ID
    Value: !Sub ${PublicApi}
  ####################
```

2. In your `tests/smoke/web` folder, add the `test_scopes.py` file. It should look something like the following:

```
import pytest
from czt.smoke.scopes import ALL_ROLES, get_scope_tests, ScopeTest
from tests.smoke.common import CONFIGURATION

# This configuration allows the configuration of what roles can call what scopes.

SCOPE_TEST_CONFIGURATIONS = {
    "insights:get-insight": {'allowed_roles': ALL_ROLES},
    "insights:get-all-insights": {'allowed_roles': ALL_ROLES},
    "insights:export-insights": {'allowed_roles': ALL_ROLES},
    "insights:get-all-comments": {'allowed_roles': ALL_ROLES},
    "insights:create-comment": {'allowed_roles': ALL_ROLES},
    "insights:update-comment": {'allowed_roles': ALL_ROLES},
    "insights:get-all-resources": {'allowed_roles': ALL_ROLES},
    "insights:export-resources": {'allowed_roles': ALL_ROLES},
    "insights:get-summary": {'allowed_roles': ALL_ROLES},
    "insights:update-insight": {'allowed_roles': ALL_ROLES},

    #  You can specify to skip a scope if the tests don't apply of don't work out well for that scope
    "insights:create-insight": {'skip': True}, 

    # If now 'allowed_roles' are specified, then it is assumed that only the organizer and super-user can call the route
    "insights:delete-insight": {}
    
    # Any method/routes not specified, but found in the APIGateway will be tested with default settings.
    # If a method/route is found in the gateway, but not in the `setup-template.json` that test will fail.
}

# You must add this function (named exactly as is) and it must call `get_scope_tests`
def pytest_generate_tests(metafunc):
    get_scope_tests(metafunc, SCOPE_TEST_CONFIGURATIONS, CONFIGURATION)


@pytest.mark.smoke
def test_scopes(namespace, credentials, public_api_url, scope_test: ScopeTest):
    scope_test.run(namespace, credentials, public_api_url)

```

If a valid AWS console token is not available in the environment, the tests will not be enumerated successfully. This is because calls must be made to AWS APIGateway to get the routes.
By default only routes that begin with `/organizations/` will be used. This can be overridden by adding the `route_prefix` parameter when calling `get_scope_tests`.


