Metadata-Version: 2.4
Name: cz-ops-cli
Version: 3.15.0
Summary: CloudZero Internal Operations command line interface.
Author-email: CloudZero <support@cloudzero.com>
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://github.com/Cloudzero/cz-ops-cli
Keywords: CloudZero,czops,ops,cli
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: MacOS
Classifier: Operating System :: Unix
Requires-Python: <3.14,>=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: cloudzero-identifiers>=1.11.5
Requires-Dist: urllib3>=2.6.3
Requires-Dist: boto3>=1.42.44
Requires-Dist: botocore>=1.42.44
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: pyaml>=26.2.1
Requires-Dist: httpx>=0.28.1
Requires-Dist: cachetools>=7.0.0
Requires-Dist: mako>=1.3.10
Requires-Dist: voluptuous>=0.16.0
Requires-Dist: base58>=2.1.1
Requires-Dist: tabulate>=0.9.0
Requires-Dist: faker>=40.4.0
Requires-Dist: toolz>=1.1.0
Requires-Dist: numpy>=2.4.2
Requires-Dist: packaging>=26.0
Requires-Dist: pydantic>=2.12.5
Requires-Dist: pydantic-settings>=2.12.0
Requires-Dist: email-validator>=2.3.0
Requires-Dist: rich>=14.3.2
Requires-Dist: python-dateutil>=2.9.0.post0
Requires-Dist: flask>=3.1.2
Requires-Dist: flask-cors>=6.0.2
Requires-Dist: pyjwt[crypto]>=2.11.0
Requires-Dist: term-image>=0.7.2
Requires-Dist: polars>=1.38.1
Requires-Dist: cryptography>=46.0.4
Requires-Dist: docopt-ng>=0.9.0
Requires-Dist: tomli-w>=1.2.0
Dynamic: license-file

# CloudZero Operations CLI

CloudZero Operations Command Line Interface (`czops`) provides administrative functions for managing CloudZero platform systems.



## Documentation

📚 **[Complete Documentation](docs/index.md)** - Full documentation index

### Quick Links
- **[Installation Guide](docs/guides/installation.md)** - Get started with installation
- **[Getting Started](docs/guides/getting-started.md)** - Basic usage and examples
- **[Command Reference](docs/guides/commands.md)** - All available commands
- **[Contributing](docs/development/contributing.md)** - Development and contribution guide


## Overview

The CloudZero Operations CLI is an internal tool designed for CloudZero employees to configure, manage, and troubleshoot CloudZero platform services. It provides a comprehensive set of commands for organization management, connection administration, API interaction, and system configuration.

### Recent Improvements (v3.0.3)

* **uv Only**: Now exclusively uses `uv tool` for installation and management - pipx no longer supported
* **Migration Guide**: Complete guide for migrating from pipx to uv with step-by-step instructions
* **Simplified Installation**: Optional global configuration for CloudZero PyPI repository
* **Configure Command Removed**: Settings auto-created with defaults - no manual configuration needed
* **Backend-Owned CUR Rules**: CUR validation rules stored in DynamoDB with tiered validation
* **Enhanced Healthcheck**: Composite key support for multi-type connections, all CURs displayed
* **Simplified API**: `get_organizations()` returns single format, removed 333 lines of dead code

## Features

* **Organization Management**: List, create, disable/enable, and configure CloudZero organizations
* **Connection Administration**: Manage AWS account connections, healthchecks, resource collection, and cache management
* **API Schema Recording**: Record and export API schemas for documentation and development
* **QuickSight Integration**: Create, delete, and manage QuickSight dashboards and datasets with SPICE reporting
* **Notification Management**: Send and list notifications for CloudZero organizations
* **Anomaly Detection**: Configure and update anomaly detection settings with enhanced dry-run capabilities
* **Document Management**: Upload, list, and manage organization documents
* **Auto-Link Management**: Enable, disable, and configure automatic AWS account linking with Okta authentication
* **Cache Management**: Advanced SQLite-based caching with intelligent refresh strategies and real-time status notifications
* **Shell Access**: Access shell environments for connected AWS accounts

Run `czops --help` for a complete list of available commands.

## Installation

### Production Installation

For standard installation instructions, follow the [CloudZero internal guide for installing cz-ops-cli](https://cloudzero.atlassian.net/wiki/spaces/CZ/pages/955449345/How+to+Install+and+use+CZOPS).

#### Installation Method (uv tool)

**Using uv tool:**

1. **Install uv** (if not already installed):
   ```bash
   curl -LsSf https://astral.sh/uv/install.sh | sh

   # Restart your shell or source your profile
   source ~/.zshrc  # or ~/.bash_profile
   ```

2. **Optional: Configure global uv settings** for easier CloudZero package installation:
   ```bash
   # Create uv config directory
   mkdir -p ~/.config/uv

   # Create global config file
   cat > ~/.config/uv/uv.toml << 'EOF'
   # Global uv configuration
   # This adds CloudZero's PyPI repository to all uv operations

   # Add CloudZero PyPI as an extra index URL
   extra-index-url = ["https://pypi.cloudzero.com"]

   # Trust the CloudZero PyPI host
   allow-insecure-host = ["pypi.cloudzero.com"]
   EOF
   ```

3. **Install cz-ops-cli:**
   ```bash
   # With global config (step 2):
   uv tool install cz-ops-cli

   # Without global config:
   uv tool install cz-ops-cli --extra-index-url https://pypi.cloudzero.com
   ```

4. **Verify installation:**
   ```bash
   czops --version
   czops --help
   ```

#### Migrating from pipx to uv tool

**IMPORTANT:** pipx is no longer supported. If you previously installed cz-ops-cli with pipx, you must migrate to uv tool.

**Migration steps:**
1. **Uninstall pipx version:** `pipx uninstall cz-ops-cli`
2. **Install uv** (if not already installed):
   ```bash
   curl -LsSf https://astral.sh/uv/install.sh | sh
   source ~/.zshrc  # or ~/.bash_profile
   ```
3. **Install with uv:** `uv tool install cz-ops-cli --extra-index-url https://pypi.cloudzero.com`
4. **Verify:** `czops --version`

**Why uv only:**
- ⚡ **Significantly faster** installations and upgrades
- 🔧 **Better dependency resolution** and conflict handling
- 🚀 **Modern toolchain** - official CloudZero standard
- 📦 **Consistent** with project's development environment

#### Upgrade to Latest Version

```bash
# With global config:
uv tool upgrade cz-ops-cli

# Without global config:
uv tool upgrade cz-ops-cli --extra-index-url https://pypi.cloudzero.com
```

#### Tool Management

```bash
uv tool list                  # List installed tools
uv tool uninstall cz-ops-cli  # Uninstall
uv tool upgrade cz-ops-cli    # Upgrade to latest
```

### Development Setup

For local development, the setup process is simple with `uv`:

1. **Clone the repository**
   ```bash
   git clone https://github.com/Cloudzero/cz-ops-cli.git
   cd cz-ops-cli
   ```

2. **Install dependencies and create virtual environment**
   ```bash
   uv sync
   ```

   This single command will:
   - Automatically create a `.venv` virtual environment
   - Install all dependencies (including dev dependencies)
   - Set up the development environment

3. **Activate the virtual environment (recommended for development)**
   ```bash
   source .venv/bin/activate
   ```

### Running Commands During Development

**Option 1: With activated virtual environment (recommended)**
```bash
source .venv/bin/activate
czops --help
```

**Option 2: Using uv run (without activating venv)**
```bash
uv run czops --help
```

## Development Commands

The project provides several script commands to help with development:

| Command | Description |
|---------|-------------|
| `./scripts/help.sh` | View available commands |
| `./scripts/init.sh` | Set up dependencies for development |
| `./scripts/test.sh` | Run the test suite |
| `./scripts/lint.sh` | Run the linter (ruff) |
| `./scripts/build.sh` | Build the project (runs lint and test first) |
| `./scripts/clean.sh` | Clean temporary files and build artifacts |

## Development Workflow

This project follows a **Feature Branch → Develop → Main** workflow with automated deployment.

### Branching Strategy

```
feature/fix-branch → develop (PR + review) → main (clean FF rebase + auto-deploy)
```

**Key Principles:**
- All new work starts from `develop` branch
- Feature branches are merged to `develop` via PR
- Only `develop` can be merged to `main`
- `main` is never ahead of `develop`
- Linear history is maintained (no merge commits)
- Automated deployment occurs from both branches

### Quick Start for Contributors

1. **Create feature branch:**
   ```bash
   git checkout develop
   git pull origin develop
   git checkout -b feature/your-feature-name
   ```

2. **Develop and test:**
   ```bash
   # Make changes
   ./scripts/test.sh    # Run tests
   ./scripts/lint.sh    # Run linter
   ```

3. **Create PR to develop:**
   ```bash
   git push -u origin feature/your-feature-name
   # Create PR via GitHub UI: feature/your-feature-name → develop
   ```

4. **After PR approval:**
   - Branch is merged to `develop`
   - Automated deployment to development environment
   - Delete feature branch

### Release Process

The CloudZero cz-ops-cli follows a structured release process with automated tagging and deployment. This section provides a comprehensive guide for creating releases.

#### 📋 **Release Checklist**

**1. Prepare Release (on develop branch):**
   ```bash
   # Update version numbers (both files must match)
   # - pyproject.toml: [project] version = "X.X.X"
   # - src/__init__.py: __version__ = "X.X.X"

   # Create detailed release notes
   # - docs/releases/X.X.X.md (following project template)
   # - Update CHANGELOG.md with entries

   # Validate everything works
   ./scripts/test.sh    # All tests must pass
   ./scripts/lint.sh    # Zero linting errors
   ./scripts/build.sh   # Must build successfully
   ```

**2. Commit and Tag on Develop:**
   ```bash
   # Commit all changes
   git add -A
   git commit -m "Release X.X.X - Brief Description

   - Update version numbers in pyproject.toml and src/__init__.py
   - Add comprehensive release notes in docs/releases/X.X.X.md
   - Update CHANGELOG.md with detailed changes
   - [List other significant changes]"

   # Create and push tag
   git tag -a vX.X.X -m "Version X.X.X"
   git push origin develop
   git push origin vX.X.X
   ```

**3. Create Release PR to Main:**
   ```bash
   # Create PR: develop → main
   gh pr create --title "Release X.X.X - Brief description" --base main --head develop
   ```

**4. After Release PR Approval and Merge:**
   - ✅ **Automated production deployment** to CloudZero PyPI
   - 🏷️ **Intelligent tag management** (see Tag Behavior section below)
   - 📦 **GitHub release creation** with notes and artifacts
   - 🔄 **Automatic branch synchronization** (develop synced with main)

#### 🏷️ **Git Tag Behavior and Management**

The release process uses **intelligent tag management** to handle scenarios where tags are created on develop before merging to main.

**Scenario 1: Tag Created on Develop First (Recommended)**
```bash
# Current common workflow:
develop: [commits] ← vX.X.X tag created here first
main:    [older commits]

# After rebase develop → main:
develop: [same commits]
main:    [same commits] ← vX.X.X tag still points to develop

# Deploy-main workflow automatically handles this:
main:    [same commits] ← vX.X.X tag moved here by GitHub Actions
develop: [same commits] ← auto-synced with main (includes moved tag)
```

**GitHub Actions Tag Management Logic:**
1. **Tag Validation**: Checks if tag exists and where it points
2. **Smart Relocation**: If tag exists but points to wrong commit, it:
   - Deletes the existing tag (locally and remotely)
   - Creates new tag pointing to main branch merge commit
   - Pushes the updated tag
3. **Preservation**: If tag already points to correct commit, no action taken
4. **Creation**: If tag doesn't exist, creates it at main branch commit

**Tag Relocation Process (Automated):**
```yaml
# From .github/workflows/deploy.main.yml
if git tag exists but points to different commit:
  - Delete existing tag: git tag -d vX.X.X
  - Remove from remote: git push origin :refs/tags/vX.X.X
  - Create new tag: git tag -a vX.X.X -m "Release vX.X.X - moved to main"
  - Push updated tag: git push origin vX.X.X
```

**Benefits of This Approach:**
- ✅ **No tag conflicts** - System handles duplicate tags gracefully
- ✅ **Consistent final state** - Tags always point to production commits
- ✅ **Preserves workflow flexibility** - Supports tagging on develop or main
- ✅ **Automatic correction** - Fixes any tag misalignment without manual intervention

#### 🚀 **Automated Deployment Pipeline**

**Develop Branch Deployment:**
- **Trigger**: Push to develop branch (after CI success)
- **Version**: Appends `-dev` suffix (e.g., `1.2.3-dev` becomes `1.2.3.dev0`)
- **Target**: Development environment on CloudZero PyPI
- **Artifacts**: Development build with dev suffix

**Main Branch Deployment:**
- **Trigger**: Push to main branch (after CI success)
- **Version**: Production version (e.g., `1.2.3`)
- **Target**: Production environment on CloudZero PyPI
- **Artifacts**: Production build, GitHub release, tag management
- **Post-Deploy**: Automatic sync of develop with main

**Branch Synchronization:**
- **Automatic**: `sync-develop-with-main` workflow runs after successful main deployment
- **Process**: Rebases develop onto main to ensure alignment
- **Fallback**: Creates GitHub issue if automatic sync fails
- **Result**: Develop and main stay synchronized with shared tag history

#### 🛠️ **Release Troubleshooting**

**Common Issues and Solutions:**

1. **Tag Already Exists Error:**
   ```bash
   # This is handled automatically by GitHub Actions
   # No manual intervention needed
   ```

2. **Build Failures:**
   ```bash
   # Always run before releasing:
   ./scripts/test.sh && ./scripts/lint.sh && ./scripts/build.sh
   ```

3. **Version Mismatch:**
   ```bash
   # Ensure both files have identical version:
   grep version pyproject.toml
   grep __version__ src/__init__.py
   ```

4. **Branch Sync Failures:**
   ```bash
   # Manual sync if automated process fails:
   git checkout develop
   git rebase origin/main
   git push --force-with-lease origin develop
   ```

#### 📊 **Release Validation**

**Pre-Release Checklist:**
- [ ] Version updated in both `pyproject.toml` and `src/__init__.py`
- [ ] Release notes created in `docs/releases/X.X.X.md`
- [ ] CHANGELOG.md updated with comprehensive entries
- [ ] All tests pass: `./scripts/test.sh`
- [ ] Linting clean: `./scripts/lint.sh`
- [ ] Build successful: `./scripts/build.sh`
- [ ] Tag created and pushed from develop branch

**Post-Release Verification:**
- [ ] Production deployment successful
- [ ] GitHub release created with correct artifacts
- [ ] Tag points to main branch commit
- [ ] Develop branch automatically synced
- [ ] Package available on CloudZero PyPI

### Automated Deployment

The project uses GitHub Actions for automated deployment:

- **Develop branch**: Deploys development versions (with `-dev` suffix)
- **Main branch**: Deploys production versions and creates GitHub releases
- **Auto-sync**: Keeps `develop` aligned with `main` after releases

#### Automated Tagging
- **Git tags** are created automatically after successful production deployment
- **GitHub releases** are created with release notes and build artifacts
- **Tag protection** prevents duplicate tags from being created
- **Branch sync** ensures develop receives the new tags immediately

### Branch Protection

- **main**: Protected, requires PR reviews, only accepts PRs from `develop`
- **develop**: Protected, requires PR reviews, accepts PRs from feature branches
- **feature branches**: No protection, can be pushed to directly by creators

For detailed workflow documentation, see `CLAUDE.md`.

### Testing

To run all tests:
```bash
./scripts/test.sh
# or directly with pytest
pytest
```

To run a specific test file:
```bash
pytest tests/path/to/test_file.py -v
```

To run a specific test:
```bash
pytest tests/path/to/test_file.py::test_function_name -v
```

### Quick System Tests

For quick system testing, these commands are useful:
```bash
# List the CloudZero organization
czops org list 02fa7d30-c3de-4e0a-8f1e-2de120e7fd23

# List connections for the CloudZero organization (using full command or alias)
czops connection list 02fa7d30-c3de-4e0a-8f1e-2de120e7fd23
czops conn list 02fa7d30-c3de-4e0a-8f1e-2de120e7fd23

# List the CloudZero organization by name
czops org list cloudzero

# Check cache status
czops cache check
```

For troubleshooting, use the `--debug` flag to see additional information and the full request/response data.

## Project Structure

- `/src/`: Main source code directory
  - `/api/`: API client interfaces to CloudZero backend services
  - `/features/`: CLI command implementations organized by feature
  - `/utils/`: Shared utility functions and helpers (generic, reusable)
    - `/utils/core/`: Basic utilities and decorators (benchmark, retry, timeout, etc.)
    - `/utils/time/`: Date/time utilities for parsing, formatting, and conversion
    - `/utils/data/`: Data structure manipulation (dict search, filtering, JSON encoding)
    - `/utils/files/`: File operations (downloading, validation, reading)
    - `/utils/console/`: User interaction and terminal interface utilities
    - `/utils/validation/`: Input and format validation utilities
    - `/utils/aws/`: AWS-specific but generic utilities
  - `/lib/`: CloudZero-specific utilities and integrations
    - `api_recorder.py`: Records API calls for schema generation
    - `csv_processor.py`: Processes CloudZero-specific CSV data
    - `formatting.py`: CloudZero-specific data formatting
    - `interface.py`: CloudZero CLI interface components
    - `slack.py`: Slack integration for CloudZero notifications
    - `version.py`: Version compatibility checking
  - `/schemas/`: Pydantic models and schema definitions

## Configuration

The CLI provides several configuration options:

### Authentication Configuration
```bash
czops configure --auth
```
This configures your API credentials for accessing CloudZero services.

### Cache Management

The CLI uses an advanced SQLite-based caching system for improved performance:

```bash
czops cache check          # View cache statistics and status
czops cache clear          # Clear local cached data
czops cache refresh        # Trigger backend cache refresh
```

**Cache Features:**
- **Automatic Refresh**: Individual items refresh after 1 hour, full cache after 3 hours
- **Smart Lookups**: When searching by ID, only that item is refreshed if stale
- **Status Notifications**: Real-time feedback shows cache age and refresh operations
- **Data Enrichment**: Automatically adds `connection_count` and `record_sources` to cached data

## API Schema Recording

The CLI includes functionality to record V2 API requests/responses for schema generation:
- Storage in `~/.cloudzero/api_records/` with JSON files per endpoint
- Commands:
  - `czops api-schemas list` - Show all recorded endpoints with record counts
  - `czops api-schemas show <endpoint> [--all]` - Display details for a specific endpoint
    - Use `--all` option to view all recorded data instead of just the latest record
  - `czops api-schemas export [<endpoint>]` - Export records to JSON files (all endpoints if none specified)
  - `czops api-schemas purge --age <DAYS>` - Remove records older than specified days
- Useful for generating Pydantic models from actual API traffic
- Records are automatically timestamped for age-based management
- Records can be purged based on age to manage storage and remove outdated information

## Requirements

- Python 3.11, 3.12, or 3.13
- AWS credentials for some operations
- CloudZero API credentials

## License

Proprietary - All Rights Reserved, CloudZero.

## Support

For issues, contact the CloudZero tools team at `#cz-tools` on internal Slack.
