Metadata-Version: 2.4
Name: cloudzero-telemetry-data-tool
Version: 0.5.0
Summary: CloudZero Telemetry Data Tool (czdt) - High-performance data analysis tool for processing and analyzing telementry CSV datasets
Project-URL: Homepage, https://github.com/cloudzero/research-telemetry-data-tool
Project-URL: Documentation, https://github.com/cloudzero/research-telemetry-data-tool/blob/main/README.md
Project-URL: Repository, https://github.com/cloudzero/research-telemetry-data-tool
Project-URL: Issues, https://github.com/cloudzero/research-telemetry-data-tool/issues
Author-email: CloudZero <support@cloudzero.com>
License: PROPRIETARY SOFTWARE LICENSE
        
        CloudZero, Inc. - All Rights Reserved
        
        Copyright (c) 2024 CloudZero, Inc.
        
        NOTICE: This software and its documentation are proprietary to CloudZero, Inc. and contain confidential and trade secret information belonging to CloudZero, Inc.
        
        CONFIDENTIAL AND PROPRIETARY
        
        This software, including all source code, object code, documentation, and related materials (collectively, the "Software"), is the confidential and proprietary property of CloudZero, Inc. ("CloudZero"). The Software contains trade secrets and confidential information of CloudZero and is protected by applicable intellectual property laws, including without limitation, copyright laws and trade secret laws.
        
        ACCESS RESTRICTIONS
        
        Access to and use of this Software is strictly limited to authorized CloudZero employees and contractors who have signed appropriate confidentiality agreements and have a legitimate business need to access the Software in connection with their work for CloudZero.
        
        PROHIBITED USES
        
        The following activities are strictly prohibited:
        
        1. Copying, reproducing, or distributing the Software or any portion thereof without prior written authorization from CloudZero
        2. Reverse engineering, decompiling, disassembling, or attempting to derive the source code of the Software
        3. Modifying, adapting, altering, translating, or creating derivative works based on the Software without authorization
        4. Sharing, transmitting, or disclosing the Software or any information contained therein to any third party
        5. Using the Software for any purpose other than authorized CloudZero business activities
        6. Removing, altering, or obscuring any proprietary notices, labels, or marks on the Software
        
        OWNERSHIP
        
        CloudZero retains all right, title, and interest in and to the Software, including all intellectual property rights therein. No ownership rights are transferred to users of the Software.
        
        TERMINATION
        
        This license terminates automatically upon:
        - Termination of employment or contractor relationship with CloudZero
        - Violation of any term of this license
        - Written notice from CloudZero
        
        Upon termination, all access to the Software must cease immediately, and all copies of the Software must be returned or destroyed.
        
        DATA PROTECTION
        
        The Software may process confidential customer data and proprietary business information. Users must comply with all applicable CloudZero data protection policies and procedures.
        
        COMPLIANCE
        
        Use of this Software is subject to all applicable CloudZero policies, including but not limited to:
        - Information Security Policy
        - Code of Conduct
        - Data Governance Policy
        - Intellectual Property Policy
        
        DISCLAIMER
        
        THE SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. CLOUDZERO DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
        
        LIMITATION OF LIABILITY
        
        IN NO EVENT SHALL CLOUDZERO BE LIABLE FOR ANY DAMAGES ARISING FROM THE USE OF THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO DIRECT, INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL, OR PUNITIVE DAMAGES.
        
        GOVERNING LAW
        
        This license shall be governed by and construed in accordance with the laws of the State of Delaware, without regard to conflict of law principles.
        
        CONTACT
        
        For questions regarding this license or to request permission for uses not expressly permitted herein, contact:
        
        CloudZero, Inc.
        Legal Department
        Email: legal@cloudzero.com
        
        By accessing or using this Software, you acknowledge that you have read, understood, and agree to be bound by the terms of this license.
        
        Last Updated: December 2024
License-File: LICENSE
Keywords: cloudzero,csv,data-analysis,telemetry
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: Utilities
Requires-Python: >=3.12
Requires-Dist: polars>=1.30.0
Requires-Dist: python-dateutil>=2.8.2
Requires-Dist: pytz>=2024.2
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=14.0.0
Provides-Extra: build
Requires-Dist: build>=1.0.0; extra == 'build'
Requires-Dist: twine>=5.0.0; extra == 'build'
Provides-Extra: dev
Requires-Dist: pytest-cov>=6.1.1; extra == 'dev'
Requires-Dist: pytest>=8.3.5; extra == 'dev'
Requires-Dist: ruff>=0.11.11; extra == 'dev'
Requires-Dist: s3pypi>=0.11.0; extra == 'dev'
Description-Content-Type: text/markdown

# CloudZero Telemetry Data Tool (czdt)

**CONFIDENTIAL & PROPRIETARY** - Internal CloudZero Research Project

A high-performance data analysis tool for processing and analyzing large-scale CSV datasets, with specialized features for CloudZero's telemetry platform integration.

**WARNING: INTERNAL USE ONLY** - This codebase contains proprietary CloudZero technology and is strictly confidential. Not for external distribution.

## Project Overview

CloudZero Data Tool (czdt) is a specialized CLI application for analyzing large-scale CSV datasets. This tool provides high-performance data processing capabilities for CloudZero's telemetry data ingestion and cost attribution workflows. The tool efficiently handles massive datasets (84M+ records) to generate business insights for our platform.

## Features

### Core Analysis Capabilities
- **High-Performance Data Processing**: Built with Polars for efficient handling of massive datasets (84M+ records)
- **Flexible Summary Analysis**: Custom aggregations with support for count, sum, avg operations
- **Time-Based Grouping**: Automatic datetime detection with hourly/daily grouping capabilities
- **Column Analysis**: Detailed statistics, value counts, and data quality insights
- **Interactive CLI**: Rich command-line interface with beautiful progress bars and table visualizations

### Performance Optimizations
- **Lazy Evaluation**: Memory-efficient processing with deferred computation
- **Column Selection**: Load only required columns to reduce memory usage
- **Schema Caching**: JSON manifest system for faster subsequent loads
- **Column Analysis Caching**: Explore mode caches analysis results for instant subsequent column selections (~15,000x speedup)
- **Sampling**: Process subsets of large datasets for quick analysis
- **Fast Mode**: Enable all optimizations automatically for maximum performance

### Data Export & Integration
- **CSV Export**: Export summary results to CSV files for further analysis
- **CloudZero Telemetry Integration**: Native support for our Telemetry API specifications
- **Rate Limit Compliance**: Built-in handling for CloudZero's 100 records/second limits
- **Batch Processing**: Optimized for 5MB request sizes (~10,000 records)

### Data Quality & Filtering
- **Null Value Handling**: Smart filtering of rows with missing data
- **Data Type Detection**: Automatic schema inference with datetime pattern recognition
- **Error Handling**: Robust parsing with fallback mechanisms
- **Data Validation**: Comprehensive error checking and user feedback

### Enterprise Development
- **Python 3.12**: Latest Python version for optimal performance
- **Modern Tooling**: uv package manager, pytest testing, ruff linting
- **Comprehensive Testing**: 50+ unit tests covering all functionality
- **Type Safety**: Full type hints and static analysis

## Data Analysis Capabilities

### Business Intelligence
- **Revenue Analytics**: Cost patterns, pricing optimization, customer lifetime value
- **Performance Metrics**: Latency analysis, model efficiency, resource utilization
- **Customer Segmentation**: Usage patterns by API key, behavior analysis
- **Search Analytics**: Query pattern optimization for our platform
- **Internal Cost Attribution**: CloudZero internal vs external usage breakdown
- **Time-Series Analysis**: Hourly and daily trending with automatic datetime detection

### CloudZero Platform Integration
- **Telemetry API Formatting**: Ready for direct ingestion into our platform
- **Rate Limit Optimization**: Built-in handling for our 100 records/second limits
- **Batch Processing**: Optimized for 5MB request sizes (~10,000 records)
- **Error Handling**: Retry logic for 503 "slow down" responses

## Installation & Setup

### Prerequisites
- Python 3.12+ (Latest version for optimal performance)
- [uv](https://github.com/astral-sh/uv) package manager
- Access to CloudZero internal systems

### Quick Start
```bash
# Clone from CloudZero GitHub
git clone https://github.com/Cloudzero/research-telemetry-data-tool.git
cd research-telemetry-data-tool

# Install dependencies
uv sync

# Build and install the package
./scripts/build.sh

# Verify installation
czdt --help
```

### Build Instructions

The project uses `uv` for fast, reliable builds:

```bash
# Build the package (creates wheel and tar.gz)
uv build

# This will:
# 1. Create wheel and source distributions
# 2. Handle dependencies automatically
# 3. Generate optimized packages
```

**Build outputs:**
- `dist/cloudzero_telemetry_data_tool-X.X.X-py3-none-any.whl` (wheel package)
- `dist/cloudzero_telemetry_data_tool-X.X.X.tar.gz` (source distribution)

### Package Publishing

For internal CloudZero distribution:

```bash
# First, build the package
uv build

# Publish to internal PyPI (requires AWS credentials)
s3pypi upload --bucket pypi.cloudzero.com --force --index.html --s3-put-args='ACL=public-read' --profile cz-prime.s3pypi-deploy dist/*

# Install from internal PyPI
pip install --index-url https://pypi.cloudzero.com/ cloudzero-telemetry-data-tool
```

**AWS Profile Requirements:**
- Use the `cz-prime.s3pypi-deploy` AWS profile for publishing
- This profile has write access to the `pypi.cloudzero.com` S3 bucket in the `cz-prime` AWS account
- Contact DevOps if you need access to this AWS profile

**Automated Build & Publish:**
```bash
# Use the automated script (handles build, test, lint, and publish)
python scripts/build_and_publish.py
```
The script will prompt you for your AWS profile name and handle the complete build and publish workflow.

### Execution Methods

You can run czdt in three ways:

#### Method 1: CLI Command (Recommended)
```bash
# Use the installed czdt command directly
czdt --in data/file.csv --explore --column MODEL_SKU
czdt --in data/file.csv --summarize "MODEL:count(*)"
```

#### Method 2: Using uv run (Alternative)
```bash
# No need to activate virtual environment
uv run python src/main.py --in data/file.csv --column MODEL_SKU
```

#### Method 3: Direct execution (Development)
```bash
# Activate virtual environment once
source .venv/bin/activate

# Then run commands directly
python src/main.py --in data/file.csv --explore --column MODEL_SKU
python src/main.py --in data/file.csv --summarize "MODEL:count(*)"
```

**When to use each:**
- **czdt command**: Fastest and most convenient, works from any directory
- **uv run**: Ensures correct environment, good for scripts/CI/CD
- **Direct execution**: Development and debugging

### Extending Analysis with Claude Code

For advanced analysis and custom data exploration, install and use Claude Code:

1. **Install Claude Code**: Follow the installation guide at [docs.anthropic.com/claude-code](https://docs.anthropic.com/claude-code)
2. **Launch in project**: Run `claude-code` in the project directory
3. **AI-Powered Analysis**: Use Claude Code to:
   - Generate custom analysis scripts for specific customer data patterns
   - Create advanced visualizations and reports
   - Develop new aggregation and filtering capabilities
   - Optimize performance for large dataset processing
   - Build custom CloudZero telemetry formatters

**Example Claude Code prompts:**
- "Analyze customer usage patterns in the customer-*.csv files and identify cost optimization opportunities"
- "Create a new analysis script that compares search vs non-search query costs by customer segment"
- "Generate a telemetry formatter that batches data according to CloudZero API rate limits"

## CLI Reference

### Required Arguments
- `--in FILE`: Input CSV file path (required for all operations)

### Analysis Options
- `--explore`: Interactive column exploration - browse and analyze columns one by one with detailed insights. First analysis builds cache for instant subsequent selections.
- `--column NAME`: Start exploration with specific column (requires `--explore`)
- `--summarize SPEC`: Create pivot table/summary analysis with custom aggregations
- `--analyze`: Interactive CloudZero telemetry mapping analysis to identify how CSV data maps to telemetry records

### Summarize Analysis Format
```
--summarize "GROUP_COLS:AGGREGATIONS"

GROUP_COLS: Comma-separated columns to group by
AGGREGATIONS: Comma-separated operations like count(COL), sum(COL), avg(COL)

Example: "MODEL_SKU,RUNTIME_MODEL_NAME:count(LLM_INPUT_TOKENS),sum(CT_TOTAL_COMPLETION_TOKENS)"
```

### Time-Based Grouping
- `--group-by {hour,day}`: Group summary by time period (requires `--summarize`)
- `--datetime-col COLUMN`: Specify datetime column for time grouping (auto-detected if not provided)

### Data Filtering  
- `--allow-none`: Include rows with null values in summary results (default: exclude nulls)
- `--sample N`: Process only N sample rows for faster analysis on large datasets

### Performance Options
- `--fast`: Enable all performance optimizations (recommended for files >100MB)
- `--lazy`: Use lazy loading for memory efficiency (useful for files >1GB)
- `--summarize-limit N`: Maximum number of summary rows to display (default: show all)

### Output Options
- `--out FILE`: Export results to CSV file instead of displaying on screen (summarize mode only)

### CloudZero Telemetry Analysis
- `--analyze`: CloudZero telemetry analysis - identify key columns for cost allocation and unit metrics
- `--costformation FILE`: Path to costformation.yaml file for cost dimension suggestions
- `--config FILE`: Configuration file (JSON/YAML) with predefined telemetry mapping answers for automation

### Schema Management
- `--generate-schema FOLDER`: Generate schema manifest for all CSV files in specified folder (speeds up future loads)

## Usage Examples

### Basic Analysis
```bash
# Interactive column exploration (recommended for data discovery)
czdt --in data/customer-pplx_api_usage_april.csv --explore

# Analyze specific column
czdt --in data/customer-pplx_api_usage_april.csv --explore --column MODEL_SKU

# Show all available options
czdt --help

# Alternative: using uv run
uv run python src/main.py --in data/customer-pplx_api_usage_april.csv --explore
```

### Summarize Analysis
```bash
# Simple summary: count by model
czdt --in data/file.csv --summarize "MODEL_SKU:count(LLM_INPUT_TOKENS)"

# Complex summary with multiple aggregations
czdt --in data/file.csv --summarize "MODEL_SKU,RUNTIME_MODEL_NAME:count(LLM_INPUT_TOKENS),sum(CT_TOTAL_COMPLETION_TOKENS),avg(TOTAL_TIME)"

# Time-based analysis: daily grouping
czdt --in data/file.csv --summarize "MODEL_SKU:sum(TOTAL_COST)" --group-by day

# Include null values (they are excluded by default)
czdt --in data/file.csv --summarize "MODEL_SKU:count(*)" --allow-none
```

### Performance Optimization
```bash
# Fast mode for large datasets
czdt --in data/large_file.csv --summarize "MODEL:count(*)" --fast

# Sample analysis for quick insights
czdt --in data/large_file.csv --summarize "MODEL:count(*)" --sample 10000

# Lazy loading for memory efficiency
czdt --in data/large_file.csv --summarize "MODEL:count(*)" --lazy
```

### CSV Export
```bash
# Export summary results to CSV
czdt --in data/file.csv --summarize "MODEL_SKU:count(*),sum(TOTAL_COST)" --out results.csv

# Export with time grouping
czdt --in data/file.csv --summarize "MODEL:sum(COST)" --group-by day --out daily_analysis.csv
```

### Business Intelligence Examples
```bash
# Customer segmentation analysis
czdt --in data/file.csv --summarize "API_KEY_SUFFIX:sum(TOTAL_COST),count(*),avg(TOTAL_TIME)"

# Model performance comparison
czdt --in data/file.csv --summarize "RUNTIME_MODEL_NAME:avg(TOTAL_TIME),avg(TOTAL_DECODING_LATENCY)"

# Search cost optimization
czdt --in data/file.csv --summarize "NUM_SEARCH_QUERIES:sum(TOTAL_COST),avg(TOTAL_TIME)"

# Internal vs external usage
czdt --in data/file.csv --explore --column IS_PERPLEXITY_INTERNAL
```

### CloudZero Telemetry Analysis
```bash
# Interactive telemetry mapping analysis
czdt --in data/file.csv --analyze

# With costformation.yaml for dimension suggestions
czdt --in data/file.csv --analyze --costformation costformation.yaml

# Automated analysis with configuration file
czdt --in data/file.csv --analyze --config telemetry_config.json

# The analysis will:
# 1. Ask if you want allocation or unit cost telemetry
# 2. Analyze temporal columns for HOURLY/DAILY granularity
# 3. Interactive temporal selection:
#    - Choose which date/time column to use
#    - Select granularity (HOURLY or DAILY) with usage recommendations
#    - Show final temporal configuration summary
# 4. Present column candidates for:
#    - element-name (single column or composite from multiple columns)
#      * Top 10 string columns by cardinality, excluding datetime columns
#      * Option to create composite element-names by combining 2+ columns
#      * Configurable column order and separator for composite names
#    - value (numeric data)
#    - filters (allocation) or associated-costs (unit cost)
# 5. Generate 5 sample JSON telemetry records
# 6. Save customer_analysis_report_<filename>.json for future use
```

#### Telemetry Configuration Format
Create JSON or YAML files to automate analysis:

**Single Element-Name:**
```json
{
  "telemetry_type": "allocation",
  "selections": {
    "element_name": "service_name",
    "value": "request_count", 
    "timestamp": "timestamp",
    "granularity": "daily",
    "filters": ["customer_id", "environment", "region"]
  }
}
```

**Composite Element-Name:**
```json
{
  "telemetry_type": "allocation",
  "selections": {
    "element_name": {
      "columns": ["service_name", "endpoint_path"],
      "separator": "|"
    },
    "value": "request_count",
    "timestamp": "timestamp",
    "granularity": "hourly",
    "filters": ["customer_id", "environment", "region"]
  }
}
```

**YAML Format:**
```yaml
telemetry_type: unit
selections:
  element_name:
    columns: ["model_name", "customer_tier"]
    separator: "|"
  value: request_count
  timestamp: timestamp
  granularity: daily
  associated_costs:
    - cost_usd
    - response_time_ms
```

### CloudZero Telemetry Preparation
```bash
# Cost attribution by model for telemetry ingestion
czdt --in data/file.csv --summarize "MODEL_SKU:sum(TOTAL_COST),count(TOTAL_COST)" --out telemetry_data.csv

# Time-based telemetry analysis
czdt --in data/file.csv --summarize "MODEL_SKU:sum(TOTAL_COST)" --group-by hour --out hourly_telemetry.csv
```

## Project Structure

```
├── data/                                    # Confidential datasets (gitignored)
│   ├── customer-pplx_api_usage_april.csv  # Primary Perplexity dataset (84M+ records)
│   ├── manifest.json                      # Schema cache for fast loading
│   ├── C0909AE5-0FB6-449F-A109-81570AEF79FE.png  # Analysis screenshot
│   └── customer-*.csv                     # Additional customer data files
├── src/                                   # Core analysis engine
│   └── main.py                           # Main analysis script with Rich UI (1200+ lines)
├── tests/                                # Comprehensive test suite
│   ├── __init__.py
│   ├── test_analyze_data.py             # Original unit tests
│   ├── test_analyze_data_comprehensive.py  # Extended test coverage (50+ tests)
│   └── data/                            # Test datasets
│       ├── mini_test_data.csv           # 5-row minimal test data
│       ├── test_pplx_data.csv          # 10-row comprehensive test data
│       └── manifest.json                # Test schema manifest
├── .gitignore                           # Git exclusions (customer data, system files)
├── .python-version                      # Python 3.12 version pinning
├── CLAUDE.md                            # CloudZero development standards & AI attribution
├── LICENSE                              # Proprietary CloudZero license
├── pyproject.toml                       # Project configuration and dependencies
├── README.md                            # This documentation
└── uv.lock                              # Dependency lock file
```

## CloudZero Integration Specifications

### Telemetry API Compliance
- **Rate Limits**: 100 records/second, 8.64M records/day maximum
- **Request Size**: 5MB maximum (~10,000 records of 512 bytes)
- **Operations**: POST (create), SUM (aggregate), REPLACE (update), DELETE
- **Error Handling**: Exponential backoff for 503 responses
- **Authentication**: CloudZero API key integration

### Data Pipeline Integration
- **Allocation Telemetry**: Customer service request tracking
- **Unit Metric Telemetry**: Feature-level database query monitoring
- **Cost Attribution**: Multi-dimensional cost breakdowns
- **Real-time Processing**: Stream processing capabilities for live data

## Development Standards

### CloudZero Code Quality
```bash
# Using uv run (recommended for CI/CD)
uv run pytest
uv run ruff check
uv run ruff format

# OR with activated venv (faster for development)
source .venv/bin/activate
pytest
ruff check
ruff format
```

## Performance Optimization Guide

### Large Dataset Processing
For files > 1GB, the tool automatically enables optimizations:
- **Lazy Loading**: Defers computation until necessary
- **Column Selection**: Only loads columns needed for analysis
- **Schema Caching**: Reuses column types from manifest.json
- **Progress Tracking**: Real-time feedback during processing

### Memory Management
```bash
# Enable all optimizations automatically
czdt --in large_file.csv --summarize "MODEL:count(*)" --fast

# Manual memory optimization
czdt --in large_file.csv --summarize "MODEL:count(*)" --lazy

# Sample for quick analysis (process subset)
czdt --in large_file.csv --summarize "MODEL:count(*)" --sample 50000
```

### Schema Management
The tool automatically generates and uses schema manifests for faster loading:
```bash
# Generate schema manifest for directory
czdt --generate-schema data/

# Manifest is automatically used when available
# Located at data/manifest.json
```

## CSV Export Features

### Export Capabilities
- **Full Results**: All summary combinations exported (no display limits)
- **Clean Headers**: Proper column naming (e.g., `COUNT_LLM_INPUT_TOKENS`)
- **Large Dataset Support**: Efficient export of millions of records
- **Time Grouping**: Export includes datetime grouping columns

### Export Examples
```bash
# Simple export
czdt --in data/file.csv --summarize "MODEL:count(*)" --out model_analysis.csv

# Complex export with time grouping
czdt --in data/file.csv \
  --summarize "MODEL_SKU,RUNTIME_MODEL_NAME:count(LLM_INPUT_TOKENS),sum(CT_TOTAL_COMPLETION_TOKENS)" \
  --group-by day --allow-none --out daily_model_analysis.csv

# Performance optimized export
czdt --in data/large_file.csv \
  --summarize "MODEL:sum(TOTAL_COST)" --fast --out cost_analysis.csv
```

### Dependencies
- **polars**: High-performance DataFrame operations
- **rich**: Terminal UI and progress visualization
- **pytest**: CloudZero testing standard
- **ruff**: Fast linting and formatting

## Sample Business Insights

The tool generates comprehensive reports including:

- **Time-Based Trends**: Hourly/daily usage patterns and cost distribution
- **Data Quality**: Null value analysis and data completeness metrics

## Quick Reference

### Most Common Commands
```bash
# Interactive exploration (recommended for discovery)
czdt --in data/file.csv --explore

# Specific column analysis
czdt --in data/file.csv --explore --column MODEL_SKU

# Multiple commands using czdt
czdt --in data/file.csv --summarize "MODEL:count(*)"
czdt --in data/file.csv --summarize "MODEL:count(*)" --out results.csv
czdt --in data/file.csv --summarize "MODEL:sum(COST)" --group-by day
```

### Performance Tips
- Use `--fast` for files > 1GB
- Use `--sample N` for quick insights on large datasets
- Files are automatically optimized when > 1GB
- Schema manifests are automatically generated and reused

## Security & Compliance

- **Data Classification**: CONFIDENTIAL - Contains customer usage patterns
- **Access Control**: CloudZero employees only with business need
- **Storage**: Encrypted at rest, secure CloudZero infrastructure only
- **Transmission**: HTTPS/TLS for all data transfers
- **Retention**: Follow CloudZero data governance policies

### Customer Data Handling

**WARNING: CRITICAL** - Customer confidential data must NEVER be checked into GitHub.

- **Source Location**: All customer data for this project is stored in `/Accounts/<customer name>` Google Shared Drive
- **Local Analysis**: Customer data may be copied to the `data/` folder for analysis and experimentation
- **Protection Mechanism**: Prefix all customer data files with `customer-` (e.g., `customer-pplx_usage.csv`)
- **Git Ignore**: Files prefixed with `customer-` are automatically ignored by git and will not be pushed to GitHub
- **Verification**: Always verify customer data files are properly prefixed before running git commands

**Example:**
```bash
# CORRECT: Customer data with proper prefix
data/customer-pplx_api_usage_april.csv
data/customer-acme_corp_analysis.json

# WRONG: Customer data without prefix - WILL BE COMMITTED TO GITHUB
data/pplx_api_usage_april.csv
data/acme_corp_analysis.json
```

## Internal Collaboration

### For CloudZero Team Members
1. Create feature branch from `main`
2. Follow CloudZero coding standards
3. Run full test suite before commits
4. Request code review from team lead
5. Merge only after approval and CI/CD success

### Questions & Support
- **Slack**: #octo

## Roadmap & Future Enhancements

- [ ] Real-time streaming data integration
- [ ] Advanced ML model performance prediction
- [ ] Automated cost optimization recommendations
- [ ] CloudZero dashboard integration
- [ ] Customer churn prediction analytics

---

**Copyright 2025 CloudZero, Inc. - All Rights Reserved**  
This software and documentation are proprietary to CloudZero and contain confidential and trade secret information. Unauthorized use, reproduction, or distribution is strictly prohibited.