.ai-context

# AI Assistant Context for UTF Strings Library

## Project Overview
This is a high-performance UTF-8/16/32 string processing library written in modern C++23. The library provides cross-platform support for Unicode string manipulation with comprehensive testing and benchmarking.

## Key Information for AI Assistants

### Project Structure
- **Language**: C++23 (requires GCC 13+, Clang 18+, MSVC 2022+)
- **Build System**: CMake 3.25+ with Conan package manager
- **Testing**: Google Test framework with extensive unit tests
- **Benchmarking**: Google Benchmark for performance measurement
- **Fuzzing**: libFuzzer for security testing
- **CI/CD**: GitHub Actions with comprehensive workflows

### Important Directories
- `include/utf/`: Public header files (main API)
- `src/`: Implementation source files
- `tests/`: Unit tests using Google Test
- `benchmarks/`: Performance benchmarks
- `fuzz/`: Fuzz testing harnesses
- `.github/workflows/`: CI/CD workflows
- `docs/`: Documentation and performance reports

### Coding Standards
- **C++ Standard**: C++23 with no extensions
- **Formatting**: clang-format (enforced in CI)
- **Naming**: snake_case for functions/variables, PascalCase for types
- **Headers**: Use #pragma once, include guards for compatibility
- **Namespaces**: All code in `utf` namespace
- **License**: BSD 2-Clause License, Copyright (c) 2025 William Sollers

### License Header Requirements
All new C++ files (.cpp, .hpp, .h) must include this license header at the top:
```cpp
/*
 * Copyright (c) 2025 William Sollers
 * 
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 * 
 * 1. Redistributions of source code must retain the above copyright notice, this
 *    list of conditions and the following disclaimer.
 * 
 * 2. Redistributions in binary form must reproduce the above copyright notice,
 *    this list of conditions and the following disclaimer in the documentation
 *    and/or other materials provided with the distribution.
 * 
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
 * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
 * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */
```

### Build Commands
```bash
# Quick setup (Debug)
./bootstrap_cmake.sh --compiler gcc --config Debug

# Release build
./bootstrap_cmake.sh --compiler gcc --config Release

# With fuzzing (Clang only)
./bootstrap_cmake.sh --compiler clang --config Debug --enable-fuzz

# Run tests
cd build/build && ./utf_strings-tests

# Run benchmarks
cd build/build && ./utf_strings-bench
```

### Key Files to Understand
- `CMakeLists.txt`: Main build configuration
- `conanfile.py`: Dependency management
- `include/utf/utf_codepoints.hpp`: Main library header
- `include/utf/version.hpp`: Version information
- `src/utf_codepoints.cpp`: Core implementation

### Common Tasks
1. **Adding new UTF conversion functions**: Update both header and implementation
2. **Adding tests**: Create test files in `tests/` directory
3. **Performance optimization**: Add benchmarks in `benchmarks/`
4. **Security testing**: Add fuzz targets in `fuzz/`

### Dependencies
- **Build**: Conan, CMake, C++23 compiler
- **Testing**: GTest (via Conan)
- **Benchmarking**: Google Benchmark (via Conan)
- **Optional**: gperftools for profiling

### Platform Support
- **Linux**: GCC 13+, Clang 18+ (primary development)
- **Windows**: MSVC 2022, Clang-CL (CI tested)

### 🚨 MANDATORY CODE REVIEW REQUIREMENTS

**CRITICAL**: All code must undergo comprehensive review before every push to origin using these parameters:

#### **Code Review Parameters**

1. **🔒 Security Analysis** (HIGHEST PRIORITY)
   - [ ] Check for undefined behavior (UB) - memory safety violations
   - [ ] Validate memory safety - no dangling pointers, use-after-free
   - [ ] Look for buffer overflows and underflows
   - [ ] Check for integer overflows/underflows
   - [ ] Assess input validation (bounds checking, null checks)
   - [ ] Verify proper error handling for security-critical paths

2. **⚡ Performance Analysis**
   - [ ] Evaluate algorithmic complexity (Big-O analysis)
   - [ ] Check for unnecessary allocations and copies
   - [ ] Identify performance bottlenecks
   - [ ] Verify move semantics used appropriately
   - [ ] Check for redundant temporary objects

3. **🐛 Correctness Issues**
   - [ ] Identify bugs and logic errors
   - [ ] Check edge cases (empty inputs, max values, boundary conditions)
   - [ ] Validate error handling (exceptions, optional returns)
   - [ ] Ensure proper initialization of all variables
   - [ ] Check const correctness and immutability

4. **🚀 C++ Core Guidelines & Modern C++23**
   - [ ] Validate RAII usage (Resource Acquisition Is Initialization)
   - [ ] Check exception safety (basic/strong/no-throw guarantee)
   - [ ] Verify proper use of `[[nodiscard]]`, `noexcept` attributes
   - [ ] Check for appropriate use of `std::optional`, `std::variant`
   - [ ] Assess template metaprogramming best practices

5. **🏗️ Design Issues**
   - [ ] Evaluate API design consistency
   - [ ] Check naming conventions consistency
   - [ ] Assess abstraction levels and encapsulation
   - [ ] Review for unnecessary complexity or over-engineering
   - [ ] Verify dependency management

6. **📚 Documentation & Testing**
   - [ ] Check for adequate inline comments
   - [ ] Verify API documentation completeness
   - [ ] Ensure comprehensive unit test coverage
   - [ ] Check for edge case testing
   - [ ] Validate error path testing

#### **Review Output Requirements**

**Categorize ALL findings by severity:**
- 🔴 **Critical** - Must fix before production (security, UB, crashes)
- 🟡 **Important** - Should fix (performance, correctness, maintainability)  
- 🟢 **Nice to have** - Optional improvements (style, minor optimizations)

**Provide Score Card:**
Rate each category (A+ to F) with overall grade and production readiness assessment.

**For Critical Issues:**
- Provide exact code fixes
- Explain the issue clearly
- Show before/after code examples

#### **Key Focus Areas for UTF Strings Library:**
- **Production readiness** - Code must be deployable
- **Security** - Especially UB, overflow, memory safety in UTF processing
- **Performance optimization** - UTF processing efficiency is critical
- **Modern C++23 practices** - Leverage language features appropriately
- **Cross-platform compatibility** - Must work on Linux/Windows/macOS

**Standard**: Every change must have ZERO 🔴 Critical issues before merging.
- **Architectures**: x64 (primary), others untested

### Performance Considerations
- Focus on zero-copy operations where possible
- SIMD optimization opportunities (AVX2, NEON)
- Memory alignment considerations for UTF-32
- Branch prediction optimization for common cases

### Security Focus
- Input validation for all UTF conversion functions
- Buffer overflow prevention
- Fuzz testing for all encoding variants
- Static analysis with multiple tools (Cppcheck, Clang-tidy, etc.)

### Code Quality Requirements
- **Modern C++23 Idioms**: All code must use idiomatic modern C++23 patterns
- **No Legacy C++**: Avoid C++98/03/11 patterns; prefer C++23 alternatives
- **Comprehensive Testing**: Every new function/class requires unit tests
- **Test-First Development**: Write tests alongside or before implementation
- **100% Test Pass Rate**: All tests must pass before pushing to remote
- **CI Validation**: Code must pass all CI checks (tests, formatting, security scans)

### Git Workflow
- **Main branch**: `main` (protected)
- **Releases**: Tagged as `v*.*.* ` (semantic versioning)
- **CI**: All PRs must pass comprehensive CI including tests, formatting, security scans
- **Commits**: Use conventional commit format (feat:, fix:, docs:, etc.)

## Helpful Context for Code Generation

When suggesting code:
1. **Always use C++23 features** when beneficial - prefer modern idioms over legacy patterns
2. **Write idiomatic modern C++23** - use concepts, ranges, constexpr, auto, structured bindings
3. **Include comprehensive unit tests** - every new function/class must have corresponding tests
4. **Ensure all tests pass** - validate that suggested tests actually work and cover edge cases
5. **Include proper error handling** and input validation with modern exception safety
6. **Consider endianness** for multi-byte encodings (UTF-16/32) using the project's endian policy
7. **Consider performance implications** - this is a performance-focused library with benchmarking
8. **Follow existing patterns** in the codebase for consistency
9. **Include appropriate documentation** comments using Doxygen style
10. **Validate before suggesting** - ensure code compiles and tests pass before recommending

## Critical Requirements for AI Assistants

### C++23 Modern Idioms Required
- Use `auto` for type deduction where appropriate
- Prefer `constexpr` and `consteval` for compile-time computation
- Use concepts for template constraints instead of SFINAE
- Leverage ranges and views for algorithms
- Use structured bindings for multiple return values
- Prefer `std::optional` and `std::expected` for error handling
- Use `std::string_view` and `std::span` for non-owning references

### Testing Requirements (NON-NEGOTIABLE)
- **Every new function requires unit tests** - no exceptions
- **Tests must cover**: normal cases, edge cases, error conditions, invalid inputs
- **Tests must pass** before any code is considered complete
- **Use Google Test framework** following existing patterns
- **Include performance tests** for critical path functions
- **Consider fuzz testing** for input parsing functions

### Pre-Push Validation Checklist
**MANDATORY: Before any code can be pushed to remote, ALL of the following MUST pass:**

#### Build Validation (REQUIRED)
1. ✅ **Full clean build succeeds** (`./bootstrap_cmake.sh --compiler clang --config Debug`)
2. ✅ **No compiler warnings or errors** in Debug configuration
3. ✅ **Release build also succeeds** (if making performance-critical changes)
4. ✅ **All CMake configuration completes successfully**

#### Testing Validation (REQUIRED)  
5. ✅ **All unit tests pass** (`./utf_strings-tests` - 100% pass rate required)
6. ✅ **All benchmarks run successfully** (`./utf_strings-bench`)
7. ✅ **Performance tests show no significant regression** (>10% slowdown requires justification)
8. ✅ **Fuzz tests pass** for any input parsing code (if applicable)

#### Code Quality Validation (REQUIRED)
9. ✅ **Code formatting is correct** (`cmake --build --preset=conan-debug --target format-check`)
10. ✅ **Static analysis passes** (will be validated in CI but check locally if possible)
11. ✅ **No memory leaks** in tests (use sanitizers: `--enable-sanitizers`)

#### Final Checks (REQUIRED)
12. ✅ **Git status is clean** (no untracked files, staged changes committed)
13. ✅ **Commit message follows conventional format** (feat:, fix:, docs:, etc.)
14. ✅ **All new C++ files have proper license headers** (BSD 2-Clause with William Sollers copyright)

**❌ DO NOT PUSH if ANY of these checks fail. Fix all issues before pushing to remote.**

## Current Version
- Version: 0.0.2 (performance monitoring release)
- See `include/utf/version.hpp` for version constants

## Questions to Ask
When helping with this project, consider asking:
- What UTF encodings are involved?
- Are there performance requirements?
- Should this handle invalid sequences gracefully?
- Is this for a specific platform or cross-platform?
- Are there security considerations?