wip: [01-stabilize] paused at task 1/1 - OCR Hallucination Immune logic via Semantic delta window and fret-isolation
This commit is contained in:
361
.agent/knowledge/awesome_claude/scripts/README.md
Normal file
361
.agent/knowledge/awesome_claude/scripts/README.md
Normal file
@@ -0,0 +1,361 @@
|
||||
# Scripts Directory
|
||||
|
||||
This directory contains all automation scripts for managing the Awesome Claude Code repository. The scripts work together to provide a complete workflow for resource management, from addition to pull request submission.
|
||||
|
||||
**Important Note**: While the primary submission workflow has moved to GitHub Issues for better user experience, we maintain these manual scripts for several critical purposes:
|
||||
- **Backup submission method** when the automated Issues workflow is unavailable
|
||||
- **Administrative tasks** requiring direct CSV manipulation
|
||||
- **Testing and debugging** the automation pipeline
|
||||
- **Emergency recovery** when automated systems fail
|
||||
|
||||
|
||||
## Overview
|
||||
|
||||
The scripts implement a CSV-first workflow where `THE_RESOURCES_TABLE.csv` serves as the single source of truth for all resources. The README.md is generated from this CSV data using templates.
|
||||
|
||||
## Repo Root Resolution
|
||||
|
||||
Scripts should never assume the current working directory or rely on fragile parent traversal. Use repo-root discovery (walk up to `pyproject.toml`) and resolve paths from there. File paths should be built from `REPO_ROOT` (e.g., `REPO_ROOT / "THE_RESOURCES_TABLE.csv"`).
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
```
|
||||
|
||||
### Imports and working directory
|
||||
|
||||
Most scripts import modules as `scripts.*`. Those imports resolve reliably when:
|
||||
- you run from the repo root (default for local usage and GitHub Actions), or
|
||||
- you set `PYTHONPATH` to the repo root, or
|
||||
- you use `python -m` with the package path.
|
||||
|
||||
If a script fails with `ModuleNotFoundError: scripts`, run it from the repo root or set `PYTHONPATH`.
|
||||
|
||||
### Running scripts with `python -m`
|
||||
|
||||
When invoking scripts, prefer module paths (dot notation) and omit the `.py` suffix:
|
||||
|
||||
```bash
|
||||
python -m scripts.readme.generate_readme
|
||||
python -m scripts.validation.validate_links
|
||||
```
|
||||
|
||||
This only works for modules with a CLI entrypoint (`if __name__ == "__main__":`).
|
||||
|
||||
## Directory Structure
|
||||
|
||||
- `badges/` - Badge notification automation (core + manual)
|
||||
- `categories/` - Category tooling and config helpers
|
||||
- `graphics/` - Logo and branding SVG generation
|
||||
- `ids/` - Resource ID generation utilities
|
||||
- `maintenance/` - Repo chores and maintenance scripts
|
||||
- `testing/` - Test and integration utilities (including `test_regenerate_cycle.py`)
|
||||
- `archive/` - Temporary holding area for deprecated scripts
|
||||
- `readme/` - README generation pipeline, generators, helpers, markup, SVG templates
|
||||
- `resources/` - Resource submission, sorting, and CSV utilities
|
||||
- `ticker/` - Repo ticker data fetch + SVG generation
|
||||
- `utils/` - Shared git helpers
|
||||
- `validation/` - URL and submission validation scripts
|
||||
|
||||
## Category System
|
||||
|
||||
### `categories/category_utils.py`
|
||||
**Purpose**: Unified category management system
|
||||
**Usage**: `from scripts.categories.category_utils import category_manager`
|
||||
**Features**:
|
||||
- Singleton pattern for efficient data loading
|
||||
- Reads categories from `templates/categories.yaml`
|
||||
- Provides methods for category lookup, validation, and ordering
|
||||
- Used by all scripts that need category information
|
||||
|
||||
### Adding New Categories
|
||||
To add a new category:
|
||||
1. Edit `templates/categories.yaml` and add your category with:
|
||||
- `id`: Unique identifier
|
||||
- `name`: Display name
|
||||
- `prefix`: ID prefix (e.g., "cmd" for Slash-Commands)
|
||||
- `icon`: Emoji icon
|
||||
- `order`: Sort order
|
||||
- `description`: Markdown description
|
||||
- `subcategories`: Optional list of subcategories
|
||||
2. Update `.github/ISSUE_TEMPLATE/recommend-resource.yml` to add the category to the dropdown
|
||||
3. If subcategories were added, run `make generate-toc-assets` to create subcategory TOC SVGs
|
||||
4. Run `make generate` to update the README
|
||||
|
||||
All scripts automatically use the new category without any code changes.
|
||||
|
||||
## Automated Backend Scripts
|
||||
|
||||
These scripts power the GitHub Issues-based submission workflow and are executed automatically by GitHub Actions:
|
||||
|
||||
### `resources/parse_issue_form.py`
|
||||
**Purpose**: Parses GitHub issue form submissions and extracts resource data
|
||||
**Usage**: Called by `validate-resource-submission.yml` workflow
|
||||
**Features**:
|
||||
- Extracts structured data from issue body
|
||||
- Validates form field completeness
|
||||
- Converts form data to resource format
|
||||
- Provides validation feedback as issue comments
|
||||
|
||||
### `resources/create_resource_pr.py`
|
||||
**Purpose**: Creates pull requests from approved resource submissions
|
||||
**Usage**: Called by `approve-resource-submission.yml` workflow
|
||||
**Features**:
|
||||
- Generates unique resource IDs
|
||||
- Adds resources to CSV database
|
||||
- Creates feature branches automatically
|
||||
- Opens PR with proper linking to original issue
|
||||
- Handles pre-commit hooks gracefully
|
||||
|
||||
## Core Workflow Scripts (Manual/Admin Use)
|
||||
|
||||
### 1. `resources/resource_utils.py`
|
||||
**Purpose**: CSV append helpers and PR content generation
|
||||
**Usage**: Imported by `resources/create_resource_pr.py`
|
||||
**Notes**:
|
||||
- Keeps CSV writes aligned to header order
|
||||
- Generates standardized PR content for automated submissions
|
||||
|
||||
### 2. `readme/generate_readme.py`
|
||||
**Purpose**: Generates multiple README styles from CSV data using templates
|
||||
**Usage**: `make generate`
|
||||
**Features**:
|
||||
- Template-based generation from `templates/README_EXTRA.template.md` (and other templates)
|
||||
- Configurable root style via `acc-config.yaml`
|
||||
- Dynamic style selector and repo ticker via placeholders
|
||||
- Hierarchical table of contents generation
|
||||
- Preserves custom sections from template
|
||||
- Automatic backup before generation
|
||||
- **GitHub Stats Integration**: Automatically adds collapsible repository statistics for GitHub resources
|
||||
- Displays stars, forks, issues, and other metrics via GitHub Stats API
|
||||
- Uses disclosure elements (`<details>`) to keep the main list clean
|
||||
- Works with all GitHub URL formats (repository root, blob URLs, etc.)
|
||||
|
||||
#### Collapsible Sections
|
||||
The generated README uses collapsible `<details>` elements for better navigation:
|
||||
- **Categories WITHOUT subcategories**: Wrapped in `<details open>` (fully collapsible)
|
||||
- **Categories WITH subcategories**: Use regular headers (subcategories are collapsible)
|
||||
- **All subcategories**: Wrapped in `<details open>` elements
|
||||
- **Table of Contents**: Main wrapper and nested categories use `<details open>`
|
||||
|
||||
**Note on anchor links**: Initially, all categories were made collapsible, but this caused issues with anchor links from the Table of Contents - links couldn't navigate to subcategories when their parent category was collapsed. The current design balances navigation and collapsibility.
|
||||
|
||||
### 2a. `readme/helpers/generate_toc_assets.py`
|
||||
**Purpose**: Regenerates subcategory TOC SVG assets from `templates/categories.yaml`
|
||||
**Usage**: `make generate-toc-assets`
|
||||
**Features**:
|
||||
- Creates/updates `toc-sub-*.svg` and `toc-sub-*-light-anim-scanline.svg` files in `assets/`
|
||||
- Uses `regenerate_sub_toc_svgs()` from `readme_assets.py` with categories from `category_manager`
|
||||
- Should be run after adding or modifying subcategories in `templates/categories.yaml`
|
||||
- SVGs are used by the Visual (Extra) README style for subcategory TOC rows
|
||||
|
||||
### 2b. `ticker/generate_ticker_svg.py`
|
||||
**Purpose**: Generates animated SVG tickers showing featured projects
|
||||
**Usage**: `python scripts/ticker/generate_ticker_svg.py`
|
||||
**Features**:
|
||||
- Reads repo stats from `data/repo-ticker.csv`
|
||||
- Generates three ticker themes: dark (CRT), light (vintage), awesome (minimal)
|
||||
- Displays repo name, owner, stars, and daily delta
|
||||
- Seamless horizontal scrolling animation
|
||||
|
||||
### 2c. `ticker/fetch_repo_ticker_data.py`
|
||||
**Purpose**: Fetches GitHub statistics for repos tracked in the ticker
|
||||
**Usage**: `python scripts/ticker/fetch_repo_ticker_data.py`
|
||||
**Features**:
|
||||
- Queries GitHub API for stars, forks, watchers
|
||||
- Calculates deltas from previous run
|
||||
- Outputs to `data/repo-ticker.csv`
|
||||
- Requires `GITHUB_TOKEN` environment variable
|
||||
|
||||
### 4. `validation/validate_links.py`
|
||||
**Purpose**: Validates all URLs in the CSV database
|
||||
**Usage**: `make validate`
|
||||
**Features**:
|
||||
- Batch URL validation with progress bar
|
||||
- GitHub API integration for repository checks
|
||||
- License detection from GitHub repos
|
||||
- Last modified date fetching
|
||||
- Exponential backoff for rate limiting
|
||||
- Override support from `.templates/resource-overrides.yaml`
|
||||
- JSON output for CI/CD integration
|
||||
|
||||
### 5. `resources/download_resources.py`
|
||||
**Purpose**: Downloads resources from GitHub repositories
|
||||
**Usage**: `make download-resources`
|
||||
**Features**:
|
||||
- Downloads files from GitHub repositories
|
||||
- Respects license restrictions
|
||||
- Category and license filtering
|
||||
- Rate limiting support
|
||||
- Progress tracking
|
||||
- Creates organized directory structure
|
||||
|
||||
## Helper Modules
|
||||
|
||||
### 6. `utils/git_utils.py`
|
||||
**Purpose**: Git and GitHub utility functions
|
||||
**Interface**:
|
||||
- `get_github_username()`: Retrieves GitHub username
|
||||
- `get_current_branch()`: Gets active git branch
|
||||
- `create_branch()`: Creates new git branch
|
||||
- `commit_changes()`: Commits with message
|
||||
- `push_to_remote()`: Pushes branch to remote
|
||||
- GitHub CLI integration utilities
|
||||
|
||||
### 7. `utils/github_utils.py`
|
||||
**Purpose**: Shared GitHub API helpers
|
||||
**Interface**:
|
||||
- `parse_github_url()`: Parse GitHub URLs into API endpoints
|
||||
- `get_github_client()`: Pygithub client with request pacing
|
||||
- `github_request_json()`: JSON requests via PyGithub requester
|
||||
|
||||
### 8. `validation/validate_single_resource.py`
|
||||
**Purpose**: Validates individual resources
|
||||
**Usage**: `make validate-single URL=...`
|
||||
**Interface**:
|
||||
- `validate_single_resource()`: Validates URL and fetches metadata using kwargs
|
||||
- Used by issue submission validation and manual validation workflows
|
||||
- Supports both regular URLs and GitHub repositories
|
||||
|
||||
### 9. `resources/sort_resources.py`
|
||||
**Purpose**: Sorts CSV entries by category hierarchy
|
||||
**Usage**: `make sort` (called automatically by `make generate`)
|
||||
**Features**:
|
||||
- Maintains consistent ordering
|
||||
- Sorts by: Category → Sub-Category → Display Name
|
||||
- Uses category order from `categories.yaml`
|
||||
- Preserves CSV structure and formatting
|
||||
|
||||
## Utility Scripts
|
||||
|
||||
### 10. `ids/generate_resource_id.py`
|
||||
**Purpose**: Interactive resource ID generator
|
||||
**Usage**: `python scripts/ids/generate_resource_id.py`
|
||||
**Features**:
|
||||
- Interactive prompts for display name, link, and category
|
||||
- Shows all available categories from `categories.yaml`
|
||||
- Displays generated ID and CSV row preview
|
||||
|
||||
### 11. `ids/resource_id.py`
|
||||
**Purpose**: Shared resource ID generation module
|
||||
**Usage**: `from resource_id import generate_resource_id`
|
||||
**Features**:
|
||||
- Central function used by all ID generation scripts
|
||||
- Uses category prefixes from `categories.yaml`
|
||||
- Ensures consistent ID generation across the project
|
||||
|
||||
### 12. `badges/badge_notification_core.py`
|
||||
**Purpose**: Core functionality for badge notification system
|
||||
**Usage**: `from scripts.badges.badge_notification_core import BadgeNotificationCore`
|
||||
**Features**:
|
||||
- Shared notification logic used by other badge scripts
|
||||
- Input validation and sanitization
|
||||
- GitHub API interaction utilities
|
||||
- Template rendering for notification messages
|
||||
|
||||
### 13. `badges/badge_notification.py`
|
||||
**Purpose**: Action-only notifier for merged resource PRs
|
||||
**Usage**: Used by `notify-on-merge.yml` (not intended for manual execution)
|
||||
**Features**:
|
||||
- Sends a single notification issue to the resource repository
|
||||
- Uses `badge_notification_core.py` for shared logic
|
||||
|
||||
### 14. `graphics/generate_logo_svgs.py`
|
||||
**Purpose**: Generates SVG logos for the repository
|
||||
**Usage**: `python -m scripts.graphics.generate_logo_svgs`
|
||||
**Features**:
|
||||
- Creates consistent branding assets
|
||||
- Generates light/dark logo variants
|
||||
- Supports dark/light mode variants
|
||||
- Used for README badges and documentation
|
||||
|
||||
## Workflow Integration
|
||||
|
||||
### Primary Workflow (GitHub Issues)
|
||||
|
||||
**For Users**: Recommend resources through the GitHub Issue form at `.github/ISSUE_TEMPLATE/recommend-resource.yml`
|
||||
1. User fills out the issue form
|
||||
2. `validate-resource-submission.yml` workflow validates the submission automatically
|
||||
3. Maintainer reviews and uses `/approve` command
|
||||
4. `approve-resource-submission.yml` workflow creates the PR automatically
|
||||
|
||||
### Manual Backup Workflows (Make Commands)
|
||||
|
||||
These commands remain available for maintainers and emergency situations:
|
||||
|
||||
#### Adding a Resource Manually
|
||||
```bash
|
||||
make generate # Regenerate README
|
||||
make validate # Validate all links
|
||||
```
|
||||
|
||||
### Maintenance Tasks
|
||||
```bash
|
||||
make sort # Sort CSV entries
|
||||
make validate # Check all links
|
||||
make download-resources # Archive resources
|
||||
make generate-toc-assets # Regenerate subcategory TOC SVGs (after adding subcategories)
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Scripts respect these configuration files:
|
||||
- `.templates/resource-overrides.yaml`: Manual overrides for resources
|
||||
- `.env`: Environment variables (not tracked in git)
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `GITHUB_TOKEN`: For API rate limiting (optional but recommended)
|
||||
- `AWESOME_CC_PAT_PUBLIC_REPO`: For badge notifications
|
||||
- `AWESOME_CC_FORK_REMOTE`: Git remote name for fork (default: origin)
|
||||
- `AWESOME_CC_UPSTREAM_REMOTE`: Git remote name for upstream (default: upstream)
|
||||
|
||||
## Development Notes
|
||||
|
||||
1. All scripts include comprehensive error handling
|
||||
2. Progress bars and user feedback for long operations
|
||||
3. Backup creation before destructive operations
|
||||
4. Consistent use of pathlib for cross-platform compatibility
|
||||
5. Type hints and docstrings throughout
|
||||
6. Scripts can be run standalone or through Make targets
|
||||
|
||||
### Naming Conventions
|
||||
|
||||
**Status Lines category** (2025-09-16): The "Statusline" category was renamed to "Status Lines" (title case, plural) for consistency with other categories like "Hooks". This change was made throughout:
|
||||
- Category name: "Status Lines" (was "Statusline" or "Status line")
|
||||
- The `id` remains `statusline` to preserve backward compatibility
|
||||
- CSV entries updated to use "Status Lines" as the category value
|
||||
- All display text uses the title case plural form "Status Lines"
|
||||
|
||||
This ensures consistent title case and pluralization across categories. If issues arise with status line resources, verify that the category name matches "Status Lines" in CSV entries.
|
||||
|
||||
### Announcements System
|
||||
|
||||
**YAML Format** (2025-09-17): Announcements migrated from Markdown to YAML format for better structure and rendering:
|
||||
|
||||
**File**: `templates/announcements.yaml`
|
||||
|
||||
**Structure**:
|
||||
```yaml
|
||||
- date: "YYYY-MM-DD"
|
||||
title: "Announcement Title" # Optional
|
||||
items:
|
||||
- "Simple text item"
|
||||
- summary: "Collapsible item"
|
||||
text: "Detailed description that can be expanded"
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Automatically renders as nested collapsible sections in README
|
||||
- Each date group is collapsible
|
||||
- Individual items can be simple text or collapsible with summary/text
|
||||
- Supports multi-line text in detailed descriptions
|
||||
- Falls back to `.md` file if YAML doesn't exist for backward compatibility
|
||||
|
||||
## Future Considerations
|
||||
|
||||
- Additional validation rules could be added
|
||||
- More sophisticated duplicate detection
|
||||
0
.agent/knowledge/awesome_claude/scripts/__init__.py
Normal file
0
.agent/knowledge/awesome_claude/scripts/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
||||
# Archive
|
||||
|
||||
Temporary holding area for deprecated scripts that are no longer wired into the
|
||||
active toolchain, but are kept for reference while being evaluated for removal.
|
||||
@@ -0,0 +1 @@
|
||||
"""Archived/deprecated scripts."""
|
||||
@@ -0,0 +1,53 @@
|
||||
# Badge Issue Notification Setup Guide
|
||||
|
||||
## Overview
|
||||
This system creates friendly notification issues on GitHub repositories when they are **newly** featured in the Awesome Claude Code list. It only notifies for new additions, not existing entries.
|
||||
|
||||
## Prerequisites
|
||||
1. Python 3.11+
|
||||
2. PyGithub library (installed automatically via pyproject.toml)
|
||||
|
||||
## GitHub Action Setup
|
||||
|
||||
### 1. Required Setup
|
||||
Add your Personal Access Token as a repository secret named `AWESOME_CC_PAT_PUBLIC_REPO`:
|
||||
1. Go to Settings → Secrets and variables → Actions
|
||||
2. Click "New repository secret"
|
||||
3. Name: `AWESOME_CC_PAT_PUBLIC_REPO`
|
||||
4. Value: Your Personal Access Token with `public_repo` scope
|
||||
|
||||
### 2. Automatic Triggers
|
||||
The action automatically runs when resource PRs are merged by the automation bot.
|
||||
|
||||
## How It Works
|
||||
|
||||
### Issue Creation Process
|
||||
1. Extracts the GitHub URL and resource name from the merged PR
|
||||
2. Runs `scripts/badges/badge_notification.py` to send a single notification issue
|
||||
|
||||
### Issue Content
|
||||
- Friendly greeting and announcement
|
||||
- Description of Awesome Claude Code
|
||||
- Two badge style options (standard and flat)
|
||||
- Clear markdown snippets for easy copying
|
||||
- No action required message
|
||||
|
||||
### Duplicate Prevention
|
||||
- Checks for existing issues by the bot
|
||||
|
||||
## Features
|
||||
|
||||
### Advantages Over PR Approach
|
||||
- ✅ Non-intrusive - just information
|
||||
- ✅ No code changes required
|
||||
- ✅ Maintainers can close anytime
|
||||
- ✅ Much simpler implementation
|
||||
- ✅ No fork/branch management
|
||||
- ✅ Faster processing
|
||||
|
||||
### Error Handling
|
||||
- Gracefully handles:
|
||||
- Private repositories
|
||||
- Disabled issues
|
||||
- Rate limiting
|
||||
- Invalid URLs
|
||||
@@ -0,0 +1,108 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Badge Issue Notification (GitHub Actions only).
|
||||
|
||||
Creates a single notification issue in a specified GitHub repository
|
||||
when a resource PR is merged. This script is designed for automated
|
||||
use in GitHub Actions and is not intended for manual execution.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
# Try to load .env file if it exists
|
||||
try:
|
||||
from dotenv import load_dotenv # type: ignore[import]
|
||||
|
||||
load_dotenv()
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from scripts.badges.badge_notification_core import BadgeNotificationCore # noqa: E402
|
||||
|
||||
|
||||
def main():
|
||||
"""Main execution for automated notification via GitHub Actions."""
|
||||
|
||||
# Get inputs from environment variables (set by GitHub Actions)
|
||||
repo_url = os.environ.get("REPOSITORY_URL", "").strip()
|
||||
resource_name = os.environ.get("RESOURCE_NAME", "").strip() or None
|
||||
description = os.environ.get("DESCRIPTION", "").strip() or None
|
||||
# Validate required inputs
|
||||
if not repo_url:
|
||||
print("Error: REPOSITORY_URL environment variable is required")
|
||||
sys.exit(1)
|
||||
|
||||
# Get GitHub token
|
||||
github_token = os.environ.get("AWESOME_CC_PAT_PUBLIC_REPO")
|
||||
if not github_token:
|
||||
print("Error: AWESOME_CC_PAT_PUBLIC_REPO environment variable is required")
|
||||
print("This token needs 'public_repo' scope to create issues in external repositories")
|
||||
sys.exit(1)
|
||||
|
||||
# Log the operation
|
||||
print(f"Sending notification to: {repo_url}")
|
||||
if resource_name:
|
||||
print(f"Resource name: {resource_name}")
|
||||
if description:
|
||||
print(f"Description: {description[:100]}...")
|
||||
|
||||
try:
|
||||
# Initialize the core notification system
|
||||
notifier = BadgeNotificationCore(github_token)
|
||||
|
||||
# Send the notification using the core module
|
||||
result = notifier.create_notification_issue(
|
||||
repo_url=repo_url,
|
||||
resource_name=resource_name,
|
||||
description=description,
|
||||
)
|
||||
|
||||
# Handle the result
|
||||
if result["success"]:
|
||||
print(f"✅ Success! Issue created: {result['issue_url']}")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print(f"❌ Failed: {result['message']}")
|
||||
|
||||
# Provide helpful guidance based on error
|
||||
if "Security validation failed" in result["message"]:
|
||||
print("\n🛡️ SECURITY: Dangerous content detected in input")
|
||||
print(" The operation was aborted for security reasons.")
|
||||
print(" Check the resource name and description for:")
|
||||
print(" - HTML tags or JavaScript")
|
||||
print(" - Protocol handlers (javascript:, data:, etc.)")
|
||||
print(" - Event handlers (onclick=, onerror=, etc.)")
|
||||
elif "Invalid or dangerous" in result["message"]:
|
||||
print("\n💡 Tip: Ensure the URL is a valid GitHub repository URL")
|
||||
print(" Format: https://github.com/owner/repository")
|
||||
elif "Rate limit" in result["message"]:
|
||||
print("\n💡 Tip: GitHub API rate limit reached. Please wait and try again.")
|
||||
elif "Permission denied" in result["message"]:
|
||||
print("\n💡 Tip: Ensure your PAT has 'public_repo' scope")
|
||||
elif "not found or private" in result["message"]:
|
||||
print("\n💡 Tip: The repository may be private or deleted")
|
||||
elif "issues disabled" in result["message"]:
|
||||
print("\n💡 Tip: The repository has issues disabled in settings")
|
||||
|
||||
sys.exit(1)
|
||||
|
||||
except ValueError as e:
|
||||
# Handle initialization errors (e.g., missing token)
|
||||
print(f"❌ Error: {e}")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
# Handle unexpected errors
|
||||
print(f"❌ Unexpected error: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,461 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Core module for badge notification system
|
||||
Shared functionality for both automated and manual badge notifications
|
||||
Includes security hardening, rate limiting, and error handling
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from github import Github
|
||||
from github.GithubException import (
|
||||
BadCredentialsException,
|
||||
GithubException,
|
||||
RateLimitExceededException,
|
||||
UnknownObjectException,
|
||||
)
|
||||
|
||||
from scripts.utils.github_utils import get_github_client, parse_github_url
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class RateLimiter:
|
||||
"""Handle GitHub API rate limiting with exponential backoff"""
|
||||
|
||||
def __init__(self):
|
||||
self.last_request_time = 0
|
||||
self.request_count = 0
|
||||
self.backoff_seconds = 1
|
||||
self.max_backoff = 60
|
||||
|
||||
def check_rate_limit(self, github_client: Github) -> dict:
|
||||
"""Check current rate limit status"""
|
||||
try:
|
||||
rate_limit = github_client.get_rate_limit()
|
||||
core = rate_limit.resources.core
|
||||
return {
|
||||
"remaining": core.remaining,
|
||||
"limit": core.limit,
|
||||
"reset_time": core.reset.timestamp(),
|
||||
"should_pause": core.remaining < 100,
|
||||
"should_stop": core.remaining < 10,
|
||||
}
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not check rate limit: {e}")
|
||||
return {
|
||||
"remaining": -1,
|
||||
"limit": -1,
|
||||
"reset_time": 0,
|
||||
"should_pause": False,
|
||||
"should_stop": False,
|
||||
}
|
||||
|
||||
def wait_if_needed(self, github_client: Github):
|
||||
"""Wait if rate limiting requires it"""
|
||||
status = self.check_rate_limit(github_client)
|
||||
|
||||
if status["should_stop"]:
|
||||
wait_time = max(0, status["reset_time"] - time.time())
|
||||
logger.warning(
|
||||
f"Rate limit nearly exhausted. Waiting {wait_time:.0f} seconds until reset"
|
||||
)
|
||||
time.sleep(wait_time + 1)
|
||||
elif status["should_pause"]:
|
||||
logger.info(
|
||||
f"Rate limit low ({status['remaining']} remaining). "
|
||||
f"Pausing {self.backoff_seconds} seconds"
|
||||
)
|
||||
time.sleep(self.backoff_seconds)
|
||||
self.backoff_seconds = min(self.backoff_seconds * 2, self.max_backoff)
|
||||
else:
|
||||
# Reset backoff if we're doing well
|
||||
if status["remaining"] > 1000:
|
||||
self.backoff_seconds = 1
|
||||
|
||||
def handle_rate_limit_error(self, error: RateLimitExceededException):
|
||||
"""Handle rate limit exception"""
|
||||
reset_time = error.headers.get("X-RateLimit-Reset", "0") if error.headers else "0"
|
||||
wait_time = max(0, int(reset_time) - time.time())
|
||||
logger.error(f"Rate limit exceeded. Waiting {wait_time} seconds until reset")
|
||||
time.sleep(wait_time + 1)
|
||||
|
||||
|
||||
class BadgeNotificationCore:
|
||||
"""Core functionality for badge notifications with security hardening"""
|
||||
|
||||
# Configuration
|
||||
ISSUE_TITLE = "🎉 Your project has been featured in Awesome Claude Code!"
|
||||
NOTIFICATION_LABEL = "awesome-claude-code"
|
||||
GITHUB_URL_BASE = "https://github.com/hesreallyhim/awesome-claude-code"
|
||||
|
||||
def __init__(self, github_token: str):
|
||||
"""Initialize with GitHub token"""
|
||||
if not github_token:
|
||||
raise ValueError("GitHub token is required")
|
||||
|
||||
self.github = get_github_client(token=github_token)
|
||||
self.rate_limiter = RateLimiter()
|
||||
|
||||
@staticmethod
|
||||
def validate_input_safety(text: str, field_name: str = "input") -> tuple[bool, str]:
|
||||
"""
|
||||
Validate that input text is safe for use in GitHub issues.
|
||||
Returns (is_safe, reason_if_unsafe)
|
||||
|
||||
This does NOT modify the input - it only checks for dangerous content.
|
||||
If dangerous content is found, the operation should be aborted.
|
||||
"""
|
||||
if not text:
|
||||
return True, ""
|
||||
|
||||
# Check for dangerous protocol handlers
|
||||
dangerous_protocols = [
|
||||
"javascript:",
|
||||
"data:",
|
||||
"vbscript:",
|
||||
"file:",
|
||||
"about:",
|
||||
"chrome:",
|
||||
"ms-",
|
||||
]
|
||||
for protocol in dangerous_protocols:
|
||||
if protocol.lower() in text.lower():
|
||||
reason = f"Dangerous protocol '{protocol}' detected in {field_name}"
|
||||
logger.warning(f"SECURITY: {reason} - Content: {text[:100]}")
|
||||
return False, reason
|
||||
|
||||
# Check for HTML/script injection attempts
|
||||
dangerous_patterns = [
|
||||
"<script",
|
||||
"</script",
|
||||
"<iframe",
|
||||
"<embed",
|
||||
"<object",
|
||||
"<applet",
|
||||
"<meta",
|
||||
"<link",
|
||||
"onclick=",
|
||||
"onload=",
|
||||
"onerror=",
|
||||
"onmouseover=",
|
||||
"onfocus=",
|
||||
]
|
||||
for pattern in dangerous_patterns:
|
||||
if pattern.lower() in text.lower():
|
||||
reason = f"HTML injection attempt detected in {field_name}: {pattern}"
|
||||
logger.warning(f"SECURITY: {reason} - Content: {text[:100]}")
|
||||
return False, reason
|
||||
|
||||
# Check for excessive length (DoS prevention)
|
||||
max_length = 5000 # Reasonable limit for resource descriptions
|
||||
if len(text) > max_length:
|
||||
reason = f"{field_name} exceeds maximum length ({len(text)} > {max_length})"
|
||||
logger.warning(f"SECURITY: {reason}")
|
||||
return False, reason
|
||||
|
||||
# Check for null bytes (can cause issues in various systems)
|
||||
if "\x00" in text:
|
||||
reason = f"Null byte detected in {field_name}"
|
||||
logger.warning(f"SECURITY: {reason}")
|
||||
return False, reason
|
||||
|
||||
# Check for control characters (except newline and tab)
|
||||
control_chars = [chr(i) for i in range(0, 32) if i not in [9, 10, 13]]
|
||||
for char in control_chars:
|
||||
if char in text:
|
||||
reason = f"Control character (ASCII {ord(char)}) detected in {field_name}"
|
||||
logger.warning(f"SECURITY: {reason}")
|
||||
return False, reason
|
||||
|
||||
return True, ""
|
||||
|
||||
@staticmethod
|
||||
def validate_github_url(url: str) -> bool:
|
||||
"""
|
||||
Strictly validate GitHub URL format
|
||||
Prevents command injection and other URL-based attacks
|
||||
"""
|
||||
if not url:
|
||||
return False
|
||||
|
||||
# Only allow HTTPS GitHub URLs
|
||||
if not url.startswith("https://github.com/"):
|
||||
return False
|
||||
|
||||
# Check for dangerous characters that could be used for injection
|
||||
dangerous_chars = [
|
||||
";",
|
||||
"|",
|
||||
"&",
|
||||
"`",
|
||||
"$",
|
||||
"(",
|
||||
")",
|
||||
"{",
|
||||
"}",
|
||||
"<",
|
||||
">",
|
||||
"\n",
|
||||
"\r",
|
||||
"\\",
|
||||
"'",
|
||||
'"',
|
||||
]
|
||||
if any(char in url for char in dangerous_chars):
|
||||
return False
|
||||
|
||||
# Strict regex for GitHub URLs
|
||||
# Only allow alphanumeric, dash, dot, underscore in owner/repo names
|
||||
pattern = r"^https://github\.com/[\w\-\.]+/[\w\-\.]+(?:\.git)?/?$"
|
||||
if not re.match(pattern, url):
|
||||
return False
|
||||
|
||||
# Check for path traversal attempts
|
||||
return ".." not in url
|
||||
|
||||
def create_issue_body(self, resource_name: str, description: str = "") -> str:
|
||||
"""Create issue body with badge options after validating inputs"""
|
||||
# Validate inputs - DO NOT modify them
|
||||
is_safe, reason = self.validate_input_safety(resource_name, "resource_name")
|
||||
if not is_safe:
|
||||
raise ValueError(f"Security validation failed: {reason}")
|
||||
|
||||
if description:
|
||||
is_safe, reason = self.validate_input_safety(description, "description")
|
||||
if not is_safe:
|
||||
raise ValueError(f"Security validation failed: {reason}")
|
||||
|
||||
# Use the ORIGINAL, unmodified values in the template
|
||||
# If they were unsafe, we would have thrown an exception above
|
||||
final_description = (
|
||||
description
|
||||
if description
|
||||
else f"Your project {resource_name} provides valuable resources "
|
||||
f"for the Claude Code community."
|
||||
)
|
||||
|
||||
# Use the original values directly
|
||||
return f"""Hello! 👋
|
||||
|
||||
I'm excited to let you know that **{resource_name}** has been featured in the
|
||||
[Awesome Claude Code]({self.GITHUB_URL_BASE}) list!
|
||||
|
||||
## About Awesome Claude Code
|
||||
Awesome Claude Code is a curated collection of the best slash-commands, CLAUDE.md files,
|
||||
CLI tools, and other resources for enhancing Claude Code workflows. Your project has been
|
||||
recognized for its valuable contribution to the Claude Code community.
|
||||
|
||||
## Your Listing
|
||||
{final_description}
|
||||
|
||||
You can find your entry here: [View in Awesome Claude Code]({self.GITHUB_URL_BASE})
|
||||
|
||||
## Show Your Recognition! 🏆
|
||||
If you'd like to display a badge in your README to show that your project is featured,
|
||||
you can use one of these:
|
||||
|
||||
### Option 1: Standard Badge
|
||||
```markdown
|
||||
[]({self.GITHUB_URL_BASE})
|
||||
```
|
||||
[]({self.GITHUB_URL_BASE})
|
||||
|
||||
### Option 2: Flat Badge
|
||||
```markdown
|
||||
[]({self.GITHUB_URL_BASE})
|
||||
```
|
||||
[]({self.GITHUB_URL_BASE})
|
||||
|
||||
## No Action Required
|
||||
This is just a friendly notification - no action is required on your part.
|
||||
Feel free to close this issue at any time.
|
||||
|
||||
Thank you for contributing to the Claude Code ecosystem! 🙏
|
||||
|
||||
---
|
||||
*This notification was sent because your project was added to the Awesome Claude Code list. This is a one-time notification.*""" # noqa: E501
|
||||
|
||||
def can_create_label(self, repo) -> bool:
|
||||
"""Check if we can create labels (requires write access)"""
|
||||
try:
|
||||
# Apply rate limiting
|
||||
self.rate_limiter.wait_if_needed(self.github)
|
||||
|
||||
# Try to create or get the label
|
||||
try:
|
||||
repo.get_label(self.NOTIFICATION_LABEL)
|
||||
return True # Label already exists
|
||||
except UnknownObjectException:
|
||||
# Label doesn't exist, try to create it
|
||||
repo.create_label(
|
||||
self.NOTIFICATION_LABEL, "f39c12", "Featured in Awesome Claude Code"
|
||||
)
|
||||
return True
|
||||
except GithubException as e:
|
||||
if e.status == 403:
|
||||
logger.info(f"No permission to create labels in {repo.full_name}")
|
||||
else:
|
||||
logger.warning(f"Could not create label for {repo.full_name}: {e}")
|
||||
return False
|
||||
except Exception as e:
|
||||
logger.warning(f"Unexpected error creating label for {repo.full_name}: {e}")
|
||||
return False
|
||||
|
||||
def create_notification_issue(
|
||||
self,
|
||||
repo_url: str,
|
||||
resource_name: str | None = None,
|
||||
description: str | None = None,
|
||||
) -> dict:
|
||||
"""
|
||||
Create a notification issue in the specified repository
|
||||
|
||||
Returns dict with: success, message, issue_url, repo_url
|
||||
"""
|
||||
result = {
|
||||
"repo_url": repo_url,
|
||||
"success": False,
|
||||
"message": "",
|
||||
"issue_url": None,
|
||||
}
|
||||
|
||||
# Validate and parse URL
|
||||
if not self.validate_github_url(repo_url):
|
||||
result["message"] = "Invalid or dangerous GitHub URL format"
|
||||
return result
|
||||
|
||||
_, is_github, owner, repo_name = parse_github_url(repo_url)
|
||||
if not is_github or not owner or not repo_name:
|
||||
result["message"] = "Invalid or dangerous GitHub URL format"
|
||||
return result
|
||||
|
||||
repo_full_name = f"{owner}/{repo_name}"
|
||||
|
||||
# Use resource name from input or default to repo name
|
||||
if not resource_name:
|
||||
resource_name = repo_name
|
||||
|
||||
# Skip Anthropic repositories
|
||||
if "anthropic" in owner.lower() or "anthropic" in repo_name.lower():
|
||||
result["message"] = "Skipping Anthropic repository"
|
||||
return result
|
||||
|
||||
try:
|
||||
# Apply rate limiting
|
||||
self.rate_limiter.wait_if_needed(self.github)
|
||||
|
||||
# Get the repository
|
||||
repo = self.github.get_repo(repo_full_name)
|
||||
|
||||
# Try to create or use label
|
||||
labels = []
|
||||
if self.can_create_label(repo):
|
||||
labels = [self.NOTIFICATION_LABEL]
|
||||
|
||||
# Create the issue body (this will validate inputs and throw if unsafe)
|
||||
try:
|
||||
issue_body = self.create_issue_body(resource_name, description or "")
|
||||
except ValueError as e:
|
||||
# Security validation failed - abort the operation
|
||||
result["message"] = str(e)
|
||||
logger.error(f"Security validation failed for {repo_full_name}: {e}")
|
||||
return result
|
||||
|
||||
# Apply rate limiting before creating issue
|
||||
self.rate_limiter.wait_if_needed(self.github)
|
||||
|
||||
# Create the issue
|
||||
issue = repo.create_issue(title=self.ISSUE_TITLE, body=issue_body, labels=labels)
|
||||
|
||||
result["success"] = True
|
||||
result["message"] = "Issue created successfully"
|
||||
result["issue_url"] = issue.html_url
|
||||
|
||||
except UnknownObjectException:
|
||||
result["message"] = "Repository not found or private"
|
||||
except BadCredentialsException:
|
||||
result["message"] = "Invalid GitHub token"
|
||||
except RateLimitExceededException as e:
|
||||
self.rate_limiter.handle_rate_limit_error(e)
|
||||
result["message"] = "Rate limit exceeded - please try again later"
|
||||
except GithubException as e:
|
||||
if e.status == 410:
|
||||
result["message"] = "Repository has issues disabled"
|
||||
elif e.status == 403:
|
||||
if "Resource not accessible" in str(e):
|
||||
result["message"] = "Insufficient permissions - requires public_repo scope"
|
||||
else:
|
||||
result["message"] = "Permission denied - check PAT permissions"
|
||||
else:
|
||||
logger.error(f"GitHub API error for {repo_full_name}: {e}")
|
||||
result["message"] = f"GitHub API error (status {e.status})"
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error for {repo_full_name}: {e}")
|
||||
result["message"] = f"Unexpected error: {str(e)[:100]}"
|
||||
|
||||
return result
|
||||
|
||||
|
||||
class ManualNotificationTracker:
|
||||
"""Optional state tracking for manual notifications"""
|
||||
|
||||
def __init__(self, tracking_file: str = ".manual_notifications.json"):
|
||||
self.tracking_file = Path(tracking_file)
|
||||
self.history = self._load_history()
|
||||
|
||||
def _load_history(self) -> list:
|
||||
"""Load notification history from file"""
|
||||
if self.tracking_file.exists():
|
||||
try:
|
||||
with open(self.tracking_file) as f:
|
||||
return json.load(f)
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not load history: {e}")
|
||||
return []
|
||||
|
||||
def _save_history(self):
|
||||
"""Save notification history to file"""
|
||||
try:
|
||||
with open(self.tracking_file, "w") as f:
|
||||
json.dump(self.history, f, indent=2)
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not save history: {e}")
|
||||
|
||||
def record_notification(self, repo_url: str, issue_url: str, resource_name: str = ""):
|
||||
"""Record a manual notification"""
|
||||
entry = {
|
||||
"repo_url": repo_url,
|
||||
"issue_url": issue_url,
|
||||
"resource_name": resource_name,
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
}
|
||||
self.history.append(entry)
|
||||
self._save_history()
|
||||
|
||||
def get_notification_count(self, repo_url: str, time_window_hours: int = 24) -> int:
|
||||
"""Get count of recent notifications for a repository"""
|
||||
cutoff = datetime.now().timestamp() - (time_window_hours * 3600)
|
||||
count = 0
|
||||
|
||||
for entry in self.history:
|
||||
if entry["repo_url"] == repo_url:
|
||||
try:
|
||||
timestamp = datetime.fromisoformat(entry["timestamp"]).timestamp()
|
||||
if timestamp > cutoff:
|
||||
count += 1
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return count
|
||||
|
||||
def has_recent_notification(self, repo_url: str, time_window_hours: int = 24) -> bool:
|
||||
"""Check if repository was notified recently"""
|
||||
return self.get_notification_count(repo_url, time_window_hours) > 0
|
||||
@@ -0,0 +1,449 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script to automate adding a new category to awesome-claude-code.
|
||||
This handles all the necessary file updates and regenerates the README.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
# Add repo root to path for imports
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from scripts.categories.category_utils import category_manager # noqa: E402
|
||||
|
||||
|
||||
class CategoryAdder:
|
||||
"""Handles the process of adding a new category to the repository."""
|
||||
|
||||
def __init__(self, repo_root: Path):
|
||||
"""Initialize the CategoryAdder with the repository root path."""
|
||||
self.repo_root = repo_root
|
||||
self.templates_dir = repo_root / "templates"
|
||||
self.github_dir = repo_root / ".github" / "ISSUE_TEMPLATE"
|
||||
|
||||
def get_max_order(self) -> int:
|
||||
"""Get the maximum order value from existing categories."""
|
||||
categories = category_manager.get_categories_for_readme()
|
||||
if not categories:
|
||||
return 0
|
||||
return max(cat.get("order", 0) for cat in categories)
|
||||
|
||||
def add_category_to_yaml(
|
||||
self,
|
||||
category_id: str,
|
||||
name: str,
|
||||
prefix: str,
|
||||
icon: str,
|
||||
description: str,
|
||||
order: int | None = None,
|
||||
subcategories: list[str] | None = None,
|
||||
) -> bool:
|
||||
"""
|
||||
Add a new category to categories.yaml.
|
||||
|
||||
Args:
|
||||
category_id: The ID for the category (e.g., "alternative-clients")
|
||||
name: Display name (e.g., "Alternative Clients")
|
||||
prefix: ID prefix for resources (e.g., "client")
|
||||
icon: Emoji icon for the category
|
||||
description: Markdown description of the category
|
||||
order: Order in the list (if None, will be added at the end)
|
||||
subcategories: List of subcategory names (defaults to ["General"])
|
||||
|
||||
Returns:
|
||||
True if successful, False otherwise
|
||||
"""
|
||||
categories_file = self.templates_dir / "categories.yaml"
|
||||
|
||||
# Load existing categories
|
||||
with open(categories_file, encoding="utf-8") as f:
|
||||
data = yaml.safe_load(f)
|
||||
|
||||
if not data or "categories" not in data:
|
||||
print("Error: Invalid categories.yaml structure")
|
||||
return False
|
||||
|
||||
# Check if category already exists
|
||||
for cat in data["categories"]:
|
||||
if cat["id"] == category_id:
|
||||
print(f"Category '{category_id}' already exists")
|
||||
return False
|
||||
|
||||
# Determine order
|
||||
if order is None:
|
||||
order = self.get_max_order() + 1
|
||||
|
||||
# Prepare subcategories
|
||||
if subcategories is None:
|
||||
subcategories = ["General"]
|
||||
|
||||
subcats_data = [{"id": sub.lower().replace(" ", "-"), "name": sub} for sub in subcategories]
|
||||
|
||||
# Create new category entry
|
||||
new_category = {
|
||||
"id": category_id,
|
||||
"name": name,
|
||||
"prefix": prefix,
|
||||
"icon": icon,
|
||||
"description": description,
|
||||
"order": order,
|
||||
"subcategories": subcats_data,
|
||||
}
|
||||
|
||||
# If inserting with specific order, update other categories' orders
|
||||
if order <= self.get_max_order():
|
||||
for cat in data["categories"]:
|
||||
if cat.get("order", 0) >= order:
|
||||
cat["order"] = cat.get("order", 0) + 1
|
||||
|
||||
# Add the new category
|
||||
data["categories"].append(new_category)
|
||||
|
||||
# Sort categories by order
|
||||
data["categories"] = sorted(data["categories"], key=lambda x: x.get("order", 999))
|
||||
|
||||
# Write back to file
|
||||
with open(categories_file, "w", encoding="utf-8") as f:
|
||||
yaml.dump(data, f, default_flow_style=False, sort_keys=False, allow_unicode=True)
|
||||
|
||||
print(f"✅ Added '{name}' to categories.yaml with order {order}")
|
||||
return True
|
||||
|
||||
def update_issue_template(self, name: str) -> bool:
|
||||
"""
|
||||
Update the GitHub issue template to include the new category.
|
||||
|
||||
Args:
|
||||
name: Display name of the category
|
||||
|
||||
Returns:
|
||||
True if successful, False otherwise
|
||||
"""
|
||||
template_file = self.github_dir / "recommend-resource.yml"
|
||||
|
||||
with open(template_file, encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
|
||||
# Find the category dropdown section
|
||||
lines = content.split("\n")
|
||||
in_category_section = False
|
||||
category_start_idx = -1
|
||||
category_end_idx = -1
|
||||
|
||||
for i, line in enumerate(lines):
|
||||
if "id: category" in line:
|
||||
in_category_section = True
|
||||
continue
|
||||
|
||||
if in_category_section:
|
||||
if "options:" in line:
|
||||
category_start_idx = i + 1
|
||||
elif category_start_idx > 0 and line.strip() and not line.strip().startswith("-"):
|
||||
category_end_idx = i
|
||||
break
|
||||
|
||||
if category_start_idx < 0:
|
||||
print("Error: Could not find category options in issue template")
|
||||
return False
|
||||
|
||||
# Extract existing categories
|
||||
existing_categories = []
|
||||
for i in range(category_start_idx, category_end_idx):
|
||||
line = lines[i].strip()
|
||||
if line.startswith("- "):
|
||||
existing_categories.append(line[2:])
|
||||
|
||||
# Check if category already exists
|
||||
if name in existing_categories:
|
||||
print(f"Category '{name}' already exists in issue template")
|
||||
return True
|
||||
|
||||
# Find where to insert (before Official Documentation)
|
||||
insert_idx = category_start_idx
|
||||
for i in range(category_start_idx, category_end_idx):
|
||||
if "Official Documentation" in lines[i]:
|
||||
insert_idx = i
|
||||
break
|
||||
|
||||
# Insert the new category
|
||||
lines.insert(insert_idx, f" - {name}")
|
||||
|
||||
# Write back to file
|
||||
with open(template_file, "w", encoding="utf-8") as f:
|
||||
f.write("\n".join(lines))
|
||||
|
||||
print(f"✅ Added '{name}' to GitHub issue template")
|
||||
return True
|
||||
|
||||
def generate_readme(self) -> bool:
|
||||
"""Generate the README using make generate."""
|
||||
print("\n📝 Generating README...")
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["make", "generate"],
|
||||
cwd=self.repo_root,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
if result.returncode != 0:
|
||||
print("Error generating README:")
|
||||
if result.stderr:
|
||||
print(result.stderr)
|
||||
return False
|
||||
|
||||
print("✅ README generated successfully")
|
||||
return True
|
||||
|
||||
except FileNotFoundError:
|
||||
print("Error: 'make' command not found")
|
||||
return False
|
||||
|
||||
def create_commit(self, name: str) -> bool:
|
||||
"""Create a commit with the changes."""
|
||||
print("\n📦 Creating commit...")
|
||||
|
||||
try:
|
||||
# Stage the changes
|
||||
files_to_stage = [
|
||||
"templates/categories.yaml",
|
||||
".github/ISSUE_TEMPLATE/recommend-resource.yml",
|
||||
"README.md",
|
||||
]
|
||||
|
||||
for file in files_to_stage:
|
||||
subprocess.run(
|
||||
["git", "add", file],
|
||||
cwd=self.repo_root,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
|
||||
# Create commit
|
||||
commit_message = f"""Add new category: {name}
|
||||
|
||||
- Add {name} category to templates/categories.yaml
|
||||
- Update GitHub issue template to include {name}
|
||||
- Regenerate README with new category section
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
|
||||
Co-Authored-By: Claude <noreply@anthropic.com>"""
|
||||
|
||||
result = subprocess.run(
|
||||
["git", "commit", "-m", commit_message],
|
||||
cwd=self.repo_root,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
if result.returncode != 0:
|
||||
if "nothing to commit" in result.stdout:
|
||||
print("No changes to commit")
|
||||
else:
|
||||
print("Error creating commit:")
|
||||
if result.stderr:
|
||||
print(result.stderr)
|
||||
return False
|
||||
else:
|
||||
print(f"✅ Created commit for '{name}' category")
|
||||
|
||||
return True
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Error with git operations: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def interactive_mode(adder: CategoryAdder) -> None:
|
||||
"""Run the script in interactive mode, prompting for all inputs."""
|
||||
print("=" * 60)
|
||||
print("ADD NEW CATEGORY TO AWESOME CLAUDE CODE")
|
||||
print("=" * 60)
|
||||
print()
|
||||
|
||||
# Get category details
|
||||
name = input("Enter category display name (e.g., 'Alternative Clients'): ").strip()
|
||||
if not name:
|
||||
print("Error: Name is required")
|
||||
sys.exit(1)
|
||||
|
||||
# Generate ID from name
|
||||
category_id = name.lower().replace(" ", "-").replace("&", "and")
|
||||
suggested_id = category_id
|
||||
category_id = input(f"Enter category ID (default: '{suggested_id}'): ").strip() or suggested_id
|
||||
|
||||
# Generate prefix from name
|
||||
suggested_prefix = name.lower().split()[0][:6]
|
||||
prefix = input(f"Enter ID prefix (default: '{suggested_prefix}'): ").strip() or suggested_prefix
|
||||
|
||||
# Get icon
|
||||
icon = input("Enter emoji icon (e.g., 🔌): ").strip() or "📦"
|
||||
|
||||
# Get description
|
||||
print("\nEnter description (can be multiline, enter '---' on a new line to finish):")
|
||||
description_lines = []
|
||||
while True:
|
||||
line = input()
|
||||
if line == "---":
|
||||
break
|
||||
description_lines.append(line)
|
||||
|
||||
description = "\n".join(description_lines)
|
||||
if description and not description.startswith(">"):
|
||||
description = "> " + description.replace("\n", "\n> ")
|
||||
|
||||
# Get order
|
||||
max_order = adder.get_max_order()
|
||||
order_input = input(
|
||||
f"Enter order position (1-{max_order + 1}, default: {max_order + 1}): "
|
||||
).strip()
|
||||
order = int(order_input) if order_input else max_order + 1
|
||||
|
||||
# Get subcategories
|
||||
print("\nSubcategories Configuration:")
|
||||
print("Most categories only need 'General'. Add more only if you need specific groupings.")
|
||||
print("Examples:")
|
||||
print(" - For simple categories: Just press Enter (uses 'General')")
|
||||
print(" - For complex categories: General, Advanced, Experimental")
|
||||
print("\nEnter subcategories (comma-separated, default: 'General'):")
|
||||
subcats_input = input("> ").strip()
|
||||
subcategories = (
|
||||
[s.strip() for s in subcats_input.split(",") if s.strip()] if subcats_input else ["General"]
|
||||
)
|
||||
|
||||
# Ensure General is always included if not explicitly added
|
||||
if subcategories and "General" not in subcategories:
|
||||
print("\nNote: Consider including 'General' as a catch-all subcategory.")
|
||||
|
||||
# Confirm
|
||||
print("\n" + "=" * 60)
|
||||
print("SUMMARY")
|
||||
print("=" * 60)
|
||||
print(f"Name: {name}")
|
||||
print(f"ID: {category_id}")
|
||||
print(f"Prefix: {prefix}")
|
||||
print(f"Icon: {icon}")
|
||||
print(f"Order: {order}")
|
||||
print(f"Subcategories: {', '.join(subcategories)}")
|
||||
print(f"Description:\n{description}")
|
||||
print("=" * 60)
|
||||
|
||||
confirm = input("\nProceed with adding this category? (y/n): ").strip().lower()
|
||||
if confirm != "y":
|
||||
print("Cancelled")
|
||||
sys.exit(0)
|
||||
|
||||
# Add the category
|
||||
if not adder.add_category_to_yaml(
|
||||
category_id, name, prefix, icon, description, order, subcategories
|
||||
):
|
||||
sys.exit(1)
|
||||
|
||||
if not adder.update_issue_template(name):
|
||||
sys.exit(1)
|
||||
|
||||
if not adder.generate_readme():
|
||||
sys.exit(1)
|
||||
|
||||
# Ask about commit
|
||||
commit_confirm = input("\nCreate a commit with these changes? (y/n): ").strip().lower()
|
||||
if commit_confirm == "y":
|
||||
adder.create_commit(name)
|
||||
|
||||
print("\n✨ Category added successfully!")
|
||||
print("\n📝 Note: The category will appear in the Table of Contents only after")
|
||||
print(" resources are added to it. This is by design to keep the ToC clean.")
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point for the script."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Add a new category to awesome-claude-code",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
%(prog)s # Interactive mode
|
||||
%(prog)s --name "My Category" --prefix "mycat" --icon "🎯"
|
||||
%(prog)s --name "Tools" --order 5 --subcategories "CLI,GUI,Web"
|
||||
""",
|
||||
)
|
||||
|
||||
parser.add_argument("--name", help="Display name for the category")
|
||||
parser.add_argument("--id", help="Category ID (defaults to slugified name)")
|
||||
parser.add_argument("--prefix", help="ID prefix for resources")
|
||||
parser.add_argument("--icon", default="📦", help="Emoji icon for the category")
|
||||
parser.add_argument(
|
||||
"--description", help="Description of the category (will be prefixed with '>')"
|
||||
)
|
||||
parser.add_argument("--order", type=int, help="Order position in the list")
|
||||
parser.add_argument(
|
||||
"--subcategories",
|
||||
help="Comma-separated list of subcategories (default: General)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-commit", action="store_true", help="Don't create a commit after adding"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Get repository root
|
||||
adder = CategoryAdder(REPO_ROOT)
|
||||
|
||||
# If name is provided, run in non-interactive mode
|
||||
if args.name:
|
||||
# Generate defaults for missing arguments
|
||||
category_id = args.id or args.name.lower().replace(" ", "-").replace("&", "and")
|
||||
prefix = args.prefix or args.name.lower().split()[0][:6]
|
||||
description = args.description or f"> **{args.name}** category for awesome-claude-code"
|
||||
if not description.startswith(">"):
|
||||
description = "> " + description
|
||||
|
||||
subcategories = (
|
||||
[s.strip() for s in args.subcategories.split(",")]
|
||||
if args.subcategories
|
||||
else ["General"]
|
||||
)
|
||||
|
||||
# Add the category
|
||||
if not adder.add_category_to_yaml(
|
||||
category_id,
|
||||
args.name,
|
||||
prefix,
|
||||
args.icon,
|
||||
description,
|
||||
args.order,
|
||||
subcategories,
|
||||
):
|
||||
sys.exit(1)
|
||||
|
||||
if not adder.update_issue_template(args.name):
|
||||
sys.exit(1)
|
||||
|
||||
if not adder.generate_readme():
|
||||
sys.exit(1)
|
||||
|
||||
if not args.no_commit:
|
||||
adder.create_commit(args.name)
|
||||
|
||||
print("\n✨ Category added successfully!")
|
||||
print("\n📝 Note: The category will appear in the Table of Contents only after")
|
||||
print(" resources are added to it. This is by design to keep the ToC clean.")
|
||||
else:
|
||||
# Run in interactive mode
|
||||
interactive_mode(adder)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,137 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Unified category utilities for awesome-claude-code.
|
||||
Provides a single source of truth for all category-related operations.
|
||||
|
||||
Usage:
|
||||
from scripts.categories.category_utils import category_manager
|
||||
|
||||
# Get all categories
|
||||
categories = category_manager.get_all_categories()
|
||||
|
||||
# Get category by name
|
||||
cat = category_manager.get_category_by_name("Status Lines")
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any, ClassVar
|
||||
|
||||
import yaml
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
|
||||
class CategoryManager:
|
||||
"""Singleton class for managing category definitions."""
|
||||
|
||||
_instance: ClassVar[CategoryManager | None] = None
|
||||
_data: ClassVar[dict[str, Any] | None] = None
|
||||
|
||||
def __new__(cls):
|
||||
"""Ensure only one instance exists."""
|
||||
if cls._instance is None:
|
||||
cls._instance = super().__new__(cls)
|
||||
return cls._instance
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the manager (only loads data once)."""
|
||||
if self._data is None:
|
||||
self._load_categories()
|
||||
|
||||
def _load_categories(self) -> None:
|
||||
"""Load category definitions from the unified YAML file."""
|
||||
categories_path = REPO_ROOT / "templates" / "categories.yaml"
|
||||
|
||||
with open(categories_path, encoding="utf-8") as f:
|
||||
type(self)._data = yaml.safe_load(f)
|
||||
|
||||
def get_all_categories(self) -> list[str]:
|
||||
"""Get list of all category names."""
|
||||
if self._data is None:
|
||||
return []
|
||||
return [cat["name"] for cat in self._data["categories"]]
|
||||
|
||||
def get_category_prefixes(self) -> dict[str, str]:
|
||||
"""Get mapping of category names to ID prefixes."""
|
||||
if self._data is None:
|
||||
return {}
|
||||
return {cat["name"]: cat["prefix"] for cat in self._data["categories"]}
|
||||
|
||||
def get_category_by_name(self, name: str) -> dict[str, Any] | None:
|
||||
"""Get category configuration by name."""
|
||||
if not self._data or "categories" not in self._data:
|
||||
return None
|
||||
for cat in self._data["categories"]:
|
||||
if cat["name"] == name:
|
||||
return cat
|
||||
return None
|
||||
|
||||
def get_category_by_id(self, cat_id: str) -> dict[str, Any] | None:
|
||||
"""Get category configuration by ID."""
|
||||
if not self._data or "categories" not in self._data:
|
||||
return None
|
||||
for cat in self._data["categories"]:
|
||||
if cat["id"] == cat_id:
|
||||
return cat
|
||||
return None
|
||||
|
||||
def get_all_subcategories(self) -> list[dict[str, str]]:
|
||||
"""Get all subcategories with their parent category names."""
|
||||
subcategories = []
|
||||
|
||||
if not self._data or "categories" not in self._data:
|
||||
return []
|
||||
|
||||
for cat in self._data["categories"]:
|
||||
if "subcategories" in cat:
|
||||
for subcat in cat["subcategories"]:
|
||||
subcategories.append(
|
||||
{
|
||||
"parent": cat["name"],
|
||||
"name": subcat["name"],
|
||||
"full_name": f"{cat['name']}: {subcat['name']}",
|
||||
}
|
||||
)
|
||||
|
||||
return subcategories
|
||||
|
||||
def get_subcategories_for_category(self, category_name: str) -> list[str]:
|
||||
"""Get subcategories for a specific category."""
|
||||
cat = self.get_category_by_name(category_name)
|
||||
if not cat or "subcategories" not in cat:
|
||||
return []
|
||||
|
||||
return [subcat["name"] for subcat in cat["subcategories"]]
|
||||
|
||||
def validate_category_subcategory(self, category: str, subcategory: str | None) -> bool:
|
||||
"""Validate that a subcategory belongs to the given category."""
|
||||
if not subcategory:
|
||||
return True
|
||||
|
||||
cat = self.get_category_by_name(category)
|
||||
if not cat:
|
||||
return False
|
||||
|
||||
if "subcategories" not in cat:
|
||||
return False
|
||||
|
||||
return any(subcat["name"] == subcategory for subcat in cat["subcategories"])
|
||||
|
||||
def get_categories_for_readme(self) -> list[dict[str, Any]]:
|
||||
"""Get categories in order for README generation."""
|
||||
if not self._data or "categories" not in self._data:
|
||||
return []
|
||||
categories = sorted(self._data["categories"], key=lambda x: x.get("order", 999))
|
||||
return categories
|
||||
|
||||
def get_toc_config(self) -> dict[str, Any]:
|
||||
"""Get table of contents configuration."""
|
||||
return self._data.get("toc", {}) if self._data else {}
|
||||
|
||||
|
||||
# Create singleton instance for import
|
||||
category_manager = CategoryManager()
|
||||
@@ -0,0 +1,91 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate responsive SVG logos for the Awesome Claude Code repository.
|
||||
|
||||
This script creates:
|
||||
- Light and dark theme versions of the ASCII art logo
|
||||
- The same logo is used for all screen sizes (scales responsively)
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
# ASCII art for the desktop version
|
||||
ASCII_ART = [
|
||||
" █████┐ ██┐ ██┐███████┐███████┐ ██████┐ ███┐ ███┐███████┐",
|
||||
"██┌──██┐██│ ██│██┌────┘██┌────┘██┌───██┐████┐ ████│██┌────┘",
|
||||
"███████│██│ █┐ ██│█████┐ ███████┐██│ ██│██┌████┌██│█████┐",
|
||||
"██┌──██│██│███┐██│██┌──┘ └────██│██│ ██│██│└██┌┘██│██┌──┘",
|
||||
"██│ ██│└███┌███┌┘███████┐███████│└██████┌┘██│ └─┘ ██│███████┐",
|
||||
"└─┘ └─┘ └──┘└──┘ └──────┘└──────┘ └─────┘ └─┘ └─┘└──────┘",
|
||||
"",
|
||||
"────────────────────────────────────────────────────────────────────────────────────",
|
||||
"",
|
||||
" ██████┐██┐ █████┐ ██┐ ██┐██████┐ ███████┐ ██████┐ ██████┐ ██████┐ ███████┐",
|
||||
"██┌────┘██│ ██┌──██┐██│ ██│██┌──██┐██┌────┘ ██┌────┘██┌───██┐██┌──██┐██┌────┘",
|
||||
"██│ ██│ ███████│██│ ██│██│ ██│█████┐ ██│ ██│ ██│██│ ██│█████┐",
|
||||
"██│ ██│ ██┌──██│██│ ██│██│ ██│██┌──┘ ██│ ██│ ██│██│ ██│██┌──┘",
|
||||
"└██████┐███████┐██│ ██│└██████┌┘██████┌┘███████┐ └██████┐└██████┌┘██████┌┘███████┐",
|
||||
" └─────┘└──────┘└─┘ └─┘ └─────┘ └─────┘ └──────┘ └─────┘ └─────┘ └─────┘ └──────┘",
|
||||
]
|
||||
|
||||
|
||||
def generate_logo_svg(theme: str = "light") -> str:
|
||||
"""Generate SVG with full ASCII art for all screen sizes."""
|
||||
fill_color = "#24292e" if theme == "light" else "#e1e4e8"
|
||||
|
||||
svg_lines = [
|
||||
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 920 320" '
|
||||
'preserveAspectRatio="xMidYMid meet">',
|
||||
" <style>",
|
||||
" text {",
|
||||
" font-family: 'Courier New', Courier, monospace;",
|
||||
" font-size: 14px;",
|
||||
f" fill: {fill_color};",
|
||||
" white-space: pre;",
|
||||
" }",
|
||||
" </style>",
|
||||
]
|
||||
|
||||
# Add each line of ASCII art as a text element
|
||||
y_position = 25
|
||||
for line in ASCII_ART:
|
||||
svg_lines.append(f' <text x="10" y="{y_position}">{line}</text>')
|
||||
y_position += 20
|
||||
|
||||
svg_lines.append("</svg>")
|
||||
return "\n".join(svg_lines)
|
||||
|
||||
|
||||
def main():
|
||||
"""Generate all logo SVG files."""
|
||||
# Get the project root directory
|
||||
assets_dir = REPO_ROOT / "assets"
|
||||
|
||||
# Create assets directory if it doesn't exist
|
||||
assets_dir.mkdir(exist_ok=True)
|
||||
|
||||
# Generate logo SVGs (same for all screen sizes)
|
||||
logo_light = generate_logo_svg("light")
|
||||
logo_dark = generate_logo_svg("dark")
|
||||
|
||||
# Write files
|
||||
files_to_write = {
|
||||
"logo-light.svg": logo_light,
|
||||
"logo-dark.svg": logo_dark,
|
||||
}
|
||||
|
||||
for filename, content in files_to_write.items():
|
||||
filepath = assets_dir / filename
|
||||
filepath.write_text(content, encoding="utf-8")
|
||||
print(f"✅ Generated: {filepath}")
|
||||
|
||||
print("\n🎨 All logo SVG files have been generated successfully!")
|
||||
print("📝 Run 'make generate' to update the README with the new logos.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Simple script to generate a resource ID for manual CSV additions.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from scripts.categories.category_utils import category_manager
|
||||
from scripts.ids.resource_id import generate_resource_id
|
||||
|
||||
|
||||
def main():
|
||||
print("Resource ID Generator")
|
||||
print("=" * 40)
|
||||
|
||||
# Get input
|
||||
display_name = input("Display Name: ").strip()
|
||||
primary_link = input("Primary Link: ").strip()
|
||||
|
||||
categories = category_manager.get_all_categories()
|
||||
print("\nAvailable categories:")
|
||||
for i, cat in enumerate(categories, 1):
|
||||
print(f"{i}. {cat}")
|
||||
|
||||
cat_choice = input("\nSelect category number: ").strip()
|
||||
try:
|
||||
category = categories[int(cat_choice) - 1]
|
||||
except (ValueError, IndexError):
|
||||
print("Invalid category selection. Using custom category.")
|
||||
category = input("Enter custom category: ").strip()
|
||||
|
||||
# Generate ID
|
||||
resource_id = generate_resource_id(display_name, primary_link, category)
|
||||
|
||||
print(f"\nGenerated ID: {resource_id}")
|
||||
print("\nCSV Row Preview:")
|
||||
print(f"ID: {resource_id}")
|
||||
print(f"Display Name: {display_name}")
|
||||
print(f"Category: {category}")
|
||||
print(f"Primary Link: {primary_link}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
38
.agent/knowledge/awesome_claude/scripts/ids/resource_id.py
Normal file
38
.agent/knowledge/awesome_claude/scripts/ids/resource_id.py
Normal file
@@ -0,0 +1,38 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Shared resource ID generation functionality.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from scripts.categories.category_utils import category_manager # noqa: E402
|
||||
|
||||
|
||||
def generate_resource_id(display_name: str, primary_link: str, category: str) -> str:
|
||||
"""
|
||||
Generate a stable resource ID from display name, link, and category.
|
||||
|
||||
Args:
|
||||
display_name: The display name of the resource
|
||||
primary_link: The primary URL of the resource
|
||||
category: The category name
|
||||
|
||||
Returns:
|
||||
A resource ID in format: {prefix}-{hash}
|
||||
"""
|
||||
# Get category prefix mapping
|
||||
prefixes = category_manager.get_category_prefixes()
|
||||
prefix = prefixes.get(category, "res")
|
||||
|
||||
# Generate hash from display name + primary link
|
||||
content = f"{display_name}{primary_link}"
|
||||
hash_value = hashlib.sha256(content.encode()).hexdigest()[:8]
|
||||
|
||||
return f"{prefix}-{hash_value}"
|
||||
@@ -0,0 +1,250 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Repository health check script for the Awesome Claude Code repository.
|
||||
|
||||
This script checks active GitHub repositories listed in THE_RESOURCES_TABLE.csv for:
|
||||
- Number of open issues
|
||||
- Date of last push or PR merge (last updated)
|
||||
|
||||
Exits with error if any repository:
|
||||
- Has not been updated in over 6 months AND
|
||||
- Has more than 2 open issues
|
||||
|
||||
If a repository has been deleted, the script continues without exiting.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import requests
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from scripts.utils.github_utils import parse_github_url
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||
|
||||
load_dotenv()
|
||||
|
||||
GITHUB_TOKEN = os.getenv("GITHUB_TOKEN", "")
|
||||
USER_AGENT = "awesome-claude-code Repository Health Check/1.0"
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
INPUT_FILE = REPO_ROOT / "THE_RESOURCES_TABLE.csv"
|
||||
HEADERS = {"User-Agent": USER_AGENT, "Accept": "application/vnd.github+json"}
|
||||
if GITHUB_TOKEN:
|
||||
HEADERS["Authorization"] = f"Bearer {GITHUB_TOKEN}"
|
||||
|
||||
# Thresholds
|
||||
MONTHS_THRESHOLD = 6
|
||||
OPEN_ISSUES_THRESHOLD = 2
|
||||
|
||||
|
||||
def get_repo_info(owner, repo):
|
||||
"""
|
||||
Fetch repository information from GitHub API.
|
||||
Returns a dict with:
|
||||
- open_issues: number of open issues
|
||||
- last_updated: date of last push (ISO format string)
|
||||
- exists: whether the repo exists (False if 404)
|
||||
Returns None if API call fails for other reasons.
|
||||
"""
|
||||
api_url = f"https://api.github.com/repos/{owner}/{repo}"
|
||||
|
||||
try:
|
||||
response = requests.get(api_url, headers=HEADERS, timeout=10)
|
||||
|
||||
if response.status_code == 404:
|
||||
logger.warning(f"Repository {owner}/{repo} not found (deleted or private)")
|
||||
return {"exists": False, "open_issues": 0, "last_updated": None}
|
||||
|
||||
if response.status_code == 403:
|
||||
logger.error(f"Rate limit or forbidden for {owner}/{repo}")
|
||||
return None
|
||||
|
||||
if response.status_code != 200:
|
||||
logger.error(f"Failed to fetch {owner}/{repo}: HTTP {response.status_code}")
|
||||
return None
|
||||
|
||||
data = response.json()
|
||||
|
||||
return {
|
||||
"exists": True,
|
||||
"open_issues": data.get("open_issues_count", data.get("open_issues", 0)),
|
||||
"last_updated": data.get("pushed_at"), # ISO 8601 timestamp
|
||||
}
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
logger.error(f"Error fetching repository info for {owner}/{repo}: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def is_outdated(last_updated_str, months_threshold):
|
||||
"""
|
||||
Check if a repository hasn't been updated in more than months_threshold months.
|
||||
"""
|
||||
if not last_updated_str:
|
||||
return True # Consider it outdated if we don't have a date
|
||||
|
||||
try:
|
||||
last_updated = datetime.fromisoformat(last_updated_str.replace("Z", "+00:00"))
|
||||
now = datetime.now(UTC)
|
||||
threshold_date = now - timedelta(days=months_threshold * 30)
|
||||
return last_updated < threshold_date
|
||||
except (ValueError, AttributeError) as e:
|
||||
logger.warning(f"Could not parse date '{last_updated_str}': {e}")
|
||||
return True
|
||||
|
||||
|
||||
def check_repos_health(
|
||||
csv_file, months_threshold=MONTHS_THRESHOLD, issues_threshold=OPEN_ISSUES_THRESHOLD
|
||||
):
|
||||
"""
|
||||
Check health of all active GitHub repositories in the CSV.
|
||||
Returns a list of problematic repos.
|
||||
"""
|
||||
problematic_repos = []
|
||||
checked_repos = 0
|
||||
deleted_repos = []
|
||||
|
||||
logger.info(f"Reading repository list from {csv_file}")
|
||||
|
||||
try:
|
||||
with open(csv_file, encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
|
||||
for row in reader:
|
||||
# Check if Active is TRUE
|
||||
active = row.get("Active", "").strip().upper()
|
||||
if active != "TRUE":
|
||||
continue
|
||||
|
||||
primary_link = row.get("Primary Link", "").strip()
|
||||
if not primary_link:
|
||||
continue
|
||||
|
||||
# Extract owner and repo from GitHub URL
|
||||
_, is_github, owner, repo = parse_github_url(primary_link)
|
||||
if not is_github or not owner or not repo:
|
||||
# Not a GitHub repository URL
|
||||
continue
|
||||
|
||||
checked_repos += 1
|
||||
resource_name = row.get("Display Name", primary_link)
|
||||
logger.info(f"Checking {owner}/{repo} ({resource_name})")
|
||||
|
||||
# Get repository information
|
||||
repo_info = get_repo_info(owner, repo)
|
||||
|
||||
if repo_info is None:
|
||||
# API error - log but continue
|
||||
logger.warning(f"Could not fetch info for {owner}/{repo}, skipping")
|
||||
continue
|
||||
|
||||
if not repo_info["exists"]:
|
||||
# Repository deleted - log but continue
|
||||
deleted_repos.append(
|
||||
{"name": resource_name, "url": primary_link, "owner": owner, "repo": repo}
|
||||
)
|
||||
continue
|
||||
|
||||
# Check if repo is problematic
|
||||
open_issues = repo_info["open_issues"]
|
||||
last_updated = repo_info["last_updated"]
|
||||
outdated = is_outdated(last_updated, months_threshold)
|
||||
|
||||
if outdated and open_issues > issues_threshold:
|
||||
problematic_repos.append(
|
||||
{
|
||||
"name": resource_name,
|
||||
"url": primary_link,
|
||||
"owner": owner,
|
||||
"repo": repo,
|
||||
"open_issues": open_issues,
|
||||
"last_updated": last_updated,
|
||||
}
|
||||
)
|
||||
logger.warning(
|
||||
f"⚠️ {owner}/{repo}: "
|
||||
f"Last updated {last_updated or 'unknown'}, "
|
||||
f"{open_issues} open issues"
|
||||
)
|
||||
|
||||
except FileNotFoundError:
|
||||
logger.error(f"CSV file not found: {csv_file}")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
logger.error(f"Error reading CSV file: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
logger.info(f"\n{'=' * 60}")
|
||||
logger.info("Summary:")
|
||||
logger.info(f" Total active GitHub repositories checked: {checked_repos}")
|
||||
logger.info(f" Deleted/unavailable repositories: {len(deleted_repos)}")
|
||||
logger.info(f" Problematic repositories: {len(problematic_repos)}")
|
||||
|
||||
if deleted_repos:
|
||||
logger.info(f"\n{'=' * 60}")
|
||||
logger.info("Deleted/Unavailable Repositories:")
|
||||
for repo in deleted_repos:
|
||||
logger.info(f" - {repo['name']} ({repo['owner']}/{repo['repo']})")
|
||||
|
||||
return problematic_repos
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Check health of GitHub repositories in THE_RESOURCES_TABLE.csv"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--csv-file",
|
||||
default=INPUT_FILE,
|
||||
help=f"Path to CSV file (default: {INPUT_FILE})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--months",
|
||||
type=int,
|
||||
default=MONTHS_THRESHOLD,
|
||||
help=f"Months threshold for outdated repos (default: {MONTHS_THRESHOLD})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--issues",
|
||||
type=int,
|
||||
default=OPEN_ISSUES_THRESHOLD,
|
||||
help=f"Open issues threshold (default: {OPEN_ISSUES_THRESHOLD})",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
problematic_repos = check_repos_health(args.csv_file, args.months, args.issues)
|
||||
|
||||
if problematic_repos:
|
||||
logger.error(f"\n{'=' * 60}")
|
||||
logger.error("❌ HEALTH CHECK FAILED")
|
||||
logger.error(
|
||||
f"Found {len(problematic_repos)} repository(ies) that have not been updated in over "
|
||||
f"{args.months} months and have more than {args.issues} open issues:\n"
|
||||
)
|
||||
|
||||
for repo in problematic_repos:
|
||||
logger.error(f" • {repo['name']}")
|
||||
logger.error(f" URL: {repo['url']}")
|
||||
logger.error(f" Last updated: {repo['last_updated'] or 'Unknown'}")
|
||||
logger.error(f" Open issues: {repo['open_issues']}")
|
||||
logger.error("")
|
||||
|
||||
sys.exit(1)
|
||||
else:
|
||||
logger.info(f"\n{'=' * 60}")
|
||||
logger.info("✅ HEALTH CHECK PASSED")
|
||||
logger.info("All active repositories are healthy!")
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,223 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Update Last Modified and GitHub release info for active GitHub repos in THE_RESOURCES_TABLE.csv.
|
||||
|
||||
Uses two GitHub REST API calls per repository:
|
||||
- /repos/{owner}/{repo}/commits?per_page=1 (latest commit on default branch)
|
||||
- /repos/{owner}/{repo}/releases/latest (latest release)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import requests
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
try:
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
DEFAULT_CSV_PATH = os.path.join(REPO_ROOT, "THE_RESOURCES_TABLE.csv")
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||
|
||||
GITHUB_TOKEN = os.getenv("GITHUB_TOKEN", "")
|
||||
USER_AGENT = "awesome-claude-code GitHub Release Sync/1.0"
|
||||
HEADERS = {"User-Agent": USER_AGENT, "Accept": "application/vnd.github+json"}
|
||||
if GITHUB_TOKEN:
|
||||
HEADERS["Authorization"] = f"Bearer {GITHUB_TOKEN}"
|
||||
|
||||
|
||||
def format_commit_date(commit_date: str | None) -> str | None:
|
||||
if not commit_date:
|
||||
return None
|
||||
try:
|
||||
dt = datetime.fromisoformat(commit_date.replace("Z", "+00:00"))
|
||||
return dt.strftime("%Y-%m-%d:%H-%M-%S")
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def parse_github_repo(url: str | None) -> tuple[str | None, str | None]:
|
||||
if not url or not isinstance(url, str):
|
||||
return None, None
|
||||
match = re.match(r"https?://github\.com/([^/]+)/([^/]+)", url.strip())
|
||||
if not match:
|
||||
return None, None
|
||||
owner, repo = match.groups()
|
||||
repo = repo.split("?", 1)[0].split("#", 1)[0]
|
||||
repo = repo.removesuffix(".git")
|
||||
return owner, repo
|
||||
|
||||
|
||||
def github_get(url: str, params: dict | None = None) -> requests.Response:
|
||||
response = requests.get(url, headers=HEADERS, params=params, timeout=10)
|
||||
if response.status_code == 403 and response.headers.get("X-RateLimit-Remaining") == "0":
|
||||
reset_time = int(response.headers.get("X-RateLimit-Reset", 0))
|
||||
sleep_time = max(reset_time - int(time.time()), 0) + 1
|
||||
logger.warning("GitHub rate limit hit. Sleeping for %s seconds.", sleep_time)
|
||||
time.sleep(sleep_time)
|
||||
response = requests.get(url, headers=HEADERS, params=params, timeout=10)
|
||||
return response
|
||||
|
||||
|
||||
def fetch_last_commit_date(owner: str, repo: str) -> tuple[str | None, str]:
|
||||
api_url = f"https://api.github.com/repos/{owner}/{repo}/commits"
|
||||
response = github_get(api_url, params={"per_page": 1})
|
||||
|
||||
if response.status_code == 200:
|
||||
data = response.json()
|
||||
if isinstance(data, list) and data:
|
||||
commit = data[0]
|
||||
commit_date = (
|
||||
commit.get("commit", {}).get("committer", {}).get("date")
|
||||
or commit.get("commit", {}).get("author", {}).get("date")
|
||||
or commit.get("committer", {}).get("date")
|
||||
or commit.get("author", {}).get("date")
|
||||
)
|
||||
return format_commit_date(commit_date), "ok"
|
||||
return None, "empty"
|
||||
if response.status_code == 404:
|
||||
return None, "not_found"
|
||||
return None, f"http_{response.status_code}"
|
||||
|
||||
|
||||
def fetch_latest_release(owner: str, repo: str) -> tuple[str | None, str | None, str]:
|
||||
api_url = f"https://api.github.com/repos/{owner}/{repo}/releases/latest"
|
||||
response = github_get(api_url)
|
||||
|
||||
if response.status_code == 200:
|
||||
data = response.json()
|
||||
published_at = data.get("published_at") or data.get("created_at")
|
||||
return format_commit_date(published_at), data.get("tag_name"), "ok"
|
||||
if response.status_code == 404:
|
||||
return None, None, "no_release"
|
||||
return None, None, f"http_{response.status_code}"
|
||||
|
||||
|
||||
def update_release_data(csv_path: str, max_rows: int | None = None, dry_run: bool = False) -> None:
|
||||
with open(csv_path, encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
rows = list(reader)
|
||||
fieldnames = list(reader.fieldnames or [])
|
||||
|
||||
required_columns = ["Last Modified", "Latest Release", "Release Version", "Release Source"]
|
||||
for column in required_columns:
|
||||
if column not in fieldnames:
|
||||
fieldnames.append(column)
|
||||
|
||||
processed = 0
|
||||
skipped = 0
|
||||
updated = 0
|
||||
errors = 0
|
||||
|
||||
for _, row in enumerate(rows):
|
||||
if max_rows and processed >= max_rows:
|
||||
logger.info("Reached max limit (%s). Stopping.", max_rows)
|
||||
break
|
||||
|
||||
if row.get("Active", "").strip().upper() != "TRUE":
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
primary_link = (row.get("Primary Link") or "").strip()
|
||||
owner, repo = parse_github_repo(primary_link)
|
||||
if not owner or not repo:
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
processed += 1
|
||||
display_name = row.get("Display Name", primary_link)
|
||||
logger.info("[%s] Updating %s (%s/%s)", processed, display_name, owner, repo)
|
||||
|
||||
row_changed = False
|
||||
|
||||
commit_date, commit_status = fetch_last_commit_date(owner, repo)
|
||||
if commit_status == "not_found":
|
||||
logger.warning("Repository not found: %s/%s", owner, repo)
|
||||
elif commit_date and row.get("Last Modified") != commit_date:
|
||||
row["Last Modified"] = commit_date
|
||||
row_changed = True
|
||||
|
||||
release_date, release_version, release_status = fetch_latest_release(owner, repo)
|
||||
if release_status == "no_release":
|
||||
if row.get("Latest Release") or row.get("Release Version") or row.get("Release Source"):
|
||||
row["Latest Release"] = ""
|
||||
row["Release Version"] = ""
|
||||
row["Release Source"] = ""
|
||||
row_changed = True
|
||||
elif release_status == "ok":
|
||||
new_release_date = release_date or ""
|
||||
new_release_version = release_version or ""
|
||||
new_release_source = "github-releases" if (release_date or release_version) else ""
|
||||
if row.get("Latest Release") != new_release_date:
|
||||
row["Latest Release"] = new_release_date
|
||||
row_changed = True
|
||||
if row.get("Release Version") != new_release_version:
|
||||
row["Release Version"] = new_release_version
|
||||
row_changed = True
|
||||
if row.get("Release Source") != new_release_source:
|
||||
row["Release Source"] = new_release_source
|
||||
row_changed = True
|
||||
else:
|
||||
logger.warning(
|
||||
"Release fetch failed for %s/%s (status: %s)",
|
||||
owner,
|
||||
repo,
|
||||
release_status,
|
||||
)
|
||||
errors += 1
|
||||
|
||||
if row_changed:
|
||||
updated += 1
|
||||
|
||||
if dry_run:
|
||||
logger.info("[DRY RUN] No changes written to CSV.")
|
||||
return
|
||||
|
||||
with open(csv_path, "w", encoding="utf-8", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=fieldnames)
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
|
||||
logger.info("Updated rows: %s", updated)
|
||||
logger.info("Skipped rows: %s", skipped)
|
||||
logger.info("Errors: %s", errors)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Update GitHub commit and release data for active resources"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--csv-file",
|
||||
default=DEFAULT_CSV_PATH,
|
||||
help="Path to THE_RESOURCES_TABLE.csv",
|
||||
)
|
||||
parser.add_argument("--max", type=int, help="Process at most N resources")
|
||||
parser.add_argument("--dry-run", action="store_true", help="Do not write changes")
|
||||
args = parser.parse_args()
|
||||
|
||||
if not os.path.exists(args.csv_file):
|
||||
logger.error("CSV file not found: %s", args.csv_file)
|
||||
sys.exit(1)
|
||||
|
||||
update_release_data(args.csv_file, max_rows=args.max, dry_run=args.dry_run)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
0
.agent/knowledge/awesome_claude/scripts/py.typed
Normal file
0
.agent/knowledge/awesome_claude/scripts/py.typed
Normal file
@@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Template-based README generator for the Awesome Claude Code repository.
|
||||
Reads resource metadata from CSV and generates README using templates.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.readme.generators.awesome import AwesomeReadmeGenerator
|
||||
from scripts.readme.generators.base import ReadmeGenerator
|
||||
from scripts.readme.generators.flat import (
|
||||
FLAT_CATEGORIES,
|
||||
FLAT_SORT_TYPES,
|
||||
ParameterizedFlatListGenerator,
|
||||
)
|
||||
from scripts.readme.generators.minimal import MinimalReadmeGenerator
|
||||
from scripts.readme.generators.visual import VisualReadmeGenerator
|
||||
from scripts.readme.helpers.readme_assets import generate_flat_badges
|
||||
from scripts.readme.helpers.readme_config import get_root_style
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
STYLE_GENERATORS: dict[str, type[ReadmeGenerator]] = {
|
||||
"extra": VisualReadmeGenerator,
|
||||
"classic": MinimalReadmeGenerator,
|
||||
"awesome": AwesomeReadmeGenerator,
|
||||
"flat": ParameterizedFlatListGenerator,
|
||||
}
|
||||
|
||||
PRIMARY_STYLE_IDS = tuple(
|
||||
style_id
|
||||
for style_id, generator_cls in STYLE_GENERATORS.items()
|
||||
if generator_cls is not ParameterizedFlatListGenerator
|
||||
)
|
||||
|
||||
|
||||
def build_root_generator(
|
||||
style_id: str,
|
||||
csv_path: str,
|
||||
template_dir: str,
|
||||
assets_dir: str,
|
||||
repo_root: str,
|
||||
) -> ReadmeGenerator:
|
||||
"""Return the generator instance for a root style."""
|
||||
style_id = style_id.lower()
|
||||
generator_cls = STYLE_GENERATORS.get(style_id)
|
||||
if generator_cls is None:
|
||||
raise ValueError(f"Unknown root style: {style_id}")
|
||||
if generator_cls is ParameterizedFlatListGenerator:
|
||||
return ParameterizedFlatListGenerator(
|
||||
csv_path,
|
||||
template_dir,
|
||||
assets_dir,
|
||||
repo_root,
|
||||
category_slug="all",
|
||||
sort_type="az",
|
||||
)
|
||||
return generator_cls(csv_path, template_dir, assets_dir, repo_root)
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point - generates all README versions."""
|
||||
repo_root = REPO_ROOT
|
||||
|
||||
csv_path = str(repo_root / "THE_RESOURCES_TABLE.csv")
|
||||
template_dir = str(repo_root / "templates")
|
||||
assets_dir = str(repo_root / "assets")
|
||||
|
||||
print("=== README Generation ===")
|
||||
|
||||
# Generate flat list badges first
|
||||
print("\n--- Generating flat list badges ---")
|
||||
generate_flat_badges(assets_dir, FLAT_SORT_TYPES, FLAT_CATEGORIES)
|
||||
print("✅ Flat list badges generated")
|
||||
|
||||
# Generate primary styles under README_ALTERNATIVES/
|
||||
main_generators = [
|
||||
STYLE_GENERATORS[style_id](csv_path, template_dir, assets_dir, str(repo_root))
|
||||
for style_id in PRIMARY_STYLE_IDS
|
||||
]
|
||||
|
||||
for generator in main_generators:
|
||||
resolved_path = generator.resolved_output_path
|
||||
print(f"\n--- Generating {resolved_path} ---")
|
||||
try:
|
||||
resource_count, backup_path = generator.generate()
|
||||
print(f"✅ {resolved_path} generated successfully")
|
||||
print(f"📊 Generated with {resource_count} active resources")
|
||||
if backup_path:
|
||||
print(f"📁 Backup saved at: {backup_path}")
|
||||
except Exception as e:
|
||||
print(f"❌ Error generating {resolved_path}: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# Generate all flat list combinations (categories × sort types = 44 files)
|
||||
print("\n--- Generating flat list views ---")
|
||||
flat_count = 0
|
||||
for category_slug in FLAT_CATEGORIES:
|
||||
for sort_type in FLAT_SORT_TYPES:
|
||||
generator = ParameterizedFlatListGenerator(
|
||||
csv_path,
|
||||
template_dir,
|
||||
assets_dir,
|
||||
str(repo_root),
|
||||
category_slug=category_slug,
|
||||
sort_type=sort_type,
|
||||
)
|
||||
try:
|
||||
resource_count, _ = generator.generate()
|
||||
flat_count += 1
|
||||
# Only print summary for first of each category
|
||||
if sort_type == "az":
|
||||
print(f" 📂 {category_slug}: {resource_count} resources")
|
||||
except Exception as e:
|
||||
print(f"❌ Error generating {generator.output_filename}: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"✅ Generated {flat_count} flat list views")
|
||||
|
||||
# Generate root README after all alternatives exist
|
||||
root_style = get_root_style()
|
||||
root_generator = build_root_generator(
|
||||
root_style,
|
||||
csv_path,
|
||||
template_dir,
|
||||
assets_dir,
|
||||
str(repo_root),
|
||||
)
|
||||
print(f"\n--- Generating README.md (root style: {root_style}) ---")
|
||||
try:
|
||||
resource_count, backup_path = root_generator.generate(output_path="README.md")
|
||||
print("✅ README.md generated successfully")
|
||||
print(f"📊 Generated with {resource_count} active resources")
|
||||
if backup_path:
|
||||
print(f"📁 Backup saved at: {backup_path}")
|
||||
except Exception as e:
|
||||
print(f"❌ Error generating README.md: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
print("\n=== Generation Complete ===")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
"""README generator implementations."""
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Awesome README generator implementation."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.readme.generators.base import ReadmeGenerator
|
||||
from scripts.readme.markup.awesome import (
|
||||
format_resource_entry as format_awesome_resource_entry,
|
||||
)
|
||||
from scripts.readme.markup.awesome import (
|
||||
generate_repo_ticker as generate_awesome_repo_ticker,
|
||||
)
|
||||
from scripts.readme.markup.awesome import (
|
||||
generate_section_content as generate_awesome_section_content,
|
||||
)
|
||||
from scripts.readme.markup.awesome import (
|
||||
generate_toc as generate_awesome_toc,
|
||||
)
|
||||
from scripts.readme.markup.awesome import (
|
||||
generate_weekly_section as generate_awesome_weekly_section,
|
||||
)
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
|
||||
class AwesomeReadmeGenerator(ReadmeGenerator):
|
||||
"""Generator for awesome-list-style README variant with clean markdown formatting."""
|
||||
|
||||
@property
|
||||
def template_filename(self) -> str:
|
||||
return "README_AWESOME.template.md"
|
||||
|
||||
@property
|
||||
def output_filename(self) -> str:
|
||||
return "README_ALTERNATIVES/README_AWESOME.md"
|
||||
|
||||
@property
|
||||
def style_id(self) -> str:
|
||||
return "awesome"
|
||||
|
||||
def format_resource_entry(self, row: dict, include_separator: bool = True) -> str:
|
||||
"""Format resource in awesome list style: - [Name](url) by [Author](link) - Description."""
|
||||
return format_awesome_resource_entry(row, include_separator=include_separator)
|
||||
|
||||
def generate_toc(self) -> str:
|
||||
"""Generate plain markdown TOC for awesome list style."""
|
||||
return generate_awesome_toc(self.categories, self.csv_data)
|
||||
|
||||
def generate_weekly_section(self) -> str:
|
||||
"""Generate weekly section with plain markdown for awesome list."""
|
||||
return generate_awesome_weekly_section(self.csv_data)
|
||||
|
||||
def generate_section_content(self, category: dict, section_index: int) -> str:
|
||||
"""Generate section with plain markdown headers in awesome list format."""
|
||||
_ = section_index
|
||||
return generate_awesome_section_content(category, self.csv_data)
|
||||
|
||||
def generate_repo_ticker(self) -> str:
|
||||
"""Generate the awesome-style animated SVG repo ticker."""
|
||||
return generate_awesome_repo_ticker()
|
||||
|
||||
def generate_banner_image(self, output_path: Path) -> str:
|
||||
"""Generate centered banner image for Awesome style README."""
|
||||
repo_root = find_repo_root(Path(__file__))
|
||||
banner_file = "assets/awesome-claude-code-social-clawd-leo.png"
|
||||
|
||||
# Calculate relative path from output location to banner
|
||||
banner_abs = repo_root / banner_file
|
||||
rel_path = Path(os.path.relpath(banner_abs, start=output_path.parent)).as_posix()
|
||||
|
||||
return f"""<p align="center">
|
||||
<picture>
|
||||
<img src="{rel_path}" alt="Awesome Claude Code" width="600">
|
||||
</picture>
|
||||
</p>"""
|
||||
@@ -0,0 +1,278 @@
|
||||
"""Shared base class and helpers for README generators."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import csv
|
||||
import os
|
||||
import shutil
|
||||
from abc import ABC, abstractmethod
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import yaml # type: ignore[import-untyped]
|
||||
|
||||
from scripts.readme.helpers.readme_config import get_root_style
|
||||
from scripts.readme.helpers.readme_paths import (
|
||||
ensure_generated_header,
|
||||
resolve_asset_tokens,
|
||||
)
|
||||
from scripts.readme.helpers.readme_utils import build_general_anchor_map
|
||||
from scripts.readme.markup.shared import generate_style_selector, load_announcements
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
|
||||
def load_template(template_path: str) -> str:
|
||||
"""Load a template file."""
|
||||
with open(template_path, encoding="utf-8") as f:
|
||||
return f.read()
|
||||
|
||||
|
||||
def load_overrides(template_dir: str) -> dict:
|
||||
"""Load resource overrides."""
|
||||
override_path = os.path.join(template_dir, "resource-overrides.yaml")
|
||||
if not os.path.exists(override_path):
|
||||
return {}
|
||||
|
||||
with open(override_path, encoding="utf-8") as f:
|
||||
data = yaml.safe_load(f)
|
||||
return data.get("overrides", {})
|
||||
|
||||
|
||||
def apply_overrides(row: dict, overrides: dict) -> dict:
|
||||
"""Apply overrides to a resource row."""
|
||||
resource_id = row.get("ID", "")
|
||||
if not resource_id or resource_id not in overrides:
|
||||
return row
|
||||
|
||||
override_config = overrides[resource_id]
|
||||
|
||||
for field, value in override_config.items():
|
||||
if field in ["skip_validation", "notes"]:
|
||||
continue
|
||||
if field.endswith("_locked"):
|
||||
continue
|
||||
|
||||
if field == "license":
|
||||
row["License"] = value
|
||||
elif field == "active":
|
||||
row["Active"] = value
|
||||
elif field == "description":
|
||||
row["Description"] = value
|
||||
|
||||
return row
|
||||
|
||||
|
||||
def create_backup(file_path: str, keep_latest: int = 1) -> str | None:
|
||||
"""Create a backup of the file if it exists, pruning older backups."""
|
||||
if not os.path.exists(file_path):
|
||||
return None
|
||||
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
backup_dir = os.path.join(REPO_ROOT, ".myob", "backups")
|
||||
os.makedirs(backup_dir, exist_ok=True)
|
||||
|
||||
backup_filename = f"{os.path.basename(file_path)}.{timestamp}.bak"
|
||||
backup_path = os.path.join(backup_dir, backup_filename)
|
||||
|
||||
shutil.copy2(file_path, backup_path)
|
||||
if keep_latest > 0:
|
||||
basename = os.path.basename(file_path)
|
||||
backups = []
|
||||
for name in os.listdir(backup_dir):
|
||||
if name.startswith(f"{basename}.") and name.endswith(".bak"):
|
||||
backups.append(os.path.join(backup_dir, name))
|
||||
backups.sort(key=os.path.getmtime, reverse=True)
|
||||
for stale_path in backups[keep_latest:]:
|
||||
with contextlib.suppress(OSError):
|
||||
os.remove(stale_path)
|
||||
return backup_path
|
||||
|
||||
|
||||
class ReadmeGenerator(ABC):
|
||||
"""Base class for README generation with shared logic."""
|
||||
|
||||
def __init__(self, csv_path: str, template_dir: str, assets_dir: str, repo_root: str) -> None:
|
||||
self.csv_path = csv_path
|
||||
self.template_dir = template_dir
|
||||
self.assets_dir = assets_dir
|
||||
self.repo_root = repo_root
|
||||
self.csv_data: list[dict] = []
|
||||
self.categories: list[dict] = []
|
||||
self.overrides: dict = {}
|
||||
self.announcements: str = ""
|
||||
self.footer: str = ""
|
||||
self.general_anchor_map: dict = {}
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def template_filename(self) -> str:
|
||||
"""Return the template filename to use."""
|
||||
...
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def output_filename(self) -> str:
|
||||
"""Return the preferred output filename for this style."""
|
||||
...
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def style_id(self) -> str:
|
||||
"""Return the style ID for this generator (extra, classic, awesome, flat)."""
|
||||
...
|
||||
|
||||
@property
|
||||
def is_root_style(self) -> bool:
|
||||
"""Check if this generator produces the root README style."""
|
||||
return self.style_id == get_root_style()
|
||||
|
||||
@property
|
||||
def resolved_output_path(self) -> str:
|
||||
"""Get the resolved output path for this generator."""
|
||||
if self.output_filename == "README.md":
|
||||
return f"README_ALTERNATIVES/README_{self.style_id.upper()}.md"
|
||||
return self.output_filename
|
||||
|
||||
def get_style_selector(self, output_path: Path) -> str:
|
||||
"""Generate the style selector HTML for this README."""
|
||||
return generate_style_selector(self.style_id, output_path)
|
||||
|
||||
@abstractmethod
|
||||
def format_resource_entry(self, row: dict, include_separator: bool = True) -> str:
|
||||
"""Format a single resource entry."""
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def generate_toc(self) -> str:
|
||||
"""Generate the table of contents."""
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def generate_weekly_section(self) -> str:
|
||||
"""Generate the weekly additions section."""
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def generate_section_content(self, category: dict, section_index: int) -> str:
|
||||
"""Generate content for a category section."""
|
||||
...
|
||||
|
||||
def generate_repo_ticker(self) -> str:
|
||||
"""Generate the repo ticker section."""
|
||||
return ""
|
||||
|
||||
def generate_banner_image(self, output_path: Path) -> str:
|
||||
"""Generate banner image HTML. Override in subclasses to add a banner."""
|
||||
_ = output_path
|
||||
return ""
|
||||
|
||||
def load_csv_data(self) -> list[dict]:
|
||||
"""Load and filter active resources from CSV."""
|
||||
csv_data = []
|
||||
with open(self.csv_path, newline="", encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
for row in reader:
|
||||
row = apply_overrides(row, self.overrides)
|
||||
if row["Active"].upper() == "TRUE":
|
||||
csv_data.append(row)
|
||||
return csv_data
|
||||
|
||||
def load_categories(self) -> list[dict]:
|
||||
"""Load categories from the category manager."""
|
||||
from scripts.categories.category_utils import category_manager
|
||||
|
||||
return category_manager.get_categories_for_readme()
|
||||
|
||||
def load_overrides(self) -> dict:
|
||||
"""Load resource overrides from YAML."""
|
||||
return load_overrides(self.template_dir)
|
||||
|
||||
def load_announcements(self) -> str:
|
||||
"""Load announcements from YAML."""
|
||||
return load_announcements(self.template_dir)
|
||||
|
||||
def load_footer(self) -> str:
|
||||
"""Load footer template from file."""
|
||||
footer_path = os.path.join(self.template_dir, "footer.template.md")
|
||||
try:
|
||||
with open(footer_path, encoding="utf-8") as f:
|
||||
return f.read()
|
||||
except FileNotFoundError:
|
||||
print(f"⚠️ Warning: Footer template not found at {footer_path}")
|
||||
return ""
|
||||
|
||||
def build_general_anchor_map(self) -> dict:
|
||||
"""Build anchor map for General subcategories."""
|
||||
return build_general_anchor_map(self.categories, self.csv_data)
|
||||
|
||||
def create_backup(self, output_path: str) -> str | None:
|
||||
"""Create backup of existing file."""
|
||||
return create_backup(output_path)
|
||||
|
||||
def generate(self, output_path: str | None = None) -> tuple[int, str | None]:
|
||||
"""Generate the README to the default or provided output path."""
|
||||
resolved_path = output_path or self.resolved_output_path
|
||||
output_path = os.path.join(self.repo_root, resolved_path)
|
||||
self.overrides = self.load_overrides()
|
||||
self.csv_data = self.load_csv_data()
|
||||
self.categories = self.load_categories()
|
||||
self.announcements = self.load_announcements()
|
||||
self.footer = self.load_footer()
|
||||
self.general_anchor_map = self.build_general_anchor_map()
|
||||
|
||||
template_path = os.path.join(self.template_dir, self.template_filename)
|
||||
template = load_template(template_path)
|
||||
|
||||
toc_content = self.generate_toc()
|
||||
weekly_section = self.generate_weekly_section()
|
||||
|
||||
body_sections = []
|
||||
for section_index, category in enumerate(self.categories):
|
||||
section_content = self.generate_section_content(category, section_index)
|
||||
body_sections.append(section_content)
|
||||
|
||||
readme_content = template
|
||||
readme_content = readme_content.replace("{{ANNOUNCEMENTS}}", self.announcements)
|
||||
readme_content = readme_content.replace("{{WEEKLY_SECTION}}", weekly_section)
|
||||
readme_content = readme_content.replace("{{TABLE_OF_CONTENTS}}", toc_content)
|
||||
readme_content = readme_content.replace(
|
||||
"{{BODY_SECTIONS}}", "\n<br>\n\n".join(body_sections)
|
||||
)
|
||||
readme_content = readme_content.replace("{{FOOTER}}", self.footer)
|
||||
readme_content = readme_content.replace(
|
||||
"{{STYLE_SELECTOR}}", self.get_style_selector(Path(output_path))
|
||||
)
|
||||
readme_content = readme_content.replace("{{REPO_TICKER}}", self.generate_repo_ticker())
|
||||
readme_content = readme_content.replace(
|
||||
"{{BANNER_IMAGE}}", self.generate_banner_image(Path(output_path))
|
||||
)
|
||||
|
||||
readme_content = ensure_generated_header(readme_content)
|
||||
readme_content = resolve_asset_tokens(
|
||||
readme_content, Path(output_path), Path(self.repo_root)
|
||||
)
|
||||
output_dir = os.path.dirname(output_path)
|
||||
if output_dir and not os.path.exists(output_dir):
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
backup_path = self.create_backup(output_path)
|
||||
|
||||
try:
|
||||
with open(output_path, "w", encoding="utf-8") as f:
|
||||
f.write(readme_content)
|
||||
except Exception as e:
|
||||
if backup_path:
|
||||
print(f"❌ Error writing {resolved_path}: {e}")
|
||||
print(f" Backup preserved at: {backup_path}")
|
||||
raise
|
||||
|
||||
return len(self.csv_data), backup_path
|
||||
|
||||
@property
|
||||
def alternative_output_path(self) -> str:
|
||||
"""Return the output path for this style under README_ALTERNATIVES/."""
|
||||
if self.output_filename == "README.md":
|
||||
return f"README_ALTERNATIVES/README_{self.style_id.upper()}.md"
|
||||
return self.output_filename
|
||||
@@ -0,0 +1,260 @@
|
||||
"""Flat list README generator implementation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.readme.generators.base import ReadmeGenerator, load_template
|
||||
from scripts.readme.helpers.readme_paths import (
|
||||
ensure_generated_header,
|
||||
resolve_asset_tokens,
|
||||
)
|
||||
from scripts.readme.helpers.readme_utils import parse_resource_date
|
||||
from scripts.readme.markup.flat import (
|
||||
generate_category_navigation as generate_flat_category_navigation,
|
||||
)
|
||||
from scripts.readme.markup.flat import (
|
||||
generate_navigation as generate_flat_navigation,
|
||||
)
|
||||
from scripts.readme.markup.flat import (
|
||||
generate_resources_table as generate_flat_resources_table,
|
||||
)
|
||||
from scripts.readme.markup.flat import (
|
||||
generate_sort_navigation as generate_flat_sort_navigation,
|
||||
)
|
||||
from scripts.readme.markup.flat import (
|
||||
get_default_template as get_flat_default_template,
|
||||
)
|
||||
|
||||
# Category definitions: slug -> (csv_value, display_name, badge_color)
|
||||
FLAT_CATEGORIES = {
|
||||
"all": (None, "All", "#71717a"),
|
||||
"tooling": ("Tooling", "Tooling", "#3b82f6"),
|
||||
"commands": ("Slash-Commands", "Commands", "#8b5cf6"),
|
||||
"claude-md": ("CLAUDE.md Files", "CLAUDE.md", "#ec4899"),
|
||||
"workflows": ("Workflows & Knowledge Guides", "Workflows", "#14b8a6"),
|
||||
"hooks": ("Hooks", "Hooks", "#f97316"),
|
||||
"skills": ("Agent Skills", "Skills", "#eab308"),
|
||||
"styles": ("Output Styles", "Styles", "#06b6d4"),
|
||||
"statusline": ("Status Lines", "Status", "#84cc16"),
|
||||
"docs": ("Official Documentation", "Docs", "#6366f1"),
|
||||
"clients": ("Alternative Clients", "Clients", "#f43f5e"),
|
||||
}
|
||||
|
||||
# Sort type definitions: slug -> (display_name, badge_color, description)
|
||||
FLAT_SORT_TYPES = {
|
||||
"az": ("A - Z", "#6366f1", "alphabetically by name"),
|
||||
"updated": ("UPDATED", "#f472b6", "by last updated date"),
|
||||
"created": ("CREATED", "#34d399", "by date created"),
|
||||
"releases": ("RELEASES", "#f59e0b", "by latest release (30 days)"),
|
||||
}
|
||||
|
||||
|
||||
class ParameterizedFlatListGenerator(ReadmeGenerator):
|
||||
"""Unified generator for flat list READMEs with category filtering and sort options."""
|
||||
|
||||
DAYS_THRESHOLD = 30 # For releases filter
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
csv_path: str,
|
||||
template_dir: str,
|
||||
assets_dir: str,
|
||||
repo_root: str,
|
||||
category_slug: str = "all",
|
||||
sort_type: str = "az",
|
||||
) -> None:
|
||||
super().__init__(csv_path, template_dir, assets_dir, repo_root)
|
||||
self.category_slug = category_slug
|
||||
self.sort_type = sort_type
|
||||
self._category_info = FLAT_CATEGORIES.get(category_slug, FLAT_CATEGORIES["all"])
|
||||
self._sort_info = FLAT_SORT_TYPES.get(sort_type, FLAT_SORT_TYPES["az"])
|
||||
|
||||
@property
|
||||
def template_filename(self) -> str:
|
||||
return "README_FLAT.template.md"
|
||||
|
||||
@property
|
||||
def output_filename(self) -> str:
|
||||
return (
|
||||
f"README_ALTERNATIVES/README_FLAT_{self.category_slug.upper()}"
|
||||
f"_{self.sort_type.upper()}.md"
|
||||
)
|
||||
|
||||
@property
|
||||
def style_id(self) -> str:
|
||||
return "flat"
|
||||
|
||||
def format_resource_entry(self, row: dict, include_separator: bool = True) -> str:
|
||||
"""Not used for flat list."""
|
||||
_ = include_separator
|
||||
return ""
|
||||
|
||||
def generate_toc(self) -> str:
|
||||
"""Not used for flat list."""
|
||||
return ""
|
||||
|
||||
def generate_weekly_section(self) -> str:
|
||||
"""Not used for flat list."""
|
||||
return ""
|
||||
|
||||
def generate_section_content(self, category: dict, section_index: int) -> str:
|
||||
"""Not used for flat list."""
|
||||
_ = category, section_index
|
||||
return ""
|
||||
|
||||
def get_filtered_resources(self) -> list[dict]:
|
||||
"""Get resources filtered by category."""
|
||||
csv_category_value = self._category_info[0]
|
||||
if csv_category_value is None:
|
||||
return list(self.csv_data)
|
||||
return [r for r in self.csv_data if r.get("Category", "").strip() == csv_category_value]
|
||||
|
||||
def sort_resources(self, resources: list[dict]) -> list[dict]:
|
||||
"""Sort resources according to sort_type."""
|
||||
if self.sort_type == "az":
|
||||
return sorted(resources, key=lambda x: (x.get("Display Name", "") or "").lower())
|
||||
if self.sort_type == "updated":
|
||||
with_dates = []
|
||||
for row in resources:
|
||||
last_modified = row.get("Last Modified", "").strip()
|
||||
parsed = parse_resource_date(last_modified) if last_modified else None
|
||||
with_dates.append((parsed, row))
|
||||
with_dates.sort(
|
||||
key=lambda x: (x[0] is None, x[0] if x[0] else datetime.min),
|
||||
reverse=True,
|
||||
)
|
||||
return [r for _, r in with_dates]
|
||||
if self.sort_type == "created":
|
||||
with_dates = []
|
||||
for row in resources:
|
||||
repo_created = row.get("Repo Created", "").strip()
|
||||
parsed = parse_resource_date(repo_created) if repo_created else None
|
||||
with_dates.append((parsed, row))
|
||||
with_dates.sort(
|
||||
key=lambda x: (x[0] is None, x[0] if x[0] else datetime.min),
|
||||
reverse=True,
|
||||
)
|
||||
return [r for _, r in with_dates]
|
||||
if self.sort_type == "releases":
|
||||
cutoff = datetime.now() - timedelta(days=self.DAYS_THRESHOLD)
|
||||
recent = []
|
||||
for row in resources:
|
||||
release_date_str = row.get("Latest Release", "")
|
||||
if not release_date_str:
|
||||
continue
|
||||
try:
|
||||
release_date = datetime.strptime(release_date_str, "%Y-%m-%d:%H-%M-%S")
|
||||
except ValueError:
|
||||
continue
|
||||
if release_date >= cutoff:
|
||||
row["_parsed_release_date"] = release_date
|
||||
recent.append(row)
|
||||
recent.sort(key=lambda x: x.get("_parsed_release_date", datetime.min), reverse=True)
|
||||
return recent
|
||||
return resources
|
||||
|
||||
def generate_sort_navigation(self) -> str:
|
||||
"""Generate sort option badges."""
|
||||
return generate_flat_sort_navigation(
|
||||
self.category_slug,
|
||||
self.sort_type,
|
||||
FLAT_SORT_TYPES,
|
||||
)
|
||||
|
||||
def generate_category_navigation(self) -> str:
|
||||
"""Generate category filter badges."""
|
||||
return generate_flat_category_navigation(
|
||||
self.category_slug,
|
||||
self.sort_type,
|
||||
FLAT_CATEGORIES,
|
||||
)
|
||||
|
||||
def generate_navigation(self) -> str:
|
||||
"""Generate combined navigation (sort + category)."""
|
||||
return generate_flat_navigation(
|
||||
self.category_slug,
|
||||
self.sort_type,
|
||||
FLAT_CATEGORIES,
|
||||
FLAT_SORT_TYPES,
|
||||
)
|
||||
|
||||
def generate_resources_table(self) -> str:
|
||||
"""Generate the resources table as HTML with shields.io badges for GitHub resources."""
|
||||
resources = self.get_filtered_resources()
|
||||
sorted_resources = self.sort_resources(resources)
|
||||
return generate_flat_resources_table(sorted_resources, self.sort_type)
|
||||
|
||||
def _get_default_template(self) -> str:
|
||||
"""Return default template content."""
|
||||
return get_flat_default_template()
|
||||
|
||||
def generate(self, output_path: str | None = None) -> tuple[int, str | None]:
|
||||
"""Generate the flat list README for a category/sort pair."""
|
||||
resolved_path = output_path or self.resolved_output_path
|
||||
self.overrides = self.load_overrides()
|
||||
self.csv_data = self.load_csv_data()
|
||||
|
||||
template_path = os.path.join(self.template_dir, self.template_filename)
|
||||
if not os.path.exists(template_path):
|
||||
template = self._get_default_template()
|
||||
else:
|
||||
template = load_template(template_path)
|
||||
|
||||
resources = self.get_filtered_resources()
|
||||
sorted_resources = self.sort_resources(resources)
|
||||
navigation = generate_flat_navigation(
|
||||
self.category_slug,
|
||||
self.sort_type,
|
||||
FLAT_CATEGORIES,
|
||||
FLAT_SORT_TYPES,
|
||||
)
|
||||
resources_table = generate_flat_resources_table(sorted_resources, self.sort_type)
|
||||
|
||||
generated_date = datetime.now().strftime("%Y-%m-%d")
|
||||
_, cat_display, _ = self._category_info
|
||||
_, _, sort_desc = self._sort_info
|
||||
|
||||
releases_disclaimer = ""
|
||||
if self.sort_type == "releases":
|
||||
releases_disclaimer = (
|
||||
"\n> **Note:** Latest release data is pulled from GitHub Releases only. "
|
||||
"Projects without GitHub Releases will not show release info here. "
|
||||
"Please verify with the project directly.\n"
|
||||
)
|
||||
|
||||
output_path = os.path.join(self.repo_root, resolved_path)
|
||||
|
||||
readme_content = template
|
||||
readme_content = readme_content.replace(
|
||||
"{{STYLE_SELECTOR}}", self.get_style_selector(Path(output_path))
|
||||
)
|
||||
readme_content = readme_content.replace("{{NAVIGATION}}", navigation)
|
||||
readme_content = readme_content.replace("{{RELEASES_DISCLAIMER}}", releases_disclaimer)
|
||||
readme_content = readme_content.replace("{{RESOURCES_TABLE}}", resources_table)
|
||||
readme_content = readme_content.replace("{{RESOURCE_COUNT}}", str(len(sorted_resources)))
|
||||
readme_content = readme_content.replace("{{CATEGORY_NAME}}", cat_display)
|
||||
readme_content = readme_content.replace("{{SORT_DESC}}", sort_desc)
|
||||
readme_content = readme_content.replace("{{GENERATED_DATE}}", generated_date)
|
||||
|
||||
readme_content = ensure_generated_header(readme_content)
|
||||
readme_content = resolve_asset_tokens(
|
||||
readme_content, Path(output_path), Path(self.repo_root)
|
||||
)
|
||||
output_dir = os.path.dirname(output_path)
|
||||
if output_dir and not os.path.exists(output_dir):
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
backup_path = self.create_backup(output_path)
|
||||
|
||||
try:
|
||||
with open(output_path, "w", encoding="utf-8") as f:
|
||||
f.write(readme_content)
|
||||
except Exception as e:
|
||||
if backup_path:
|
||||
print(f"Error writing {resolved_path}: {e}")
|
||||
print(f" Backup preserved at: {backup_path}")
|
||||
raise
|
||||
|
||||
return len(sorted_resources), backup_path
|
||||
@@ -0,0 +1,48 @@
|
||||
"""Minimal README generator implementation."""
|
||||
|
||||
from scripts.readme.generators.base import ReadmeGenerator
|
||||
from scripts.readme.markup.minimal import (
|
||||
format_resource_entry as format_minimal_resource_entry,
|
||||
)
|
||||
from scripts.readme.markup.minimal import (
|
||||
generate_section_content as generate_minimal_section_content,
|
||||
)
|
||||
from scripts.readme.markup.minimal import (
|
||||
generate_toc as generate_minimal_toc,
|
||||
)
|
||||
from scripts.readme.markup.minimal import (
|
||||
generate_weekly_section as generate_minimal_weekly_section,
|
||||
)
|
||||
|
||||
|
||||
class MinimalReadmeGenerator(ReadmeGenerator):
|
||||
"""Generator for plain markdown README classic variant."""
|
||||
|
||||
@property
|
||||
def template_filename(self) -> str:
|
||||
return "README_CLASSIC.template.md"
|
||||
|
||||
@property
|
||||
def output_filename(self) -> str:
|
||||
return "README_ALTERNATIVES/README_CLASSIC.md"
|
||||
|
||||
@property
|
||||
def style_id(self) -> str:
|
||||
return "classic"
|
||||
|
||||
def format_resource_entry(self, row: dict, include_separator: bool = True) -> str:
|
||||
"""Format resource as plain markdown with collapsible GitHub stats."""
|
||||
return format_minimal_resource_entry(row, include_separator=include_separator)
|
||||
|
||||
def generate_toc(self) -> str:
|
||||
"""Generate plain markdown nested details TOC."""
|
||||
return generate_minimal_toc(self.categories, self.csv_data)
|
||||
|
||||
def generate_weekly_section(self) -> str:
|
||||
"""Generate weekly section with plain markdown."""
|
||||
return generate_minimal_weekly_section(self.csv_data)
|
||||
|
||||
def generate_section_content(self, category: dict, section_index: int) -> str:
|
||||
"""Generate section with plain markdown headers."""
|
||||
_ = section_index
|
||||
return generate_minimal_section_content(category, self.csv_data)
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Visual README generator implementation."""
|
||||
|
||||
from scripts.readme.generators.base import ReadmeGenerator
|
||||
from scripts.readme.markup.visual import (
|
||||
format_resource_entry as format_visual_resource_entry,
|
||||
)
|
||||
from scripts.readme.markup.visual import (
|
||||
generate_repo_ticker as generate_visual_repo_ticker,
|
||||
)
|
||||
from scripts.readme.markup.visual import (
|
||||
generate_section_content as generate_visual_section_content,
|
||||
)
|
||||
from scripts.readme.markup.visual import (
|
||||
generate_toc_from_categories as generate_visual_toc,
|
||||
)
|
||||
from scripts.readme.markup.visual import (
|
||||
generate_weekly_section as generate_visual_weekly_section,
|
||||
)
|
||||
|
||||
|
||||
class VisualReadmeGenerator(ReadmeGenerator):
|
||||
"""Generator for visual/themed README variant with SVG assets."""
|
||||
|
||||
@property
|
||||
def template_filename(self) -> str:
|
||||
return "README_EXTRA.template.md"
|
||||
|
||||
@property
|
||||
def output_filename(self) -> str:
|
||||
return "README_ALTERNATIVES/README_EXTRA.md"
|
||||
|
||||
@property
|
||||
def style_id(self) -> str:
|
||||
return "extra"
|
||||
|
||||
def format_resource_entry(self, row: dict, include_separator: bool = True) -> str:
|
||||
"""Format resource with SVG badges and visible GitHub stats."""
|
||||
return format_visual_resource_entry(
|
||||
row,
|
||||
assets_dir=self.assets_dir,
|
||||
include_separator=include_separator,
|
||||
)
|
||||
|
||||
def generate_toc(self) -> str:
|
||||
"""Generate terminal-style SVG TOC."""
|
||||
return generate_visual_toc(
|
||||
self.categories,
|
||||
self.csv_data,
|
||||
self.general_anchor_map,
|
||||
)
|
||||
|
||||
def generate_weekly_section(self) -> str:
|
||||
"""Generate latest additions section with header SVG."""
|
||||
return generate_visual_weekly_section(self.csv_data, assets_dir=self.assets_dir)
|
||||
|
||||
def generate_section_content(self, category: dict, section_index: int) -> str:
|
||||
"""Generate section with SVG headers and desc boxes."""
|
||||
return generate_visual_section_content(
|
||||
category,
|
||||
self.csv_data,
|
||||
self.general_anchor_map,
|
||||
assets_dir=self.assets_dir,
|
||||
section_index=section_index,
|
||||
)
|
||||
|
||||
def generate_repo_ticker(self) -> str:
|
||||
"""Generate the animated SVG repo ticker for visual theme."""
|
||||
return generate_visual_repo_ticker()
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Regenerate subcategory TOC SVGs from categories.yaml.
|
||||
|
||||
Run after adding or modifying subcategories in templates/categories.yaml
|
||||
to create/update the corresponding TOC row SVG assets used by the
|
||||
Visual (Extra) README style.
|
||||
|
||||
Usage:
|
||||
python -m scripts.readme.generate_toc_assets
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.categories.category_utils import category_manager
|
||||
from scripts.readme.helpers.readme_assets import regenerate_sub_toc_svgs
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
|
||||
def main() -> None:
|
||||
repo_root = find_repo_root(Path(__file__))
|
||||
assets_dir = str(repo_root / "assets")
|
||||
categories = category_manager.get_categories_for_readme()
|
||||
regenerate_sub_toc_svgs(categories, assets_dir)
|
||||
print("✅ Subcategory TOC SVGs regenerated")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,345 @@
|
||||
"""SVG asset generation and file helpers for README generation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import glob
|
||||
import os
|
||||
import re
|
||||
|
||||
from scripts.readme.helpers.readme_utils import format_category_dir_name
|
||||
from scripts.readme.svg_templates.badges import (
|
||||
generate_resource_badge_svg,
|
||||
render_flat_category_badge_svg,
|
||||
render_flat_sort_badge_svg,
|
||||
)
|
||||
from scripts.readme.svg_templates.dividers import (
|
||||
generate_desc_box_light_svg,
|
||||
generate_section_divider_light_svg,
|
||||
)
|
||||
from scripts.readme.svg_templates.dividers import (
|
||||
generate_entry_separator_svg as _generate_entry_separator_svg,
|
||||
)
|
||||
from scripts.readme.svg_templates.headers import (
|
||||
generate_category_header_light_svg,
|
||||
render_h2_svg,
|
||||
render_h3_svg,
|
||||
)
|
||||
from scripts.readme.svg_templates.toc import (
|
||||
_normalize_svg_root,
|
||||
generate_toc_header_light_svg,
|
||||
generate_toc_row_light_svg,
|
||||
generate_toc_row_svg,
|
||||
generate_toc_sub_light_svg,
|
||||
generate_toc_sub_svg,
|
||||
)
|
||||
|
||||
|
||||
def create_h2_svg_file(text: str, filename: str, assets_dir: str, icon: str = "") -> str:
|
||||
"""Create an animated hero-centered H2 header SVG file."""
|
||||
svg_content = render_h2_svg(text, icon=icon)
|
||||
|
||||
filepath = os.path.join(assets_dir, filename)
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
|
||||
return filename
|
||||
|
||||
|
||||
def create_h3_svg_file(text: str, filename: str, assets_dir: str) -> str:
|
||||
"""Create an animated minimal-inline H3 header SVG file."""
|
||||
svg_content = render_h3_svg(text)
|
||||
|
||||
filepath = os.path.join(assets_dir, filename)
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
if not svg_content.endswith("\n"):
|
||||
f.write("\n")
|
||||
|
||||
return filename
|
||||
|
||||
|
||||
def ensure_category_header_exists(
|
||||
category_id: str,
|
||||
title: str,
|
||||
section_number: str,
|
||||
assets_dir: str,
|
||||
icon: str = "",
|
||||
always_regenerate: bool = True,
|
||||
) -> tuple[str, str]:
|
||||
"""Ensure category header SVGs exist, generating them if needed."""
|
||||
safe_name = category_id.replace("-", "_")
|
||||
dark_filename = f"header_{safe_name}.svg"
|
||||
light_filename = f"header_{safe_name}-light-v3.svg"
|
||||
|
||||
dark_path = os.path.join(assets_dir, dark_filename)
|
||||
if always_regenerate or not os.path.exists(dark_path):
|
||||
create_h2_svg_file(title, dark_filename, assets_dir, icon=icon)
|
||||
|
||||
light_path = os.path.join(assets_dir, light_filename)
|
||||
if always_regenerate or not os.path.exists(light_path):
|
||||
svg_content = generate_category_header_light_svg(title, section_number)
|
||||
with open(light_path, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
|
||||
return (dark_filename, light_filename)
|
||||
|
||||
|
||||
def ensure_section_divider_exists(variant: int, assets_dir: str) -> tuple[str, str]:
|
||||
"""Ensure section divider SVG exists, generating if needed."""
|
||||
dark_filename = "section-divider-alt2.svg"
|
||||
light_filename = f"section-divider-light-manual-v{variant}.svg"
|
||||
|
||||
light_path = os.path.join(assets_dir, light_filename)
|
||||
if not os.path.exists(light_path):
|
||||
svg_content = generate_section_divider_light_svg(variant)
|
||||
with open(light_path, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
|
||||
return (dark_filename, light_filename)
|
||||
|
||||
|
||||
def ensure_desc_box_exists(position: str, assets_dir: str) -> str:
|
||||
"""Ensure desc box SVG exists, generating if needed."""
|
||||
filename = f"desc-box-{position}-light.svg"
|
||||
filepath = os.path.join(assets_dir, filename)
|
||||
|
||||
if not os.path.exists(filepath):
|
||||
svg_content = generate_desc_box_light_svg(position)
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
|
||||
return filename
|
||||
|
||||
|
||||
def ensure_toc_row_exists(
|
||||
category_id: str,
|
||||
directory_name: str,
|
||||
description: str,
|
||||
assets_dir: str,
|
||||
always_regenerate: bool = True,
|
||||
) -> str:
|
||||
"""Ensure TOC row SVG exists, generating if needed."""
|
||||
filename = f"toc-row-{category_id}.svg"
|
||||
filepath = os.path.join(assets_dir, filename)
|
||||
|
||||
if always_regenerate or not os.path.exists(filepath):
|
||||
svg_content = generate_toc_row_svg(directory_name, description)
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
|
||||
return filename
|
||||
|
||||
|
||||
def ensure_toc_sub_exists(
|
||||
subcat_id: str,
|
||||
directory_name: str,
|
||||
description: str,
|
||||
assets_dir: str,
|
||||
always_regenerate: bool = True,
|
||||
) -> str:
|
||||
"""Ensure TOC subcategory SVG exists, generating if needed."""
|
||||
filename = f"toc-sub-{subcat_id}.svg"
|
||||
filepath = os.path.join(assets_dir, filename)
|
||||
|
||||
if always_regenerate or not os.path.exists(filepath):
|
||||
svg_content = generate_toc_sub_svg(directory_name, description)
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
|
||||
return filename
|
||||
|
||||
|
||||
def get_category_svg_filename(category_id: str) -> str:
|
||||
"""Map category ID to SVG filename."""
|
||||
svg_map = {
|
||||
"skills": "toc-row-skills.svg",
|
||||
"workflows": "toc-row-workflows.svg",
|
||||
"tooling": "toc-row-tooling.svg",
|
||||
"statusline": "toc-row-statusline.svg",
|
||||
"hooks": "toc-row-custom.svg",
|
||||
"slash-commands": "toc-row-commands.svg",
|
||||
"claude-md-files": "toc-row-config.svg",
|
||||
"alternative-clients": "toc-row-clients.svg",
|
||||
"official-documentation": "toc-row-docs.svg",
|
||||
}
|
||||
return svg_map.get(category_id, f"toc-row-{category_id}.svg")
|
||||
|
||||
|
||||
def get_subcategory_svg_filename(subcat_id: str) -> str:
|
||||
"""Map subcategory ID to SVG filename."""
|
||||
svg_map = {
|
||||
"general": "toc-sub-general.svg",
|
||||
"ide-integrations": "toc-sub-ide.svg",
|
||||
"usage-monitors": "toc-sub-monitors.svg",
|
||||
"orchestrators": "toc-sub-orchestrators.svg",
|
||||
"config-managers": "toc-sub-config-managers.svg",
|
||||
"version-control-git": "toc-sub-git.svg",
|
||||
"code-analysis-testing": "toc-sub-code-analysis.svg",
|
||||
"context-loading-priming": "toc-sub-context.svg",
|
||||
"documentation-changelogs": "toc-sub-documentation.svg",
|
||||
"ci-deployment": "toc-sub-ci.svg",
|
||||
"project-task-management": "toc-sub-project-mgmt.svg",
|
||||
"miscellaneous": "toc-sub-misc.svg",
|
||||
"language-specific": "toc-sub-language.svg",
|
||||
"domain-specific": "toc-sub-domain.svg",
|
||||
"project-scaffolding-mcp": "toc-sub-scaffolding.svg",
|
||||
"ralph-wiggum": "toc-sub-ralph-wiggum.svg",
|
||||
}
|
||||
return svg_map.get(subcat_id, f"toc-sub-{subcat_id}.svg")
|
||||
|
||||
|
||||
def get_category_header_svg(category_id: str) -> tuple[str, str]:
|
||||
"""Map category ID to pre-made header SVG filenames (dark and light variants)."""
|
||||
header_map = {
|
||||
"skills": ("header_agent_skills.svg", "header_agent_skills-light-v3.svg"),
|
||||
"workflows": (
|
||||
"header_workflows_knowledge_guides.svg",
|
||||
"header_workflows_knowledge_guides-light-v3.svg",
|
||||
),
|
||||
"tooling": ("header_tooling.svg", "header_tooling-light-v3.svg"),
|
||||
"statusline": ("header_status_lines.svg", "header_status_lines-light-v3.svg"),
|
||||
"hooks": ("header_hooks.svg", "header_hooks-light-v3.svg"),
|
||||
"slash-commands": (
|
||||
"header_slash_commands.svg",
|
||||
"header_slash_commands-light-v3.svg",
|
||||
),
|
||||
"claude-md-files": (
|
||||
"header_claudemd_files.svg",
|
||||
"header_claudemd_files-light-v3.svg",
|
||||
),
|
||||
"alternative-clients": (
|
||||
"header_alternative_clients.svg",
|
||||
"header_alternative_clients-light-v3.svg",
|
||||
),
|
||||
"official-documentation": (
|
||||
"header_official_documentation.svg",
|
||||
"header_official_documentation-light-v3.svg",
|
||||
),
|
||||
}
|
||||
return header_map.get(
|
||||
category_id, (f"header_{category_id}.svg", f"header_{category_id}-light-v3.svg")
|
||||
)
|
||||
|
||||
|
||||
_section_divider_counter = 0
|
||||
|
||||
|
||||
def get_section_divider_svg() -> tuple[str, str]:
|
||||
"""Get the next section divider SVG filenames."""
|
||||
global _section_divider_counter
|
||||
variant = (_section_divider_counter % 3) + 1
|
||||
_section_divider_counter += 1
|
||||
return ("section-divider-alt2.svg", f"section-divider-light-manual-v{variant}.svg")
|
||||
|
||||
|
||||
def normalize_toc_svgs(assets_dir: str) -> None:
|
||||
"""Normalize TOC row/sub SVGs to enforce consistent display height/anchoring."""
|
||||
patterns = ["toc-row-*.svg", "toc-sub-*.svg", "toc-header*.svg"]
|
||||
for pattern in patterns:
|
||||
for path in glob.glob(os.path.join(assets_dir, pattern)):
|
||||
with open(path, encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
|
||||
match = re.search(r"<svg[^>]*>", content)
|
||||
if not match:
|
||||
continue
|
||||
|
||||
root_tag = match.group(0)
|
||||
is_header = "toc-header" in os.path.basename(path)
|
||||
target_width = 400
|
||||
target_height = 48 if is_header else 40
|
||||
|
||||
normalized_tag = _normalize_svg_root(root_tag, target_width, target_height)
|
||||
if normalized_tag != root_tag:
|
||||
content = content.replace(root_tag, normalized_tag, 1)
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
f.write(content)
|
||||
|
||||
|
||||
def regenerate_main_toc_svgs(categories: list[dict], assets_dir: str) -> None:
|
||||
"""Regenerate main category TOC row SVGs with standardized styling."""
|
||||
for category in categories:
|
||||
display_dir = format_category_dir_name(category.get("name", ""), category.get("id", ""))
|
||||
description = category.get("description", "")
|
||||
|
||||
dark_filename = get_category_svg_filename(category.get("id", ""))
|
||||
dark_path = os.path.join(assets_dir, dark_filename)
|
||||
svg_content = generate_toc_row_svg(display_dir, description)
|
||||
with open(dark_path, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
|
||||
light_path = dark_path.replace(".svg", "-light-anim-scanline.svg")
|
||||
light_svg = generate_toc_row_light_svg(display_dir, description)
|
||||
with open(light_path, "w", encoding="utf-8") as f:
|
||||
f.write(light_svg)
|
||||
|
||||
|
||||
def regenerate_sub_toc_svgs(categories: list[dict], assets_dir: str) -> None:
|
||||
"""Regenerate subcategory TOC SVGs to keep sizing consistent."""
|
||||
for category in categories:
|
||||
subcats = category.get("subcategories", [])
|
||||
for subcat in subcats:
|
||||
display_dir = subcat.get("name", "")
|
||||
description = subcat.get("description", "")
|
||||
dark_filename = get_subcategory_svg_filename(subcat.get("id", ""))
|
||||
dark_path = os.path.join(assets_dir, dark_filename)
|
||||
svg_content = generate_toc_sub_svg(display_dir, description)
|
||||
with open(dark_path, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
|
||||
light_path = dark_path.replace(".svg", "-light-anim-scanline.svg")
|
||||
light_svg = generate_toc_sub_light_svg(display_dir, description)
|
||||
with open(light_path, "w", encoding="utf-8") as f:
|
||||
f.write(light_svg)
|
||||
|
||||
|
||||
def regenerate_toc_header(assets_dir: str) -> None:
|
||||
"""Regenerate the light-mode TOC header for consistent sizing."""
|
||||
light_header_path = os.path.join(assets_dir, "toc-header-light-anim-scanline.svg")
|
||||
light_header_svg = generate_toc_header_light_svg()
|
||||
with open(light_header_path, "w", encoding="utf-8") as f:
|
||||
f.write(light_header_svg)
|
||||
|
||||
|
||||
def save_resource_badge_svg(display_name: str, author_name: str, assets_dir: str) -> str:
|
||||
"""Save a resource name SVG badge to the assets directory and return the filename."""
|
||||
safe_name = re.sub(r"[^a-zA-Z0-9]", "-", display_name.lower())
|
||||
safe_name = re.sub(r"-+", "-", safe_name).strip("-")
|
||||
filename = f"badge-{safe_name}.svg"
|
||||
|
||||
svg_content = generate_resource_badge_svg(display_name, author_name)
|
||||
|
||||
filepath = os.path.join(assets_dir, filename)
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(svg_content)
|
||||
if not svg_content.endswith("\n"):
|
||||
f.write("\n")
|
||||
|
||||
return filename
|
||||
|
||||
|
||||
def generate_entry_separator_svg() -> str:
|
||||
"""Generate a small separator SVG between entries in vintage manual style."""
|
||||
return _generate_entry_separator_svg()
|
||||
|
||||
|
||||
def ensure_separator_svg_exists(assets_dir: str) -> str:
|
||||
"""Return the animated entry separator SVG filename."""
|
||||
_ = assets_dir
|
||||
return "entry-separator-light-animated.svg"
|
||||
|
||||
|
||||
def generate_flat_badges(assets_dir: str, sort_types: dict, categories: dict) -> None:
|
||||
"""Generate all sort and category badge SVGs."""
|
||||
for slug, (display, color, _) in sort_types.items():
|
||||
svg = render_flat_sort_badge_svg(display, color)
|
||||
filepath = os.path.join(assets_dir, f"badge-sort-{slug}.svg")
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(svg)
|
||||
|
||||
for slug, (_, display, color) in categories.items():
|
||||
width = max(70, len(display) * 10 + 30)
|
||||
svg = render_flat_category_badge_svg(display, color, width)
|
||||
filepath = os.path.join(assets_dir, f"badge-cat-{slug}.svg")
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(svg)
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Configuration loader for README generation."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import yaml # type: ignore[import-untyped]
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
|
||||
def load_config() -> dict:
|
||||
"""Load configuration from acc-config.yaml."""
|
||||
config_path = REPO_ROOT / "acc-config.yaml"
|
||||
try:
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
return yaml.safe_load(f)
|
||||
except FileNotFoundError:
|
||||
print(f"Warning: acc-config.yaml not found at {config_path}, using defaults")
|
||||
return {
|
||||
"readme": {"root_style": "extra"},
|
||||
"styles": {
|
||||
"extra": {
|
||||
"name": "Extra",
|
||||
"badge": "badge-style-extra.svg",
|
||||
"highlight_color": "#6a6a8a",
|
||||
"filename": "README_EXTRA.md",
|
||||
},
|
||||
"classic": {
|
||||
"name": "Classic",
|
||||
"badge": "badge-style-classic.svg",
|
||||
"highlight_color": "#c9a227",
|
||||
"filename": "README_CLASSIC.md",
|
||||
},
|
||||
"awesome": {
|
||||
"name": "Awesome",
|
||||
"badge": "badge-style-awesome.svg",
|
||||
"highlight_color": "#cc3366",
|
||||
"filename": "README_AWESOME.md",
|
||||
},
|
||||
"flat": {
|
||||
"name": "Flat",
|
||||
"badge": "badge-style-flat.svg",
|
||||
"highlight_color": "#71717a",
|
||||
"filename": "README_FLAT_ALL_AZ.md",
|
||||
},
|
||||
},
|
||||
"style_order": ["extra", "classic", "flat", "awesome"],
|
||||
}
|
||||
|
||||
|
||||
# Global config instance
|
||||
CONFIG = load_config()
|
||||
|
||||
|
||||
def get_root_style() -> str:
|
||||
"""Get the root README style from config."""
|
||||
readme_config = CONFIG.get("readme", {})
|
||||
return readme_config.get("root_style") or readme_config.get("default_style", "extra")
|
||||
|
||||
|
||||
def get_style_selector_target(style_id: str) -> str:
|
||||
"""Get the selector link target for a style, accounting for root style config."""
|
||||
root_style = get_root_style()
|
||||
styles = CONFIG.get("styles", {})
|
||||
style_config = styles.get(style_id, {})
|
||||
filename = style_config.get("filename")
|
||||
if not filename:
|
||||
if style_id == "flat":
|
||||
filename = "README_FLAT_ALL_AZ.md"
|
||||
else:
|
||||
filename = f"README_{style_id.upper()}.md"
|
||||
filename = os.path.basename(filename)
|
||||
|
||||
if style_id == root_style:
|
||||
return "README.md"
|
||||
return f"README_ALTERNATIVES/{filename}"
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Path resolution helpers for README generation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
GENERATED_HEADER = "<!-- GENERATED FILE: do not edit directly -->"
|
||||
|
||||
ASSET_PATH_PATTERN = re.compile(
|
||||
r"\{\{ASSET_PATH\(\s*(?P<quote>['\"])(?P<path>[^'\"]+)(?P=quote)\s*\)\}\}"
|
||||
)
|
||||
ASSET_URL_PATTERN = re.compile(r"asset:([A-Za-z0-9_.\-/]+)")
|
||||
|
||||
|
||||
def asset_path_token(filename: str) -> str:
|
||||
"""Return a tokenized asset reference for templates/markup."""
|
||||
filename = filename.lstrip("/")
|
||||
return f"{{{{ASSET_PATH('{filename}')}}}}"
|
||||
|
||||
|
||||
def ensure_generated_header(content: str) -> str:
|
||||
"""Prepend the generated-file header if missing."""
|
||||
if content.startswith(GENERATED_HEADER):
|
||||
return content
|
||||
return f"{GENERATED_HEADER}\n{content.lstrip(chr(10))}"
|
||||
|
||||
|
||||
def resolve_asset_tokens(content: str, output_path: Path, repo_root: Path | None = None) -> str:
|
||||
"""Resolve asset tokens into relative paths for the output location."""
|
||||
repo_root = repo_root or find_repo_root(output_path)
|
||||
base_dir = output_path.parent
|
||||
assets_dir = repo_root / "assets"
|
||||
|
||||
rel_assets = Path(os.path.relpath(assets_dir, start=base_dir)).as_posix()
|
||||
if rel_assets == ".":
|
||||
rel_assets = "assets"
|
||||
rel_assets = rel_assets.rstrip("/")
|
||||
|
||||
def join_asset(path: str) -> str:
|
||||
path = path.lstrip("/")
|
||||
if not rel_assets:
|
||||
return path
|
||||
return f"{rel_assets}/{path}"
|
||||
|
||||
content = content.replace("{{ASSET_PREFIX}}", f"{rel_assets}/")
|
||||
|
||||
content = ASSET_PATH_PATTERN.sub(lambda match: join_asset(match.group("path")), content)
|
||||
content = ASSET_URL_PATTERN.sub(lambda match: join_asset(match.group(1)), content)
|
||||
|
||||
return content
|
||||
|
||||
|
||||
def resolve_relative_link(from_path: Path, to_path: Path, repo_root: Path | None = None) -> str:
|
||||
"""Return a relative link between two files, normalized for README links."""
|
||||
repo_root = repo_root or find_repo_root(from_path)
|
||||
from_path = from_path.resolve()
|
||||
to_path = (repo_root / to_path).resolve() if not to_path.is_absolute() else to_path.resolve()
|
||||
|
||||
rel_path = Path(os.path.relpath(to_path, start=from_path.parent)).as_posix()
|
||||
|
||||
if to_path == repo_root / "README.md":
|
||||
if rel_path in (".", "README.md"):
|
||||
return "./"
|
||||
if rel_path.endswith("/README.md"):
|
||||
return rel_path[: -len("README.md")]
|
||||
|
||||
return rel_path
|
||||
@@ -0,0 +1,191 @@
|
||||
"""Shared utility helpers for README generation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
def extract_github_owner_repo(url: str) -> tuple[str, str] | None:
|
||||
"""Extract owner and repo from any GitHub URL."""
|
||||
patterns = [
|
||||
r"github\.com/([^/]+)/([^/]+?)(?:\.git)?/?$", # repo root
|
||||
r"github\.com/([^/]+)/([^/]+)/(?:blob|tree|issues|pull|releases)", # with path
|
||||
r"github\.com/([^/]+)/([^/]+)/?", # general fallback
|
||||
]
|
||||
for pattern in patterns:
|
||||
match = re.search(pattern, url)
|
||||
if match:
|
||||
owner, repo = match.groups()[:2]
|
||||
repo = repo.split("/")[0].split("?")[0].split("#")[0]
|
||||
if owner and repo:
|
||||
return (owner, repo)
|
||||
return None
|
||||
|
||||
|
||||
def format_stars(num: int) -> str:
|
||||
"""Format star count with K/M suffix."""
|
||||
if num >= 1_000_000:
|
||||
return f"{num / 1_000_000:.1f}M"
|
||||
if num >= 1000:
|
||||
return f"{num / 1000:.1f}K"
|
||||
return str(num)
|
||||
|
||||
|
||||
def format_delta(delta: int) -> str:
|
||||
"""Format delta with +/- prefix."""
|
||||
if delta > 0:
|
||||
return f"+{delta}"
|
||||
if delta < 0:
|
||||
return str(delta)
|
||||
return ""
|
||||
|
||||
|
||||
def get_anchor_suffix_for_icon(icon: str | None) -> str:
|
||||
"""Generate the anchor suffix for a section with a trailing emoji icon.
|
||||
|
||||
GitHub strips simple emoji codepoints and turns them into a dash. If the emoji
|
||||
includes a variation selector (U+FE00 to U+FE0F), the variation selector is
|
||||
URL-encoded and appended after the dash.
|
||||
"""
|
||||
if not icon:
|
||||
return ""
|
||||
|
||||
vs_char = next((char for char in icon if 0xFE00 <= ord(char) <= 0xFE0F), None)
|
||||
if vs_char:
|
||||
vs_bytes = vs_char.encode("utf-8")
|
||||
url_encoded = "".join(f"%{byte:02X}" for byte in vs_bytes)
|
||||
return f"-{url_encoded}"
|
||||
|
||||
return "-"
|
||||
|
||||
|
||||
def generate_toc_anchor(
|
||||
title: str,
|
||||
icon: str | None = None,
|
||||
has_back_to_top_in_heading: bool = False,
|
||||
) -> str:
|
||||
"""Generate a TOC anchor for a heading.
|
||||
|
||||
Centralizes anchor generation logic across all README styles.
|
||||
|
||||
Args:
|
||||
title: The heading text (e.g., "Agent Skills")
|
||||
icon: Optional trailing emoji icon (e.g., "🤖"). Each emoji adds a dash.
|
||||
has_back_to_top_in_heading: True if heading contains 🔝 back-to-top link,
|
||||
which adds an additional trailing dash to the anchor.
|
||||
|
||||
Returns:
|
||||
The anchor string without the leading '#' (e.g., "agent-skills-")
|
||||
"""
|
||||
base = title.lower().replace(" ", "-").replace("&", "").replace("/", "").replace(".", "")
|
||||
suffix = get_anchor_suffix_for_icon(icon)
|
||||
back_to_top_suffix = "-" if has_back_to_top_in_heading else ""
|
||||
return f"{base}{suffix}{back_to_top_suffix}"
|
||||
|
||||
|
||||
def generate_subcategory_anchor(
|
||||
title: str,
|
||||
general_counter: int = 0,
|
||||
has_back_to_top_in_heading: bool = False,
|
||||
) -> tuple[str, int]:
|
||||
"""Generate a TOC anchor for a subcategory heading.
|
||||
|
||||
Handles the special case of multiple "General" subcategories which need
|
||||
unique anchors (general, general-1, general-2, etc.).
|
||||
|
||||
Args:
|
||||
title: The subcategory name (e.g., "General", "IDE Integrations")
|
||||
general_counter: Current count of "General" subcategories seen so far
|
||||
has_back_to_top_in_heading: True if heading contains 🔝 back-to-top link
|
||||
|
||||
Returns:
|
||||
Tuple of (anchor_string, updated_general_counter)
|
||||
"""
|
||||
base = title.lower().replace(" ", "-").replace("&", "").replace("/", "")
|
||||
back_to_top_suffix = "-" if has_back_to_top_in_heading else ""
|
||||
|
||||
if title == "General":
|
||||
if general_counter == 0:
|
||||
anchor = f"general{back_to_top_suffix}"
|
||||
else:
|
||||
# GitHub uses double-dash before counter when back-to-top present
|
||||
separator = "-" if has_back_to_top_in_heading else ""
|
||||
anchor = f"general-{separator}{general_counter}"
|
||||
return anchor, general_counter + 1
|
||||
|
||||
return f"{base}{back_to_top_suffix}", general_counter
|
||||
|
||||
|
||||
def sanitize_filename_from_anchor(anchor: str) -> str:
|
||||
"""Convert an anchor string to a tidy filename fragment."""
|
||||
name = anchor.rstrip("-")
|
||||
name = name.replace("-", "_")
|
||||
name = re.sub(r"_+", "_", name)
|
||||
return name.strip("_")
|
||||
|
||||
|
||||
def build_general_anchor_map(categories: list[dict], csv_data: list[dict] | None = None) -> dict:
|
||||
"""Build a map of (category, 'General') -> anchor string shared by TOC and body."""
|
||||
general_map: dict[tuple[str, str], str] = {}
|
||||
|
||||
for category in categories:
|
||||
category_name = category.get("name", "")
|
||||
category_id = category.get("id", "")
|
||||
subcategories = category.get("subcategories", [])
|
||||
|
||||
for subcat in subcategories:
|
||||
sub_title = subcat["name"]
|
||||
if sub_title != "General":
|
||||
continue
|
||||
|
||||
include_subcategory = True
|
||||
if csv_data is not None:
|
||||
resources = [
|
||||
r
|
||||
for r in csv_data
|
||||
if r["Category"] == category_name
|
||||
and r.get("Sub-Category", "").strip() == sub_title
|
||||
]
|
||||
include_subcategory = bool(resources)
|
||||
|
||||
if not include_subcategory:
|
||||
continue
|
||||
|
||||
anchor = f"{category_id}-general"
|
||||
general_map[(category_id, sub_title)] = anchor
|
||||
|
||||
return general_map
|
||||
|
||||
|
||||
def parse_resource_date(date_string: str | None) -> datetime | None:
|
||||
"""Parse a date string that may include timestamp information."""
|
||||
if not date_string:
|
||||
return None
|
||||
|
||||
date_string = date_string.strip()
|
||||
|
||||
date_formats = [
|
||||
"%Y-%m-%d:%H-%M-%S",
|
||||
"%Y-%m-%d",
|
||||
]
|
||||
|
||||
for fmt in date_formats:
|
||||
try:
|
||||
return datetime.strptime(date_string, fmt)
|
||||
except ValueError:
|
||||
continue
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def format_category_dir_name(name: str, category_id: str | None = None) -> str:
|
||||
"""Convert category name to display text for TOC rows."""
|
||||
overrides = {
|
||||
"workflows": "WORKFLOWS_&_GUIDES/",
|
||||
}
|
||||
if category_id and category_id in overrides:
|
||||
return overrides[category_id]
|
||||
|
||||
slug = re.sub(r"[^A-Za-z0-9]+", "_", name).strip("_").upper()
|
||||
return slug + "/"
|
||||
170
.agent/knowledge/awesome_claude/scripts/readme/markup/awesome.py
Normal file
170
.agent/knowledge/awesome_claude/scripts/readme/markup/awesome.py
Normal file
@@ -0,0 +1,170 @@
|
||||
"""Awesome-list README markdown rendering helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from scripts.readme.helpers.readme_paths import asset_path_token
|
||||
from scripts.readme.helpers.readme_utils import (
|
||||
generate_subcategory_anchor,
|
||||
generate_toc_anchor,
|
||||
parse_resource_date,
|
||||
)
|
||||
|
||||
|
||||
def format_resource_entry(row: dict, include_separator: bool = True) -> str:
|
||||
"""Format resource in awesome list style."""
|
||||
_ = include_separator
|
||||
display_name = row["Display Name"]
|
||||
primary_link = row["Primary Link"]
|
||||
author_name = row.get("Author Name", "").strip()
|
||||
author_link = row.get("Author Link", "").strip()
|
||||
description = row.get("Description", "").strip()
|
||||
removed_from_origin = row.get("Removed From Origin", "").strip().upper() == "TRUE"
|
||||
|
||||
entry_parts: list[str] = []
|
||||
|
||||
if primary_link:
|
||||
entry_parts.append(f"[{display_name}]({primary_link})")
|
||||
else:
|
||||
entry_parts.append(display_name)
|
||||
|
||||
if author_name:
|
||||
if author_link:
|
||||
entry_parts.append(f" by [{author_name}]({author_link})")
|
||||
else:
|
||||
entry_parts.append(f" by {author_name}")
|
||||
|
||||
if description:
|
||||
desc = description.rstrip()
|
||||
if not desc.endswith((".", "!", "?")):
|
||||
desc += "."
|
||||
entry_parts.append(f" - {desc}")
|
||||
|
||||
result = "- " + "".join(entry_parts)
|
||||
|
||||
if removed_from_origin:
|
||||
result += " *(Removed from origin)*"
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def generate_toc(categories: list[dict], csv_data: list[dict]) -> str:
|
||||
"""Generate plain markdown TOC for awesome list style."""
|
||||
toc_lines: list[str] = []
|
||||
toc_lines.append("## Contents")
|
||||
toc_lines.append("")
|
||||
|
||||
general_counter = 0
|
||||
|
||||
for category in categories:
|
||||
section_title = category.get("name", "")
|
||||
icon = category.get("icon", "")
|
||||
subcategories = category.get("subcategories", [])
|
||||
|
||||
anchor = generate_toc_anchor(section_title, icon=icon)
|
||||
display_title = f"{section_title} {icon}" if icon else section_title
|
||||
|
||||
if subcategories:
|
||||
category_name = category.get("name", "")
|
||||
has_resources = any(r["Category"] == category_name for r in csv_data)
|
||||
|
||||
if has_resources:
|
||||
toc_lines.append(f"- [{display_title}](#{anchor})")
|
||||
|
||||
for subcat in subcategories:
|
||||
sub_title = subcat["name"]
|
||||
|
||||
resources = [
|
||||
r
|
||||
for r in csv_data
|
||||
if r["Category"] == category_name
|
||||
and r.get("Sub-Category", "").strip() == sub_title
|
||||
]
|
||||
|
||||
if resources:
|
||||
sub_anchor, general_counter = generate_subcategory_anchor(
|
||||
sub_title, general_counter
|
||||
)
|
||||
toc_lines.append(f" - [{sub_title}](#{sub_anchor})")
|
||||
else:
|
||||
toc_lines.append(f"- [{display_title}](#{anchor})")
|
||||
|
||||
return "\n".join(toc_lines).strip()
|
||||
|
||||
|
||||
def generate_weekly_section(csv_data: list[dict]) -> str:
|
||||
"""Generate weekly section with plain markdown for awesome list."""
|
||||
lines: list[str] = []
|
||||
lines.append("## Latest Additions")
|
||||
lines.append("")
|
||||
|
||||
resources_sorted_by_date: list[tuple[datetime, dict]] = []
|
||||
for row in csv_data:
|
||||
date_added = row.get("Date Added", "").strip()
|
||||
if date_added:
|
||||
parsed_date = parse_resource_date(date_added)
|
||||
if parsed_date:
|
||||
resources_sorted_by_date.append((parsed_date, row))
|
||||
resources_sorted_by_date.sort(key=lambda x: x[0], reverse=True)
|
||||
|
||||
latest_additions: list[dict[str, str]] = []
|
||||
cutoff_date = datetime.now() - timedelta(days=7)
|
||||
for dated_resource in resources_sorted_by_date:
|
||||
if dated_resource[0] >= cutoff_date or len(latest_additions) < 3:
|
||||
latest_additions.append(dated_resource[1])
|
||||
else:
|
||||
break
|
||||
|
||||
for resource in latest_additions:
|
||||
lines.append(format_resource_entry(resource, include_separator=False))
|
||||
|
||||
lines.append("")
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
|
||||
|
||||
def generate_section_content(category: dict, csv_data: list[dict]) -> str:
|
||||
"""Generate section with plain markdown headers in awesome list format."""
|
||||
lines: list[str] = []
|
||||
|
||||
title = category.get("name", "")
|
||||
icon = category.get("icon", "")
|
||||
description = category.get("description", "").strip()
|
||||
category_name = category.get("name", "")
|
||||
subcategories = category.get("subcategories", [])
|
||||
|
||||
header_text = f"{title} {icon}" if icon else title
|
||||
lines.append(f"## {header_text}")
|
||||
lines.append("")
|
||||
|
||||
if description:
|
||||
lines.append(f"> {description}")
|
||||
lines.append("")
|
||||
|
||||
for subcat in subcategories:
|
||||
sub_title = subcat["name"]
|
||||
resources = [
|
||||
r
|
||||
for r in csv_data
|
||||
if r["Category"] == category_name and r.get("Sub-Category", "").strip() == sub_title
|
||||
]
|
||||
|
||||
if resources:
|
||||
lines.append(f"### {sub_title}")
|
||||
lines.append("")
|
||||
|
||||
for resource in resources:
|
||||
lines.append(format_resource_entry(resource, include_separator=False))
|
||||
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
|
||||
|
||||
def generate_repo_ticker() -> str:
|
||||
"""Generate the awesome-style animated SVG repo ticker."""
|
||||
return f"""<div align="center">
|
||||
|
||||
<img src="{asset_path_token("repo-ticker-awesome.svg")}" alt="Featured Claude Code Projects" width="100%">
|
||||
|
||||
</div>"""
|
||||
215
.agent/knowledge/awesome_claude/scripts/readme/markup/flat.py
Normal file
215
.agent/knowledge/awesome_claude/scripts/readme/markup/flat.py
Normal file
@@ -0,0 +1,215 @@
|
||||
"""Flat list README markup rendering helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from scripts.readme.helpers.readme_paths import asset_path_token
|
||||
from scripts.readme.helpers.readme_utils import extract_github_owner_repo
|
||||
|
||||
|
||||
def generate_shields_badges(owner: str, repo: str) -> str:
|
||||
"""Generate shields.io badge HTML for a GitHub repository."""
|
||||
badge_types = [
|
||||
("stars", f"https://img.shields.io/github/stars/{owner}/{repo}"),
|
||||
("forks", f"https://img.shields.io/github/forks/{owner}/{repo}"),
|
||||
("issues", f"https://img.shields.io/github/issues/{owner}/{repo}"),
|
||||
("prs", f"https://img.shields.io/github/issues-pr/{owner}/{repo}"),
|
||||
("created", f"https://img.shields.io/github/created-at/{owner}/{repo}"),
|
||||
("last-commit", f"https://img.shields.io/github/last-commit/{owner}/{repo}"),
|
||||
("release-date", f"https://img.shields.io/github/release-date/{owner}/{repo}"),
|
||||
("version", f"https://img.shields.io/github/v/release/{owner}/{repo}"),
|
||||
("license", f"https://img.shields.io/github/license/{owner}/{repo}"),
|
||||
]
|
||||
|
||||
badges = []
|
||||
for alt, url in badge_types:
|
||||
badges.append(f'<img src="{url}?style=flat-square" alt="{alt}">')
|
||||
|
||||
return " ".join(badges)
|
||||
|
||||
|
||||
def generate_sort_navigation(
|
||||
category_slug: str,
|
||||
sort_type: str,
|
||||
sort_types: dict,
|
||||
) -> str:
|
||||
"""Generate sort option badges."""
|
||||
lines = ['<p align="center">']
|
||||
for slug, (display, color, _) in sort_types.items():
|
||||
filename = f"README_FLAT_{category_slug.upper()}_{slug.upper()}.md"
|
||||
is_selected = slug == sort_type
|
||||
style = f' style="border: 3px solid {color}; border-radius: 6px;"' if is_selected else ""
|
||||
lines.append(
|
||||
f' <a href="{filename}"><img src="{asset_path_token(f"badge-sort-{slug}.svg")}" '
|
||||
f'alt="{display}" height="48"{style}></a>'
|
||||
)
|
||||
lines.append("</p>")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def generate_category_navigation(
|
||||
category_slug: str,
|
||||
sort_type: str,
|
||||
categories: dict,
|
||||
) -> str:
|
||||
"""Generate category filter badges."""
|
||||
lines = ['<p align="center">']
|
||||
for slug, (_, display, color) in categories.items():
|
||||
filename = f"README_FLAT_{slug.upper()}_{sort_type.upper()}.md"
|
||||
is_selected = slug == category_slug
|
||||
style = f' style="border: 2px solid {color}; border-radius: 4px;"' if is_selected else ""
|
||||
lines.append(
|
||||
f' <a href="{filename}"><img src="{asset_path_token(f"badge-cat-{slug}.svg")}" '
|
||||
f'alt="{display}" height="28"{style}></a>'
|
||||
)
|
||||
lines.append("</p>")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def generate_navigation(
|
||||
category_slug: str,
|
||||
sort_type: str,
|
||||
categories: dict,
|
||||
sort_types: dict,
|
||||
) -> str:
|
||||
"""Generate combined navigation (sort + category)."""
|
||||
sort_nav = generate_sort_navigation(category_slug, sort_type, sort_types)
|
||||
cat_nav = generate_category_navigation(category_slug, sort_type, categories)
|
||||
_, _, sort_desc = sort_types[sort_type]
|
||||
_, cat_display, _ = categories[category_slug]
|
||||
|
||||
current_info = f"**{cat_display}** sorted {sort_desc}"
|
||||
if sort_type == "releases":
|
||||
current_info += " (past 30 days)"
|
||||
|
||||
return f"""{sort_nav}
|
||||
<p align="center"><strong>Category:</strong></p>
|
||||
{cat_nav}
|
||||
<p align="center"><em>Currently viewing: {current_info}</em></p>"""
|
||||
|
||||
|
||||
def generate_resources_table(sorted_resources: list[dict], sort_type: str) -> str:
|
||||
"""Generate the resources table as HTML with shields.io badges for GitHub resources."""
|
||||
if not sorted_resources:
|
||||
if sort_type == "releases":
|
||||
return "*No releases in the past 30 days for this category.*"
|
||||
return "*No resources found in this category.*"
|
||||
|
||||
lines: list[str] = ["<table>", "<thead>", "<tr>"]
|
||||
|
||||
if sort_type == "releases":
|
||||
num_cols = 5
|
||||
lines.extend(
|
||||
[
|
||||
"<th>Resource</th>",
|
||||
"<th>Version</th>",
|
||||
"<th>Source</th>",
|
||||
"<th>Release Date</th>",
|
||||
"<th>Description</th>",
|
||||
]
|
||||
)
|
||||
else:
|
||||
num_cols = 4
|
||||
lines.extend(
|
||||
[
|
||||
"<th>Resource</th>",
|
||||
"<th>Category</th>",
|
||||
"<th>Sub-Category</th>",
|
||||
"<th>Description</th>",
|
||||
]
|
||||
)
|
||||
|
||||
lines.extend(["</tr>", "</thead>", "<tbody>"])
|
||||
|
||||
for row in sorted_resources:
|
||||
display_name = row.get("Display Name", "").strip()
|
||||
primary_link = row.get("Primary Link", "").strip()
|
||||
author_name = row.get("Author Name", "").strip()
|
||||
author_link = row.get("Author Link", "").strip()
|
||||
|
||||
if primary_link:
|
||||
resource_html = f'<a href="{primary_link}"><b>{display_name}</b></a>'
|
||||
else:
|
||||
resource_html = f"<b>{display_name}</b>"
|
||||
|
||||
if author_name and author_link:
|
||||
author_html = f'<a href="{author_link}">{author_name}</a>'
|
||||
else:
|
||||
author_html = author_name or ""
|
||||
|
||||
resource_cell = f"{resource_html}<br>by {author_html}" if author_html else resource_html
|
||||
|
||||
lines.append("<tr>")
|
||||
lines.append(f"<td>{resource_cell}</td>")
|
||||
|
||||
if sort_type == "releases":
|
||||
version = row.get("Release Version", "").strip() or "-"
|
||||
source = row.get("Release Source", "").strip()
|
||||
source_display = {
|
||||
"github-releases": "GitHub",
|
||||
"npm": "npm",
|
||||
"pypi": "PyPI",
|
||||
"crates": "crates.io",
|
||||
"homebrew": "Homebrew",
|
||||
"readme": "README",
|
||||
}.get(source, source or "-")
|
||||
release_date = row.get("Latest Release", "")[:10] if row.get("Latest Release") else "-"
|
||||
description = row.get("Description", "").strip()
|
||||
|
||||
lines.append(f"<td>{version}</td>")
|
||||
lines.append(f"<td>{source_display}</td>")
|
||||
lines.append(f"<td>{release_date}</td>")
|
||||
lines.append(f"<td>{description}</td>")
|
||||
else:
|
||||
category = row.get("Category", "").strip() or "-"
|
||||
sub_category = row.get("Sub-Category", "").strip() or "-"
|
||||
description = row.get("Description", "").strip()
|
||||
|
||||
lines.append(f"<td>{category}</td>")
|
||||
lines.append(f"<td>{sub_category}</td>")
|
||||
lines.append(f"<td>{description}</td>")
|
||||
|
||||
lines.append("</tr>")
|
||||
|
||||
if primary_link:
|
||||
github_info = extract_github_owner_repo(primary_link)
|
||||
if github_info:
|
||||
owner, repo = github_info
|
||||
badges = generate_shields_badges(owner, repo)
|
||||
lines.append("<tr>")
|
||||
lines.append(f'<td colspan="{num_cols}">{badges}</td>')
|
||||
lines.append("</tr>")
|
||||
|
||||
lines.extend(["</tbody>", "</table>"])
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def get_default_template() -> str:
|
||||
"""Return default template content."""
|
||||
return """<!--lint disable remark-lint:awesome-badge-->
|
||||
|
||||
{{STYLE_SELECTOR}}
|
||||
|
||||
# Awesome Claude Code (Flat)
|
||||
|
||||
[](https://awesome.re)
|
||||
|
||||
A flat list view of all resources. Category: **{{CATEGORY_NAME}}** | Sorted: {{SORT_DESC}}
|
||||
|
||||
---
|
||||
|
||||
## Sort By:
|
||||
|
||||
{{NAVIGATION}}
|
||||
|
||||
---
|
||||
|
||||
## Resources
|
||||
{{RELEASES_DISCLAIMER}}
|
||||
{{RESOURCES_TABLE}}
|
||||
|
||||
---
|
||||
|
||||
**Total Resources:** {{RESOURCE_COUNT}}
|
||||
|
||||
**Last Generated:** {{GENERATED_DATE}}
|
||||
"""
|
||||
188
.agent/knowledge/awesome_claude/scripts/readme/markup/minimal.py
Normal file
188
.agent/knowledge/awesome_claude/scripts/readme/markup/minimal.py
Normal file
@@ -0,0 +1,188 @@
|
||||
"""Minimal README markdown rendering helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from scripts.readme.helpers.readme_utils import (
|
||||
generate_subcategory_anchor,
|
||||
generate_toc_anchor,
|
||||
parse_resource_date,
|
||||
)
|
||||
from scripts.utils.github_utils import parse_github_url
|
||||
|
||||
|
||||
def format_resource_entry(row: dict, include_separator: bool = True) -> str:
|
||||
"""Format resource as plain markdown with collapsible GitHub stats."""
|
||||
_ = include_separator
|
||||
display_name = row["Display Name"]
|
||||
primary_link = row["Primary Link"]
|
||||
author_name = row.get("Author Name", "").strip()
|
||||
author_link = row.get("Author Link", "").strip()
|
||||
description = row.get("Description", "").strip()
|
||||
license_info = row.get("License", "").strip()
|
||||
removed_from_origin = row.get("Removed From Origin", "").strip().upper() == "TRUE"
|
||||
|
||||
entry_parts = [f"[`{display_name}`]({primary_link})"]
|
||||
|
||||
if author_name:
|
||||
if author_link:
|
||||
entry_parts.append(f" by [{author_name}]({author_link})")
|
||||
else:
|
||||
entry_parts.append(f" by {author_name}")
|
||||
|
||||
entry_parts.append(" ")
|
||||
|
||||
if license_info and license_info != "NOT_FOUND":
|
||||
entry_parts.append(f" ⚖️ {license_info}")
|
||||
|
||||
result = "".join(entry_parts)
|
||||
|
||||
if description:
|
||||
result += f" \n{description}" + ("* " if removed_from_origin else "")
|
||||
|
||||
if removed_from_origin:
|
||||
result += "\n<sub>* Removed from origin</sub>"
|
||||
|
||||
if primary_link and not removed_from_origin:
|
||||
_, is_github, owner, repo = parse_github_url(primary_link)
|
||||
if is_github and owner and repo:
|
||||
base_url = "https://github-readme-stats-fork-orpin.vercel.app/api/pin/"
|
||||
stats_url = f"{base_url}?repo={repo}&username={owner}&all_stats=true&stats_only=true"
|
||||
result += "\n\n<details>"
|
||||
result += "\n<summary>📊 GitHub Stats</summary>"
|
||||
result += f"\n\n"
|
||||
result += "\n\n</details>"
|
||||
result += "\n<br>"
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def generate_toc(categories: list[dict], csv_data: list[dict]) -> str:
|
||||
"""Generate plain markdown nested details TOC."""
|
||||
toc_lines: list[str] = []
|
||||
toc_lines.append("## Contents [🔝](#awesome-claude-code)")
|
||||
toc_lines.append("")
|
||||
toc_lines.append("<details open>")
|
||||
toc_lines.append("<summary>Table of Contents</summary>")
|
||||
toc_lines.append("")
|
||||
|
||||
general_counter = 0
|
||||
# CLASSIC style headings include [🔝](#awesome-claude-code) which adds another dash
|
||||
has_back_to_top = True
|
||||
|
||||
for category in categories:
|
||||
section_title = category.get("name", "")
|
||||
icon = category.get("icon", "")
|
||||
subcategories = category.get("subcategories", [])
|
||||
|
||||
anchor = generate_toc_anchor(
|
||||
section_title, icon=icon, has_back_to_top_in_heading=has_back_to_top
|
||||
)
|
||||
|
||||
if subcategories:
|
||||
toc_lines.append("- <details open>")
|
||||
toc_lines.append(f' <summary><a href="#{anchor}">{section_title}</a></summary>')
|
||||
toc_lines.append("")
|
||||
|
||||
for subcat in subcategories:
|
||||
sub_title = subcat["name"]
|
||||
|
||||
category_name = category.get("name", "")
|
||||
resources = [
|
||||
r
|
||||
for r in csv_data
|
||||
if r["Category"] == category_name
|
||||
and r.get("Sub-Category", "").strip() == sub_title
|
||||
]
|
||||
|
||||
if resources:
|
||||
sub_anchor, general_counter = generate_subcategory_anchor(
|
||||
sub_title, general_counter, has_back_to_top_in_heading=has_back_to_top
|
||||
)
|
||||
toc_lines.append(f" - [{sub_title}](#{sub_anchor})")
|
||||
|
||||
toc_lines.append("")
|
||||
toc_lines.append(" </details>")
|
||||
else:
|
||||
toc_lines.append(f"- [{section_title}](#{anchor})")
|
||||
|
||||
toc_lines.append("")
|
||||
|
||||
toc_lines.append("</details>")
|
||||
return "\n".join(toc_lines).strip()
|
||||
|
||||
|
||||
def generate_weekly_section(csv_data: list[dict]) -> str:
|
||||
"""Generate weekly section with plain markdown."""
|
||||
lines: list[str] = []
|
||||
lines.append("## Latest Additions ✨ [🔝](#awesome-claude-code)")
|
||||
lines.append("")
|
||||
|
||||
resources_sorted_by_date: list[tuple[datetime, dict]] = []
|
||||
for row in csv_data:
|
||||
date_added = row.get("Date Added", "").strip()
|
||||
if date_added:
|
||||
parsed_date = parse_resource_date(date_added)
|
||||
if parsed_date:
|
||||
resources_sorted_by_date.append((parsed_date, row))
|
||||
resources_sorted_by_date.sort(key=lambda x: x[0], reverse=True)
|
||||
|
||||
latest_additions: list[dict[str, str]] = []
|
||||
cutoff_date = datetime.now() - timedelta(days=7)
|
||||
for dated_resource in resources_sorted_by_date:
|
||||
if dated_resource[0] >= cutoff_date or len(latest_additions) < 3:
|
||||
latest_additions.append(dated_resource[1])
|
||||
else:
|
||||
break
|
||||
|
||||
lines.append("")
|
||||
for resource in latest_additions:
|
||||
lines.append(format_resource_entry(resource, include_separator=False))
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
|
||||
|
||||
def generate_section_content(category: dict, csv_data: list[dict]) -> str:
|
||||
"""Generate section with plain markdown headers."""
|
||||
lines: list[str] = []
|
||||
|
||||
title = category.get("name", "")
|
||||
icon = category.get("icon", "")
|
||||
description = category.get("description", "").strip()
|
||||
category_name = category.get("name", "")
|
||||
subcategories = category.get("subcategories", [])
|
||||
|
||||
header_text = f"{title} {icon}" if icon else title
|
||||
lines.append(f"## {header_text} [🔝](#awesome-claude-code)")
|
||||
lines.append("")
|
||||
|
||||
if description:
|
||||
lines.append(f"> {description}")
|
||||
lines.append("")
|
||||
|
||||
for subcat in subcategories:
|
||||
sub_title = subcat["name"]
|
||||
resources = [
|
||||
r
|
||||
for r in csv_data
|
||||
if r["Category"] == category_name and r.get("Sub-Category", "").strip() == sub_title
|
||||
]
|
||||
|
||||
if resources:
|
||||
lines.append("<details open>")
|
||||
lines.append(
|
||||
f'<summary><h3>{sub_title} <a href="#awesome-claude-code">🔝</a></h3></summary>'
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
for resource in resources:
|
||||
lines.append(format_resource_entry(resource, include_separator=False))
|
||||
lines.append("")
|
||||
|
||||
lines.append("</details>")
|
||||
lines.append("")
|
||||
|
||||
lines.append("<br>")
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
113
.agent/knowledge/awesome_claude/scripts/readme/markup/shared.py
Normal file
113
.agent/knowledge/awesome_claude/scripts/readme/markup/shared.py
Normal file
@@ -0,0 +1,113 @@
|
||||
"""Markdown/HTML rendering helpers shared across README styles."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import yaml # type: ignore[import-untyped]
|
||||
|
||||
from scripts.readme.helpers.readme_config import CONFIG, get_style_selector_target
|
||||
from scripts.readme.helpers.readme_paths import asset_path_token, resolve_relative_link
|
||||
|
||||
|
||||
def generate_style_selector(
|
||||
current_style: str, output_path: Path, repo_root: Path | None = None
|
||||
) -> str:
|
||||
"""Generate the style selector HTML for a README."""
|
||||
styles = CONFIG.get("styles", {})
|
||||
style_order = CONFIG.get("style_order", ["extra", "classic", "flat", "awesome"])
|
||||
|
||||
lines = ['<h3 align="center">Pick Your Style:</h3>', '<p align="center">']
|
||||
|
||||
for style_id in style_order:
|
||||
style_config = styles.get(style_id, {})
|
||||
name = style_config.get("name", style_id.title())
|
||||
badge = style_config.get("badge", f"badge-style-{style_id}.svg")
|
||||
highlight_color = style_config.get("highlight_color", "#666666")
|
||||
|
||||
target_path = Path(get_style_selector_target(style_id))
|
||||
href = resolve_relative_link(output_path, target_path, repo_root)
|
||||
|
||||
if style_id == current_style:
|
||||
style_attr = f' style="border: 2px solid {highlight_color}; border-radius: 4px;"'
|
||||
else:
|
||||
style_attr = ""
|
||||
|
||||
badge_src = asset_path_token(badge)
|
||||
lines.append(
|
||||
f'<a href="{href}"><img src="{badge_src}" alt="{name}" height="28"{style_attr}></a>'
|
||||
)
|
||||
|
||||
lines.append("</p>")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def load_announcements(template_dir: str) -> str:
|
||||
"""Load announcements from the announcements.yaml file and format as markdown."""
|
||||
announcements_path = os.path.join(template_dir, "announcements.yaml")
|
||||
if os.path.exists(announcements_path):
|
||||
with open(announcements_path, encoding="utf-8") as f:
|
||||
announcements_data = yaml.safe_load(f)
|
||||
|
||||
if not announcements_data:
|
||||
return ""
|
||||
|
||||
markdown_lines = []
|
||||
markdown_lines.append("### Announcements [🔝](#awesome-claude-code)")
|
||||
markdown_lines.append("")
|
||||
|
||||
markdown_lines.append("<details open>")
|
||||
markdown_lines.append("<summary>View Announcements</summary>")
|
||||
markdown_lines.append("")
|
||||
|
||||
for entry in announcements_data:
|
||||
date = entry.get("date", "")
|
||||
title = entry.get("title", "")
|
||||
items = entry.get("items", [])
|
||||
|
||||
markdown_lines.append("- <details open>")
|
||||
|
||||
if title:
|
||||
markdown_lines.append(f" <summary>{date} - {title}</summary>")
|
||||
else:
|
||||
markdown_lines.append(f" <summary>{date}</summary>")
|
||||
|
||||
markdown_lines.append("")
|
||||
|
||||
for item in items:
|
||||
if isinstance(item, str):
|
||||
markdown_lines.append(f" - {item}")
|
||||
elif isinstance(item, dict):
|
||||
summary = item.get("summary", "")
|
||||
text = item.get("text", "")
|
||||
|
||||
if summary and text:
|
||||
markdown_lines.append(" - <details open>")
|
||||
markdown_lines.append(f" <summary>{summary}</summary>")
|
||||
markdown_lines.append("")
|
||||
|
||||
text_lines = text.strip().split("\n")
|
||||
for i, line in enumerate(text_lines):
|
||||
if i == 0:
|
||||
markdown_lines.append(f" - {line}")
|
||||
else:
|
||||
markdown_lines.append(f" {line}")
|
||||
|
||||
markdown_lines.append("")
|
||||
markdown_lines.append(" </details>")
|
||||
elif summary:
|
||||
markdown_lines.append(f" - {summary}")
|
||||
elif text:
|
||||
markdown_lines.append(f" - {text}")
|
||||
|
||||
markdown_lines.append("")
|
||||
|
||||
markdown_lines.append(" </details>")
|
||||
markdown_lines.append("")
|
||||
|
||||
markdown_lines.append("</details>")
|
||||
|
||||
return "\n".join(markdown_lines).strip()
|
||||
|
||||
return ""
|
||||
414
.agent/knowledge/awesome_claude/scripts/readme/markup/visual.py
Normal file
414
.agent/knowledge/awesome_claude/scripts/readme/markup/visual.py
Normal file
@@ -0,0 +1,414 @@
|
||||
"""Visual README markup rendering helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.readme.helpers.readme_assets import (
|
||||
create_h3_svg_file,
|
||||
ensure_category_header_exists,
|
||||
ensure_separator_svg_exists,
|
||||
get_category_svg_filename,
|
||||
get_section_divider_svg,
|
||||
get_subcategory_svg_filename,
|
||||
save_resource_badge_svg,
|
||||
)
|
||||
from scripts.readme.helpers.readme_paths import asset_path_token
|
||||
from scripts.readme.helpers.readme_utils import (
|
||||
generate_toc_anchor,
|
||||
parse_resource_date,
|
||||
sanitize_filename_from_anchor,
|
||||
)
|
||||
from scripts.utils.github_utils import parse_github_url
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
|
||||
def format_resource_entry(
|
||||
row: dict,
|
||||
assets_dir: str | None = None,
|
||||
include_separator: bool = True,
|
||||
) -> str:
|
||||
"""Format a single resource entry with vintage manual styling for light mode."""
|
||||
display_name = row["Display Name"]
|
||||
primary_link = row["Primary Link"]
|
||||
author_name = row.get("Author Name", "").strip()
|
||||
description = row.get("Description", "").strip()
|
||||
removed_from_origin = row.get("Removed From Origin", "").strip().upper() == "TRUE"
|
||||
|
||||
parts: list[str] = []
|
||||
|
||||
if assets_dir:
|
||||
badge_filename = save_resource_badge_svg(display_name, author_name, assets_dir)
|
||||
parts.append(f'<a href="{primary_link}">')
|
||||
parts.append(f'<img src="{asset_path_token(badge_filename)}" alt="{display_name}">')
|
||||
parts.append("</a>")
|
||||
else:
|
||||
parts.append(f"[`{display_name}`]({primary_link})")
|
||||
if author_name:
|
||||
parts.append(f" by {author_name}")
|
||||
|
||||
if description:
|
||||
parts.append(" \n")
|
||||
parts.append(f"_{description}_" + ("*" if removed_from_origin else ""))
|
||||
|
||||
if removed_from_origin:
|
||||
parts.append(" \n")
|
||||
parts.append("<sub>* Removed from origin</sub>")
|
||||
|
||||
if primary_link and not removed_from_origin:
|
||||
_, is_github, owner, repo = parse_github_url(primary_link)
|
||||
|
||||
if is_github and owner and repo:
|
||||
base_url = "https://github-readme-stats-fork-orpin.vercel.app/api/pin/"
|
||||
stats_url = (
|
||||
f"{base_url}?repo={repo}&username={owner}&all_stats=true&stats_only=true"
|
||||
"&hide_border=true&bg_color=00000000&icon_color=FF0000&text_color=FF0000"
|
||||
)
|
||||
parts.append(" \n")
|
||||
parts.append(f"")
|
||||
|
||||
if include_separator and assets_dir:
|
||||
separator_filename = ensure_separator_svg_exists(assets_dir)
|
||||
parts.append("\n\n")
|
||||
parts.append('<div align="center">')
|
||||
parts.append(f'<img src="{asset_path_token(separator_filename)}" alt="">')
|
||||
parts.append("</div>")
|
||||
parts.append("\n")
|
||||
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def generate_weekly_section(
|
||||
csv_data: list[dict],
|
||||
assets_dir: str | None = None,
|
||||
) -> str:
|
||||
"""Generate the latest additions section that appears above Contents."""
|
||||
lines: list[str] = []
|
||||
|
||||
lines.append('<div align="center">')
|
||||
lines.append(" <picture>")
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: dark)" '
|
||||
f'srcset="{asset_path_token("latest-additions-header.svg")}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: light)" '
|
||||
f'srcset="{asset_path_token("latest-additions-header-light.svg")}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <img src="{asset_path_token("latest-additions-header-light.svg")}" '
|
||||
'alt="LATEST ADDITIONS">'
|
||||
)
|
||||
lines.append(" </picture>")
|
||||
lines.append("</div>")
|
||||
lines.append("")
|
||||
|
||||
resources_sorted_by_date: list[tuple[datetime, dict]] = []
|
||||
for row in csv_data:
|
||||
date_added = row.get("Date Added", "").strip()
|
||||
if date_added:
|
||||
parsed_date = parse_resource_date(date_added)
|
||||
if parsed_date:
|
||||
resources_sorted_by_date.append((parsed_date, row))
|
||||
resources_sorted_by_date.sort(key=lambda x: x[0], reverse=True)
|
||||
|
||||
latest_additions: list[dict] = []
|
||||
cutoff_date = datetime.now() - timedelta(days=7)
|
||||
for dated_resource in resources_sorted_by_date:
|
||||
if dated_resource[0] >= cutoff_date or len(latest_additions) < 3:
|
||||
latest_additions.append(dated_resource[1])
|
||||
else:
|
||||
break
|
||||
|
||||
for resource in latest_additions:
|
||||
lines.append(
|
||||
format_resource_entry(
|
||||
resource,
|
||||
assets_dir=assets_dir,
|
||||
include_separator=False,
|
||||
)
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
|
||||
|
||||
def generate_toc_from_categories(
|
||||
categories: list[dict] | None = None,
|
||||
csv_data: list[dict] | None = None,
|
||||
general_map: dict | None = None,
|
||||
) -> str:
|
||||
"""Generate simple table of contents as vertical list of SVG rows."""
|
||||
if categories is None:
|
||||
from scripts.categories.category_utils import category_manager
|
||||
|
||||
categories = category_manager.get_categories_for_readme()
|
||||
|
||||
toc_header = f"""<!-- Directory Tree Terminal - Theme Adaptive -->
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="{asset_path_token("toc-header.svg")}">
|
||||
<source media="(prefers-color-scheme: light)" srcset="{asset_path_token("toc-header-light-anim-scanline.svg")}">
|
||||
<img src="{asset_path_token("toc-header-light-anim-scanline.svg")}" alt="Directory Listing" height="48" \
|
||||
style="height:48px;max-width:none;">
|
||||
</picture>"""
|
||||
|
||||
toc_lines = [
|
||||
'<div style="overflow-x:auto;white-space:nowrap;text-align:left;">',
|
||||
f'<div style="height:48px;width:400px;overflow:hidden;display:block;">{toc_header}</div>',
|
||||
]
|
||||
|
||||
for category in categories:
|
||||
section_title = category["name"]
|
||||
category_name = category.get("name", "")
|
||||
category_id = category.get("id", "")
|
||||
# EXTRA style uses explicit IDs with trailing dash (no icon in anchor)
|
||||
anchor = generate_toc_anchor(section_title, icon=None, has_back_to_top_in_heading=True)
|
||||
|
||||
svg_filename = get_category_svg_filename(category_id)
|
||||
|
||||
dark_svg = svg_filename
|
||||
light_svg = svg_filename.replace(".svg", "-light-anim-scanline.svg")
|
||||
toc_lines.append('<div style="height:40px;width:400px;overflow:hidden;display:block;">')
|
||||
toc_lines.append(f'<a href="#{anchor}">')
|
||||
toc_lines.append(" <picture>")
|
||||
toc_lines.append(
|
||||
f' <source media="(prefers-color-scheme: dark)" srcset="{asset_path_token(dark_svg)}">'
|
||||
)
|
||||
toc_lines.append(
|
||||
f' <source media="(prefers-color-scheme: light)" srcset="{asset_path_token(light_svg)}">'
|
||||
)
|
||||
toc_lines.append(
|
||||
f' <img src="{asset_path_token(light_svg)}" alt="{section_title}" '
|
||||
'height="40" style="height:40px;max-width:none;">'
|
||||
)
|
||||
toc_lines.append(" </picture>")
|
||||
toc_lines.append("</a>")
|
||||
toc_lines.append("</div>")
|
||||
|
||||
subcategories = category.get("subcategories", [])
|
||||
|
||||
if subcategories:
|
||||
for subcat in subcategories:
|
||||
sub_title = subcat["name"]
|
||||
subcat_id = subcat.get("id", "")
|
||||
|
||||
include_subcategory = True
|
||||
if csv_data is not None:
|
||||
resources = [
|
||||
r
|
||||
for r in csv_data
|
||||
if r["Category"] == category_name
|
||||
and r.get("Sub-Category", "").strip() == sub_title
|
||||
]
|
||||
include_subcategory = bool(resources)
|
||||
|
||||
if include_subcategory:
|
||||
sub_anchor = (
|
||||
sub_title.lower().replace(" ", "-").replace("&", "").replace("/", "")
|
||||
)
|
||||
|
||||
if sub_title == "General":
|
||||
if general_map is not None:
|
||||
sub_anchor = general_map.get((category_id, sub_title), "general")
|
||||
else:
|
||||
sub_anchor = f"{category_id}-general"
|
||||
|
||||
svg_filename = get_subcategory_svg_filename(subcat_id)
|
||||
|
||||
dark_svg = svg_filename
|
||||
light_svg = svg_filename.replace(".svg", "-light-anim-scanline.svg")
|
||||
toc_lines.append(
|
||||
'<div style="height:40px;width:400px;overflow:hidden;display:block;">'
|
||||
)
|
||||
toc_lines.append(f'<a href="#{sub_anchor}">')
|
||||
toc_lines.append(" <picture>")
|
||||
toc_lines.append(
|
||||
f' <source media="(prefers-color-scheme: dark)" '
|
||||
f'srcset="{asset_path_token(dark_svg)}">'
|
||||
)
|
||||
toc_lines.append(
|
||||
f' <source media="(prefers-color-scheme: light)" '
|
||||
f'srcset="{asset_path_token(light_svg)}">'
|
||||
)
|
||||
toc_lines.append(
|
||||
f' <img src="{asset_path_token(light_svg)}" alt="{sub_title}" '
|
||||
'height="40" style="height:40px;max-width:none;">'
|
||||
)
|
||||
toc_lines.append(" </picture>")
|
||||
toc_lines.append("</a>")
|
||||
toc_lines.append("</div>")
|
||||
|
||||
toc_lines.append("</div>")
|
||||
|
||||
return "\n".join(toc_lines).strip()
|
||||
|
||||
|
||||
def generate_section_content(
|
||||
category: dict,
|
||||
csv_data: list[dict],
|
||||
general_map: dict | None = None,
|
||||
assets_dir: str | None = None,
|
||||
section_index: int = 0,
|
||||
) -> str:
|
||||
"""Generate content for a category based on CSV data."""
|
||||
lines: list[str] = []
|
||||
|
||||
category_id = category.get("id", "")
|
||||
title = category.get("name", "")
|
||||
icon = category.get("icon", "")
|
||||
description = category.get("description", "").strip()
|
||||
category_name = category.get("name", "")
|
||||
subcategories = category.get("subcategories", [])
|
||||
|
||||
dark_divider, light_divider = get_section_divider_svg()
|
||||
lines.append('<div align="center">')
|
||||
lines.append(" <picture>")
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: dark)" srcset="{asset_path_token(dark_divider)}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: light)" srcset="{asset_path_token(light_divider)}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <img src="{asset_path_token(light_divider)}" alt="" width="100%" style="max-width: 800px;">'
|
||||
)
|
||||
lines.append(" </picture>")
|
||||
lines.append("</div>")
|
||||
lines.append("")
|
||||
|
||||
# EXTRA style uses explicit IDs with trailing dash (no icon in anchor)
|
||||
anchor_id = generate_toc_anchor(title, icon=None, has_back_to_top_in_heading=True)
|
||||
|
||||
section_number = str(section_index + 1).zfill(2)
|
||||
display_title = title
|
||||
if category_id == "workflows":
|
||||
display_title = "Workflows & Guides"
|
||||
assert assets_dir is not None
|
||||
dark_header, light_header = ensure_category_header_exists(
|
||||
category_id,
|
||||
display_title,
|
||||
section_number,
|
||||
assets_dir,
|
||||
icon=icon,
|
||||
always_regenerate=True,
|
||||
)
|
||||
|
||||
lines.append(f'<h2 id="{anchor_id}">')
|
||||
lines.append('<div align="center">')
|
||||
lines.append(" <picture>")
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: dark)" srcset="{asset_path_token(dark_header)}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: light)" srcset="{asset_path_token(light_header)}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <img src="{asset_path_token(light_header)}" alt="{title}" style="max-width: 600px;">'
|
||||
)
|
||||
lines.append(" </picture>")
|
||||
lines.append("</div>")
|
||||
lines.append("</h2>")
|
||||
lines.append('<div align="right"><a href="#awesome-claude-code">🔝 Back to top</a></div>')
|
||||
lines.append("")
|
||||
|
||||
if description:
|
||||
lines.append("")
|
||||
lines.append('<div align="center">')
|
||||
lines.append(" <picture>")
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: dark)" '
|
||||
f'srcset="{asset_path_token("desc-box-top.svg")}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: light)" '
|
||||
f'srcset="{asset_path_token("desc-box-top-light.svg")}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <img src="{asset_path_token("desc-box-top-light.svg")}" alt="" '
|
||||
'width="100%" style="max-width: 900px;">'
|
||||
)
|
||||
lines.append(" </picture>")
|
||||
lines.append("</div>")
|
||||
lines.append(f"<h3 id='{anchor_id}' align='center'>{description}</h3>")
|
||||
lines.append('<div align="center">')
|
||||
lines.append(" <picture>")
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: dark)" '
|
||||
f'srcset="{asset_path_token("desc-box-bottom.svg")}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <source media="(prefers-color-scheme: light)" '
|
||||
f'srcset="{asset_path_token("desc-box-bottom-light.svg")}">'
|
||||
)
|
||||
lines.append(
|
||||
f' <img src="{asset_path_token("desc-box-bottom-light.svg")}" alt="" '
|
||||
'width="100%" style="max-width: 900px;">'
|
||||
)
|
||||
lines.append(" </picture>")
|
||||
lines.append("</div>")
|
||||
|
||||
for subcat in subcategories:
|
||||
sub_title = subcat["name"]
|
||||
|
||||
resources = [
|
||||
r
|
||||
for r in csv_data
|
||||
if r["Category"] == category_name and r.get("Sub-Category", "").strip() == sub_title
|
||||
]
|
||||
|
||||
if resources:
|
||||
lines.append("")
|
||||
|
||||
sub_anchor = sub_title.lower().replace(" ", "-").replace("&", "").replace("/", "")
|
||||
|
||||
if sub_title == "General":
|
||||
if general_map is not None:
|
||||
sub_anchor = general_map.get((category_id, sub_title), "general")
|
||||
else:
|
||||
sub_anchor = f"{category_id}-general"
|
||||
|
||||
sub_anchor_id = sub_anchor
|
||||
|
||||
safe_filename = sanitize_filename_from_anchor(sub_anchor)
|
||||
svg_filename = f"subheader_{safe_filename}.svg"
|
||||
|
||||
assets_root = str(REPO_ROOT / "assets")
|
||||
create_h3_svg_file(sub_title, svg_filename, assets_root)
|
||||
|
||||
lines.append(f'<details open id="{sub_anchor_id}">')
|
||||
lines.append(
|
||||
f'<summary><span><picture><img src="{asset_path_token(svg_filename)}" '
|
||||
f'alt="{sub_title}" align="absmiddle"></picture></span></summary>'
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
for resource in resources:
|
||||
lines.append(
|
||||
format_resource_entry(
|
||||
resource,
|
||||
assets_dir=assets_dir,
|
||||
)
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
lines.append("</details>")
|
||||
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
|
||||
|
||||
def generate_repo_ticker() -> str:
|
||||
"""Generate the animated SVG repo ticker for visual theme."""
|
||||
return f"""<div align="center">
|
||||
|
||||
<br />
|
||||
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="{asset_path_token("repo-ticker.svg")}">
|
||||
<source media="(prefers-color-scheme: light)" srcset="{asset_path_token("repo-ticker-light.svg")}">
|
||||
<img src="{asset_path_token("repo-ticker-light.svg")}" alt="Featured Claude Code Projects" width="100%">
|
||||
</picture>
|
||||
|
||||
</div>"""
|
||||
@@ -0,0 +1,98 @@
|
||||
"""SVG renderers for badges."""
|
||||
|
||||
|
||||
def generate_resource_badge_svg(display_name, author_name=""):
|
||||
"""Generate SVG content for a resource name badge with theme-adaptive colors.
|
||||
|
||||
Uses CSS media queries to switch between light and dark color schemes.
|
||||
- Light: dark text on transparent background
|
||||
- Dark: light text on transparent background
|
||||
"""
|
||||
# Get first two letters/initials for the box
|
||||
words = display_name.split()
|
||||
if len(words) >= 2:
|
||||
initials = words[0][0].upper() + words[1][0].upper()
|
||||
else:
|
||||
initials = display_name[:2].upper()
|
||||
|
||||
# Escape XML special characters
|
||||
name_escaped = (
|
||||
display_name.replace("&", "&")
|
||||
.replace("<", "<")
|
||||
.replace(">", ">")
|
||||
.replace('"', """)
|
||||
)
|
||||
author_escaped = (
|
||||
author_name.replace("&", "&")
|
||||
.replace("<", "<")
|
||||
.replace(">", ">")
|
||||
.replace('"', """)
|
||||
if author_name
|
||||
else ""
|
||||
)
|
||||
|
||||
# Calculate width based on text length (approximate) - larger fonts need more space
|
||||
name_width = len(display_name) * 10
|
||||
author_width = (len(author_name) * 7 + 35) if author_name else 0 # 35px for "by "
|
||||
text_width = name_width + author_width + 70 # 70px for box + padding
|
||||
svg_width = max(220, min(700, text_width))
|
||||
|
||||
# Calculate position for author text
|
||||
name_end_x = 48 + name_width
|
||||
|
||||
# Build author text element if author provided
|
||||
author_element = ""
|
||||
if author_name:
|
||||
author_element = f"""
|
||||
<text class="author" x="{name_end_x + 10}" y="30" font-family="system-ui, -apple-system, 'Helvetica Neue', sans-serif" font-size="14" font-weight="400">by {author_escaped}</text>"""
|
||||
|
||||
svg = f"""<svg width="{svg_width}" height="44" xmlns="http://www.w3.org/2000/svg">
|
||||
<style>
|
||||
@media (prefers-color-scheme: light) {{
|
||||
.line {{ stroke: #5c5247; }}
|
||||
.box {{ stroke: #5c5247; }}
|
||||
.initials {{ fill: #c96442; }}
|
||||
.name {{ fill: #3d3530; }}
|
||||
.author {{ fill: #5c5247; opacity: 0.7; }}
|
||||
}}
|
||||
@media (prefers-color-scheme: dark) {{
|
||||
.line {{ stroke: #888; }}
|
||||
.box {{ stroke: #888; }}
|
||||
.initials {{ fill: #ff6b4a; }}
|
||||
.name {{ fill: #e8e8e8; }}
|
||||
.author {{ fill: #aaa; opacity: 0.8; }}
|
||||
}}
|
||||
</style>
|
||||
|
||||
<!-- Thin top line -->
|
||||
<line class="line" x1="4" y1="6" x2="{svg_width - 4}" y2="6" stroke-width="1.25" opacity="0.4"/>
|
||||
|
||||
<!-- Initials box -->
|
||||
<rect class="box" x="4" y="12" width="32" height="26" fill="none" stroke-width="2.25" opacity="0.6"/>
|
||||
<text class="initials" x="20" y="30" font-family="'Courier New', Courier, monospace" font-size="14" font-weight="700" text-anchor="middle">{initials}</text>
|
||||
|
||||
<!-- Resource name -->
|
||||
<text class="name" x="48" y="30" font-family="system-ui, -apple-system, 'Helvetica Neue', sans-serif" font-size="17" font-weight="600">{name_escaped}</text>{author_element}
|
||||
|
||||
<!-- Bottom rule -->
|
||||
<line class="line" x1="48" y1="37" x2="{svg_width - 4}" y2="37" stroke-width="1.25" opacity="0.5"/>
|
||||
</svg>"""
|
||||
return svg
|
||||
|
||||
|
||||
def render_flat_sort_badge_svg(display: str, color: str) -> str:
|
||||
"""Render a flat-list sort badge SVG."""
|
||||
return f"""<svg xmlns="http://www.w3.org/2000/svg" width="180" height="48" viewBox="0 0 180 48">
|
||||
<rect x="0" y="0" width="180" height="48" fill="#1a1a2e"/>
|
||||
<rect x="0" y="0" width="6" height="48" fill="{color}"/>
|
||||
<text x="93" y="32" font-family="'SF Mono', 'Consolas', monospace" font-size="18" font-weight="700" fill="#e2e8f0" text-anchor="middle" letter-spacing="1">{display}</text>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def render_flat_category_badge_svg(display: str, color: str, width: int) -> str:
|
||||
"""Render a flat-list category badge SVG."""
|
||||
return f"""<svg xmlns="http://www.w3.org/2000/svg" width="{width}" height="28" viewBox="0 0 {width} 28">
|
||||
<rect x="0" y="0" width="{width}" height="28" fill="#27272a"/>
|
||||
<rect x="0" y="0" width="4" height="28" fill="{color}"/>
|
||||
<text x="{width // 2 + 2}" y="19" font-family="'SF Mono', 'Consolas', monospace" font-size="12" font-weight="600" fill="#d4d4d8" text-anchor="middle">{display}</text>
|
||||
</svg>"""
|
||||
@@ -0,0 +1,290 @@
|
||||
"""SVG renderers for section dividers and boxes."""
|
||||
|
||||
|
||||
def generate_section_divider_light_svg(variant=1):
|
||||
"""Generate a light-mode section divider SVG.
|
||||
|
||||
Args:
|
||||
variant: 1, 2, or 3 for different styles
|
||||
"""
|
||||
if variant == 1:
|
||||
# Diagram/schematic style with nodes
|
||||
return """<svg width="900" height="40" xmlns="http://www.w3.org/2000/svg">
|
||||
<!-- Vintage Technical Manual Style - Diagram Variant (BOLD) -->
|
||||
|
||||
<!-- Ghost line for layered effect -->
|
||||
<line x1="82" y1="18" x2="818" y2="18" stroke="#5c5247" stroke-width="1.5" opacity="0.2"/>
|
||||
|
||||
<!-- Main horizontal rule -->
|
||||
<line x1="80" y1="20" x2="820" y2="20" stroke="#5c5247" stroke-width="1.75" opacity="0.65"/>
|
||||
|
||||
<!-- Technical node markers along the line -->
|
||||
<g fill="none" stroke="#5c5247">
|
||||
<!-- Left terminal node -->
|
||||
<circle cx="80" cy="20" r="5" stroke-width="1.5" opacity="0.7"/>
|
||||
<circle cx="80" cy="20" r="2.5" fill="#5c5247" opacity="0.55"/>
|
||||
<circle cx="82" cy="22" r="1.5" fill="#5c5247" opacity="0.25"/>
|
||||
|
||||
<!-- Intermediate nodes -->
|
||||
<circle cx="200" cy="20" r="3.5" stroke-width="1.25" opacity="0.55"/>
|
||||
<circle cx="350" cy="20" r="3.5" stroke-width="1.25" opacity="0.55"/>
|
||||
|
||||
<!-- Center node - emphasized -->
|
||||
<circle cx="450" cy="20" r="6" stroke-width="1.5" opacity="0.65"/>
|
||||
<circle cx="450" cy="20" r="3.5" fill="#c96442" opacity="0.75"/>
|
||||
<circle cx="452" cy="22" r="2" fill="#c96442" opacity="0.35"/>
|
||||
|
||||
<!-- Intermediate nodes -->
|
||||
<circle cx="550" cy="20" r="3.5" stroke-width="1.25" opacity="0.55"/>
|
||||
<circle cx="700" cy="20" r="3.5" stroke-width="1.25" opacity="0.55"/>
|
||||
|
||||
<!-- Right terminal node -->
|
||||
<circle cx="820" cy="20" r="5" stroke-width="1.5" opacity="0.7"/>
|
||||
<circle cx="820" cy="20" r="2.5" fill="#5c5247" opacity="0.55"/>
|
||||
<circle cx="818" cy="22" r="1.5" fill="#5c5247" opacity="0.25"/>
|
||||
</g>
|
||||
|
||||
<!-- Measurement ticks -->
|
||||
<g stroke="#5c5247" opacity="0.4">
|
||||
<line x1="140" y1="15" x2="140" y2="25" stroke-width="1"/>
|
||||
<line x1="142" y1="16" x2="142" y2="24" stroke-width="0.75" opacity="0.5"/>
|
||||
<line x1="260" y1="15" x2="260" y2="25" stroke-width="1"/>
|
||||
<line x1="380" y1="15" x2="380" y2="25" stroke-width="1"/>
|
||||
<line x1="382" y1="16" x2="382" y2="24" stroke-width="0.75" opacity="0.5"/>
|
||||
<line x1="520" y1="15" x2="520" y2="25" stroke-width="1"/>
|
||||
<line x1="640" y1="15" x2="640" y2="25" stroke-width="1"/>
|
||||
<line x1="642" y1="16" x2="642" y2="24" stroke-width="0.75" opacity="0.5"/>
|
||||
<line x1="760" y1="15" x2="760" y2="25" stroke-width="1"/>
|
||||
</g>
|
||||
|
||||
<!-- Directional arrows at ends -->
|
||||
<g stroke="#5c5247" stroke-width="1.5" fill="none" opacity="0.5">
|
||||
<path d="M 52 20 L 65 20 M 58 15 L 65 20 L 58 25"/>
|
||||
<path d="M 848 20 L 835 20 M 842 15 L 835 20 L 842 25"/>
|
||||
</g>
|
||||
</svg>"""
|
||||
|
||||
elif variant == 2:
|
||||
# Wave/organic style
|
||||
return """<svg width="900" height="40" xmlns="http://www.w3.org/2000/svg">
|
||||
<!-- Vintage Technical Manual Style - Wave Variant -->
|
||||
|
||||
<!-- Ghost wave -->
|
||||
<path d="M 50 22 Q 150 12, 250 20 T 450 18 T 650 22 T 850 18"
|
||||
fill="none" stroke="#5c5247" stroke-width="1" opacity="0.2"/>
|
||||
|
||||
<!-- Main wave line -->
|
||||
<path d="M 50 20 Q 150 10, 250 18 T 450 16 T 650 20 T 850 16"
|
||||
fill="none" stroke="#5c5247" stroke-width="1.75" opacity="0.5"/>
|
||||
|
||||
<!-- Circle accents -->
|
||||
<g fill="#5c5247">
|
||||
<circle cx="50" cy="20" r="4" opacity="0.5"/>
|
||||
<circle cx="52" cy="22" r="2" opacity="0.25"/>
|
||||
<circle cx="250" cy="18" r="3" opacity="0.35"/>
|
||||
<circle cx="450" cy="16" r="4" opacity="0.45"/>
|
||||
<circle cx="452" cy="18" r="2.5" fill="#c96442" opacity="0.6"/>
|
||||
<circle cx="650" cy="20" r="3" opacity="0.35"/>
|
||||
<circle cx="850" cy="16" r="4" opacity="0.5"/>
|
||||
<circle cx="848" cy="18" r="2" opacity="0.25"/>
|
||||
</g>
|
||||
|
||||
<!-- Tick marks -->
|
||||
<g stroke="#5c5247" opacity="0.35">
|
||||
<line x1="150" y1="12" x2="150" y2="24" stroke-width="1.25"/>
|
||||
<line x1="350" y1="14" x2="350" y2="22" stroke-width="1.25"/>
|
||||
<line x1="550" y1="14" x2="550" y2="24" stroke-width="1.25"/>
|
||||
<line x1="750" y1="12" x2="750" y2="22" stroke-width="1.25"/>
|
||||
</g>
|
||||
</svg>"""
|
||||
|
||||
else: # variant == 3
|
||||
# Bracket style with layered drafts
|
||||
return """<svg width="900" height="40" xmlns="http://www.w3.org/2000/svg">
|
||||
<!-- Vintage Technical Manual Style - Bracket Variant -->
|
||||
|
||||
<!-- Ghost lines -->
|
||||
<line x1="82" y1="18" x2="818" y2="18" stroke="#5c5247" stroke-width="1" opacity="0.15"/>
|
||||
|
||||
<!-- Main horizontal line -->
|
||||
<line x1="80" y1="20" x2="820" y2="20" stroke="#5c5247" stroke-width="1.75" opacity="0.5"/>
|
||||
|
||||
<!-- Corner brackets - left -->
|
||||
<g fill="none" stroke="#5c5247">
|
||||
<path d="M 50,20 L 50,35 M 50,20 L 80,20" stroke-width="2" opacity="0.5"/>
|
||||
<path d="M 53,18 L 53,33 M 53,18 L 78,18" stroke-width="1" opacity="0.2"/>
|
||||
</g>
|
||||
|
||||
<!-- Corner brackets - right -->
|
||||
<g fill="none" stroke="#5c5247">
|
||||
<path d="M 850,20 L 850,35 M 850,20 L 820,20" stroke-width="2" opacity="0.5"/>
|
||||
<path d="M 847,18 L 847,33 M 847,18 L 822,18" stroke-width="1" opacity="0.2"/>
|
||||
</g>
|
||||
|
||||
<!-- Corner dots -->
|
||||
<g fill="#5c5247">
|
||||
<circle cx="50" cy="20" r="4" opacity="0.45"/>
|
||||
<circle cx="52" cy="22" r="2" opacity="0.2"/>
|
||||
<circle cx="850" cy="20" r="4" opacity="0.45"/>
|
||||
<circle cx="848" cy="22" r="2" opacity="0.2"/>
|
||||
</g>
|
||||
|
||||
<!-- Center accent -->
|
||||
<circle cx="450" cy="20" r="5" fill="none" stroke="#5c5247" stroke-width="1.5" opacity="0.5"/>
|
||||
<circle cx="450" cy="20" r="2.5" fill="#c96442" opacity="0.6"/>
|
||||
|
||||
<!-- Tick marks with doubles -->
|
||||
<g stroke="#5c5247" opacity="0.35">
|
||||
<line x1="180" y1="14" x2="180" y2="26" stroke-width="1.25"/>
|
||||
<line x1="182" y1="15" x2="182" y2="25" stroke-width="0.75" opacity="0.5"/>
|
||||
<line x1="320" y1="15" x2="320" y2="25" stroke-width="1.25"/>
|
||||
<line x1="580" y1="15" x2="580" y2="25" stroke-width="1.25"/>
|
||||
<line x1="720" y1="14" x2="720" y2="26" stroke-width="1.25"/>
|
||||
<line x1="722" y1="15" x2="722" y2="25" stroke-width="0.75" opacity="0.5"/>
|
||||
</g>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def generate_desc_box_light_svg(position="top"):
|
||||
"""Generate a light-mode description box SVG (top or bottom).
|
||||
|
||||
Args:
|
||||
position: "top" or "bottom"
|
||||
"""
|
||||
if position == "top":
|
||||
return """<svg width="900" height="40" xmlns="http://www.w3.org/2000/svg">
|
||||
<!-- Vintage Technical Manual - BOLD layered drafts (top) -->
|
||||
|
||||
<!-- Ghost/draft lines -->
|
||||
<line x1="30" y1="13" x2="870" y2="13" stroke="#5c5247" stroke-width="1.5" opacity="0.15"/>
|
||||
<line x1="26" y1="17" x2="875" y2="17" stroke="#5c5247" stroke-width="1" opacity="0.12"/>
|
||||
|
||||
<!-- Main horizontal line -->
|
||||
<line x1="28" y1="15" x2="872" y2="15" stroke="#5c5247" stroke-width="2" opacity="0.5"/>
|
||||
|
||||
<!-- Secondary lines - partial, offset -->
|
||||
<line x1="45" y1="21" x2="620" y2="21" stroke="#5c5247" stroke-width="1" opacity="0.25"/>
|
||||
<line x1="48" y1="23" x2="580" y2="23" stroke="#5c5247" stroke-width="0.75" opacity="0.15"/>
|
||||
|
||||
<!-- Short accent lines on right -->
|
||||
<line x1="720" y1="10" x2="850" y2="10" stroke="#5c5247" stroke-width="1" opacity="0.22"/>
|
||||
<line x1="740" y1="8" x2="830" y2="8" stroke="#5c5247" stroke-width="0.75" opacity="0.12"/>
|
||||
|
||||
<!-- Bold tick marks -->
|
||||
<g stroke="#5c5247" opacity="0.4">
|
||||
<line x1="95" y1="8" x2="95" y2="26" stroke-width="1.5"/>
|
||||
<line x1="97" y1="9" x2="97" y2="24" stroke-width="1" opacity="0.5"/>
|
||||
<line x1="175" y1="10" x2="175" y2="22" stroke-width="1.5"/>
|
||||
<line x1="270" y1="7" x2="270" y2="27" stroke-width="1.5"/>
|
||||
<line x1="272" y1="9" x2="272" y2="25" stroke-width="1" opacity="0.5"/>
|
||||
<line x1="390" y1="9" x2="390" y2="24" stroke-width="1.5"/>
|
||||
<line x1="530" y1="10" x2="530" y2="23" stroke-width="1.5"/>
|
||||
<line x1="600" y1="7" x2="600" y2="27" stroke-width="1.5"/>
|
||||
<line x1="720" y1="9" x2="720" y2="24" stroke-width="1.5"/>
|
||||
<line x1="820" y1="7" x2="820" y2="27" stroke-width="1.5"/>
|
||||
</g>
|
||||
|
||||
<!-- Bold circles -->
|
||||
<g fill="#5c5247">
|
||||
<circle cx="130" cy="15" r="3" opacity="0.35"/>
|
||||
<circle cx="133" cy="17" r="2" opacity="0.2"/>
|
||||
<circle cx="330" cy="16" r="2.5" opacity="0.3"/>
|
||||
<circle cx="480" cy="15" r="3.5" opacity="0.35"/>
|
||||
<circle cx="560" cy="17" r="2" opacity="0.28"/>
|
||||
<circle cx="660" cy="15" r="3" opacity="0.32"/>
|
||||
<circle cx="790" cy="14" r="2.5" opacity="0.3"/>
|
||||
</g>
|
||||
|
||||
<!-- Corner dots -->
|
||||
<g fill="#5c5247">
|
||||
<circle cx="20" cy="15" r="5" opacity="0.5"/>
|
||||
<circle cx="22" cy="17" r="3" opacity="0.25"/>
|
||||
<circle cx="880" cy="15" r="5" opacity="0.5"/>
|
||||
<circle cx="878" cy="17" r="3" opacity="0.25"/>
|
||||
</g>
|
||||
|
||||
<!-- Corner brackets -->
|
||||
<g fill="none" stroke="#5c5247">
|
||||
<path d="M 6,15 L 6,38 M 6,15 L 28,15" stroke-width="2.5" opacity="0.55"/>
|
||||
<path d="M 9,13 L 9,36 M 9,13 L 30,13" stroke-width="1.5" opacity="0.2"/>
|
||||
<path d="M 894,15 L 894,38 M 894,15 L 872,15" stroke-width="2.5" opacity="0.55"/>
|
||||
<path d="M 891,13 L 891,36 M 891,13 L 870,13" stroke-width="1.5" opacity="0.2"/>
|
||||
</g>
|
||||
</svg>"""
|
||||
else: # bottom
|
||||
return """<svg width="900" height="40" xmlns="http://www.w3.org/2000/svg">
|
||||
<!-- Vintage Technical Manual - BOLD layered drafts (bottom) -->
|
||||
|
||||
<!-- Ghost/draft lines -->
|
||||
<line x1="30" y1="27" x2="870" y2="27" stroke="#5c5247" stroke-width="1.5" opacity="0.15"/>
|
||||
<line x1="26" y1="23" x2="875" y2="23" stroke="#5c5247" stroke-width="1" opacity="0.12"/>
|
||||
|
||||
<!-- Main horizontal line -->
|
||||
<line x1="28" y1="25" x2="872" y2="25" stroke="#5c5247" stroke-width="2" opacity="0.5"/>
|
||||
|
||||
<!-- Secondary lines -->
|
||||
<line x1="280" y1="19" x2="855" y2="19" stroke="#5c5247" stroke-width="1" opacity="0.25"/>
|
||||
<line x1="320" y1="17" x2="852" y2="17" stroke="#5c5247" stroke-width="0.75" opacity="0.15"/>
|
||||
|
||||
<!-- Short accent lines on left -->
|
||||
<line x1="50" y1="30" x2="180" y2="30" stroke="#5c5247" stroke-width="1" opacity="0.22"/>
|
||||
<line x1="70" y1="32" x2="160" y2="32" stroke="#5c5247" stroke-width="0.75" opacity="0.12"/>
|
||||
|
||||
<!-- Bold tick marks -->
|
||||
<g stroke="#5c5247" opacity="0.4">
|
||||
<line x1="80" y1="14" x2="80" y2="32" stroke-width="1.5"/>
|
||||
<line x1="82" y1="16" x2="82" y2="30" stroke-width="1" opacity="0.5"/>
|
||||
<line x1="210" y1="17" x2="210" y2="30" stroke-width="1.5"/>
|
||||
<line x1="370" y1="14" x2="370" y2="32" stroke-width="1.5"/>
|
||||
<line x1="500" y1="16" x2="500" y2="31" stroke-width="1.5"/>
|
||||
<line x1="630" y1="14" x2="630" y2="32" stroke-width="1.5"/>
|
||||
<line x1="632" y1="16" x2="632" y2="30" stroke-width="1" opacity="0.5"/>
|
||||
<line x1="760" y1="16" x2="760" y2="30" stroke-width="1.5"/>
|
||||
<line x1="820" y1="14" x2="820" y2="32" stroke-width="1.5"/>
|
||||
</g>
|
||||
|
||||
<!-- Bold circles -->
|
||||
<g fill="#5c5247">
|
||||
<circle cx="140" cy="25" r="3" opacity="0.35"/>
|
||||
<circle cx="143" cy="23" r="2" opacity="0.2"/>
|
||||
<circle cx="290" cy="24" r="2.5" opacity="0.3"/>
|
||||
<circle cx="440" cy="25" r="3.5" opacity="0.35"/>
|
||||
<circle cx="570" cy="23" r="2" opacity="0.28"/>
|
||||
<circle cx="700" cy="25" r="3" opacity="0.32"/>
|
||||
<circle cx="850" cy="24" r="2.5" opacity="0.3"/>
|
||||
</g>
|
||||
|
||||
<!-- Corner dots -->
|
||||
<g fill="#5c5247">
|
||||
<circle cx="20" cy="25" r="5" opacity="0.5"/>
|
||||
<circle cx="22" cy="23" r="3" opacity="0.25"/>
|
||||
<circle cx="880" cy="25" r="5" opacity="0.5"/>
|
||||
<circle cx="878" cy="23" r="3" opacity="0.25"/>
|
||||
</g>
|
||||
|
||||
<!-- Corner brackets (inverted for bottom) -->
|
||||
<g fill="none" stroke="#5c5247">
|
||||
<path d="M 6,25 L 6,2 M 6,25 L 28,25" stroke-width="2.5" opacity="0.55"/>
|
||||
<path d="M 9,27 L 9,4 M 9,27 L 30,27" stroke-width="1.5" opacity="0.2"/>
|
||||
<path d="M 894,25 L 894,2 M 894,25 L 872,25" stroke-width="2.5" opacity="0.55"/>
|
||||
<path d="M 891,27 L 891,4 M 891,27 L 870,27" stroke-width="1.5" opacity="0.2"/>
|
||||
</g>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def generate_entry_separator_svg():
|
||||
"""Generate a small separator SVG between entries in vintage manual style.
|
||||
|
||||
Uses bolder 'layered drafts' aesthetic with ghost circles for depth.
|
||||
"""
|
||||
return """<svg width="200" height="12" xmlns="http://www.w3.org/2000/svg">
|
||||
<g opacity="0.55">
|
||||
<circle cx="88" cy="6" r="2.5" fill="#c4baa8"/>
|
||||
<circle cx="100" cy="6" r="3.5" fill="#c96442"/>
|
||||
<circle cx="112" cy="6" r="2.5" fill="#c4baa8"/>
|
||||
<!-- Ghost circles for layered effect -->
|
||||
<circle cx="90" cy="7" r="1.5" fill="#c4baa8" opacity="0.4"/>
|
||||
<circle cx="102" cy="7" r="2" fill="#c96442" opacity="0.3"/>
|
||||
<circle cx="110" cy="7" r="1.5" fill="#c4baa8" opacity="0.4"/>
|
||||
</g>
|
||||
</svg>"""
|
||||
@@ -0,0 +1,226 @@
|
||||
"""SVG renderers for section headers."""
|
||||
|
||||
|
||||
def render_h2_svg(text: str, icon: str = "") -> str:
|
||||
"""Create an animated hero-centered H2 header SVG string.
|
||||
|
||||
Args:
|
||||
text: The header text (e.g., "Agent Skills")
|
||||
icon: Optional icon to append (e.g., an emoji)
|
||||
"""
|
||||
# Build display text with optional icon
|
||||
display_text = f"{text} {icon}" if icon else text
|
||||
|
||||
# Escape XML special characters
|
||||
text_escaped = display_text.replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
|
||||
# Calculate viewBox bounds based on text length
|
||||
# Text is centered at x=400, font-size 38px ~ 22px per char, emoji ~ 50px
|
||||
text_width = len(text) * 22 + (50 if icon else 0)
|
||||
half_text = text_width / 2
|
||||
# Ensure we include decorations (x=187 to x=613) plus text bounds with generous padding
|
||||
left_bound = int(min(180, 400 - half_text - 30))
|
||||
right_bound = int(max(620, 400 + half_text + 30))
|
||||
viewbox_width = right_bound - left_bound
|
||||
|
||||
return f"""<svg width="100%" height="100" viewBox="{left_bound} 0 {viewbox_width} 100" xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<!-- Subtle glow for hero text - reduced blur for better readability -->
|
||||
<filter id="heroGlow" x="-10%" y="-10%" width="120%" height="120%">
|
||||
<feGaussianBlur stdDeviation="1" result="coloredBlur"/>
|
||||
<feMerge>
|
||||
<feMergeNode in="coloredBlur"/>
|
||||
<feMergeNode in="SourceGraphic"/>
|
||||
</feMerge>
|
||||
</filter>
|
||||
|
||||
<!-- Hero gradient - brighter, more saturated colors for contrast -->
|
||||
<linearGradient id="heroGrad" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" stop-color="#FF8855" stop-opacity="1">
|
||||
<animate attributeName="stop-color" values="#FF8855;#FFAA77;#FF8855" dur="5s" repeatCount="indefinite"/>
|
||||
</stop>
|
||||
<stop offset="50%" stop-color="#FFAA77" stop-opacity="1"/>
|
||||
<stop offset="100%" stop-color="#FF8855" stop-opacity="1">
|
||||
<animate attributeName="stop-color" values="#FF8855;#FFCC99;#FF8855" dur="5s" repeatCount="indefinite"/>
|
||||
</stop>
|
||||
</linearGradient>
|
||||
|
||||
<!-- Accent line gradient -->
|
||||
<linearGradient id="accentLine" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" stop-color="#FFB088" stop-opacity="0"/>
|
||||
<stop offset="50%" stop-color="#FF8855" stop-opacity="1">
|
||||
<animate attributeName="stop-opacity" values="0.8;1;0.8" dur="3s" repeatCount="indefinite"/>
|
||||
</stop>
|
||||
<stop offset="100%" stop-color="#FFB088" stop-opacity="0"/>
|
||||
</linearGradient>
|
||||
|
||||
<!-- Radial glow background - more subtle -->
|
||||
<radialGradient id="bgGlow">
|
||||
<stop offset="0%" stop-color="#FF8C5A" stop-opacity="0.08">
|
||||
<animate attributeName="stop-opacity" values="0.05;0.12;0.05" dur="4s" repeatCount="indefinite"/>
|
||||
</stop>
|
||||
<stop offset="100%" stop-color="#FF8C5A" stop-opacity="0"/>
|
||||
</radialGradient>
|
||||
</defs>
|
||||
|
||||
<!-- Background glow - more subtle -->
|
||||
<ellipse cx="400" cy="50" rx="300" ry="40" fill="url(#bgGlow)"/>
|
||||
|
||||
<!-- Top accent line -->
|
||||
<line x1="200" y1="20" x2="600" y2="20" stroke="url(#accentLine)" stroke-width="2" stroke-linecap="round">
|
||||
<animate attributeName="stroke-width" values="2;2.5;2" dur="3s" repeatCount="indefinite"/>
|
||||
</line>
|
||||
|
||||
<!-- Main hero text - larger, bolder, with subtle dark outline for contrast -->
|
||||
<text x="400" y="58" font-family="system-ui, -apple-system, sans-serif" font-size="38" font-weight="900" fill="url(#heroGrad)" text-anchor="middle" filter="url(#heroGlow)" letter-spacing="0.5" stroke="#221111" stroke-width="0.5" paint-order="stroke fill">
|
||||
{text_escaped}
|
||||
</text>
|
||||
|
||||
<!-- Bottom accent line -->
|
||||
<line x1="200" y1="80" x2="600" y2="80" stroke="url(#accentLine)" stroke-width="2" stroke-linecap="round">
|
||||
<animate attributeName="stroke-width" values="2;2.5;2" dur="3s" begin="1.5s" repeatCount="indefinite"/>
|
||||
</line>
|
||||
|
||||
<!-- Decorative corner elements -->
|
||||
<g opacity="0.6">
|
||||
<!-- Top left -->
|
||||
<path d="M 195,16 L 195,24 M 195,20 L 187,20" stroke="#FF8855" stroke-width="2" stroke-linecap="round">
|
||||
<animate attributeName="opacity" values="0.5;0.9;0.5" dur="3s" repeatCount="indefinite"/>
|
||||
</path>
|
||||
<!-- Top right -->
|
||||
<path d="M 605,16 L 605,24 M 605,20 L 613,20" stroke="#FF8855" stroke-width="2" stroke-linecap="round">
|
||||
<animate attributeName="opacity" values="0.5;0.9;0.5" dur="3s" begin="0.5s" repeatCount="indefinite"/>
|
||||
</path>
|
||||
<!-- Bottom left -->
|
||||
<path d="M 195,76 L 195,84 M 195,80 L 187,80" stroke="#FFAA77" stroke-width="2" stroke-linecap="round">
|
||||
<animate attributeName="opacity" values="0.5;0.9;0.5" dur="3s" begin="1s" repeatCount="indefinite"/>
|
||||
</path>
|
||||
<!-- Bottom right -->
|
||||
<path d="M 605,76 L 605,84 M 605,80 L 613,80" stroke="#FFAA77" stroke-width="2" stroke-linecap="round">
|
||||
<animate attributeName="opacity" values="0.5;0.9;0.5" dur="3s" begin="1.5s" repeatCount="indefinite"/>
|
||||
</path>
|
||||
</g>
|
||||
|
||||
<!-- Floating accent particles - reduced opacity -->
|
||||
<g opacity="0.35">
|
||||
<circle cx="250" cy="35" r="2" fill="#FFCBA4">
|
||||
<animate attributeName="cy" values="35;30;35" dur="4s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0;0.5;0" dur="4s" repeatCount="indefinite"/>
|
||||
</circle>
|
||||
<circle cx="550" cy="45" r="2.5" fill="#FFB088">
|
||||
<animate attributeName="cy" values="45;40;45" dur="4.5s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0;0.6;0" dur="4.5s" repeatCount="indefinite"/>
|
||||
</circle>
|
||||
<circle cx="320" cy="68" r="1.5" fill="#FF9B70">
|
||||
<animate attributeName="cy" values="68;63;68" dur="3.5s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0;0.4;0" dur="3.5s" repeatCount="indefinite"/>
|
||||
</circle>
|
||||
</g>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def render_h3_svg(text: str) -> str:
|
||||
"""Create an animated minimal-inline H3 header SVG string."""
|
||||
# Escape XML special characters
|
||||
text_escaped = text.replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
|
||||
# Calculate approximate text width (rough estimate: 10px per character for 18px font)
|
||||
text_width = len(text) * 10
|
||||
total_width = text_width + 50 # Add padding for decorative elements
|
||||
|
||||
return f"""<svg width="100%" height="36" viewBox="0 0 {total_width} 36" xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<!-- Very subtle glow -->
|
||||
<filter id="minimalGlow">
|
||||
<feGaussianBlur stdDeviation="1" result="coloredBlur"/>
|
||||
<feMerge>
|
||||
<feMergeNode in="coloredBlur"/>
|
||||
<feMergeNode in="SourceGraphic"/>
|
||||
</feMerge>
|
||||
</filter>
|
||||
|
||||
<!-- Simple gradient -->
|
||||
<linearGradient id="minimalGrad" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" stop-color="#FF6B35" stop-opacity="1"/>
|
||||
<stop offset="100%" stop-color="#8B5A3C" stop-opacity="1"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<!-- Left decorative element -->
|
||||
<g>
|
||||
<line x1="0" y1="18" x2="12" y2="18" stroke="#FF6B35" stroke-width="3" stroke-linecap="round" opacity="0.8">
|
||||
<animate attributeName="x2" values="12;16;12" dur="3s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0.7;1;0.7" dur="3s" repeatCount="indefinite"/>
|
||||
</line>
|
||||
<circle cx="18" cy="18" r="2" fill="#FF8C5A" opacity="0.7">
|
||||
<animate attributeName="r" values="2;2.5;2" dur="3s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0.6;0.9;0.6" dur="3s" repeatCount="indefinite"/>
|
||||
</circle>
|
||||
</g>
|
||||
|
||||
<!-- Header text -->
|
||||
<text x="30" y="24" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="600" fill="url(#minimalGrad)" filter="url(#minimalGlow)">
|
||||
{text_escaped}
|
||||
<animate attributeName="opacity" values="0.93;1;0.93" dur="4s" repeatCount="indefinite"/>
|
||||
</text>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def generate_category_header_light_svg(title, section_number="01"):
|
||||
"""Generate a light-mode category header SVG in vintage technical manual style.
|
||||
|
||||
Args:
|
||||
title: The category title (e.g., "Agent Skills", "Tooling")
|
||||
section_number: Two-digit section number (e.g., "01", "02")
|
||||
"""
|
||||
# Escape XML special characters
|
||||
title_escaped = title.replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
|
||||
# Calculate text width for positioning
|
||||
title_width = len(title) * 14 # Approximate width per character
|
||||
line_end_x = max(640, 220 + title_width + 50)
|
||||
|
||||
return f"""<svg width="100%" height="80" viewBox="150 0 500 80" xmlns="http://www.w3.org/2000/svg">
|
||||
<!--
|
||||
Vintage Technical Manual Style - Header (Auto-generated)
|
||||
Clean, authoritative, reference manual aesthetic
|
||||
-->
|
||||
|
||||
<!-- Section number box -->
|
||||
<g>
|
||||
<rect x="160" y="22" width="36" height="36" fill="none" stroke="#5c5247" stroke-width="2" opacity="0.6"/>
|
||||
<text x="178" y="48"
|
||||
font-family="'Courier New', Courier, monospace"
|
||||
font-size="20"
|
||||
font-weight="700"
|
||||
fill="#c96442"
|
||||
text-anchor="middle">
|
||||
{section_number}
|
||||
</text>
|
||||
</g>
|
||||
|
||||
<!-- Main title -->
|
||||
<text x="220" y="47"
|
||||
font-family="system-ui, -apple-system, 'Helvetica Neue', sans-serif"
|
||||
font-size="28"
|
||||
font-weight="600"
|
||||
fill="#3d3530"
|
||||
letter-spacing="0.5">
|
||||
{title_escaped}
|
||||
</text>
|
||||
|
||||
<!-- Horizontal rule extending from title -->
|
||||
<line x1="220" y1="58" x2="{line_end_x}" y2="58" stroke="#5c5247" stroke-width="1.75" opacity="0.45"/>
|
||||
|
||||
<!-- Reference dots pattern (like page markers) -->
|
||||
<g fill="#5c5247" opacity="0.3">
|
||||
<circle cx="{line_end_x - 60}" cy="35" r="1"/>
|
||||
<circle cx="{line_end_x - 45}" cy="35" r="1"/>
|
||||
<circle cx="{line_end_x - 30}" cy="35" r="1"/>
|
||||
<circle cx="{line_end_x - 15}" cy="35" r="1"/>
|
||||
<circle cx="{line_end_x}" cy="35" r="1"/>
|
||||
</g>
|
||||
|
||||
<!-- Thin top line -->
|
||||
<line x1="160" y1="15" x2="{line_end_x}" y2="15" stroke="#5c5247" stroke-width="1.75" opacity="0.45"/>
|
||||
</svg>"""
|
||||
@@ -0,0 +1,259 @@
|
||||
"""SVG renderers for table-of-contents elements."""
|
||||
|
||||
import re
|
||||
|
||||
|
||||
def generate_toc_row_svg(directory_name, description):
|
||||
"""Generate a dark-mode TOC row SVG in CRT terminal style.
|
||||
|
||||
Args:
|
||||
directory_name: The directory name (e.g., "agent-skills/")
|
||||
description: Short description for the comment
|
||||
"""
|
||||
# Escape XML entities
|
||||
desc_escaped = description.replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
dir_escaped = directory_name.replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
|
||||
return f"""<svg width="400" height="40" viewBox="0 0 400 40" xmlns="http://www.w3.org/2000/svg" preserveAspectRatio="xMinYMid meet">
|
||||
<defs>
|
||||
<filter id="crtGlow">
|
||||
<feGaussianBlur stdDeviation="0.2" result="coloredBlur"/>
|
||||
<feMerge>
|
||||
<feMergeNode in="coloredBlur"/>
|
||||
<feMergeNode in="SourceGraphic"/>
|
||||
</feMerge>
|
||||
</filter>
|
||||
|
||||
<pattern id="scanlines" x="0" y="0" width="100%" height="4" patternUnits="userSpaceOnUse">
|
||||
<rect x="0" y="0" width="100%" height="2" fill="#000000" opacity="0.25"/>
|
||||
</pattern>
|
||||
|
||||
<linearGradient id="phosphor" x1="0%" y1="0%" x2="0%" y2="100%">
|
||||
<stop offset="0%" style="stop-color:#0f380f;stop-opacity:1"/>
|
||||
<stop offset="100%" style="stop-color:#0a2f0a;stop-opacity:1"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<!-- Background -->
|
||||
<rect width="400" height="40" fill="#1a1a1a"/>
|
||||
<rect x="7" y="0" width="393" height="40" fill="url(#phosphor)"/>
|
||||
<rect x="7" y="0" width="393" height="40" fill="url(#scanlines)"/>
|
||||
|
||||
<!-- Hover highlight -->
|
||||
<rect x="7" y="0" width="393" height="40" fill="#33ff33" opacity="0">
|
||||
<animate attributeName="opacity" values="0;0.05;0" dur="2s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
|
||||
<!-- Content -->
|
||||
<g filter="url(#crtGlow)">
|
||||
<text x="20" y="25" font-family="monospace" font-size="16" fill="#66ff66">
|
||||
drwxr-xr-x
|
||||
</text>
|
||||
<text x="140" y="25" font-family="monospace" font-size="16" fill="#33ff33" font-weight="bold">
|
||||
{dir_escaped}
|
||||
<animate attributeName="opacity" values="1;0.95;1" dur="0.1s" repeatCount="indefinite"/>
|
||||
</text>
|
||||
<!--
|
||||
<text x="400" y="25" font-family="monospace" font-size="14" fill="#449944" opacity="1">
|
||||
# {desc_escaped}
|
||||
</text>
|
||||
-->
|
||||
</g>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def generate_toc_row_light_svg(directory_name, description):
|
||||
"""Generate a light-mode TOC row SVG in vintage manual style."""
|
||||
_ = description # Reserved for future use
|
||||
dir_escaped = directory_name.replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
|
||||
return f"""<svg width="400" height="40" viewBox="0 0 400 40" xmlns="http://www.w3.org/2000/svg" preserveAspectRatio="xMinYMid meet">
|
||||
<defs>
|
||||
<linearGradient id="paperBg" x1="0%" y1="0%" x2="0%" y2="100%">
|
||||
<stop offset="0%" style="stop-color:#faf8f3"/>
|
||||
<stop offset="100%" style="stop-color:#f5f0e6"/>
|
||||
</linearGradient>
|
||||
<pattern id="leaderDots" x="0" y="0" width="10" height="4" patternUnits="userSpaceOnUse">
|
||||
<circle cx="2" cy="2" r="0.8" fill="#8a7b6f" opacity="0.5"/>
|
||||
</pattern>
|
||||
</defs>
|
||||
|
||||
<!-- Background -->
|
||||
<rect width="400" height="36" fill="url(#paperBg)"/>
|
||||
<line x1="2" y1="0" x2="2" y2="36" stroke="#c4baa8" stroke-width="1"/>
|
||||
<line x1="398" y1="0" x2="398" y2="36" stroke="#c4baa8" stroke-width="1"/>
|
||||
|
||||
<!-- Section number -->
|
||||
<text x="32" y="24"
|
||||
font-family="'Courier New', Courier, monospace"
|
||||
font-size="14"
|
||||
font-weight="700"
|
||||
fill="#c96442"
|
||||
text-anchor="middle">
|
||||
01
|
||||
</text>
|
||||
|
||||
<!-- Section title -->
|
||||
<text x="120" y="24"
|
||||
font-family="Georgia, 'Times New Roman', serif"
|
||||
font-size="14"
|
||||
fill="#3d3530">
|
||||
{dir_escaped}
|
||||
</text>
|
||||
|
||||
<!-- Leader dots -->
|
||||
<rect x="210" y="20" width="140" height="4" fill="url(#leaderDots)"/>
|
||||
|
||||
<!-- Page/section reference -->
|
||||
<text x="370" y="24"
|
||||
font-family="'Courier New', Courier, monospace"
|
||||
font-size="12"
|
||||
fill="#5c5247"
|
||||
text-anchor="end"
|
||||
opacity="0.7">
|
||||
§1
|
||||
</text>
|
||||
|
||||
<!-- Bottom rule -->
|
||||
<line x1="20" y1="34" x2="380" y2="34" stroke="#c4baa8" stroke-width="0.5" opacity="0.3"/>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def generate_toc_header_light_svg():
|
||||
"""Generate a compact light-mode TOC header with fixed width and centered title."""
|
||||
return """<svg width="400" height="48" viewBox="0 0 400 48" xmlns="http://www.w3.org/2000/svg" preserveAspectRatio="xMinYMid meet">
|
||||
<defs>
|
||||
<linearGradient id="tocHeaderBg" x1="0%" y1="0%" x2="0%" y2="100%">
|
||||
<stop offset="0%" style="stop-color:#faf8f3"/>
|
||||
<stop offset="100%" style="stop-color:#f3eee4"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<rect x="0.5" y="0.5" width="399" height="47" rx="3" ry="3" fill="url(#tocHeaderBg)" stroke="#c4baa8" stroke-width="1"/>
|
||||
|
||||
<!-- Center title -->
|
||||
<text x="200" y="28"
|
||||
font-family="Georgia, 'Times New Roman', serif"
|
||||
font-size="17"
|
||||
font-weight="600"
|
||||
fill="#3d3530"
|
||||
text-anchor="middle"
|
||||
letter-spacing="2">
|
||||
CONTENTS
|
||||
</text>
|
||||
|
||||
<!-- Decorative diamonds -->
|
||||
<g fill="#5c5247" opacity="0.65">
|
||||
<path d="M 118 24 L 124 18 L 130 24 L 124 30 Z"/>
|
||||
<path d="M 282 24 L 288 18 L 294 24 L 288 30 Z"/>
|
||||
</g>
|
||||
|
||||
<!-- Light scan indicator -->
|
||||
<rect x="-40" y="2" width="3" height="44" fill="#d2c5b4" opacity="0.16">
|
||||
<animate attributeName="x" values="-40;420;420;-40" keyTimes="0;0.28;0.98;1" dur="7s" repeatCount="indefinite" />
|
||||
</rect>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def generate_toc_sub_svg(directory_name, description):
|
||||
"""Generate a dark-mode TOC subcategory row SVG.
|
||||
|
||||
Args:
|
||||
directory_name: The subdirectory name (e.g., "general/")
|
||||
description: Short description for the comment
|
||||
"""
|
||||
_ = description # Reserved for future use
|
||||
dir_escaped = directory_name.replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
|
||||
return f"""<svg height="40" width="400" viewBox="0 0 400 40" xmlns="http://www.w3.org/2000/svg" preserveAspectRatio="xMinYMid meet">
|
||||
<defs>
|
||||
<filter id="crtGlow">
|
||||
<feGaussianBlur stdDeviation="0.5" result="coloredBlur"/>
|
||||
<feMerge>
|
||||
<feMergeNode in="coloredBlur"/>
|
||||
<feMergeNode in="SourceGraphic"/>
|
||||
</feMerge>
|
||||
</filter>
|
||||
|
||||
<pattern id="scanlines" x="0" y="0" width="100%" height="4" patternUnits="userSpaceOnUse">
|
||||
<rect x="0" y="0" width="100%" height="2" fill="#000000" opacity="0.25"/>
|
||||
</pattern>
|
||||
|
||||
<linearGradient id="phosphor" x1="0%" y1="0%" x2="0%" y2="100%">
|
||||
<stop offset="0%" style="stop-color:#0f380f;stop-opacity:1"/>
|
||||
<stop offset="100%" style="stop-color:#0a2f0a;stop-opacity:1"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<!-- Background -->
|
||||
<rect width="400" height="40" fill="#1a1a1a"/>
|
||||
<rect x="7" y="0" width="393" height="40" fill="url(#phosphor)"/>
|
||||
<rect x="7" y="0" width="393" height="40" fill="url(#scanlines)"/>
|
||||
|
||||
<!-- Content -->
|
||||
<g filter="url(#crtGlow)">
|
||||
<text x="18" y="25" font-family="monospace" font-size="12" fill="#66ff66" opacity="0.8">
|
||||
|-
|
||||
</text>
|
||||
<text x="56" y="25" font-family="monospace" font-size="13" fill="#33ff33">
|
||||
{dir_escaped}
|
||||
</text>
|
||||
</g>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def generate_toc_sub_light_svg(directory_name, description):
|
||||
"""Generate a light-mode TOC subcategory row SVG."""
|
||||
_ = description # Reserved for future use
|
||||
dir_escaped = directory_name.replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
|
||||
return f"""<svg width="400" height="40" viewBox="0 0 400 40" xmlns="http://www.w3.org/2000/svg" preserveAspectRatio="xMinYMid meet">
|
||||
<defs>
|
||||
<linearGradient id="paperBgSub" x1="0%" y1="0%" x2="0%" y2="100%">
|
||||
<stop offset="0%" style="stop-color:#fbfaf6"/>
|
||||
<stop offset="100%" style="stop-color:#f4efe5"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<rect width="400" height="36" fill="url(#paperBgSub)"/>
|
||||
<line x1="2" y1="0" x2="2" y2="36" stroke="#c4baa8" stroke-width="1"/>
|
||||
<line x1="398" y1="0" x2="398" y2="36" stroke="#c4baa8" stroke-width="1"/>
|
||||
|
||||
<text x="22" y="24"
|
||||
font-family="'Courier New', Courier, monospace"
|
||||
font-size="12"
|
||||
fill="#c96442"
|
||||
opacity="0.8">
|
||||
|-
|
||||
</text>
|
||||
<text x="60" y="24"
|
||||
font-family="Georgia, 'Times New Roman', serif"
|
||||
font-size="13"
|
||||
fill="#3d3530">
|
||||
{dir_escaped}
|
||||
</text>
|
||||
|
||||
<line x1="20" y1="33" x2="380" y2="33" stroke="#c4baa8" stroke-width="0.5" opacity="0.3"/>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def _normalize_svg_root(tag: str, target_width: int, target_height: int) -> str:
|
||||
"""Ensure root SVG tag enforces target width/height, viewBox, and left anchoring."""
|
||||
|
||||
def ensure_attr(svg_tag: str, name: str, value: str) -> str:
|
||||
if re.search(rf'{name}="[^"]*"', svg_tag):
|
||||
return re.sub(rf'{name}="[^"]*"', f'{name}="{value}"', svg_tag)
|
||||
# Insert before closing ">"
|
||||
return svg_tag.rstrip(">") + f' {name}="{value}">'
|
||||
|
||||
# Force consistent width/height
|
||||
svg_tag = ensure_attr(tag, "width", str(target_width))
|
||||
svg_tag = ensure_attr(svg_tag, "height", str(target_height))
|
||||
|
||||
# Ensure preserveAspectRatio anchors left and keeps aspect
|
||||
svg_tag = ensure_attr(svg_tag, "preserveAspectRatio", "xMinYMid meet")
|
||||
|
||||
# Enforce viewBox to match target dimensions
|
||||
svg_tag = ensure_attr(svg_tag, "viewBox", f"0 0 {target_width} {target_height}")
|
||||
|
||||
return svg_tag
|
||||
@@ -0,0 +1,320 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Create a pull request with a new resource addition.
|
||||
This script is called by the GitHub Action after approval.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import contextlib
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from scripts.ids.resource_id import generate_resource_id
|
||||
from scripts.readme.generate_readme import main as generate_readmes
|
||||
from scripts.resources.resource_utils import append_to_csv, generate_pr_content
|
||||
from scripts.validation.validate_links import (
|
||||
get_github_commit_dates_from_url,
|
||||
get_latest_release_info,
|
||||
)
|
||||
|
||||
|
||||
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
|
||||
"""Run a command and return the result."""
|
||||
return subprocess.run(cmd, capture_output=True, text=True, check=check)
|
||||
|
||||
|
||||
def create_unique_branch_name(base_name: str) -> str:
|
||||
"""Create a unique branch name with timestamp."""
|
||||
timestamp = datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||
return f"{base_name}-{timestamp}"
|
||||
|
||||
|
||||
def get_badge_filename(display_name: str) -> str:
|
||||
"""Compute the badge filename for a resource.
|
||||
|
||||
Uses the same logic as save_resource_badge_svg in generate_readme.py.
|
||||
"""
|
||||
safe_name = re.sub(r"[^a-zA-Z0-9]", "-", display_name.lower())
|
||||
safe_name = re.sub(r"-+", "-", safe_name).strip("-")
|
||||
return f"badge-{safe_name}.svg"
|
||||
|
||||
|
||||
def validate_generated_outputs(status_stdout: str, repo_root: str) -> None:
|
||||
"""Verify expected outputs exist and no unexpected files are changed."""
|
||||
expected_readme = os.path.join(repo_root, "README.md")
|
||||
expected_csv = os.path.join(repo_root, "THE_RESOURCES_TABLE.csv")
|
||||
expected_readme_dir = os.path.join(repo_root, "README_ALTERNATIVES")
|
||||
|
||||
if not os.path.isfile(expected_readme):
|
||||
raise Exception(f"Missing generated README: {expected_readme}")
|
||||
if not os.path.isfile(expected_csv):
|
||||
raise Exception(f"Missing CSV: {expected_csv}")
|
||||
if not os.path.isdir(expected_readme_dir):
|
||||
raise Exception(f"Missing README directory: {expected_readme_dir}")
|
||||
if not glob.glob(os.path.join(expected_readme_dir, "*.md")):
|
||||
raise Exception(f"No README alternatives found in {expected_readme_dir}")
|
||||
|
||||
changed_paths = []
|
||||
for line in status_stdout.splitlines():
|
||||
if not line.strip():
|
||||
continue
|
||||
path = line[3:]
|
||||
if " -> " in path:
|
||||
path = path.split(" -> ", 1)[1]
|
||||
changed_paths.append(path)
|
||||
|
||||
allowed_files = {"README.md", "THE_RESOURCES_TABLE.csv"}
|
||||
allowed_prefixes = ("README_ALTERNATIVES/", "assets/")
|
||||
ignored_files = {"resource_data.json", "pr_result.json"}
|
||||
unexpected = [
|
||||
path
|
||||
for path in changed_paths
|
||||
if path not in ignored_files
|
||||
and path not in allowed_files
|
||||
and not path.startswith(allowed_prefixes)
|
||||
]
|
||||
if unexpected:
|
||||
raise Exception(f"Unexpected changes outside generated outputs: {', '.join(unexpected)}")
|
||||
|
||||
|
||||
def write_step_outputs(outputs: dict[str, str]) -> None:
|
||||
"""Write outputs for GitHub Actions, if available."""
|
||||
output_path = os.environ.get("GITHUB_OUTPUT")
|
||||
if not output_path:
|
||||
return
|
||||
|
||||
try:
|
||||
with open(output_path, "a", encoding="utf-8") as f:
|
||||
for key, value in outputs.items():
|
||||
if value is None:
|
||||
value = ""
|
||||
value_str = str(value)
|
||||
if "\n" in value_str or "\r" in value_str:
|
||||
f.write(f"{key}<<EOF\n{value_str}\nEOF\n")
|
||||
else:
|
||||
f.write(f"{key}={value_str}\n")
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to write step outputs: {e}", file=sys.stderr)
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point."""
|
||||
parser = argparse.ArgumentParser(description="Create PR from approved resource submission")
|
||||
parser.add_argument("--issue-number", required=True, help="Issue number")
|
||||
parser.add_argument("--resource-data", required=True, help="Path to resource data JSON file")
|
||||
args = parser.parse_args()
|
||||
|
||||
# Load resource data
|
||||
with open(args.resource_data) as f:
|
||||
resource_data = json.load(f)
|
||||
|
||||
# If the validation returned a structure with 'data' field, extract it
|
||||
if isinstance(resource_data, dict) and "data" in resource_data:
|
||||
resource_data = resource_data["data"]
|
||||
|
||||
# Generate resource ID
|
||||
resource_id = generate_resource_id(
|
||||
resource_data["display_name"], resource_data["primary_link"], resource_data["category"]
|
||||
)
|
||||
|
||||
# Fetch commit dates from GitHub if applicable
|
||||
primary_link = resource_data["primary_link"]
|
||||
first_commit_date, last_commit_date = get_github_commit_dates_from_url(primary_link)
|
||||
|
||||
if last_commit_date:
|
||||
print(f"Fetched Last Modified date from GitHub: {last_commit_date}", file=sys.stderr)
|
||||
else:
|
||||
print(
|
||||
"Could not fetch Last Modified date from GitHub (non-GitHub URL or API error)",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
if first_commit_date:
|
||||
print(f"Fetched First Commit date from GitHub: {first_commit_date}", file=sys.stderr)
|
||||
|
||||
# Fetch latest release info
|
||||
release_date, release_version, release_source = get_latest_release_info(
|
||||
primary_link, resource_data["display_name"]
|
||||
)
|
||||
if release_date:
|
||||
print(
|
||||
f"Fetched release info: {release_version} from {release_source} ({release_date})",
|
||||
file=sys.stderr,
|
||||
)
|
||||
else:
|
||||
print("No release info found (no GitHub releases or non-GitHub URL)", file=sys.stderr)
|
||||
|
||||
# Prepare the complete resource data
|
||||
resource = {
|
||||
"id": resource_id,
|
||||
"display_name": resource_data["display_name"],
|
||||
"category": resource_data["category"],
|
||||
"subcategory": resource_data.get("subcategory", ""),
|
||||
"primary_link": primary_link,
|
||||
"secondary_link": resource_data.get("secondary_link", ""),
|
||||
"author_name": resource_data["author_name"],
|
||||
"author_link": resource_data["author_link"],
|
||||
"license": resource_data.get("license", "NOT_FOUND"),
|
||||
"description": resource_data["description"],
|
||||
"last_modified": last_commit_date or "", # Set from GitHub API
|
||||
"repo_created": first_commit_date or "", # First commit date from GitHub API
|
||||
"latest_release": release_date or "", # Latest release date
|
||||
"release_version": release_version or "", # Release version (e.g., v1.2.3)
|
||||
"release_source": release_source or "", # Release source (npm, pypi, github-releases)
|
||||
}
|
||||
|
||||
# Create branch name based on category and display name
|
||||
safe_name = resource_data["display_name"].lower()
|
||||
safe_name = "".join(c if c.isalnum() or c in "-_" else "-" for c in safe_name)
|
||||
safe_name = safe_name.strip("-")[:50] # Limit length
|
||||
|
||||
branch_base = f"add-resource/{resource_data['category'].lower().replace(' ', '-')}/{safe_name}"
|
||||
branch_name = create_unique_branch_name(branch_base)
|
||||
|
||||
try:
|
||||
# Ensure we're on main and up to date
|
||||
run_command(["git", "checkout", "main"])
|
||||
run_command(["git", "pull", "origin", "main"])
|
||||
|
||||
# Create new branch
|
||||
try:
|
||||
run_command(["git", "checkout", "-b", branch_name])
|
||||
except subprocess.CalledProcessError as e:
|
||||
# Branch might already exist, try checking it out
|
||||
print(f"Failed to create branch, trying to checkout: {e}", file=sys.stderr)
|
||||
run_command(["git", "checkout", branch_name])
|
||||
|
||||
# Add resource to CSV
|
||||
if not append_to_csv(resource):
|
||||
raise Exception("Failed to add resource to CSV")
|
||||
|
||||
# Sort the CSV
|
||||
print("Sorting CSV after adding resource", file=sys.stderr)
|
||||
sort_result = run_command(
|
||||
["python3", "-m", "scripts.resources.sort_resources"], check=False
|
||||
)
|
||||
if sort_result.returncode != 0:
|
||||
print(f"Warning: CSV sorting failed: {sort_result.stderr}", file=sys.stderr)
|
||||
else:
|
||||
print("CSV sorted successfully", file=sys.stderr)
|
||||
|
||||
# Generate all README variants
|
||||
print("Generating README files...", file=sys.stderr)
|
||||
try:
|
||||
with contextlib.redirect_stdout(sys.stderr):
|
||||
generate_readmes()
|
||||
print("README generation completed successfully", file=sys.stderr)
|
||||
except Exception as e:
|
||||
print(f"ERROR generating README: {e}", file=sys.stderr)
|
||||
raise
|
||||
|
||||
# Check if README was modified
|
||||
status_result = run_command(["git", "status", "--porcelain"])
|
||||
print(f"Git status after README generation:\n{status_result.stdout}", file=sys.stderr)
|
||||
|
||||
repo_root = str(REPO_ROOT)
|
||||
validate_generated_outputs(status_result.stdout, repo_root)
|
||||
|
||||
# Compute badge path and check if it was generated
|
||||
badge_filename = get_badge_filename(resource_data["display_name"])
|
||||
badge_path = os.path.join(repo_root, "assets", badge_filename)
|
||||
badge_warning = ""
|
||||
|
||||
# Stage changes for generated outputs (README variants + badges)
|
||||
files_to_stage = ["THE_RESOURCES_TABLE.csv", "README.md", "README_ALTERNATIVES", "assets"]
|
||||
|
||||
if os.path.exists(badge_path):
|
||||
print(f"Badge file found: {badge_filename}", file=sys.stderr)
|
||||
else:
|
||||
print(f"Warning: Badge file not generated: {badge_path}", file=sys.stderr)
|
||||
badge_warning = (
|
||||
f"\n\n> **Warning**: Badge SVG (`assets/{badge_filename}`) was not generated. "
|
||||
"Manual attention may be required."
|
||||
)
|
||||
|
||||
run_command(["git", "add", "-A", "--", *files_to_stage])
|
||||
|
||||
# Commit
|
||||
commit_message = f"Add resource: {resource_data['display_name']}\n\n"
|
||||
commit_message += f"Category: {resource_data['category']}\n"
|
||||
if resource_data.get("subcategory"):
|
||||
commit_message += f"Sub-category: {resource_data['subcategory']}\n"
|
||||
commit_message += f"Author: {resource_data['author_name']}\n"
|
||||
commit_message += f"From issue: #{args.issue_number}"
|
||||
|
||||
run_command(["git", "commit", "-m", commit_message])
|
||||
|
||||
# Push branch
|
||||
run_command(["git", "push", "origin", branch_name])
|
||||
|
||||
# Create PR
|
||||
pr_title = f"Add resource: {resource_data['display_name']}"
|
||||
pr_body = generate_pr_content(resource)
|
||||
pr_body += badge_warning # Empty string if badge was generated successfully
|
||||
pr_body += f"\n\n---\n\nResolves #{args.issue_number}"
|
||||
|
||||
# Use gh CLI to create PR
|
||||
result = run_command(
|
||||
[
|
||||
"gh",
|
||||
"pr",
|
||||
"create",
|
||||
"--title",
|
||||
pr_title,
|
||||
"--body",
|
||||
pr_body,
|
||||
"--base",
|
||||
"main",
|
||||
"--head",
|
||||
branch_name,
|
||||
]
|
||||
)
|
||||
|
||||
# Extract PR URL from output
|
||||
pr_url = result.stdout.strip()
|
||||
|
||||
# Output result
|
||||
result = {
|
||||
"success": True,
|
||||
"pr_url": pr_url,
|
||||
"branch_name": branch_name,
|
||||
"resource_id": resource_id,
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error in create_resource_pr: {e}", file=sys.stderr)
|
||||
import traceback
|
||||
|
||||
traceback.print_exc(file=sys.stderr)
|
||||
result = {
|
||||
"success": False,
|
||||
"error": str(e),
|
||||
"branch_name": branch_name if "branch_name" in locals() else None,
|
||||
}
|
||||
|
||||
write_step_outputs(
|
||||
{
|
||||
"success": "true" if result["success"] else "false",
|
||||
"pr_url": result.get("pr_url") or "",
|
||||
"branch_name": result.get("branch_name") or "",
|
||||
"resource_id": result.get("resource_id") or "",
|
||||
"error": result.get("error") or "",
|
||||
}
|
||||
)
|
||||
print(json.dumps(result))
|
||||
return 0 if result["success"] else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,190 @@
|
||||
"""
|
||||
Detect informal resource submissions that didn't use the issue template.
|
||||
|
||||
Returns a confidence score and recommended action:
|
||||
- score >= 0.6: High confidence - auto-close with firm warning
|
||||
- 0.4 <= score < 0.6: Medium confidence - gentle warning, leave open
|
||||
- score < 0.4: Low confidence - no action
|
||||
|
||||
Usage:
|
||||
Set ISSUE_TITLE and ISSUE_BODY environment variables, then run:
|
||||
python -m scripts.resources.detect_informal_submission
|
||||
|
||||
Outputs GitHub Actions outputs:
|
||||
- action: "none" | "warn" | "close"
|
||||
- confidence: float (0.0 to 1.0) formatted as percentage
|
||||
- matched_signals: comma-separated list of matched signals
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class Action(Enum):
|
||||
NONE = "none"
|
||||
WARN = "warn" # Medium confidence: warn but don't close
|
||||
CLOSE = "close" # High confidence: warn and close
|
||||
|
||||
|
||||
@dataclass
|
||||
class DetectionResult:
|
||||
confidence: float
|
||||
action: Action
|
||||
matched_signals: list[str]
|
||||
|
||||
|
||||
# Template field labels - VERY strong indicator (from the issue form)
|
||||
# Matching 3+ of these is almost certainly a copy-paste from template without using form
|
||||
TEMPLATE_FIELD_LABELS = [
|
||||
"display name:",
|
||||
"category:",
|
||||
"sub-category:",
|
||||
"primary link:",
|
||||
"author name:",
|
||||
"author link:",
|
||||
"license:",
|
||||
"description:",
|
||||
"validate claims:",
|
||||
"specific task",
|
||||
"specific prompt",
|
||||
"additional comments:",
|
||||
]
|
||||
|
||||
# Strong signals: clear intent to submit/recommend a resource
|
||||
STRONG_SIGNALS: list[tuple[str, str]] = [
|
||||
(r"\b(recommend(ing)?|submit(ting)?|submission)\b", "submission language"),
|
||||
(r"\b(add|include).*\b(resource|tool|plugin)\b", "add resource request"),
|
||||
(r"\b(please add|check.?out|made this|created this|built this)\b", "creator language"),
|
||||
(
|
||||
r"\b(would be great|might be useful|could be added|should be added)\b",
|
||||
"suggestion language",
|
||||
),
|
||||
(r"\bnew (tool|plugin|skill|hook|command)\b", "new resource mention"),
|
||||
]
|
||||
|
||||
# Medium signals: contextual indicators
|
||||
MEDIUM_SIGNALS: list[tuple[str, str]] = [
|
||||
(r"github\.com/[\w-]+/[\w-]+", "GitHub repo URL"),
|
||||
(r"\b(plugin|skill|hook|slash.?command|claude\.md)\b", "resource type mention"),
|
||||
(r"\b(agent skills?|tooling|workflows?|status.?lines?)\b", "category mention"),
|
||||
(r"\bhttps?://\S+", "contains URL"),
|
||||
(r"\bMIT|Apache|GPL|BSD|ISC\b", "license mention"),
|
||||
]
|
||||
|
||||
# Negative signals: reduce score if these are present (likely bug/question)
|
||||
NEGATIVE_SIGNALS: list[tuple[str, str]] = [
|
||||
(r"\b(bug|error|crash|broken|fix|issue|problem)\b", "bug-like language"),
|
||||
(r"\b(how (do|can|to)|what is|why does)\b", "question language"),
|
||||
(r"\b(not working|doesn't work|failed)\b", "failure language"),
|
||||
]
|
||||
|
||||
HIGH_THRESHOLD = 0.6
|
||||
MEDIUM_THRESHOLD = 0.4
|
||||
|
||||
|
||||
def count_template_field_matches(text: str) -> int:
|
||||
"""Count how many template field labels appear in the text."""
|
||||
text_lower = text.lower()
|
||||
return sum(1 for label in TEMPLATE_FIELD_LABELS if label in text_lower)
|
||||
|
||||
|
||||
def calculate_confidence(title: str, body: str) -> DetectionResult:
|
||||
"""Calculate confidence that this is an informal resource submission."""
|
||||
text = f"{title}\n{body}".lower()
|
||||
score = 0.0
|
||||
matched: list[str] = []
|
||||
|
||||
# Check for template field labels (VERY strong indicator)
|
||||
# 3+ matches = almost certainly tried to copy template format
|
||||
template_matches = count_template_field_matches(text)
|
||||
if template_matches >= 3:
|
||||
# This is a near-certain match - set high score immediately
|
||||
score += 0.7
|
||||
matched.append(f"template-fields: {template_matches} matches")
|
||||
elif template_matches >= 1:
|
||||
score += 0.2 * template_matches
|
||||
matched.append(f"template-fields: {template_matches} matches")
|
||||
|
||||
# Check strong signals (+0.3 each, max contribution ~0.9)
|
||||
for pattern, name in STRONG_SIGNALS:
|
||||
if re.search(pattern, text, re.IGNORECASE):
|
||||
score += 0.3
|
||||
matched.append(f"strong: {name}")
|
||||
|
||||
# Check medium signals (+0.15 each)
|
||||
for pattern, name in MEDIUM_SIGNALS:
|
||||
if re.search(pattern, text, re.IGNORECASE):
|
||||
score += 0.15
|
||||
matched.append(f"medium: {name}")
|
||||
|
||||
# Check negative signals (-0.2 each)
|
||||
for pattern, name in NEGATIVE_SIGNALS:
|
||||
if re.search(pattern, text, re.IGNORECASE):
|
||||
score -= 0.2
|
||||
matched.append(f"negative: {name}")
|
||||
|
||||
# Clamp score to [0, 1]
|
||||
score = max(0.0, min(1.0, score))
|
||||
|
||||
# Determine action based on thresholds
|
||||
if score >= HIGH_THRESHOLD:
|
||||
action = Action.CLOSE
|
||||
elif score >= MEDIUM_THRESHOLD:
|
||||
action = Action.WARN
|
||||
else:
|
||||
action = Action.NONE
|
||||
|
||||
return DetectionResult(confidence=score, action=action, matched_signals=matched)
|
||||
|
||||
|
||||
def sanitize_output(value: str) -> str:
|
||||
"""Sanitize a value for safe use in GitHub Actions outputs.
|
||||
|
||||
Prevents:
|
||||
- Newline injection (could add fake output variables)
|
||||
- Carriage return injection
|
||||
- Null byte injection
|
||||
"""
|
||||
# Remove characters that could break GITHUB_OUTPUT format or cause injection
|
||||
return value.replace("\n", " ").replace("\r", " ").replace("\0", "")
|
||||
|
||||
|
||||
def set_github_output(name: str, value: str) -> None:
|
||||
"""Set a GitHub Actions output variable safely."""
|
||||
# Sanitize both name and value to prevent injection attacks
|
||||
safe_name = sanitize_output(name)
|
||||
safe_value = sanitize_output(value)
|
||||
|
||||
github_output = os.environ.get("GITHUB_OUTPUT")
|
||||
if github_output:
|
||||
with open(github_output, "a") as f:
|
||||
f.write(f"{safe_name}={safe_value}\n")
|
||||
else:
|
||||
# For local testing, just print
|
||||
print(f"::set-output name={safe_name}::{safe_value}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Entry point for GitHub Actions."""
|
||||
title = os.environ.get("ISSUE_TITLE", "")
|
||||
body = os.environ.get("ISSUE_BODY", "")
|
||||
|
||||
result = calculate_confidence(title, body)
|
||||
|
||||
# Output results for GitHub Actions
|
||||
set_github_output("action", result.action.value)
|
||||
set_github_output("confidence", f"{result.confidence:.0%}")
|
||||
set_github_output("matched_signals", ", ".join(result.matched_signals))
|
||||
|
||||
# Also print for logging
|
||||
print(f"Confidence: {result.confidence:.2%}")
|
||||
print(f"Action: {result.action.value}")
|
||||
print(f"Matched signals: {result.matched_signals}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,497 @@
|
||||
"""
|
||||
Download resources from the Awesome Claude Code repository CSV file.
|
||||
|
||||
This script downloads all active resources (or filtered subset) from GitHub
|
||||
repositories listed in the resource-metadata.csv file. It respects rate
|
||||
limiting and organizes downloads by category.
|
||||
|
||||
Resources are saved to two locations:
|
||||
- Archive directory: All resources regardless of license (.myob/downloads/)
|
||||
- Hosted directory: Only open-source licensed resources (resources/)
|
||||
|
||||
Note: Authentication is optional but recommended to avoid rate limiting:
|
||||
- Unauthenticated: 60 requests/hour
|
||||
- Authenticated: 5,000 requests/hour
|
||||
export GITHUB_TOKEN=your_github_token
|
||||
|
||||
Usage:
|
||||
python download_resources.py [options]
|
||||
|
||||
Options:
|
||||
--category CATEGORY Filter by specific category
|
||||
--license LICENSE Filter by license type
|
||||
--max-downloads N Limit number of downloads (for testing)
|
||||
--output-dir DIR Custom archive directory (default: .myob/downloads)
|
||||
--hosted-dir DIR Custom hosted directory (default: resources)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import requests
|
||||
import yaml # type: ignore[import-untyped]
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from scripts.utils.github_utils import parse_github_resource_url
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
# Load environment variables from .myob/.env
|
||||
load_dotenv()
|
||||
|
||||
# Constants
|
||||
USER_AGENT = "awesome-claude-code Downloader/1.0"
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
CSV_FILE = REPO_ROOT / "THE_RESOURCES_TABLE.csv"
|
||||
DEFAULT_OUTPUT_DIR = ".myob/downloads"
|
||||
HOSTED_OUTPUT_DIR = "resources"
|
||||
|
||||
# Setup headers with optional GitHub token
|
||||
HEADERS = {
|
||||
"User-Agent": USER_AGENT,
|
||||
"Accept": "application/vnd.github.v3.raw",
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
}
|
||||
github_token = os.environ.get("GITHUB_TOKEN")
|
||||
if github_token:
|
||||
# Use Bearer token format as per GitHub API documentation
|
||||
HEADERS["Authorization"] = f"Bearer {github_token}"
|
||||
print("Using authenticated requests (5,000/hour limit)")
|
||||
else:
|
||||
print("Using unauthenticated requests (60/hour limit)")
|
||||
|
||||
# Open source licenses that allow hosting
|
||||
OPEN_SOURCE_LICENSES = {
|
||||
"MIT",
|
||||
"MIT+CC",
|
||||
"Apache-2.0",
|
||||
"BSD-2-Clause",
|
||||
"BSD-3-Clause",
|
||||
"GPL-2.0",
|
||||
"GPL-3.0",
|
||||
"LGPL-2.1",
|
||||
"LGPL-3.0",
|
||||
"MPL-2.0",
|
||||
"ISC",
|
||||
"0BSD",
|
||||
"Unlicense",
|
||||
"CC0-1.0",
|
||||
"CC-BY-4.0",
|
||||
"CC-BY-SA-4.0",
|
||||
"AGPL-3.0",
|
||||
"EPL-2.0",
|
||||
"BSL-1.0",
|
||||
}
|
||||
|
||||
# Category name mapping - removed to use sanitized names for both directories
|
||||
# Keeping the mapping dict empty for now in case we need it later
|
||||
_CATEGORY_MAPPING: dict[str, str] = {}
|
||||
|
||||
|
||||
def sanitize_filename(name: str) -> str:
|
||||
"""Sanitize a string to be safe for use as a filename."""
|
||||
# Replace spaces with hyphens and remove/replace problematic characters
|
||||
# Added commas and other special chars that could cause issues
|
||||
name = re.sub(r'[<>:"/\\|?*,;]', "", name)
|
||||
name = re.sub(r"\s+", "-", name)
|
||||
name = name.strip("-.")
|
||||
return name[:255] # Max filename length
|
||||
|
||||
|
||||
def download_github_file(
|
||||
url_info: dict[str, str], output_path: str, retry_count: int = 0, max_retries: int = 3
|
||||
) -> bool:
|
||||
"""
|
||||
Download a file from GitHub using the API.
|
||||
Returns True if successful, False otherwise.
|
||||
"""
|
||||
response: requests.Response | None = None
|
||||
try:
|
||||
if url_info["type"] == "file":
|
||||
# Download single file
|
||||
api_url = (
|
||||
f"https://api.github.com/repos/{url_info['owner']}/"
|
||||
f"{url_info['repo']}/contents/{url_info['path']}?ref={url_info['branch']}"
|
||||
)
|
||||
response = requests.get(api_url, headers=HEADERS, timeout=30)
|
||||
|
||||
# Log response details
|
||||
if response.status_code != 200:
|
||||
print(f" API Response: {response.status_code}")
|
||||
print(
|
||||
" Headers: X-RateLimit-Remaining"
|
||||
f"={response.headers.get('X-RateLimit-Remaining', 'N/A')}"
|
||||
)
|
||||
print(f" Response: {response.text[:300]}...")
|
||||
|
||||
if response.status_code == 200:
|
||||
# Create directory if needed
|
||||
os.makedirs(os.path.dirname(output_path), exist_ok=True)
|
||||
|
||||
# Write file content
|
||||
with open(output_path, "wb") as f:
|
||||
f.write(response.content)
|
||||
return True
|
||||
else:
|
||||
print(f" Failed to get file content - Status: {response.status_code}")
|
||||
|
||||
elif url_info["type"] == "dir":
|
||||
# List directory contents
|
||||
api_url = (
|
||||
f"https://api.github.com/repos/{url_info['owner']}/{url_info['repo']}/contents/"
|
||||
f"{url_info['path']}?ref={url_info['branch']}"
|
||||
)
|
||||
# Update headers to use proper Accept header for directory listing
|
||||
dir_headers = HEADERS.copy()
|
||||
dir_headers["Accept"] = "application/vnd.github+json"
|
||||
response = requests.get(api_url, headers=dir_headers, timeout=30)
|
||||
|
||||
# Log response details
|
||||
if response.status_code != 200:
|
||||
print(f" API Response: {response.status_code}")
|
||||
print(
|
||||
f" Headers: X-RateLimit-Remaining="
|
||||
f"{response.headers.get('X-RateLimit-Remaining', 'N/A')}"
|
||||
)
|
||||
print(f" Response: {response.text[:300]}...")
|
||||
|
||||
if response.status_code == 200:
|
||||
# Create directory
|
||||
os.makedirs(output_path, exist_ok=True)
|
||||
|
||||
# Download each file in the directory
|
||||
items = response.json()
|
||||
for item in items:
|
||||
if item["type"] == "file":
|
||||
file_path = os.path.join(output_path, item["name"])
|
||||
# Download the file content
|
||||
file_response = requests.get(
|
||||
item["download_url"], headers=HEADERS, timeout=30
|
||||
)
|
||||
if file_response.status_code != 200:
|
||||
print(
|
||||
f" File download failed: {item['name']} - "
|
||||
f"Status: {file_response.status_code}"
|
||||
)
|
||||
if file_response.status_code == 200:
|
||||
with open(file_path, "wb") as f:
|
||||
f.write(file_response.content)
|
||||
return True
|
||||
|
||||
elif url_info["type"] == "gist":
|
||||
# Download gist
|
||||
api_url = f"https://api.github.com/gists/{url_info['gist_id']}"
|
||||
# Update headers to use proper Accept header for gist API
|
||||
gist_headers = HEADERS.copy()
|
||||
gist_headers["Accept"] = "application/vnd.github+json"
|
||||
response = requests.get(api_url, headers=gist_headers, timeout=30)
|
||||
|
||||
# Log response details
|
||||
if response.status_code != 200:
|
||||
print(f" API Response: {response.status_code}")
|
||||
print(
|
||||
f" Headers: X-RateLimit-Remaining="
|
||||
f"{response.headers.get('X-RateLimit-Remaining', 'N/A')}"
|
||||
)
|
||||
print(f" Response: {response.text[:300]}...")
|
||||
|
||||
if response.status_code == 200:
|
||||
gist_data = response.json()
|
||||
# Create directory for gist
|
||||
os.makedirs(output_path, exist_ok=True)
|
||||
|
||||
# Download each file in the gist
|
||||
for filename, file_info in gist_data["files"].items():
|
||||
file_path = os.path.join(output_path, filename)
|
||||
with open(file_path, "w", encoding="utf-8") as f:
|
||||
f.write(file_info["content"])
|
||||
return True
|
||||
|
||||
# Handle rate limiting
|
||||
if response and response.status_code == 429:
|
||||
reset_time = response.headers.get("X-RateLimit-Reset")
|
||||
if reset_time:
|
||||
reset_datetime = datetime.fromtimestamp(int(reset_time))
|
||||
print(
|
||||
f" Rate limit will reset at: {reset_datetime.strftime('%Y-%m-%d %H:%M:%S')}"
|
||||
)
|
||||
raise requests.exceptions.HTTPError("Rate limited")
|
||||
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
if retry_count < max_retries:
|
||||
wait_time = (2**retry_count) + random.uniform(1, 2)
|
||||
print(f" Retry in {wait_time:.1f}s... (Error: {str(e)})")
|
||||
time.sleep(wait_time)
|
||||
return download_github_file(url_info, output_path, retry_count + 1, max_retries)
|
||||
|
||||
print(f" Failed after {max_retries} retries: {str(e)}")
|
||||
return False
|
||||
|
||||
|
||||
def load_overrides() -> dict[str, Any]:
|
||||
"""Load resource overrides from template directory."""
|
||||
template_dir = REPO_ROOT / "templates"
|
||||
override_path = os.path.join(template_dir, "resource-overrides.yaml")
|
||||
if not os.path.exists(override_path):
|
||||
return {}
|
||||
|
||||
with open(override_path, encoding="utf-8") as f:
|
||||
data = yaml.safe_load(f)
|
||||
return data.get("overrides", {})
|
||||
|
||||
|
||||
def apply_overrides(row: dict[str, str], overrides: dict[str, Any]) -> dict[str, str]:
|
||||
"""Apply overrides to a resource row.
|
||||
|
||||
Override values are applied for resource downloading. Any field set in
|
||||
the override configuration is automatically locked by validation scripts.
|
||||
"""
|
||||
resource_id = row.get("ID", "")
|
||||
if not resource_id or resource_id not in overrides:
|
||||
return row
|
||||
|
||||
override_config = overrides[resource_id]
|
||||
|
||||
# Apply overrides (excluding control/metadata fields and legacy locked flags)
|
||||
for field, value in override_config.items():
|
||||
# Skip special control/metadata fields
|
||||
if field in ["skip_validation", "notes"]:
|
||||
continue
|
||||
|
||||
# Skip any legacy *_locked flags (no longer needed)
|
||||
if field.endswith("_locked"):
|
||||
continue
|
||||
|
||||
# Apply override values
|
||||
if field == "license":
|
||||
row["License"] = value
|
||||
elif field == "active":
|
||||
row["Active"] = value
|
||||
elif field == "description":
|
||||
row["Description"] = value
|
||||
|
||||
return row
|
||||
|
||||
|
||||
def process_resources(
|
||||
category_filter: str | None = None,
|
||||
license_filter: str | None = None,
|
||||
max_downloads: int | None = None,
|
||||
output_dir: str = DEFAULT_OUTPUT_DIR,
|
||||
hosted_dir: str = HOSTED_OUTPUT_DIR,
|
||||
) -> None:
|
||||
"""
|
||||
Process and download resources from the CSV file.
|
||||
"""
|
||||
start_time = datetime.now()
|
||||
print(f"Starting download at: {start_time.strftime('%Y-%m-%d %H:%M:%S')}")
|
||||
print(f"Archive directory (all resources): {output_dir}")
|
||||
print(f"Hosted directory (open-source only): {hosted_dir}")
|
||||
|
||||
# Check rate limit status
|
||||
try:
|
||||
rate_check = requests.get("https://api.github.com/rate_limit", headers=HEADERS, timeout=10)
|
||||
if rate_check.status_code == 200:
|
||||
rate_data = rate_check.json()
|
||||
core_limit = rate_data.get("rate", {})
|
||||
print("\nGitHub API Rate Limit Status:")
|
||||
print(
|
||||
f" Remaining: {core_limit.get('remaining', 'N/A')}/"
|
||||
f"{core_limit.get('limit', 'N/A')}"
|
||||
)
|
||||
if core_limit.get("reset"):
|
||||
reset_time = datetime.fromtimestamp(core_limit["reset"])
|
||||
print(f" Resets at: {reset_time.strftime('%Y-%m-%d %H:%M:%S')}")
|
||||
except Exception as e:
|
||||
print(f"Could not check rate limit: {e}")
|
||||
|
||||
# Load overrides
|
||||
overrides = load_overrides()
|
||||
if overrides:
|
||||
print(f"\nLoaded {len(overrides)} resource overrides")
|
||||
|
||||
# Track statistics
|
||||
total_resources = 0
|
||||
downloaded = 0
|
||||
skipped = 0
|
||||
failed = 0
|
||||
|
||||
# Read CSV
|
||||
with open("./THE_RESOURCES_TABLE.csv", newline="", encoding="utf-8") as file:
|
||||
reader = csv.DictReader(file)
|
||||
|
||||
for row in reader:
|
||||
# Apply overrides to the row
|
||||
row = apply_overrides(row, overrides)
|
||||
# Check if we've reached the download limit
|
||||
if max_downloads and downloaded >= max_downloads:
|
||||
print(f"\nReached download limit ({max_downloads}). Stopping.")
|
||||
break
|
||||
|
||||
# Skip inactive resources
|
||||
if row["Active"].upper() != "TRUE":
|
||||
continue
|
||||
|
||||
total_resources += 1
|
||||
|
||||
# Apply filters
|
||||
if category_filter and row["Category"] != category_filter:
|
||||
continue
|
||||
|
||||
if license_filter and row.get("License", "") != license_filter:
|
||||
continue
|
||||
|
||||
# Get the URL (prefer primary link)
|
||||
url = row["Primary Link"].strip() or row["Secondary Link"].strip()
|
||||
if not url:
|
||||
continue
|
||||
|
||||
display_name = row["Display Name"]
|
||||
original_category = row["Category"]
|
||||
category = sanitize_filename(original_category.lower().replace(" & ", "-"))
|
||||
|
||||
# Use same sanitized category name for both directories
|
||||
resource_license = row.get("License", "NOT_FOUND").strip()
|
||||
|
||||
print(f"\n[{downloaded + 1}] Processing: {display_name}")
|
||||
print(f" URL: {url}")
|
||||
print(f" Category: {original_category} -> '{category}'")
|
||||
|
||||
# Parse GitHub URL
|
||||
url_info = parse_github_resource_url(url)
|
||||
if not url_info:
|
||||
print(" Skipped: Not a GitHub URL")
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
# Determine output paths
|
||||
safe_name = sanitize_filename(display_name)
|
||||
print(f" Sanitized name: '{display_name}' -> '{safe_name}'")
|
||||
|
||||
# Primary path for archive (all resources)
|
||||
if url_info["type"] == "gist":
|
||||
resource_path = os.path.join(output_dir, category, f"{safe_name}-gist")
|
||||
hosted_path = (
|
||||
os.path.join(hosted_dir, category, safe_name)
|
||||
if resource_license in OPEN_SOURCE_LICENSES
|
||||
else None
|
||||
)
|
||||
elif url_info["type"] == "repo":
|
||||
resource_path = os.path.join(output_dir, category, safe_name)
|
||||
print(" Skipped: Full repository downloads not implemented")
|
||||
skipped += 1
|
||||
continue
|
||||
elif url_info["type"] == "dir":
|
||||
resource_path = os.path.join(output_dir, category, safe_name)
|
||||
hosted_path = (
|
||||
os.path.join(hosted_dir, category, safe_name)
|
||||
if resource_license in OPEN_SOURCE_LICENSES
|
||||
else None
|
||||
)
|
||||
else: # file
|
||||
# Extract filename from path
|
||||
filename = os.path.basename(url_info["path"])
|
||||
resource_path = os.path.join(output_dir, category, safe_name, filename)
|
||||
hosted_path = (
|
||||
os.path.join(hosted_dir, category, safe_name, filename)
|
||||
if resource_license in OPEN_SOURCE_LICENSES
|
||||
else None
|
||||
)
|
||||
|
||||
# Download the resource to archive
|
||||
print(f" Downloading to archive: {resource_path}")
|
||||
print(f" License: {resource_license}")
|
||||
if hosted_path:
|
||||
print(f" Will copy to hosted: {hosted_path}")
|
||||
|
||||
download_success = download_github_file(url_info, resource_path)
|
||||
|
||||
if download_success:
|
||||
print(" ✅ Downloaded successfully")
|
||||
downloaded += 1
|
||||
|
||||
# If open-source licensed, also copy to hosted directory
|
||||
if hosted_path and resource_license in OPEN_SOURCE_LICENSES:
|
||||
print(f" 📦 Copying to hosted directory: {hosted_path}")
|
||||
try:
|
||||
import shutil
|
||||
|
||||
os.makedirs(os.path.dirname(hosted_path), exist_ok=True)
|
||||
|
||||
if os.path.isdir(resource_path):
|
||||
print(
|
||||
f" Source is directory with "
|
||||
f"{len(os.listdir(resource_path))} items"
|
||||
)
|
||||
shutil.copytree(resource_path, hosted_path, dirs_exist_ok=True)
|
||||
else:
|
||||
print(" Source is file")
|
||||
shutil.copy2(resource_path, hosted_path)
|
||||
print(" ✅ Copied to hosted directory")
|
||||
except Exception as e:
|
||||
print(f" ⚠️ Failed to copy to hosted directory: {e}")
|
||||
print(f" Error type: {type(e).__name__}")
|
||||
import traceback
|
||||
|
||||
print(f" Traceback: {traceback.format_exc()}")
|
||||
else:
|
||||
print(" ❌ Download failed")
|
||||
failed += 1
|
||||
|
||||
# Rate limiting delay
|
||||
time.sleep(random.uniform(1, 2))
|
||||
|
||||
# Summary
|
||||
end_time = datetime.now()
|
||||
duration = end_time - start_time
|
||||
|
||||
print(f"\n{'=' * 60}")
|
||||
print(f"Download completed at: {end_time.strftime('%Y-%m-%d %H:%M:%S')}")
|
||||
print(f"Total execution time: {duration}")
|
||||
print("\nSummary:")
|
||||
print(f" Total resources found: {total_resources}")
|
||||
print(f" Downloaded: {downloaded}")
|
||||
print(f" Skipped: {skipped}")
|
||||
print(f" Failed: {failed}")
|
||||
print(f"{'=' * 60}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Main entry point."""
|
||||
parser = argparse.ArgumentParser(description="Download resources from awesome-claude-code CSV")
|
||||
parser.add_argument("--category", help="Filter by specific category")
|
||||
parser.add_argument("--license", help="Filter by license type")
|
||||
parser.add_argument("--max-downloads", type=int, help="Limit number of downloads")
|
||||
parser.add_argument("--output-dir", default=DEFAULT_OUTPUT_DIR, help="Archive output directory")
|
||||
parser.add_argument(
|
||||
"--hosted-dir",
|
||||
default=HOSTED_OUTPUT_DIR,
|
||||
help="Hosted output directory for open-source resources",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Create output directories if needed
|
||||
Path(args.output_dir).mkdir(parents=True, exist_ok=True)
|
||||
Path(args.hosted_dir).mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Process resources
|
||||
process_resources(
|
||||
category_filter=args.category,
|
||||
license_filter=args.license,
|
||||
max_downloads=args.max_downloads,
|
||||
output_dir=args.output_dir,
|
||||
hosted_dir=args.hosted_dir,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,292 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Parse GitHub Issue form data from resource submissions.
|
||||
Validates the data and returns structured JSON.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from scripts.categories.category_utils import category_manager
|
||||
from scripts.validation.validate_single_resource import validate_single_resource
|
||||
|
||||
|
||||
def parse_issue_body(issue_body: str) -> dict[str, str]:
|
||||
"""
|
||||
Parse GitHub issue form body into structured data.
|
||||
|
||||
GitHub issue forms are rendered as markdown with specific patterns:
|
||||
- Headers (###) indicate field labels
|
||||
- Values follow the headers
|
||||
- Checkboxes are rendered as - [x] or - [ ]
|
||||
"""
|
||||
data = {}
|
||||
|
||||
# Split into sections by ### headers
|
||||
sections = re.split(r"###\s+", issue_body)
|
||||
|
||||
for section in sections:
|
||||
if not section.strip():
|
||||
continue
|
||||
|
||||
lines = section.strip().split("\n")
|
||||
if not lines:
|
||||
continue
|
||||
|
||||
# First line is the field label
|
||||
label = lines[0].strip()
|
||||
|
||||
# Rest is the value (skip empty lines)
|
||||
value_lines = [
|
||||
line
|
||||
for line in lines[1:]
|
||||
if line.strip() and not line.strip().startswith("_No response_")
|
||||
]
|
||||
value = "\n".join(value_lines).strip()
|
||||
|
||||
# Map form labels to data fields
|
||||
if "Display Name" in label:
|
||||
data["display_name"] = value
|
||||
data["_original_display_name"] = value # Track original for warning
|
||||
elif "Category" in label and "Sub-Category" not in label:
|
||||
data["category"] = value
|
||||
# If this is a slash command, we'll validate/fix the display name later
|
||||
elif "Sub-Category" in label:
|
||||
# Set to "General" as default if empty or "None / Not Applicable"
|
||||
if not value or "None" in value or "Not Applicable" in value:
|
||||
data["subcategory"] = "General"
|
||||
else:
|
||||
# Strip the category prefix if present
|
||||
# (e.g., "Slash-Commands: " from "Slash-Commands: Context Loading & Priming")
|
||||
if ":" in value:
|
||||
data["subcategory"] = value.split(":", 1)[1].strip()
|
||||
else:
|
||||
data["subcategory"] = value
|
||||
elif "Primary Link" in label:
|
||||
data["primary_link"] = value
|
||||
elif "Secondary Link" in label:
|
||||
data["secondary_link"] = value
|
||||
elif "Author Name" in label:
|
||||
data["author_name"] = value
|
||||
elif "Author Link" in label:
|
||||
data["author_link"] = value
|
||||
elif "License" in label and "Other License" not in label:
|
||||
data["license"] = value
|
||||
elif "Other License" in label:
|
||||
if value:
|
||||
data["license"] = value # Override with custom license
|
||||
elif "Description" in label:
|
||||
data["description"] = value
|
||||
|
||||
# Fix slash command display names
|
||||
if data.get("category") == "Slash-Commands" and data.get("display_name"):
|
||||
display_name = data["display_name"]
|
||||
|
||||
# Ensure it starts with a slash
|
||||
if not display_name.startswith("/"):
|
||||
display_name = "/" + display_name
|
||||
|
||||
# Ensure it's a single string (no spaces, only hyphens, underscores, colons allowed)
|
||||
# Replace spaces with hyphens
|
||||
display_name = display_name.replace(" ", "-")
|
||||
|
||||
# Remove any characters that aren't alphanumeric, slash, hyphen, underscore, or colon
|
||||
display_name = re.sub(r"[^a-zA-Z0-9/_:-]", "", display_name)
|
||||
|
||||
# Ensure it's lowercase (convention for slash commands)
|
||||
display_name = display_name.lower()
|
||||
|
||||
# Ensure only one leading slash - remove any extra slashes at the beginning
|
||||
while display_name.startswith("//"):
|
||||
display_name = display_name[1:]
|
||||
|
||||
data["display_name"] = display_name
|
||||
|
||||
return data
|
||||
|
||||
|
||||
def validate_parsed_data(data: dict[str, str]) -> tuple[bool, list[str], list[str]]:
|
||||
"""
|
||||
Validate the parsed data meets all requirements.
|
||||
Returns (is_valid, errors, warnings)
|
||||
"""
|
||||
errors = []
|
||||
warnings = []
|
||||
|
||||
# Check required fields
|
||||
required_fields = [
|
||||
"display_name",
|
||||
"category",
|
||||
"primary_link",
|
||||
"author_name",
|
||||
"author_link",
|
||||
"description",
|
||||
]
|
||||
|
||||
for field in required_fields:
|
||||
if not data.get(field, "").strip():
|
||||
errors.append(f"Required field '{field}' is missing or empty")
|
||||
|
||||
# Validate category
|
||||
valid_categories = category_manager.get_all_categories()
|
||||
if data.get("category") not in valid_categories:
|
||||
errors.append(
|
||||
f"Invalid category: {data.get('category')}. "
|
||||
f"Must be one of: {', '.join(valid_categories)}"
|
||||
)
|
||||
|
||||
# Sub-category validation is no longer needed since we strip the prefix
|
||||
# The form already ensures subcategories match their parent categories
|
||||
|
||||
# Check if slash command display name was modified
|
||||
if (
|
||||
data.get("category") == "Slash-Commands"
|
||||
and "_original_display_name" in data
|
||||
and data["display_name"] != data["_original_display_name"]
|
||||
):
|
||||
warnings.append(
|
||||
f"Display name was automatically corrected from "
|
||||
f"'{data['_original_display_name']}' to '{data['display_name']}'. "
|
||||
"Slash commands must start with '/' and contain no spaces."
|
||||
)
|
||||
|
||||
# Additional validation for slash commands - check for multiple slashes
|
||||
if data.get("category") == "Slash-Commands" and data.get("display_name"):
|
||||
display_name = data["display_name"]
|
||||
# Check if there are multiple slashes anywhere in the command
|
||||
slash_count = display_name.count("/")
|
||||
if slash_count > 1:
|
||||
errors.append(
|
||||
f"Slash command '{display_name}' contains multiple slashes. "
|
||||
"Slash commands must have exactly one slash at the beginning."
|
||||
)
|
||||
|
||||
# Validate URLs
|
||||
url_fields = ["primary_link", "secondary_link", "author_link"]
|
||||
for field in url_fields:
|
||||
value = data.get(field, "").strip()
|
||||
if value and field != "secondary_link": # secondary is optional
|
||||
if not value.startswith("https://"):
|
||||
errors.append(f"{field} must start with https://")
|
||||
elif " " in value:
|
||||
errors.append(f"{field} contains spaces")
|
||||
|
||||
# Validate license
|
||||
if data.get("license") == "No License / Not Specified":
|
||||
data["license"] = "NOT_FOUND"
|
||||
warnings.append("No license specified - consider adding one for open source projects")
|
||||
|
||||
# Check description length
|
||||
description = data.get("description", "")
|
||||
if len(description) > 500:
|
||||
errors.append("Description is too long (max 500 characters)")
|
||||
elif len(description) < 10:
|
||||
errors.append("Description is too short (min 10 characters)")
|
||||
|
||||
# Check for common issues
|
||||
if data.get("display_name", "").lower() in ["test", "testing", "example"]:
|
||||
warnings.append("Display name appears to be a test entry")
|
||||
|
||||
return len(errors) == 0, errors, warnings
|
||||
|
||||
|
||||
def check_for_duplicates(data: dict[str, str]) -> list[str]:
|
||||
"""Check if resource already exists in the CSV."""
|
||||
warnings: list[str] = []
|
||||
|
||||
csv_path = os.path.join(REPO_ROOT, "THE_RESOURCES_TABLE.csv")
|
||||
if not os.path.exists(csv_path):
|
||||
return warnings
|
||||
|
||||
import csv
|
||||
|
||||
primary_link = data.get("primary_link", "").lower()
|
||||
display_name = data.get("display_name", "").lower()
|
||||
|
||||
with open(csv_path, encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
for row in reader:
|
||||
# Check for duplicate URL
|
||||
if row.get("Primary Link", "").lower() == primary_link:
|
||||
warnings.append(
|
||||
f"A resource with this primary link already exists: {row.get('Display Name')}"
|
||||
)
|
||||
# Check for similar names
|
||||
elif row.get("Display Name", "").lower() == display_name:
|
||||
warnings.append(
|
||||
f"A resource with the same name already exists: {row.get('Display Name')}"
|
||||
)
|
||||
|
||||
return warnings
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point for the script."""
|
||||
# Get issue body from environment variable
|
||||
issue_body = os.environ.get("ISSUE_BODY", "")
|
||||
if not issue_body:
|
||||
print(json.dumps({"valid": False, "errors": ["No issue body provided"], "data": {}}))
|
||||
return 1
|
||||
|
||||
# Parse the issue body
|
||||
parsed_data = parse_issue_body(issue_body)
|
||||
|
||||
# Check if --validate flag is passed
|
||||
validate_mode = "--validate" in sys.argv
|
||||
|
||||
if validate_mode:
|
||||
# Full validation mode
|
||||
is_valid, errors, warnings = validate_parsed_data(parsed_data)
|
||||
|
||||
# Check for duplicates
|
||||
duplicate_warnings = check_for_duplicates(parsed_data)
|
||||
warnings.extend(duplicate_warnings)
|
||||
|
||||
# If basic validation passed, do URL validation
|
||||
if is_valid and parsed_data.get("primary_link"):
|
||||
url_valid, enriched_data, url_errors = validate_single_resource(
|
||||
primary_link=parsed_data.get("primary_link", ""),
|
||||
secondary_link=parsed_data.get("secondary_link", ""),
|
||||
display_name=parsed_data.get("display_name", ""),
|
||||
category=parsed_data.get("category", ""),
|
||||
license=parsed_data.get("license", "NOT_FOUND"),
|
||||
subcategory=parsed_data.get("subcategory", ""),
|
||||
author_name=parsed_data.get("author_name", ""),
|
||||
author_link=parsed_data.get("author_link", ""),
|
||||
description=parsed_data.get("description", ""),
|
||||
)
|
||||
|
||||
if not url_valid:
|
||||
is_valid = False
|
||||
errors.extend(url_errors)
|
||||
else:
|
||||
# Update with enriched data (license from GitHub, etc.)
|
||||
parsed_data.update(enriched_data)
|
||||
|
||||
# Remove temporary tracking field
|
||||
if "_original_display_name" in parsed_data:
|
||||
del parsed_data["_original_display_name"]
|
||||
|
||||
result = {"valid": is_valid, "errors": errors, "warnings": warnings, "data": parsed_data}
|
||||
else:
|
||||
# Simple parse mode - just return the parsed data
|
||||
# Remove temporary tracking field
|
||||
if "_original_display_name" in parsed_data:
|
||||
del parsed_data["_original_display_name"]
|
||||
result = parsed_data
|
||||
|
||||
# Print compact JSON (no newlines) to make it easier to extract
|
||||
print(json.dumps(result))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Helpers for resource CSV updates and PR content generation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import os
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
__all__ = ["append_to_csv", "generate_pr_content"]
|
||||
|
||||
|
||||
def append_to_csv(data: dict[str, str]) -> bool:
|
||||
"""Append the new resource to THE_RESOURCES_TABLE.csv using header order."""
|
||||
csv_path = os.path.join(REPO_ROOT, "THE_RESOURCES_TABLE.csv")
|
||||
|
||||
try:
|
||||
with open(csv_path, encoding="utf-8", newline="") as f:
|
||||
reader = csv.reader(f)
|
||||
headers = next(reader, None)
|
||||
except Exception as e:
|
||||
print(f"Error reading CSV header: {e}")
|
||||
return False
|
||||
|
||||
if not headers:
|
||||
print("Error reading CSV header: missing header row")
|
||||
return False
|
||||
|
||||
now = datetime.now().strftime("%Y-%m-%d:%H-%M-%S")
|
||||
value_map = {
|
||||
"ID": data.get("id", ""),
|
||||
"Display Name": data.get("display_name", ""),
|
||||
"Category": data.get("category", ""),
|
||||
"Sub-Category": data.get("subcategory", ""),
|
||||
"Primary Link": data.get("primary_link", ""),
|
||||
"Secondary Link": data.get("secondary_link", ""),
|
||||
"Author Name": data.get("author_name", ""),
|
||||
"Author Link": data.get("author_link", ""),
|
||||
"Active": data.get("active", "TRUE"),
|
||||
"Date Added": data.get("date_added", now),
|
||||
"Last Modified": data.get("last_modified", ""),
|
||||
"Last Checked": data.get("last_checked", now),
|
||||
"License": data.get("license", ""),
|
||||
"Description": data.get("description", ""),
|
||||
"Removed From Origin": data.get("removed_from_origin", "FALSE"),
|
||||
"Stale": data.get("stale", "FALSE"),
|
||||
"Repo Created": data.get("repo_created", ""),
|
||||
"Latest Release": data.get("latest_release", ""),
|
||||
"Release Version": data.get("release_version", ""),
|
||||
"Release Source": data.get("release_source", ""),
|
||||
}
|
||||
|
||||
missing_headers = [key for key in value_map if key not in headers]
|
||||
if missing_headers:
|
||||
print(f"Error reading CSV header: missing columns {', '.join(missing_headers)}")
|
||||
return False
|
||||
|
||||
row = {header: value_map.get(header, "") for header in headers}
|
||||
|
||||
try:
|
||||
with open(csv_path, "a", newline="", encoding="utf-8") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=headers)
|
||||
writer.writerow(row)
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"Error writing to CSV: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def generate_pr_content(data: dict[str, str]) -> str:
|
||||
"""Generate PR template content."""
|
||||
is_github = "github.com" in data["primary_link"]
|
||||
|
||||
content = f"""### Resource Information
|
||||
|
||||
- **Display Name**: {data["display_name"]}
|
||||
- **Category**: {data["category"]}
|
||||
- **Sub-Category**: {data["subcategory"] if data["subcategory"] else "N/A"}
|
||||
- **Primary Link**: {data["primary_link"]}
|
||||
- **Author Name**: {data["author_name"]}
|
||||
- **Author Link**: {data["author_link"]}
|
||||
- **License**: {data["license"] if data["license"] else "Not specified"}
|
||||
|
||||
### Description
|
||||
|
||||
{data["description"]}
|
||||
|
||||
### Automated Notification
|
||||
|
||||
- [{"x" if is_github else " "}] This is a GitHub-hosted resource and will receive an automatic
|
||||
notification issue when merged"""
|
||||
|
||||
return content
|
||||
@@ -0,0 +1,133 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Sort THE_RESOURCES_TABLE.csv by category, sub-category, and display name.
|
||||
|
||||
This utility ensures resources are properly ordered for consistent presentation
|
||||
in the generated README and other outputs.
|
||||
"""
|
||||
|
||||
import csv
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
|
||||
def sort_resources(csv_path: Path) -> None:
|
||||
"""Sort resources in the CSV file by category, sub-category,
|
||||
and display name."""
|
||||
# Load category order from category_utils
|
||||
from scripts.categories.category_utils import category_manager
|
||||
|
||||
category_order = []
|
||||
categories = []
|
||||
try:
|
||||
categories = category_manager.get_categories_for_readme()
|
||||
category_order = [cat["name"] for cat in categories]
|
||||
except Exception as e:
|
||||
print(f"Warning: Could not load category order from category_utils: {e}")
|
||||
print("Using alphabetical sorting instead.")
|
||||
|
||||
# Create a mapping for sort order
|
||||
category_sort_map = {cat: idx for idx, cat in enumerate(category_order)}
|
||||
|
||||
# Create subcategory order mappings for each category
|
||||
subcategory_sort_maps = {}
|
||||
for category in categories:
|
||||
if "subcategories" in category:
|
||||
subcat_order = [sub["name"] for sub in category["subcategories"]]
|
||||
subcategory_sort_maps[category["name"]] = {
|
||||
name: idx for idx, name in enumerate(subcat_order)
|
||||
}
|
||||
|
||||
# Read the CSV data
|
||||
with open(csv_path, encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
headers = reader.fieldnames
|
||||
rows = list(reader)
|
||||
|
||||
# Sort the rows
|
||||
# First by Category (using custom order), then by Sub-Category
|
||||
# (using defined order from YAML), then by Display Name
|
||||
def subcategory_sort_key(category, subcat):
|
||||
"""Sort subcategories by their defined order in categories.yaml"""
|
||||
if not subcat:
|
||||
return 999 # Empty sorts last
|
||||
|
||||
# Get the sort map for this category
|
||||
if category in subcategory_sort_maps:
|
||||
subcat_map = subcategory_sort_maps[category]
|
||||
return subcat_map.get(subcat, 998) # Unknown subcategories sort second-to-last
|
||||
|
||||
# If no sort map, fall back to alphabetical
|
||||
return 997 # Categories without defined subcategory order
|
||||
|
||||
sorted_rows = sorted(
|
||||
rows,
|
||||
key=lambda row: (
|
||||
category_sort_map.get(row.get("Category", ""), 999), # Unknown categories sort last
|
||||
subcategory_sort_key(row.get("Category", ""), row.get("Sub-Category", "")),
|
||||
row.get("Display Name", "").lower(),
|
||||
),
|
||||
)
|
||||
|
||||
# Write the sorted data back
|
||||
with open(csv_path, "w", encoding="utf-8", newline="") as f:
|
||||
if headers:
|
||||
writer = csv.DictWriter(f, fieldnames=headers)
|
||||
writer.writeheader()
|
||||
writer.writerows(sorted_rows)
|
||||
|
||||
print(f"✓ Sorted {len(sorted_rows)} resources in {csv_path}")
|
||||
|
||||
# Print summary of categories
|
||||
category_counts: dict[str, dict[str, int]] = {}
|
||||
for row in sorted_rows:
|
||||
category_name = row.get("Category", "Unknown")
|
||||
subcat = row.get("Sub-Category", "") or "None"
|
||||
if category_name not in category_counts:
|
||||
category_counts[category_name] = {}
|
||||
if subcat not in category_counts[category_name]:
|
||||
category_counts[category_name][subcat] = 0
|
||||
category_counts[category_name][subcat] += 1
|
||||
|
||||
print("\nCategory Summary:")
|
||||
# Sort categories using the same custom order
|
||||
sorted_categories = sorted(
|
||||
category_counts.keys(), key=lambda cat: category_sort_map.get(cat, 999)
|
||||
)
|
||||
for category_name in sorted_categories:
|
||||
print(f" {category_name}:")
|
||||
# Sort subcategories using the same order as in the CSV sorting
|
||||
sorted_subcats = sorted(
|
||||
category_counts[category_name].keys(),
|
||||
key=lambda s: subcategory_sort_key(category_name, s if s != "None" else ""),
|
||||
)
|
||||
for subcat in sorted_subcats:
|
||||
count = category_counts[category_name][subcat]
|
||||
if subcat == "None":
|
||||
print(f" (no sub-category): {count} items")
|
||||
else:
|
||||
print(f" {subcat}: {count} items")
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point."""
|
||||
# Default to THE_RESOURCES_TABLE.csv in parent directory
|
||||
csv_path = REPO_ROOT / "THE_RESOURCES_TABLE.csv"
|
||||
|
||||
if len(sys.argv) > 1:
|
||||
csv_path = Path(sys.argv[1])
|
||||
|
||||
if not csv_path.exists():
|
||||
print(f"Error: CSV file not found at {csv_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
sort_resources(csv_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
"""Testing helpers package."""
|
||||
@@ -0,0 +1,156 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Integration regeneration cycle test for README outputs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
CONFIG_PATH = REPO_ROOT / "acc-config.yaml"
|
||||
README_PATH = REPO_ROOT / "README.md"
|
||||
|
||||
ROOT_STYLE_RE = re.compile(r"^(?P<indent>\s*)root_style:\s*(?P<value>\S+)\s*$", re.M)
|
||||
STYLE_ORDER_RE = re.compile(r"^(style_order:\s*\n)(?P<items>(?:\s*-\s*.*\n?)+)", re.M)
|
||||
|
||||
|
||||
def run(cmd: list[str]) -> None:
|
||||
subprocess.run(cmd, cwd=REPO_ROOT, check=True)
|
||||
|
||||
|
||||
def git_status() -> str:
|
||||
result = subprocess.run(
|
||||
["git", "status", "--porcelain"],
|
||||
cwd=REPO_ROOT,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def read_config_text() -> str:
|
||||
return CONFIG_PATH.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def write_config_text(text: str) -> None:
|
||||
CONFIG_PATH.write_text(text, encoding="utf-8")
|
||||
|
||||
|
||||
def set_root_style(text: str, root_style: str) -> str:
|
||||
if not ROOT_STYLE_RE.search(text):
|
||||
raise RuntimeError("root_style not found in acc-config.yaml")
|
||||
return ROOT_STYLE_RE.sub(rf"\g<indent>root_style: {root_style}", text, count=1)
|
||||
|
||||
|
||||
def get_style_order(text: str) -> list[str]:
|
||||
match = STYLE_ORDER_RE.search(text)
|
||||
if not match:
|
||||
raise RuntimeError("style_order not found in acc-config.yaml")
|
||||
items: list[str] = []
|
||||
for line in match.group("items").splitlines():
|
||||
line = line.strip()
|
||||
if line.startswith("-"):
|
||||
items.append(line[1:].strip())
|
||||
if not items:
|
||||
raise RuntimeError("style_order is empty in acc-config.yaml")
|
||||
return items
|
||||
|
||||
|
||||
def set_style_order(text: str, style_order: list[str]) -> str:
|
||||
if not STYLE_ORDER_RE.search(text):
|
||||
raise RuntimeError("style_order not found in acc-config.yaml")
|
||||
block = "style_order:\n" + "".join(f" - {style}\n" for style in style_order)
|
||||
return STYLE_ORDER_RE.sub(block, text, count=1)
|
||||
|
||||
|
||||
def read_readme() -> str:
|
||||
return README_PATH.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def selector_order_from_content(content: str) -> list[str]:
|
||||
matches = re.findall(r"badge-style-([a-z0-9_-]+)\.svg", content)
|
||||
if not matches:
|
||||
raise RuntimeError("Could not determine style selector order from README.md")
|
||||
ordered: list[str] = []
|
||||
for item in matches:
|
||||
if item not in ordered:
|
||||
ordered.append(item)
|
||||
return ordered
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if git_status():
|
||||
print("Error: working tree must be clean before running test-regenerate-cycle")
|
||||
return 1
|
||||
|
||||
original_text = read_config_text()
|
||||
|
||||
try:
|
||||
run(["make", "test-regenerate"])
|
||||
|
||||
current_text = read_config_text()
|
||||
style_order = get_style_order(current_text)
|
||||
|
||||
if len(style_order) < 2:
|
||||
raise RuntimeError("style_order must contain at least two entries")
|
||||
|
||||
first_style = style_order[0]
|
||||
second_style = style_order[1]
|
||||
|
||||
updated_text = set_root_style(current_text, first_style)
|
||||
write_config_text(updated_text)
|
||||
|
||||
run(["make", "test-regenerate-allow-diff"])
|
||||
first_content = read_readme()
|
||||
|
||||
updated_text = set_root_style(updated_text, second_style)
|
||||
write_config_text(updated_text)
|
||||
|
||||
run(["make", "test-regenerate-allow-diff"])
|
||||
root_content = read_readme()
|
||||
if root_content == first_content:
|
||||
raise RuntimeError("README.md did not change after root_style update")
|
||||
|
||||
new_order = style_order[1:] + style_order[:1] if len(style_order) > 1 else style_order
|
||||
updated_text = set_style_order(updated_text, new_order)
|
||||
write_config_text(updated_text)
|
||||
|
||||
previous_content = root_content
|
||||
run(["make", "test-regenerate-allow-diff"])
|
||||
root_content = read_readme()
|
||||
if root_content == previous_content:
|
||||
raise RuntimeError("README.md did not change after style_order update")
|
||||
|
||||
selector_order = selector_order_from_content(root_content)
|
||||
if selector_order[: len(new_order)] != new_order:
|
||||
raise RuntimeError("Style selector order does not match updated style_order")
|
||||
|
||||
write_config_text(original_text)
|
||||
run(["make", "test-regenerate", "ALLOW_DIRTY=1"])
|
||||
|
||||
if git_status():
|
||||
raise RuntimeError("Working tree is dirty after restoring configuration")
|
||||
except subprocess.CalledProcessError as exc:
|
||||
print(f"Error: command failed: {exc}", file=sys.stderr)
|
||||
write_config_text(original_text)
|
||||
with contextlib.suppress(subprocess.CalledProcessError):
|
||||
run(["make", "test-regenerate", "ALLOW_DIRTY=1"])
|
||||
return exc.returncode
|
||||
except Exception as exc:
|
||||
print(f"Error: {exc}", file=sys.stderr)
|
||||
write_config_text(original_text)
|
||||
with contextlib.suppress(subprocess.CalledProcessError):
|
||||
run(["make", "test-regenerate", "ALLOW_DIRTY=1"])
|
||||
return 1
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,263 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate TOC anchors against GitHub-rendered HTML.
|
||||
|
||||
This utility compares the anchor links in our generated README's table of contents
|
||||
against the actual anchor IDs that GitHub generates when rendering the markdown.
|
||||
|
||||
Usage:
|
||||
# Validate AWESOME style (default)
|
||||
python -m scripts.testing.validate_toc_anchors
|
||||
|
||||
# Validate specific style
|
||||
python -m scripts.testing.validate_toc_anchors --style classic
|
||||
python -m scripts.testing.validate_toc_anchors --style extra
|
||||
python -m scripts.testing.validate_toc_anchors --style flat
|
||||
|
||||
# Validate with custom paths
|
||||
python -m scripts.testing.validate_toc_anchors --html path/to/github.html --readme README.md
|
||||
|
||||
# Generate new fixture (requires manual step to download HTML from GitHub)
|
||||
python -m scripts.testing.validate_toc_anchors --generate-expected
|
||||
|
||||
To obtain the GitHub HTML:
|
||||
1. Push your README to GitHub
|
||||
2. View the rendered README page
|
||||
3. Open browser dev tools (F12)
|
||||
4. Find the <article> element containing the README content
|
||||
5. Copy the inner HTML to tests/fixtures/github-html/<style>.html
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
FIXTURES_DIR = REPO_ROOT / "tests" / "fixtures" / "github-html"
|
||||
EXPECTED_ANCHORS_PATH = REPO_ROOT / "tests" / "fixtures" / "expected_toc_anchors.txt"
|
||||
|
||||
# Style configurations: (html_fixture, readme_path)
|
||||
# HTML fixture names indicate root vs non-root placement on GitHub
|
||||
STYLE_CONFIGS = {
|
||||
"awesome": (FIXTURES_DIR / "awesome-root.html", REPO_ROOT / "README.md"),
|
||||
"classic": (
|
||||
FIXTURES_DIR / "classic-non-root.html",
|
||||
REPO_ROOT / "README_ALTERNATIVES" / "README_CLASSIC.md",
|
||||
),
|
||||
"extra": (
|
||||
FIXTURES_DIR / "extra-non-root.html",
|
||||
REPO_ROOT / "README_ALTERNATIVES" / "README_EXTRA.md",
|
||||
),
|
||||
"flat": (
|
||||
FIXTURES_DIR / "flat-non-root.html",
|
||||
REPO_ROOT / "README_ALTERNATIVES" / "README_FLAT_ALL_AZ.md",
|
||||
),
|
||||
}
|
||||
|
||||
DEFAULT_HTML_PATH = STYLE_CONFIGS["awesome"][0]
|
||||
DEFAULT_README_PATH = STYLE_CONFIGS["awesome"][1]
|
||||
|
||||
|
||||
def extract_github_anchor_ids(html_content: str) -> set[str]:
|
||||
"""Extract heading anchor IDs from GitHub-rendered HTML.
|
||||
|
||||
GitHub prefixes heading IDs with 'user-content-' in the rendered HTML.
|
||||
"""
|
||||
pattern = r'id="user-content-([^"]*)"'
|
||||
matches = re.findall(pattern, html_content)
|
||||
return set(matches)
|
||||
|
||||
|
||||
def extract_toc_anchors_from_readme(readme_content: str) -> set[str]:
|
||||
"""Extract TOC anchor links from README markdown.
|
||||
|
||||
Handles both markdown links [text](#anchor) and HTML href="#anchor".
|
||||
"""
|
||||
# Markdown style: [text](#anchor)
|
||||
md_pattern = r"\]\(#([^)]+)\)"
|
||||
md_matches = re.findall(md_pattern, readme_content)
|
||||
|
||||
# HTML style: href="#anchor"
|
||||
html_pattern = r'href="#([^"]+)"'
|
||||
html_matches = re.findall(html_pattern, readme_content)
|
||||
|
||||
all_anchors = set(md_matches + html_matches)
|
||||
|
||||
# Filter out back-to-top links (these aren't TOC entries)
|
||||
all_anchors.discard("awesome-claude-code")
|
||||
|
||||
return all_anchors
|
||||
|
||||
|
||||
def normalize_anchor(anchor: str) -> str:
|
||||
"""Normalize anchor for comparison (URL decode)."""
|
||||
return urllib.parse.unquote(anchor)
|
||||
|
||||
|
||||
def compare_anchors(
|
||||
github_anchors: set[str],
|
||||
toc_anchors: set[str],
|
||||
) -> tuple[set[str], set[str], set[str]]:
|
||||
"""Compare GitHub anchors with TOC anchors.
|
||||
|
||||
Returns:
|
||||
Tuple of (matched, missing_in_github, extra_in_github)
|
||||
"""
|
||||
# Normalize TOC anchors (URL decode)
|
||||
toc_normalized = {normalize_anchor(a) for a in toc_anchors}
|
||||
|
||||
# Filter GitHub anchors to only include those that could be TOC entries
|
||||
# (headings in the body sections, not meta sections like "contents", "license")
|
||||
toc_relevant_github = github_anchors.copy()
|
||||
|
||||
matched = toc_normalized & toc_relevant_github
|
||||
missing_in_github = toc_normalized - toc_relevant_github
|
||||
extra_in_github = toc_relevant_github - toc_normalized
|
||||
|
||||
return matched, missing_in_github, extra_in_github
|
||||
|
||||
|
||||
def validate(
|
||||
html_path: Path = DEFAULT_HTML_PATH,
|
||||
readme_path: Path = DEFAULT_README_PATH,
|
||||
verbose: bool = True,
|
||||
) -> bool:
|
||||
"""Validate TOC anchors against GitHub HTML.
|
||||
|
||||
Returns True if validation passes, False otherwise.
|
||||
"""
|
||||
if not html_path.exists():
|
||||
print(f"❌ GitHub HTML file not found: {html_path}")
|
||||
print(" Download the rendered HTML from GitHub and save it to this path.")
|
||||
print(" See module docstring for instructions.")
|
||||
return False
|
||||
|
||||
if not readme_path.exists():
|
||||
print(f"❌ README file not found: {readme_path}")
|
||||
return False
|
||||
|
||||
html_content = html_path.read_text(encoding="utf-8")
|
||||
readme_content = readme_path.read_text(encoding="utf-8")
|
||||
|
||||
github_anchors = extract_github_anchor_ids(html_content)
|
||||
toc_anchors = extract_toc_anchors_from_readme(readme_content)
|
||||
|
||||
if verbose:
|
||||
print(f"📄 GitHub HTML: {html_path}")
|
||||
print(f"📄 README: {readme_path}")
|
||||
print(f" Found {len(github_anchors)} GitHub anchor IDs")
|
||||
print(f" Found {len(toc_anchors)} TOC anchors")
|
||||
print()
|
||||
|
||||
matched, missing_in_github, extra_in_github = compare_anchors(github_anchors, toc_anchors)
|
||||
|
||||
# Filter extra_in_github to only show content section anchors
|
||||
# (exclude meta sections that aren't in TOC)
|
||||
meta_sections = {
|
||||
"contents",
|
||||
"latest-additions",
|
||||
"contributing-",
|
||||
"license",
|
||||
"growing-thanks-to-you",
|
||||
"recommend-a-new-resource-here",
|
||||
"pick-your-style",
|
||||
"awesome-claude-code",
|
||||
}
|
||||
extra_in_github = extra_in_github - meta_sections
|
||||
|
||||
success = True
|
||||
|
||||
if missing_in_github:
|
||||
print("❌ TOC anchors NOT FOUND in GitHub HTML (BROKEN LINKS):")
|
||||
for anchor in sorted(missing_in_github):
|
||||
print(f" #{anchor}")
|
||||
success = False
|
||||
else:
|
||||
print("✅ All TOC anchors found in GitHub HTML")
|
||||
|
||||
if extra_in_github and verbose:
|
||||
print("\n⚠️ GitHub headings not in TOC (informational only):")
|
||||
for anchor in sorted(extra_in_github):
|
||||
print(f" #{anchor}")
|
||||
|
||||
if success:
|
||||
print(f"\n🎉 Validation passed! {len(matched)} TOC anchors verified.")
|
||||
|
||||
return success
|
||||
|
||||
|
||||
def generate_expected_anchors(
|
||||
html_path: Path = DEFAULT_HTML_PATH,
|
||||
output_path: Path = EXPECTED_ANCHORS_PATH,
|
||||
) -> None:
|
||||
"""Generate expected anchors file from current GitHub HTML."""
|
||||
if not html_path.exists():
|
||||
print(f"❌ GitHub HTML file not found: {html_path}")
|
||||
return
|
||||
|
||||
html_content = html_path.read_text(encoding="utf-8")
|
||||
anchors = extract_github_anchor_ids(html_content)
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_text("\n".join(sorted(anchors)) + "\n", encoding="utf-8")
|
||||
print(f"✅ Generated expected anchors file: {output_path}")
|
||||
print(f" Contains {len(anchors)} anchor IDs")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Validate TOC anchors against GitHub-rendered HTML",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--style",
|
||||
choices=list(STYLE_CONFIGS.keys()),
|
||||
default="awesome",
|
||||
help="README style to validate (default: awesome)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--html",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Path to GitHub-rendered HTML file (overrides --style)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--readme",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Path to README.md file (overrides --style)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--generate-expected",
|
||||
action="store_true",
|
||||
help="Generate expected anchors fixture file",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--quiet",
|
||||
"-q",
|
||||
action="store_true",
|
||||
help="Only show errors",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Resolve paths from style or explicit arguments
|
||||
html_path = args.html if args.html else STYLE_CONFIGS[args.style][0]
|
||||
readme_path = args.readme if args.readme else STYLE_CONFIGS[args.style][1]
|
||||
|
||||
if args.generate_expected:
|
||||
generate_expected_anchors(html_path)
|
||||
return 0
|
||||
|
||||
success = validate(html_path, readme_path, verbose=not args.quiet)
|
||||
return 0 if success else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,209 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Fetch GitHub repository data for the stock ticker banner.
|
||||
|
||||
This script queries the GitHub Search API for repositories matching
|
||||
"claude code" or "claude-code" in their name, readme, or description,
|
||||
calculates deltas compared to previous data, and saves the results to CSV.
|
||||
"""
|
||||
|
||||
import csv
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import requests
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
|
||||
def load_previous_data(csv_path: Path) -> dict[str, dict[str, int]]:
|
||||
"""
|
||||
Load previous repository data from CSV file.
|
||||
|
||||
Args:
|
||||
csv_path: Path to previous CSV file
|
||||
|
||||
Returns:
|
||||
Dictionary mapping full_name to metrics dict
|
||||
"""
|
||||
if not csv_path.exists():
|
||||
return {}
|
||||
|
||||
previous = {}
|
||||
with csv_path.open("r", encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
for row in reader:
|
||||
previous[row["full_name"]] = {
|
||||
"stars": int(row["stars"]),
|
||||
"watchers": int(row["watchers"]),
|
||||
"forks": int(row["forks"]),
|
||||
}
|
||||
|
||||
print(f"✓ Loaded {len(previous)} repositories from previous data")
|
||||
return previous
|
||||
|
||||
|
||||
def fetch_repos(token: str) -> list[dict[str, Any]]:
|
||||
"""
|
||||
Fetch repositories from GitHub Search API.
|
||||
|
||||
Args:
|
||||
token: GitHub authentication token
|
||||
|
||||
Returns:
|
||||
List of repository data dictionaries
|
||||
"""
|
||||
# GitHub Search API endpoint
|
||||
url = "https://api.github.com/search/repositories"
|
||||
|
||||
# Search query
|
||||
query = '"claude code" claude-code in:name,readme,description'
|
||||
|
||||
# Parameters for the API request
|
||||
params: dict[str, str | int] = {
|
||||
"q": query,
|
||||
"per_page": 100, # Maximum results per page
|
||||
"page": 1,
|
||||
"sort": "relevance", # Sort by relevance (default)
|
||||
}
|
||||
|
||||
# Headers with authentication
|
||||
headers = {
|
||||
"Accept": "application/vnd.github+json",
|
||||
"Authorization": f"Bearer {token}",
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
}
|
||||
|
||||
try:
|
||||
response = requests.get(url, params=params, headers=headers, timeout=30)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
repos = []
|
||||
for item in data.get("items", []):
|
||||
repos.append(
|
||||
{
|
||||
"full_name": item["full_name"],
|
||||
"stars": item["stargazers_count"],
|
||||
"watchers": item["watchers_count"],
|
||||
"forks": item["forks_count"],
|
||||
"url": item["html_url"],
|
||||
}
|
||||
)
|
||||
|
||||
print(f"✓ Fetched {len(repos)} repositories from search")
|
||||
return repos
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
print(f"✗ Error fetching data from GitHub API: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def calculate_deltas(
|
||||
repos: list[dict[str, Any]],
|
||||
previous: dict[str, dict[str, int]],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""
|
||||
Calculate deltas for each repository compared to previous data.
|
||||
|
||||
For repos not in previous data:
|
||||
- If there is no previous baseline at all, set deltas to 0.
|
||||
- Otherwise, treat the repo as new and set deltas to current values.
|
||||
|
||||
Args:
|
||||
repos: List of current repository data
|
||||
previous: Dictionary of previous repository data
|
||||
Returns:
|
||||
List of repository data with deltas
|
||||
"""
|
||||
repos_with_deltas = []
|
||||
has_previous = bool(previous)
|
||||
|
||||
for repo in repos:
|
||||
full_name = repo["full_name"]
|
||||
|
||||
if full_name in previous:
|
||||
# Calculate deltas from previous data
|
||||
prev = previous[full_name]
|
||||
repo["stars_delta"] = repo["stars"] - prev["stars"]
|
||||
repo["watchers_delta"] = repo["watchers"] - prev["watchers"]
|
||||
repo["forks_delta"] = repo["forks"] - prev["forks"]
|
||||
else:
|
||||
# New repo vs previous snapshot.
|
||||
# If there is no prior snapshot, use 0 deltas as a baseline.
|
||||
if not has_previous:
|
||||
repo["stars_delta"] = 0
|
||||
repo["watchers_delta"] = 0
|
||||
repo["forks_delta"] = 0
|
||||
else:
|
||||
repo["stars_delta"] = repo["stars"]
|
||||
repo["watchers_delta"] = repo["watchers"]
|
||||
repo["forks_delta"] = repo["forks"]
|
||||
|
||||
repos_with_deltas.append(repo)
|
||||
|
||||
return repos_with_deltas
|
||||
|
||||
|
||||
def save_to_csv(repos: list[dict[str, Any]], output_path: Path) -> None:
|
||||
"""
|
||||
Save repository data to CSV file.
|
||||
|
||||
Args:
|
||||
repos: List of repository data dictionaries
|
||||
output_path: Path to output CSV file
|
||||
"""
|
||||
# Create data directory if it doesn't exist
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Write CSV
|
||||
with output_path.open("w", newline="", encoding="utf-8") as f:
|
||||
fieldnames = [
|
||||
"full_name",
|
||||
"stars",
|
||||
"watchers",
|
||||
"forks",
|
||||
"stars_delta",
|
||||
"watchers_delta",
|
||||
"forks_delta",
|
||||
"url",
|
||||
]
|
||||
writer = csv.DictWriter(f, fieldnames=fieldnames)
|
||||
|
||||
writer.writeheader()
|
||||
writer.writerows(repos)
|
||||
|
||||
print(f"✓ Saved {len(repos)} repositories to {output_path}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Main function to orchestrate the data fetching and saving."""
|
||||
# Get GitHub token from environment
|
||||
token = os.environ.get("GITHUB_TOKEN")
|
||||
if not token:
|
||||
print("✗ GITHUB_TOKEN environment variable not set", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Load previous data
|
||||
previous_path = REPO_ROOT / "data" / "repo-ticker-previous.csv"
|
||||
previous_data = load_previous_data(previous_path)
|
||||
|
||||
# Fetch repository data
|
||||
print("Fetching repository data from GitHub API...")
|
||||
repos = fetch_repos(token)
|
||||
|
||||
# Calculate deltas
|
||||
print("Calculating deltas...")
|
||||
repos_with_deltas = calculate_deltas(repos, previous_data)
|
||||
|
||||
# Save to CSV
|
||||
output_path = REPO_ROOT / "data" / "repo-ticker.csv"
|
||||
save_to_csv(repos_with_deltas, output_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,727 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate dynamic stock ticker SVGs from repository data.
|
||||
|
||||
This script reads the repo ticker CSV and generates animated SVG files
|
||||
with a random sampling of repositories for both dark and light themes.
|
||||
Displays deltas for each metric with color coding.
|
||||
"""
|
||||
|
||||
import csv
|
||||
import os
|
||||
import random
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import requests
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
|
||||
_STAR_DELTA_CACHE: dict[str, int] = {}
|
||||
|
||||
|
||||
def fetch_recent_star_delta(
|
||||
full_name: str, token: str, since: datetime, cache: dict[str, int]
|
||||
) -> int | None:
|
||||
"""Fetch the number of stars added since the cutoff time."""
|
||||
if full_name in cache:
|
||||
return cache[full_name]
|
||||
|
||||
owner, repo = full_name.split("/", 1)
|
||||
headers = {
|
||||
"Accept": "application/vnd.github+json",
|
||||
"Authorization": f"Bearer {token}",
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
}
|
||||
query = """
|
||||
query($owner: String!, $name: String!, $cursor: String) {
|
||||
repository(owner: $owner, name: $name) {
|
||||
stargazers(first: 100, after: $cursor, orderBy: {field: STARRED_AT, direction: DESC}) {
|
||||
edges { starredAt }
|
||||
pageInfo { hasNextPage endCursor }
|
||||
}
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
delta = 0
|
||||
cursor: str | None = None
|
||||
cutoff = since.astimezone(UTC)
|
||||
|
||||
while True:
|
||||
payload = {"query": query, "variables": {"owner": owner, "name": repo, "cursor": cursor}}
|
||||
response = requests.post(
|
||||
"https://api.github.com/graphql", json=payload, headers=headers, timeout=20
|
||||
)
|
||||
if response.status_code != 200:
|
||||
return None
|
||||
data = response.json()
|
||||
if "errors" in data:
|
||||
return None
|
||||
|
||||
stargazers = data.get("data", {}).get("repository", {}).get("stargazers", {})
|
||||
edges = stargazers.get("edges", [])
|
||||
page_info = stargazers.get("pageInfo", {})
|
||||
|
||||
for edge in edges:
|
||||
starred_at = edge.get("starredAt")
|
||||
if not starred_at:
|
||||
continue
|
||||
starred_dt = datetime.fromisoformat(starred_at.replace("Z", "+00:00"))
|
||||
if starred_dt < cutoff:
|
||||
cache[full_name] = delta
|
||||
return delta
|
||||
delta += 1
|
||||
|
||||
if not page_info.get("hasNextPage"):
|
||||
break
|
||||
cursor = page_info.get("endCursor")
|
||||
|
||||
cache[full_name] = delta
|
||||
return delta
|
||||
|
||||
|
||||
def apply_recent_star_deltas(repos: list[dict[str, Any]]) -> None:
|
||||
"""Replace stars_delta with counts from the last 24 hours when possible."""
|
||||
token = os.getenv("GITHUB_TOKEN", "").strip()
|
||||
if not token:
|
||||
return
|
||||
|
||||
cutoff = datetime.now(UTC) - timedelta(days=1)
|
||||
for repo in repos:
|
||||
delta = fetch_recent_star_delta(repo["full_name"], token, cutoff, _STAR_DELTA_CACHE)
|
||||
if delta is not None:
|
||||
repo["stars_delta"] = delta
|
||||
|
||||
|
||||
def format_number(num: int) -> str:
|
||||
"""
|
||||
Format a number with K/M suffix for display.
|
||||
|
||||
Args:
|
||||
num: The number to format
|
||||
|
||||
Returns:
|
||||
Formatted string (e.g., "1.2K", "15.3K")
|
||||
"""
|
||||
if num >= 1_000_000:
|
||||
return f"{num / 1_000_000:.1f}M"
|
||||
elif num >= 1000:
|
||||
return f"{num / 1000:.1f}K"
|
||||
else:
|
||||
return str(num)
|
||||
|
||||
|
||||
def format_delta(delta: int) -> str:
|
||||
"""
|
||||
Format a delta with +/- prefix.
|
||||
|
||||
Args:
|
||||
delta: The delta value
|
||||
|
||||
Returns:
|
||||
Formatted string (e.g., "+5", "-2", "0")
|
||||
"""
|
||||
if delta > 0:
|
||||
return f"+{format_number(delta)}"
|
||||
elif delta < 0:
|
||||
return format_number(delta)
|
||||
else:
|
||||
return "0"
|
||||
|
||||
|
||||
def truncate_repo_name(name: str, max_length: int = 20) -> str:
|
||||
"""
|
||||
Truncate a repository name if it exceeds max_length.
|
||||
|
||||
Args:
|
||||
name: The repository name to truncate
|
||||
max_length: Maximum length before truncation (default: 20)
|
||||
|
||||
Returns:
|
||||
Truncated string with ellipsis if needed (e.g., "very-long-repositor...")
|
||||
"""
|
||||
if len(name) <= max_length:
|
||||
return name
|
||||
return name[:max_length] + "..."
|
||||
|
||||
|
||||
def get_delta_color(delta: int, colors: dict[str, str]) -> str:
|
||||
"""
|
||||
Get color for delta based on value.
|
||||
|
||||
Args:
|
||||
delta: The delta value
|
||||
colors: Color scheme dictionary
|
||||
|
||||
Returns:
|
||||
Color string
|
||||
"""
|
||||
if delta > 0:
|
||||
return colors["delta_positive"]
|
||||
elif delta < 0:
|
||||
return colors["delta_negative"]
|
||||
else:
|
||||
return colors["delta_neutral"]
|
||||
|
||||
|
||||
def load_repos(csv_path: Path) -> list[dict[str, Any]]:
|
||||
"""
|
||||
Load repository data from CSV file.
|
||||
|
||||
Args:
|
||||
csv_path: Path to CSV file
|
||||
|
||||
Returns:
|
||||
List of repository dictionaries
|
||||
"""
|
||||
repos = []
|
||||
with csv_path.open("r", encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
for row in reader:
|
||||
repos.append(
|
||||
{
|
||||
"full_name": row["full_name"],
|
||||
"stars": int(row["stars"]),
|
||||
"watchers": int(row["watchers"]),
|
||||
"forks": int(row["forks"]),
|
||||
"stars_delta": int(row.get("stars_delta", 0)),
|
||||
"watchers_delta": int(row.get("watchers_delta", 0)),
|
||||
"forks_delta": int(row.get("forks_delta", 0)),
|
||||
}
|
||||
)
|
||||
return repos
|
||||
|
||||
|
||||
def generate_repo_group(
|
||||
repo: dict[str, Any], x_offset: int, colors: dict[str, str], flip: bool
|
||||
) -> str:
|
||||
"""
|
||||
Generate SVG group element for a single repository.
|
||||
|
||||
Args:
|
||||
repo: Repository data
|
||||
x_offset: X position offset for this group
|
||||
colors: Color scheme dictionary
|
||||
|
||||
Returns:
|
||||
SVG group element as string
|
||||
"""
|
||||
# Format deltas (commented out - unused but may be needed later)
|
||||
# stars_delta = format_delta(repo["stars_delta"])
|
||||
# watchers_delta = format_delta(repo["watchers_delta"])
|
||||
# forks_delta = format_delta(repo["forks_delta"])
|
||||
|
||||
# Get delta colors (commented out - unused but may be needed later)
|
||||
# stars_delta_color = get_delta_color(repo["stars_delta"], colors)
|
||||
# watchers_delta_color = get_delta_color(repo["watchers_delta"], colors)
|
||||
# forks_delta_color = get_delta_color(repo["forks_delta"], colors)
|
||||
|
||||
# Split full_name into owner and repo
|
||||
parts = repo["full_name"].split("/", 1)
|
||||
owner = parts[0] if len(parts) > 0 else ""
|
||||
repo_name = parts[1] if len(parts) > 1 else ""
|
||||
|
||||
# Truncate repo name to avoid overlap with long names (slightly longer cutoff for mobile)
|
||||
truncated_repo_name = truncate_repo_name(repo_name, max_length=24)
|
||||
|
||||
# Stars snippet placed after owner
|
||||
delta_text = format_delta(repo["stars_delta"])
|
||||
show_delta = delta_text != "0"
|
||||
approx_char_width = 12 # rough monospace estimate for 24px owner font
|
||||
owner_start_x = 140
|
||||
owner_font_size = 24
|
||||
|
||||
def star_snippet(y_pos: int) -> str:
|
||||
star_str = f"{format_number(repo['stars'])} ⭐"
|
||||
delta_str = f" {delta_text}" if show_delta else ""
|
||||
metrics = f" | {star_str}{delta_str}"
|
||||
star_x = owner_start_x + (len(owner) * approx_char_width) + 22
|
||||
return f"""
|
||||
<text x="{star_x}" y="{y_pos}" font-family="'Courier New', monospace" font-size="16" font-weight="bold"
|
||||
fill="{colors["stars"]}">{metrics}</text>"""
|
||||
|
||||
if not flip:
|
||||
# Names on top, owner just below
|
||||
return f""" <!-- Repo: {repo["full_name"]} -->
|
||||
<g transform="translate({x_offset}, 0)">
|
||||
<!-- Repo name -->
|
||||
<text x="140" y="32" font-family="'Courier New', monospace" font-size="34" font-weight="bold"
|
||||
fill="{colors["text"]}">{truncated_repo_name}</text>
|
||||
<!-- Owner name -->
|
||||
<text x="{owner_start_x}" y="64" font-family="'Courier New', monospace" font-size="{owner_font_size}" font-weight="normal"
|
||||
fill="{colors["text"]}" opacity="1.0">{owner}</text>{star_snippet(64)}
|
||||
</g>"""
|
||||
else:
|
||||
# Owner on top, name just below (in lower half)
|
||||
return f""" <!-- Repo: {repo["full_name"]} -->
|
||||
<g transform="translate({x_offset}, 0)">
|
||||
<!-- Owner name -->
|
||||
<text x="{owner_start_x}" y="102" font-family="'Courier New', monospace" font-size="{owner_font_size}" font-weight="normal"
|
||||
fill="{colors["text"]}" opacity="1.0">{owner}</text>{star_snippet(102)}
|
||||
<!-- Repo name -->
|
||||
<text x="140" y="132" font-family="'Courier New', monospace" font-size="34" font-weight="bold"
|
||||
fill="{colors["text"]}">{truncated_repo_name}</text>
|
||||
</g>"""
|
||||
|
||||
|
||||
def generate_ticker_svg(repos: list[dict[str, Any]], theme: str = "dark") -> str:
|
||||
"""
|
||||
Generate complete ticker SVG.
|
||||
|
||||
Args:
|
||||
repos: List of repository data
|
||||
theme: "dark" or "light"
|
||||
|
||||
Returns:
|
||||
Complete SVG as string
|
||||
"""
|
||||
# Filter out repos owned by hesreallyhim
|
||||
filtered_repos = [r for r in repos if not r["full_name"].startswith("hesreallyhim/")]
|
||||
|
||||
# Sample 10 random repos and duplicate for seamless scrolling
|
||||
sampled = random.sample(filtered_repos, min(10, len(filtered_repos)))
|
||||
apply_recent_star_deltas(sampled)
|
||||
|
||||
# Color schemes
|
||||
if theme == "dark":
|
||||
colors = {
|
||||
"bg_start": "#001a00",
|
||||
"bg_mid": "#002200",
|
||||
"bg_opacity_start": "0.95",
|
||||
"bg_opacity_mid": "0.98",
|
||||
"border_1": "#33ff33",
|
||||
"border_2": "#00ffff",
|
||||
"border_3": "#66ff66",
|
||||
"border_4": "#00ff99",
|
||||
"label_bg": "#001100",
|
||||
"label_title": "#33ff33",
|
||||
"label_subtitle": "#00ffff",
|
||||
"pulse": "#33ff33",
|
||||
"text": "#ffffff",
|
||||
"stars": "#00ffff",
|
||||
"watchers": "#66ff66",
|
||||
"forks": "#00ff99",
|
||||
"fade_color": "#001a00",
|
||||
"delta_positive": "#33ff33",
|
||||
"delta_negative": "#ff3333",
|
||||
"delta_neutral": "#888888",
|
||||
"glow_blur": "0.1",
|
||||
}
|
||||
else: # light
|
||||
colors = {
|
||||
"bg_start": "#fff8f0",
|
||||
"bg_mid": "#fff5eb",
|
||||
"bg_opacity_start": "0.98",
|
||||
"bg_opacity_mid": "1",
|
||||
"border_1": "#FF6B35",
|
||||
"border_2": "#9C4EFF",
|
||||
"border_3": "#FFD700",
|
||||
"border_4": "#FF6B35",
|
||||
"label_bg": "#fff0e6",
|
||||
"label_title": "#FF6B35",
|
||||
"label_subtitle": "#9C4EFF",
|
||||
"pulse": "#FF6B35",
|
||||
"text": "#2d2d2d",
|
||||
"stars": "#9C4EFF",
|
||||
"watchers": "#FF6B35",
|
||||
"forks": "#FFD700",
|
||||
"fade_color": "#fff8f0",
|
||||
"delta_positive": "#00aa00",
|
||||
"delta_negative": "#cc0000",
|
||||
"delta_neutral": "#888888",
|
||||
"glow_blur": "0.15",
|
||||
}
|
||||
|
||||
# Generate repo groups: all 10 + first 4 repeated for seamless loop
|
||||
repo_groups = []
|
||||
x_pos = 0
|
||||
|
||||
# All sampled repos
|
||||
for idx, repo in enumerate(sampled):
|
||||
group_svg = generate_repo_group(repo, x_pos, colors, flip=bool(idx % 2))
|
||||
repo_groups.append(group_svg)
|
||||
x_pos += 300 # Space between repos (compact stock ticker style)
|
||||
|
||||
# Primary content width (what we scroll through before looping)
|
||||
primary_width = x_pos
|
||||
|
||||
# Append first 4 repos to fill the visible gap during loop reset
|
||||
for idx, repo in enumerate(sampled[:4]):
|
||||
group_svg = generate_repo_group(repo, x_pos, colors, flip=bool(idx % 2))
|
||||
repo_groups.append(group_svg)
|
||||
x_pos += 300
|
||||
|
||||
repos_svg = "\n".join(repo_groups)
|
||||
|
||||
# Calculate animation duration based on primary content (10 repos)
|
||||
duration = max(28, primary_width // 55) # slightly slower to aid legibility on mobile
|
||||
|
||||
return f"""<svg width="900" height="150" xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<!-- Gradient for ticker background -->
|
||||
<linearGradient id="tickerBg" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" style="stop-color:{colors["bg_start"]};stop-opacity:{colors["bg_opacity_start"]}"/>
|
||||
<stop offset="50%" style="stop-color:{colors["bg_mid"]};stop-opacity:{colors["bg_opacity_mid"]}"/>
|
||||
<stop offset="100%" style="stop-color:{colors["bg_start"]};stop-opacity:{colors["bg_opacity_start"]}"/>
|
||||
</linearGradient>
|
||||
|
||||
<!-- Gradient for border lines -->
|
||||
<linearGradient id="borderGrad" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" style="stop-color:{colors["border_1"]};stop-opacity:0.85">
|
||||
<animate attributeName="stop-opacity" values="0.85;1;0.85" dur="2s" repeatCount="indefinite"/>
|
||||
</stop>
|
||||
<stop offset="33%" style="stop-color:{colors["border_2"]};stop-opacity:0.9">
|
||||
<animate attributeName="stop-opacity" values="0.9;1;0.9" dur="2.2s" repeatCount="indefinite"/>
|
||||
</stop>
|
||||
<stop offset="66%" style="stop-color:{colors["border_3"]};stop-opacity:0.85">
|
||||
<animate attributeName="stop-opacity" values="0.85;1;0.85" dur="1.8s" repeatCount="indefinite"/>
|
||||
</stop>
|
||||
<stop offset="100%" style="stop-color:{colors["border_4"]};stop-opacity:0.85">
|
||||
<animate attributeName="stop-opacity" values="0.85;1;0.85" dur="2s" repeatCount="indefinite"/>
|
||||
</stop>
|
||||
</linearGradient>
|
||||
|
||||
<!-- Text glow effect -->
|
||||
<filter id="textGlow">
|
||||
<feGaussianBlur stdDeviation="{colors["glow_blur"]}" result="coloredBlur"/>
|
||||
<feMerge>
|
||||
<feMergeNode in="coloredBlur"/>
|
||||
<feMergeNode in="SourceGraphic"/>
|
||||
</feMerge>
|
||||
</filter>
|
||||
|
||||
<!-- Strong glow for metrics -->
|
||||
<filter id="metricGlow">
|
||||
<feGaussianBlur stdDeviation="0.2" result="coloredBlur"/>
|
||||
<feMerge>
|
||||
<feMergeNode in="coloredBlur"/>
|
||||
<feMergeNode in="SourceGraphic"/>
|
||||
</feMerge>
|
||||
</filter>
|
||||
|
||||
<!-- Edge fade effects -->
|
||||
<linearGradient id="leftFade" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" style="stop-color:{colors["fade_color"]};stop-opacity:1"/>
|
||||
<stop offset="100%" style="stop-color:{colors["fade_color"]};stop-opacity:0"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="rightFade" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" style="stop-color:{colors["fade_color"]};stop-opacity:0"/>
|
||||
<stop offset="100%" style="stop-color:{colors["fade_color"]};stop-opacity:1"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<!-- Background panel -->
|
||||
<rect width="900" height="150" fill="url(#tickerBg)" rx="8"/>
|
||||
|
||||
<!-- Top border -->
|
||||
<rect x="0" y="2" width="900" height="2" fill="url(#borderGrad)" rx="1"/>
|
||||
|
||||
<!-- Bottom border -->
|
||||
<rect x="0" y="146" width="900" height="2" fill="url(#borderGrad)" rx="1"/>
|
||||
|
||||
<!-- Midline for grouping reference -->
|
||||
<line x1="0" y1="75" x2="900" y2="75" stroke="{colors["border_2"]}" stroke-width="2" stroke-dasharray="8 6" opacity="0.6"/>
|
||||
|
||||
<!-- Ticker label on left -->
|
||||
<rect x="0" y="0" width="120" height="150" fill="{colors["label_bg"]}" opacity="0.95" rx="8"/>
|
||||
<text x="60" y="46" font-family="'Courier New', monospace" font-size="18" font-weight="bold"
|
||||
fill="{colors["label_title"]}" text-anchor="middle" filter="url(#textGlow)">
|
||||
CLAUDE CODE
|
||||
</text>
|
||||
<text x="60" y="68" font-family="'Courier New', monospace" font-size="18" font-weight="bold"
|
||||
fill="{colors["label_subtitle"]}" text-anchor="middle" filter="url(#textGlow)">
|
||||
REPOS LIVE
|
||||
</text>
|
||||
<text x="60" y="92" font-family="'Courier New', monospace" font-size="14" font-weight="bold"
|
||||
fill="{colors["delta_positive"]}" text-anchor="middle">
|
||||
DAILY Δ
|
||||
</text>
|
||||
|
||||
<!-- Animated pulse indicator -->
|
||||
<circle cx="60" cy="118" r="5" fill="{colors["pulse"]}" filter="url(#metricGlow)">
|
||||
<animate attributeName="r" values="4;6;4" dur="1.5s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0.6;1;0.6" dur="1.5s" repeatCount="indefinite"/>
|
||||
</circle>
|
||||
|
||||
<!-- Divider line after label -->
|
||||
<rect x="122" y="18" width="2" height="114" fill="url(#borderGrad)" rx="1"/>
|
||||
|
||||
<!-- Scrolling ticker content area -->
|
||||
<clipPath id="tickerClip">
|
||||
<rect x="130" y="0" width="770" height="150"/>
|
||||
</clipPath>
|
||||
|
||||
<g clip-path="url(#tickerClip)">
|
||||
<!-- Single ticker strip with seamless loop (10 repos + first 4 repeated) -->
|
||||
<g id="tickerItems">
|
||||
<animateTransform
|
||||
attributeName="transform"
|
||||
attributeType="XML"
|
||||
type="translate"
|
||||
from="0 0"
|
||||
to="-{primary_width} 0"
|
||||
dur="{duration}s"
|
||||
repeatCount="indefinite"/>
|
||||
|
||||
{repos_svg}
|
||||
</g>
|
||||
</g>
|
||||
|
||||
<!-- Edge fade effects -->
|
||||
<rect x="130" y="0" width="50" height="100" fill="url(#leftFade)"/>
|
||||
<rect x="850" y="0" width="50" height="100" fill="url(#rightFade)"/>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def generate_awesome_repo_group(repo: dict[str, Any], x_offset: int, flip: bool) -> str:
|
||||
"""
|
||||
Generate SVG group element for a single repository in awesome style.
|
||||
Uses same layout as original ticker but with clean, minimal styling.
|
||||
|
||||
Args:
|
||||
repo: Repository data
|
||||
x_offset: X position offset for this group
|
||||
flip: Whether to flip the layout (owner on top vs bottom)
|
||||
|
||||
Returns:
|
||||
SVG group element as string
|
||||
"""
|
||||
# Split full_name into owner and repo
|
||||
parts = repo["full_name"].split("/", 1)
|
||||
owner = parts[0] if len(parts) > 0 else ""
|
||||
repo_name = parts[1] if len(parts) > 1 else ""
|
||||
|
||||
# Truncate repo name (same as original)
|
||||
truncated_repo_name = truncate_repo_name(repo_name, max_length=24)
|
||||
|
||||
# Stars snippet - same layout as original but with clean colors
|
||||
delta_text = format_delta(repo["stars_delta"])
|
||||
show_delta = delta_text != "0"
|
||||
approx_char_width = 12
|
||||
owner_start_x = 140
|
||||
owner_font_size = 24
|
||||
|
||||
# Color scheme - clean, muted
|
||||
text_color = "#24292e" # GitHub dark
|
||||
owner_color = "#586069" # GitHub secondary
|
||||
stars_color = "#6a737d" # Muted gray
|
||||
delta_positive = "#22863a"
|
||||
delta_negative = "#cb2431"
|
||||
|
||||
# Delta color
|
||||
if repo["stars_delta"] > 0:
|
||||
delta_color = delta_positive
|
||||
elif repo["stars_delta"] < 0:
|
||||
delta_color = delta_negative
|
||||
else:
|
||||
delta_color = stars_color
|
||||
|
||||
def star_snippet(y_pos: int) -> str:
|
||||
star_str = f"{format_number(repo['stars'])} ★"
|
||||
delta_str = f" {delta_text}" if show_delta else ""
|
||||
star_x = owner_start_x + (len(owner) * approx_char_width) + 22
|
||||
# Stars in muted gray, delta in appropriate color
|
||||
result = f"""
|
||||
<text x="{star_x}" y="{y_pos}" font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="16" font-weight="500"
|
||||
fill="{stars_color}">| {star_str}</text>"""
|
||||
if show_delta:
|
||||
delta_x = star_x + (len(f"| {star_str}") * 9) + 5
|
||||
result += f"""
|
||||
<text x="{delta_x}" y="{y_pos}" font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="16" font-weight="500"
|
||||
fill="{delta_color}">{delta_str}</text>"""
|
||||
return result
|
||||
|
||||
font_family = "-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif"
|
||||
|
||||
if not flip:
|
||||
# Names on top, owner just below
|
||||
return f""" <!-- Repo: {repo["full_name"]} -->
|
||||
<g transform="translate({x_offset}, 0)">
|
||||
<!-- Repo name -->
|
||||
<text x="140" y="32" font-family="{font_family}" font-size="34" font-weight="600"
|
||||
fill="{text_color}">{truncated_repo_name}</text>
|
||||
<!-- Owner name -->
|
||||
<text x="{owner_start_x}" y="64" font-family="{font_family}" font-size="{owner_font_size}" font-weight="400"
|
||||
fill="{owner_color}">{owner}</text>{star_snippet(64)}
|
||||
</g>"""
|
||||
else:
|
||||
# Owner on top, name just below (in lower half)
|
||||
return f""" <!-- Repo: {repo["full_name"]} -->
|
||||
<g transform="translate({x_offset}, 0)">
|
||||
<!-- Owner name -->
|
||||
<text x="{owner_start_x}" y="102" font-family="{font_family}" font-size="{owner_font_size}" font-weight="400"
|
||||
fill="{owner_color}">{owner}</text>{star_snippet(102)}
|
||||
<!-- Repo name -->
|
||||
<text x="140" y="132" font-family="{font_family}" font-size="34" font-weight="600"
|
||||
fill="{text_color}">{truncated_repo_name}</text>
|
||||
</g>"""
|
||||
|
||||
|
||||
def generate_awesome_ticker_svg(repos: list[dict[str, Any]]) -> str:
|
||||
"""
|
||||
Generate awesome-style ticker SVG - same layout as original but clean, minimal styling.
|
||||
|
||||
Args:
|
||||
repos: List of repository data
|
||||
|
||||
Returns:
|
||||
Complete SVG as string
|
||||
"""
|
||||
# Filter out repos owned by hesreallyhim
|
||||
filtered_repos = [r for r in repos if not r["full_name"].startswith("hesreallyhim/")]
|
||||
|
||||
# Sample 10 random repos (same as regular ticker)
|
||||
sampled = random.sample(filtered_repos, min(10, len(filtered_repos)))
|
||||
apply_recent_star_deltas(sampled)
|
||||
|
||||
# Generate repo groups with alternating layout (same as original)
|
||||
repo_groups = []
|
||||
x_pos = 0
|
||||
|
||||
for idx, repo in enumerate(sampled):
|
||||
group_svg = generate_awesome_repo_group(repo, x_pos, flip=bool(idx % 2))
|
||||
repo_groups.append(group_svg)
|
||||
x_pos += 300 # Same spacing as original
|
||||
|
||||
primary_width = x_pos
|
||||
|
||||
# Append first 4 repos for seamless loop (same as original)
|
||||
for idx, repo in enumerate(sampled[:4]):
|
||||
group_svg = generate_awesome_repo_group(repo, x_pos, flip=bool(idx % 2))
|
||||
repo_groups.append(group_svg)
|
||||
x_pos += 300
|
||||
|
||||
repos_svg = "\n".join(repo_groups)
|
||||
|
||||
# Animation duration (same calculation as original)
|
||||
duration = max(28, primary_width // 55)
|
||||
|
||||
# Colors
|
||||
bg_color = "#ffffff"
|
||||
border_color = "#e1e4e8"
|
||||
label_bg = "#f6f8fa"
|
||||
label_title = "#24292e"
|
||||
label_subtitle = "#586069"
|
||||
fade_color = "#ffffff"
|
||||
|
||||
return f"""<svg width="900" height="150" xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<!-- Edge fade effects -->
|
||||
<linearGradient id="awesomeLeftFade" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" style="stop-color:{fade_color};stop-opacity:1"/>
|
||||
<stop offset="100%" style="stop-color:{fade_color};stop-opacity:0"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="awesomeRightFade" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" style="stop-color:{fade_color};stop-opacity:0"/>
|
||||
<stop offset="100%" style="stop-color:{fade_color};stop-opacity:1"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<!-- Clean background -->
|
||||
<rect width="900" height="150" fill="{bg_color}" rx="8"/>
|
||||
|
||||
<!-- Top border -->
|
||||
<rect x="0" y="2" width="900" height="1" fill="{border_color}"/>
|
||||
|
||||
<!-- Bottom border -->
|
||||
<rect x="0" y="147" width="900" height="1" fill="{border_color}"/>
|
||||
|
||||
<!-- Midline for grouping reference -->
|
||||
<line x1="0" y1="75" x2="900" y2="75" stroke="{border_color}" stroke-width="1" stroke-dasharray="8 6" opacity="0.6"/>
|
||||
|
||||
<!-- Ticker label on left -->
|
||||
<rect x="0" y="0" width="120" height="150" fill="{label_bg}" rx="8"/>
|
||||
<text x="60" y="50" font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="14" font-weight="600"
|
||||
fill="{label_title}" text-anchor="middle">CLAUDE CODE</text>
|
||||
<text x="60" y="72" font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="14" font-weight="600"
|
||||
fill="{label_subtitle}" text-anchor="middle">PROJECTS</text>
|
||||
<text x="60" y="100" font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12" font-weight="500"
|
||||
fill="{label_subtitle}" text-anchor="middle">Daily Δ</text>
|
||||
|
||||
<!-- Simple indicator dot -->
|
||||
<circle cx="60" cy="122" r="4" fill="#22863a" opacity="0.8"/>
|
||||
|
||||
<!-- Divider line after label -->
|
||||
<rect x="122" y="18" width="1" height="114" fill="{border_color}"/>
|
||||
|
||||
<!-- Scrolling ticker content area -->
|
||||
<clipPath id="awesomeTickerClip">
|
||||
<rect x="130" y="0" width="770" height="150"/>
|
||||
</clipPath>
|
||||
|
||||
<g clip-path="url(#awesomeTickerClip)">
|
||||
<!-- Single ticker strip with seamless loop -->
|
||||
<g id="awesomeTickerItems">
|
||||
<animateTransform
|
||||
attributeName="transform"
|
||||
attributeType="XML"
|
||||
type="translate"
|
||||
from="0 0"
|
||||
to="-{primary_width} 0"
|
||||
dur="{duration}s"
|
||||
repeatCount="indefinite"/>
|
||||
|
||||
{repos_svg}
|
||||
</g>
|
||||
</g>
|
||||
|
||||
<!-- Edge fade effects -->
|
||||
<rect x="130" y="0" width="50" height="150" fill="url(#awesomeLeftFade)"/>
|
||||
<rect x="850" y="0" width="50" height="150" fill="url(#awesomeRightFade)"/>
|
||||
</svg>"""
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Main function to generate ticker SVGs."""
|
||||
csv_path = REPO_ROOT / "data" / "repo-ticker.csv"
|
||||
output_dir = REPO_ROOT / "assets"
|
||||
|
||||
# Check if CSV exists
|
||||
if not csv_path.exists():
|
||||
print(f"⚠ CSV file not found at {csv_path}")
|
||||
print("⚠ Using static ticker SVGs (already created)")
|
||||
return
|
||||
|
||||
# Load repos
|
||||
print(f"Loading repository data from {csv_path}...")
|
||||
repos = load_repos(csv_path)
|
||||
print(f"✓ Loaded {len(repos)} repositories")
|
||||
|
||||
if len(repos) == 0:
|
||||
print("⚠ No repositories found in CSV")
|
||||
print("⚠ Using static ticker SVGs (already created)")
|
||||
return
|
||||
|
||||
# Generate SVGs
|
||||
print("Generating ticker SVGs...")
|
||||
|
||||
# Dark theme
|
||||
dark_svg = generate_ticker_svg(repos, "dark")
|
||||
dark_path = output_dir / "repo-ticker.svg"
|
||||
with dark_path.open("w", encoding="utf-8") as f:
|
||||
f.write(dark_svg)
|
||||
print(f"✓ Generated dark theme: {dark_path}")
|
||||
|
||||
# Light theme
|
||||
light_svg = generate_ticker_svg(repos, "light")
|
||||
light_path = output_dir / "repo-ticker-light.svg"
|
||||
with light_path.open("w", encoding="utf-8") as f:
|
||||
f.write(light_svg)
|
||||
print(f"✓ Generated light theme: {light_path}")
|
||||
|
||||
# Awesome theme (clean, minimal)
|
||||
awesome_svg = generate_awesome_ticker_svg(repos)
|
||||
awesome_path = output_dir / "repo-ticker-awesome.svg"
|
||||
with awesome_path.open("w", encoding="utf-8") as f:
|
||||
f.write(awesome_svg)
|
||||
print(f"✓ Generated awesome theme: {awesome_path}")
|
||||
|
||||
print("✓ Ticker SVGs generated successfully!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
269
.agent/knowledge/awesome_claude/scripts/utils/git_utils.py
Normal file
269
.agent/knowledge/awesome_claude/scripts/utils/git_utils.py
Normal file
@@ -0,0 +1,269 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Git and GitHub utility functions for command-line operations."""
|
||||
|
||||
import logging
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class GitUtils:
|
||||
"""Handle git and GitHub operations."""
|
||||
|
||||
def __init__(self, logger: logging.Logger | None = None):
|
||||
"""
|
||||
Initialize GitUtils.
|
||||
|
||||
Args:
|
||||
logger: Optional logger instance. If not provided, creates a
|
||||
default logger.
|
||||
"""
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
|
||||
def check_command_exists(self, command: str) -> bool:
|
||||
"""
|
||||
Check if a command is available in the system PATH.
|
||||
|
||||
Args:
|
||||
command: Command name to check (e.g., 'git', 'gh')
|
||||
|
||||
Returns:
|
||||
True if command exists, False otherwise
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[command, "--version"], capture_output=True, text=True, check=False
|
||||
)
|
||||
return result.returncode == 0
|
||||
except (subprocess.SubprocessError, FileNotFoundError):
|
||||
return False
|
||||
|
||||
def run_command(self, cmd: list[str], error_msg: str = "") -> bool:
|
||||
"""
|
||||
Run a command and check if it succeeds.
|
||||
|
||||
Args:
|
||||
cmd: Command to run as list of strings
|
||||
error_msg: Optional error message to log on failure
|
||||
|
||||
Returns:
|
||||
True if command succeeds, False otherwise
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, check=False)
|
||||
if result.returncode != 0:
|
||||
if error_msg:
|
||||
self.logger.error(f"{error_msg}: {result.stderr}")
|
||||
return False
|
||||
return True
|
||||
except Exception as e:
|
||||
if error_msg:
|
||||
self.logger.error(f"{error_msg}: {e}")
|
||||
return False
|
||||
|
||||
def is_git_installed(self) -> bool:
|
||||
"""Check if git is installed."""
|
||||
return self.check_command_exists("git")
|
||||
|
||||
def is_gh_installed(self) -> bool:
|
||||
"""Check if GitHub CLI (gh) is installed."""
|
||||
return self.check_command_exists("gh")
|
||||
|
||||
def is_gh_authenticated(self) -> bool:
|
||||
"""Check if GitHub CLI is authenticated."""
|
||||
try:
|
||||
# Try to get the current user - this will fail if not authenticated
|
||||
result = subprocess.run(
|
||||
["gh", "api", "user", "-q", ".login"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
# If we get a username back, we're authenticated
|
||||
return result.returncode == 0 and result.stdout.strip() != ""
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def get_github_username(self) -> str | None:
|
||||
"""
|
||||
Get GitHub username from gh CLI.
|
||||
|
||||
Returns:
|
||||
GitHub username or None if not available
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["gh", "api", "user", "--jq", ".login"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
username = result.stdout.strip()
|
||||
return username if username else None
|
||||
except subprocess.CalledProcessError:
|
||||
self.logger.warning("Could not get GitHub username from gh CLI")
|
||||
return None
|
||||
|
||||
def get_git_config(self, key: str) -> str | None:
|
||||
"""
|
||||
Get a git configuration value.
|
||||
|
||||
Args:
|
||||
key: Git config key (e.g., 'user.name', 'user.email')
|
||||
|
||||
Returns:
|
||||
Config value or None if not set
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "config", key], capture_output=True, text=True, check=False
|
||||
)
|
||||
value = result.stdout.strip()
|
||||
return value if value else None
|
||||
except subprocess.SubprocessError:
|
||||
return None
|
||||
|
||||
def check_remote_exists(self, remote_name: str = "origin") -> bool:
|
||||
"""
|
||||
Check if a git remote exists.
|
||||
|
||||
Args:
|
||||
remote_name: Name of the remote to check
|
||||
|
||||
Returns:
|
||||
True if remote exists, False otherwise
|
||||
"""
|
||||
return self.run_command(
|
||||
["git", "remote", "get-url", remote_name],
|
||||
f"Remote '{remote_name}' not found",
|
||||
)
|
||||
|
||||
def get_remote_url(self, remote_name: str = "origin") -> str | None:
|
||||
"""
|
||||
Get URL for a git remote.
|
||||
|
||||
Args:
|
||||
remote_name: Name of the remote
|
||||
|
||||
Returns:
|
||||
Remote URL or None if remote doesn't exist
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "remote", "get-url", remote_name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
return result.stdout.strip()
|
||||
except subprocess.CalledProcessError:
|
||||
return None
|
||||
|
||||
def get_remote_type(self, remote_name: str = "origin") -> str | None:
|
||||
"""
|
||||
Detect whether a git remote uses SSH or HTTPS.
|
||||
|
||||
Args:
|
||||
remote_name: Name of the remote to check
|
||||
|
||||
Returns:
|
||||
"ssh" or "https" or None if remote doesn't exist
|
||||
"""
|
||||
url = self.get_remote_url(remote_name)
|
||||
if not url:
|
||||
return None
|
||||
|
||||
if url.startswith("git@") or url.startswith("ssh://"):
|
||||
return "ssh"
|
||||
elif url.startswith("https://"):
|
||||
return "https"
|
||||
else:
|
||||
self.logger.warning(f"Unknown remote URL format: {url}")
|
||||
return None
|
||||
|
||||
def is_working_directory_clean(self) -> bool:
|
||||
"""
|
||||
Check if working directory has no uncommitted changes.
|
||||
|
||||
Returns:
|
||||
True if clean, False if there are uncommitted changes
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "status", "--porcelain"], capture_output=True, text=True
|
||||
)
|
||||
return not result.stdout.strip()
|
||||
except subprocess.SubprocessError:
|
||||
return False
|
||||
|
||||
def get_uncommitted_files(self) -> str | None:
|
||||
"""
|
||||
Get list of uncommitted files.
|
||||
|
||||
Returns:
|
||||
Output of git status --porcelain or None on error
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "status", "--porcelain"], capture_output=True, text=True
|
||||
)
|
||||
return result.stdout.strip()
|
||||
except subprocess.SubprocessError:
|
||||
return None
|
||||
|
||||
def stage_file(self, filepath: Path, cwd: Path | None = None) -> bool:
|
||||
"""
|
||||
Stage a file for commit.
|
||||
|
||||
Args:
|
||||
filepath: Path to file to stage
|
||||
cwd: Working directory for the command
|
||||
|
||||
Returns:
|
||||
True if successful, False otherwise
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "add", str(filepath)],
|
||||
cwd=cwd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
self.logger.error(f"Failed to stage file: {result.stderr}")
|
||||
return False
|
||||
return True
|
||||
except subprocess.SubprocessError as e:
|
||||
self.logger.error(f"Error staging file: {e}")
|
||||
return False
|
||||
|
||||
def check_file_modified(self, filepath: Path, cwd: Path | None = None) -> bool:
|
||||
"""
|
||||
Check if a file has been modified (staged or unstaged).
|
||||
|
||||
Args:
|
||||
filepath: Path to check
|
||||
cwd: Working directory for the command
|
||||
|
||||
Returns:
|
||||
True if file is modified, False otherwise
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "diff", "--name-only", str(filepath)],
|
||||
cwd=cwd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
unstaged = bool(result.stdout.strip())
|
||||
|
||||
result = subprocess.run(
|
||||
["git", "diff", "--cached", "--name-only", str(filepath)],
|
||||
cwd=cwd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
staged = bool(result.stdout.strip())
|
||||
|
||||
return unstaged or staged
|
||||
except subprocess.SubprocessError:
|
||||
return False
|
||||
172
.agent/knowledge/awesome_claude/scripts/utils/github_utils.py
Normal file
172
.agent/knowledge/awesome_claude/scripts/utils/github_utils.py
Normal file
@@ -0,0 +1,172 @@
|
||||
"""GitHub-related utilities shared across scripts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
from urllib.parse import quote
|
||||
|
||||
from github import Auth, Github
|
||||
|
||||
_DEFAULT_SECONDS_BETWEEN_REQUESTS = 0.5
|
||||
_DEFAULT_GITHUB_USER_AGENT = "awesome-claude-code bot"
|
||||
_GITHUB_CLIENTS: dict[tuple[str | None, str | None, float], Github] = {}
|
||||
|
||||
|
||||
def _normalize_repo_name(repo: str) -> str:
|
||||
if repo.endswith(".git"):
|
||||
return repo[: -len(".git")]
|
||||
return repo
|
||||
|
||||
|
||||
def get_github_client(
|
||||
token: str | None = None,
|
||||
user_agent: str = _DEFAULT_GITHUB_USER_AGENT,
|
||||
seconds_between_requests: float = _DEFAULT_SECONDS_BETWEEN_REQUESTS,
|
||||
) -> Github:
|
||||
"""Return a cached PyGithub client with optional pacing."""
|
||||
key = (token, user_agent, seconds_between_requests)
|
||||
if key not in _GITHUB_CLIENTS:
|
||||
auth = Auth.Token(token) if token else None
|
||||
_GITHUB_CLIENTS[key] = Github(
|
||||
auth=auth,
|
||||
user_agent=user_agent,
|
||||
seconds_between_requests=seconds_between_requests,
|
||||
)
|
||||
return _GITHUB_CLIENTS[key]
|
||||
|
||||
|
||||
def github_request_json(
|
||||
api_url: str,
|
||||
params: dict[str, object] | None = None,
|
||||
token: str | None = None,
|
||||
user_agent: str = _DEFAULT_GITHUB_USER_AGENT,
|
||||
seconds_between_requests: float = _DEFAULT_SECONDS_BETWEEN_REQUESTS,
|
||||
) -> tuple[int, dict[str, object], object | None]:
|
||||
"""Request JSON from the GitHub API using PyGithub's requester."""
|
||||
if token is None:
|
||||
token = os.getenv("GITHUB_TOKEN") or None
|
||||
client = get_github_client(
|
||||
token=token,
|
||||
user_agent=user_agent,
|
||||
seconds_between_requests=seconds_between_requests,
|
||||
)
|
||||
status, headers, body = client.requester.requestJson(
|
||||
"GET",
|
||||
api_url,
|
||||
parameters=params,
|
||||
headers={"Accept": "application/vnd.github+json"},
|
||||
)
|
||||
if not body:
|
||||
return status, headers, None
|
||||
try:
|
||||
data = json.loads(body)
|
||||
except json.JSONDecodeError:
|
||||
data = body
|
||||
return status, headers, data
|
||||
|
||||
|
||||
def parse_github_url(url: str) -> tuple[str, bool, str | None, str | None]:
|
||||
"""
|
||||
Parse GitHub URL and return API endpoint if it's a GitHub repository content URL.
|
||||
Returns (api_url, is_github, owner, repo) tuple.
|
||||
"""
|
||||
# Match GitHub blob or tree URLs - capture everything after /blob/ or /tree/ as one group
|
||||
github_pattern = r"https://github\.com/([^/]+)/([^/]+)/(blob|tree)/(.+)"
|
||||
match = re.match(github_pattern, url)
|
||||
|
||||
if match:
|
||||
owner, repo, _, branch_and_path = match.groups() # _ is blob_or_tree, which we don't need
|
||||
repo = _normalize_repo_name(repo)
|
||||
|
||||
# Split on the first occurrence of a path starting with . or containing a file extension
|
||||
# Common patterns: .github/, .claude/, src/, file.ext
|
||||
parts = branch_and_path.split("/")
|
||||
|
||||
# Find where the file path likely starts
|
||||
branch_parts = []
|
||||
path_parts: list[str] = []
|
||||
found_path_start = False
|
||||
|
||||
for i, part in enumerate(parts):
|
||||
if not found_path_start:
|
||||
# Check if this looks like the start of a file path
|
||||
if (
|
||||
part.startswith(".") # Hidden directories like .github, .claude
|
||||
or "." in part # Files with extensions
|
||||
or part in ["src", "lib", "bin", "scripts", "docs", "test", "tests"]
|
||||
): # Common directories
|
||||
found_path_start = True
|
||||
path_parts = parts[i:]
|
||||
else:
|
||||
branch_parts.append(part)
|
||||
|
||||
# If we didn't find an obvious path start, treat the last part as the path
|
||||
if not path_parts and parts:
|
||||
branch_parts = parts[:-1] if len(parts) > 1 else parts
|
||||
path_parts = parts[-1:] if len(parts) > 1 else []
|
||||
|
||||
branch = "/".join(branch_parts) if branch_parts else "main"
|
||||
path = "/".join(path_parts)
|
||||
|
||||
# URL-encode the branch name to handle slashes
|
||||
encoded_branch = quote(branch, safe="")
|
||||
api_url = (
|
||||
f"https://api.github.com/repos/{owner}/{repo}/contents/{path}?ref={encoded_branch}"
|
||||
)
|
||||
return api_url, True, owner, repo
|
||||
|
||||
# Check if it's a repository root URL
|
||||
github_repo_pattern = r"https://github\.com/([^/]+)/([^/]+)(?:/.*)?$"
|
||||
match = re.match(github_repo_pattern, url)
|
||||
if match:
|
||||
owner, repo = match.groups()
|
||||
repo = _normalize_repo_name(repo)
|
||||
api_url = f"https://api.github.com/repos/{owner}/{repo}"
|
||||
return api_url, True, owner, repo
|
||||
|
||||
return url, False, None, None
|
||||
|
||||
|
||||
def parse_github_resource_url(url: str) -> dict[str, str] | None:
|
||||
"""
|
||||
Parse GitHub URL and extract owner, repo, branch, and path.
|
||||
Returns a dict with keys: owner, repo, branch, path, type.
|
||||
"""
|
||||
patterns = {
|
||||
# File in repository
|
||||
"file": r"https://github\.com/([^/]+)/([^/]+)/(?:blob|raw)/([^/]+)/(.+)",
|
||||
# Directory in repository
|
||||
"dir": r"https://github\.com/([^/]+)/([^/]+)/tree/([^/]+)/(.+)",
|
||||
# Repository root
|
||||
"repo": r"https://github\.com/([^/]+)/([^/]+)/?$",
|
||||
# Gist
|
||||
"gist": r"https://gist\.github\.com/([^/]+)/([^/#]+)",
|
||||
}
|
||||
|
||||
for url_type, pattern in patterns.items():
|
||||
match = re.match(pattern, url)
|
||||
if match:
|
||||
if url_type == "gist":
|
||||
return {
|
||||
"type": "gist",
|
||||
"owner": match.group(1),
|
||||
"gist_id": match.group(2),
|
||||
}
|
||||
elif url_type == "repo":
|
||||
return {
|
||||
"type": "repo",
|
||||
"owner": match.group(1),
|
||||
"repo": _normalize_repo_name(match.group(2)),
|
||||
}
|
||||
else:
|
||||
return {
|
||||
"type": url_type,
|
||||
"owner": match.group(1),
|
||||
"repo": _normalize_repo_name(match.group(2)),
|
||||
"branch": match.group(3),
|
||||
"path": match.group(4),
|
||||
}
|
||||
|
||||
return None
|
||||
15
.agent/knowledge/awesome_claude/scripts/utils/repo_root.py
Normal file
15
.agent/knowledge/awesome_claude/scripts/utils/repo_root.py
Normal file
@@ -0,0 +1,15 @@
|
||||
"""Repo root discovery helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def find_repo_root(start: Path) -> Path:
|
||||
"""Locate the repo root by walking upward until pyproject.toml exists."""
|
||||
p = start.resolve()
|
||||
while not (p / "pyproject.toml").exists():
|
||||
if p.parent == p:
|
||||
raise RuntimeError("Repo root not found")
|
||||
p = p.parent
|
||||
return p
|
||||
1015
.agent/knowledge/awesome_claude/scripts/validation/validate_links.py
Normal file
1015
.agent/knowledge/awesome_claude/scripts/validation/validate_links.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,173 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Single resource validation script for the Awesome Claude Code repository.
|
||||
Validates a single resource before adding it to the CSV.
|
||||
|
||||
This script is used by the issue submission validator and manual validation
|
||||
workflows to validate resources before they are committed to the CSV file.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from scripts.utils.repo_root import find_repo_root
|
||||
|
||||
REPO_ROOT = find_repo_root(Path(__file__))
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from scripts.validation.validate_links import validate_url
|
||||
|
||||
|
||||
def validate_single_resource(
|
||||
*,
|
||||
primary_link: str,
|
||||
secondary_link: str = "",
|
||||
display_name: str = "",
|
||||
category: str = "",
|
||||
license: str = "NOT_FOUND",
|
||||
**kwargs: Any,
|
||||
) -> tuple[bool, dict[str, Any], list[str]]:
|
||||
"""
|
||||
Validate a single resource before adding to CSV.
|
||||
|
||||
Args:
|
||||
primary_link: Required URL to validate
|
||||
secondary_link: Optional secondary URL
|
||||
display_name: Name of the resource
|
||||
category: Resource category
|
||||
license: License information (defaults to "NOT_FOUND")
|
||||
**kwargs: Additional fields that may be present in the resource
|
||||
|
||||
Returns:
|
||||
Tuple of (is_valid, enriched_data, errors):
|
||||
- is_valid: Boolean indicating if resource passes validation
|
||||
- enriched_data: Original data enriched with license and last_modified info
|
||||
- errors: List of validation error messages
|
||||
"""
|
||||
errors = []
|
||||
enriched_data = {
|
||||
"primary_link": primary_link,
|
||||
"secondary_link": secondary_link,
|
||||
"display_name": display_name,
|
||||
"category": category,
|
||||
"license": license,
|
||||
**kwargs,
|
||||
}
|
||||
|
||||
# Validate primary link
|
||||
primary_url = primary_link.strip()
|
||||
if not primary_url:
|
||||
errors.append("Primary link is required")
|
||||
return False, enriched_data, errors
|
||||
|
||||
print(f"Validating primary URL: {primary_url}")
|
||||
primary_valid, primary_status, license_info, last_modified = validate_url(primary_url)
|
||||
|
||||
if not primary_valid:
|
||||
errors.append(f"Primary URL validation failed: {primary_status}")
|
||||
else:
|
||||
print("✓ Primary URL is valid")
|
||||
|
||||
# Enrich with GitHub data if available
|
||||
if license_info and license_info != "NOT_FOUND":
|
||||
enriched_data["license"] = license_info
|
||||
print(f"✓ Found license: {license_info}")
|
||||
|
||||
if last_modified:
|
||||
enriched_data["last_modified"] = last_modified
|
||||
print(f"✓ Found last modified date: {last_modified}")
|
||||
|
||||
# Validate secondary link if present
|
||||
secondary_url = secondary_link.strip()
|
||||
if secondary_url:
|
||||
print(f"Validating secondary URL: {secondary_url}")
|
||||
secondary_valid, secondary_status, _, _ = validate_url(secondary_url)
|
||||
|
||||
if not secondary_valid:
|
||||
errors.append(f"Secondary URL validation failed: {secondary_status}")
|
||||
else:
|
||||
print("✓ Secondary URL is valid")
|
||||
|
||||
# Set active status
|
||||
is_valid = len(errors) == 0
|
||||
enriched_data["active"] = "TRUE" if is_valid else "FALSE"
|
||||
enriched_data["last_checked"] = datetime.now().strftime("%Y-%m-%d:%H-%M-%S")
|
||||
|
||||
return is_valid, enriched_data, errors
|
||||
|
||||
|
||||
def validate_resource_from_dict(
|
||||
resource_dict: dict[str, str],
|
||||
) -> tuple[bool, dict[str, Any], list[str]]:
|
||||
"""
|
||||
Convenience function for validating a resource dictionary.
|
||||
Maps common field names to expected format.
|
||||
"""
|
||||
# Extract known fields and pass the rest as kwargs
|
||||
is_valid, enriched_data, errors = validate_single_resource(
|
||||
primary_link=resource_dict.get("primary_link", ""),
|
||||
secondary_link=resource_dict.get("secondary_link", ""),
|
||||
display_name=resource_dict.get("display_name", ""),
|
||||
category=resource_dict.get("category", ""),
|
||||
license=resource_dict.get("license", "NOT_FOUND"),
|
||||
**{
|
||||
k: v
|
||||
for k, v in resource_dict.items()
|
||||
if k not in ["primary_link", "secondary_link", "display_name", "category", "license"]
|
||||
},
|
||||
)
|
||||
|
||||
# Map enriched data back to original field names
|
||||
if "license" in enriched_data and enriched_data["license"] != "NOT_FOUND":
|
||||
resource_dict["license"] = enriched_data["license"]
|
||||
if "last_modified" in enriched_data:
|
||||
resource_dict["last_modified"] = enriched_data["last_modified"]
|
||||
if "last_checked" in enriched_data:
|
||||
resource_dict["last_checked"] = enriched_data["last_checked"]
|
||||
|
||||
return is_valid, resource_dict, errors
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
Command-line interface for testing single resource validation.
|
||||
"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description="Validate a single resource")
|
||||
parser.add_argument("url", help="Primary URL to validate")
|
||||
parser.add_argument("--secondary", help="Secondary URL to validate")
|
||||
parser.add_argument("--name", default="Test Resource", help="Resource name")
|
||||
args = parser.parse_args()
|
||||
|
||||
print(f"\nValidating resource: {args.name}")
|
||||
print("=" * 50)
|
||||
|
||||
is_valid, enriched_data, errors = validate_single_resource(
|
||||
primary_link=args.url,
|
||||
secondary_link=args.secondary or "",
|
||||
display_name=args.name,
|
||||
category="Test",
|
||||
)
|
||||
|
||||
print("\nValidation Results:")
|
||||
print("=" * 50)
|
||||
print(f"Valid: {'✓ Yes' if is_valid else '✗ No'}")
|
||||
|
||||
if errors:
|
||||
print("\nErrors:")
|
||||
for error in errors:
|
||||
print(f" - {error}")
|
||||
|
||||
print("\nEnriched Data:")
|
||||
for key, value in enriched_data.items():
|
||||
if value and key not in ["primary_link", "secondary_link", "display_name", "category"]:
|
||||
print(f" {key}: {value}")
|
||||
|
||||
return 0 if is_valid else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user