docs: add comprehensive api endpoints documentation and expand cli functionalityv1.0.3

1 files changed, 462 insertions, 0 deletions

diff --git a/api_endpoints_summary.md b/api_endpoints_summary.md
new file mode 100644
index 0000000..721e6d6
--- /dev/null
+++ b/api_endpoints_summary.md
@@ -0,0 +1,462 @@
+# DB.org.ai API Endpoints Summary
+
+## Account Management
+
+### `/me` - Get Account Stats
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: None
+- **Response**: `main.AccountStatsResponse`
+- **Description**: Get account information and usage statistics for authenticated user
+
+---
+
+## Admin Service
+
+### `/admin/accounts` - List Accounts
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: None
+- **Response**: Object with arrays of `main.AccountResponse`
+- **Description**: Get a list of all accounts
+
+### `/admin/accounts` - Create Account
+- **Method**: POST
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Body: `main.CreateAccountRequest` (required)
+- **Response**: `main.AccountResponse`
+- **Description**: Create a new user account with API key
+
+### `/admin/accounts/{api_key}` - Delete Account
+- **Method**: DELETE
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `api_key` (string, required) - Account API Key
+- **Response**: Object with string properties
+- **Description**: Delete an account by API key
+
+### `/admin/accounts/{api_key}/credits` - Set Credits
+- **Method**: PUT
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `api_key` (string, required) - Account API Key
+  - Body: `main.SetCreditsRequest` (required)
+- **Response**: `main.AccountResponse`
+- **Description**: Set account credits to a specific amount
+
+### `/admin/accounts/{api_key}/credits` - Add Credits
+- **Method**: POST
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `api_key` (string, required) - Account API Key
+  - Body: `main.AddCreditsRequest` (required)
+- **Response**: `main.AccountResponse`
+- **Description**: Add credits to an existing account
+
+### `/admin/accounts/{api_key}/disable` - Disable/Enable Account
+- **Method**: PATCH
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `api_key` (string, required) - Account API Key
+  - Body: `main.DisableAccountRequest` (required)
+- **Response**: `main.AccountResponse`
+- **Description**: Disable or enable an account by setting the disabled flag
+
+---
+
+## BreachForum Service
+
+### `/breachforum/search` - Search BreachForum Data
+- **Method**: GET
+- **Authentication**: None
+- **Parameters**: 
+  - Query: `search` (string, required) - Search query
+  - Query: `max_hits` (integer, default: 10) - Maximum number of hits to return
+- **Response**: `api.BreachForumSearchResponse`
+- **Description**: Search breachdetect index for BreachForum messages and detections
+
+---
+
+## BSSID Service
+
+### `/bssid/{bssid}` - BSSID Location Lookup
+- **Method**: GET
+- **Authentication**: None
+- **Parameters**: 
+  - Path: `bssid` (string, required) - BSSID/MAC address (format: aa:bb:cc:dd:ee:ff)
+  - Query: `a` (boolean, default: false) - Show all related results instead of exact match only (alias for 'all')
+  - Query: `all` (boolean, default: false) - Show all related results instead of exact match only
+  - Query: `google` (boolean, default: false) - Include Google Maps URL for the location
+  - Query: `osm` (boolean, default: false) - Include OpenStreetMap URL for the location
+- **Response**: Array of `api.BSSIDResult`
+- **Description**: Lookup geographic location of a WiFi access point by its BSSID using Apple's location services
+
+---
+
+## Buckets Service
+
+### `/buckets/buckets` - Search Public Buckets
+- **Method**: GET
+- **Authentication**: Required (X-API-Key header)
+- **Parameters**: 
+  - Header: `X-API-Key` (string, required) - API Key
+  - Query: `limit` (integer, default: 1000) - Number of results to return
+  - Query: `start` (integer, default: 0) - Starting offset for pagination
+- **Response**: `api.BucketsSearchResponse`
+- **Description**: List public buckets with file counts
+
+### `/buckets/files` - Search Public Bucket Files
+- **Method**: GET
+- **Authentication**: Required (X-API-Key header)
+- **Parameters**: 
+  - Header: `X-API-Key` (string, required) - API Key
+  - Query: `keywords` (string) - Search keywords
+  - Query: `extensions` (string) - File extensions (comma-separated, e.g. 'sql,db,xlsx')
+  - Query: `buckets` (string) - Filter by bucket names (comma-separated)
+  - Query: `limit` (integer, default: 1000) - Number of results to return
+  - Query: `start` (integer, default: 0) - Starting offset for pagination
+- **Response**: `api.FilesSearchResponse`
+- **Description**: Search public S3, Azure, GCP, and DigitalOcean buckets for exposed files
+
+---
+
+## Crawl Service
+
+### `/crawl/{domain}` - Crawl Domain
+- **Method**: GET
+- **Authentication**: None
+- **Parameters**: 
+  - Path: `domain` (string, required) - Domain to crawl (can include http:// or https://)
+  - Query: `subdomains` (boolean) - Also discover and crawl all subdomains using subfinder (default: false)
+- **Response**: Server-Sent Events (SSE) stream
+- **Description**: Resolves a domain using httpx and crawls it using katana with depth 3 and JavaScript link extraction
+
+---
+
+## DNS Service
+
+### `/dns/tld/{term}` - Check NXDOMAIN for All TLDs
+- **Method**: GET
+- **Authentication**: None
+- **Parameters**: 
+  - Path: `term` (string, required) - Domain name prefix to check
+  - Query: `showOnly` (string) - Filter results: 'exists', 'nxdomain', or empty (show all)
+- **Response**: `api.DomainResult` (NDJSON stream)
+- **Description**: Streams NDJSON results checking each TLD with tech detection
+
+---
+
+## Files Service
+
+### `/files/{url}` - Search Open Directory Files
+- **Method**: GET
+- **Authentication**: None (Free OSINT endpoint)
+- **Parameters**: 
+  - Path: `url` (string, required) - Search term to match in URLs
+  - Query: `filename` (string) - Search term to match in filenames
+  - Query: `extension` (string) - Filter by file extension(s) - comma-separated
+  - Query: `exclude` (string) - Exclude file extension(s) - comma-separated (default: html,HTML)
+  - Query: `size` (integer) - Number of results to return (max 40, default 10)
+  - Query: `from` (integer) - Starting offset for pagination (default 0)
+- **Response**: Object with additional properties
+- **Description**: Search for files in open directories using various filters
+
+---
+
+## Geo Service
+
+### `/geo` - Search Address Information
+- **Method**: GET
+- **Authentication**: Required (X-API-Key header)
+- **Parameters**: 
+  - Query: `street` (string, required) - Street address
+  - Query: `city` (string, required) - City
+  - Query: `state` (string, required) - State (2-letter code)
+  - Query: `zip` (string, required) - ZIP code
+  - Header: `X-API-Key` (string, required) - API Key
+- **Response**: Object with additional properties
+- **Description**: Returns address information including residents, property details, and demographics
+
+---
+
+## GitHub Service
+
+### `/github/leads` - GitHub Leads Scanner
+- **Method**: GET
+- **Authentication**: None
+- **Parameters**: 
+  - Query: `q` (string, required) - Search query for GitHub repositories
+  - Query: `sort` (string, default: "stars") - Sort method (stars, forks, updated)
+  - Query: `exclude` (string) - Comma-separated terms to exclude from search
+- **Response**: `api.LeadResult` (NDJSON stream)
+- **Description**: Scans GitHub repositories for commit author information based on search query
+
+---
+
+## NPD Service
+
+### `/npd/search` - Search NPD Breach Data
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Query: `id` (string) - ID
+  - Query: `firstname` (string) - First name
+  - Query: `lastname` (string) - Last name
+  - Query: `middlename` (string) - Middle name
+  - Query: `dob` (string) - Date of birth
+  - Query: `ssn` (string) - Social security number
+  - Query: `phone1` (string) - Phone number
+  - Query: `address` (string) - Address
+  - Query: `city` (string) - City
+  - Query: `st` (string) - State
+  - Query: `zip` (string) - ZIP code
+  - Query: `county_name` (string) - County name
+  - Query: `name_suff` (string) - Name suffix
+  - Query: `aka1fullname` (string) - AKA 1 full name
+  - Query: `aka2fullname` (string) - AKA 2 full name
+  - Query: `aka3fullname` (string) - AKA 3 full name
+  - Query: `alt1dob` (string) - Alternate DOB 1
+  - Query: `alt2dob` (string) - Alternate DOB 2
+  - Query: `alt3dob` (string) - Alternate DOB 3
+  - Query: `startdat` (string) - Start date
+  - Query: `max_hits` (integer, default: 10) - Maximum number of hits to return
+  - Query: `sort_by` (string) - Sort by field
+- **Response**: `api.NPDSearchResponse`
+- **Description**: Search NPD breach data by various fields
+
+---
+
+## Reddit Service
+
+### `/reddit/r/{subreddit}` - Get Subreddit Posts
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `subreddit` (string, required) - Subreddit name
+- **Response**: `api.SubredditResponse`
+- **Description**: Get up to 1000 recent posts from a subreddit
+
+### `/reddit/r/{subreddit}/comments` - Get Subreddit Comments
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `subreddit` (string, required) - Subreddit name
+- **Response**: `api.SubredditResponse`
+- **Description**: Get up to 1000 recent comments from a subreddit
+
+### `/reddit/user/{username}/about` - Get User About
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `username` (string, required) - Reddit username
+- **Response**: `api.UserResponse`
+- **Description**: Get user profile information
+
+### `/reddit/user/{username}/comments` - Get User Comments
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `username` (string, required) - Reddit username
+- **Response**: `api.UserResponse`
+- **Description**: Get up to 1000 recent comments from a user
+
+### `/reddit/user/{username}/posts` - Get User Posts
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `username` (string, required) - Reddit username
+- **Response**: `api.UserResponse`
+- **Description**: Get up to 1000 recent posts from a user
+
+---
+
+## Shortlinks Service
+
+### `/shortlinks` - Search Brute Forced Short Links
+- **Method**: GET
+- **Authentication**: Required (X-API-Key header)
+- **Parameters**: 
+  - Header: `X-API-Key` (string, required) - API Key
+  - Query: `keywords` (string) - Search keywords
+  - Query: `ext` (string) - File extensions (comma-separated, e.g. 'pdf,docx,xlsx')
+  - Query: `order` (string) - Sort by property (size, timestamp)
+  - Query: `direction` (string) - Sort direction (asc, desc)
+  - Query: `regexp` (boolean, default: false) - Treat keywords as regular expression
+  - Query: `limit` (integer, default: 100) - Number of results to return
+  - Query: `start` (integer, default: 0) - Starting offset for pagination
+- **Response**: `api.ShortlinksSearchResponse`
+- **Description**: Search for exposed URLs discovered through brute forcing URL shortener services
+
+---
+
+## Skiptrace (Premium) Service
+
+### `/prem/skiptrace/email/{email}` - Search Email Address
+- **Method**: GET
+- **Authentication**: Required (X-API-Key header, Premium Required)
+- **Parameters**: 
+  - Path: `email` (string, required) - Email address
+  - Header: `X-API-Key` (string, required) - API Key (Premium Required)
+- **Response**: Object with additional properties
+- **Description**: Premium endpoint - Search for email address
+
+### `/prem/skiptrace/people/report/{sx_key}/{selection}` - Get Detailed Person Report
+- **Method**: GET
+- **Authentication**: Required (X-API-Key header, Premium Required)
+- **Parameters**: 
+  - Path: `sx_key` (string, required) - Search session key
+  - Path: `selection` (integer, required) - Person selection (1-based index)
+  - Header: `X-API-Key` (string, required) - API Key (Premium Required)
+- **Response**: Object with additional properties
+- **Description**: Premium endpoint - Get detailed report for selected person
+
+### `/prem/skiptrace/people/search` - Search People by Name
+- **Method**: GET
+- **Authentication**: Required (X-API-Key header, Premium Required)
+- **Parameters**: 
+  - Query: `first_name` (string, required) - First name
+  - Query: `last_name` (string, required) - Last name
+  - Query: `city` (string) - City
+  - Query: `state` (string) - State (2-letter code)
+  - Query: `age` (string) - Age
+  - Header: `X-API-Key` (string, required) - API Key (Premium Required)
+- **Response**: Object with additional properties
+- **Description**: Premium endpoint - Search for people by name
+
+### `/prem/skiptrace/phone/{phone}` - Search Phone Number
+- **Method**: GET
+- **Authentication**: Required (X-API-Key header, Premium Required)
+- **Parameters**: 
+  - Path: `phone` (string, required) - Phone number (10 digits, no +1 prefix)
+  - Header: `X-API-Key` (string, required) - API Key (Premium Required)
+- **Response**: Object with additional properties
+- **Description**: Premium endpoint - Search for phone number
+
+---
+
+## Stealer Logs Service
+
+### `/sl/search` - Search Stealer Logs
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Query: `query` (string, required) - Search query
+  - Query: `max_hits` (integer, default: 10) - Maximum number of hits to return
+  - Query: `sort_by` (string, enum: ["ingest_timestamp", "date_posted"]) - Sort by field
+  - Query: `ingest_start_date` (string) - Ingest timestamp start date (Quickwit date format)
+  - Query: `ingest_end_date` (string) - Ingest timestamp end date (Quickwit date format)
+  - Query: `posted_start_date` (string) - Date posted start date (Quickwit date format)
+  - Query: `posted_end_date` (string) - Date posted end date (Quickwit date format)
+  - Query: `format` (string, default: "json") - Response format: json or custom format like 'ulp', 'up', 'pul', etc.
+- **Response**: `api.QuickwitSearchResponse`
+- **Description**: Search stealer logs
+
+---
+
+## Username Service
+
+### `/username/{username}` - Check Username Availability
+- **Method**: GET
+- **Authentication**: None
+- **Parameters**: 
+  - Path: `username` (string, required) - Username to check
+  - Query: `sites` (array, string) - Specific sites to check (comma-separated)
+  - Query: `fuzzy` (boolean, default: false) - Enable fuzzy validation mode
+  - Query: `max_tasks` (integer, default: 50) - Maximum concurrent tasks
+- **Response**: `api.SiteResult` (NDJSON stream)
+- **Description**: Check username availability across hundreds of websites using WhatsMyName dataset
+
+---
+
+## X (Twitter) Service
+
+### `/x/first/{username}` - Get First 20 Followers
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `username` (string, required) - Twitter/X username
+- **Response**: `db_org_ai_services_x_api.SearchResponse`
+- **Description**: Retrieves the first 20 followers of a Twitter/X account
+
+### `/x/history/{username}` - Get Username History
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `username` (string, required) - Twitter/X username
+- **Response**: `db_org_ai_services_x_api.SearchResponse`
+- **Description**: Retrieves the username history and previous usernames for a Twitter/X user
+
+### `/x/nfl/{username}` - Get Notable Followers
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `username` (string, required) - Twitter/X username
+- **Response**: `db_org_ai_services_x_api.SearchResponse`
+- **Description**: Retrieves the notable followers (influential accounts) following a Twitter/X account
+
+### `/x/replies/{tweet_id}` - Fetch Tweet Replies
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `tweet_id` (string, required) - Tweet ID
+  - Query: `limit` (integer) - Maximum number of replies to fetch (default: 100)
+- **Response**: `api.ScrapedReply` (NDJSON stream)
+- **Description**: Fetches all replies for a given tweet ID and streams results as NDJSON
+
+### `/x/search/{query}` - Search Tweets
+- **Method**: GET
+- **Authentication**: Required (ApiKeyAuth)
+- **Parameters**: 
+  - Path: `query` (string, required) - Search term
+  - Query: `limit` (integer) - Maximum number of tweets to fetch (default: 100)
+- **Response**: `api.ScrapedTweet` (NDJSON stream)
+- **Description**: Searches Twitter/X for tweets matching the given search term (Costs 5 credits per 500 tweets)
+
+### `/x/tweets/{username}` - Scrape Tweets by Username
+- **Method**: GET
+- **Authentication**: None (Free OSINT endpoint)
+- **Parameters**: 
+  - Path: `username` (string, required) - Twitter/X username
+- **Response**: `api.TweetsStreamResponse` (NDJSON stream)
+- **Description**: Discovers tweet IDs from Internet Archive CDX API and fetches tweet content using Twitter's oEmbed API
+
+---
+
+## Authentication Summary
+
+### ApiKeyAuth
+- **Header**: `X-API-Key`
+- **Used by**: Admin, Account, NPD, Reddit, Stealer Logs, X (most endpoints)
+
+### X-API-Key Header
+- **Header**: `X-API-Key`
+- **Used by**: Buckets, Geo, Shortlinks, Skiptrace (Premium)
+
+### No Authentication Required
+- **Services**: BreachForum, BSSID, Crawl, DNS, Files, GitHub, Username, X (tweets endpoint)
+
+---
+
+## Response Format Summary
+
+### Standard JSON Responses
+- Most endpoints return structured JSON with specific schema types
+- Error responses typically follow `{ "error": "message" }` format
+
+### Streaming Responses
+- **NDJSON**: Username, X (replies, search), GitHub, DNS
+- **SSE**: Crawl
+- **Text**: Stealer Logs (when format != "json")
+
+### Credits Information
+- Many paid endpoints include `credits` object with `remaining` and `unlimited` fields
+- Premium endpoints require premium account access
+
+---
+
+## Cost Information
+- **X Search**: 5 credits per 500 tweets (rounded up)
+- **Premium Skiptrace**: Pricing configured in database with discount support
+- **Geo Service**: Pricing configured in database with discount support
+- **Buckets/Shortlinks**: Uses credit system with payment required (402 response)
\ No newline at end of file