Application Security and Privacy teams have a new internal tool for discovering Personally Identifiable Information (PII). Released on October 16, the PIIHunter security tool scans PostgreSQL databases and Jira projects to detect accidental data exposure using multiple layers of analysis.
Detection Methods and Features
PIIHunter combines three detection layers for comprehensive coverage. It uses deterministic regex patterns to find emails, phone numbers, Social Security Numbers, and credit cards. For entity recognition, such as names, locations, and dates, it relies on NER/ML models like Microsoft Presidio or spaCy. Additionally, teams can use optional self-hosted Large Language Models (LLMs) to classify ambiguous free-text data.
The tool includes smart exception filters to suppress expected content, such as @mentions and company email addresses. It analyzes representative data samples to minimize performance impact through intelligent sampling. Every scan is logged with its configuration, timestamps, and model versions to maintain a comprehensive audit trail, while privacy-safe logging automatically redacts PII in all logs and stored samples. Jira scan results are automatically saved to a dated JSON or CSV file on every run.
Database Schema Scan
To scan a PostgreSQL database for PII across columns, users must first set up the environment and download the approximately 800MB spaCy NER model.
# Clone the repository
cd PIIHunter
# Create virtual environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Download spaCy NER model (~800MB)
python -m spacy download en_core_web_lgNext, create a read-only role in the database.
CREATE ROLE pii_scanner_readonly WITH LOGIN PASSWORD 'your_secure_password';
GRANT CONNECT ON DATABASE your_database TO pii_scanner_readonly;
GRANT USAGE ON SCHEMA public, app_data TO pii_scanner_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public, app_data TO pii_scanner_readonly;Configure the environment variables.
cp .env.example .env
# Fill in DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORDThe tool creates a security_pii schema to store scan results. Run the following commands to discover columns, run the detection, and view reports.
# Preview candidate columns (no data read)
python -m pii_scanner.cli discover
# Run PII detection
python -m pii_scanner.cli scan --sample-size 1000
# View results
python -m pii_scanner.cli reportTo test the database connection, use this command.
python -m pii_scanner.cli test-connectionJira Scan Integration
PIIHunter can scan Jira issues and comments for accidental PII exposure by polling for activity in the last specified hours, which requires no server. Database setup is not required for this feature, but the base installation is necessary.
cd PIIHunter
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python -m spacy download en_core_web_lgTo configure Jira access:
- Go to
https://id.atlassian.com, navigate to Security, and select API tokens. - Click Create API token, give it a name, and copy the token.
Add the credentials to the configuration file.
JIRA_URL=https://yourcompany.atlassian.net
[email protected]
JIRA_API_TOKEN=your_api_token_here
# Optional — leave empty to scan all projects
# JIRA_PROJECT_KEYS=ENG,OPS,SUPPORT
# Emails from these domains will NOT be flagged as PII
COMPANY_EMAIL_DOMAINS=yourcompany.comThe tool verifies credentials, lists accessible projects, and fetches a sample of recent issues. If it fails, users should check their API token and ensure the email matches the account.
Testing connection to https://yourcompany.atlassian.net...
Auth ✓ logged in as Jane Smith ([email protected])
Projects ✓ 3 accessible: ENG, OPS, SUPPORT
Project ENG ✓ accessible
Recent issues (no time filter):
ENG-101 Fix login timeout on mobile
ENG-100 Update password policyRun the scan with a custom lookback window.
# Scan last 24 hours
python -m pii_scanner.cli jira scan
# Custom lookback window
python -m pii_scanner.cli jira scan --hours 48Real-Time Jira Webhooks
For real-time scanning, PIIHunter can start an HTTP server that receives Jira events and scans them instantly. This requires a publicly reachable endpoint and the base installation.
# Secret used to verify requests genuinely come from Jira
JIRA_WEBHOOK_SECRET=your_webhook_secret
JIRA_WEBHOOK_PORT=8080Start the webhook server.
python -m pii_scanner.cli jira webhook
# Custom port
python -m pii_scanner.cli jira webhook --port 9090Configure the webhook in Jira:
- In Jira, go to Project Settings, select Webhooks, and click Create webhook.
- Set the URL to
http://yourserver:8080/webhook. - Under Events, select
Issue → createdandComment → created. - Set the secret to match the webhook secret in the configuration file and save.
Jira will now send a POST request to the server for every new issue and comment, allowing the scanner to process each one immediately.
Deployment and Security Guidelines
PIIHunter is designed strictly for internal use within a secure, controlled environment. It operates with a security-first design, utilizing read-only database access, making no external API calls, and ensuring minimal PII retention. For production webhook setups, the developers warn against exposing uvicorn directly to the internet.
This article was written by AI from reporting by kitploit.com.