﻿================================================================================
                       SURICATA IDS CONFIGURATION README
================================================================================

IMPORTANT NOTICE:
This host is managed by IDSTower via Ansible automation. Any manual 
modifications made directly on this host may be overwritten during the 
next automated deployment. For persistent configuration changes, please 
use the IDSTower web interface.

================================================================================
                              FILE STRUCTURE
================================================================================

Suricata Configuration Directory: /etc/suricata/

INSTANCE-SPECIFIC CONFIGURATIONS:
Each Suricata instance is completely isolated with its own configuration files.
There are NO shared configuration files between instances.

For each instance named <NAME>, all files are located under:
/etc/suricata/instance-<NAME>/

CONFIGURATION FILES PER INSTANCE:
- /etc/suricata/instance-<NAME>/suricata.yaml          - Base Suricata configuration with default settings
- /etc/suricata/instance-<NAME>/instance-config.yaml   - IDSTower settings from GUI (overrides suricata.yaml)
- /etc/suricata/instance-<NAME>/custom-config.yaml     - Custom configuration overrides (highest priority)
- /etc/suricata/instance-<NAME>/classification.config  - Alert classification definitions
- /etc/suricata/instance-<NAME>/reference.config       - Reference URL mappings  
- /etc/suricata/instance-<NAME>/threshold.config       - Alert thresholding configuration

CONFIGURATION FILE HIERARCHY:
The configuration files are processed in the following order (later files override earlier ones):
1. suricata.yaml          - Default Suricata settings and templates
2. instance-config.yaml   - IDSTower-managed settings from the web interface
3. custom-config.yaml     - Your custom configuration overrides

RULES DIRECTORIES PER INSTANCE:
- /etc/suricata/instance-<NAME>/rules/
  - idstower_suricata.rules                 - Main detection rules from IDSTower
  - idstower_iprep_indicators.rules         - IP reputation rules
  - idstower_datasets_indicators.rules      - Dataset-based detection rules

IP REPUTATION DIRECTORIES PER INSTANCE:
- /etc/suricata/instance-<NAME>/iprep/
  - categories.txt                          - IP reputation categories
  - idstower_ip_indicators.list             - IP addresses and CIDRs for reputation

DATASETS DIRECTORIES PER INSTANCE:
- /etc/suricata/instance-<NAME>/datasets/
  - idstower_fqdn_indicators.list           - Domain names for detection
  - idstower_sha256_indicators_score*.list  - File hashes (score 1-10)

MANAGEMENT SCRIPTS (SHARED):
- /etc/suricata/suricata_rules_update.sh         - Updates detection rules
- /etc/suricata/suricata_indicators_update.sh    - Updates IoCs/indicators
- /etc/suricata/suricata_logs_deletion.sh        - Manages log retention
- /etc/suricata/README.txt                       - This documentation file

================================================================================
                              LOG FILES
================================================================================

INSTANCE-SPECIFIC LOG DIRECTORIES:
Each Suricata instance writes its logs to its own dedicated directory under:
/var/log/suricata/instance-<NAME>/

This ensures complete isolation between instances and makes log management easier.

LOG FILES PER INSTANCE:
Each instance directory contains the following log files:

CORE LOG FILES:
- suricata.log                   - Main Suricata service log (text format)
- suricata.log.json              - Main Suricata service log (JSON format)
- suricata-stats.json            - Performance statistics and counters
- suricata-alerts.json           - Security alerts in JSON format
- suricata-nsm.json              - Network security monitoring events

WORKER-SPECIFIC FILES:
When using multi-threading, some log files may have worker numbers appended:
- suricata-alerts.1.json         - Alerts from worker thread 1
- suricata-alerts.2.json         - Alerts from worker thread 2
- suricata-nsm.1.json            - NSM events from worker thread 1
- suricata-nsm.3.json            - NSM events from worker thread 3
- (additional numbered files for each worker thread)

EXAMPLE LOG DIRECTORY STRUCTURE:
/var/log/suricata/instance-prod/
|-- suricata.log                 - 66K
|-- suricata.log.json           - 106K
|-- suricata-stats.json         - 7.2G
|-- suricata-alerts.1.json      - 0 bytes (no alerts on this worker)
|-- suricata-alerts.2.json      - 38M
|-- suricata-alerts.4.json      - 31M
|-- suricata-alerts.6.json      - 32M
|-- suricata-alerts.8.json      - 31M
|-- suricata-nsm.1.json         - 108M
|-- suricata-nsm.3.json         - 104M
|-- suricata-nsm.5.json         - 106M
|-- suricata-nsm.7.json         - 102M
+-- suricata-nsm.9.json         - 125M

HOW TO VIEW LOGS (replace <NAME> with your instance name):
# View recent alerts for a specific instance
tail -f /var/log/suricata/instance-<NAME>/suricata-alerts.*.json

# View main service log for an instance
tail -f /var/log/suricata/instance-<NAME>/suricata.log

# View JSON service log for an instance
tail -f /var/log/suricata/instance-<NAME>/suricata.log.json

# View specific instance service and logs
sudo journalctl -u suricata-instance-<NAME>.service -f

# Search for specific alerts in instance logs (example: malware detection)
grep -i malware /var/log/suricata/instance-<NAME>/suricata-alerts.*.json

# View statistics for a specific instance
tail /var/log/suricata/instance-<NAME>/suricata-stats.json

# Monitor all alerts across all instances
tail -f /var/log/suricata/instance-*/suricata-alerts.*.json

# List all instance log directories
ls -la /var/log/suricata/

# Check disk usage per instance
du -sh /var/log/suricata/instance-*

================================================================================
                        IMMEDIATE RULES AND IOC UPDATES
================================================================================

IMPORTANT: Do not manually edit rules or IOC files directly as they will be 
overwritten by automated updates. Instead, use IDSTower GUI for persistent 
changes or run update scripts for immediate deployment.

FORCING IMMEDIATE UPDATES:
Instead of waiting for the automatic 5-minute update cycle, you can force 
immediate updates using the management scripts:

FORCE RULES UPDATE:
Run the rules update script to immediately pull the latest rules:

sudo bash /etc/suricata/suricata_rules_update.sh \
  --api-url <YOUR-IDSTOWER-HOSTNAME> \
  --api-key <YOUR-HOST-API-KEY> \
  --suricata-config-dir /etc/suricata/ \
  --instances "1,2" \
  --skip-delay

This will:
- Download the latest rules from IDSTower for all configured instances
- Reload rules on each instance without restarting services (unless necessary)

FORCE IOC/INDICATORS UPDATE:
Run the indicators update script to immediately pull the latest threat intelligence:

sudo bash /etc/suricata/suricata_indicators_update.sh \
  --api-url <YOUR-IDSTOWER-HOSTNAME> \
  --api-key <YOUR-HOST-API-KEY> \
  --suricata-config-dir /etc/suricata/ \
  --instances "1,2" \
  --skip-delay

This will:
- Download the latest Indicators (aka IOCs) from IDSTower for all configured instances
- Reload indicators on each instance without restarting services (unless necessary)

FINDING YOUR CONFIGURATION:
To find the host API key and instance names for your host:
sudo crontab -l

This will show the automated cron jobs with the correct:
- API URL (--api-url)
- API key (--api-key) 
- Instance names (--instances)
- Configuration directory (--suricata-config-dir)

To view other options for the update scripts:
sudo bash /etc/suricata/suricata_rules_update.sh --help
sudo bash /etc/suricata/suricata_indicators_update.sh --help

VERIFY UPDATES APPLIED:
After running update scripts, verify the changes were applied:

# Check when rules were last updated
ls -la /etc/suricata/instance-<NAME>/rules/

# Check when indicators were last updated  
ls -la /etc/suricata/instance-<NAME>/iprep/
ls -la /etc/suricata/instance-<NAME>/datasets/

# Reload rules for specific instance (if needed)
sudo suricatasc -c "ruleset-reload-rules" /var/run/suricata-instance-<NAME>.socket

# Check rule statistics for an instance
sudo suricatasc -c "ruleset-stats" /var/run/suricata-instance-<NAME>.socket

AUTOMATED UPDATE SCHEDULE:
- Rules updates: Every 5 minutes (automatic)
- IOC updates: Every 5 minutes (automatic)
- Use the scripts above only when immediate updates are needed

To view the run result of the last automated update:
sudo cat /var/log/idstower/rules_updates.log
sudo cat /var/log/idstower/indicators_updates.log

================================================================================
                            SERVICE MANAGEMENT
================================================================================

SURICATA SERVICES:
Each instance runs as a separate systemd service:
- Instance services: suricata-instance-<NAME>.service

COMMON COMMANDS (replace <NAME> with your instance name):
# Check service status
sudo systemctl status suricata-instance-<NAME>.service

# Start/stop/restart service
sudo systemctl start suricata-instance-<NAME>.service
sudo systemctl stop suricata-instance-<NAME>.service
sudo systemctl restart suricata-instance-<NAME>.service

# View service logs
sudo journalctl -u suricata-instance-<NAME>.service -f

# Reload rules without restart
sudo suricatasc -c "ruleset-reload-rules" /var/run/suricata-instance-<NAME>.socket

# Check Suricata version
sudo suricatasc -c "version" /var/run/suricata-instance-<NAME>.socket

# List all Suricata instance services
sudo systemctl list-units "suricata-instance-*.service"

================================================================================
                            IMPORTANT WARNINGS
================================================================================

[!] ANSIBLE MANAGEMENT WARNING:
This host is automatically managed by IDSTower using Ansible. Any manual 
changes made directly to configuration files, rules, or indicators will be 
OVERWRITTEN during the next automated deployment cycle.

[!] INSTANCE ISOLATION:
Each Suricata instance is completely isolated. Configuration changes made to 
one instance will NOT affect other instances. Always specify the correct 
instance name when making changes.

[!] PERSISTENT CHANGES:
To make persistent changes to Suricata configuration, Use the IDSTower web 
interface. Manual changes to configuration files will be lost.

================================================================================
                         TROUBLESHOOTING
================================================================================

COMMON ISSUES (replace <NAME> with your instance name):

1. Suricata instance not starting:
   - Check config syntax: sudo suricata -T -c /etc/suricata/instance-<NAME>/suricata.yaml
   - View logs: sudo journalctl -u suricata-instance-<NAME>.service

2. No alerts generated for an instance:
   - Verify interface configuration in instance-<NAME>/suricata.yaml
   - Check if traffic is reaching the sensor
   - Ensure rules are loaded: sudo suricatasc -c "ruleset-stats" /var/run/suricata-instance-<NAME>.socket

3. High CPU usage on specific instance:
   - Review instance-<NAME>/suricata.yaml performance settings
   - Check rules efficiency for that instance
   - Monitor via: sudo suricatasc -c "dump-counters" /var/run/suricata-instance-<NAME>.socket

4. Disk space issues:
   - Check log retention: /etc/suricata/suricata_logs_deletion.sh
   - Verify log rotation is working
   - Monitor: df -h /var/log/suricata

5. Multiple instances conflicts:
   - Ensure each instance uses different interfaces or ports
   - Check for socket file conflicts
   - Verify unique cluster-id settings per instance

DEBUGGING SPECIFIC INSTANCES:
# Test instance configuration
sudo suricata -T -c /etc/suricata/instance-<NAME>/suricata.yaml

# Check instance socket connectivity  
sudo suricatasc -c "version" /var/run/suricata-instance-<NAME>.socket

# View instance-specific performance
sudo suricatasc -c "dump-counters" /var/run/suricata-instance-<NAME>.socket

SUPPORT:
For additional support, refer to the IDSTower documentation at https://idstower.com/docs
or contact IDSTower support at support@idstower.com

================================================================================
                              END OF README
================================================================================