This utility provides an optimized tool for migrating Rspamd fuzzy backend
data between Redis instances with the following features:
* Non-blocking SCAN-based iteration through Redis keys
* Filter exports by specific fuzzy flags (e.g., flag 1, 8, 11)
* Automatic detection and migration of shingles (32 per text hash)
* TTL preservation for all keys
* Binary Storable format for efficient serialization
* Single-pass algorithm with O(N) complexity instead of O(N*M)
* Redis pipelining for minimal network round-trips
* Configurable batch sizes for memory and performance tuning
* Detailed statistics including per-flag distribution
* Comprehensive POD documentation
Performance optimizations:
- Large SCAN batches (default 5000) for fast key iteration
- Pipeline size of 500 operations for maximum throughput
- ~800x faster than naive approach for large datasets
- Single-pass shingle matching instead of per-hash SCAN operations
Usage:
# Export fuzzy hashes with flag filtering
fuzzy_redis_migrate.pl --source-host redis1 --flags 1 8 --export backup.dat
# Import to another Redis instance
fuzzy_redis_migrate.pl --dest-host redis2 --import backup.dat
# View full documentation
perldoc utils/fuzzy_redis_migrate.pl
* [Minor] Small fix for error messages
* [Feature] Create rspamadm util to decrypt header
* [Feature] Create python example to encrypt/decrypt header
* [Minor] Small clean up
* [Minor] Change c-rspamadm util to lua-rspamadm util
* [Minor] Small clean up
* [Minor] Add some debug
* [Feature] Add secretbox command
* [Minor] Debug
* [Minor] Add additional return for encrypted string(noce + encrypted string
* [Minor] Small debug
* [Minor] Add a way to provide encrypted text concatenated with nonce
* [Minor] Add nonce to encrypt text
* [Minor] Clean up
* [Minor] Clean up unused variable
* [Minor] Small fix
* [Minor] Fix return issue
* [Minor] Add blake2b for key derivation
* [Minor] Small upgrade to debug
* [Minor] Small clean up
* [Minor] Change return to more convenient form
* [Minor] Change print to test form
* [Test] Provide tests for encrypt/decrypt with rspamadm util and python script
* [Minor] Change python to python3
* [Minor] Add stderr check
* [Minor] Make the function return nonce+text
* [Minor] Change unit tests to new return format
* [Minor] Add flag to manage encodings
* [Minor] Add --encoding argument to manage encodings
* [Minor] Change tests for new input format
* [Minor] Fix lua format
* [Minor] Small fix
* [Minor] Provide full support for new return format of maybe_encrypt_header
* [Test] Test small fix
* [Test] Small fix
* [Minor] Clean up
* [Minor] Small fix for name of variable
* [Minor] Small clean up
* [Minor] Change format of command to a mre convenient
* [Minor] Change tests to be same as a format of a command
* [Minor] Change description of flags
* [Minor] Small fix
---------
Co-authored-by: Ivan Stakhov <50211739+LeftTry@users.noreply.github.com>