Rapid spam filtering system https://rspamd.com/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

7139 lines
345 KiB

Release 3.14.1 * [Feature] Composites: Add inverted index for fast composite rule evaluation * [Feature] Composites: Add bloom filter for fast negative symbol lookups * [Feature] Composites: Add statistics tracking and control protocol command * [Feature] Composites: Precompute atom types at config time * [Feature] Multimap: Add combinator option for selector rules * [Feature] SPF: Add rspamadm spf_flatten tool with macro preservation * [Feature] URL: Add deep processing architecture with C-to-Lua filter consultation * [Feature] URL: Add obfuscated URL detection to url_suspect plugin * [Feature] URL: Add rspamd_util.decode_html_entities for HTML entity decoding * [Feature] lua_shape: Add new validation library as tableshape replacement * [Feature] lua_shape: Add T.callable() type for function validation * [Feature] lua_shape: Add callable defaults support * [Feature] Whitelist: Auto-mark symbols with SYMBOL_TYPE_FINE flag * [Fix] url_suspect: Fix plugin causing massive false positives * [Fix] url_suspect: Optimize for high URL volume messages * [Fix] Network: Prevent infinite loop in split_networks_into_chunks() * [Fix] Memory: Fix leak in custom tokenizer result handling * [Fix] Composites: Fix group matchers handling in inverted index * [Fix] Composites: Improve atom polarity detection in inverted index * [Fix] Composites: Copy expression string to memory pool for Lua composites * [Fix] lua_shape: Fix transform logic and tableshape compatibility * [Fix] lua_shape: Fix registry to recursively resolve nested schemas * [Fix] lua_shape: Improve error safety * [Fix] Settings: Keep groups_*/symbols_* fields for runtime processing * [Fix] URL: Encode redirect URLs to handle unencoded spaces and special characters * [Fix] external_relay: Fix mixins and confighelp * [Fix] RBL: Fix plugin transform schemas * [WebUI] Update D3 libs with bug fixes and validation * [WebUI] Restore hover colors for symbols * [WebUI] Fix hover behavior outside status tables * [Minor] Whitelist: Use contemporary API for maps * [Minor] Migrate all plugins and libraries from tableshape to lua_shape * [Minor] Simplify configuration by removing use_*_map flags * [Minor] Add plugins registry with reworked mixins * [Test] Add comprehensive tests for URL deep processing * [Test] Isolate url_suspect tests with symbols_enabled
1 month ago
Release 3.14.0 * [Feature] Fuzzy check: Add HTML fuzzy hashing for structural similarity matching * [Feature] Fuzzy check: Add per-rule text_hashes toggle for HTML-only fuzzy rules * [Feature] Fuzzy check: Add structured checks configuration with backward compatibility * [Feature] Fuzzy storage: Implement full TCP protocol support with auto-switch * [Feature] Fuzzy check: Add TCP connection management and error handling * [Feature] URL: Add task:get_cta_urls() API for proper CTA domain extraction * [Feature] URL: Move CTA processing into dedicated module * [Feature] URL: Add url:get_hash() method for efficient deduplication without string conversion * [Feature] GPT: Add web search context support with Redis caching * [Feature] HTML: Add infrastructure for async URL rewriting with Lua bindings * [Feature] HTML: Add task:rewrite_html_urls() and task:get_html_urls() Lua API * [Feature] WebUI: Implement dark mode with theme toggle and auto detection * [Feature] Aliases: Add advanced resolution with loop detection for converging paths * [Feature] Milter: Add ESMTP argument parsing with Lua API access * [Feature] Milter: Add per-recipient ESMTP args parsing and metadata access * [Feature] Milter: Support array of positions for remove_headers operations * [Feature] Proxy: Add client IP preservation in message headers through chain * [Feature] Rspamc: Add milter.add_headers object format support to --mime * [Feature] Configwizard: Add Postfix integration wizard using postconf utility * [Feature] Build: Add comprehensive BSD workflows (FreeBSD, NetBSD, OpenBSD) with Lua version selection * [Feature] Build: Add automated code review GitHub Actions workflow with Rspamd-specific guidelines * [Feature] Build: Add Docker-based integration test suite with ASAN and real corpus * [Feature] Build: Add automatic public suffix list synchronization * [Feature] Multimap: Add support for symbols with leading numerals * [Feature] DMARC: Add Auto-Reply-To and Precedence headers to prevent out-of-office replies * [Feature] Platform: Add NetBSD memory usage tracking support * [Feature] Utilities: Add fuzzy Redis migration utility * [Feature] Bayes: Allow skipping local/authenticated mail in autolearn condition * [Feature] ARC: Add DKIM signing key API for flexible ARC signing * [Feature] Logger: Add type specifiers support for better formatting * [Feature] Heap: Add rspamd_heap_push_slot to eliminate double allocation * [Fix] DNS: Preserve req->pos during reply validation to prevent packet truncation on UDP-to-TCP retransmits * [Fix] DNS: Regenerate transaction ID before copying to TCP buffer to avoid collisions * [Fix] DNS: Fix nameserver round-robin when using /etc/resolv.conf * [Fix] DNS: Fix TCP uninitialized memory leak * [Fix] DMARC: Add batching and forced GC for Redis connections to prevent pool exhaustion * [Fix] DMARC: Validate and normalize batch_size to prevent fractional indexing and loop errors * [Fix] DMARC: Refactor reporting to use helper functions and async maps * [Fix] Allocator: Fix jemalloc/system malloc mixing in getline() to prevent crashes * [Fix] Allocator: Fix allocator mismatches in hiredis * [Fix] Allocator: Fix allocator mismatches in libucl * [Fix] Hyperscan: Use runtime version instead of compile-time for database validation * [Fix] Hyperscan: Auto-recreate invalid unserialized cache files on version mismatch * [Fix] Memory: Fix leaks in fuzzy storage khash tables * [Fix] Memory: Fix leaks in upstream address parsing * [Fix] Memory: Fix leaks in *-any address parsing * [Fix] Memory: Fix OpenSSL providers cleanup * [Fix] Memory: Fix UCL object memory leak in Lua integration * [Fix] Memory: Fix stat metadata tokenization leak * [Fix] Fuzzy TCP: Fix double-release in fuzzy_tcp_session * [Fix] Fuzzy TCP: Fix refcount leak in destructor * [Fix] Fuzzy TCP: Fix timeout handling and buffer overflow * [Fix] Fuzzy TCP: Fix endianness mismatch in framing protocol * [Fix] Fuzzy TCP: Fix race conditions and fd reuse bugs * [Fix] Fuzzy TCP: Use pure ev_timer for session timeouts * [Fix] Fuzzy TCP: Fix server replies and client event handling * [Fix] Shutdown: Keep srv events active during shutdown to track auxiliary processes * [Fix] ARC: Restore strict header ordering to comply with RFC 8617 * [Fix] ARC: Add ed25519 key support * [Fix] Composites: Implement two-phase evaluation for postfilter dependencies * [Fix] Composites: Use null-terminated string for symbol lookup * [Fix] URL: Refactor extraction to prevent DoS with hash-based deduplication * [Fix] URL: Add 50k URL limit with warning for DoS protection * [Fix] URL: Skip HTML_DISPLAYED URLs in CTA detection * [Fix] URL: Fix CTA priority preservation in extract_specific_urls * [Fix] Bayes: Improve Redis server discovery * [Fix] Bayes: Only bypass learn when header value matches * [Fix] ESMTP: Robust per-recipient parsing in milter with safe cursor advance * [Fix] ESMTP: Refcount ESMTP args in proxy_session_refresh to avoid use-after-free * [Fix] ESMTP: Correct Lua stack cleanup in lua_task_get_rcpt_esmtp_args * [Fix] HTML: Correct attribute value offset calculation for URL rewriting * [Fix] HTML: Add HTML entity encoding for URL rewriting * [Fix] HTML: Fix segfault due to incorrect HTML features access * [Fix] HTML: Fix frequency-based ordering in domain hashing * [Fix] HTML: Fix shingles hash generation bugs * [Fix] HTML: Fix memory leaks in shingles generation * [Fix] HTML: Fix memory management in html_cta.process_html_links * [Fix] HTML: Fix CSS class normalization in fuzzy tokens * [Fix] HTML: Fix cache key collision between text and HTML fuzzy hashes * [Fix] OpenBSD: Fix kinfo_proc structure member names * [Fix] OpenBSD: Disable Hyperscan (not available) * [Fix] FreeBSD: Fix zstd package name * [Fix] FreeBSD: Add IGNORE_OSVERSION for package version mismatches * [Fix] NetBSD: Setup pkgin and PKG_PATH before installing packages * [Fix] NetBSD: Fix missing dependencies and package names * [Fix] BSD: Remove -j flag from ninja in all BSD workflows * [Fix] Multimap: Handle symbols with leading numerals * [Fix] Aliases: Prevent creation of malformed email addresses * [Fix] Aliases: Fix alias loop detection for converging paths * [Fix] Aliases: Fix is_local_domain to support backend objects * [Fix] Aliases: Correct to_local when no recipients present * [Fix] Aliases: Fix set_addr validation to prevent malformed addresses * [Fix] MIME: Remove Authentication-Results and anonymize envelope-from in Received headers * [Fix] Mempool: Prevent double-free in destructor cleanup * [Fix] Rspamadm: Unbreak dnstool command * [Fix] Integration tests: Fix ASAN configuration and startup diagnostics * [Minor] Replace GHashTable with khash in fuzzy_check.c and lua_textpart_get_cta_urls * [Minor] Update cache key prefix to match module name in llm_search_context * [Minor] Fix llm_search_context to follow Rspamd idioms * [Minor] Refactor llm_search_context to use lua_cache module * [Minor] Address review comments in various modules * [Minor] Fix droid usage * [Minor] Use GPT-5 Codex for code reviews * [Minor] Update libucl with automatic stack management * [Rework] Prioritize CTA URLs in redirector and Lua helpers * [Rework] RBL configuration: Add new from selectors, content_urls checks, and lower_utf8 for hashed domains * [Rework] Make Bayes learn guards configurable * [Rework] Refactor element visibility control to use Bootstrap classes * [Rework] Use postconf utility for Postfix configuration in configwizard * [Rework] Remove Lua-level HTTP header parsing in ESMTP args getters * [Rework] Add CFG_REF_* macros with debug logging for config refcounting * [Rework] Move OpenSSL providers from global to libs_ctx * [Rework] Convert heap to fully intrusive kvec-based implementation * [Rework] Add specialized pool types for long-lived and short-lived allocations * [Rework] Improve memory pool destructors with smart preallocation based on pool type * [Project] Restrict code review workflow to authorized maintainers * [Project] Add Claude Code and Cursor AI assistant configuration * [WebUI] Replace Glyphicons with FontAwesome SVG icons * [WebUI] Update CodeJar to version 4.3.0 * [WebUI] Update Node.js and ESLint * [WebUI] Update D3-based visualization libs * [WebUI] Replace deprecated alert-error class with alert-danger * [WebUI] Add search syntax hint to history table filter input * [WebUI] Fix theme toggle default to auto * [WebUI] Keep classifiers list when request is skipped * [WebUI] Repopulate classifier dropdown * [WebUI] Add comment for removeEventListener * [WebUI] Fix icon rendering race condition in tab initialization * [Test] Add comprehensive Lua unit tests for HTML URL rewriting * [Test] Add unit tests for HTML URL rewriting patch engine * [Test] Add functional tests for HTML fuzzy hashing * [Test] Add ARC chain verification tests with multiple signatures * [Test] Add e2e for classifier dropdown population * [Test] Multimap symbol with leading numerals * [Test] Sync public suffix list automatically * [Test] Update JS linters * [Test] Fix integration test environment variable passing * [Test] Add detailed error output for integration test failures
2 months ago
Release 3.13.2 * [Feature] Fuzzy check: Add separate encryption keys for read and write operations * [Feature] DKIM: Add ED25519 support for DKIM signing and verification with OpenSSL version checks * [Feature] Vault: Add HashiCorp Vault KV version 2 support for DKIM key management * [Feature] MetaDefender: Add MetaDefender Cloud Lua module for SHA256 hash lookups * [Feature] LLM: Add user/domain context support for LLM-based classification with Redis-based conversation context * [Feature] DMARC: Add RUA address exclusion configuration option * [Fix] DKIM: Fix relaxed bodyhash calculation for lines with only spaces to comply with RFC 6376 * [Fix] DKIM: Fix ED25519 key loading to prevent memory corruption in union handling * [Fix] HTTP maps: Enforce server-controlled refresh intervals and prevent aggressive polling * [Fix] HTTP maps: Prevent time_t overflow in expires header processing * [Fix] Once received plugin: Fix duplicate symbol addition by changing break to return * [Fix] Redis: Propagate unused Sentinel options properly * [Fix] Fuzzy check: Fix reply decryption when using separate read/write keys * [Fix] Fuzzy check: Add fallback when only one specific encryption key is set * [Fix] Fuzzy check: Fix duplicate key filtering in reply decryption * [Fix] Fuzzy ping: Allow read/write servers configuration * [Minor] Fuzzy check: Refactor encryption key selection into helper functions * [Minor] Fuzzy check: Stop early when found a correct key * [Minor] Add cursor rules for development
3 months ago
Release 3.13.1 * [Feature] Archive module: Full support for encrypted ZIP archives with ZipCrypto and AES encryption * [Feature] Archive module: Both reading and writing of AES-encrypted ZIP archives is supported * [Feature] Archive module: Updated Lua bindings for libarchive * [Feature] Encrypted maps: Support for encrypted maps to enable new distribution scenarios * [Feature] Redis TLS: Configurable TLS connections in Redis backend * [Feature] Map helpers alignment: Enforce 64-byte alignment to prevent unaligned memory access * [Feature] Enhanced CLI for secretbox with additional security test coverage * [Fix] MIME encoding: Major overhauls and multiple fixes for MIME encoding logic * [Fix] MIME encoding: Improved handling and decoding of UTF-8 in MIME headers * [Fix] Learning system: Numerous fixes to learn checks and autolearn flag handling * [Fix] Learning system: Prevention of duplicate message learning * [Fix] Learning system: Extended multiclass learning test coverage * [Fix] Critical: Fixed bug when converting zero-length strings to numbers * [Fix] Critical: Fixed XML prolog detection in lua_magic module * [Fix] Build: Fixed build issues on 32-bit platforms * [Fix] Compatibility: Improved compatibility with Lua versions above 5.1 * [Fix] Empty input: Addressed issues with empty input handling in lua_magic * [Fix] Testing: Improved stability of automated testing with multiple test fixes * [Fix] Minor compatibility improvements (buffer allocation, missing cmath include)
3 months ago
Release 3.13.0 * [Conf] Add defaults * [Conf] Fix JB IDE damage * [Feature] Add a signal from main to workers for workers ready state * [Feature] Add lua_util.fold_header_with_encoding * [Feature] Add some convenience options to rspamc * [Feature] Add some more OS utility functions * [Feature] Add symbols proxy for piecewise changes * [Feature] Allow lua callback maps to be filled line by line * [Feature] Allow selectors in regexp maps expressions * [Feature] Allow to pass expression flags in the regexp plugin * [Feature] Detect part types in mime parser * [Feature] Resolve DNS nameservers names using getaddrinfo * [Fix] Bayes: Try to be bug-to-bug compatible * [Fix] Check skip_hashes for the returned hashes * [Fix] Fix DL lists initialisations * [Fix] Fix double free in the client... * [Fix] Fix end-to-end proxy compression * [Fix] Fix l= calculations again * [Fix] Fix lua state setting ambiguity * [Fix] Fix order of descriptor closing * [Fix] Fix probabilities overflow * [Fix] Fix rules setup * [Fix] Fix statfiles ordering * [Fix] Fix various corner cases and tests * [Fix] Fix whitelist options in the arc module * [Fix] GPT: Fix occasional damage * [Fix] GPT: fix processing of messages with no subject * [Fix] Prevent WebUI crash with empty RRD * [Fix] Store html attributes that are empty * [Fix] Try to fix learned order * [Fix] Use C++20 standard consistently to resolve ODR violations * [Fix] Use a more straightforward approach for learn cache * [Fix] fix error check in lua_dkim_tools.lua * [Project] Add CTA analytics engine * [Project] Add ability to create custom tokenizers for languages * [Project] Add controller learn endpoints * [Project] Add support of granular timeouts to plugins and maps * [Project] Add tests and fix stuff * [Project] Add tests for LLM provider, fix various issues with metatokens * [Project] Apply changes to bayes_expiry plugin * [Project] Create an isolated API for external tokenizers * [Project] Extract more features from HTML messages * [Project] Fix Lua API and some constexpr compatibility * [Project] Fix binary classification and lua scripts * [Project] Fix more calculation issues * [Project] Fix other classification and learning issues * [Project] Fix scoped compilation again * [Project] Fix symbols finalisation * [Project] Fix unlearn stuff * [Project] Fix various issues * [Project] Fix various other issues * [Project] Further updates * [Project] Implement backoff for upstreams revival * [Project] Implement more flexible http timeouts * [Project] Implement scoped compilation * [Project] Implement scoped regexp cache system * [Project] Multi-class classification project baseline * [Project] Rework rspamc to allow training of different neural types * [Project] Rework system of html tags to allow more tag types * [Project] Rework tokenizers initialisation * [Project] Some rework of the CTA defaults * [Project] Start implementation of the rules maps * [Project] Start to implement better revive strategy for upstreams * [Project] Store regexp rules state to avoid incomplete/orphaned rules * [Project] Support more common html attributes * [Project] Take button weight into consideration * [Project] Use re_cache scopes for maps * [Rework] Fix logger format string mismatch * [Rework] MIME detection via Lua Magic; enforce cfg in Lua task API * [Rework] Return back N-ary optimizations for arithmetic-alike expressions * [Rework] Use GLib agnostic type for words * [Rework]Refactor MIME detection via Lua Magic; enforce cfg in Lua task API * [Rules] Make bitcoin expression to use explicit flags
4 months ago
Release 3.12.0 * [CritFix] In lua-ucl disable macros and file variables by default * [Feature] Add keep-alive support * [Feature] Add some convenience methods * [Feature] Add support for separate read and write servers in fuzzy check * [Feature] Allow CDB files as external maps * [Feature] Allow to specify Redis version * [Feature] Allow to specify extra headers in Rspamd proxy * [Feature] Allow to specify log tag in proxy * [Feature] Allow to specify max log tag length for all log messages * [Feature] Allow to use HTTPS when connection to backends in proxy * [Feature] Output content for all maps * [Feature] Plugin to integrate with Contextal platform * [Feature] Show all maps status * [Fix] Add fail check for cfg transform for some corner cases * [Fix] Add header with reason everytime (not only for ham) and use correct value for header * [Fix] Add null check for master_conn->up in proxy backend error handler * [Fix] Allow 'Hash' in Access-Control-Allow-Headers * [Fix] Arc: Use tonumber when comparing * [Fix] As we have replxx library, always use it * [Fix] Backport some issues from libucl * [Fix] Filter invalid domains in fuzzy extra data * [Fix] Fix maps ids * [Fix] Fix race condition in maps loading by unlocking backend on switch * [Fix] Fix static maps description passing * [Fix] Fix variable propagation (no functional change) * [Fix] Fix various issues * [Fix] Greylist: Improve body hash calculations * [Fix] Known senders: More recipients test logic * [Fix] Known senders: Use the same logic as in the replies module * [Fix] Prevent crashes when accessing upstream address in self-scan mode * [Fix] Really fix local objects filtering, sigh... * [Fix] Update default URL for openphish * [Fix] Use bundled libfmt everywhere * [Fix] Use safe parsers everywhere except configuration * [Fix] correct logic error in milter_headers.lua: skip_wanted() * [Fix] initialize ollama result table * [Fix] libmime: declare comparators const for doctest 2.4.12 compatibility * [Project] Modernize cmake * [Project] Rework OSDep * [Rework] Replies: consider all recipients and use smtp ones * [Rework] Store shared maps data separately * [Rework] Use locks/loaded per backend for all maps
7 months ago
Release 3.11.1 * [Feature] Add 'noop' redis backend for scripts running * [Feature] Add Redis caching framework * [Feature] Add UTF8 `sub` and `len` variants for rspamd_text * [Feature] Allow adding timers to task (respecting symbols) * [Feature] Allow additional categories to be defined in GPT * [Feature] Allow fine-grained control on keys permissions * [Feature] Allow individual `read_only` flag per key * [Feature] Allow multiple lua scripts for fuzzy storage * [Feature] Allow to add periodic functions in Lua API * [Feature] Allow to disable rbls from map * [Feature] Allow to hash any Lua types * [Feature] Allow to store shingles as opaque Lua data * [Feature] Cache LLM replies * [Feature] GPT: Add ollama support * [Feature] GPT: Support reason adding * [Feature] Improve prompt and use plaintext instead of JSON * [Feature] Lua_task: Allow to load data into the existing task * [Feature] More additions * [Feature] Pass shingles to Lua scripts * [Feature] Preliminary implementation of LLM based anonymizing * [Feature] Support LLM models consensus * [Feature] Try to check maps earlier if their expires is too long * [Feature] Use debug module name from caller in lua_cache * [Fix] Add timer update before timer setting * [Fix] Allow to work with no ratelimits * [Fix] Always create ratelimit buckets * [Fix] Avoid collision hacks in mempool variables hash * [Fix] Expire neural ham and spam sets so they not hangup in redis indefinitely * [Fix] Fix crash on FreeBSD when Rspamd is built without hyperscan * [Fix] Make table digests consistent * [Fix] RBL: fix use of `content_urls` and `images` inside `checks` * [Fix] Verify key type to match DKIM signature type * [Fix] connIP is not correctly added to request * [Fix] properly close multipart/related boundary when adding text footer * [Rework] GPT: Use cache framework
10 months ago
Release 3.11.0 * [Conf] Add lua.local.d folder * [Conf] Add more ways to extend Rspamd configuration * [Conf] Allow to install examples conditionally * [Conf] Fix comments in elastic.conf * [Conf] Install examples * [Conf] Use full hostnames for SURBL * [CritFix] Allow to reupload scripts when Redis is restarted * [CritFix] Fix ARC-Seal signing * [CritFix] Properly check dynamic keys in fuzzy storage * [Feature] Add LRU cache for last filled ratelimit buckets * [Feature] Add include/exclude logic for headers * [Feature] Add ratelimit util to manage last filled ratelimit buckets * [Feature] Dump current ratelimits for fuzzy keys * [Feature] GPT: Try harder to find JSON in NN reply * [Feature] Improve address rotation algorithm * [Feature] Show stats in rspamadm * [Feature] Unify displayed part selection * [Fix] Avoid null-bytes in Log-Tag header value. * [Fix] Another story about char sign * [Fix] Check DNS limits when resolving MX/PTR records * [Fix] Dmarc: Properly encode structured headers * [Fix] Do not abort when OpenSSL is broken, report that to a user * [Fix] Fix dealing with happy eyeballs in SPF * [Fix] Fix memory leak in `lua_new_text` invocations * [Fix] Fix usage of the logging function * [Fix] Load "default" provider as well as "legacy" * [Fix] Milter headers: fold before encoding * [Fix] More fixes to rfc2047 encoding * [Fix] More things to fix * [Fix] Phishing: Fix many cases of FP for the same domains * [Fix] Remove USPOOF_SINGLE_SCRIPT_CONFUSABLE from the checks * [Fix] Restore propagation of spam/ham learns * [Fix] Sigh, another fix * [Fix] Some fixes to `rspamadm ratelimit` * [Fix] Some more fixes * [Fix] Use `dkim_signing` for `sign_headers` option * [Fix] Use correct type for keylen in lua_ucl_newindex * [Fix] Use cumulative timeout when dealing with TCP connections * [Fix] Use unsigned char for ragel machines * [Project] Add routine to strip attachments * [Project] Add tool to rspamadm * [Project] Initial stuff to implement messages anonymization * [Project] Keep Changelog updated * [Project] Various fixes and `rspamadm mime strip` command * [Rework] Allow `Content-Encoding` standard header for zstd compression * [Rework] Breaking: Actualize elastic module, support Elastic 8 & OpenSearch 2, add index policy with logs retention and many more * [Rework] Multimap should use only distinct text parts for content matching * [Rework] Replace fastutf with simdutf * [Rework] Rewrite rfc2047 encoding as it was totally broken * [Rework] Update hiredis to 1.2.0
1 year ago
Release 3.10.0 * [Conf] Add SenderScore RPBL return codes * [Conf] Add SenderScore Reputationlist RBL * [Conf] Increase scores for strange things in the archives * [CritFix] The max size for signing key is actually 65 bytes for p256 * [Feature] Add rspamadm secretbox command * [Feature] Add rspamd_cxx_unit_cryptobox for unit testing * [Feature] Add support for OpenSSL 3.0 * [Feature] Add support for OpenSSL 3.0 for DKIM * [Feature] Add tests for rspamd_cryptobox * [Feature] Add tooling to encrypt strings in Lua * [Feature] Allow differen modes for fuzzy rules * [Feature] Allow to set negative group score limit via `min_score` * [Feature] DMARC: Implement reporting.only_domains setting * [Fix] Add workaround for the bug in OpenSSL < 3.0.8 * [Fix] Another fix for Redis schema * [Fix] Another safe-guards for learning/classifying an empty message * [Fix] Apply the same workaround for signing keys * [Fix] Check message before trying to dereference pointer * [Fix] Do not install doctest stuff * [Fix] Encode headers that we send to milter add header * [Fix] Fix DCC `rep` handling * [Fix] Fix Redis scripts uploading when Redis is not ready * [Fix] Fix and rework various parts * [Fix] Fix compatibility with OSSL 1.0 * [Fix] Fix glib internals by setting locale in `rspamc` * [Fix] GPT: Fix bug in condition check * [Fix] Get rid of EVP_PKEY_CTX_set1_rsa_keygen_pubexp * [Fix] Iterate over dynamic keys in fuzzy storage * [Fix] Make tostring in UCL a bit less brain-damaged * [Fix] More bogus sizes fix * [Fix] Preserve the previous behaviour of RDNS_* checks * [Fix] Rework DMARC to correctly handle spaces in DMARC records Issue: #4906 * [Fix] Sign key != encryption key, omg * [Fix] Unify lua symbols registration * [Fix] Use proper keys when doing asymmetric encryption * [Project] Add API method to push unwrapped UCL object * [Project] Add more stuff to transparent UCL * [Project] Add parsing of key limits and expire date * [Project] Add ratelimit parsing for fuzzy keys * [Project] Allow manipulations with opaque UCL objects * [Project] Allow to change log tag from HTTP request * [Project] Implement expiration * [Project] Implement per-key ratelimit * [Project] Move ratelimit parsing stuff to a common library * [Project] Remove NIST (OpenSSL) mode from cryptobox * [Project] Remove NIST mode from everywhere * [Project] Rework ratelimits check * [Project] Some more fixes * [Project] Start support of MIME UTF8 * [Project] Try to allow more transparent access of ucl elements * [Rework] Allow more flexible keypair encoding * [Rework] Breaking: Rewrite cfg transform and remove legacy * [Rework] Change fuzzy error symbols * [Rework] Change the logic of skipping symbols * [Rework] Clean up legacy code * [Rework] Implement new replies logic on the server's side * [Rework] Remove control block support * [Rework] Resolve rdns in a separate function * [Rework] Use __builtin_cpu_supports where possible * [Rework] Use a more straight structure for DKIM keys * [Rules] Fix some old rules
1 year ago
Release 3.9.0 * [CritFix] Protect regexp matcher from regexps with empty patterns * [Feature] Allow adding X-CMAE-Score header * [Feature] Allow custom milter quarantine and tempfail messages * [Feature] Allow to specify minimum weight in GPT plugin * [Feature] Cloudmark: Add scores_symbols setting * [Feature] Further optimization to the hot path * [Feature] Make min_resolve_interval configurable * [Feature] Report slow synchronous rules * [Feature] Rspamc: add `files-list` option * [Feature] Support metrics command in normal/proxy workers * [Feature] Support reply in message pack format * [Feature] Use msgpack in the client * [Feature] Verify sanity of l= tag in DKIM * [Feature] milter_headers: support not removing headers * [Fix] Allow autolearn for the controller worker * [Fix] Allow spaces in DMARC records * [Fix] Allow to set 0 as number of rows to disable roll history * [Fix] Always set the unlearn flag when relearning * [Fix] Apply detection phase if fasttext could not detect language * [Fix] Backport fixes from libucl * [Fix] Backport multiple fixes from libucl * [Fix] Change expiration logic for redis_cache * [Fix] Check nconns when firing the final termination event * [Fix] Do not apply tableshape for known senders * [Fix] Do not crash if symbol is missing in the metric * [Fix] Do not read out-of-boundary when doing base64 encoding * [Fix] Do not save multipatterns to FS in certain cases * [Fix] Fix another corner case that allows candidates to be freed without init * [Fix] Fix documentation nesting * [Fix] Fix dynamic_symbols in the multimap plugin * [Fix] Fix inconsistent nesting in mime parts * [Fix] Fix parsing of maps definitions * [Fix] Fix shared memory proxying when compression is set * [Fix] Fuzzy add/delete handlers are badly broken for compression * [Fix] Honor dynamic thresholds for greylisting module * [Fix] If we have one statfile disabled we need to disable all * [Fix] Increase/decrease hash_key value for tokens depending on is_unlearn * [Fix] Libucl: Fix messagepack autodetection * [Fix] Resgore the collaterally removed line * [Fix] Unset autolearn flag if we learn explicitly * [Fix] Update `nconns` in proxy * [Fix] Update greylisting threshold as it can be easily dynamic * [Fix] Update is_spam in learned_ids when relearning * [Fix] Use explicit timeouts in the metadata exporter * [Fix] metric_exporter: avoid sending `nil` in metric values * [Project] Add GPT plugin * [Project] Add a tool to perform statistical analysis of classifiers * [Project] Finally fix slow timer behaviour * [Project] Implement fuzzy check retransmits backpressure * [Project] Improve stats processing * [Project] Reduce default window size of OSB tokenizer to 2 * [Project] Rework grow factor * [Project] Start optimization of userdata hashing * [Project] Use libarchive for troublesome archives * [Rework] Allow metrics endpoint to be enabled for each scanner * [Rework] Allow multiple conditions for dkim signing when using vault/redis * [Rework] Introduce dynamic_rate_limit for ratelimits * [Rework] Serialize control commands * [Rework] Update fmt to version 11 * [Rework] Use direct hashing instead of string hashing * [Rules] Added rules for detecting likely malware
2 years ago
Release 3.7.1 * [CritFix] Fix leak in `gzip` function * [Feature] Add ICAP Content-Type and Filename * [Feature] Add `logging`->`task_max_elts` option * [Feature] Add utility to split string like stuff for C++ code * [Feature] Allow to set HTTP auth parameters for the maps * [Feature] Check for plugin configuration errors on `configtest` * [Feature] `known_senders` plugin * [Feature] Use backward-cpp instead of manual libunwind stuff * [Feature] rbl: support checking numeric URLs in isolation * [Fix] CMakeLists.txt remove whitespace added by linter as it makes tests fail * [Fix] Change Date: header location to conform with RFC * [Fix] Correct format pattern for RE tree tempfile name * [Fix] Correct format string for unw_word_t * [Fix] Do not accept invalid ucl object types * [Fix] Do not pollute public headers with libev internals * [Fix] Do not set output type if list application failed * [Fix] Fix `url:set_redirected` method * [Fix] Fix format string and some length issues * [Fix] Fix grammar definition for content-disposition attributes * [Fix] Fix lua schema enrichment logic for Redis params * [Fix] Fix lua stack corruption when logging large tables * [Fix] Fix merge table utility * [Fix] Fix output of non-RSA DKIM keys * [Fix] Fix some corner cases of single-host urls parsing * [Fix] Fix various issues in the `url_redirector` plugin * [Fix] MISSING_MIMEOLE: avoid matching messages from Android GMail app (#4561) * [Fix] Prevent DNSWL sabotage * [Fix] Try to fix unzip function * [Fix] rbl: really fix dependency registration when symbols_prefixes is used * [Fix] rspamadm mime: arguments beginning with letter `t` * [Rework] Breaking: return back to semver * [Rework] Move rcl logic to C++
2 years ago
Release 3.6 * [Conf] Add `one_shot` to some specific multimap rules * [Conf] Add language detection configuration * [Conf] Add missing attributes for the language detection configuration * [Conf] Remove outdated composite rules * [Feature] Add `sentinel_password` option * [Feature] Add ability to deny specific fuzzy flags by default * [Feature] Add controller endpoint to get fuzzy hashes from messages * [Feature] Add extra symbol when URL redirector reaches nested limit * [Feature] Add function to transliterate utf8 to ascii with some normalisation * [Feature] Add html parsing limit * [Feature] Add order to urls structure * [Feature] Add some missing functions to `lua_rsa` library * [Feature] Allow fuzzy workers to exchange blocked information * [Feature] Allow to have weak flags in fuzzy storage * [Feature] Allow to read options from maps in the multimap plugin * [Feature] Allow to use other methods when fasttext detection is enabled * [Feature] Count stats per key per flag * [Feature] Finish all features of dkim_keygen in Lua * [Feature] Khash: Allow static initialisation * [Feature] Maps: Add on_load support * [Feature] Preliminary implementation of dynamic composites * [Feature] Process HTML parts before text ones * [Feature] Reorganise struct rspamd_url to be 64 bytes size * [Feature] Save fuzzy ratelimit buckets * [Feature] Use in-place deflation for strings * [Feature] external_relay: add ip_map strategy * [Fix] Avoid race between config new/free by using a counter * [Fix] Do not use `rspamadm.dkim_keygen` * [Fix] Feed fasttext language model with the pre-tokenized words * [Fix] Fix `rspamd_has_only_html_part` * [Fix] Fix an old issue with order of destruction race between redis pool and lua * [Fix] Fix format string usage * [Fix] Fix parsing due to old bug revealed * [Fix] Fix parsing of the mask values that are invalid * [Fix] Ignore non-unique stop words * [Fix] Include the last character when parsing the last header with no value * [Fix] More fixes to fuzzystat * [Fix] Set proper counter * [Fix] Try harder to clean pending bucket * [Fix] Try harder to remove bad hyperscan files * [Fix] Update stats before encryption... * [Fix] dmarc gramar - allow spaces before ";" * [Fix] rbl: fix dependency registration when symbols_prefixes is used * [Fix] remove obsolete rspamd-redirector files * [Fix] test external_relay: count should always be the last rule, as it have no matching condition * [Project] Allow to register multimap symbols dynamically * [Project] Implement fasttext language detection * [Rework] Default max shots must not influence options * [Rework] Write dkim keygen tool in lua * [Rules] Add thread hijacking composite rule
2 years ago
3 years ago
Release 3.4 * [CritFix] Restore compatibility with the integrations and headers alterations * [Feature] Milter_headers: Add `x-rspamd-action` routine * [Feature] Share hyperscan database among processes * [Fix] Another corner case in url parsing * [Fix] Another fix for the enable password * [Fix] Another try to fix close method in lua_tcp * [Fix] Fix additional fields in the Redis schema * [Fix] Fix emoji joiner FP * [Fix] Fix favicon.ico Content-Type header * [Fix] Fix hang when close is used * [Fix] Lua_tcp: Sigh, another try to fix `close` invocation * [Fix] Mx_check: Cache the fact of a missing MX record * [Fix] Try to fix parsing of the unencoded `>` characters in html attributes * [Fix] Try to fix the case where password == enable_password * [Project] (Re)implement hyperscan caching * [Project] Rework cleanup * [Project] Synchronize hyperscan caches via the main process * [Rework] Convert multipattern to use hyperscan tools * [Rework] Make http normalize path function a generic function * [Rework] Split locked and unlocked files, as mmap does not need flock normally * [Rework] Start movement of the hyperscan related routines into a single unit * [Rework] Store the current worker, so other libraries could use this information * [Rework] Use blocking socket for IPC between main and workers * [Rework] Use more predictable size for commands buffers * [Rules] Do not insert ONCE_RECEIVED_STRICT on RDNS missing * [Rules] Reduce score of HTTP_TO_HTTPS - subject to remove completely
3 years ago
Release 3.3 * [Conf] Add missing groups for whitelist module symbols * [CritFix] Neural: Fix keys regression after #3968 * [Feature] Accept upstream in lua_tcp * [Feature] Add ability to statically maintain disabled/enabled patterns * [Feature] Add function to store upstreams for HTTP urls * [Feature] Allow augmentations set in Lua API * [Feature] Allow lua_http module to accept upstreams * [Feature] Allow to limit write access to fuzzy storage by key * [Feature] Allow to sort symbols output * [Feature] Check content for binary stuff before dumping it to Lua * [Feature] Implement symbols augmentations * [Fix] Add missing flags * [Fix] Add more sanity checks for rua in dmarc_report * [Fix] Adjust length of the fuzzy checks for short text parts * [Fix] Another try to fix add headers compatibility logic * [Fix] Another try to fix race condition in the runtime destruction * [Fix] Avoid cyclic references in symcache and fix memory leaks * [Fix] Avoid overriding IP with Sender IP * [Fix] BAD_REP_POLICIES did not trigger when message was classified as spam by Bayes * [Fix] Bind AF_UNIX DGRAM client connection to annonymous address * [Fix] Disable IPv6 lookups for Blocklist.de RBL * [Fix] Distinguish dynamic and static items * [Fix] Dkim: Ignore unknown DKIM kv pairs as stated in RFC * [Fix] Dmarc report: Use local timezone instead of GMT * [Fix] Do not exclude authenticated users from URIBL lookups * [Fix] Empty envelopes should not be emitted as arrays (json+messagepack) when populated envelopes are objects. This greatly complicates decoding in strictly typed languages. * [Fix] External_relay: Restore the originating hostname check * [Fix] Fix DKIM keys with spaces still allowing errors on invalid base64 * [Fix] Fix copying of sockaddr_un addresses * [Fix] Fix crash with cname replies * [Fix] Fix dependencies propagation * [Fix] Fix iteration over milter headers * [Fix] Fix ordering when sorting symcache * [Fix] Fix reading of the cached maps * [Fix] Fix several issues with the HTTP keepalive parsing * [Fix] Fix stack smashing * [Fix] Fix synchronous auth/select in lua_redis * [Fix] Fix various symcache issues * [Fix] Ignore all (I hope) unknown DKIM signature KV pairs * [Fix] Ignore directories in RarV5 archives * [Fix] Libucl: avoid memory leak on objects merging * [Fix] Lua_tcp: Another try to fix closing logic * [Fix] Mempool: Fix alloc_array function to actually multiply nmembers by size * [Fix] Only check allowed fuzzy worker update ips for non-unix sockets * [Fix] Plug memory leak in regexp destruction with pcre2 * [Fix] Properly check the original email flag * [Fix] Properly deal with `get_symbol/get_metric_symbol` ambiguity * [Fix] Properly parse expressions atoms * [Fix] Properly set `Host` in rspamd_proxy * [Fix] Rbl: Fix received positioned checks * [Fix] Remove check for a score with no symbol being registered * [Fix] Same fix for lua_tcp * [Fix] Skip cname records when processing SPF records * [Fix] Skip sending dmarc reports in no-opt mode fixes https://github.com/rspamd/rspamd/issues/4241 * [Fix] Stop slow timer on task destruction * [Fix] Symcache: Do not use C style comparators in C++ sorts * [Fix] Try to avoid a corner case for `@` pattern * [Fix] Try to fix dkim reputation adjustements * [Fix] Try to fix passthrough results processing logic * [Fix] Try to fix the mess with read only flag * [Fix] Upstreams: Don't ignore revive_time config option * [Fix] Use proper format string, sigh... * [Fix] Use space category in ragel automata to resolve space characters * [Fix] Zstd: Fix compression with the new Zstd API * [Fix] milter_headers: Header fields may be inserted at wrong position. * [Project] Add experimental HTTP statistics backend * [Project] Add more methods for symbols addition * [Project] Add raii_sink file helper * [Project] Add some more methods * [Project] Add symbols processing methods * [Project] Allow `=` separated augmentations to be treated as kv pairs * [Project] Allow to extract augmentation values * [Project] Few more methods * [Project] Fix on conditions * [Project] Further efforts to make a more consistent architecture * [Project] Further rework * [Project] Further rework tracking * [Project] Further split of the code * [Project] Get rid of C style ctors/dtors * [Project] Http_stat: Notice statfiles when creating runtime * [Project] Implement dynamic items lookup and processing * [Project] Implement item finalization * [Project] Implement more methods * [Project] Implement runtime creation * [Project] Implement settings processing + some neats * [Project] Implement some conditions checks * [Project] Implement validation logic * [Project] More methods * [Project] Move runtime cache part to a separate unit * [Project] Move some more methods * [Project] Re-implement counters method * [Project] Reimplement dependencies processing * [Project] Remove obsoleted methods * [Project] Remove old code (finally) * [Project] Rework symbols execution * [Project] Some more adjustments in symbols registration * [Project] Start rewrite symcache in c++ * [Project] Support augmentations with values * [Project] Symcache: Use ordered filters to avoid extra lookups * [Rework] Another movement * [Rework] Augmentations can now imply flags * [Rework] Further steps * [Rework] Further work on deps processing * [Rework] Implement cache resorting * [Rework] Isolate disable/enable logic for the configuration ucl objects * [Rework] Move item implementation to a separate header * [Rework] Multimap: Avoid prefilters usage where augmentations can be used * [Rework] Pass upstream when sending TCP requests * [Rework] Re-implement cache sorting * [Rework] Reimplement saving/loading the cache items * [Rework] Reiterate on priorities * [Rework] Rework files structure * [Rework] Rewrite rspamc in C++ * [Rework] Simplify scores check and extend it to pre/post filters * [Rework] Switch minimum C++ standard version to C++20 * [Rework] Try to fix the mess with types & flags * [Rework] Use another version of hash table from the same author * [Rework] Use dynamic items for calling callbacks * [Rework] Use dynamic items in the callbacks * [Rework] Use hash map for id->symbol mappings * [Rework] Use khash instead of uthash in rdns compression logic
3 years ago
Release 3.2 * [Conf] Score MIME_OBFUSCATED_ARCHIVE to 8 points * [Conf] Set one_shot for URIBL rules by default * [CritFix] Fix upstreams name resolution when there is also a port * [Feature] Add ROC feature to neural network plugin * [Feature] Add public suffic compilation utility * [Feature] Add support of Cloudmark * [Feature] Allow hyperscan for ppc64, as vectorscan now suports it. * [Feature] Allow to skip DNS resolution for keep-alive connections * [Feature] Aws_s3: Allow to store large parts separately * [Feature] BIMI: Add preliminary version of the BIMI plugin * [Feature] JSON endpoint for querying maps * [Feature] Lua_magic: Add a sane CSV heuristic * [Feature] Lua_mime: Add schema for message transfer * [Feature] Output average scan time in /stat endpoint * [Feature] Show average scan time in `rspamc stat` output * [Fix] Add guards to avoid race condition on TCP connection * [Fix] Allow spaces in DKIM key records * [Fix] Apply the similar fix to the url_reputation * [Fix] Avoid overwriting whitelisted_signers_map * [Fix] Backport PR from libucl * [Fix] Clear SSL errors * [Fix] ClickHouse cleanup of old partitions * [Fix] Do not double call error handler on ssl errors in the timeout path * [Fix] Do not forget to clear pointers on IOC reset * [Fix] External_relay: Remove useless check of the map value * [Fix] Find suspicious url encodings that could break url extraction * [Fix] Fix HTTP(s) client timeout * [Fix] Fix exclude flags setting * [Fix] Fix expanding of the variables * [Fix] Fix host header usage in lua_http * [Fix] Fix http maps shared memory cache cleanup * [Fix] Fix logic in HTML processing FSM * [Fix] Fix parsing of the compound mailto urls * [Fix] Fix processing captures from pcre2 * [Fix] Fix removing from khash * [Fix] Fix stuctured headers pushing * [Fix] Further fix for i386 compilation * [Fix] Improve duplicate settings error reporting * [Fix] Lua: task:remove_result didn't work in some cases * [Fix] Output service parts as well * [Fix] Phishing: Deal with phishing + redirected URL * [Fix] Phishing: Fix finding domains in the phishing map * [Fix] Plug memory leak by using mempool for a copied address * [Fix] Properly find the request and the number of requested entries * [Fix] Rbl: Fix inversed logic of the url_full_hostname * [Fix] Read file maps if they were not pre-read during preload * [Fix] Restrict x86_64 assembly to x86_64 * [Fix] Return a real number of recipients when dealing with aliases * [Fix] Rework unshedule DNS request function * [Fix] Support definition of ungrouped symbol in conf file, use group info from lua or other conf file * [Fix] Unschedule DNS request when clearing IO channel * [Fix] When checking for phishing, we need to convert punicode -> UTF8, not vice versa * [Fix] lua_cfg_transform - actions without score (discard) * [Fix] lua_cfg_transform - silly break break actions * [Fix] ratelimit - symbol per bucket * [Project] BIMI: Fix helper integration issues * [Project] Further DNS over TCP architecturing * [Project] Rdns: Add more functions for TCP based requests * [Project] Rdns: Add preliminary reading logic for TCP channels * [Project] Rdns: Add reaper for inactive TCP connections * [Project] Rdns: Add timeout logic for TCP requests * [Project] Rdns: Do not treat TCP channels failure as fatal * [Project] Rdns: Fix TCP connection mess * [Project] Rdns: Fix TCP stuff cleanup * [Project] Rdns: Fix various ownership issues * [Project] Rdns: Implement TCP writing logic * [Project] Rdns: Initial support of TCP IO channels * [Project] Rdns: More fixes in TCP handling * [Project] Rdns: Restore the previous EDNS0 size * [Project] Rdns: Send truncated replies via TCP * [Project] Rdns: Unregister TCP requests * [Rework] Allow to restore SSL handlers after keepalive pooling * [Rework] Allow to set a different behaviour for actions from settings * [Rework] Include SSL flag into keepalive hash * [Rework] Make `rspamadm dmarc_report` default behaviour more sane * [Rework] Mempool: Use explicit alignment * [Rework] Rdns: Use faster and more compact hash table for DNS requests * [Rework] Rework SSL flag operations * [Rework] Take disabled flag into account * [Rework] Timeouts are now global per event and not reseted by IO activity * [Rework] Use xxh3 as a default hash and fix memory/alignment issues * [Rules] Fix old rules to stop global functions usage * [Rules] Fix symbol for DKIM temporary failure * [Rules] Remove ancient and inefficient rules * [Rules] Slightly reduce MULTIPLE_FROM score
4 years ago
Release 3.1 * [Feature] Add junk_threshold for autolearn * [Feature] Add neural test command * [Feature] Antivirus: Allow to set fake eicar patterns for testing AV engines * [Feature] Lua_cdb: Add cdb building interface * [Feature] Ratelimit: Add per bucket configurations * [Feature] S3: Allow to store structured data in messagepack * [Fix] Add concept of uncancellable events to prevent use-after-free * [Fix] Add temporary guard to prevent linked list exploitation * [Fix] Another rework of the ucl hashing * [Fix] Another try to fix references safety * [Fix] Another try to fix rspamd_text passing in the selectors * [Fix] Avoid copy for received structure as it has raw C pointers * [Fix] Avoid dangling reference * [Fix] Correctly check numeric URLs in URL DNS lists * [Fix] Delete the correct pointer type * [Fix] Dmarc: Always lowercase domain * [Fix] Fix compilation of the hyperscan databases with errors * [Fix] Fix hash table lookup * [Fix] Fix http message flag shift * [Fix] Fix parsing of the from_hostname when it is an IP address * [Fix] Fix parsing of the unquoted attributes in HTML * [Fix] Fix passing of rspamd_text in selectors pipelines * [Fix] Fix rubbish QP sequences decoding * [Fix] Fix some complicated case with the closing tags parsing * [Fix] Fix the case when l tag is too small * [Fix] Html: Fix the case where only bgcolor is explicitly set * [Fix] Libucl: Fix deletion from ucl objects * [Fix] Namespace and add metadata for OpenMetrics, fix interleaving * [Fix] Plug memory leak in http settings reload * [Fix] Preserve SPF top record in the mempool variable * [Fix] Remove aarch64 GC64 workaround * [Fix] Remove bogus G_LIKELY * [Fix] Spf: Do not parse non TXT DNS replies as TXT replies * [Fix] Try to use on_connect/on_disconnect callbacks to handle internal Redis failures * [Fix] buffer overflow in rspamc counters * [Fix] fix static building * [Fix] lua_scanners - message_min_words logic * [Fix] src/lua/lua_mimepart.c: fix null dereference * [Project] Add constant iterators * [Project] Add helper library to handle mime strings in a more safe matter * [Project] Add preliminary support of CDB bayes dump * [Project] Add trim operations * [Project] Allow mempool allocated mime strings * [Project] Cdb: Finish backend implementation * [Project] Cdb: Fix configuration load * [Project] Cdb: Use shared data between cdb statfiles * [Project] Cdb: continue statistics backend implementation * [Project] Finish received headers rework part * [Project] Move C++ specific declarations to C++ header * [Project] Rework received headers parsing to C++ * [Project] Start using of the new received structure * [Project] Start work on cdb backend * [Rework] Further rework of the redis pool * [Rework] Redis_pool: fix issues found * [Rework] Rework learn and add classify condition * [Rework] Save invisible content to a separate buffer * [Rework] Start rewriting of the redis pool logic * [Rules] Improve zero font rule Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
4 years ago
Release 3.0 * [Conf] Align ARC scores with DKIM scores * [CritFix] Neural: Fix sorting application * [Feature] Add a simple dumper for bayes tokens * [Feature] Add lua_maps.fill_config_maps function * [Feature] Add preliminary exporter to AWS S3 * [Feature] Add preliminary restore bayes support * [Feature] Add race condition protection against hs_helper restarts * [Feature] Add rspamd_utf8_strcmp utility * [Feature] Add zstd streaming API * [Feature] Allow to log severity level explicitly * [Feature] Allow to save and show attachment name when inserting AV scan results * [Feature] Allow to sort urls for Lua * [Feature] Allow to specify different timeouts/retransmits for fuzzy rules * [Feature] Aws_s3: Allow to compress data stored * [Feature] CMakeLists.txt: Change check and run-test to use rspamd-test-cxx * fixes #3807 * [Feature] Dmarc_report: allow sending reports in batches * [Feature] Fuzzy_check: Allow to disable subject when making short text hash * [Feature] Lua_cryptobox: Add keyed ssl hash functions via HMAC * [Feature] Lua_task: Add get_urls_filtered method * [Feature] Make monitored checks less frequent * [Feature] Milter_headers: Add x-rspamd-pre-result header * [Feature] Neural: Allow to balance FP/FN for the network * [Feature] Ppopagate monitored errors from rbl module * [Feature] Pyzor calculate score dynamically Count - WL-Count of default_score in percent * [Feature] Rbl: Distinguish flattened and non-flattened selectors in RBL requests * [Feature] Re-add pyzor support * [Feature] Settings: add ip_map check and rework structure slightly * [Feature] Spamassassin: Allow to set the default priority for SA scores * [Feature] Strip smtp comments from message id * [Feature] add SYSTEM_ZSTD cmake option To use the system zstd instead on the bundled version * [Feature] external_relay plugin * [Feature] rspamadm clickhouse neural_train subcommand * [Fix] #3400 milter_headers: fix inverted logic for extended_headers_rcpt * [Fix] ASN: fix _FAIL symbol for when main symbol is disabled * [Fix] Add a special logic for text part with no text extraction * [Fix] Add diacritics flag for several eu languages * [Fix] Another FSM fix to accomodate possibility of multiple consequent ? * [Fix] Avoid curse of dynamic array referencing * [Fix] Avoid reinitialising neural settings * [Fix] Check remain before processing TXT records * [Fix] Enable error multiplier on http errors * [Fix] Finally rework parsing entities logic * [Fix] Fix '==' parsing in the content type attributes parser * [Fix] Fix IPv6 expansion for SPF macros * [Fix] Fix Mozilla Message-ID detection * [Fix] Fix an edge case in BITCOIN_ADDR rule * [Fix] Fix brain-damaged behaviour when http request has a custom Host header * [Fix] Fix check of limits in email address parsing * [Fix] Fix copy&paste error and rework * [Fix] Fix expressions logic for and/or and float values * [Fix] Fix fuzzy retransmits * [Fix] Fix http maps with no or invalid expires data * [Fix] Fix last quote character parsing in the content-type state machine * [Fix] Fix normalisation flags propagation * [Fix] Fix overflow when appending many broken tags * [Fix] Fix parsing of rfc2047 tokens with '?' inside * [Fix] Fix phishing flag set * [Fix] Fix rfc2047 embedded into rfc2231 pieces in special headers * [Fix] Fix round-robin rotation * [Fix] Fix searching for symbols * [Fix] Fix storing of the regexps inside variant * [Fix] Fix tokenization near exceptions * [Fix] Fix visibility calculations * [Fix] Html: Attach inline tags to the structure * [Fix] Html: Do not treat empty tags as block tags * [Fix] Ical: Do not extract urls from all flags using merely specific ones * [Fix] Initialise symcache even if it cannot be loaded properly * [Fix] Lua_fuzzy: Remove text parts check when checking image dimensions * [Fix] Lua_maps: Fix adjustments for the map type in the complex map definitions * [Fix] Lua_task: Fix deleted symbols in has_symbol/get_symbol * [Fix] Move metric and symcache link from validation to the init stage * [Fix] Oletools: Another try to fix table sorting * [Fix] One more default behaviour fix * [Fix] Phishing: Rework urls processing * [Fix] RBL: was missing some config schema * [Fix] Replies: Fix 'Reply-To' handling in task:get_reply_sender * [Fix] Rework metrics handling * [Fix] Save symcache on exit * [Fix] Selectors: Filter nil elements in lists * [Fix] Selectors: Properly fix implicit tostring for nils * [Fix] Try to fix some broken code in DMARC reporting plugin * [Fix] Urls: Fix processing of html urls when it comes to the flags * [Fix] Use proper buffer length * [Fix] Various visibility fixes * [Fix]: ASN: dns cb func should also return in case of an error * [Project] Add a simple css rule definition * [Project] Add css style skeleton * [Project] Add css syntax (adopted from ebnf) * [Project] Add css_selectors * [Project] Add doctest unit testing library * [Project] Add expected library * [Project] Add fmt library for simple string ops * [Project] Add fu2 library to better functions abstractions * [Project] Add hashing method * [Project] Add parsers skeleton * [Project] Add preliminary support of vcard parser * [Project] Add process exceptions for invisible text * [Project] Add some methods for css parser * [Project] Allow static libstdc++ * [Project] Another whitespace hack * [Project] CSS: Various fixes in the declarations and values parsing * [Project] Cpp: Add robin-hood hash map library * [Project] Css: Add AST debug * [Project] Css: Add colors conversion functions * [Project] Css: Add dimensions handling * [Project] Css: Add display value support * [Project] Css: Add frozen library from https://github.com/serge-sans-paille/frozen/ * [Project] Css: Add opacity support * [Project] Css: Add parser helpers to simplify debugging * [Project] Css: Add preliminary stylesheet support * [Project] Css: Add rules processing functions and tests * [Project] Css: Add simple selectors unit tests * [Project] Css: Add some c++ unit tests * [Project] Css: Add some debug methods * [Project] Css: Add some debug statements for the css parser * [Project] Css: Add some logical skeleton for declarations parser * [Project] Css: Add url/function tokens * [Project] Css: Allow at rules parsing * [Project] Css: Declarations parsing logic skeleton * [Project] Css: Enable conditional css parsing support from the HTML parser * [Project] Css: Finish generic lexer cases * [Project] Css: Fix HSL conversion * [Project] Css: Fix minus parsing * [Project] Css: Fix parser consumers nesting * [Project] Css: Fix parsing of the qualified rules * [Project] Css: Fix rules merging * [Project] Css: Further fixes to lexer * [Project] Css: Further steps to parse css colors + rework * [Project] Css: Further work on parser's methods * [Project] Css: Implement backlog of css tokens * [Project] Css: Implement numbers and ident parsers * [Project] Css: Implement simple css selectors lookup * [Project] Css: Implement styles merging * [Project] Css: Make debug strings json like to simplify tests * [Project] Css: Minor adjustments * [Project] Css: More meat to the lexer * [Project] Css: Move some of the tests to the doctest * [Project] Css: Projected a parser * [Project] Css: Properties attachment logic * [Project] Css: Remove ragel from build targets (maybe keep for reference) * [Project] Css: Rework css block structure * [Project] Css: Rework flags of css properties * [Project] Css: Rework tokens structure * [Project] Css: Several fixes + tests * [Project] Css: Simplify checks * [Project] Css: Simplify debug code * [Project] Css: Start css selectors parsing logic * [Project] Css: Start semantic parsing for rules * [Project] Css: Start stylesheet implementation * [Project] Css: Tidy up lambdas * [Project] Css: rework tokeniser * [Project] Dmarc: Add dmarc report tool (WIP) * [Project] Dmarc: Add munging configuration * [Project] Dmarc: Add preliminary munging logic * [Project] Dmarc: Fix header removal * [Project] Dmarc: Fix munging logic * [Project] Dmarc: Use full recipient address instead of a domain map * [Project] Dmarc: Use zlists for dmarc reports * [Project] Dmarc_report: Add message generation logic * [Project] Dmarc_report: Add preliminary sending support * [Project] Fix lua bindings * [Project] Fix xml/sgml tags processing * [Project] Handle new modification * [Project] Html/CSS: Add transform from a CSS rule to html block * [Project] Html/CSS: Link html and css styles * [Project] Html/CSS: Switch styles parsing to css parser * [Project] Html/Css: Fix some issues found * [Project] Html/Css: Implement visibility rules for a block * [Project] Html: Add more tests cases and fix some more corner issues * [Project] Html: Add rows display type support * [Project] Html: Allow decode entities function to normalise spaces + unit tests * [Project] Html: Another rework of the tags structure * [Project] Html: Another try to fix unbalanced cases * [Project] Html: Fix crossing spans * [Project] Html: Fix parent propagation * [Project] Html: Further rework of the html parsing stuff * [Project] Html: Implement logic for tags pairing * [Project] Html: Implement rawtext state machine * [Project] Html: Insert closing tags as well :( * [Project] Html: More fixes * [Project] Html: More fixes * [Project] Html: More spaces logic fixes * [Project] Html: One more attempt to write text content * [Project] Html: Replace \0 in html content * [Project] Html: Rework img/a tags handling * [Project] Html: Rework propagation method * [Project] Html: Rework tags placement * [Project] Html: Rework transparency logic * [Project] Html: Support 'hidden' attribute * [Project] Html: Try another approach to append tags content * [Project] Html: Try to deal with bad unknown tags properly * [Project] Lua_aws: Add canonicalisation utility * [Project] Lua_aws: Add function to produce AWS Authorisation header * [Project] Lua_aws: Implement request signing * [Project] Lua_mime: Add lua_mime.modify_headers routine * [Project] Lua_task: Add modify_header method * [Project] Lua_task: Allow to extract modified headers * [Project] Make unescape code public for unit testing * [Project] More fixes for closed tags * [Project] More fixes to calculations * [Project] Rework API for the modified headers * [Project] Rework html visibility rule * [Project] Skeleton of the css library * [Project] Start headers modification API structure * [Project] Start working on AWS Lua API * [Project] Use lua_mime to modify headers * [Project] Use modified headers on dkim signing * [Project] Use string_view to constexpr variant unpacking * [Rework] Add composites manager concept * [Rework] Add tags definitions * [Rework] Allow C code to be compiled with C++ compiler * [Rework] Clickhouse: Store url flags * [Rework] Composites: Rewrite the composites logic * [Rework] Composites: Start rework of the composites framework * [Rework] Dmarc: Move check policy function to the common utils * [Rework] Dmarc: Rework reports keys structure * [Rework] Further work to make html content private * [Rework] Html/CSS: Remove css C bindings as they are useless now * [Rework] Html/CSS: Rework Lua bindings * [Rework] Html/Css: Start rework of the html blocks * [Rework] Html: Add images processing logic * [Rework] Html: Add traverse function * [Rework] Html: Another steps to get rid of gnode * [Rework] Html: Convert to variant * [Rework] Html: Deal with the utf_content part * [Rework] Html: Final rework part for the html processing code * [Rework] Html: Fix Lua bindings * [Rework] Html: Forgot to add the internal include * [Rework] Html: Further html urls rework * [Rework] Html: Further rework of the tags content extraction * [Rework] Html: Make parameters as a vector again * [Rework] Html: Move blocks part * [Rework] Html: Move images processing stuff * [Rework] Html: Rework lua bindings * [Rework] Html: Start html text extraction rework * [Rework] Html: Start refactoring of the html tags handling * [Rework] Html: Start removing of GNode stuff * [Rework] Html: Start rework of the html content structure * [Rework] Lua_magic: Try to detect text parts with 8bit characters for non-utf8 encodings * [Rework] Move HTML url functions and rework them * [Rework] Move and adopt entities handling logic * [Rework] Move common and rarely used dmarc code to the library * [Rework] Move compression routines outside of rspamd_util library * [Rework] Move entities/tags handling * [Rework] Phishing: Split from redirectors usage * [Rework] Redesign html blocks propagation logic * [Rework] Remove tag name string * [Rework] Rename phished url to a linked url * [Rework] Reorganize dmarc plugin and remove unsupported reporting code * [Rework] Reputation: Use more flexible types in get/set functions * [Rework] Require proper C++ environment for Rspamd build * [Rework] Rework extended urls output * [Rework] Rework tags parsing machine * [Rework] Slightly improve old regexp API * [Rework] Start conversion of the redis pool code to c++ * [Rework] Try to resolve failed upstreams more agressively * [Rework] Use C++ utf8 library with unit tests to trim whitespaces * [Rework] Use C++ version for unicode normalisation * [Rework] Use C++ version of the lua threads pool * [Rules] Add raw addresses to MULTIPLE_FROM options * [Rules] Another fix to HTTP_TO_HTTPS rule * [Rules] Do not trigger HTML_SHORT_LINK_IMG on external images * [Rules] Extend FORGED_X_MAILER * [Rules] Extend OLD_X_MAILER * [Rules] Fix CTYPE_MIXED_BOGUS for text attachments * [Rules] Fix FPs for CTYPE_MIXED_BOGUS * [Rules] Fix HTTP_TO_HTTPS rule * [Rules] Fix HTTP_TO_HTTPS rule * [Rules] Fix zerofont rule (partially) * [Rules] Micro-optimize X_PHP_EVAL * [Rules] Reduce default weight for R_MISSING_CHARSET Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
4 years ago
Release 2.7 * [Conf] Add R_DKIM_PERMFAIL to the metric * [CritFix] Dkim: Fix simple canonicalisation if multiple signatures are presented * [CritFix] Fix controller paths normalisation * [Feature] Add INVALID_DATE rule * [Feature] Add controller endpoint for training neural * [Feature] Add sanity checks for actions thresholds * [Feature] Add support of '==' and '!=' in Rspamd expressions * [Feature] Composites: Improve composite atoms parser * [Feature] Docker: use Debian slim variant * [Feature] Elastic: Add some missing fields * [Feature] Extract text from img alt attributes * [Feature] Improve charset detection logic * [Feature] Lua_clickhouse: Add optional row callback for large selections * [Feature] Lua_dns_resolver: Add idna_convert_utf8 method * [Feature] Lua_mime: Add ability to do multipattern replacement * [Feature] Lua_trie: Allow to report start of the match * [Feature] Multimap: support adding map values as extra options * [Feature] Neural: Move PCA learning to a subprocess * [Feature] RBL: support matching content/image URLs only * [Feature] RBL: support use of multiple selectors * [Feature] Reputation: Allow to specify ip masks * [Feature] Support SMIME signed messages container * [Feature] Support multiple conditions for symbols * [Feature] Support ping in milter mode * [Feature] Support rspamd_text in selector regexps * [Feature] Use own daemonization routine * [Feature] Vadesecure: Implement settings_outbound feature as recommended by Vade * [Feature] `rspamadm clickhouse` command * [Feature] allow hyperscan for aarch64 * [Fix] Allow to set priorities between post init scripts * [Fix] Allow to use maps for strings that are not zero terminated * [Fix] Apply max_lua_urls limit for emails as well * [Fix] Arc: Fix CV check on signing * [Fix] Arc: Fix signing of the broken ARC chains * [Fix] Clickhouse: escape carriage return * [Fix] Composites: Allow partial match * [Fix] Deduct type of a table methods * [Fix] Do not load errored hyperscan database * [Fix] Do not process links in ignored html tags * [Fix] Fix ClamAV result for cached encrypted file (#3395) * [Fix] Fix canonicalisation when l= tag is presented * [Fix] Fix flag shift * [Fix] Fix handling of skip/skip_process http flags * [Fix] Fix html attachments checks * [Fix] Fix issue with pushing binary formats to Lua strings * [Fix] Fix logging for rspamadm * [Fix] Fix off-by-one with init check * [Fix] Fix parsing of escape characters in quoted pairs * [Fix] Fix pushing ucl strings with \0 inside * [Fix] Fix quoted-printable soft newlines bugged case * [Fix] Fix settings in case actions are set to null (#3415) * [Fix] Fix several issues with auth results producing * [Fix] Fix smtp comments exclusion * [Fix] Fix smtp date syntax definition * [Fix] Fix substring search in case if srchlen == inlen * [Fix] Fix text selectors * [Fix] Honour `systemd` setting when logging to console (#3514) * [Fix] Html: Add entities collisions prevention logic (e.g. for mathml entities) * [Fix] Lua_auth_results: Quote potentially bad values in AR header * [Fix] Multimap: Fix flags usage * [Fix] Multimap: Fix scoring for combined maps * [Fix] Plug GList * leak in redis pool * [Fix] RBL: allow for multiple matches of the same label if types are different * [Fix] Rely on libev checks for file maps * [Fix] Restore simple dkim canonicalisation mode * [Fix] Return MimeCharset as we work with emails... * [Fix] Spamassassin: Fix pcre_only flags * [Fix] Spamassassin: Preserve 'pcre_only' flag when dealing with regexp replacements * [Fix] Try to fix GError leak * [Fix] Try to fix a mess with settings loading by adding priorities * [Fix] Try to move setings initialisation to a later stage * [Fix] Use dup fd in milter handler to avoid races with the proxy * [Fix] Use message pointer to avoid obsolete data to be cached * [Project] Rbl: Migrate to `checks` * [Project] Rbl: Move config code outside of the plugin * [Project] Ressurect empty prefilters as connection filters * [Project] Support connection filters registration from Lua * [Rework] Add final cleanup logic * [Rework] Add preliminary support of hyperscan caching for re maps * [Rework] Add stale cache removal * [Rework] Clickhouse: Improve performance * [Rework] Distinguish between strict config test mode * [Rework] Furhter logging improvements * [Rework] Milter_headers: improve extended_headers_rcpt support * [Rework] Move parsers to a separate lua library * [Rework] Neural: Skip composite symbols * [Rework] Rbl: Rework defaults logic * [Rework] Some tunes to cache saving * [Rework] Track maps origins * [Rework] Use full crypto hash for regexp maps * [Rules] Remove broken rule Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
5 years ago
Release 2.6 * [Conf] Add missing symbols * [Conf] Add missing symbols * [Conf] Fix fat-fingers typo * [Conf] Fix wrong comment in options.inc * [Conf] Neural: Fix the default name for max_trains * [Conf] Register a known symbol * [Conf] Spf: Add R_SPF_PERMFAIL symbol * [CritFix] Arc: Fix ARC validation for chains of signatures * [CritFix] Distinguish socketpairs between different fuzzy workers * [CritFix] Fix IDNA dots parsing * [CritFix] Fix test assertion method * [CritFix] Fix usage of crypto_sign it should be crypto_sign_detached! * [Feature] Add BOUNCE rule * [Feature] Add controller plugins support and selectors plugin * [Feature] Add maps query method * [Feature] Add minimal delay to fuzzy storage * [Feature] Add multiple base32 alphabets for decoding * [Feature] Add preliminary support of BCH addresses * [Feature] Add query_specific endpoint * [Feature] Allow multiple base32 encodings in Lua API * [Feature] Allow to specify nonces manually * [Feature] Controller: Allow to pass query arguments to the lua webui plugins * [Feature] Fuzzy_check: Add gen_hashes command * [Feature] Fuzzy_check: Add weight_threshold option for fuzzy rules * [Feature] Implement address retry on connection failure * [Feature] Improve limits in pdf scanning * [Feature] Initial support of subscribe command in lua_redis * [Feature] Lua_cryptobox: Add secretbox API * [Feature] Lua_text: Add encoding methods * [Feature] Milter_headers: Allow to activate routines via users settings * [Feature] PDF: Add timeouts for expensive operations * [Feature] Preliminary maps addon for controller * [Feature] Split pdf processing object and output object to allow GC * [Feature] Support BLIS blas library * [Feature] Support input vectorisation by recvmmsg call * [Feature] Support multiple base32 alphabets * [Feature] add queueid, uid, messageid and specific symbols to selectors [Minor] use only selectors to fill vars in force_actions message * [Feature] allow variables in force_actions messages * [Feature] extend lua api * [Fix] #3249 * [Fix] Allow to adjust neurons in the hidden layer * [Fix] Another try to fix email names parsing * [Fix] Arc: Allow to reuse authentication results when doing multi-stage signing * [Fix] Arc: Fix bug with arc chains verification where i>1 * [Fix] Arc: Sort headers by their i= value * [Fix] Change neural plugin's loss function * [Fix] Deal with double eqsigns when decoding headers * [Fix] Default ANN names in clickhouse * [Fix] Disable reuseport for TCP sockets as it causes too many troubles * [Fix] Disable text detection heuristics for encrypted parts * [Fix] Distinguish DKIM keys by md5 * [Fix] Distinguish type from flags in register_symbol * [Fix] Dmarc: Unbreak reporting after cf2ae3292ac93da8b6e0624b48a62828a51803c9 * [Fix] Do not flag pre-result of virus scanners as least if action is reject * [Fix] Do not use GC64 workaround on 32bit platforms, omg * [Fix] Exclude damaged urls from html parser * [Fix] Fix FREEMAIL_REPLYTO_NEQ_FROM_DOM * [Fix] Fix FROM_NEQ_ENVFROM * [Fix] Fix FWD_GOOGLE rule (#1815) * [Fix] Fix adding of the empty archive file for gzip * [Fix] Fix aliases in forged recipients and limit number of iterations * [Fix] Fix authentication results insertion * [Fix] Fix calling of methods in selectors * [Fix] Fix clen length for hiredis... * [Fix] Fix endless loop if broken arc chain has been found * [Fix] Fix false - operation * [Fix] Fix get_urls table invocation * [Fix] Fix group based composites * [Fix] Fix headers passing in rspamd_proxy * [Fix] Fix incomplete utf8 sequences handling * [Fix] Fix lua_next invocation * [Fix] Fix lua_parse_symbol_type function logic * [Fix] Fix multiple listen configuration * [Fix] Fix occasional encryption of the cached data * [Fix] Fix parsing boundaries with spaces * [Fix] Fix passing of methods arguments * [Fix] Fix poor man allocator algorithm * [Fix] Fix regexp selector and add flattening * [Fix] Fix rfc base32 encode ordering (skip inverse bits) * [Fix] Fix rfc based base32 decoding * [Fix] Fix sockets leak in the client * [Fix] Fix storing of the original smtp from * [Fix] Fix types check and types usage in lua_cryptobox * [Fix] Fix unused results * [Fix] Fuzzy_check: Disable shingles for short texts (really) * [Fix] Ical: Fix identation grammar * [Fix] Improve part:is_attachment logic * [Fix] Mmap return value must be checked versus MAP_FAILED * [Fix] One more fix to skip images that are not urls * [Fix] Pdf: Support some weird objects with no newline before endobj * [Fix] Rbl: Fix ignore_defaults in conjunction with ignore_whitelists * [Fix] Restore support for `for` and `id` parts in received headers * [Fix] Segmentation fault in contrib/lua-lpeg/lpvm.c on ppc64el * [Fix] Skip spaces at the boundary end * [Fix] Slashing fix: fix captures matching API * [Fix] Spamassassin: Rework metas processing * [Fix] Store reference of upstream list in upstreams objects * [Fix] Understand utf8 in content-disposition parser * [Fix] Unify selectors digest functions * [Fix] Use `abs` value when checking composites * [Fix] Use strict IDNA for utf8 DNS names + add sanity checks for DNS names * [Fix] Use unsigned char and better support of utf8 in ragel parser * [Fix] add missing selector_cache declaration * [Project] Add `L` flag for regexps to save start of the match in Hyperscan * [Project] Add `lower` method to lua_text * [Project] Add a simple matrix Lua library * [Project] Add implicit bitcoincash prefix * [Project] Add linalg ffi library for prototyping * [Project] Add methods to append data to fuzzy requests * [Project] Add routine to call a generic lua function * [Project] Add ssyev method interface * [Project] Add tensors index method * [Project] Add text:sub method * [Project] Allow rspamd_text based selectors * [Project] Allow to specify re_conditions for regular expressions * [Project] Attach extensions to the binary fuzzy commands * [Project] Bitcoin: BTC cash addresses needs some checksum validation * [Project] Cleanup the redis script * [Project] Convert bitcoin rules to the new regexp conditions feature * [Project] Detect memrchr in systems that supports it * [Project] Do not listen sockets in the main process * [Project] Implement 'probabilistic' learn mode for ANN * [Project] Implement BTC polymod in C as it requires 64 bit ops * [Project] Implement bitcoin cash validation in a proper way * [Project] Implement extensions logic for fuzzy storage * [Project] Implement symbols insertion in multiple results mode * [Project] Lua_text: Add method memchr * [Project] Neural: Add PCA loading logic * [Project] Neural: Fix PCA based learning * [Project] Neural: Fix matrix gemm * [Project] Neural: Further PCA fixes * [Project] Neural: Implement PCA in learning * [Project] Neural: Implement PCA learning * [Project] Neural: Implement PCA on ANN forward * [Project] Neural: Implement PCA serialisation * [Project] Neural: Start PCA implementation * [Project] Neural: Use C version of scatter matrix producing * [Project] Preliminary support of lua conditions for regexps * [Project] Preliminary usage of the reuseport * [Project] Process composites separately for each shadow result * [Project] Remove old code * [Project] Rework scan result functions to support shadow results * [Project] Rework some more functions to work with shadow results * [Project] Some more fixes * [Project] Start results chain implementation * [Project] Support fun iterators on rspamd_text objects * [Project] Support multiply, minus and divide operators in expressions * [Project] Tensor: Move scatter matrix calculation to C * [Rework] Allow to specify exat metric result when adding a symbol * [Rework] Change and improve openblas detection and usage * [Rework] Close listen sockets in main after fork * [Rework] Further rework of lua urls extraction API * [Rework] Lua_cryptobox: Allow to store output of the hash function * [Rework] Lua_task: Add more methods to deal with shadow results * [Rework] Modernize logging for expressions * [Rework] Remove empty prefilters feature - we are not prepared... * [Rework] Remove old FindLua module, disable lua fallback when LuaJIT is enabled * [Rework] Rework and refactor forged recipients plugin * [Rework] Rework expressions processing * [Rework] Rework fuzzy commands processing * [Rework] Rework url flags handling API * [Rework] Rework urls extraction * [Rework] Split operations processing and add more debug logs * [Rework] Update zstd to 1.4.5 * [Rework] Use google-ced instead of libicu chardet as the former sucks * [Rework] add alias util:parse_addr for util:parse_mail_address * [Rework] get rid of util:parse_addr duplicating the util:parse_mail_address, replace where used * [Rules] Allow prefix for bitcoin cash addresses * [Rules] More fixes for bitcoin cash addresses decoding * [Rules] Refactor bleach32 addresses handling Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
5 years ago
Release 2.5 * [Conf] Mark Rspamd emailbl as ignore whitelist * [Conf] RBL: Add missing emails = true option * [Feature] Add support for scripts in fuzzy storage * [Feature] Arc: Add whitelisted_signers_map option * [Feature] Implement hosts file processing * [Feature] Neural: Introduce classes bias that allows non-equal classes learning * [Feature] Update libev to 4.33 * [Fix] Another brain damage html standard adoptions * [Fix] Another fix for brain damaged obs-fws state * [Fix] Fix flags that caused force_actions failure * [Fix] Fix logging issue * [Fix] Fix lua symbols scores registration when config does not define scores * [Fix] Fix opaque maps logic * [Fix] Fix parsing of the html tags with no spaces after attributes * [Fix] Fix some corner cases in urls parsing, add limits * [Fix] Fix tlds extraction if custom composition rules are used * [Fix] Fix variables replacement in mempool * [Fix] Improve base64 detection * [Fix] Normalize dynamic scores in ANN correctly * [Fix] Plug memory leak introduced by #3153 * [Fix] Stat_redis_backend: Fix memory leak and simplify learn path * [Fix] Try hard to deal with ghost workers * [Fix] metadata_exporter default formatter * [Rework] Change the way to extract URLs when dealing with alternative parts * [Rework] Fix various url extraction issues * [Rework] Re cache: Load compiled hyperscan in the main process as well * [Rework] Re cache: Load hyperscan early * [Rework] Rework URL structure: adjust tld part * [Rework] Rework URL structure: host field * [Rework] Rework URL structure: more structure optimisations * [Rework] Rework URL structure: user field * [Rework] URL: Another update for urls extraction logic * [Rework] Urls: Improve query urls handling * [Rework] Urls: adopt html related stuff * [Rework] Urls: more rework of the urls sets * [Rework] Urls: process query urls in HTML urls correctly * [Rework] Urls: rework urls hash structure * [Rework] Urls: update lua libraries * [Rework] Use multiple search tries for different url extraction types Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
6 years ago
Release 2.4 * [CritFix] Fix parsing of the content type attributes * [Feature] Clickhouse: Add extra columns support * [Feature] Rbl: Add url_compose_map option for RBL rules * [Fix] 'R' flag is for all headers regexp * [Fix] Allow to reset settings id from Lua (e.g. because of the priority) * [Fix] Avoid collisions in mempool variables by changing fuzzy caching logic * [Fix] Avoid strdup usage for symbols options * [Fix] Do not trust stat(2) it lies * [Fix] Filter all options for symbols to have sane characters * [Fix] Fix all headers iteration * [Fix] Fix allowed_settings for neural * [Fix] Fix listen socket parsing * [Fix] Fix maps expressions evaluation * [Fix] Fix sentinel connections leak by using async connections * [Fix] Fix smtp message on passthrough result * [Fix] Fix tld compositon rules * [Fix] Fuzzy_storage: Do not check for shingles if a direct hash has been found * [Fix] Lua_mime: Do not perform QP encoding for 7bit parts * [Fix] Neural: Distinguish missing symbols from symbols with low scores * [Fix] Support listening on systemd sockets by name * [Project] Add lua_urls_compose library * [Project] Allow to set a custom log function to the logger * [Project] CDB maps: Start making cdb a first class citizen * [Project] Clickhouse: Add extra columns concept * [Project] Fix urls composition rules, add unit tests * [Project] Unify cdb maps * [Rework] Logger infrastructure rework * [Rework] Refactor libraries structure * [Rework] Rework SSL caching * [Rework] Update snowball stemmer to 2.0 and remove all crap aside of UTF8 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
6 years ago
Release 2.3 * [Conf] SPF is no longer a C module * [Conf] Update spamtrap map path example * [CritFix] Fix html entities decoding * [CritFix] Fix re cache when mix of pcre and hyperscan is used * [Feature] Allow milter code to deal with multiple headers * [Feature] Antivirus: Add avast support * [Feature] Dkim_signing: Allow to sign via milter_headers * [Feature] Implement content hashes * [Feature] Lua_text: Add regexp split iterator method * [Feature] Lua_text: Implement flattening of the input tables * [Feature] Send quit command to Redis * [Feature] Speed up is_ascii function * [Feature] Spf: Add external_relay option * [Fix] Avoid double escaping * [Fix] Fix O(N^2) algorithm * [Fix] Fix arc seal validation * [Fix] Fix base tag processing according to stupid HTML renderer behaviour * [Fix] Fix dealing with `\0` in ucl strings and JSON * [Fix] Fix gpg parts misdetection * [Fix] Fix ignored symbols exporting * [Fix] Fix processing of numeric url's * [Fix] Fix processing of the closed tcp connections * [Fix] Fix regexp type check for pcre2 * [Fix] Fix urls encode function * [Fix] Fix urls shifting when doing decode to include separators * [Fix] Fix white on white rule and add is_leaf flag * [Fix] Further fixes in charset detection * [Fix] Ignore diacritics in chartable module for specific languages * [Fix] Limit size of symbols options by max_opts_len option * [Fix] More fixes in html tag content calculations * [Fix] Plug memory leak in fuzzy storage * [Fix] Process high priority settings even if settings/id has been specified * [Fix] Select a different upstream on last retransmit * [Fix] Treat soft hyphen as zero width space * [Fix] Try harder to watch the lifetime of the key_stat * [Fix] Use ipv6-mapped-ipv4 addresses in radix trie * [Project] Add logic to break execution when processing symbols* * [Project] Add methods to set specific content for mime parts from Lua * [Project] Lua_content: support PDF files * [Project] Move dns_tool to using of the rspamd_spf from FFI module * [Project] Preliminary SPF plugin in Lua * [Project] Show debug stat for memory pool * [Project] Some rework about specific data that is now tagged * [Project] Start reworking of the mempool structure * [Rework] Allow to add userdata as symbols options * [Rework] Change mime part specifics handling * [Rework] Move LRU SPF cache from spf plugin * [Rework] Rework HTML tags content attachment * [Rework] Rework options hash structure * [Rework] Start lua_content library * [Rework] Stop using of uthash for http headers * [Rework] Use faster hashing approach for memory pools variables * [Rules] Add PDF related rules Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
6 years ago
Release 2.2 * [Conf] Antivirus: Fix the default config * [Feature] Add verdict library in lua * [Feature] Allow exception when choosing upstream * [Feature] Allow to disable symbols from the metric config * [Feature] Allow to limit maps per specific worker * [Feature] Always validate Rspamd protocol output * [Feature] Antivirus: Add preliminary virustotal support * [Feature] Clickhouse: Rework Clickhouse collection logic * [Feature] Improve base64 usage * [Feature] Shutdown timeout is now associated with task timeout * [Fix] #3129 Multiple classifiers on redis working incorrectly * [Fix] Allow real upstreams configuration * [Fix] Another try to fix slow callbacks and timers * [Fix] Check results of write message as SSL can bork them * [Fix] Clickhouse: Avoid potential races in collection * [Fix] Clickhouse: Fix periodic script * [Fix] Fail DNS upstream on each retransmit attempt * [Fix] Fix consistent hashing when upstreams are marked inactive * [Fix] Fix issues found * [Fix] Fix off-by-one in retries for the proxy * [Fix] Fix termination * [Fix] Fix upstreams exclusion logic * [Fix] Fix utf8 validation for symbols options and empty strings * [Fix] Oops, fix maps reload * [Fix] Rbl: Allow utf8 lookups for IDN domains * [Fix] Sigh, another try to fix brain-damaged openssl * [Project] Add fast utf8 validation library * [Project] Use own utf8 validation instead of glib * [Rework] Another phase of finish actions rework * [Rework] Further cmake system rework * [Rework] Further isolation of the controller's functions * [Rework] Make cmake structure more modular * [Rework] Move cmake modules to a dedicated path * [Rework] Replace controller functions by any scanner worker if needed * [Rework] Rework final scripts logic * [Rework] Rewrite rspamd_str_make_utf_valid function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
6 years ago
Release 2.1 * [Conf] Update neural.conf * [CritFix] Fix dkim verification for multiple headers listed * [Feature] Add support of uudecode * [Feature] Allow to explicitly set events backend * [Feature] Implement configurable limits for SPF lookups * [Feature] Lua_scanners: Use lua magic for inclusion/exclusion logic * [Feature] Multimap: Do not check files in office archives * [Feature] Neural: Add sampling when storing training vectors * [Feature] SPF: Allow to disable AAAA checks in configuration * [Feature] Spf: Add limits configuration support * [Feature] Store etag in cached HTTP maps + better logging * [Feature] Support segwit BTC addresses, fix LTC verification * [Feature] Support uuencoding * [Fix] Add configurable number of threads for OpenBLAS * [Fix] Add workaround for ragel 7 in hyperscan related maps code * [Fix] Another fix for numeric urls parsing * [Fix] Correct EMA time calculations * [Fix] Do not treat archives as text * [Fix] Do not use strdup on data extracted from lua * [Fix] Fix a failure calcuating URL reputation. * [Fix] Fix crash due to constructors init order * [Fix] Fix crash on parts with no cd * [Fix] Fix empty prefilters that require mime structures * [Fix] Fix event loop creation * [Fix] Fix issues sending DMARC reports. * [Fix] Fix misprint * [Fix] Fix saving of the file maps * [Fix] Fix size calculations when converting from utf16 * [Fix] Fix support of disable_monitoring in rbl * [Fix] Fix use-after-free * [Fix] Fix zip files check to relax requirements * [Fix] Important hiredis fixes * [Fix] Lot's of fixes in maps check logic * [Fix] Lua_tcp: Deal with temporary fails on write * [Fix] Lua_tcp: Make write errors fatal and rework error handlers * [Fix] Meta: Filter some more values * [Fix] Neural: Add protection agains infinities * [Fix] Oops, fix math.huge invocation * [Fix] Plug memory leak * [Fix] Sigh, another email to string fix * [Fix] Try to fix another ownership race in ssl connection * [Fix] Uuencode: Fix parsing of corrupted uuencode * [Fix] lua_scanners - razor rename need_check function * [Rework] Require CMake 3.9 to work, remove manual lto crap Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
6 years ago
Release 2.0 * [Conf] Add BROKEN_HEADERS_MAILLIST composite * [Conf] Add path to greylist-whitelist-domains.inc * [Conf] Clarify documentation in the config files * [Conf] Introduce maps.d directories * [Conf] Log settings id by default * [Conf] Make LEAKED_PASSWORD_SCAM a composite rule again * [Conf] Move all surbl/emails rules to rbl * [Conf] Register new Spamhaus codes * [Conf] Remove configs for deleted modules * [Conf] Remove surbl parts, fix hash_format attribute * [Conf] Show autolearn sample * [Conf] Slashing: Change default stats backend to Redis * [Conf] Surbl: Utilise new `check_emails` option * [Conf] Update header * [Conf] Use multi-prefixes RBLs in the default config * [CritFix] Deal with case-sensivity in Content-Disposition parser * [CritFix] Eliminate old endpoint * [CritFix] Fix case sensivity when parsing Content-Type * [CritFix] Fix loading of DKIM public keys * [CritFix] Fix procesing of urls * [CritFix] Fix whitelisting when both spf and dkim are required to be valid * [CritFix] Langdet: Fix language detection where no stop words found * [Feature] Add description to the groups * [Feature] Add limit for number of URLs in Lua * [Feature] Add logging of groups to the log_format * [Feature] Add lua_smtp library * [Feature] Add maps cache and type refinement * [Feature] Add p0f scanner * [Feature] Adopt emails module to use lua_maps * [Feature] Allow options matching in composites * [Feature] Allow selectors in rbl module * [Feature] Allow to output group results * [Feature] Asn: Allow to use bgpdump when NET::MRT is broken * [Feature] Calculate tokens occurrences distribution * [Feature] Clickhouse: Add authenticated user and settings id columns * [Feature] Clickhouse: Store groups data * [Feature] Clickhouse: Utilise LowCardinality feature * [Feature] Implement Redis prefixes registration logic * [Feature] Implement settings id propagation between deps * [Feature] Improve AV results caching * [Feature] Improve autolearning * [Feature] Improve logging locking logic (remove it actually) * [Feature] Improve settings processing * [Feature] Langdet: Limit number of stop words to be checked * [Feature] Libucl: Allow to sort keys in ucl objects * [Feature] Lua_config: Extend get symbols method * [Feature] Lua_maps: Allow static maps for key-value pairs * [Feature] Lua_mimepart: Add function filter_words * [Feature] Lua_selectors: Add `words` selector * [Feature] Lua_selectors: Add sort and uniq transform functions * [Feature] Lua_selectors: Allow table arguments for selectors * [Feature] Lua_tcp: Add preliminary support of SSL connections * [Feature] Lua_trie: More flexible API * [Feature] Lua_util: Add filter_specific_url function * [Feature] Lua_util: table_digest can now recursively traverse tables * [Feature] Maillist: Improve detection * [Feature] Maps: Allow caching for complex maps * [Feature] Monitored: Support random lookups * [Feature] Multimap: Add combined maps prototype * [Feature] Multimap: Add dependend maps via redis keys selectors * [Feature] Multimap: Allow multiple email addresses matches * [Feature] Multimap: Also check detected charset when do filename checks * [Feature] Output number of messages processed to proctitle * [Feature] Perform clean SSL shutdown * [Feature] Performance: Do not use base64 SIMD version for bad inputs * [Feature] RBL: Support bit results in replies * [Feature] RBL: Support type specific prefixes * [Feature] Ratelimit: Consider number of SMTP recipients * [Feature] Rbl: Add ability to check urls * [Feature] Rbl: Add resolve_ip based RBLs * [Feature] Rbl: Make config checks much more strict * [Feature] Rbl: Support per-rule whitelists * [Feature] Rbl: Support process script * [Feature] Rbl: Support replyto addresses * [Feature] SURBL: Allow to check email domains * [Feature] Selectors: Add `list` generator * [Feature] Selectors: Add `specific_urls` extractor * [Feature] Selectors: Add flatten function * [Feature] Selectors: Support filter_map and apply_map functions * [Feature] Store Clickhouse data outside of lua alloc * [Feature] Support caching for encrypted files and macros * [Feature] Support images when extracting urls * [Feature] Support more hyperscan flags * [Feature] Support protocol flags * [Feature] URL: Apply stringprep to hostnames to filter garbage * [Feature] Upstreams: Add lazy resolving logic to all upstreams * [Feature] Upstreams: Set noresolve flag on numeric upstreams * [Feature] Use `scores` in apply section * [Feature] Use maps logic from lua_maps for multimap * [Feature] Use random monitored in rbl module * [Feature] lua_scanners - add Razor support * [Fix] Add another safe-guard in urls processing * [Fix] Add debug to ssl, fixed write hangs * [Fix] Add missing groups to C callback symbols * [Fix] Add more checks for ghosts symbols * [Fix] Allow to enable or add new actions via settings * [Fix] Allow to set 0 size for spf/dkim caches * [Fix] Another bunch of fixes towards protocol mess * [Fix] Another fix to deal with bad URLs * [Fix] Arc: Another bunch of fixes for arc signing * [Fix] Arc: More arc signing fixes * [Fix] Avoid another overflow in fpconv * [Fix] Clickhouse: Fix quoting * [Fix] Clickhouse: Fix retention query quoting * [Fix] Distinguish empty and non-empty prefilters * [Fix] Distinguish remote and local addrs parsing * [Fix] Do not assert if length of sig is bad, just fail verification * [Fix] Do not assert if we have broken mime boundary in the headers * [Fix] Do not call implicit strlen to avoid issues * [Fix] Do not count images urls when checking url regexps for compatibility * [Fix] Do not output rbl suffix in symbol option * [Fix] Do not use config pool to avoid issues with double reload * [Fix] Do not use ephemeral string * [Fix] Do not use lightuserdata for traceback * [Fix] Do not use priority in metric registration * [Fix] Emails: Check email sanity before testing on BL * [Fix] Emails: Fix misprint in key name * [Fix] Escape utf in regexp to dodge ragel/hyperscan issue * [Fix] Extend task_timeout to postfilters stage * [Fix] Fix ARC signing after fixing another bug in it... * [Fix] Fix AV scan logic * [Fix] Fix DMARC_NA behaviour in case of no valid policies * [Fix] Fix LRU hash iteration logic * [Fix] Fix alignment mess * [Fix] Fix configuring symbols without scores * [Fix] Fix disabling of the actions * [Fix] Fix dkim signing exceptions * [Fix] Fix embedded images linking logic * [Fix] Fix events leak * [Fix] Fix eviction corner case * [Fix] Fix fuzzy image score calculation #2962 * [Fix] Fix hang in fuzzy_learn when explicit rotation is set * [Fix] Fix headers propagation logic * [Fix] Fix hearbeats restart issue * [Fix] Fix history reset * [Fix] Fix log parameter * [Fix] Fix lua_ip_equal logic * [Fix] Fix more issues with nested messages + tests * [Fix] Fix normalization of non-alphabet based languages * [Fix] Fix offsets when parsing message/rfc822 in multipart * [Fix] Fix options in rbl symbols * [Fix] Fix out of bound access in lua logger * [Fix] Fix out-of-bound read in qp decode * [Fix] Fix parent CTE propagation * [Fix] Fix parsing of the received headers with empty part * [Fix] Fix pending checks for events * [Fix] Fix printing of NULL pointer with fixed length * [Fix] Fix race condition in watcher handler * [Fix] Fix read-after-end in quoted printable decoding * [Fix] Fix redis sentinel support * [Fix] Fix registry leak in case of DNS errors * [Fix] Fix reload logic * [Fix] Fix sending of large entries via HTTPS * [Fix] Fix settings reload * [Fix] Fix some more corner cases for fpconv * [Fix] Fix trie code when there are regexps and Hyperscan is absent * [Fix] Further fixes to printing of the FP numbers * [Fix] Fuzzy_check: Fix timeouts * [Fix] Grrr, fix empty ip case * [Fix] Html: Fix processing of fjlig entity * [Fix] Lang_det: Try better to distinguish Chinese and Japanese * [Fix] Lua_mime: Fix reversed extensions map * [Fix] Lua_task: Fix message-less API * [Fix] Lua_tcp: Report connection failures * [Fix] Lua_tcp: Various fixes and debugging improvements * [Fix] Metadata_exporter: This plugin is idempotent not a postfilter * [Fix] More fixes to extract_specific_urls * [Fix] More stages fixes * [Fix] Neural: Another bunch of fixes * [Fix] Neural: use version in ANN key profile * [Fix] Postpone lua state destruction to allow lua dtors to be used * [Fix] Prefer surbl/emails rule on rbl to preserve compatibility * [Fix] RBL: Fix behaviour of emails_domainonly * [Fix] Ratelimit: Fix dynamic score * [Fix] Rbl: Fix emailbl functions * [Fix] Really fix hyperscan workaround * [Fix] Set sanity limits for pcre2 * [Fix] Settings: Fix settings check flags * [Fix] Sort keys when getting data from Lua when filling rules * [Fix] Statistics: Do not query Redis tokens when there are no learns * [Fix] Stop IO event on write finished in http connection * [Fix] Use heuristically detected text parts data * [Fix] Various fixes to QP encoding algorithm * [Fix] Various fixes to SSL state machine handler * [Fix] Various fixes to asn module * [Fix] Workaround for empty charset in rfc2231 encoding * [Project] Switch from torch to KANN * [Project] Add heartbeat events * [Project] Add preliminary support of the Kaspersky Scan Engine * [Project] Add preliminary version of maps expressions * [Project] Add preprocessed settings to the config structure * [Project] Add simple forward propagation function * [Project] Add small helpers for migration simplifications * [Project] Allow to replace body in milter * [Project] Bundle libev * [Project] First refactoring step libevent->libev * [Project] Implement syntax highlighting for Lua * [Project] Lua_magic: Adopt lua_magic stuff in mime_types * [Project] Remove libfann, gd and other unsupported stuff * [Project] Remove torch * [Project] Rework upstreams * [Rework] Allow execution of async events when hs compiles regexps * [Rework] Bayes expiry: eliminate `default` expiration mode * [Rework] Dkim: Remove signing code * [Rework] Dkim_signing: Move sign condition to dkim_signing * [Rework] Do not lowercase all data send to ClickHouse * [Rework] Drop url tags * [Rework] Eliminate lua_squeeze as it has shown no improvements * [Rework] Eliminate virtual scan time as it is useless * [Rework] Lua core: Use lightuserdata to index classes * [Rework] Lua_util: Another rework for extract_specific_urls * [Rework] Migrate from ip_score to reputation * [Rework] Move mime modification functions to lua_mime library * [Rework] Rbl: Major whitelisting logic rework * [Rework] Remove deprecated plugins * [Rework] Remove log helper worker * [Rework] Remove rspamd.classifiers.lua * [Rework] Rename filter.h to a more sane name * [Rework] Reorganise selectors implementation * [Rework] Replace linenoise with replxx * [Rework] Reputation: Remove ipnet from the ip reputation * [Rework] Reputation: Slashing - change name of symbols * [Rework] Rework children operations * [Rework] Rework config reload * [Rework] Rework expression API * [Rework] Rework image urls processing * [Rework] Rework initialisation to reduce static leaks count * [Rework] Rework request headers processing * [Rework] Slashing: Change versioning schema - move to 2.0 * [Rework] Slashing: Turn off postfilters when passthrough result is set * [Rework] Start moving to replxx * [Rework] Stop support of signed HTTP maps to simplify code * [Rework] Store ASN as UInt32 in ClickHouse * [Rework] Url_redirector: Rewrite plugin * [Rework] Use a dedicated library for autolearn * [Rework] Use libsodium instead of hand crafted crypto implementations * [Rework] Use opaque structure to store a table of mime headers * [Rules] Add dedicated bitcoin addresses filter rule * [Rules] Add more detection to LEAKED_PASSWORD_SCAM * [Rules] Catch LTC addresses * [Rules] Reduce weight of RSPAMD_EMAILBL * [Rules] Rework LEAKED_PASSWORD_SCAM rule one more time Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
6 years ago
Release 1.9.3 * [Conf] Add IP_SCORE_FREEMAIL composite rule * [Feature] Add cryptobox method to generate dkim keypairs * [Feature] Add fast hashes to lua cryptobox hash * [Feature] Add least passthrough results * [Feature] Allow oversign if exists mode * [Feature] Clickhouse: Modernise table initial schema * [Feature] Implement IUF interface for specific fast hashes * [Feature] Lua_util: Allow to obfuscate different fields * [Feature] Tune memory management in Rspamd and Lua * [Fix] Avoid buffer overflow when printing long lua strings * [Fix] Change the default oversigning headers to a more sane list * [Fix] Clickhouse: Do not store digest as it is not needed now * [Fix] Clickhouse: Fix lots of storage issues * [Fix] Clickhouse: Support custom actions * [Fix] Deny URLs where hostname is bogus * [Fix] Do not blacklist mail by SPF/DMARC for local/authed users * [Fix] Fix DoS caused by bug in glib * [Fix] Fix UCL parsing of the multiline strings * [Fix] Fix buffer overflow when printing small floats * [Fix] Fix init code for servers keypairs cache * [Fix] Fix issue with urls with no tld (e.g. IP) * [Fix] Fix memory in arc signing logic * [Fix] Fix memory leak in language detector during reloads * [Fix] Fix mixed case content type processing * [Fix] Fix processing of the ip urls in file * [Fix] Fix use after free * [Fix] HTML: Fix `size` attribute processing * [Fix] Hum, it seems that 99ff1c8 was not correct * [Fix] Lua_task: Fix task:get_from method * [Fix] Preserve fd when mapping file to scan * [Fix] Re-use milter_headers settings when doing arc signing * [Fix] Set dmarc force action as least action * [Fix] Switch to GMT * [Fix] allow PKCS7 signatures to be text/plain, too * [Project] Add initial version of the vault management tool * [Project] Add vault support for DKIM and ARC signing * [Project] Implement keys rotation in the vault * [Project] Improve dkim keys generation for vault * [Project] Improve keys creation in rspamadm vault * [Rework] Move lua_worker to a dedicated unit * [WebUI] Add URL fragments (#) support * [WebUI] Fix AJAX request URL Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
7 years ago
Release 1.9.1 * [Conf] Add vendor groups for symbols * [Feature] Add `rspamadm template` command * [Feature] Allow to add messages from settings * [Feature] Allow unconnected DNS servers operations * [Feature] Check limits after being set, migrate to uint64 * [Feature] Greylist: Allow to disable greylisting depending on symbols * [Feature] Improve lua binary strings output * [Feature] Mime_types: Implement user configurable extension filters * [Feature] Mime_types: When no extension defined, detect it by content * [Feature] Preprocess config files using jinja templates * [Feature] Replies: Filter replies sender to limit whitelisting to direct messages * [Feature] Treat all tags with HREF as a potential hyperlinks * [Feature] Validate BTC addresses in LEAKED_PASSWORD_SCAM * [Fix] Add crash safety for HTTP async routines * [Fix] Another fix for Redis sentinel * [Fix] Clickhouse: Fix table schema upload * [Fix] Core: Fix squeezed dependencies handling for virtual symbols * [Fix] Finally fix default parameters parsing in actions section * [Fix] Fix ES sending logic (restore from coroutines mess) * [Fix] Fix finishing script for clickhouse collection * [Fix] Fix priority for regexp symbols registriation * [Fix] Fix various issues found by PVS Studio * [Fix] Initialize lua debugging earlier * [Fix] Neural: Fix training * [Fix] Rework cached Redis logic to avoid sentinels breaking * [Fix] SURBL: Fix regression in surbl module * [Fix] Fix double signing in the milter * [Project] Add support of HTTP proxy in requests * [Rework] Change lua global variables registration * [Rework] Rework HTML content urls extraction * [Rework] Start rework of aliasing in Rspamd * [WebUI] Combine Scan and Learning into one tab * [WebUI] Fix symbol score input type * [WebUI] Show grayed out pie * [WebUI] Update Throughput summary values dynamically Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
7 years ago
Release 1.9.0 * [Conf] Add missing includes * [Conf] Move to options * [Conf] Rbl: DWL is actually special whitelist * [Conf] Relax some uribl rules * [Conf] Remove abuse.ch * [CritFix] Html: Entities are not valid within tag params values * [Feature] Add `rspamadm mime sign` tool * [Feature] Add configgraph utility * [Feature] Add dedicated ZW spaces detection for URLs * [Feature] Add flag to url object when visible part is url_like * [Feature] Add method task:lookup_words * [Feature] Add pyzor support (by crosenberg) * [Feature] Allow to add upstream watchers to Lua API * [Feature] Allow to set rewrite subject pattern from settings * [Feature] Better escaping of unicode * [Feature] Clickhouse: Allow to store subject in Clickhouse * [Feature] Core: Add QP encoding utility * [Feature] Core: Add libmagic detection for all parts * [Feature] Core: Add support for gzip archives * [Feature] Core: Allow to construct scan tasks from raw data * [Feature] Core: Detect charset in archived files * [Feature] Core: Ignore and mark invisible spaces * [Feature] Core: Normalise zero-width spaces in urls * [Feature] Core: Process data urls for images * [Feature] Core: Relax quoted-printable encoding * [Feature] Core: Support RFC2231 encoding in headers * [Feature] Core: Support telephone URLs * [Feature] Core: allow to emit soft reject on task timeout * [Feature] DCC: Add bulkness and reputation checks to dcc * [Feature] Elastic: Modernize plugin * [Feature] Export visible part of url to lua * [Feature] Fuzzy_storage: add preliminary support of rate limits * [Feature] HTML: Specially treat data urls in HTML * [Feature] Implement event watchers for upstreams * [Feature] Implement includes tracing in Lua * [Feature] Improve dkim part in configwizard * [Feature] Lua_scanners: Add VadeSecure engine support * [Feature] Lua_task: Add flexible method to get specific urls * [Feature] Mime_types: Add MIME_BAD_UNICODE rule * [Feature] Mime_types: Use detected content type as well * [Feature] Plugins: Add preliminary version of the external services plugin * [Feature] Query sentinel on master errors * [Feature] Regexp: Allow local lua functions in Rspamd regexp module * [Feature] Rspamadm: Allow to append footers to plain messages * [Feature] Rspamadm: Allow to rewrite headers in messages * [Feature] Selectors: Add `ipmask` processor * [Feature] Settings: Allow hostname match * [Feature] Settings: Allow local when selecting settings * [Feature] Settings: Allow multiple selectors * [Feature] Settings: Allow to inverse conditions * [Feature] Support User-Agent in HTTP requests * [Feature] Support ed25519 dkim keys generation * [Feature] Try to filter bad unicode types during normalisation * [Feature] external_services - oletools (olefy) support * [Feature] lua_scanners - icap protocol support * [Feature] lua_scanners - spamassassin spam scanner * [Fix] Add filter for absurdic URLs * [Fix] Add some more cases for Received header * [Fix] Allow to disable/enable composite symbols * [Fix] Arc: Use a separated list of headers for arc signing * [Fix] Archive: Final fixes for 7z archives * [Fix] Clickhouse: Fix database usage * [Fix] Controller: Make save stats timer persistent * [Fix] Core: Detect encrypted rarv5 archives * [Fix] Core: Don't detect language twice * [Fix] Core: Fix address rotation bug * [Fix] Core: Fix content calculations for message parts * [Fix] Core: Fix emails comments parsing and other issues * [Fix] Core: Fix etags support * [Fix] Core: Fix headers folding on the last token * [Fix] Core: Fix iso-8859-16 encoding * [Fix] Core: Fix log_urls flag (and encrypted logging) * [Fix] Core: Fix part length when dealing with boundaries * [Fix] Core: Fix parts distance calculations * [Fix] Core: Fix processing of NDNs of certain type * [Fix] Core: Implement logic to find some bad characters in URLs * [Fix] Core: treat nodes with ttl properly in lru cache * [Fix] Fix Content-Type parsing * [Fix] Fix HTTP headers signing case * [Fix] Fix control interface * [Fix] Fix deletion of the duplicate headers * [Fix] Fix emails filtering in emails module * [Fix] Fix greylisting log message and logic * [Fix] Fix issues with storing of the accepted addr in rspamd control * [Fix] Fix maps object update race condition * [Fix] Fix memor leaks and whitespace processing * [Fix] Fix processing of null bytes in headers * [Fix] Fix rcpt_mime and from_mime in user settings * [Fix] Fix rfc2047 decoding for CD headers * [Fix] Fix rfc2231 for Content-Disposition header * [Fix] Fix setting of the subject pattern in config * [Fix] Greylist: fix records checking * [Fix] HTML: Another HTML comments exception fix * [Fix] HTML: Another entities decoding logic fix * [Fix] HTML: Fix HTML comments with many dashes * [Fix] HTML: Fix entities in HTML attributes * [Fix] HTML: Fix some more SGML tags issues * [Fix] Ignore whitespaces at the end of value in DKIM records * [Fix] MID module: Fix DKIM domain matching * [Fix] Milter_headers: Fix remove_upstream_spam_flag and modernise config * [Fix] Mime_parser: Fix issue with parsing of the trailing garbadge * [Fix] Mime_parser: Fix parsing of mime parts without closing boundary * [Fix] Multimap: Fix operating with userdata * [Fix] Process orphaned `symbols` section * [Fix] Rdns: Fix multiple replies in fake replies * [Fix] Rework groups scores definitions * [Fix] Set proper element when reading data from Sentinel * [Fix] Set rspamd user to initialise supplementary groups on reload * [Fix] Settings: Fix selectors usage * [Fix] Sort data received from Sentinel to avoid constant replacing * [Fix] groups.conf - filename typo * [Fix] lua_scanner - oletools typos, logging * [Fix] lua_scanners - actions and symbol_fail * [Fix] lua_scanners - fix luacheck * [Fix] lua_scanners - kaspersky - response with fname * [Fix] lua_scanners - savapi redis prefix * [Fix] tests - antivirus - fprot symbols * [Project] Add concept of flexible actions * [Project] Add heuristical from parser to received parser * [Project] Add new flags to clickhouse, redis and elastic exporters * [Project] Attach new received parser * [Project] Fallback to callbacks from coroutines * [Project] Implement keep-alive support in lua_http * [Project] Lua_udp: Implement fully functional client * [Project] Plug keepalive knobs into http connection handling * [Project] Rspamadm: Add `modify` tool * [Rework] Convert rspamd-server to a shared library * [Rework] Dcc: Rework DCC plugin * [Rework] Enable explicit coroutines symbols * [Rework] Rework telephone urls parsing logic * [Rework] Rewrite RBL module * [Rework] Settings: Rework settings check * [Rework] Slashing: Distinguish lualibdir, pluginsdir and sharedir * [Rework] Unify task_timeout * [Rework] Use VEX instructions in assembly, relocate * [WebUI] Notify user if uploaded data was not learned * [WebUI] Remove redundant condition Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
7 years ago
Release 1.8.3 * [CritFix] Make flags mutually exclusive for mime parts * [CritFix] Strictly deny unencoded bad utf8 sequences in headers * [Feature] Add Kaspersky antivirus support * [Feature] Add method to get dkim results * [Feature] Add more words regexp classes * [Feature] Allow to choose words format in `rspamadm mime` * [Feature] Allow to get all types of words from Lua * [Feature] Allow to get task flags in C expressions * [Feature] Allow to require encryption when accepting connections * [Feature] Ignore bogus whitespaces in the words * [Feature] Implement more strict configuration tests * [Feature] Improve SPF results in Authentication-Results * [Feature] Support ClickHouse database * [Fix] Add failsafety for utf8 regexps * [Fix] Do not trigger BROKEN_CONTENT_TYPE on innocent text parts * [Fix] Emit error if connection has been terminated with no stop pattern * [Fix] Fix boundaries checks in embedded messages * [Fix] Fix double free * [Fix] Perform policy downgrade on sample out, add tests * [Fix] Properly escape utf8 regexps in hyperscan mode * [Fix] Selectors - attachments args condition * [Fix] Some fixes for raw parts * [Fix] Treat learning errors as non-fatal * [Fix] Use tld when looking for DKIM domains * [Project] Words unicode structure rework * [Project] Add preliminary Redis Sentinel support * [Project] Improve Authentication-Results header * [Project] Rework DKIM checks results * [Project] Use more generalised API to produce meta words Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
7 years ago
Release 1.8.2 * [Conf] Add DWL support in the default configuration * [Conf] Disable rspamd_update by default (again) * [Conf] Fix configuration sample for ratelimit * [CritFix] Disable broken url tags by default * [CritFix] Fix \0 processing when doing RSA sign * [CritFix] Fix adding symbols to their primary groups * [Feature] Add `rspamadm cookie` utility * [Feature] Add specialised functions for generating encrypted cookies * [Feature] Add support of cookies in replies module * [Feature] Add support of words regexps * [Feature] Allow to add 3rd party clang plugins * [Feature] Allow to create lua regexps from glob or plain patterns * [Feature] Allow to set custom limits for upstream lists * [Feature] Detect orphaned parts and attach them to message * [Feature] Filter tokens in bayes * [Feature] Fold b= value when doing arc sealing * [Feature] Ignore cookies in the future and too old in the past * [Feature] Skip stop words in statistics * [Feature] Store stop words and allow to query them * [Feature] Support query arguments in controller's custom commands * [Feature] Tune upstream limits in Rspamd proxy * [Feature] Use different callback symbols for different uribls * [Feature] Write DKIM selector in dkim allow/reject symbols * [Fix] Add obs_fws state support to eoh state machine * [Fix] Add sanity check when applying mime boundaries heuristic * [Fix] Antivirus - virus names with 0 were recognized as tables * [Fix] Disable headernames in bayes temporarily * [Fix] Do not allow syntax errors in include files... * [Fix] Do not allow to merge an object with an array (or vice versa) * [Fix] Don't perform forged recipients check for missing recipients * [Fix] Fix DKIM based RBLs * [Fix] Fix actrie implementation (sync from upstream), fixed OOB read * [Fix] Fix explicit methods call in selectors * [Fix] Fix extraction of additional parts * [Fix] Fix finalization for internal plugins * [Fix] Fix override_defaults function * [Fix] Fix squeezed symbols when using settings * [Fix] Fix urls insertion in Clickhouse module * [Fix] Furhter fixes to ratelimits logic * [Fix] Ignore signatures when looking for boundaries * [Fix] Properly set learned count * [Fix] Really fix ratelimits configuration and work * [Fix] Remove ambigious format flag from printf * [Fix] Restore URLs exporting in ClickHouse plugin * [Fix] Rework bayes calculations... * [Fix] Switch from chi-square to naive for large Fisher value * [Fix] Treat normal password as enable password if there is no enable password * [Fix] Use proper syntax for making DNS requests * [Fix] Various fixes in embedded plugins * [Project] Change fuzzy check selection logic to lua_fuzzy library * [Project] Rework async events and symbols * [Project] Move all metatokens in Bayes to lua_stat from C * [WebUI] Add history rows per page control Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
7 years ago
Release 1.8.1 * [CritFix] Fix options insertion * [CritFix] Fix words decay one more time (affects long messages) * [CritFix] Increase default words_decay * [CritFix] Plug memory leak in redis pool * [Feature] Add `check_violation` feature to DKIM/ARC signing * [Feature] Add only unique elements to Clickhouse url arrays * [Feature] Allow `g+:` and `g-:` composite atoms * [Feature] Allow dkim domains check in surbl * [Feature] Allow maps with HTTP auth * [Feature] Allow to disable actions by users settings * [Feature] Extend whitelisting options * [Feature] Store url object in images * [Feature] Use verdict instead of the plain action in plugins * [Fix] Allow to call fstring append with NULL string * [Fix] DCC - luacheck * [Fix] Do not load torch on each rspamadm invocation * [Fix] Fix boundaries detection and rework stop words algorithm * [Fix] Fix dependencies for DNS_SIGNED symbol * [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit * [Fix] Fix groups mess * [Fix] Fix groups mess * [Fix] Fix parsing address with comments * [Fix] Fix resolving in DMARC reports * [Fix] Fix various issues with parsing of the received headers * [Fix] Fix watchers issue in lua_tcp when doing no resolving * [Fix] Plug memory leak in language detector (affects reloads) * [Fix] Remove one letter stop words * [Fix] Slashing: backport chunk logic from libucl * [Fix] Stop libevent from using cached time in rspamadm * [Fix] Try to fix watchers chaining * [Fix] Various fixes in redis sync interface * [Fix] ip_score - respect check_authed and check_local settings from config * [Project] Rework passthrough actions * [Project] Clustering module * [Rework] Always create result for a task * [Rework] Completely rewrite DMARC checks logic * [Rework] Rework and fix whitelist plugin * [WebUI] Add symbols sorting buttons * [WebUI] Change symbols order without updating history * [WebUI] Colorize symbols * [WebUI] Do not display password form when secure_ip is set * [WebUI] Fix symbol description tooltips display * [WebUI] History: add sorting by symbol score value Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
7 years ago
Release 1.8.0 * [Feature] Add arguments schemas to processors and extractors * [Feature] Add functional selectors library * [Feature] Add generic selector to reputation module * [Feature] Add more ratelimits: by digest, by attachments data, by filenames * [Feature] Add preliminary stop words detection support * [Feature] Add pure Lua debugm function * [Feature] Add schema validation for Redis settings * [Feature] Add selectors combine function * [Feature] Add some recursion protection to lua logger * [Feature] Add support for Lua API tracing * [Feature] Allow to apply schema to arguments * [Feature] Allow to get dkim signing data directly from HTTP headers * [Feature] Allow to reuse existing authentication results * [Feature] Cache selectors results in re runtime * [Feature] Implement new text tokenizer based on libicu * [Feature] Integrate selectors framework to multimap * [Feature] Relax FORGED_RECIPIENTS * [Feature] Support (almost) all html entities * [Feature] Support adding and deletion of recipients in the milter block * [Feature] Support gathering HTTP body from fragments in lua_http * [Feature] Support multi flag in regexp and glob maps * [Feature] Support selectors in ratelimit module * [Feature] Support selectors in settings * [Feature] Use khash in HTML parser * [Feature] Use pure Lua debugm function * [Fix] Add fail-safety for destroying sessions * [Fix] Allow to add result-less fake DNS records * [Fix] Another try to fix race conditions on config unload * [Fix] Call Lua callback on DNS timeouts * [Fix] Deprecate task:inc_dns_req as it is redundant * [Fix] Do not allow events deletions on cleanup * [Fix] Do not try to process skipped messages * [Fix] Fix HTTP requests with no body * [Fix] Fix another cleanup race condition * [Fix] Fix bug in processing of pcre regexps * [Fix] Fix byte array allocation in the pool * [Fix] Fix crashes on task cleanup * [Fix] Fix dynamic buckets in ratelimits * [Fix] Fix endless loop when waiting for Rspamd to stop * [Fix] Fix lua_util.str_split in case of delimiters set * [Fix] Fix more issues with watching of async events * [Fix] Fix stop words detection and loading logic * [Fix] Fix various corner cases for language detection * [Fix] Fix watchers in lua_tcp * [Fix] Fix words decay algorithm * [Fix] Implement watchers replacement to handle nested calls * [Fix] Save faked code into fake dns record * [Fix] Show the proper frame when using lua_util.debugm * [Fix] Use fake dns records in tests * [Fix] Use unicode replacements for HTML entities * [Fix] fixed "cannot find dependency on symbol 1" issue when using replaced symbols in spamassassin rules * [Fix] partition_id is not available in old versions of CH * [Project] Add implicit conversion logic to selectors * [Project] Add initial support for selectors in regexps * [Project] Add method concept * [Project] Further changes in unicode operations * [Project] Implement Clickhouse migrations * [Project] Implement implicit conversions to userdata * [Project] Implement insert method * [Project] Implement selectors registration for regular expressions * [Project] Implement selectors support in re_cache * [Project] Improve language detector: cleanup unused files, categorize * [Project] Migrate CH data to a fat table * [Project] Rework selectors logic * [Project] Start Clickhouse utilities library * [Project] Start unicode rework * [Project] coroutine threaded model for API calls: thread pool * [Rework] Move phishtank to a DNS based service * [Rework] Rework Clickhouse plugin to use the new API * [Rework] Rework language detector * [Rework] Rework utf content processing in text parts * [WebUI] Add progress bar for AJAX requests * [WebUI] Avoid errors table reinitialization * [WebUI] Avoid history table reinitialization * [WebUI] Avoid throughput summary table reinitialization * [WebUI] Destroy summary table on disconnect * [WebUI] Fix "auth" request URL * [WebUI] Fix disabling and hiding controls on page reload * [WebUI] Fix maps loading from neighbours * [WebUI] Fix symbols sorting by score * [WebUI] Fix tables destroying * [WebUI] Fix throughput data consolidation * [WebUI] Fix upload buttons disabling * [WebUI] Notify user on module loading failure * [WebUI] Update FooTable 3.1.4 -> 3.1.6 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
7 years ago
Release 1.7.9 * [CritFix] Fix caseless comparison of equal length strings * [Feature] Add HTTP basic auth support to elastic and clickhouse plugins * [Feature] Add SPF selector to reputation * [Feature] Add support of the fallback backends for HTTP maps * [Feature] Allow to print full mime structure when extracting mime data * [Feature] Allow to split symbols in reputation plugin * [Feature] Check attachments only on AV scanners in attachments_only mode * [Feature] Disable all SSL checks if ssl_no_verify flag is set * [Feature] Implement parsing of scoped IPv6 addresses * [Feature] Improve `rspamc counters` output * [Fix] Add sanity checks when expanding SPF macros * [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack) * [Fix] Avoid one extra byte writing * [Fix] Deal with direct hash table * [Fix] Detect empty text part as text, not HTML * [Fix] Do not reduce map watch timeout for mixed http/file maps * [Fix] Fix HTML part detection heuristic * [Fix] Fix double free in redirectors cleanup * [Fix] Fix legacy history handling in the controller * [Fix] Fix messages insertion * [Fix] Fix sending string method * [Fix] Fix statconver command line arguments * [Fix] Fixed argument checking for being null * [Fix] Fixed issues reported by luacheck * [Fix] Freeze updates queue when do actual storage update * [Fix] HTTP map hash is per-backend and not per-map * [Fix] Plug memory leak in fuzzy updates * [Fix] Prefer 'MTA-Name' when producing authentication results * [Fix] Replace bad unicode sequences instead of stopping on them * [Fix] Set classifier version on learning * [Project] Reworked ratelimits * [Project] Apply topological sorting for symbols in Rspamd * [Project] Remove global contexts from C modules * [Project] Move performance critical hash tables to khash * [WebUI] Avoid unused indexes * [WebUI] Do not execute `on_success` callback * [WebUI] Fix history reset for "All SERVERS" (#2346) * [WebUI] Fix query URL for selected server * [WebUI] Fix symbols display in legacy history, * [WebUI] Hide symbols order selector for legacy history * [WebUI] Refactor query functions into one * [WebUI] Remove previously-attached event handlers * [WebUI] Save symbols to the selected server * [WebUI] Unify arguments of query functions * [WebUI] Use common query functions to get graph data * [WebUI] Use common query functions to save symbols Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.8 * [Feature] Add more extended statistics about fuzzy updates * [Feature] Add more non-conformant Received headers support * [Feature] Add preliminary function to get fuzzy hashes from text in Lua * [Feature] Allow to configure AV module rejection message * [Feature] Implement fuzzy hashes extraction in mime tool * [Feature] Improve WHITE_ON_WHITE rule * [Feature] Improve integer -> string conversion * [Feature] Reuse maps in multimap module more aggressively * [Fix] Avoid race condition in skip map as pool lifetime is not enough * [Fix] Eliminate all specific C plugins pools * [Fix] Fix DKIM check rule if DNS is unavailable * [Fix] Fix build where ucontext is defined in ucontext.h * [Fix] Fix crash in base url handling * [Fix] Fix descriptors leak in sqlite3 locking code * [Fix] Fix messages quarantine * [Fix] Fix padded numbers printing * [Fix] Fix race condition on maps reinit * [Fix] Fix regexp functions when no data is passed * [Fix] Fix specific urls extraction * [Fix] Fix styles propagation * [Fix] Improve resetting of the limit buckets * [Fix] Initialize sqlite3 properly * [Fix] Work with broken resolvers in resolv.conf * [Project] Implement HTTP maps caching * [Project] Refresh fuzzy hashes when matched * [Project] Add logic to deduplicate fuzzy updates queue * [WebUI] Add missed declarations * [WebUI] Avoid using "undefined" property * [WebUI] Do not accept passwords containing control characters * [WebUI] Do not redeclare variables * [WebUI] Enable strict mode, * [WebUI] Fix variable assignment * [WebUI] Initialize variables at declaration * [WebUI] Remove duplicated path from RequireJS config * [WebUI] Remove unused block * [WebUI] Remove unused variable * [WebUI] Remove unused variables * [WebUI] Use self-explanatory notation * [WebUI] Use type-safe equality operators Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.6 * [CritFix] Fix multiple neural networks support * [Feature] Add decryption function to keypair command * [Feature] Add gzip compression for HTTP requests in elastic module * [Feature] Add gzip methods to lua util * [Feature] Add maps based on Top Level Domains * [Feature] Add pubkey checks for dkim_signing * [Feature] Add support of fake DNS records * [Feature] Add tool to encrypt files * [Feature] Allow to add symbols using settings directly * [Feature] Allow to match private and public keys for DKIM signatures * [Feature] Allow to set task flags via settings * [Feature] Allow to specify fake DNS address from the config * [Feature] Implement signatures verification using rspamadm keypair * [Feature] Implement signing using `rspamadm keypair` * [Feature] Improve error reporting for DKIM key access issues * [Feature] Provide $HOSTNAME variable in UCL * [Feature] Rework levenshtein distance computation * [Feature] Split message parsing and processing * [Feature] Support ED25519 DKIM signatures * [Feature] Support encrypted configs in UCL * [Feature] Suppress duplicate warning on very large radix tries * [Feature] Use OSB to combine header names * [Fix] Cleanup maps data on shutdown * [Fix] Fix '~' behaviour in composites * [Fix] Fix HTTP maps updates * [Fix] Fix NIST signatures * [Fix] Fix RFC822 comments when processing a mime address * [Fix] Fix double free * [Fix] Fix dynamic settings application * [Fix] Fix for CommuniGate Pro maillist * [Fix] Fix keypair creation method to actually create keypair... * [Fix] Fix matching patterns with no paths * [Fix] Fix memory leak in parsing comments * [Fix] Fix parsing of urls with numeric password * [Fix] Fix plugins intialisation in configwizard * [Fix] Fix potential crash on reload * [Fix] Fix potential race condition for a finished HTTP connections * [Fix] Fix race-condition leak on processes reload * [Fix] Fix signing in openssl mode * [Fix] Free language detector structures * [Fix] Relax alignment requirements * [Fix] Send DMARC reports compressed * [Fix] Try to fix leak in dmarc module * [Fix] Try to plug memory leak in metric exporter * [Project] Convert rspamadm subcommands to Lua * [WebUI] Display smtp sender/recipient in history * [WebUI] Fix elements disabling in "Symbols" tab * [WebUI] Limit recipients list in history column to 3 * [WebUI] Match envelope and mime addresses following in arbitrary order * [WebUI] Update column header * [WebUI] Wrap addresses in history Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.6 * [CritFix] Fix multiple neural networks support * [Feature] Add decryption function to keypair command * [Feature] Add gzip compression for HTTP requests in elastic module * [Feature] Add gzip methods to lua util * [Feature] Add maps based on Top Level Domains * [Feature] Add pubkey checks for dkim_signing * [Feature] Add support of fake DNS records * [Feature] Add tool to encrypt files * [Feature] Allow to add symbols using settings directly * [Feature] Allow to match private and public keys for DKIM signatures * [Feature] Allow to set task flags via settings * [Feature] Allow to specify fake DNS address from the config * [Feature] Implement signatures verification using rspamadm keypair * [Feature] Implement signing using `rspamadm keypair` * [Feature] Improve error reporting for DKIM key access issues * [Feature] Provide $HOSTNAME variable in UCL * [Feature] Rework levenshtein distance computation * [Feature] Split message parsing and processing * [Feature] Support ED25519 DKIM signatures * [Feature] Support encrypted configs in UCL * [Feature] Suppress duplicate warning on very large radix tries * [Feature] Use OSB to combine header names * [Fix] Cleanup maps data on shutdown * [Fix] Fix '~' behaviour in composites * [Fix] Fix HTTP maps updates * [Fix] Fix NIST signatures * [Fix] Fix RFC822 comments when processing a mime address * [Fix] Fix double free * [Fix] Fix dynamic settings application * [Fix] Fix for CommuniGate Pro maillist * [Fix] Fix keypair creation method to actually create keypair... * [Fix] Fix matching patterns with no paths * [Fix] Fix memory leak in parsing comments * [Fix] Fix parsing of urls with numeric password * [Fix] Fix plugins intialisation in configwizard * [Fix] Fix potential crash on reload * [Fix] Fix potential race condition for a finished HTTP connections * [Fix] Fix race-condition leak on processes reload * [Fix] Fix signing in openssl mode * [Fix] Free language detector structures * [Fix] Relax alignment requirements * [Fix] Send DMARC reports compressed * [Fix] Try to fix leak in dmarc module * [Fix] Try to plug memory leak in metric exporter * [Project] Convert rspamadm subcommands to Lua * [WebUI] Display smtp sender/recipient in history * [WebUI] Fix elements disabling in "Symbols" tab * [WebUI] Limit recipients list in history column to 3 * [WebUI] Match envelope and mime addresses following in arbitrary order * [WebUI] Update column header * [WebUI] Wrap addresses in history Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.5 * [Conf] Add MSBL proposed return codes * [Conf] Add additional groups for policies * [CritFix] Do not use volatile Lua strings as UCL keys * [Feature] Add ability to add fuzzy hashes to headers * [Feature] Add function to extract most meaningful urls * [Feature] Add rule to block mixed text and encrypted parts * [Feature] Allow multiple groups for symbols * [Feature] Allow to disable lua squeezing logic * [Feature] Allow to get multipart children in Lua * [Feature] Allow to insert multiple headers from milter headers * [Feature] Allow to print scores in subject and further extensions * [Feature] Be more error-prone in squeezed rules * [Feature] Support multiple return codes in emails module * [Feature] Use EMA for calculating averages * [Feature] Use common jit cache for all regexps * [Feature] support for CommuniGate Pro self-generated messages * [Fix] Allow to have multiple values for headers as arrays * [Fix] Do not open sockets for disabled workers * [Fix] Fix AuthservId * [Fix] Fix base64 folding in Lua API * [Fix] Fix build on non-x86 platforms * [Fix] Fix cached maps logic * [Fix] Fix compatibility with old maps query logic * [Fix] Fix crash if skip_map is used * [Fix] Fix importing static maps from UCL * [Fix] Fix parsing of unix sockets * [Fix] Fix raw_mime regexp on HTML part with no text content * [Fix] Fix tables logging * [Fix] Fix vertical tab handling in libucl * [Fix] Try to fix frequency counters * [Fix] Use better sharding for ip_score * [Fix] Use multiple results from SURBL DNS reply * [Fix] When doing AV scan select a different server for retransmit Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.5 * [Conf] Add MSBL proposed return codes * [Conf] Add additional groups for policies * [CritFix] Do not use volatile Lua strings as UCL keys * [Feature] Add ability to add fuzzy hashes to headers * [Feature] Add function to extract most meaningful urls * [Feature] Add rule to block mixed text and encrypted parts * [Feature] Allow multiple groups for symbols * [Feature] Allow to disable lua squeezing logic * [Feature] Allow to get multipart children in Lua * [Feature] Allow to insert multiple headers from milter headers * [Feature] Allow to print scores in subject and further extensions * [Feature] Be more error-prone in squeezed rules * [Feature] Support multiple return codes in emails module * [Feature] Use EMA for calculating averages * [Feature] Use common jit cache for all regexps * [Feature] support for CommuniGate Pro self-generated messages * [Fix] Allow to have multiple values for headers as arrays * [Fix] Do not open sockets for disabled workers * [Fix] Fix AuthservId * [Fix] Fix base64 folding in Lua API * [Fix] Fix build on non-x86 platforms * [Fix] Fix cached maps logic * [Fix] Fix compatibility with old maps query logic * [Fix] Fix crash if skip_map is used * [Fix] Fix importing static maps from UCL * [Fix] Fix parsing of unix sockets * [Fix] Fix raw_mime regexp on HTML part with no text content * [Fix] Fix tables logging * [Fix] Fix vertical tab handling in libucl * [Fix] Try to fix frequency counters * [Fix] Use better sharding for ip_score * [Fix] Use multiple results from SURBL DNS reply * [Fix] When doing AV scan select a different server for retransmit Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.4 * [Conf] Major stock config updates: - Workers are now specified in a new format worker "type" { ... } - Enable fuzzy worker to simplify local fuzzy storages configuration - Bind all workers to localhost by default to avoid security flaws * [Conf] Make more sane fuzzy_check default settings * [CritFix] Fix ucl escape for bad symbols * [Feature] Add failure symbol for AV module * [Feature] Add lazy expiration mode for new classifier schema * [Feature] Add preliminary version of maps stats plugin * [Feature] Allow to block fuzzy requests from specific networks * [Feature] Allow to change `expire` of live statistics * [Feature] Distinguish AV failure from clean result * [Feature] Further improvements of language detector by using khash * [Feature] Further optimization of the lang_detection * [Feature] Implement cluster-aware bayes expiry * [Feature] Implement exclude patterns in rspamc * [Feature] Implement glob maps in addition to regexp maps * [Feature] Implement map statistics function for lua API * [Feature] Implement stop symbols for Clickhouse collection * [Feature] Support recipients separated by commas * [Feature] Try harder to upload scripts to the Redis server * [Feature] Upgrade t1ha distribution * [Feature] use_domain_sign_inbound * [Feature] Use scores from maps if `symbols_set` is not defined * [Fix] Add resolving version of radix map helper * [Fix] Check URL before adding implicit prefix * [Fix] Do not check pid/state when using PRNG * [Fix] Fix CentOS logrotate script for systemd * [Fix] Fix slash + dot in urls * [Fix] Fix systemd version of the logrotate script * [Fix] Propagate key when import implicit array from Lua * [Fix] Strip spaces from map keys and values * [Fix] Try to fix a specific case when processing milter protocol * [Fix] Try to fix crash when a tcp connection cannot be set * [Fix] Typo use_domain_local --> use_domain_sign_local * [Fix] Various fixes to once_received module * [Project] Store hits counters for map elements Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.4 * [Conf] Major stock config updates: - Workers are now specified in a new format worker "type" { ... } - Enable fuzzy worker to simplify local fuzzy storages configuration - Bind all workers to localhost by default to avoid security flaws * [Conf] Make more sane fuzzy_check default settings * [CritFix] Fix ucl escape for bad symbols * [Feature] Add failure symbol for AV module * [Feature] Add lazy expiration mode for new classifier schema * [Feature] Add preliminary version of maps stats plugin * [Feature] Allow to block fuzzy requests from specific networks * [Feature] Allow to change `expire` of live statistics * [Feature] Distinguish AV failure from clean result * [Feature] Further improvements of language detector by using khash * [Feature] Further optimization of the lang_detection * [Feature] Implement cluster-aware bayes expiry * [Feature] Implement exclude patterns in rspamc * [Feature] Implement glob maps in addition to regexp maps * [Feature] Implement map statistics function for lua API * [Feature] Implement stop symbols for Clickhouse collection * [Feature] Support recipients separated by commas * [Feature] Try harder to upload scripts to the Redis server * [Feature] Upgrade t1ha distribution * [Feature] use_domain_sign_inbound * [Feature] Use scores from maps if `symbols_set` is not defined * [Fix] Add resolving version of radix map helper * [Fix] Check URL before adding implicit prefix * [Fix] Do not check pid/state when using PRNG * [Fix] Fix CentOS logrotate script for systemd * [Fix] Fix slash + dot in urls * [Fix] Fix systemd version of the logrotate script * [Fix] Propagate key when import implicit array from Lua * [Fix] Strip spaces from map keys and values * [Fix] Try to fix a specific case when processing milter protocol * [Fix] Try to fix crash when a tcp connection cannot be set * [Fix] Typo use_domain_local --> use_domain_sign_local * [Fix] Various fixes to once_received module * [Project] Store hits counters for map elements Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.3 * [CritFix] Plug bad memory leak in protocol reply * [Feature] Add avx2 codec for base64 * [Feature] Add method to receive all URL flags from Lua API * [Feature] Allow to fold headers on stop characters * [Feature] Allow to set lua_cpath from options * [Feature] Allow to specify custom rejection message in milter * [Feature] Deal with unnormalised Unicode obfuscation * [Feature] Do not detect language twice for relative parts * [Feature] Implement oversigning feature * [Feature] Implement silent logging level to minimize noise in logs * [Feature] Improve URL_IN_SUBJECT rule * [Feature] Use hashing to reduce redis attack surface * [Fix] Add oversigning for the most important headers * [Fix] add 'rewrite subject' to History dropdown * [Fix] Another fix in folding algorithm * [Fix] Do not call multimap addr for parts of addr if filter is presented * [Fix] Do not clean hostname on generic reset * [Fix] Do not create pid file in no-fork mode * [Fix] Fix fold_after case to preserve multiple spaces * [Fix] Fix folding and folding tests * [Fix] Fix hostname usage in milter mode * [Fix] Fix lua RSA verify and its tests * [Fix] Fix metadata exporter send_mail backend (#2124) * [Fix] Fix processing of '\v' in libucl * [Fix] Fix shemaless URLs detection * [Fix] Fix support of multiple headers in sign_header * [Fix] Fix usage of util.parse_mail_address * [Fix] Fix weights of dynamic squeezed rules * [Fix] Leak from bucket before checking the burst * [Fix] Stop using own localtime as DST could be messy in many cases * [Fix] Treat unnormalised URLs as obscured * [Rework] Restore leaky bucket model in ratelimit plugin * [WebUI] Add messages total to throughput summary * [WebUI] Add symbols order selector to history * [WebUI] Config: Load list on demand * [WebUI] Fix modalBody for maps that appear more than once * [WebUI] History: Fix Tooltips on paging, filtering and sorting * [WebUI] Remove a previously-attached event handler * [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.3 * [CritFix] Plug bad memory leak in protocol reply * [Feature] Add avx2 codec for base64 * [Feature] Add method to receive all URL flags from Lua API * [Feature] Allow to fold headers on stop characters * [Feature] Allow to set lua_cpath from options * [Feature] Allow to specify custom rejection message in milter * [Feature] Deal with unnormalised Unicode obfuscation * [Feature] Do not detect language twice for relative parts * [Feature] Implement oversigning feature * [Feature] Implement silent logging level to minimize noise in logs * [Feature] Improve URL_IN_SUBJECT rule * [Feature] Use hashing to reduce redis attack surface * [Fix] Add oversigning for the most important headers * [Fix] add 'rewrite subject' to History dropdown * [Fix] Another fix in folding algorithm * [Fix] Do not call multimap addr for parts of addr if filter is presented * [Fix] Do not clean hostname on generic reset * [Fix] Do not create pid file in no-fork mode * [Fix] Fix fold_after case to preserve multiple spaces * [Fix] Fix folding and folding tests * [Fix] Fix hostname usage in milter mode * [Fix] Fix lua RSA verify and its tests * [Fix] Fix metadata exporter send_mail backend (#2124) * [Fix] Fix processing of '\v' in libucl * [Fix] Fix shemaless URLs detection * [Fix] Fix support of multiple headers in sign_header * [Fix] Fix usage of util.parse_mail_address * [Fix] Fix weights of dynamic squeezed rules * [Fix] Leak from bucket before checking the burst * [Fix] Stop using own localtime as DST could be messy in many cases * [Fix] Treat unnormalised URLs as obscured * [Rework] Restore leaky bucket model in ratelimit plugin * [WebUI] Add messages total to throughput summary * [WebUI] Add symbols order selector to history * [WebUI] Config: Load list on demand * [WebUI] Fix modalBody for maps that appear more than once * [WebUI] History: Fix Tooltips on paging, filtering and sorting * [WebUI] Remove a previously-attached event handler * [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.0 * [Conf] Add bayes_expiry as explicit module * [Conf] Adjust names and weights for neural network plugin * [Conf] Change updates url * [Conf] Default statistics is stored in Redis now * [Conf] Disable fann_redis module by default * [Conf] Fix default elastic configuration * [Conf] Fix double quote position * [Conf] Massive config rework for new structure of symbols and scores * [Conf] Rename Rambler BLs as they are now Rspamd's ones * [Conf] Use dedicated rspamd.com subdomains * [Conf] Use more data from rspamd.com fuzzy storage * [CritFix] Add sanity guards for badly broken HTML * [CritFix] Another errors path handling fix * [CritFix] Another portion of tokenization fixes * [CritFix] Do not send reject messages after set reply * [CritFix] Fix ARC chain verification * [CritFix] Fix crash in milter errors handler * [CritFix] Fix memory leak in spf caching logic * [CritFix] Fix milter commands pipelining * [CritFix] Fix newlines detection * [CritFix] Fix semicolons parsing in the content type * [CritFix] Plug memory leak in zstd protocol compression * [Feature] Add ability to match score in force_actions module * [Feature] Add aes-rng PRF to libottery * [Feature] Add 'composites' debug module * [Feature] Add concept of experimental modules * [Feature] Add DKIM trace symbol * [Feature] Add EBL to the default config * [Feature] Add expected ip check for emails plugin * [Feature] Add framework to manage Redis scripts * [Feature] Add framing for the new reputation generic plugin * [Feature] Add function to show plugins stat * [Feature] Add gzip compression support for clickhouse module * [Feature] Add gzip compression support for rspamd controller * [Feature] Add gzip support when sending lua http requests * [Feature] Add json output for rspamd_stats * [Feature] Add method to do a synchronous Redis connection * [Feature] Add method to get all content-type attributes in Lua * [Feature] Add `-m` flag to configdump to show modules states * [Feature] Add mime types to extensions map * [Feature] Add more features to rescore utility * [Feature] Add more gtube like patterns to test other spam actions * [Feature] Add more metafunctions, improve logging * [Feature] Add more text attributes * [Feature] Add new configwizard command to rspamadm * [Feature] Add new tooling for stats conversation * [Feature] Add old groups migration tool * [Feature] Add plugins state variable * [Feature] Add preliminary ecdsa keys support in DKIM * [Feature] Add preliminary support of idempotent symbols * [Feature] Add Redis server wizard * [Feature] Add routine to convert old style stats to a new one * [Feature] Add some sanity checks for actions and controller * [Feature] Add statistic convertation module to configwizard * [Feature] Add sugestions logic to mempool allocator * [Feature] Add support of config transform in Lua * [Feature] Add timeout to rspamc when doing corpus test * [Feature] Add tooling to convert bayes schemas * [Feature] Add torch conditional to configuration * [Feature] Add torch-decisiontree package * [Feature] Add torch-optim contrib package * [Feature] Add TTL autodetection * [Feature] Add urls reputation to the reputation framework * [Feature] Allow floating and negative values in expressions limits * [Feature] Allow multiple CTs in full extensions map * [Feature] Allow multiple fann rules * [Feature] Allow randomly select User-Agent from a list * [Feature] Allow rspamadm commands to export methods in Lua * [Feature] Allow rule specific min_bytes in fuzzy check * [Feature] Allow to adjust symbols scores from Lua * [Feature] Allow to attach stat signature to messages * [Feature] Allow to change SMTP from via milter headers * [Feature] Allow to configure monitored * [Feature] Allow to create directories in Lua API * [Feature] Allow to disable torch and skip train samples for ANN * [Feature] Allow to discard messages dynamically * [Feature] Allow to enable/disable languages from the detector * [Feature] Allow to generate DKIM keys from rspamadm API * [Feature] Allow to get CPU flags from Lua * [Feature] Allow to have high precision timestamps in logs * [Feature] Allow to insert headers into specific position * [Feature] Allow to limit redirector requests per task * [Feature] Allow to load and use dynamic ANNs with torch * [Feature] Allow to quarantine rejected messages using milter interface * [Feature] Allow to receive signing keys from mempool vars * [Feature] Allow to reserve elements in libucl * [Feature] Allow to reuse signal handlers chains * [Feature] Allow to set custom mempool variables from settings * [Feature] Allow to set headers from settings * [Feature] Allow to set Settings-Id for all connections * [Feature] Allow to skip real action and add a header instead * [Feature] Allow to skip specific hashes in fuzzy storage * [Feature] Allow to spawn asynchronous processes from Lua * [Feature] Allow to specify number of threads for ANN learning * [Feature] Allow to use global lua maps in settings * [Feature] Allow to use postfilters in composites * [Feature] Allow to verify signatures from HTTP headers in maps * [Feature] Antivirus: ordered pattern matches * [Feature] Authentication-Results: support hiding usernames * [Feature] Automatically create tables in clickhouse * [Feature] Catch next-to-last bad extension * [Feature] Check cached maps more frequently * [Feature] Check groups sanity * [Feature] Deal with obscured URLs with @ symbols * [Feature] Enhance task:store_in_file method * [Feature] Export password encryption routines to Redis * [Feature] Filter nan and inf when adding scores * [Feature] Finalize 7zip files support * [Feature] Further improvements in language detection * [Feature] Further improvements in language detection algorithm * [Feature] Generic key name expansion for Redis keys * [Feature] Hash whitelist for fuzzy_check * [Feature] Implement bayes signatures storage * [Feature] Implement buckets for Redis backend * [Feature] Implement DKIM reputation adjustments * [Feature] Implement forked workers children monitoring * [Feature] Implement headers flags in mime parser * [Feature] Implement l1/l2 regularization against the current weights * [Feature] Implement manual ANN train mode * [Feature] Implement per-user ANN support * [Feature] Implement torch based ANN learning * [Feature] Implement upstreams logic for clickhouse exporter * [Feature] Import torch to Rspamd... * [Feature] Improve allocation policy when interacting with Lua * [Feature] Improve Lua/C interaction in history_redis * [Feature] Improve multiple fuzzy results combining * [Feature] Improve parsing of DKIM keys: parse algorithm * [Feature] Improve subprocesses termination handle * [Feature] Improve symbol type parsing in Lua API * [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600) * [Feature] Milter headers: support adding/removing arbitrary headers from config * [Feature] More metatokens * [Feature] Multimap: checking of symbol options * [Feature] Multimap: template URL filter * [Feature] New bayes expiry plugin * [Feature] Periodically save rspamd stats to disk * [Feature] Preliminary import of the elasticsearch module * [Feature] Ratelimit: allow full addresses in whitelisted_rcpts * [Feature] Ratelimit: support fetching limits from Redis * [Feature] RBL: received: filtering by position & flags * [Feature] Read global maps for lua * [Feature] Redis settings: support checking multiple keys * [Feature] Rework fann plugin to be a normal post-filter * [Feature] Rework logging configuration for rspamadm case * [Feature] Rework short hashes generation to avoid FP * [Feature] Save real ucl types when exporting to Lua * [Feature] Set TCP_NODELAY for milter sockets * [Feature] Setup DKIM signing from configwizard * [Feature] Skip certain symbols from ANN classify * [Feature] Store plugins state * [Feature] Support etag for HTTP maps * [Feature] Support Expires header when using HTTP maps * [Feature] Support sending given header multiple times in lua_http * [Feature] Support sha512 in DKIM signatures * [Feature] Try to detect HTML messages better * [Feature] Use array instead of queue to reduce memory fragmentation * [Feature] Use controller port by default when connecting to local IP * [Feature] Use rdtsc where possible * [Fix] Actively load skip hashes map in fuzzy storage * [Fix] Add another workaround to display history properly * [Fix] Add definition for old glib compatibility method * [Fix] Add missing rspamadm control options to help * [Fix] Add workaround for IPv6 in sendmail * [Fix] Add workaround for system with non-XSI compatible tzset * [Fix] Allow oversigning in DKIM signatures * [Fix] Allow to check negative scores in force_actions * [Fix] Allow to have negative actions limits * [Fix] Allow to set any layers number for fann rules * [Fix] Another fix for rdtcs * [Fix] Another fix to lua xmlrpc * [Fix] Another try to deal with #1998 * [Fix] Another try to fix #1998 * [Fix] Another try to fix threading in torch * [Fix] Apply language detection when adding fuzzy hashes * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer * [Fix] Authentication Results: Fix SPF smtp.mail_from * [Fix] Auth-Results: Multiple DKIM signatures * [Fix] Avoid changing content-transfer-encoding header's value * [Fix] Better handling of the legacy protocol * [Fix] Check decoded headers sanity (e.g. by excluding \0) * [Fix] Check for magic when checking for an archive * [Fix] Cleanup mess with groups * [Fix] Clickhouse: Insertion in the symbols table * [Fix] Crash in URL processing * [Fix] Deal with another case when processing exceptions * [Fix] Deal with deeply nested messages more aggressively * [Fix] Deal with nan and inf encoding in json/ucl * [Fix] Deal with non-key arguments in lua_redis.exec_script * [Fix] Deal with unknown weight * [Fix] Deal with URLs with no slashes after protocol * [Fix] Deal with URLs wrapped in [] in text parts * [Fix] Deal with zero scores symbols * [Fix] Default monitoring domain for surbl plugin * [Fix] Delay upstream re-resolving when one upstream is defined * [Fix] Detection of maillist optimized and fixed * [Fix] DKIM signing: allow for auth_only to be false * [Fix] DMARC: require report_settings for sending reports only * [Fix] Do not allow garbadge when checking url domain * [Fix] Do not cache SPF records with PTR elements * [Fix] Do not constantly re-resolve failed upstreams with a single element * [Fix] Do not crash if no words defined * [Fix] Do not crash on empty subtype * [Fix] Do not expose spamtrap messages to SMTP reply * [Fix] Do not fail rbl plugin when there are no received or emails * [Fix] Do not ignore short words * [Fix] Do not include idempotent/nostat symbols to checksum * [Fix] Do not override groups when converting metrics * [Fix] Do not override unix socket group when group comes before owner * [Fix] Do not skip the last character * [Fix] Do not spawn too many workers by default * [Fix] Do not stop monitored on dns errors * [Fix] Do not stop parsing headers on bad IP header * [Fix] Do not strip last character in the last word * [Fix] Do not treat script content as text * [Fix] Do not try to connect to non-supported addresses * [Fix] Do not try to dereference last character * [Fix] Do not try to sign unknown domains * [Fix] Don't use whitelist/greylist maps as regexp, but as map * [Fix] Erase unknown HTML entities * [Fix] Exim Received header protocol parsing * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false * [Fix] Fix a lot of FP in chartable in mixed languages * [Fix] Fix ANN checks * [Fix] Fix ANN loading logic * [Fix] Fix another tokenization issue * [Fix] Fix autolearn parameters reading * [Fix] Fix bad archive characters stripping * [Fix] Fix bad extension check * [Fix] Fix bayes schema conversion * [Fix] Fix blacklists and DMARC in whitelist * [Fix] Fix brain-damaged torch build system * [Fix] Fix build on FreeBSD * [Fix] Fix clickhouse exporter * [Fix] Fix clickhouse schema * [Fix] Fix comparision * [Fix] Fix composites processing * [Fix] Fix connecting to a unix socket in rspamadm statconvert * [Fix] Fix couple of warnings * [Fix] Fix crashes in the rspamd_control path * [Fix] Fix deletion from hash * [Fix] Fix DKIM forgeries via multiple headers * [Fix] FIx dynamic conf plugin * [Fix] Fix emails detection * [Fix] Fix empty headers simple canonicalization * [Fix] Fix empty threshold check in greylisting module * [Fix] Fix encrypted legacy reply in fuzzy storage * [Fix] Fix enormous scores for R_WHITE_ON_WHITE * [Fix] Fix exceptions list in surbl * [Fix] Fix *_EXCESS_BASE64 rules * [Fix] Fix expire rounding * [Fix] Fix extra hits in PCRE mode for regular expressions * [Fix] Fix format strings * [Fix] Fix get_content method * [Fix] Fix groups override when defining symbols * [Fix] Fix learned count in new schema * [Fix] Fix learn errors propagation * [Fix] Fix loading of per-user redis backend for statistics * [Fix] Fix logging buffer corruption in case of repeated messages * [Fix] Fix lua cached elements invalidation * [Fix] Fix merging of the implicit arrays * [Fix] Fix mime_types scoring * [Fix] Fix multiple headers in DKIM headers list * [Fix] Fix null callee case in clang plugin * [Fix] Fix obscured url in format user@@example.com * [Fix] Fix parsing of the per-user script * [Fix] Fix priorities in rspamd_update, disable rules execution * [Fix] Fix processing of closed tags * [Fix] Fix processing of idempotent rules when autolearn fails * [Fix] Fix processing of multipart parts with no headers * [Fix] Fix processing of skip-hashes in fuzzy storage * [Fix] Fix PTR processing in SPF * [Fix] Fix pushing country to clickhouse asn table * [Fix] Fix random forests module * [Fix] Fix real IP parsing for some strange Exim received * [Fix] Fix Redis timeout setup * [Fix] Fix reload crash when hyperscan is enabled * [Fix] Fix reusing of redis connection after exec * [Fix] Fix sanity checks on macro value * [Fix] Fix setting of path and cpath for Lua * [Fix] Fix setting of signals when spawning a thread * [Fix] Fix text splitting: stack overflow (too many captures) * [Fix] Fix ticks processing * [Fix] Fix upstream addrs updating * [Fix] Fix urls/emails distinguishing found in queries * [Fix] Fix user settings check * [Fix] Fix variable increment * [Fix] Fix various issues in stat_convert * [Fix] F-PROT Antivirus infection string for all known occurences * [Fix] F-PROT Antivirus: only check return code to determine infection * [Fix] Further fixes around floating point expressions * [Fix] Further fixes to ANN module * [Fix] Further fixes to rescore tool * [Fix] Further fixes to support ES 6 * [Fix] Further tokenization fixes * [Fix] Greylisting set phase is not idempotent * [Fix] Handle proxy copy errors * [Fix] Header checks: Fix get_raw_header method * [Fix] Header checks: REPLYTO_UNPARSEABLE rule * [Fix] Kill spawned processes on termination * [Fix] Load skip map from all processes as shared cache is unavailable * [Fix] Lowercase HTTP headers to make them searchable from Lua * [Fix] Lowercase words * [Fix] Lua_http: freeing * [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found" * [Fix] Map absence is not an error * [Fix] Metadata exporter: check IP sanity * [Fix] Milter headers: custom headers: removing headers * [Fix] Milter headers: skip_local / skip_authenticated settings * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first * [Fix] mime_types: fix next-to-last extension length check * [Fix] More hacks to deal with old configs * [Fix] Move composites second pass to the dedicated stage * [Fix] Multimap: received: filtering of artificial header * [Fix] Multiple fixes in torch based ANN plugins * [Fix] Once more (#1879) fix bad extension check * [Fix] Optimize rspamd_fstring_t reallocations * [Fix] options.local_networks setting * [Fix] Parse HREF urls without explicit prefix * [Fix] Plan new event on HTTP errors * [Fix] Plug another possible memory leak * [Fix] Plug memory leak * [Fix] Plug memory leak in lua_tcp * [Fix] Plug memory leak when setting email addresses from Lua * [Fix] Propagate learn/stat errors more precisely * [Fix] Ratelimit: fix whitelisted_rcpts matching * [Fix] Ratelimit: lowercase email addresses * [Fix] RBL: received: deal with missing data (#1965) * [Fix] Rebalance and slightly rework MX check plugin * [Fix] Redis key expansion: EVAL: deal with strings * [Fix] Redis script loading in DMARC; URL tags; URL reputation * [Fix] Reject invalid bh for DKIM signatures earlier * [Fix] Relax pem signature detection * [Fix] Relax unicode properties requirements for chartable module * [Fix] Remove extra noise from dkim and arc signing * [Fix] Remove hop-by-hop headers in proxy * [Fix] Remove incorrect method `task:set_metric_subject` * [Fix] Replace space like characters in headers with plain space * [Fix] Restore old style ratelimits support * [Fix] Rework elasticsearch plugin * [Fix] Rewriting subjects via force actions module * [Fix] RPM postinstall * [Fix] Sanitize IP in history redis * [Fix] Select the correct signature when doing simple canon * [Fix] Set CLOEXEC flag on files opened * [Fix] Setting check_local / check_authed in plugins (#1954) * [Fix] Settings: avoid checking invalid IP (#1981) * [Fix] Settings: header: deal with multiple settings (#1988) * [Fix] Skip checks if both extensions are not bad * [Fix] Skip nostat tokens when get number of tokens * [Fix] Some more fixes towards emails detection * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match * [Fix] Stop using of g_slice... * [Fix] Switch rspamadm logging to message level * [Fix] Symbol 'FANNR_SPAM' has its score defined.. * [Fix] Table parameter for rspamd_config:add_doc() * [Fix] Treat 'rewrite subject' as spam action * [Fix] Try harder in passing IPv6 addresses * [Fix] Try harder to find rfc822 notifications * [Fix] Try harder to find urls * [Fix] Use decoded values when parsing mime addresses * [Fix] Use full URL when making an HTTP request * [Fix] Use greylisting threshold in greylisting module * [Fix] Use n_words attribute from ngramms * [Fix] Use raw urls when sending requests to redirector * [Fix] Use the right boolean operator on error check * [Fix] Use weight from map for fuzzy scoring * [Fix] Various fixes to elastic plugin * [Fix] Various fixes to fann_redis instantiation * [Fix] Various improvements in language detection * [Fix] Virus infection string for F-PROT Antivirus * [Fix] Virus infetction string for F-PROT Antivirus * [Fix] WebUI: use relative path for savemap (#1943) * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule * [Fix] Write configuration changes as UCL config * [Project] Add detection logic for words * [Project] Add fast debug logging infrastructure * [Project] Add more flags to languages * [Project] Add n-gramms data files * [Project] Add ngramms frequencies detector * [Project] Add random words selection logic * [Project] Add unigramms to language detection as well * [Project] Convert all C modules to fast debug infrastructure * [Project] Detect some languages based on unicode script * [Project] Enable fast debug lookup for some modules * [Project] Enable language detector init in scanner workers * [Project] Further improvements to language detector * [Project] Implement logic of ngramms application * [Project] Improve weighting in lang_detection * [Project] Initialize language detector * [Project] Preliminary version of ngramms based language detector * [Project] Preliminary version of the new stat_convert * [Project] Remove old language detector * [Project] Rework language detection ngramms structure * [Project] Start language detection project * [Project] Start rework of language detection to improve quality * [Project] Use fast debug logging check * [Rework] Add frame for new reputation based IP score module * [Rework] Continue stat_convert rework task * [Rework] Implement new version of fuzzy replies * [Rework] Improve readability of xmlrpc API * [Rework] Kill metrics!11 * [Rework] Ratelimit module * [Rework] Rename fann_redis to neural plugin * [Rework] Reorganize mime_types module * [Rework] Rework rescore utility * [Rework] Rewrite model and learning logic for rescore * [Rework] Run post-loads when all initialization is completed * [Rework] Simplify lua path initialization * [Rework] Start major stat_convert rework * [Rework] Start mempool fragmentation reduce project * [Rework] Start moving of fann redis to torch * [Rework] Stop embedding rspamadm scripts into C * [Rework] Use floating point arithmetics in Rspamd expressions * [Rework] Use frequencies distribution in language detector * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages * [WebUI] Compact graph selectors * [WebUI] Escape strings inside HTML in history * [WebUI] Fix message count in throughput summary (#1724) * [WebUI] Fix NaNs display on Throughput graph * [WebUI] Migrate widgets to D3 v4 * [WebUI] Restore passwordless login support (#2003) * [WebUI] Show symbol descriptions as tooltips in history * [WebUI] Stop using commas in pie chart tooltips * [WebUI] Update D3 and jQuery * [WebUI] Update D3Evolution 1.0.0 -> 1.1.0 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.0 * [Conf] Add bayes_expiry as explicit module * [Conf] Adjust names and weights for neural network plugin * [Conf] Change updates url * [Conf] Default statistics is stored in Redis now * [Conf] Disable fann_redis module by default * [Conf] Fix default elastic configuration * [Conf] Fix double quote position * [Conf] Massive config rework for new structure of symbols and scores * [Conf] Rename Rambler BLs as they are now Rspamd's ones * [Conf] Use dedicated rspamd.com subdomains * [Conf] Use more data from rspamd.com fuzzy storage * [CritFix] Add sanity guards for badly broken HTML * [CritFix] Another errors path handling fix * [CritFix] Another portion of tokenization fixes * [CritFix] Do not send reject messages after set reply * [CritFix] Fix ARC chain verification * [CritFix] Fix crash in milter errors handler * [CritFix] Fix memory leak in spf caching logic * [CritFix] Fix milter commands pipelining * [CritFix] Fix newlines detection * [CritFix] Fix semicolons parsing in the content type * [CritFix] Plug memory leak in zstd protocol compression * [Feature] Add ability to match score in force_actions module * [Feature] Add aes-rng PRF to libottery * [Feature] Add 'composites' debug module * [Feature] Add concept of experimental modules * [Feature] Add DKIM trace symbol * [Feature] Add EBL to the default config * [Feature] Add expected ip check for emails plugin * [Feature] Add framework to manage Redis scripts * [Feature] Add framing for the new reputation generic plugin * [Feature] Add function to show plugins stat * [Feature] Add gzip compression support for clickhouse module * [Feature] Add gzip compression support for rspamd controller * [Feature] Add gzip support when sending lua http requests * [Feature] Add json output for rspamd_stats * [Feature] Add method to do a synchronous Redis connection * [Feature] Add method to get all content-type attributes in Lua * [Feature] Add `-m` flag to configdump to show modules states * [Feature] Add mime types to extensions map * [Feature] Add more features to rescore utility * [Feature] Add more gtube like patterns to test other spam actions * [Feature] Add more metafunctions, improve logging * [Feature] Add more text attributes * [Feature] Add new configwizard command to rspamadm * [Feature] Add new tooling for stats conversation * [Feature] Add old groups migration tool * [Feature] Add plugins state variable * [Feature] Add preliminary ecdsa keys support in DKIM * [Feature] Add preliminary support of idempotent symbols * [Feature] Add Redis server wizard * [Feature] Add routine to convert old style stats to a new one * [Feature] Add some sanity checks for actions and controller * [Feature] Add statistic convertation module to configwizard * [Feature] Add sugestions logic to mempool allocator * [Feature] Add support of config transform in Lua * [Feature] Add timeout to rspamc when doing corpus test * [Feature] Add tooling to convert bayes schemas * [Feature] Add torch conditional to configuration * [Feature] Add torch-decisiontree package * [Feature] Add torch-optim contrib package * [Feature] Add TTL autodetection * [Feature] Add urls reputation to the reputation framework * [Feature] Allow floating and negative values in expressions limits * [Feature] Allow multiple CTs in full extensions map * [Feature] Allow multiple fann rules * [Feature] Allow randomly select User-Agent from a list * [Feature] Allow rspamadm commands to export methods in Lua * [Feature] Allow rule specific min_bytes in fuzzy check * [Feature] Allow to adjust symbols scores from Lua * [Feature] Allow to attach stat signature to messages * [Feature] Allow to change SMTP from via milter headers * [Feature] Allow to configure monitored * [Feature] Allow to create directories in Lua API * [Feature] Allow to disable torch and skip train samples for ANN * [Feature] Allow to discard messages dynamically * [Feature] Allow to enable/disable languages from the detector * [Feature] Allow to generate DKIM keys from rspamadm API * [Feature] Allow to get CPU flags from Lua * [Feature] Allow to have high precision timestamps in logs * [Feature] Allow to insert headers into specific position * [Feature] Allow to limit redirector requests per task * [Feature] Allow to load and use dynamic ANNs with torch * [Feature] Allow to quarantine rejected messages using milter interface * [Feature] Allow to receive signing keys from mempool vars * [Feature] Allow to reserve elements in libucl * [Feature] Allow to reuse signal handlers chains * [Feature] Allow to set custom mempool variables from settings * [Feature] Allow to set headers from settings * [Feature] Allow to set Settings-Id for all connections * [Feature] Allow to skip real action and add a header instead * [Feature] Allow to skip specific hashes in fuzzy storage * [Feature] Allow to spawn asynchronous processes from Lua * [Feature] Allow to specify number of threads for ANN learning * [Feature] Allow to use global lua maps in settings * [Feature] Allow to use postfilters in composites * [Feature] Allow to verify signatures from HTTP headers in maps * [Feature] Antivirus: ordered pattern matches * [Feature] Authentication-Results: support hiding usernames * [Feature] Automatically create tables in clickhouse * [Feature] Catch next-to-last bad extension * [Feature] Check cached maps more frequently * [Feature] Check groups sanity * [Feature] Deal with obscured URLs with @ symbols * [Feature] Enhance task:store_in_file method * [Feature] Export password encryption routines to Redis * [Feature] Filter nan and inf when adding scores * [Feature] Finalize 7zip files support * [Feature] Further improvements in language detection * [Feature] Further improvements in language detection algorithm * [Feature] Generic key name expansion for Redis keys * [Feature] Hash whitelist for fuzzy_check * [Feature] Implement bayes signatures storage * [Feature] Implement buckets for Redis backend * [Feature] Implement DKIM reputation adjustments * [Feature] Implement forked workers children monitoring * [Feature] Implement headers flags in mime parser * [Feature] Implement l1/l2 regularization against the current weights * [Feature] Implement manual ANN train mode * [Feature] Implement per-user ANN support * [Feature] Implement torch based ANN learning * [Feature] Implement upstreams logic for clickhouse exporter * [Feature] Import torch to Rspamd... * [Feature] Improve allocation policy when interacting with Lua * [Feature] Improve Lua/C interaction in history_redis * [Feature] Improve multiple fuzzy results combining * [Feature] Improve parsing of DKIM keys: parse algorithm * [Feature] Improve subprocesses termination handle * [Feature] Improve symbol type parsing in Lua API * [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600) * [Feature] Milter headers: support adding/removing arbitrary headers from config * [Feature] More metatokens * [Feature] Multimap: checking of symbol options * [Feature] Multimap: template URL filter * [Feature] New bayes expiry plugin * [Feature] Periodically save rspamd stats to disk * [Feature] Preliminary import of the elasticsearch module * [Feature] Ratelimit: allow full addresses in whitelisted_rcpts * [Feature] Ratelimit: support fetching limits from Redis * [Feature] RBL: received: filtering by position & flags * [Feature] Read global maps for lua * [Feature] Redis settings: support checking multiple keys * [Feature] Rework fann plugin to be a normal post-filter * [Feature] Rework logging configuration for rspamadm case * [Feature] Rework short hashes generation to avoid FP * [Feature] Save real ucl types when exporting to Lua * [Feature] Set TCP_NODELAY for milter sockets * [Feature] Setup DKIM signing from configwizard * [Feature] Skip certain symbols from ANN classify * [Feature] Store plugins state * [Feature] Support etag for HTTP maps * [Feature] Support Expires header when using HTTP maps * [Feature] Support sending given header multiple times in lua_http * [Feature] Support sha512 in DKIM signatures * [Feature] Try to detect HTML messages better * [Feature] Use array instead of queue to reduce memory fragmentation * [Feature] Use controller port by default when connecting to local IP * [Feature] Use rdtsc where possible * [Fix] Actively load skip hashes map in fuzzy storage * [Fix] Add another workaround to display history properly * [Fix] Add definition for old glib compatibility method * [Fix] Add missing rspamadm control options to help * [Fix] Add workaround for IPv6 in sendmail * [Fix] Add workaround for system with non-XSI compatible tzset * [Fix] Allow oversigning in DKIM signatures * [Fix] Allow to check negative scores in force_actions * [Fix] Allow to have negative actions limits * [Fix] Allow to set any layers number for fann rules * [Fix] Another fix for rdtcs * [Fix] Another fix to lua xmlrpc * [Fix] Another try to deal with #1998 * [Fix] Another try to fix #1998 * [Fix] Another try to fix threading in torch * [Fix] Apply language detection when adding fuzzy hashes * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer * [Fix] Authentication Results: Fix SPF smtp.mail_from * [Fix] Auth-Results: Multiple DKIM signatures * [Fix] Avoid changing content-transfer-encoding header's value * [Fix] Better handling of the legacy protocol * [Fix] Check decoded headers sanity (e.g. by excluding \0) * [Fix] Check for magic when checking for an archive * [Fix] Cleanup mess with groups * [Fix] Clickhouse: Insertion in the symbols table * [Fix] Crash in URL processing * [Fix] Deal with another case when processing exceptions * [Fix] Deal with deeply nested messages more aggressively * [Fix] Deal with nan and inf encoding in json/ucl * [Fix] Deal with non-key arguments in lua_redis.exec_script * [Fix] Deal with unknown weight * [Fix] Deal with URLs with no slashes after protocol * [Fix] Deal with URLs wrapped in [] in text parts * [Fix] Deal with zero scores symbols * [Fix] Default monitoring domain for surbl plugin * [Fix] Delay upstream re-resolving when one upstream is defined * [Fix] Detection of maillist optimized and fixed * [Fix] DKIM signing: allow for auth_only to be false * [Fix] DMARC: require report_settings for sending reports only * [Fix] Do not allow garbadge when checking url domain * [Fix] Do not cache SPF records with PTR elements * [Fix] Do not constantly re-resolve failed upstreams with a single element * [Fix] Do not crash if no words defined * [Fix] Do not crash on empty subtype * [Fix] Do not expose spamtrap messages to SMTP reply * [Fix] Do not fail rbl plugin when there are no received or emails * [Fix] Do not ignore short words * [Fix] Do not include idempotent/nostat symbols to checksum * [Fix] Do not override groups when converting metrics * [Fix] Do not override unix socket group when group comes before owner * [Fix] Do not skip the last character * [Fix] Do not spawn too many workers by default * [Fix] Do not stop monitored on dns errors * [Fix] Do not stop parsing headers on bad IP header * [Fix] Do not strip last character in the last word * [Fix] Do not treat script content as text * [Fix] Do not try to connect to non-supported addresses * [Fix] Do not try to dereference last character * [Fix] Do not try to sign unknown domains * [Fix] Don't use whitelist/greylist maps as regexp, but as map * [Fix] Erase unknown HTML entities * [Fix] Exim Received header protocol parsing * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false * [Fix] Fix a lot of FP in chartable in mixed languages * [Fix] Fix ANN checks * [Fix] Fix ANN loading logic * [Fix] Fix another tokenization issue * [Fix] Fix autolearn parameters reading * [Fix] Fix bad archive characters stripping * [Fix] Fix bad extension check * [Fix] Fix bayes schema conversion * [Fix] Fix blacklists and DMARC in whitelist * [Fix] Fix brain-damaged torch build system * [Fix] Fix build on FreeBSD * [Fix] Fix clickhouse exporter * [Fix] Fix clickhouse schema * [Fix] Fix comparision * [Fix] Fix composites processing * [Fix] Fix connecting to a unix socket in rspamadm statconvert * [Fix] Fix couple of warnings * [Fix] Fix crashes in the rspamd_control path * [Fix] Fix deletion from hash * [Fix] Fix DKIM forgeries via multiple headers * [Fix] FIx dynamic conf plugin * [Fix] Fix emails detection * [Fix] Fix empty headers simple canonicalization * [Fix] Fix empty threshold check in greylisting module * [Fix] Fix encrypted legacy reply in fuzzy storage * [Fix] Fix enormous scores for R_WHITE_ON_WHITE * [Fix] Fix exceptions list in surbl * [Fix] Fix *_EXCESS_BASE64 rules * [Fix] Fix expire rounding * [Fix] Fix extra hits in PCRE mode for regular expressions * [Fix] Fix format strings * [Fix] Fix get_content method * [Fix] Fix groups override when defining symbols * [Fix] Fix learned count in new schema * [Fix] Fix learn errors propagation * [Fix] Fix loading of per-user redis backend for statistics * [Fix] Fix logging buffer corruption in case of repeated messages * [Fix] Fix lua cached elements invalidation * [Fix] Fix merging of the implicit arrays * [Fix] Fix mime_types scoring * [Fix] Fix multiple headers in DKIM headers list * [Fix] Fix null callee case in clang plugin * [Fix] Fix obscured url in format user@@example.com * [Fix] Fix parsing of the per-user script * [Fix] Fix priorities in rspamd_update, disable rules execution * [Fix] Fix processing of closed tags * [Fix] Fix processing of idempotent rules when autolearn fails * [Fix] Fix processing of multipart parts with no headers * [Fix] Fix processing of skip-hashes in fuzzy storage * [Fix] Fix PTR processing in SPF * [Fix] Fix pushing country to clickhouse asn table * [Fix] Fix random forests module * [Fix] Fix real IP parsing for some strange Exim received * [Fix] Fix Redis timeout setup * [Fix] Fix reload crash when hyperscan is enabled * [Fix] Fix reusing of redis connection after exec * [Fix] Fix sanity checks on macro value * [Fix] Fix setting of path and cpath for Lua * [Fix] Fix setting of signals when spawning a thread * [Fix] Fix text splitting: stack overflow (too many captures) * [Fix] Fix ticks processing * [Fix] Fix upstream addrs updating * [Fix] Fix urls/emails distinguishing found in queries * [Fix] Fix user settings check * [Fix] Fix variable increment * [Fix] Fix various issues in stat_convert * [Fix] F-PROT Antivirus infection string for all known occurences * [Fix] F-PROT Antivirus: only check return code to determine infection * [Fix] Further fixes around floating point expressions * [Fix] Further fixes to ANN module * [Fix] Further fixes to rescore tool * [Fix] Further fixes to support ES 6 * [Fix] Further tokenization fixes * [Fix] Greylisting set phase is not idempotent * [Fix] Handle proxy copy errors * [Fix] Header checks: Fix get_raw_header method * [Fix] Header checks: REPLYTO_UNPARSEABLE rule * [Fix] Kill spawned processes on termination * [Fix] Load skip map from all processes as shared cache is unavailable * [Fix] Lowercase HTTP headers to make them searchable from Lua * [Fix] Lowercase words * [Fix] Lua_http: freeing * [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found" * [Fix] Map absence is not an error * [Fix] Metadata exporter: check IP sanity * [Fix] Milter headers: custom headers: removing headers * [Fix] Milter headers: skip_local / skip_authenticated settings * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first * [Fix] mime_types: fix next-to-last extension length check * [Fix] More hacks to deal with old configs * [Fix] Move composites second pass to the dedicated stage * [Fix] Multimap: received: filtering of artificial header * [Fix] Multiple fixes in torch based ANN plugins * [Fix] Once more (#1879) fix bad extension check * [Fix] Optimize rspamd_fstring_t reallocations * [Fix] options.local_networks setting * [Fix] Parse HREF urls without explicit prefix * [Fix] Plan new event on HTTP errors * [Fix] Plug another possible memory leak * [Fix] Plug memory leak * [Fix] Plug memory leak in lua_tcp * [Fix] Plug memory leak when setting email addresses from Lua * [Fix] Propagate learn/stat errors more precisely * [Fix] Ratelimit: fix whitelisted_rcpts matching * [Fix] Ratelimit: lowercase email addresses * [Fix] RBL: received: deal with missing data (#1965) * [Fix] Rebalance and slightly rework MX check plugin * [Fix] Redis key expansion: EVAL: deal with strings * [Fix] Redis script loading in DMARC; URL tags; URL reputation * [Fix] Reject invalid bh for DKIM signatures earlier * [Fix] Relax pem signature detection * [Fix] Relax unicode properties requirements for chartable module * [Fix] Remove extra noise from dkim and arc signing * [Fix] Remove hop-by-hop headers in proxy * [Fix] Remove incorrect method `task:set_metric_subject` * [Fix] Replace space like characters in headers with plain space * [Fix] Restore old style ratelimits support * [Fix] Rework elasticsearch plugin * [Fix] Rewriting subjects via force actions module * [Fix] RPM postinstall * [Fix] Sanitize IP in history redis * [Fix] Select the correct signature when doing simple canon * [Fix] Set CLOEXEC flag on files opened * [Fix] Setting check_local / check_authed in plugins (#1954) * [Fix] Settings: avoid checking invalid IP (#1981) * [Fix] Settings: header: deal with multiple settings (#1988) * [Fix] Skip checks if both extensions are not bad * [Fix] Skip nostat tokens when get number of tokens * [Fix] Some more fixes towards emails detection * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match * [Fix] Stop using of g_slice... * [Fix] Switch rspamadm logging to message level * [Fix] Symbol 'FANNR_SPAM' has its score defined.. * [Fix] Table parameter for rspamd_config:add_doc() * [Fix] Treat 'rewrite subject' as spam action * [Fix] Try harder in passing IPv6 addresses * [Fix] Try harder to find rfc822 notifications * [Fix] Try harder to find urls * [Fix] Use decoded values when parsing mime addresses * [Fix] Use full URL when making an HTTP request * [Fix] Use greylisting threshold in greylisting module * [Fix] Use n_words attribute from ngramms * [Fix] Use raw urls when sending requests to redirector * [Fix] Use the right boolean operator on error check * [Fix] Use weight from map for fuzzy scoring * [Fix] Various fixes to elastic plugin * [Fix] Various fixes to fann_redis instantiation * [Fix] Various improvements in language detection * [Fix] Virus infection string for F-PROT Antivirus * [Fix] Virus infetction string for F-PROT Antivirus * [Fix] WebUI: use relative path for savemap (#1943) * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule * [Fix] Write configuration changes as UCL config * [Project] Add detection logic for words * [Project] Add fast debug logging infrastructure * [Project] Add more flags to languages * [Project] Add n-gramms data files * [Project] Add ngramms frequencies detector * [Project] Add random words selection logic * [Project] Add unigramms to language detection as well * [Project] Convert all C modules to fast debug infrastructure * [Project] Detect some languages based on unicode script * [Project] Enable fast debug lookup for some modules * [Project] Enable language detector init in scanner workers * [Project] Further improvements to language detector * [Project] Implement logic of ngramms application * [Project] Improve weighting in lang_detection * [Project] Initialize language detector * [Project] Preliminary version of ngramms based language detector * [Project] Preliminary version of the new stat_convert * [Project] Remove old language detector * [Project] Rework language detection ngramms structure * [Project] Start language detection project * [Project] Start rework of language detection to improve quality * [Project] Use fast debug logging check * [Rework] Add frame for new reputation based IP score module * [Rework] Continue stat_convert rework task * [Rework] Implement new version of fuzzy replies * [Rework] Improve readability of xmlrpc API * [Rework] Kill metrics!11 * [Rework] Ratelimit module * [Rework] Rename fann_redis to neural plugin * [Rework] Reorganize mime_types module * [Rework] Rework rescore utility * [Rework] Rewrite model and learning logic for rescore * [Rework] Run post-loads when all initialization is completed * [Rework] Simplify lua path initialization * [Rework] Start major stat_convert rework * [Rework] Start mempool fragmentation reduce project * [Rework] Start moving of fann redis to torch * [Rework] Stop embedding rspamadm scripts into C * [Rework] Use floating point arithmetics in Rspamd expressions * [Rework] Use frequencies distribution in language detector * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages * [WebUI] Compact graph selectors * [WebUI] Escape strings inside HTML in history * [WebUI] Fix message count in throughput summary (#1724) * [WebUI] Fix NaNs display on Throughput graph * [WebUI] Migrate widgets to D3 v4 * [WebUI] Restore passwordless login support (#2003) * [WebUI] Show symbol descriptions as tooltips in history * [WebUI] Stop using commas in pie chart tooltips * [WebUI] Update D3 and jQuery * [WebUI] Update D3Evolution 1.0.0 -> 1.1.0 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.6.6 * [CritFix] Add sanity guards for badly broken HTML * [CritFix] Another errors path handling fix * [CritFix] Fix ARC chain verification * [CritFix] Fix crash in milter errors handler * [Feature] Allow to insert headers into specific position * [Feature] Allow to receive signing keys from mempool vars * [Feature] Authentication-Results: support hiding usernames * [Fix] Another try to deal with #1998 * [Fix] Another try to fix #1998 * [Fix] Better handling of the legacy protocol * [Fix] Check decoded headers sanity (e.g. by excluding \0) * [Fix] Deal with nan and inf encoding in json/ucl * [Fix] Deal with URLs wrapped in [] in text parts * [Fix] DKIM signing: allow for auth_only to be false * [Fix] Do not crash on empty subtype * [Fix] Do not fail rbl plugin when there are no received or emails * [Fix] Do not skip the last character * [Fix] Do not try to dereference last character * [Fix] Do not try to sign unknown domains * [Fix] Exim Received header protocol parsing * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false * [Fix] Fix bad archive characters stripping * [Fix] Fix comparision * [Fix] Fix connecting to a unix socket in rspamadm statconvert * [Fix] Fix empty headers simple canonicalization * [Fix] Fix extra hits in PCRE mode for regular expressions * [Fix] Fix parsing of the per-user script * [Fix] Fix processing of skip-hashes in fuzzy storage * [Fix] Fix Redis timeout setup * [Fix] Fix sanity checks on macro value * [Fix] Fix text splitting: stack overflow (too many captures) * [Fix] Fix urls/emails distinguishing found in queries * [Fix] F-PROT Antivirus: only check return code to determine infection * [Fix] Metadata exporter: check IP sanity * [Fix] Multimap: received: filtering of artificial header * [Fix] Plan new event on HTTP errors * [Fix] Plug another possible memory leak * [Fix] Remove hop-by-hop headers in proxy * [Fix] Sanitize IP in history redis * [Fix] Setting check_local / check_authed in plugins (#1954) * [Fix] Settings: avoid checking invalid IP (#1981) * [Fix] Try harder in passing IPv6 addresses * [Fix] WebUI: use relative path for savemap (#1943) * [WebUI] Fix message count in throughput summary (#1724) * [WebUI] Fix NaNs display on Throughput graph * [WebUI] Restore passwordless login support (#2003) Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.6.6 * [CritFix] Add sanity guards for badly broken HTML * [CritFix] Another errors path handling fix * [CritFix] Fix ARC chain verification * [CritFix] Fix crash in milter errors handler * [Feature] Allow to insert headers into specific position * [Feature] Allow to receive signing keys from mempool vars * [Feature] Authentication-Results: support hiding usernames * [Fix] Another try to deal with #1998 * [Fix] Another try to fix #1998 * [Fix] Better handling of the legacy protocol * [Fix] Check decoded headers sanity (e.g. by excluding \0) * [Fix] Deal with nan and inf encoding in json/ucl * [Fix] Deal with URLs wrapped in [] in text parts * [Fix] DKIM signing: allow for auth_only to be false * [Fix] Do not crash on empty subtype * [Fix] Do not fail rbl plugin when there are no received or emails * [Fix] Do not skip the last character * [Fix] Do not try to dereference last character * [Fix] Do not try to sign unknown domains * [Fix] Exim Received header protocol parsing * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false * [Fix] Fix bad archive characters stripping * [Fix] Fix comparision * [Fix] Fix connecting to a unix socket in rspamadm statconvert * [Fix] Fix empty headers simple canonicalization * [Fix] Fix extra hits in PCRE mode for regular expressions * [Fix] Fix parsing of the per-user script * [Fix] Fix processing of skip-hashes in fuzzy storage * [Fix] Fix Redis timeout setup * [Fix] Fix sanity checks on macro value * [Fix] Fix text splitting: stack overflow (too many captures) * [Fix] Fix urls/emails distinguishing found in queries * [Fix] F-PROT Antivirus: only check return code to determine infection * [Fix] Metadata exporter: check IP sanity * [Fix] Multimap: received: filtering of artificial header * [Fix] Plan new event on HTTP errors * [Fix] Plug another possible memory leak * [Fix] Remove hop-by-hop headers in proxy * [Fix] Sanitize IP in history redis * [Fix] Setting check_local / check_authed in plugins (#1954) * [Fix] Settings: avoid checking invalid IP (#1981) * [Fix] Try harder in passing IPv6 addresses * [Fix] WebUI: use relative path for savemap (#1943) * [WebUI] Fix message count in throughput summary (#1724) * [WebUI] Fix NaNs display on Throughput graph * [WebUI] Restore passwordless login support (#2003) Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.6.2 * [Conf] Remove Rambler email bl for now * [Conf] Switch RAMBLER_URIBL to a locally managed source * [CritFix] Switch from ragel to C for Content-Type parsing * [Feature] Add `-e` option for lua_repl * [Feature] Add per-domain emails normalisation rules * [Feature] Add sessions cache to debug dangling sessions * [Feature] Add short_text_direct_hash for fuzzy check module * [Feature] Add text_part:get_stats function * [Feature] Allow to add custom processing script for surbl * [Feature] Allow to check reply-to email * [Feature] Allow to customize spam header, remove existing spam headers * [Feature] Allow to disable specific workers in the config * [Feature] Allow to discard messages instead of rejection * [Feature] Allow to specify custom delimiter in emails plugin * [Feature] Allow to specify custom User-Agent for rspamc * [Feature] Allow to store symbols data in Clickhouse * [Feature] Allow to use HTTPS when connecting to Clickhouse * [Feature] Enable sessions cache tracking for milter connections * [Feature] Implement per-line mode in lua_repl (like `perl -p`) * [Feature] Implement rdns-curve plugin based on rspamd cryptobox * [Feature] Improve maps cached data lifetime * [Feature] Improve maps checking frequency * [Feature] Improve monitored timeouts logic * [Feature] milter_headers: add `extended_headers_rcpt` option * [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers * [Feature] Milter headers: remove-header routine * [Feature] Multimap: received filters for extracting TLDs from hostnames * [Feature] Normalize email aliases in emails module * [Feature] Re-add rambler email bl (as hashed list) * [Feature] Reload file maps more frequently * [Feature] Rework newlines strip parser one more time * [Feature] Skip updates for messages scanned via controller * [Feature] Split long DKIM public keys * [Feature] Store more data when stripping newlines * [Feature] Support SPF macros transformations * [Feature] Support suppressing DMARC reports for some domains * [Fix] Add missing `break` statement * [Fix] Allow modifiers in SPF macros * [Fix] DKIM sign tools: edge-cases around use_esld * [Fix] Do not cache SPF records with macros * [Fix] Do not overwrite score when setting pre-action * [Fix] Fix comparision logic * [Fix] Fix DKIM base64 folding for milter flagged messages * [Fix] Fix emails module configuration * [Fix] Fix folding for arc headers when milter interface is used * [Fix] Fix gmail dots removal * [Fix] Fix rspamc detection in greylist module * [Fix] Fix some more issues with HTTP maps * [Fix] Milter sessions can live forever * [Fix] Normalize fuzzy probability better * [Fix] Plug memory leak * [Fix] RBL: Fixed hashed email address lookups * [Fix] Try to deal with brain-damaged milter behaviour * [Fix] Use `\n` to fold headers for milter * [Rework] Allow to use custom callback for monitored checks * [Rework] Further steps towards one process monitoring * [Rework] Send health checks from a single worker * [WebUI] Round-up throughput summary values Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.6.2 * [Conf] Remove Rambler email bl for now * [Conf] Switch RAMBLER_URIBL to a locally managed source * [CritFix] Switch from ragel to C for Content-Type parsing * [Feature] Add `-e` option for lua_repl * [Feature] Add per-domain emails normalisation rules * [Feature] Add sessions cache to debug dangling sessions * [Feature] Add short_text_direct_hash for fuzzy check module * [Feature] Add text_part:get_stats function * [Feature] Allow to add custom processing script for surbl * [Feature] Allow to check reply-to email * [Feature] Allow to customize spam header, remove existing spam headers * [Feature] Allow to disable specific workers in the config * [Feature] Allow to discard messages instead of rejection * [Feature] Allow to specify custom delimiter in emails plugin * [Feature] Allow to specify custom User-Agent for rspamc * [Feature] Allow to store symbols data in Clickhouse * [Feature] Allow to use HTTPS when connecting to Clickhouse * [Feature] Enable sessions cache tracking for milter connections * [Feature] Implement per-line mode in lua_repl (like `perl -p`) * [Feature] Implement rdns-curve plugin based on rspamd cryptobox * [Feature] Improve maps cached data lifetime * [Feature] Improve maps checking frequency * [Feature] Improve monitored timeouts logic * [Feature] milter_headers: add `extended_headers_rcpt` option * [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers * [Feature] Milter headers: remove-header routine * [Feature] Multimap: received filters for extracting TLDs from hostnames * [Feature] Normalize email aliases in emails module * [Feature] Re-add rambler email bl (as hashed list) * [Feature] Reload file maps more frequently * [Feature] Rework newlines strip parser one more time * [Feature] Skip updates for messages scanned via controller * [Feature] Split long DKIM public keys * [Feature] Store more data when stripping newlines * [Feature] Support SPF macros transformations * [Feature] Support suppressing DMARC reports for some domains * [Fix] Add missing `break` statement * [Fix] Allow modifiers in SPF macros * [Fix] DKIM sign tools: edge-cases around use_esld * [Fix] Do not cache SPF records with macros * [Fix] Do not overwrite score when setting pre-action * [Fix] Fix comparision logic * [Fix] Fix DKIM base64 folding for milter flagged messages * [Fix] Fix emails module configuration * [Fix] Fix folding for arc headers when milter interface is used * [Fix] Fix gmail dots removal * [Fix] Fix rspamc detection in greylist module * [Fix] Fix some more issues with HTTP maps * [Fix] Milter sessions can live forever * [Fix] Normalize fuzzy probability better * [Fix] Plug memory leak * [Fix] RBL: Fixed hashed email address lookups * [Fix] Try to deal with brain-damaged milter behaviour * [Fix] Use `\n` to fold headers for milter * [Rework] Allow to use custom callback for monitored checks * [Rework] Further steps towards one process monitoring * [Rework] Send health checks from a single worker * [WebUI] Round-up throughput summary values Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.6.0 * [Conf] Add rspamd_proxy to the default configuration set * [Conf] Add sample arc module config * [Conf] Do away with systemd specifics completely * [Conf] Increase min_bytes to avoid FP * [Conf] Remove ratelimits from default configuration * [CritFix] Fix accepting on IPv6 sockets * [CritFix] Fix corruption when multiple fuzzy are defined * [CritFix] Fix learn condition in fuzzy check * [CritFix] Fix memory leak in fuzzy check * [CritFix] Fix memory leak in maps scheduling * [CritFix] Paese the last character in DKIM signature correctly * [CritFix] Zero fill sockaddr_un * [Feature] Add ability to add doc strings by example * [Feature] Add API to verify DKIM (and ARC) signatures * [Feature] Add compression/decompression to proxy * [Feature] Add count to url structure * [Feature] Add initial support of the new protocol reply * [Feature] Add Lua plugin spamtrap * [Feature] Add `monitored_address` for rbls * [Feature] Add new schema for bayes tokens * [Feature] Add preliminary ARC support to dkim code * [Feature] Add preliminary support of ARC signing * [Feature] Add rules to detect bad 8bit characters in From and To * [Feature] Add scanning support for milter protocol * [Feature] Add support for bidirectional symbols in rspamd_stats * [Feature] Add support for static maps * [Feature] Add support of maps with multiple regexps matches * [Feature] Add `text_multiplier` param * [Feature] Add the preliminary ARC plugin * [Feature] Add top redirector targets rank * [Feature] Allow async events to be registered from LUA rules * [Feature] Allow storing bayes tokens in Redis * [Feature] Allow to exclude specific domains from mx check * [Feature] Allow to have a stack of watcher finalisers * [Feature] Allow to pass hostname to `-i` flag in Rspamc * [Feature] Allow to set custom user agent in url redirector * [Feature] Allow to use custom callback when parsing resolv.conf * [Feature] Allow to use domain from authenticated user * [Feature] Bayes expiry plugin * [Feature] Check dkim sign keys for modifications * [Feature] DKIM signing: sign_networks/local address specific use_domain settings * [Feature] DMARC: Support excluding domains from sampling * [Feature] Expire processing items for URL redirector aggressively * [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option * [Feature] Implement caching for dkim body hashes * [Feature] Implement milter protocol scan reply * [Feature] Improve omograph phishing detection * [Feature] Initial support of self-scan in Rspamd proxy * [Feature] Keep track of headers in milter interface * [Feature] Milter headers: better controls for local/authenticated * [Feature] Multimap: email:domain:tld filter * [Feature] Preliminary DMARC reporting implementation * [Feature] Reuse stemmers in the cache * [Feature] Rework confighelp to load Lua plugins * [Feature] Rework hfilter to use hyperscan if possible * [Feature] Rework lua RSA API * [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers * [Feature] Start integration of milter support in proxy * [Feature] Store average words length and short words count * [Feature] Store hash of headers order and names * [Feature] Support MTA name header * [Feature] Support multiple types of dkim signing in Lua * [Feature] Support numeric arguments for Redis requests * [Feature] Use headers hash in bayes metatokens * [Feature] Use normal resolv.conf rules of rotation in Rspamd * [Feature] Use version 2 proto for checking messages * [Fix] Allow to follow symlinks when safe * [Fix] Append MX name for authentication results as required * [Fix] Change default text multiplier from 0.5 to 2.0 * [Fix] Check min_bytes for images as well * [Fix] Deal with 7bit charsets properly * [Fix] Deal with 8bit characters in email addresses * [Fix] Deal with unpaired <a> tags * [Fix] Detect confighelp in plugins initialisation * [Fix] Disable certain checks for utf spoof detection * [Fix] DKIM Signing: avoid nil index when From header is missing * [Fix] Do not add exact hashes from different parts * [Fix] Do not check DMARC if SPF or DKIM were not checked * [Fix] Do not check URLs that are resolved to be redirected * [Fix] Do not set bayes probability if we don't use it * [Fix] Do not stop on illegal unicode points - replace them * [Fix] Fix another race condition in arc checks * [Fix] Fix arc count logic * [Fix] Fix ARC signing * [Fix] Fix brain-damaged spamc protocol for now * [Fix] Fix calling for peak functions * [Fix] Fix couple of issues in FORWARDED rule * [Fix] Fix CTE propagation from parent containers to children parts * [Fix] Fix errors processing in the controller * [Fix] Fix format string in milter * [Fix] Fix issues in SPF macros parsing * [Fix] Fix logging format string * [Fix] Fix logic of cached passwords check * [Fix] Fix lowercasing of stemmed words * [Fix] Fix LRU elements removal * [Fix] Fix memory leak when accepting from unix sockets * [Fix] Fix milter connections persistence * [Fix] Fix objects merging in UCL * [Fix] Fix order of operations to avoid race condition * [Fix] Fix parsing of long regexp types * [Fix] Fix passing data to log helper when many symbols defined * [Fix] Fix pools management for milter session * [Fix] Fix processing of the watchers * [Fix] Fix queue id macro in milter * [Fix] Fix R_BAD_CTE_7BIT rule * [Fix] Fix Redis timeout set * [Fix] Fix REPLYTO_UNPARSEABLE rule * [Fix] Fix setting of email address * [Fix] Fix some more issues about duplicated fuzzy requests * [Fix] Fix spamc support in rspamd proxy * [Fix] Fix syntax error in spamtrap plugin * [Fix] Fix url counts for href urls * [Fix] Fix url handling in the protocol * [Fix] Multimap: Received IP filters with Redis * [Fix] Oops, fix d9d0fa5e86db2f4470d34395a233b450478b2f60 * [Fix] Parse rgb[a](x,x,x[,x]) css colors * [Fix] Phishing: strict_domains * [Fix] Reduce maps aggressiveness * [Fix] Reresolve upstreams even if there is a single server there * [Fix] Rspamadm grep: Disable Lua patterns in string search by default * [Fix] Skip text parts when checking binary parts in fuzzy check * [Fix] Support v2 checks in controller * [Fix] Treat empty address as valid * [Fix] Try harder to detect CTE * [Fix] Try to deal with v4 mapped to v6 addresses on accept * [Fix] Use dkim signing callback properly * [Fix] Use non-volatile memory for storing data * [Fix] Use static maps instead of ugly hack for radix_from_config * [Fix] Use the same pool for related sessions * [Rework] Continue modularisation for lua library * [Rework] Initial milter protocol support * [Rework] Make log pipes worker agnostic, add scanners API * [Rework] Move authentication results generation to a separate routine * [Rework] Move common DKIM functions to a separate lua module * [Rework] Move global functions to a separate directory * [Rework] Prepare dkim module for ARC checks * [Rework] Propagate ucl variables from the command line * [Rework] Remove multiple metrics support from Rspamd * [Rework] Stop using name 'rmilter' for the modern protocol * [Rework] Use LFU algorithm in LRU cache * [Rules] Fix received TLS rules * [Rules] Improve URL_COUNT_ODD rule * [WebUI] Fix add header filter in history * [WebUI] Use modern protocol for checking messages Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.6.0 * [Conf] Add rspamd_proxy to the default configuration set * [Conf] Add sample arc module config * [Conf] Do away with systemd specifics completely * [Conf] Increase min_bytes to avoid FP * [Conf] Remove ratelimits from default configuration * [CritFix] Fix accepting on IPv6 sockets * [CritFix] Fix corruption when multiple fuzzy are defined * [CritFix] Fix learn condition in fuzzy check * [CritFix] Fix memory leak in fuzzy check * [CritFix] Fix memory leak in maps scheduling * [CritFix] Paese the last character in DKIM signature correctly * [CritFix] Zero fill sockaddr_un * [Feature] Add ability to add doc strings by example * [Feature] Add API to verify DKIM (and ARC) signatures * [Feature] Add compression/decompression to proxy * [Feature] Add count to url structure * [Feature] Add initial support of the new protocol reply * [Feature] Add Lua plugin spamtrap * [Feature] Add `monitored_address` for rbls * [Feature] Add new schema for bayes tokens * [Feature] Add preliminary ARC support to dkim code * [Feature] Add preliminary support of ARC signing * [Feature] Add rules to detect bad 8bit characters in From and To * [Feature] Add scanning support for milter protocol * [Feature] Add support for bidirectional symbols in rspamd_stats * [Feature] Add support for static maps * [Feature] Add support of maps with multiple regexps matches * [Feature] Add `text_multiplier` param * [Feature] Add the preliminary ARC plugin * [Feature] Add top redirector targets rank * [Feature] Allow async events to be registered from LUA rules * [Feature] Allow storing bayes tokens in Redis * [Feature] Allow to exclude specific domains from mx check * [Feature] Allow to have a stack of watcher finalisers * [Feature] Allow to pass hostname to `-i` flag in Rspamc * [Feature] Allow to set custom user agent in url redirector * [Feature] Allow to use custom callback when parsing resolv.conf * [Feature] Allow to use domain from authenticated user * [Feature] Bayes expiry plugin * [Feature] Check dkim sign keys for modifications * [Feature] DKIM signing: sign_networks/local address specific use_domain settings * [Feature] DMARC: Support excluding domains from sampling * [Feature] Expire processing items for URL redirector aggressively * [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option * [Feature] Implement caching for dkim body hashes * [Feature] Implement milter protocol scan reply * [Feature] Improve omograph phishing detection * [Feature] Initial support of self-scan in Rspamd proxy * [Feature] Keep track of headers in milter interface * [Feature] Milter headers: better controls for local/authenticated * [Feature] Multimap: email:domain:tld filter * [Feature] Preliminary DMARC reporting implementation * [Feature] Reuse stemmers in the cache * [Feature] Rework confighelp to load Lua plugins * [Feature] Rework hfilter to use hyperscan if possible * [Feature] Rework lua RSA API * [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers * [Feature] Start integration of milter support in proxy * [Feature] Store average words length and short words count * [Feature] Store hash of headers order and names * [Feature] Support MTA name header * [Feature] Support multiple types of dkim signing in Lua * [Feature] Support numeric arguments for Redis requests * [Feature] Use headers hash in bayes metatokens * [Feature] Use normal resolv.conf rules of rotation in Rspamd * [Feature] Use version 2 proto for checking messages * [Fix] Allow to follow symlinks when safe * [Fix] Append MX name for authentication results as required * [Fix] Change default text multiplier from 0.5 to 2.0 * [Fix] Check min_bytes for images as well * [Fix] Deal with 7bit charsets properly * [Fix] Deal with 8bit characters in email addresses * [Fix] Deal with unpaired <a> tags * [Fix] Detect confighelp in plugins initialisation * [Fix] Disable certain checks for utf spoof detection * [Fix] DKIM Signing: avoid nil index when From header is missing * [Fix] Do not add exact hashes from different parts * [Fix] Do not check DMARC if SPF or DKIM were not checked * [Fix] Do not check URLs that are resolved to be redirected * [Fix] Do not set bayes probability if we don't use it * [Fix] Do not stop on illegal unicode points - replace them * [Fix] Fix another race condition in arc checks * [Fix] Fix arc count logic * [Fix] Fix ARC signing * [Fix] Fix brain-damaged spamc protocol for now * [Fix] Fix calling for peak functions * [Fix] Fix couple of issues in FORWARDED rule * [Fix] Fix CTE propagation from parent containers to children parts * [Fix] Fix errors processing in the controller * [Fix] Fix format string in milter * [Fix] Fix issues in SPF macros parsing * [Fix] Fix logging format string * [Fix] Fix logic of cached passwords check * [Fix] Fix lowercasing of stemmed words * [Fix] Fix LRU elements removal * [Fix] Fix memory leak when accepting from unix sockets * [Fix] Fix milter connections persistence * [Fix] Fix objects merging in UCL * [Fix] Fix order of operations to avoid race condition * [Fix] Fix parsing of long regexp types * [Fix] Fix passing data to log helper when many symbols defined * [Fix] Fix pools management for milter session * [Fix] Fix processing of the watchers * [Fix] Fix queue id macro in milter * [Fix] Fix R_BAD_CTE_7BIT rule * [Fix] Fix Redis timeout set * [Fix] Fix REPLYTO_UNPARSEABLE rule * [Fix] Fix setting of email address * [Fix] Fix some more issues about duplicated fuzzy requests * [Fix] Fix spamc support in rspamd proxy * [Fix] Fix syntax error in spamtrap plugin * [Fix] Fix url counts for href urls * [Fix] Fix url handling in the protocol * [Fix] Multimap: Received IP filters with Redis * [Fix] Oops, fix d9d0fa5e86db2f4470d34395a233b450478b2f60 * [Fix] Parse rgb[a](x,x,x[,x]) css colors * [Fix] Phishing: strict_domains * [Fix] Reduce maps aggressiveness * [Fix] Reresolve upstreams even if there is a single server there * [Fix] Rspamadm grep: Disable Lua patterns in string search by default * [Fix] Skip text parts when checking binary parts in fuzzy check * [Fix] Support v2 checks in controller * [Fix] Treat empty address as valid * [Fix] Try harder to detect CTE * [Fix] Try to deal with v4 mapped to v6 addresses on accept * [Fix] Use dkim signing callback properly * [Fix] Use non-volatile memory for storing data * [Fix] Use static maps instead of ugly hack for radix_from_config * [Fix] Use the same pool for related sessions * [Rework] Continue modularisation for lua library * [Rework] Initial milter protocol support * [Rework] Make log pipes worker agnostic, add scanners API * [Rework] Move authentication results generation to a separate routine * [Rework] Move common DKIM functions to a separate lua module * [Rework] Move global functions to a separate directory * [Rework] Prepare dkim module for ARC checks * [Rework] Propagate ucl variables from the command line * [Rework] Remove multiple metrics support from Rspamd * [Rework] Stop using name 'rmilter' for the modern protocol * [Rework] Use LFU algorithm in LRU cache * [Rules] Fix received TLS rules * [Rules] Improve URL_COUNT_ODD rule * [WebUI] Fix add header filter in history * [WebUI] Use modern protocol for checking messages Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.5.5 * [CritFix] Fix classifier learning with Redis backend * [CritFix] Fix issue when parsing encoded rfc822/messages * [Feature] Add escaped version of lua_ucl import * [Feature] Add task:headers_foreach function * [Feature] Allow to process filenames from content type * [Feature] Allow to query hashed emails * [Feature] Ignore bayes with mostly metatokens or with too few text * [Feature] Probabilistically skip metatokens * [Feature] Retrieve all virus names from SAVAPI * [Feature] Rework classifiers lua metatokens * [Feature] Store headers order * [Feature] Store text tokens inside bayes tokens * [Feature] Use cached shingles keys * [Fix] Add missing score normalisation for HFILTER_URL_ONLY * [Fix] Avoid lookup in absent hash * [Fix] Check return values from Lua functions called from C * [Fix] Do not count sending and loading time in rspamc * [Fix] Escape json strings for controller rejplies from Lua * [Fix] Fix archive scans for savapi * [Fix] Fix domain_only emails RBL * [Fix] Fix ip_score map configuration * [Fix] Fix JSON output for history_redis * [Fix] Fix one character length substrings search * [Fix] Fix parsing of non-RFC compatible Exim received * [Fix] Fix parsing of options for workers with the same type * [Fix] Fix processing of small tokens vectors * [Fix] Fix rfc2047 tokenization * [Fix] Fix typo * [Fix] More fixes for inplace decoding * [Fix] Try to avoid modifications of the original data * [Fix] URL redirector: Fix call to is_redirector * [Rework] Set token data as uint64_t instead of chars array * [WebUI] Check if neighbours' history backend versions match * [WebUI] Disable phrase connectors replacement in history filtering * [WebUI] Disable phrase connectors replacement in symbols filtering * [WebUI] Do not hide messages with bad subject, just replace it with '???' * [WebUI] Fix error message * [WebUI] Fix history v2 display * [WebUI] Fix legacy history * [WebUI] history: break To address lists on commas * [WebUI] Increase default timeout to 20 seconds * [WebUI] Save some history table space Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.5.5 * [CritFix] Fix classifier learning with Redis backend * [CritFix] Fix issue when parsing encoded rfc822/messages * [Feature] Add escaped version of lua_ucl import * [Feature] Add task:headers_foreach function * [Feature] Allow to process filenames from content type * [Feature] Allow to query hashed emails * [Feature] Ignore bayes with mostly metatokens or with too few text * [Feature] Probabilistically skip metatokens * [Feature] Retrieve all virus names from SAVAPI * [Feature] Rework classifiers lua metatokens * [Feature] Store headers order * [Feature] Store text tokens inside bayes tokens * [Feature] Use cached shingles keys * [Fix] Add missing score normalisation for HFILTER_URL_ONLY * [Fix] Avoid lookup in absent hash * [Fix] Check return values from Lua functions called from C * [Fix] Do not count sending and loading time in rspamc * [Fix] Escape json strings for controller rejplies from Lua * [Fix] Fix archive scans for savapi * [Fix] Fix domain_only emails RBL * [Fix] Fix ip_score map configuration * [Fix] Fix JSON output for history_redis * [Fix] Fix one character length substrings search * [Fix] Fix parsing of non-RFC compatible Exim received * [Fix] Fix parsing of options for workers with the same type * [Fix] Fix processing of small tokens vectors * [Fix] Fix rfc2047 tokenization * [Fix] Fix typo * [Fix] More fixes for inplace decoding * [Fix] Try to avoid modifications of the original data * [Fix] URL redirector: Fix call to is_redirector * [Rework] Set token data as uint64_t instead of chars array * [WebUI] Check if neighbours' history backend versions match * [WebUI] Disable phrase connectors replacement in history filtering * [WebUI] Disable phrase connectors replacement in symbols filtering * [WebUI] Do not hide messages with bad subject, just replace it with '???' * [WebUI] Fix error message * [WebUI] Fix history v2 display * [WebUI] Fix legacy history * [WebUI] history: break To address lists on commas * [WebUI] Increase default timeout to 20 seconds * [WebUI] Save some history table space Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.5.4 * [Conf] Add history_redis default configuration * [Feature] Add spoofed rules * [Feature] Add URL_IN_SUBJECT rule * [Feature] Allow to get task's subject * [Feature] Allow to specify maximum number of shots for symbols * [Feature] Distinguish URLs found in Subject * [Feature] Memoize LPEG grammars * [Feature] Parse else parts in SA rules * [Feature] Process subject for mixed characters * [Feature] Resolve url chains in url_redirector module * [Feature] Stat greylisted messages as greylisted not soft-rejected * [Feature] Support checking for redirector in Lua SURBL * [Feature] Support tag_exists SA function * [Feature] Work with broken rfc2047 tokens * [Fix] Check all watcher's dependencies * [Fix] Do not compile hyperscan with no SSSE3 support * [Fix] Do not crash if cannot decode qp encoded part * [Fix] Fix dependencies of DKIM when multiple signatures are found * [Fix] Fix lists in whitelist plugin * [Fix] Fix one-shot symbols weight calculations * [Fix] Fix options and shots match * [Fix] Fix order of symbol options * [Fix] Fix parsing of dot at the end of the address * [Fix] Fix parsing of lua table arguments * [Fix] Fix processing of subject words * [Fix] Fix string split memoization * [Fix] Fix templates grammar usage * [Fix] Fix various issues related to Lua stack manipulation * [Fix] Force actions: Use postfilter if we have honor_action / require_action * [Fix] Further fixes to avoid PHISHING FP * [Fix] Preserve order of options in symbols * [Fix] Rspamadm grep: deal with unusually-formatted logs * [Fix] Use hostname suffix when dealing with history * [Rework] Remove outdated SA rules * [WebUI] Add flexible columns * [WebUI] Add footable * [WebUI] Add sender, recipients and subject columns * [WebUI] Allow message-id break * [WebUI] Fix history clustering * [WebUI] Fix history display * [WebUI] Fix sorting * [WebUI] Humanize sizes * [WebUI] Initial move towards footable * [WebUI] Remove datatables * [WebUI] Replace `.values` method with `.map` * [WebUI] Rework v2 symbols display * [WebUI] Try to normalize frequencies * [WebUI] Unbreak WebUI * [WebUI] Use Footable to draw Throughput summary table Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.5.4 * [Conf] Add history_redis default configuration * [Feature] Add spoofed rules * [Feature] Add URL_IN_SUBJECT rule * [Feature] Allow to get task's subject * [Feature] Allow to specify maximum number of shots for symbols * [Feature] Distinguish URLs found in Subject * [Feature] Memoize LPEG grammars * [Feature] Parse else parts in SA rules * [Feature] Process subject for mixed characters * [Feature] Resolve url chains in url_redirector module * [Feature] Stat greylisted messages as greylisted not soft-rejected * [Feature] Support checking for redirector in Lua SURBL * [Feature] Support tag_exists SA function * [Feature] Work with broken rfc2047 tokens * [Fix] Check all watcher's dependencies * [Fix] Do not compile hyperscan with no SSSE3 support * [Fix] Do not crash if cannot decode qp encoded part * [Fix] Fix dependencies of DKIM when multiple signatures are found * [Fix] Fix lists in whitelist plugin * [Fix] Fix one-shot symbols weight calculations * [Fix] Fix options and shots match * [Fix] Fix order of symbol options * [Fix] Fix parsing of dot at the end of the address * [Fix] Fix parsing of lua table arguments * [Fix] Fix processing of subject words * [Fix] Fix string split memoization * [Fix] Fix templates grammar usage * [Fix] Fix various issues related to Lua stack manipulation * [Fix] Force actions: Use postfilter if we have honor_action / require_action * [Fix] Further fixes to avoid PHISHING FP * [Fix] Preserve order of options in symbols * [Fix] Rspamadm grep: deal with unusually-formatted logs * [Fix] Use hostname suffix when dealing with history * [Rework] Remove outdated SA rules * [WebUI] Add flexible columns * [WebUI] Add footable * [WebUI] Add sender, recipients and subject columns * [WebUI] Allow message-id break * [WebUI] Fix history clustering * [WebUI] Fix history display * [WebUI] Fix sorting * [WebUI] Humanize sizes * [WebUI] Initial move towards footable * [WebUI] Remove datatables * [WebUI] Replace `.values` method with `.map` * [WebUI] Rework v2 symbols display * [WebUI] Try to normalize frequencies * [WebUI] Unbreak WebUI * [WebUI] Use Footable to draw Throughput summary table Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.5.3 * [Conf] Add composite for hacked wordpress phishing * [CritFix] Fix base64 decoding when there are unparseable characters * [Feature] Additional symbol metadata in metadata exporter * [Feature] Add method to get protocol reply from Lua * [Feature] Add symbols when tagged rcpt/sender are normalised * [Feature] Add task:get_symbols_all() function * [Feature] Allow multiple formats of DKIM signing key * [Feature] Allow to cache and use flexible protocol reply * [Feature] Allow to set one_shot flag from register_symbol * [Feature] Allow to skip certain types of hashes when learning fuzzy * [Feature] Cache and insert scan time into the protocol * [Feature] Detect newlines in rspamc --mime * [Feature] DKIM signing: support use of maps * [Feature] Greylist: Support excluding low-scoring messages from greylisting * [Feature] Implement lua history in controller * [Feature] Implement redis history querying * [Feature] Preliminary implementation of redis history plugin * [Feature] Support using request headers in settings * [Fix] Change default template to deal with non-ASCII characters * [Fix] Deal with lists of maps in whitelist module * [Fix] DKIM signing: use domain-specific signing key * [Fix] Do not reallocate completed zstd buffer * [Fix] Do not use local_addrs in proxy * [Fix] Fix crash when resolver is undefined * [Fix] Fix double free when closing lua_tcp connections * [Fix] Fix for lua 5.3 * [Fix] Fix freeing of arrays iterators * [Fix] Fix issue with task:get_symbol and symbols with no metric * [Fix] Fix log line duplication in `rspamadm grep` * [Fix] Fix memory corruption on termination * [Fix] Fix out-of-bound access in base64 decode * [Fix] Fix ratelimit + greylisting * [Fix] Fix subject rewriting * [Fix] Fix task:set_recipients function * [Fix] Fix URI_COUNT_ODD rule * [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule * [Fix] Greylist: Suppress greylist action for whitelisted hosts too * [Fix] Metadata exporter: use rule-specific settings for emails * [Fix] Properly set missing fields in exporter * [Fix] Proxy: max_retries option * [Fix] RCPT_COUNT fixes * [Fix] Rework HAS_X_PRIO rule to match symbols conventions * [Fix] Update issues in ac-trie * [Fix] Use optimised base64 decoding in DKIM * [WebUI] Add preliminary v2 history parser * [WebUI] Allow different history parsers * [WebUI] Display symbols * [WebUI] Rework history v2 function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.5.3 * [Conf] Add composite for hacked wordpress phishing * [CritFix] Fix base64 decoding when there are unparseable characters * [Feature] Additional symbol metadata in metadata exporter * [Feature] Add method to get protocol reply from Lua * [Feature] Add symbols when tagged rcpt/sender are normalised * [Feature] Add task:get_symbols_all() function * [Feature] Allow multiple formats of DKIM signing key * [Feature] Allow to cache and use flexible protocol reply * [Feature] Allow to set one_shot flag from register_symbol * [Feature] Allow to skip certain types of hashes when learning fuzzy * [Feature] Cache and insert scan time into the protocol * [Feature] Detect newlines in rspamc --mime * [Feature] DKIM signing: support use of maps * [Feature] Greylist: Support excluding low-scoring messages from greylisting * [Feature] Implement lua history in controller * [Feature] Implement redis history querying * [Feature] Preliminary implementation of redis history plugin * [Feature] Support using request headers in settings * [Fix] Change default template to deal with non-ASCII characters * [Fix] Deal with lists of maps in whitelist module * [Fix] DKIM signing: use domain-specific signing key * [Fix] Do not reallocate completed zstd buffer * [Fix] Do not use local_addrs in proxy * [Fix] Fix crash when resolver is undefined * [Fix] Fix double free when closing lua_tcp connections * [Fix] Fix for lua 5.3 * [Fix] Fix freeing of arrays iterators * [Fix] Fix issue with task:get_symbol and symbols with no metric * [Fix] Fix log line duplication in `rspamadm grep` * [Fix] Fix memory corruption on termination * [Fix] Fix out-of-bound access in base64 decode * [Fix] Fix ratelimit + greylisting * [Fix] Fix subject rewriting * [Fix] Fix task:set_recipients function * [Fix] Fix URI_COUNT_ODD rule * [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule * [Fix] Greylist: Suppress greylist action for whitelisted hosts too * [Fix] Metadata exporter: use rule-specific settings for emails * [Fix] Properly set missing fields in exporter * [Fix] Proxy: max_retries option * [Fix] RCPT_COUNT fixes * [Fix] Rework HAS_X_PRIO rule to match symbols conventions * [Fix] Update issues in ac-trie * [Fix] Use optimised base64 decoding in DKIM * [WebUI] Add preliminary v2 history parser * [WebUI] Allow different history parsers * [WebUI] Display symbols * [WebUI] Rework history v2 function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.5.0 * [Conf] Add configurations for asn, clickhouse and dcc * [Conf] Add default config for url redirector plugin * [Conf] Add the default config for greylist module * [Conf] Allow to edit all local maps from WebUI by default * [CritFix] Deal with absent headers in DKIM * [CritFix] Do not trust remote shingles count * [CritFix] Fix bad memory leak in TLS certificates validation * [CritFix] Fix critical memory issues with radix maps * [CritFix] Fix descriptors leak on reload * [CritFix] Fix headers selection in DKIM verification * [CritFix] Fix parsing of boundaries that end with `--` * [CritFix] Repair PTR_ARRAY_FOREACH macro * [Feature] Add CORS support to the controller * [Feature] Add FROM_NAME_EXCESS_SPACE rule * [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule * [Feature] Add `caseless_hash` method to `lua_util` * [Feature] Add `rip` keyword to ratelimit module * [Feature] Add a simple benchmark for content type parsing * [Feature] Add boundaries parsing in content type * [Feature] Add charset detection for text parts * [Feature] Add content disposition parser * [Feature] Add fallback if too many updates are failing * [Feature] Add function to convert struct tm to time using timezone * [Feature] Add function to normalize HTTP paths * [Feature] Add fuzzy collection plugin * [Feature] Add fuzzy logic for images * [Feature] Add gmime parser to mime_tool * [Feature] Add heuristic to detect broken messages * [Feature] Add heuristic to find displayed URLs * [Feature] Add heuristic to process broken email addresses * [Feature] Add images normalization * [Feature] Add mechanism for disabling composites (Fixes #1270) * [Feature] Add method to create regexp from a glob pattern * [Feature] Add mime encoding manipulation routines * [Feature] Add mime tool to explore messages * [Feature] Add more meta tokens from received headers * [Feature] Add neighbours option to support Rspamd cluster in WebUI * [Feature] Add new function to parse mime addresses * [Feature] Add new methods for lua_tcp * [Feature] Add own headers decoding routine * [Feature] Add own routine to generate a message id * [Feature] Add parser for SMTP date * [Feature] Add per-task lua cache to reuse 'heavy' objects * [Feature] Add plugins list path in WebUI * [Feature] Add preliminary multipart support * [Feature] Add preliminary version of DKIM signing module * [Feature] Add profiling support in client output * [Feature] Add rfc2047 grammar * [Feature] Add rfc2047 variant for QP decoding * [Feature] Add rmilter_headers module (Fixes #1227) * [Feature] Add sse42 version of base64 decoding * [Feature] Add ssse3 and avx2 base64 decoders * [Feature] Add support of libgd * [Feature] Add the preliminary version of redirects resolver in Lua * [Feature] Add ucl_object_iterate_full function * [Feature] Add url encoding function * [Feature] Allow SOA requests in lua dns * [Feature] Allow custom parse types in lua ucl * [Feature] Allow plugins to register webui handlers * [Feature] Allow to add options explicitly to symbols * [Feature] Allow to call a callback when symbol frequency is on peak * [Feature] Allow to call redirector script from SURBL * [Feature] Allow to create variable length dkim keys * [Feature] Allow to have module specific options for Redis in plugins * [Feature] Allow to pass sign key directly from Lua * [Feature] Allow to register configuration docs from Lua API * [Feature] Allow to return options as a table * [Feature] Allow to set peak callbacks from Lua * [Feature] Allow to specify custom method for a message * [Feature] Allow to store dkim keys in Redis * [Feature] Allow to store messages in files * [Feature] Apply DCT using AAN for fuzzy signature * [Feature] Avira SAVAPI support * [Feature] Cache and simplify DCT and jpeg decode * [Feature] Cache libicu convertors * [Feature] Detect URLs with suspicious omographs * [Feature] Do not increase score for duplicate options * [Feature] Do not trust CTE, check base64 and qp strictly * [Feature] Dynamic reputation in URL reputation plugin * [Feature] Extend redis lock when learning spawned * [Feature] Filter non-utf chars from all decoded headers * [Feature] Fix phishing detection for IDNA urls * [Feature] Ignore bad symbols on base64 decoding * [Feature] Ignore too wide elements in SPF * [Feature] Implement fuzzy collection mode * [Feature] Implement helo maps in multimap * [Feature] Implement human readable buckets configuration * [Feature] Implement min-hash shingles for DCT data from images * [Feature] Implement new algorithm for fuzzy hashes of images * [Feature] Implement new unicode normalizer * [Feature] Implement quoted printable decoding * [Feature] Implement received headers flags * [Feature] Implement rspamdgrep tool * [Feature] Implement sane checksum for config file * [Feature] Implement url tags concept * [Feature] Improve detection of omographs using libicu * [Feature] Improve url redirector module * [Feature] Multimap: Received header processing * [Feature] Multiple improvements in the maps * [Feature] New URL filters in multimap * [Feature] Plugin to force actions on selected symbols * [Feature] RBL module: support hashing for emails and helo RBL * [Feature] Reuse URL tags in SURBL module * [Feature] Rework RRD ds count, add conversion path * [Feature] Rework surbl module to avoid extra redirector calls * [Feature] Send config id to the WebUI * [Feature] Simplify HTTPCrypt client support * [Feature] Skip processing for large images * [Feature] Start collection only mode implementation for fuzzy storage * [Feature] Start import of the optimized base64 decode * [Feature] Store all received headers in lua * [Feature] Store relational order of all headers in a message * [Feature] Support DKIM signing in Lua plugins * [Feature] Support HTTPCrypt client in lua_http * [Feature] Support setting SMTP message in multimap * [Feature] Support setting metric subject from Lua * [Feature] Support setting subject in force actions module * [Feature] Treat v6 mapped addresses as v4 addresses * [Feature] URL reputation plugin * [Feature] Use Redis instead of memcached in URLs redirector * [Feature] Use Rspamd rfc2047 decoder instead of gmime one * [Feature] Use a different normalization for fuzzy images * [Feature] Use normalized images in fuzzy hashes * [Feature] Use own code for parsing of date * [Feature] Use shingles for images fuzzying * [Feature] Use t1ha for hashes, allow inlining * [Feature] Use t1ha instead of metrohash and xxhash32 * [Feature] Various new features in metadata exporter module * [Feature] rmilter_headers: authentication-results (#78) * [Fix] Add additional check to mark redis connection inactive * [Fix] Add packed attribute for protocol structure * [Fix] Adopt OMOGRAPH_URL rule * [Fix] Allow static maps * [Fix] Allow to disable classifiers checks using settings and conditions * [Fix] Another try to fix 0 length maps * [Fix] Another try to fix corruption during maps reload * [Fix] Another try to fix descriptors leak * [Fix] Another try to fix reload and logger * [Fix] Antivirus module: register virtual symbols for patterns * [Fix] Avoid extensive reallocs * [Fix] Avoid mempool leak in SA plugin on reload * [Fix] Avoid race condition on saving cache and reload * [Fix] Avoid reusing g_error (Fixes #1262) * [Fix] Break pool connection on fatal redis errors * [Fix] Check for NaN properly * [Fix] Couple of fixes for date parsing * [Fix] Date header timezone adjustments (#1279) * [Fix] Deal with EOF properly * [Fix] Decode filename in content disposition * [Fix] Disable fuzzy images by default * [Fix] Disable zero-copy mode for text parts to avoid crashes * [Fix] Do not destroy session when not all finish scripts are done * [Fix] Do not greyscale images * [Fix] Do not leave parent-less workers processes on fatal errors * [Fix] Do not lowercase Content-Disposition to perform decoding * [Fix] Do not penalize characters just after numeric prefix * [Fix] Do not refork workers that are intended to die * [Fix] Do not set pre-result and update records for no `Queue-ID` messages * [Fix] Do not skip post-filters when pre-filters have set some results * [Fix] Do not stop symbols planning if async events are pending * [Fix] Do not try to set keys for unencrypted requests in proxy * [Fix] Encode URLs according to rfc3986 * [Fix] Encode URLs before sending them to the protocol * [Fix] Filter bad characters from message id * [Fix] Fix CTE detection heuristic * [Fix] Fix Content-Type in HTTP requests * [Fix] Fix IDN eslds phishing checks * [Fix] Fix adding maps from config in Lua * [Fix] Fix another reload memory issue * [Fix] Fix argument returned on redis backend errors * [Fix] Fix assertion in graph handling * [Fix] Fix body trie matching * [Fix] Fix build * [Fix] Fix byte array expansion during toutf8 conversion * [Fix] Fix charset normalisation * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added * [Fix] Fix couple of cornercases with email addresses * [Fix] Fix couple of issues * [Fix] Fix dependencies tracking for callback symbols * [Fix] Fix detection of jpeg size * [Fix] Fix errors handling in fuzzy backend initialization * [Fix] Fix fuzzy hashes count * [Fix] Fix globbing and convert lists to arrays in fuzzy_check * [Fix] Fix heuristical CTE detection for QP encoding * [Fix] Fix ignoring of bad text parts * [Fix] Fix indexes in array access, interleave loop * [Fix] Fix int64 -> double conversion * [Fix] Fix invalid memory access on reload * [Fix] Fix issues with empty updates * [Fix] Fix issues with quoted-printable encoding * [Fix] Fix keys names * [Fix] Fix lots of issues in mime parser code * [Fix] Fix lua maps load * [Fix] Fix macro name * [Fix] Fix mas group score calculations * [Fix] Fix matching of the same patterns from different tries * [Fix] Fix memory corruprtion and leak * [Fix] Fix memory leak in HTTP maps * [Fix] Fix memory leak in expression destroying * [Fix] Fix memory leak in parsing of mime names * [Fix] Fix memory leak in safe ucl iterators * [Fix] Fix memory leak on reload in plugins * [Fix] Fix modules reconfigure on reload * [Fix] Fix monitored setup fro URLBLs with IP addresses * [Fix] Fix name of var * [Fix] Fix new rrd updates * [Fix] Fix out of bounds access * [Fix] Fix parsing messages with no body * [Fix] Fix parsing of '=' character in headers * [Fix] Fix parsing of messages with no content type * [Fix] Fix plugins callbacks in webui * [Fix] Fix possible memory corruption in redis pool * [Fix] Fix probability calculations for fuzzy redis backend * [Fix] Fix processing errors in lua_tcp * [Fix] Fix processing of emails with name only * [Fix] Fix processing of non-multipart messages * [Fix] Fix processing of parts with no valid content type * [Fix] Fix race condition in SIGUSR2 handler * [Fix] Fix redis options parsing when no redis servers are defined * [Fix] Fix reload and hyperscan ready event * [Fix] Fix reload memory issue * [Fix] Fix rra_ptr conversion * [Fix] Fix rrd file conversion * [Fix] Fix setting of content-type attributes * [Fix] Fix signing headers creation in DKIM * [Fix] Fix stddev calculations * [Fix] Fix surbl plugin to work with composite maps * [Fix] Fix timezones parsing * [Fix] Fix tokens usage * [Fix] Fix urls and emails hashes * [Fix] Fix usage of unsafe ucl iterators * [Fix] Fix work with broken utf8 tokens * [Fix] Fix writing of user to roll history * [Fix] Forgotten worker * [Fix] Further memory leaks fixes * [Fix] Ignore lua metatokens in bayes for now * [Fix] Improve OMOGRAPH_URL rule * [Fix] Lua IP from string should be invalid if parsing failed * [Fix] Miltiple fixes to new lua_tcp, add debugging * [Fix] More fixes for iterators cleanup * [Fix] More fixes to logger initialization * [Fix] More heuristic fixes for phishing detection * [Fix] More leaks eliminated * [Fix] More leaks... * [Fix] More random fixes for reload... * [Fix] Multimap: Fixes for email filters * [Fix] Multiple fixes for fann module * [Fix] Multiple memory corruption fixes * [Fix] Normalize path in HTTP router * [Fix] Plug memory leak * [Fix] Plug memory leak in adding radix trees * [Fix] Plug memory leak in configuration parser * [Fix] Plug memory leak in expressions parsing during reload * [Fix] Plug memory leak in learning fuzzy storage * [Fix] Plug memory leak in lua_tcp * [Fix] Plug reload leaks * [Fix] Plug termination memory leaks * [Fix] Really increase lock lifetime * [Fix] Replies module: fix symbol weight * [Fix] Restore content type params related functions * [Fix] Set task's subject from mime subject * [Fix] Sigh, one more reload leak * [Fix] Simplify images shingles * [Fix] Some more memory issues are fixed * [Fix] Stop hardcoding of lua in C * [Fix] Stop processing of bad parts as text parts * [Fix] Strictly filter bad characters when emittin json * [Fix] Strings returned from lua are ephemeral * [Fix] Support unix sockets for lua redis * [Fix] Try to fix issues with reloading config * [Fix] Try to fix race condition in redis_pool * [Fix] Use checksum to avoid intersection between different ANNs * [Fix] Use rspamd hashes in embedded ucl * [Fix] Use sane default rewrite subject (*** SPAM *** %s) * [Fix] Various collection mode fixes * [Fix] Various fixes to mime parser * [Fix] Various reload leak fixing * [Fix] Whitelist certain extensions from archive checks * [Rework] Add preliminary implementation of the mime parser * [Rework] Adopt code for the new options * [Rework] Change logger setup interface * [Rework] Composite configuration (#1270) * [Rework] Finally remove gmime dependency from Rspamd * [Rework] Further fixes to symbols frequencies * [Rework] Implement content type parser for mime * [Rework] Kill all InternetAddressList usages * [Rework] Multiple fixes for symbols cache statistics * [Rework] Refactor struct names * [Rework] Rework images fuzzy hashes algorithm * [Rework] Rework lua_tcp to allow TCP dialog * [Rework] Start massive rework to get rid of gmime * [Rework] Start new approach for multiparts parsing * [Rework] Start rework of mime addresses * [Rework] Start rework of symbols cache updates * [Rework] Start switching to libicu * [Rework] Use a special structure for stats tokens * [Rework] Use hash tables for symbols options * [Rework] Use libicu instead of iconv for conversions * [Rework] Use new scheme to parse mime parts * [WebUI] Add Access-Control-Allow-Origin for cluster management * [WebUI] Add Throughput graph autorefreshing (#820) * [WebUI] Add Visibility.js library * [WebUI] Add basic cluster support to Throughput tab * [WebUI] Add graph legend entries for new DSes * [WebUI] Add graph tab * [WebUI] Add neighbours RRD data consolidation * [WebUI] Add preliminary save symbols clustering * [WebUI] Add server selector to navbar * [WebUI] Add soft reject to auth stats * [WebUI] Add summary to the Throughput tab * [WebUI] Allow to save maps on the cluster * [WebUI] Avoid extra graph redraw and alerts glitching * [WebUI] Be more generous with AJAX timeout * [WebUI] Disable error ring loading in `read only` mode * [WebUI] Enclose table header cells with `tr`s * [WebUI] Finish interface rework * [WebUI] Fix RRD summary pie chart position * [WebUI] Fix `All SERVERS` graph fot just one available server * [WebUI] Fix case when no cluster is defined * [WebUI] Fix compatibility with non-ES6 compliant browsers * [WebUI] Fix config ID * [WebUI] Fix configuration page partially * [WebUI] Fix disabled state * [WebUI] Fix graph dataset selector initialization * [WebUI] Fix graph selectors state resetting * [WebUI] Fix mouse events on throughput summary table area * [WebUI] Fix multiple JS issues * [WebUI] Fix pie chart displaying * [WebUI] Fix read only * [WebUI] Fix read only2 * [WebUI] Fix retarded datatables * [WebUI] Fix soft reject in pie chart * [WebUI] Fix stat widgets timers multiplication on `Refresh` click * [WebUI] Fix symbols config * [WebUI] Fix various errors with login form * [WebUI] Further fixes * [WebUI] Hide learning tab in read-only mode * [WebUI] Initial clusters support * [WebUI] Make legend entry colours more contrast * [WebUI] Move configuration tab to a separate module * [WebUI] Move history tab * [WebUI] Move symbols config as well * [WebUI] New sec to time function * [WebUI] Prevent multiple clicks on `Refresh` * [WebUI] RRD summary: Hide inner labels of tiny pie sectors * [WebUI] RRD summary: Respect undefined values * [WebUI] Reduce font size of graph's legend * [WebUI] Remove orphaned font duplicates * [WebUI] Remove unused code * [WebUI] Replace spinner with animated glyphicon * [WebUI] Reset refresh timer on server switching * [WebUI] Rework interface to use requirejs * [WebUI] Rework neighbours query function * [WebUI] Separate attributes by space * [WebUI] Set focus to password field (#1230) * [WebUI] Simplify neighbours table populating * [WebUI] Start rework of modules * [WebUI] Stop stats refreshing if the page is hidden * [WebUI] Turn d3pie's stuff into a reusable function, * [WebUI] Unify send data functions * [WebUI] Update D3Evolution to 0.0.1 * [WebUI] Update d3.js * [WebUI] Update datatables to work with the requirejs * [WebUI] Use unified tab click event handler, * [WebUI] clusters for the chart * [WebUI] fix uptime Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.5.0 * [Conf] Add configurations for asn, clickhouse and dcc * [Conf] Add default config for url redirector plugin * [Conf] Add the default config for greylist module * [Conf] Allow to edit all local maps from WebUI by default * [CritFix] Deal with absent headers in DKIM * [CritFix] Do not trust remote shingles count * [CritFix] Fix bad memory leak in TLS certificates validation * [CritFix] Fix critical memory issues with radix maps * [CritFix] Fix descriptors leak on reload * [CritFix] Fix headers selection in DKIM verification * [CritFix] Fix parsing of boundaries that end with `--` * [CritFix] Repair PTR_ARRAY_FOREACH macro * [Feature] Add CORS support to the controller * [Feature] Add FROM_NAME_EXCESS_SPACE rule * [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule * [Feature] Add `caseless_hash` method to `lua_util` * [Feature] Add `rip` keyword to ratelimit module * [Feature] Add a simple benchmark for content type parsing * [Feature] Add boundaries parsing in content type * [Feature] Add charset detection for text parts * [Feature] Add content disposition parser * [Feature] Add fallback if too many updates are failing * [Feature] Add function to convert struct tm to time using timezone * [Feature] Add function to normalize HTTP paths * [Feature] Add fuzzy collection plugin * [Feature] Add fuzzy logic for images * [Feature] Add gmime parser to mime_tool * [Feature] Add heuristic to detect broken messages * [Feature] Add heuristic to find displayed URLs * [Feature] Add heuristic to process broken email addresses * [Feature] Add images normalization * [Feature] Add mechanism for disabling composites (Fixes #1270) * [Feature] Add method to create regexp from a glob pattern * [Feature] Add mime encoding manipulation routines * [Feature] Add mime tool to explore messages * [Feature] Add more meta tokens from received headers * [Feature] Add neighbours option to support Rspamd cluster in WebUI * [Feature] Add new function to parse mime addresses * [Feature] Add new methods for lua_tcp * [Feature] Add own headers decoding routine * [Feature] Add own routine to generate a message id * [Feature] Add parser for SMTP date * [Feature] Add per-task lua cache to reuse 'heavy' objects * [Feature] Add plugins list path in WebUI * [Feature] Add preliminary multipart support * [Feature] Add preliminary version of DKIM signing module * [Feature] Add profiling support in client output * [Feature] Add rfc2047 grammar * [Feature] Add rfc2047 variant for QP decoding * [Feature] Add rmilter_headers module (Fixes #1227) * [Feature] Add sse42 version of base64 decoding * [Feature] Add ssse3 and avx2 base64 decoders * [Feature] Add support of libgd * [Feature] Add the preliminary version of redirects resolver in Lua * [Feature] Add ucl_object_iterate_full function * [Feature] Add url encoding function * [Feature] Allow SOA requests in lua dns * [Feature] Allow custom parse types in lua ucl * [Feature] Allow plugins to register webui handlers * [Feature] Allow to add options explicitly to symbols * [Feature] Allow to call a callback when symbol frequency is on peak * [Feature] Allow to call redirector script from SURBL * [Feature] Allow to create variable length dkim keys * [Feature] Allow to have module specific options for Redis in plugins * [Feature] Allow to pass sign key directly from Lua * [Feature] Allow to register configuration docs from Lua API * [Feature] Allow to return options as a table * [Feature] Allow to set peak callbacks from Lua * [Feature] Allow to specify custom method for a message * [Feature] Allow to store dkim keys in Redis * [Feature] Allow to store messages in files * [Feature] Apply DCT using AAN for fuzzy signature * [Feature] Avira SAVAPI support * [Feature] Cache and simplify DCT and jpeg decode * [Feature] Cache libicu convertors * [Feature] Detect URLs with suspicious omographs * [Feature] Do not increase score for duplicate options * [Feature] Do not trust CTE, check base64 and qp strictly * [Feature] Dynamic reputation in URL reputation plugin * [Feature] Extend redis lock when learning spawned * [Feature] Filter non-utf chars from all decoded headers * [Feature] Fix phishing detection for IDNA urls * [Feature] Ignore bad symbols on base64 decoding * [Feature] Ignore too wide elements in SPF * [Feature] Implement fuzzy collection mode * [Feature] Implement helo maps in multimap * [Feature] Implement human readable buckets configuration * [Feature] Implement min-hash shingles for DCT data from images * [Feature] Implement new algorithm for fuzzy hashes of images * [Feature] Implement new unicode normalizer * [Feature] Implement quoted printable decoding * [Feature] Implement received headers flags * [Feature] Implement rspamdgrep tool * [Feature] Implement sane checksum for config file * [Feature] Implement url tags concept * [Feature] Improve detection of omographs using libicu * [Feature] Improve url redirector module * [Feature] Multimap: Received header processing * [Feature] Multiple improvements in the maps * [Feature] New URL filters in multimap * [Feature] Plugin to force actions on selected symbols * [Feature] RBL module: support hashing for emails and helo RBL * [Feature] Reuse URL tags in SURBL module * [Feature] Rework RRD ds count, add conversion path * [Feature] Rework surbl module to avoid extra redirector calls * [Feature] Send config id to the WebUI * [Feature] Simplify HTTPCrypt client support * [Feature] Skip processing for large images * [Feature] Start collection only mode implementation for fuzzy storage * [Feature] Start import of the optimized base64 decode * [Feature] Store all received headers in lua * [Feature] Store relational order of all headers in a message * [Feature] Support DKIM signing in Lua plugins * [Feature] Support HTTPCrypt client in lua_http * [Feature] Support setting SMTP message in multimap * [Feature] Support setting metric subject from Lua * [Feature] Support setting subject in force actions module * [Feature] Treat v6 mapped addresses as v4 addresses * [Feature] URL reputation plugin * [Feature] Use Redis instead of memcached in URLs redirector * [Feature] Use Rspamd rfc2047 decoder instead of gmime one * [Feature] Use a different normalization for fuzzy images * [Feature] Use normalized images in fuzzy hashes * [Feature] Use own code for parsing of date * [Feature] Use shingles for images fuzzying * [Feature] Use t1ha for hashes, allow inlining * [Feature] Use t1ha instead of metrohash and xxhash32 * [Feature] Various new features in metadata exporter module * [Feature] rmilter_headers: authentication-results (#78) * [Fix] Add additional check to mark redis connection inactive * [Fix] Add packed attribute for protocol structure * [Fix] Adopt OMOGRAPH_URL rule * [Fix] Allow static maps * [Fix] Allow to disable classifiers checks using settings and conditions * [Fix] Another try to fix 0 length maps * [Fix] Another try to fix corruption during maps reload * [Fix] Another try to fix descriptors leak * [Fix] Another try to fix reload and logger * [Fix] Antivirus module: register virtual symbols for patterns * [Fix] Avoid extensive reallocs * [Fix] Avoid mempool leak in SA plugin on reload * [Fix] Avoid race condition on saving cache and reload * [Fix] Avoid reusing g_error (Fixes #1262) * [Fix] Break pool connection on fatal redis errors * [Fix] Check for NaN properly * [Fix] Couple of fixes for date parsing * [Fix] Date header timezone adjustments (#1279) * [Fix] Deal with EOF properly * [Fix] Decode filename in content disposition * [Fix] Disable fuzzy images by default * [Fix] Disable zero-copy mode for text parts to avoid crashes * [Fix] Do not destroy session when not all finish scripts are done * [Fix] Do not greyscale images * [Fix] Do not leave parent-less workers processes on fatal errors * [Fix] Do not lowercase Content-Disposition to perform decoding * [Fix] Do not penalize characters just after numeric prefix * [Fix] Do not refork workers that are intended to die * [Fix] Do not set pre-result and update records for no `Queue-ID` messages * [Fix] Do not skip post-filters when pre-filters have set some results * [Fix] Do not stop symbols planning if async events are pending * [Fix] Do not try to set keys for unencrypted requests in proxy * [Fix] Encode URLs according to rfc3986 * [Fix] Encode URLs before sending them to the protocol * [Fix] Filter bad characters from message id * [Fix] Fix CTE detection heuristic * [Fix] Fix Content-Type in HTTP requests * [Fix] Fix IDN eslds phishing checks * [Fix] Fix adding maps from config in Lua * [Fix] Fix another reload memory issue * [Fix] Fix argument returned on redis backend errors * [Fix] Fix assertion in graph handling * [Fix] Fix body trie matching * [Fix] Fix build * [Fix] Fix byte array expansion during toutf8 conversion * [Fix] Fix charset normalisation * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added * [Fix] Fix couple of cornercases with email addresses * [Fix] Fix couple of issues * [Fix] Fix dependencies tracking for callback symbols * [Fix] Fix detection of jpeg size * [Fix] Fix errors handling in fuzzy backend initialization * [Fix] Fix fuzzy hashes count * [Fix] Fix globbing and convert lists to arrays in fuzzy_check * [Fix] Fix heuristical CTE detection for QP encoding * [Fix] Fix ignoring of bad text parts * [Fix] Fix indexes in array access, interleave loop * [Fix] Fix int64 -> double conversion * [Fix] Fix invalid memory access on reload * [Fix] Fix issues with empty updates * [Fix] Fix issues with quoted-printable encoding * [Fix] Fix keys names * [Fix] Fix lots of issues in mime parser code * [Fix] Fix lua maps load * [Fix] Fix macro name * [Fix] Fix mas group score calculations * [Fix] Fix matching of the same patterns from different tries * [Fix] Fix memory corruprtion and leak * [Fix] Fix memory leak in HTTP maps * [Fix] Fix memory leak in expression destroying * [Fix] Fix memory leak in parsing of mime names * [Fix] Fix memory leak in safe ucl iterators * [Fix] Fix memory leak on reload in plugins * [Fix] Fix modules reconfigure on reload * [Fix] Fix monitored setup fro URLBLs with IP addresses * [Fix] Fix name of var * [Fix] Fix new rrd updates * [Fix] Fix out of bounds access * [Fix] Fix parsing messages with no body * [Fix] Fix parsing of '=' character in headers * [Fix] Fix parsing of messages with no content type * [Fix] Fix plugins callbacks in webui * [Fix] Fix possible memory corruption in redis pool * [Fix] Fix probability calculations for fuzzy redis backend * [Fix] Fix processing errors in lua_tcp * [Fix] Fix processing of emails with name only * [Fix] Fix processing of non-multipart messages * [Fix] Fix processing of parts with no valid content type * [Fix] Fix race condition in SIGUSR2 handler * [Fix] Fix redis options parsing when no redis servers are defined * [Fix] Fix reload and hyperscan ready event * [Fix] Fix reload memory issue * [Fix] Fix rra_ptr conversion * [Fix] Fix rrd file conversion * [Fix] Fix setting of content-type attributes * [Fix] Fix signing headers creation in DKIM * [Fix] Fix stddev calculations * [Fix] Fix surbl plugin to work with composite maps * [Fix] Fix timezones parsing * [Fix] Fix tokens usage * [Fix] Fix urls and emails hashes * [Fix] Fix usage of unsafe ucl iterators * [Fix] Fix work with broken utf8 tokens * [Fix] Fix writing of user to roll history * [Fix] Forgotten worker * [Fix] Further memory leaks fixes * [Fix] Ignore lua metatokens in bayes for now * [Fix] Improve OMOGRAPH_URL rule * [Fix] Lua IP from string should be invalid if parsing failed * [Fix] Miltiple fixes to new lua_tcp, add debugging * [Fix] More fixes for iterators cleanup * [Fix] More fixes to logger initialization * [Fix] More heuristic fixes for phishing detection * [Fix] More leaks eliminated * [Fix] More leaks... * [Fix] More random fixes for reload... * [Fix] Multimap: Fixes for email filters * [Fix] Multiple fixes for fann module * [Fix] Multiple memory corruption fixes * [Fix] Normalize path in HTTP router * [Fix] Plug memory leak * [Fix] Plug memory leak in adding radix trees * [Fix] Plug memory leak in configuration parser * [Fix] Plug memory leak in expressions parsing during reload * [Fix] Plug memory leak in learning fuzzy storage * [Fix] Plug memory leak in lua_tcp * [Fix] Plug reload leaks * [Fix] Plug termination memory leaks * [Fix] Really increase lock lifetime * [Fix] Replies module: fix symbol weight * [Fix] Restore content type params related functions * [Fix] Set task's subject from mime subject * [Fix] Sigh, one more reload leak * [Fix] Simplify images shingles * [Fix] Some more memory issues are fixed * [Fix] Stop hardcoding of lua in C * [Fix] Stop processing of bad parts as text parts * [Fix] Strictly filter bad characters when emittin json * [Fix] Strings returned from lua are ephemeral * [Fix] Support unix sockets for lua redis * [Fix] Try to fix issues with reloading config * [Fix] Try to fix race condition in redis_pool * [Fix] Use checksum to avoid intersection between different ANNs * [Fix] Use rspamd hashes in embedded ucl * [Fix] Use sane default rewrite subject (*** SPAM *** %s) * [Fix] Various collection mode fixes * [Fix] Various fixes to mime parser * [Fix] Various reload leak fixing * [Fix] Whitelist certain extensions from archive checks * [Rework] Add preliminary implementation of the mime parser * [Rework] Adopt code for the new options * [Rework] Change logger setup interface * [Rework] Composite configuration (#1270) * [Rework] Finally remove gmime dependency from Rspamd * [Rework] Further fixes to symbols frequencies * [Rework] Implement content type parser for mime * [Rework] Kill all InternetAddressList usages * [Rework] Multiple fixes for symbols cache statistics * [Rework] Refactor struct names * [Rework] Rework images fuzzy hashes algorithm * [Rework] Rework lua_tcp to allow TCP dialog * [Rework] Start massive rework to get rid of gmime * [Rework] Start new approach for multiparts parsing * [Rework] Start rework of mime addresses * [Rework] Start rework of symbols cache updates * [Rework] Start switching to libicu * [Rework] Use a special structure for stats tokens * [Rework] Use hash tables for symbols options * [Rework] Use libicu instead of iconv for conversions * [Rework] Use new scheme to parse mime parts * [WebUI] Add Access-Control-Allow-Origin for cluster management * [WebUI] Add Throughput graph autorefreshing (#820) * [WebUI] Add Visibility.js library * [WebUI] Add basic cluster support to Throughput tab * [WebUI] Add graph legend entries for new DSes * [WebUI] Add graph tab * [WebUI] Add neighbours RRD data consolidation * [WebUI] Add preliminary save symbols clustering * [WebUI] Add server selector to navbar * [WebUI] Add soft reject to auth stats * [WebUI] Add summary to the Throughput tab * [WebUI] Allow to save maps on the cluster * [WebUI] Avoid extra graph redraw and alerts glitching * [WebUI] Be more generous with AJAX timeout * [WebUI] Disable error ring loading in `read only` mode * [WebUI] Enclose table header cells with `tr`s * [WebUI] Finish interface rework * [WebUI] Fix RRD summary pie chart position * [WebUI] Fix `All SERVERS` graph fot just one available server * [WebUI] Fix case when no cluster is defined * [WebUI] Fix compatibility with non-ES6 compliant browsers * [WebUI] Fix config ID * [WebUI] Fix configuration page partially * [WebUI] Fix disabled state * [WebUI] Fix graph dataset selector initialization * [WebUI] Fix graph selectors state resetting * [WebUI] Fix mouse events on throughput summary table area * [WebUI] Fix multiple JS issues * [WebUI] Fix pie chart displaying * [WebUI] Fix read only * [WebUI] Fix read only2 * [WebUI] Fix retarded datatables * [WebUI] Fix soft reject in pie chart * [WebUI] Fix stat widgets timers multiplication on `Refresh` click * [WebUI] Fix symbols config * [WebUI] Fix various errors with login form * [WebUI] Further fixes * [WebUI] Hide learning tab in read-only mode * [WebUI] Initial clusters support * [WebUI] Make legend entry colours more contrast * [WebUI] Move configuration tab to a separate module * [WebUI] Move history tab * [WebUI] Move symbols config as well * [WebUI] New sec to time function * [WebUI] Prevent multiple clicks on `Refresh` * [WebUI] RRD summary: Hide inner labels of tiny pie sectors * [WebUI] RRD summary: Respect undefined values * [WebUI] Reduce font size of graph's legend * [WebUI] Remove orphaned font duplicates * [WebUI] Remove unused code * [WebUI] Replace spinner with animated glyphicon * [WebUI] Reset refresh timer on server switching * [WebUI] Rework interface to use requirejs * [WebUI] Rework neighbours query function * [WebUI] Separate attributes by space * [WebUI] Set focus to password field (#1230) * [WebUI] Simplify neighbours table populating * [WebUI] Start rework of modules * [WebUI] Stop stats refreshing if the page is hidden * [WebUI] Turn d3pie's stuff into a reusable function, * [WebUI] Unify send data functions * [WebUI] Update D3Evolution to 0.0.1 * [WebUI] Update d3.js * [WebUI] Update datatables to work with the requirejs * [WebUI] Use unified tab click event handler, * [WebUI] clusters for the chart * [WebUI] fix uptime Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.4.1 * [Feature] ASN support in Clickhouse module * [Feature] Add clickhouse plugin * [Feature] Add generic tool to add universal maps for lua modules * [Feature] Add logger.debugm to debug lua modules * [Feature] Allow to register metrics symbols using register_symbol * [Feature] Allow to specify prefix for fann_redis * [Feature] Clickhouse: support different masks for IPv4/IPv6 * [Feature] Support forcing action in antivirus plugin * [Fix] Add handling of regexp maps * [Fix] Allow backslashes in http urls * [Fix] Avoid mapping of empty files * [Fix] Do not load tld file to speed up rspamadm * [Fix] Do not resolve numeric IP addresses due to ipv6 insanity * [Fix] Filter incorrect training data * [Fix] Fix Fuzzyconvert tool when password or DB is given * [Fix] Fix build with custom glib/gmime * [Fix] Fix converting of learn count from sqlite to redis * [Fix] Fix crashes with invalid received and task:set_from_ip * [Fix] Fix external dependencies for SA module * [Fix] Fix fann_redis when number of scores has been changed * [Fix] Fix hyperscan usage for non compatible platforms * [Fix] Fix loading of maps from UCL objects * [Fix] Fix memory leak for task-less redis requests * [Fix] Fix mid module with new maps syntax * [Fix] Fix parsing of URLs with username * [Fix] Fix re cache initialisation * [Fix] Fix replacements to sanitize '%' character * [Fix] Fix set and regexp like static maps * [Fix] Fix some issues in redis settings * [Fix] Fix static IP maps * [Fix] Fix total learns counter for redis stats * [Fix] Fix usage of config during reload * [Fix] Fix various warnings and issues * [Fix] Invalidate ANN if training data is incorrect * [Fix] Miltiple fixes to fann_redis module * [Fix] More fixes for URLs with backslashes * [Fix] Properly get options for ip_score module * [Fix] Relax requirements for Received as gmail cannot RFC * [Fix] Remove or fix hyperscan incompatible regexps * [Fix] Settings: correctly read redis config * [Rework] Rework lua logger interface slightly * [Rework] Use new maps add function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.4.1 * [Feature] ASN support in Clickhouse module * [Feature] Add clickhouse plugin * [Feature] Add generic tool to add universal maps for lua modules * [Feature] Add logger.debugm to debug lua modules * [Feature] Allow to register metrics symbols using register_symbol * [Feature] Allow to specify prefix for fann_redis * [Feature] Clickhouse: support different masks for IPv4/IPv6 * [Feature] Support forcing action in antivirus plugin * [Fix] Add handling of regexp maps * [Fix] Allow backslashes in http urls * [Fix] Avoid mapping of empty files * [Fix] Do not load tld file to speed up rspamadm * [Fix] Do not resolve numeric IP addresses due to ipv6 insanity * [Fix] Filter incorrect training data * [Fix] Fix Fuzzyconvert tool when password or DB is given * [Fix] Fix build with custom glib/gmime * [Fix] Fix converting of learn count from sqlite to redis * [Fix] Fix crashes with invalid received and task:set_from_ip * [Fix] Fix external dependencies for SA module * [Fix] Fix fann_redis when number of scores has been changed * [Fix] Fix hyperscan usage for non compatible platforms * [Fix] Fix loading of maps from UCL objects * [Fix] Fix memory leak for task-less redis requests * [Fix] Fix mid module with new maps syntax * [Fix] Fix parsing of URLs with username * [Fix] Fix re cache initialisation * [Fix] Fix replacements to sanitize '%' character * [Fix] Fix set and regexp like static maps * [Fix] Fix some issues in redis settings * [Fix] Fix static IP maps * [Fix] Fix total learns counter for redis stats * [Fix] Fix usage of config during reload * [Fix] Fix various warnings and issues * [Fix] Invalidate ANN if training data is incorrect * [Fix] Miltiple fixes to fann_redis module * [Fix] More fixes for URLs with backslashes * [Fix] Properly get options for ip_score module * [Fix] Relax requirements for Received as gmail cannot RFC * [Fix] Remove or fix hyperscan incompatible regexps * [Fix] Settings: correctly read redis config * [Rework] Rework lua logger interface slightly * [Rework] Use new maps add function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.4.0 * [CritFix] Add guards for inactive redis connections * [CritFix] Another fix for proxying files using rspamd_proxy * [CritFix] Cleanup inactive redis connections * [CritFix] Do not sometimes try to exec posfilters before classification * [CritFix] Fix application of IPv6 mask * [CritFix] Fix chunked encoding when reading messages * [CritFix] Fix file mode for rspamd_proxy * [CritFix] Fix hyperscan compilation on regexp change * [CritFix] Fix issue with finding of end of lines pointers * [CritFix] Fix iteration over headers array (introduced in 1.4) * [CritFix] Fix processing of learned tokens count for redis backend * [CritFix] Fix race condition in checking of cached maps * [CritFix] Fix workers scripts by sharing workers configs * [CritFix] Introduce raw content to text parts * [CritFix] Plug memory leak and potential memory corruption * [Feature] Adaptive ratelimits * [Feature] Add ASN -> rbldnsd script for asn.rspamd.com * [Feature] Add DMARC_NA symbol * [Feature] Add F-Prot support to antivirus module * [Feature] Add HTTP backend to metadata exporter * [Feature] Add Lua API module for monitored objects * [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols * [Feature] Add R_DKIM_PERMFAIL symbol * [Feature] Add R_SPF_PERMFAIL symbol * [Feature] Add Sophos antivirus support * [Feature] Add ZSTD compression to Lua API * [Feature] Add `mid` Lua module * [Feature] Add `one_param` flag for metric symbols * [Feature] Add a generic lua classifier * [Feature] Add a very basic interface to access workers data from on_load * [Feature] Add ability to delete a hash by its data to fuzzy_check plugin * [Feature] Add ability to enable/disable symbols via dynamic_conf * [Feature] Add ability to lookup settings by key * [Feature] Add common way to disable Lua modules * [Feature] Add compression support to rspamd client * [Feature] Add condition to do antiviral check * [Feature] Add configuration for lua classifiers * [Feature] Add configuration knobs for the errors circular buffer * [Feature] Add decompression support in rspamd client * [Feature] Add errors exporter to the controller * [Feature] Add expected value for monitored DNS resources * [Feature] Add exporter from error ringbuf to ucl * [Feature] Add extended version for fann creation function * [Feature] Add ffi friendly version of process_regexp function * [Feature] Add frequency and time display to webui * [Feature] Add fuzzy_delhash command to rspamc client * [Feature] Add implementation of redis connections pool * [Feature] Add latency and offline time monitoring * [Feature] Add learning support for lua classifiers * [Feature] Add max-size and timeout options to CGP helper * [Feature] Add method to enable/disable symbols in config * [Feature] Add methods to get metric's actions and symbols from Lua * [Feature] Add mmap support to lua_text * [Feature] Add monitored object for surbl plugin * [Feature] Add more exceptions to surbl whitelist * [Feature] Add more meta-tokens to bayes * [Feature] Add neural net classifier to fann_scores module * [Feature] Add neural net serialization/deserialization * [Feature] Add new dynamic conf module * [Feature] Add periodic events support for lua_config * [Feature] Add plugin to check MX'es for the sender's domain * [Feature] Add preliminary monitored module * [Feature] Add preliminary support of dynamic conf updates in Redis * [Feature] Add preliminary version of clamav plugin * [Feature] Add redis cache to asn module * [Feature] Add replies compression * [Feature] Add spamhaus DROP dnsbl * [Feature] Add support for dictionary in client compression * [Feature] Add support for fuzzy learn and unlearn from lua * [Feature] Add support for input encryption * [Feature] Add support of min_learns to neural net classifier * [Feature] Add termination callbacks for workers * [Feature] Add user-agent for rspamc * [Feature] Add utility to perform classifier tests * [Feature] Add zstd compression library * [Feature] Allow HTTPS requests in lua_http * [Feature] Allow conditions for pre and postfilters * [Feature] Allow custom functions for ratelimits * [Feature] Allow for excluding messages from AV scanning based on size * [Feature] Allow for getting worker stats from Lua * [Feature] Allow getting task UID from Lua * [Feature] Allow parsing of mailbox messages from the commandline * [Feature] Allow plugins to publish their lua API via rspamd_plugins * [Feature] Allow to compare other systems with Rspamd * [Feature] Allow to execute Lua scripts by controller * [Feature] Allow to have a function to set custom greylist message * [Feature] Allow to iterate over multiple tags * [Feature] Allow to pass extra data from plugins to log helper * [Feature] Allow to plan new periodics at different time * [Feature] Allow to reset hashes * [Feature] Allow to run rspamadm lua just as a lua interpreter * [Feature] Allow to store settings in redis * [Feature] Allow to update dynamic conf in Redis * [Feature] Allow to use dictionaries for compression * [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua * [Feature] Allow whitelisting by IP for greylisting plugin * [Feature] Antivirus: Support whitelists & pattern-matching sig names * [Feature] Backport pack/unpack routines from Lua 5.3 * [Feature] Check settings with equal priopities in alphabetical order * [Feature] Compress neural net in redis * [Feature] Consider more tags when doing WHITE_ON_WHITE rule * [Feature] Descriptive options for DMARC failure symbols * [Feature] Descriptive options for RBL symbols * [Feature] Enable configuration for monitored objects * [Feature] Execute on_load scripts with ev_base ready * [Feature] Fann scores now uses metadata from a message * [Feature] Implement FANN threaded learning * [Feature] Implement classifying for lua classifiers * [Feature] Implement finish scripts for worker processes * [Feature] Implement monitoring for DNS resources * [Feature] Implement real priorities for pre and post filters * [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one * [Feature] Module to push metadata/messages to redis pubsub * [Feature] Monitor RBL records * [Feature] Move fann_classifier to a separate plugin * [Feature] Normalize all ANN inputs * [Feature] Preliminary version of metric exporter module * [Feature] Preserve decompression context between tasks * [Feature] Ratelimit: Support dynamic bucket size/leak rate * [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves * [Feature] Remove symbols weights on composites processing * [Feature] Return symbol scores when getting resulting symbols * [Feature] Rework lua tcp module * [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries * [Feature] Rule to identify some X-PHP-Script forgeries * [Feature] Rules for scoring Google Message-ID fixes * [Feature] Send hashes values to reply * [Feature] Set expire for dmarc reports * [Feature] Stop using cymru zone as it is unstable * [Feature] Stop using of GLists for headers, improve performance * [Feature] Store `for` in task:get_received_headers * [Feature] Store `for` part in received headers * [Feature] Store enabled flag for webui session * [Feature] Store error messages in ring buffer * [Feature] Support compressed maps * [Feature] Support excluding selected users from ratelimits * [Feature] Support looking up NS records in lua_dns * [Feature] Support modern style SURBL configuration * [Feature] Support multiple hashes in delhash path * [Feature] Support new messages in rspamc * [Feature] Support requests without reads in lua_tcp * [Feature] Support setting task message from Lua * [Feature] Track visibility of HTML elements * [Feature] Try to add CRLF when checking DKIM * [Feature] Try to guess line endings when folding headers * [Feature] Try to improve normalization function for bayes * [Feature] Use FFI to optimize SA module * [Feature] Use length based arguments for redis, allow lua_text as arg * [Feature] Use more layers for fann and another normalization * [Feature] User-defined ratelimits * [Feature] Utility to convert fuzzy storage from sqlite to redis * [Feature] Yield DMARC_DNSFAIL on lookup failure * [Fix] Adopt fuzzy storage for flexible backends * [Fix] Allow plain IP addresses in Rspamd maps * [Fix] Another fix for brain-damaged hiredis * [Fix] Another fix for rdns write errors * [Fix] Another fix for rdns_make_request_full invocation * [Fix] Another fix in DKIM canonicalization * [Fix] Another memory leak plugged * [Fix] Another try to deal with posix idiotizm * [Fix] Another try to fix RDNS events processing logic * [Fix] Avoid double frees in HEAD requests * [Fix] Avoid extra symbols for RBLs * [Fix] Banish table.maxn from Lua parts * [Fix] Check for socket error before connection in lua_tcp * [Fix] Correctly propagate redis timeouts to Lua * [Fix] Do not add extra newline in MIME mode * [Fix] Do not be cheated by system hiredis * [Fix] Do not classify when a message has not enough tokens * [Fix] Do not crash on redis errors * [Fix] Do not distinguish NXDOMAIN and NOREC for monitored * [Fix] Do not replan retransmits if merely one server is defined * [Fix] Do not use headers to calculate messages digests * [Fix] Don't force action in replies module for authenticated users/local networks * [Fix] Explicitly ban default passwords in webui * [Fix] Finally fix ambiguity between parsed and resolved spf elts * [Fix] Fix 'decoded' value in task:get_header_full() * [Fix] Fix DKIM calculations * [Fix] Fix DKIM signing for messages with no newline at the end * [Fix] Fix DNS request in monitored * [Fix] Fix DNS write errors processing * [Fix] Fix HTTP methods other than GET and POST * [Fix] Fix PERMFAIL for v6/v4 ambiguities * [Fix] Fix absurdic scores for HFILTER_URL_ONLY * [Fix] Fix actions in rolling history * [Fix] Fix actrie patterns * [Fix] Fix applying of lua dynamic confg * [Fix] Fix autolearning errors and redis cache * [Fix] Fix bayes learn_condition * [Fix] Fix build with the recent OpenSSL * [Fix] Fix caching and compressed maps * [Fix] Fix check plain text part * [Fix] Fix crash on OpenBSD in `url_email_start` * [Fix] Fix double free in SPF * [Fix] Fix extraction of shingles from redis fuzzy storage * [Fix] Fix false sharing for symbols in the cache * [Fix] Fix float usage in util:get_time * [Fix] Fix folding algorithm to deal with empty tokens * [Fix] Fix format string * [Fix] Fix format string usage in controller errors handling * [Fix] Fix handling of '\0' in lua_tcp * [Fix] Fix handling of HTTP HEAD methods * [Fix] Fix hash creation * [Fix] Fix hiredis stupidity * [Fix] Fix implicit settings module settingsup * [Fix] Fix interaction with lua GC to avoid craches * [Fix] Fix ip_score module registration * [Fix] Fix issue with empty messages and dkim * [Fix] Fix issues with CGP helper * [Fix] Fix issues with the recent SPF changes * [Fix] Fix key name to load ANN correctly * [Fix] Fix lua tcp module by saving `do_read` in callback data * [Fix] Fix memory leak in client when using compression * [Fix] Fix min_learns option * [Fix] Fix on_finish scripts and async handlers * [Fix] Fix options for SPF dnsfail symbol * [Fix] Fix parsing includes and redirects in SPF * [Fix] Fix parsing of lua comments with empty lines * [Fix] Fix parsing of unquoted HTML attributes * [Fix] Fix periodic events and redis * [Fix] Fix processing of fuzzy learns from Lua * [Fix] Fix processing of redirect in SPF includes * [Fix] Fix processing of symbols when reject limit is reached * [Fix] Fix refcounts when map is specified by IP * [Fix] Fix rspamd{session} class in Lua API * [Fix] Fix setting ratelimit key for 'ip' bucket * [Fix] Fix some cases of TLD urls detector * [Fix] Fix statconvert tool * [Fix] Fix stats for backend-less classifiers * [Fix] Fix training script for fann_redis * [Fix] Fix variable in ann module * [Fix] Fix various errors in lua dynamic conf plugin * [Fix] Forget old ANN when max_usages is reached to avoid overtrain * [Fix] Further canonicalization fixes * [Fix] Further fixes for fann_redis prefixes * [Fix] Handle failures for inactive pooled connections * [Fix] Improve multimap info message * [Fix] More fixes in ANN loading * [Fix] More fixes to fann_redis * [Fix] More issues in fann_redis * [Fix] More spaces fix in DKIM signature * [Fix] Multiple fixes to asn script, add IPv6 support * [Fix] Multiple issues in fann_redis * [Fix] No greylist rejected messages * [Fix] One more attempt to fix lua_redis * [Fix] One more check for readdir... * [Fix] Params should be treated as a hash * [Fix] Plug memory leak in regexp desctructor * [Fix] Process headers only once * [Fix] Properly handle nil values in ratelimit plugin * [Fix] Really fix redis shingles check * [Fix] Remove fann with incorrect layers count * [Fix] Remove mentions of deleted include * [Fix] Remove some incompatible functions * [Fix] Settings: fix `authenticated` parameter (#886) * [Fix] Skip MX check for authenticated users and local networks * [Fix] Slightly fix ANN routines * [Fix] Stop caching records with DNS failures * [Fix] Treat all errors in redis_pool as fatal errors for a connection * [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule * [Fix] Try to avoid race condition when using rrd * [Fix] Try to reload redis scripts if they are missing * [Fix] Unbreak once_received skipping for local networks * [Fix] Unlock ANN on error * [Fix] Use memmove for overlapping regions * [Fix] Use real size instead of displayed for core limits * [Fix] Use the correct macro to get the size of control * [Fix] Various fixes for errors ringbuffer * [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails * [Fix] mid: fix map initialization * [Fix] mid: handle incorrect rgexps in the map * [Rework] Add extract training data function to fann_redis * [Rework] Add preliminary train tests * [Rework] Add redis storage feature to fann_redis * [Rework] Adopt fuzzy storage for abstract backend * [Rework] Adopt plugins * [Rework] First reiteration on fann scores * [Rework] Implement loading/invalidating * [Rework] Make lua_redis task agnostic * [Rework] Make rspamd protocol messages useful * [Rework] Massive removal of legacy code * [Rework] More cleanup actions * [Rework] Remove legacy code never used for classifiers * [Rework] Remove outdated and unused lua_session module * [Rework] Reorganize fuzzy backend structure * [Rework] Reorganize the internal backend structure * [Rework] Restore old fann_scores, move common parts * [Rework] Rework and simplify rbl plugin * [Rework] Rework parsing of DMARC records Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.4.0 * [CritFix] Add guards for inactive redis connections * [CritFix] Another fix for proxying files using rspamd_proxy * [CritFix] Cleanup inactive redis connections * [CritFix] Do not sometimes try to exec posfilters before classification * [CritFix] Fix application of IPv6 mask * [CritFix] Fix chunked encoding when reading messages * [CritFix] Fix file mode for rspamd_proxy * [CritFix] Fix hyperscan compilation on regexp change * [CritFix] Fix issue with finding of end of lines pointers * [CritFix] Fix iteration over headers array (introduced in 1.4) * [CritFix] Fix processing of learned tokens count for redis backend * [CritFix] Fix race condition in checking of cached maps * [CritFix] Fix workers scripts by sharing workers configs * [CritFix] Introduce raw content to text parts * [CritFix] Plug memory leak and potential memory corruption * [Feature] Adaptive ratelimits * [Feature] Add ASN -> rbldnsd script for asn.rspamd.com * [Feature] Add DMARC_NA symbol * [Feature] Add F-Prot support to antivirus module * [Feature] Add HTTP backend to metadata exporter * [Feature] Add Lua API module for monitored objects * [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols * [Feature] Add R_DKIM_PERMFAIL symbol * [Feature] Add R_SPF_PERMFAIL symbol * [Feature] Add Sophos antivirus support * [Feature] Add ZSTD compression to Lua API * [Feature] Add `mid` Lua module * [Feature] Add `one_param` flag for metric symbols * [Feature] Add a generic lua classifier * [Feature] Add a very basic interface to access workers data from on_load * [Feature] Add ability to delete a hash by its data to fuzzy_check plugin * [Feature] Add ability to enable/disable symbols via dynamic_conf * [Feature] Add ability to lookup settings by key * [Feature] Add common way to disable Lua modules * [Feature] Add compression support to rspamd client * [Feature] Add condition to do antiviral check * [Feature] Add configuration for lua classifiers * [Feature] Add configuration knobs for the errors circular buffer * [Feature] Add decompression support in rspamd client * [Feature] Add errors exporter to the controller * [Feature] Add expected value for monitored DNS resources * [Feature] Add exporter from error ringbuf to ucl * [Feature] Add extended version for fann creation function * [Feature] Add ffi friendly version of process_regexp function * [Feature] Add frequency and time display to webui * [Feature] Add fuzzy_delhash command to rspamc client * [Feature] Add implementation of redis connections pool * [Feature] Add latency and offline time monitoring * [Feature] Add learning support for lua classifiers * [Feature] Add max-size and timeout options to CGP helper * [Feature] Add method to enable/disable symbols in config * [Feature] Add methods to get metric's actions and symbols from Lua * [Feature] Add mmap support to lua_text * [Feature] Add monitored object for surbl plugin * [Feature] Add more exceptions to surbl whitelist * [Feature] Add more meta-tokens to bayes * [Feature] Add neural net classifier to fann_scores module * [Feature] Add neural net serialization/deserialization * [Feature] Add new dynamic conf module * [Feature] Add periodic events support for lua_config * [Feature] Add plugin to check MX'es for the sender's domain * [Feature] Add preliminary monitored module * [Feature] Add preliminary support of dynamic conf updates in Redis * [Feature] Add preliminary version of clamav plugin * [Feature] Add redis cache to asn module * [Feature] Add replies compression * [Feature] Add spamhaus DROP dnsbl * [Feature] Add support for dictionary in client compression * [Feature] Add support for fuzzy learn and unlearn from lua * [Feature] Add support for input encryption * [Feature] Add support of min_learns to neural net classifier * [Feature] Add termination callbacks for workers * [Feature] Add user-agent for rspamc * [Feature] Add utility to perform classifier tests * [Feature] Add zstd compression library * [Feature] Allow HTTPS requests in lua_http * [Feature] Allow conditions for pre and postfilters * [Feature] Allow custom functions for ratelimits * [Feature] Allow for excluding messages from AV scanning based on size * [Feature] Allow for getting worker stats from Lua * [Feature] Allow getting task UID from Lua * [Feature] Allow parsing of mailbox messages from the commandline * [Feature] Allow plugins to publish their lua API via rspamd_plugins * [Feature] Allow to compare other systems with Rspamd * [Feature] Allow to execute Lua scripts by controller * [Feature] Allow to have a function to set custom greylist message * [Feature] Allow to iterate over multiple tags * [Feature] Allow to pass extra data from plugins to log helper * [Feature] Allow to plan new periodics at different time * [Feature] Allow to reset hashes * [Feature] Allow to run rspamadm lua just as a lua interpreter * [Feature] Allow to store settings in redis * [Feature] Allow to update dynamic conf in Redis * [Feature] Allow to use dictionaries for compression * [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua * [Feature] Allow whitelisting by IP for greylisting plugin * [Feature] Antivirus: Support whitelists & pattern-matching sig names * [Feature] Backport pack/unpack routines from Lua 5.3 * [Feature] Check settings with equal priopities in alphabetical order * [Feature] Compress neural net in redis * [Feature] Consider more tags when doing WHITE_ON_WHITE rule * [Feature] Descriptive options for DMARC failure symbols * [Feature] Descriptive options for RBL symbols * [Feature] Enable configuration for monitored objects * [Feature] Execute on_load scripts with ev_base ready * [Feature] Fann scores now uses metadata from a message * [Feature] Implement FANN threaded learning * [Feature] Implement classifying for lua classifiers * [Feature] Implement finish scripts for worker processes * [Feature] Implement monitoring for DNS resources * [Feature] Implement real priorities for pre and post filters * [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one * [Feature] Module to push metadata/messages to redis pubsub * [Feature] Monitor RBL records * [Feature] Move fann_classifier to a separate plugin * [Feature] Normalize all ANN inputs * [Feature] Preliminary version of metric exporter module * [Feature] Preserve decompression context between tasks * [Feature] Ratelimit: Support dynamic bucket size/leak rate * [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves * [Feature] Remove symbols weights on composites processing * [Feature] Return symbol scores when getting resulting symbols * [Feature] Rework lua tcp module * [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries * [Feature] Rule to identify some X-PHP-Script forgeries * [Feature] Rules for scoring Google Message-ID fixes * [Feature] Send hashes values to reply * [Feature] Set expire for dmarc reports * [Feature] Stop using cymru zone as it is unstable * [Feature] Stop using of GLists for headers, improve performance * [Feature] Store `for` in task:get_received_headers * [Feature] Store `for` part in received headers * [Feature] Store enabled flag for webui session * [Feature] Store error messages in ring buffer * [Feature] Support compressed maps * [Feature] Support excluding selected users from ratelimits * [Feature] Support looking up NS records in lua_dns * [Feature] Support modern style SURBL configuration * [Feature] Support multiple hashes in delhash path * [Feature] Support new messages in rspamc * [Feature] Support requests without reads in lua_tcp * [Feature] Support setting task message from Lua * [Feature] Track visibility of HTML elements * [Feature] Try to add CRLF when checking DKIM * [Feature] Try to guess line endings when folding headers * [Feature] Try to improve normalization function for bayes * [Feature] Use FFI to optimize SA module * [Feature] Use length based arguments for redis, allow lua_text as arg * [Feature] Use more layers for fann and another normalization * [Feature] User-defined ratelimits * [Feature] Utility to convert fuzzy storage from sqlite to redis * [Feature] Yield DMARC_DNSFAIL on lookup failure * [Fix] Adopt fuzzy storage for flexible backends * [Fix] Allow plain IP addresses in Rspamd maps * [Fix] Another fix for brain-damaged hiredis * [Fix] Another fix for rdns write errors * [Fix] Another fix for rdns_make_request_full invocation * [Fix] Another fix in DKIM canonicalization * [Fix] Another memory leak plugged * [Fix] Another try to deal with posix idiotizm * [Fix] Another try to fix RDNS events processing logic * [Fix] Avoid double frees in HEAD requests * [Fix] Avoid extra symbols for RBLs * [Fix] Banish table.maxn from Lua parts * [Fix] Check for socket error before connection in lua_tcp * [Fix] Correctly propagate redis timeouts to Lua * [Fix] Do not add extra newline in MIME mode * [Fix] Do not be cheated by system hiredis * [Fix] Do not classify when a message has not enough tokens * [Fix] Do not crash on redis errors * [Fix] Do not distinguish NXDOMAIN and NOREC for monitored * [Fix] Do not replan retransmits if merely one server is defined * [Fix] Do not use headers to calculate messages digests * [Fix] Don't force action in replies module for authenticated users/local networks * [Fix] Explicitly ban default passwords in webui * [Fix] Finally fix ambiguity between parsed and resolved spf elts * [Fix] Fix 'decoded' value in task:get_header_full() * [Fix] Fix DKIM calculations * [Fix] Fix DKIM signing for messages with no newline at the end * [Fix] Fix DNS request in monitored * [Fix] Fix DNS write errors processing * [Fix] Fix HTTP methods other than GET and POST * [Fix] Fix PERMFAIL for v6/v4 ambiguities * [Fix] Fix absurdic scores for HFILTER_URL_ONLY * [Fix] Fix actions in rolling history * [Fix] Fix actrie patterns * [Fix] Fix applying of lua dynamic confg * [Fix] Fix autolearning errors and redis cache * [Fix] Fix bayes learn_condition * [Fix] Fix build with the recent OpenSSL * [Fix] Fix caching and compressed maps * [Fix] Fix check plain text part * [Fix] Fix crash on OpenBSD in `url_email_start` * [Fix] Fix double free in SPF * [Fix] Fix extraction of shingles from redis fuzzy storage * [Fix] Fix false sharing for symbols in the cache * [Fix] Fix float usage in util:get_time * [Fix] Fix folding algorithm to deal with empty tokens * [Fix] Fix format string * [Fix] Fix format string usage in controller errors handling * [Fix] Fix handling of '\0' in lua_tcp * [Fix] Fix handling of HTTP HEAD methods * [Fix] Fix hash creation * [Fix] Fix hiredis stupidity * [Fix] Fix implicit settings module settingsup * [Fix] Fix interaction with lua GC to avoid craches * [Fix] Fix ip_score module registration * [Fix] Fix issue with empty messages and dkim * [Fix] Fix issues with CGP helper * [Fix] Fix issues with the recent SPF changes * [Fix] Fix key name to load ANN correctly * [Fix] Fix lua tcp module by saving `do_read` in callback data * [Fix] Fix memory leak in client when using compression * [Fix] Fix min_learns option * [Fix] Fix on_finish scripts and async handlers * [Fix] Fix options for SPF dnsfail symbol * [Fix] Fix parsing includes and redirects in SPF * [Fix] Fix parsing of lua comments with empty lines * [Fix] Fix parsing of unquoted HTML attributes * [Fix] Fix periodic events and redis * [Fix] Fix processing of fuzzy learns from Lua * [Fix] Fix processing of redirect in SPF includes * [Fix] Fix processing of symbols when reject limit is reached * [Fix] Fix refcounts when map is specified by IP * [Fix] Fix rspamd{session} class in Lua API * [Fix] Fix setting ratelimit key for 'ip' bucket * [Fix] Fix some cases of TLD urls detector * [Fix] Fix statconvert tool * [Fix] Fix stats for backend-less classifiers * [Fix] Fix training script for fann_redis * [Fix] Fix variable in ann module * [Fix] Fix various errors in lua dynamic conf plugin * [Fix] Forget old ANN when max_usages is reached to avoid overtrain * [Fix] Further canonicalization fixes * [Fix] Further fixes for fann_redis prefixes * [Fix] Handle failures for inactive pooled connections * [Fix] Improve multimap info message * [Fix] More fixes in ANN loading * [Fix] More fixes to fann_redis * [Fix] More issues in fann_redis * [Fix] More spaces fix in DKIM signature * [Fix] Multiple fixes to asn script, add IPv6 support * [Fix] Multiple issues in fann_redis * [Fix] No greylist rejected messages * [Fix] One more attempt to fix lua_redis * [Fix] One more check for readdir... * [Fix] Params should be treated as a hash * [Fix] Plug memory leak in regexp desctructor * [Fix] Process headers only once * [Fix] Properly handle nil values in ratelimit plugin * [Fix] Really fix redis shingles check * [Fix] Remove fann with incorrect layers count * [Fix] Remove mentions of deleted include * [Fix] Remove some incompatible functions * [Fix] Settings: fix `authenticated` parameter (#886) * [Fix] Skip MX check for authenticated users and local networks * [Fix] Slightly fix ANN routines * [Fix] Stop caching records with DNS failures * [Fix] Treat all errors in redis_pool as fatal errors for a connection * [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule * [Fix] Try to avoid race condition when using rrd * [Fix] Try to reload redis scripts if they are missing * [Fix] Unbreak once_received skipping for local networks * [Fix] Unlock ANN on error * [Fix] Use memmove for overlapping regions * [Fix] Use real size instead of displayed for core limits * [Fix] Use the correct macro to get the size of control * [Fix] Various fixes for errors ringbuffer * [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails * [Fix] mid: fix map initialization * [Fix] mid: handle incorrect rgexps in the map * [Rework] Add extract training data function to fann_redis * [Rework] Add preliminary train tests * [Rework] Add redis storage feature to fann_redis * [Rework] Adopt fuzzy storage for abstract backend * [Rework] Adopt plugins * [Rework] First reiteration on fann scores * [Rework] Implement loading/invalidating * [Rework] Make lua_redis task agnostic * [Rework] Make rspamd protocol messages useful * [Rework] Massive removal of legacy code * [Rework] More cleanup actions * [Rework] Remove legacy code never used for classifiers * [Rework] Remove outdated and unused lua_session module * [Rework] Reorganize fuzzy backend structure * [Rework] Reorganize the internal backend structure * [Rework] Restore old fann_scores, move common parts * [Rework] Rework and simplify rbl plugin * [Rework] Rework parsing of DMARC records Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years ago
Release 1.3.4 * [Feature] ASN module; support matching ASN/country in multimap * [Feature] Add SPF method in spf return result * [Feature] Add Yandex and Mail.ru forwarding rules * [Feature] Add mempool maps in multimap * [Feature] Add rule for identifying mail sent by eval()'d PHP code * [Feature] Add support of stub DNSSEC resolver to rdns * [Feature] Add task:get_digest method * [Feature] Allow for more fine-grained scoring for ip_score * [Feature] Allow to get digest of a mime part from lua * [Feature] Allow to print message digest in logs * [Feature] Fold DKIM-Signature header * [Feature] Implement encrypted logs * [Feature] Log URLs encrypted if we have log encryption pubkey * [Feature] Pass authenticated bit to lua * [Feature] Read redis backend statistics configuration from global section * [Feature] Show the exact value matched for multima symbols * [Feature] Store task checksum * [Fix] Avoid setting limits when required elements are missing * [Fix] DMARC: Fix alignment checking for subdomains * [Fix] DMARC: deal with missing and spurious spaces * [Fix] Defer insertion of results in ip_score to avoid skewing stats * [Fix] Disable DMARC for local/authorized mail * [Fix] Fix handling of proxied headers in controller * [Fix] Fix hex printing of strings * [Fix] Fix issue with spaces in maps * [Fix] Fix parsing of forwarded IP * [Fix] Fix reload in some plugins and workers * [Fix] Fix reloading on SIGHUP * [Fix] Fix some border cases for DKIM canonicalization * [Fix] Fix url maps * [Fix] Make dnssec configurable option disabled by default for now * [Fix] rspamadm statconvert: force db to be a string * [Fix] rspamadm statconvert: use db/password for learn cache * [Rework] Rework flags in rspamd logger Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.3.4 * [Feature] ASN module; support matching ASN/country in multimap * [Feature] Add SPF method in spf return result * [Feature] Add Yandex and Mail.ru forwarding rules * [Feature] Add mempool maps in multimap * [Feature] Add rule for identifying mail sent by eval()'d PHP code * [Feature] Add support of stub DNSSEC resolver to rdns * [Feature] Add task:get_digest method * [Feature] Allow for more fine-grained scoring for ip_score * [Feature] Allow to get digest of a mime part from lua * [Feature] Allow to print message digest in logs * [Feature] Fold DKIM-Signature header * [Feature] Implement encrypted logs * [Feature] Log URLs encrypted if we have log encryption pubkey * [Feature] Pass authenticated bit to lua * [Feature] Read redis backend statistics configuration from global section * [Feature] Show the exact value matched for multima symbols * [Feature] Store task checksum * [Fix] Avoid setting limits when required elements are missing * [Fix] DMARC: Fix alignment checking for subdomains * [Fix] DMARC: deal with missing and spurious spaces * [Fix] Defer insertion of results in ip_score to avoid skewing stats * [Fix] Disable DMARC for local/authorized mail * [Fix] Fix handling of proxied headers in controller * [Fix] Fix hex printing of strings * [Fix] Fix issue with spaces in maps * [Fix] Fix parsing of forwarded IP * [Fix] Fix reload in some plugins and workers * [Fix] Fix reloading on SIGHUP * [Fix] Fix some border cases for DKIM canonicalization * [Fix] Fix url maps * [Fix] Make dnssec configurable option disabled by default for now * [Fix] rspamadm statconvert: force db to be a string * [Fix] rspamadm statconvert: use db/password for learn cache * [Rework] Rework flags in rspamd logger Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.3.2 * [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL * [Feature] Add correlations report in fuzzy stats * [Feature] Add experimental CGP integration * [Feature] Add method to get urls length in a text part * [Feature] Add new methods to lua_html to access HTML tags * [Feature] Allow all types of symbols to be added via __newindex method * [Feature] Allow to create settings for authenticated users * [Feature] Allow to get block content for HTML tags * [Feature] Improve DNS failures when dealing with SPF * [Feature] Properly implement R_WHITE_ON_WHITE rule * [Feature] Remove old ugly rules * [Feature] Rspamc can now add dkim signature in mime mode * [Feature] Store content length for HTML tags * [Feature] Support reacher set of HTML colors * [Feature] Try to avoid FP for low contrast fonts detection * [Fix] Add missing HTML colors * [Fix] Add spaces to dkim signature to allow folding * [Fix] Avoid returning NaN as score on scan * [Fix] Decode entitles in href parts * [Fix] Do not cache SPF records with DNS errors * [Fix] Do not crash on cyclic depends * [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed * [Fix] Do not set absent hostname to "unknown" * [Fix] Do not stress redis with KEYS command (#791) * [Fix] Fix DMARC_BAD_POLICY symbol * [Fix] Fix HFILTER_URL module * [Fix] Fix HFILTER_URL_ONELINE rule * [Fix] Fix buffering in CGP integration * [Fix] Fix colors propagation from parent nodes * [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey * [Fix] Fix dependencies id sanity check * [Fix] Fix folding for semicolon separated tokens * [Fix] Fix largest possible TLD behaviour * [Fix] Fix last token folding * [Fix] Fix length calculations in white on white rule * [Fix] Fix multiple request headers structure * [Fix] Fix multiple values headers freeing * [Fix] Fix parsing of background color * [Fix] Fix printing from field in log_urls * [Fix] Fix processing of last element of DMARC policies * [Fix] Further fixes for HTML colors * [Fix] Further fixes for multiple values headers * [Fix] Further fixes for white on white rule * [Fix] Further fixes in HTML tags parsing * [Fix] Ignore content type/subtype case * [Fix] Increase score of R_WHITE_ON_WHITE * [Fix] Parse CGP envelope data * [Fix] Propagate colors in HTML * [Fix] Restore multiple values headers in protocol * [Fix] Restore multiple values in headers processing * [Fix] Some more changes to tag's content length calculations * [Fix] Some more fixes for low contrast fonts detector * [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..]) * [Fix] Trigger HTML_SHORT_LINK_IMG on any external image * [Fix] rspamd_stats: remove deprecated defined(@array) Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.3.2 * [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL * [Feature] Add correlations report in fuzzy stats * [Feature] Add experimental CGP integration * [Feature] Add method to get urls length in a text part * [Feature] Add new methods to lua_html to access HTML tags * [Feature] Allow all types of symbols to be added via __newindex method * [Feature] Allow to create settings for authenticated users * [Feature] Allow to get block content for HTML tags * [Feature] Improve DNS failures when dealing with SPF * [Feature] Properly implement R_WHITE_ON_WHITE rule * [Feature] Remove old ugly rules * [Feature] Rspamc can now add dkim signature in mime mode * [Feature] Store content length for HTML tags * [Feature] Support reacher set of HTML colors * [Feature] Try to avoid FP for low contrast fonts detection * [Fix] Add missing HTML colors * [Fix] Add spaces to dkim signature to allow folding * [Fix] Avoid returning NaN as score on scan * [Fix] Decode entitles in href parts * [Fix] Do not cache SPF records with DNS errors * [Fix] Do not crash on cyclic depends * [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed * [Fix] Do not set absent hostname to "unknown" * [Fix] Do not stress redis with KEYS command (#791) * [Fix] Fix DMARC_BAD_POLICY symbol * [Fix] Fix HFILTER_URL module * [Fix] Fix HFILTER_URL_ONELINE rule * [Fix] Fix buffering in CGP integration * [Fix] Fix colors propagation from parent nodes * [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey * [Fix] Fix dependencies id sanity check * [Fix] Fix folding for semicolon separated tokens * [Fix] Fix largest possible TLD behaviour * [Fix] Fix last token folding * [Fix] Fix length calculations in white on white rule * [Fix] Fix multiple request headers structure * [Fix] Fix multiple values headers freeing * [Fix] Fix parsing of background color * [Fix] Fix printing from field in log_urls * [Fix] Fix processing of last element of DMARC policies * [Fix] Further fixes for HTML colors * [Fix] Further fixes for multiple values headers * [Fix] Further fixes for white on white rule * [Fix] Further fixes in HTML tags parsing * [Fix] Ignore content type/subtype case * [Fix] Increase score of R_WHITE_ON_WHITE * [Fix] Parse CGP envelope data * [Fix] Propagate colors in HTML * [Fix] Restore multiple values headers in protocol * [Fix] Restore multiple values in headers processing * [Fix] Some more changes to tag's content length calculations * [Fix] Some more fixes for low contrast fonts detector * [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..]) * [Fix] Trigger HTML_SHORT_LINK_IMG on any external image * [Fix] rspamd_stats: remove deprecated defined(@array) Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.3.1 * [CritFix] Fix catena passwords validation * [CritFix] Fix crash when the first received is faked * [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid * [Feature] Allow for matching hostnames in multimap (#773) * [Feature] Allow for setting action based on DMARC disposition * [Feature] Allow limiting of the inbound message size * [Feature] Allow maps with multiple symbols and scores * [Feature] Allow regexps in the emails maps * [Feature] Allow to register metric symbols from multimap * [Feature] Allow to reset redis tokens instead of appendig values * [Feature] Allow to store strings in radix maps * [Feature] Check UTF validity when there are utf regexps in a map * [Feature] Correctly work when there is no hard reject action * [Feature] Implement dependencies for maps * [Fix] Another effort to unbreak sqlite locking * [Fix] Avoid crash when closing mmapped file * [Fix] Do not break history on NaN in required score * [Fix] Ensure that hyperscan cache written is written properly * [Fix] Filter NaN from scores in history * [Fix] Fix DNSBL maps * [Fix] Fix another locking issue in sqlite * [Fix] Fix another locking issue with mapped files * [Fix] Fix deadlock in mmaped file stats * [Fix] Fix dependencies in multimap plugin * [Fix] Fix emails module configuration * [Fix] Fix greylist plugin (#755) * [Fix] Fix greylisting plugin variable usage * [Fix] Fix installed permissions for rspamd_stats * [Fix] Fix locking in mmapped statistics * [Fix] Fix paths in tests * [Fix] Fix prefilter mode for multimap * [Fix] Forgot to commit leftover changes * [Fix] Really fix local.d includes * [Fix] Restore selective greylisting behaviour * [Fix] Set max size on per connection basis * [Fix] Use temporary storage for hyperscan cache * [Rework] Remove systemd socket activation Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.3.1 * [CritFix] Fix catena passwords validation * [CritFix] Fix crash when the first received is faked * [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid * [Feature] Allow for matching hostnames in multimap (#773) * [Feature] Allow for setting action based on DMARC disposition * [Feature] Allow limiting of the inbound message size * [Feature] Allow maps with multiple symbols and scores * [Feature] Allow regexps in the emails maps * [Feature] Allow to register metric symbols from multimap * [Feature] Allow to reset redis tokens instead of appendig values * [Feature] Allow to store strings in radix maps * [Feature] Check UTF validity when there are utf regexps in a map * [Feature] Correctly work when there is no hard reject action * [Feature] Implement dependencies for maps * [Fix] Another effort to unbreak sqlite locking * [Fix] Avoid crash when closing mmapped file * [Fix] Do not break history on NaN in required score * [Fix] Ensure that hyperscan cache written is written properly * [Fix] Filter NaN from scores in history * [Fix] Fix DNSBL maps * [Fix] Fix another locking issue in sqlite * [Fix] Fix another locking issue with mapped files * [Fix] Fix deadlock in mmaped file stats * [Fix] Fix dependencies in multimap plugin * [Fix] Fix emails module configuration * [Fix] Fix greylist plugin (#755) * [Fix] Fix greylisting plugin variable usage * [Fix] Fix installed permissions for rspamd_stats * [Fix] Fix locking in mmapped statistics * [Fix] Fix paths in tests * [Fix] Fix prefilter mode for multimap * [Fix] Forgot to commit leftover changes * [Fix] Really fix local.d includes * [Fix] Restore selective greylisting behaviour * [Fix] Set max size on per connection basis * [Fix] Use temporary storage for hyperscan cache * [Rework] Remove systemd socket activation Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.2.4 * Implement new multipattern matcher that uses hyperscan if possible * Use mutlipattern for lua_trie code * Add utility methods for multipattern * Use multipattern in url matcher * Add escape functions for hyperscan * Allow to optimize lua -> C transition by flattening table args * Optimize hot paths in SA plugin * Optimize rspamd_re_cache_type_from_string * Allow empty tries * Fix extraction of URLs from Subject * Allow to have different flags for different patterns in multipattern * Add common directory for hyperscan cache to config * Implement caching for hyperscan multipattern * Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev) * Allow multipattern scans to be nested for the case of hyperscan * Simplify SURBL redirector search code and avoid ac_trie * Add two way substring search algorithm * Avoid acism usage to find gtube pattern * Fix processing of empty headers * Allow to disable pthread mutexes on broken platforms * Make web interface not send password in query strings (#585) by @fatalbanana * Add maximum delay to ratelimit module * Backport fix for empty files inclusion from libucl * Fix settings id setup * Add min_learns option to classifiers * Use more clever to utf8 conversion strategy * Fix disabling of virtual symbols in the settings * Rework settings to work properly in metric-less configuration * Set the default limit for classifier * Fix ttl based expiration from LRU cache * Rework DKIM module to use OpenSSL for digests * Fix mailto urls parsing with hyperscan * Do not set obscured flag for urls starting with spaces * Fix crash on redis learn * Fix ratelimit ctime setting Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.2.4 * Implement new multipattern matcher that uses hyperscan if possible * Use mutlipattern for lua_trie code * Add utility methods for multipattern * Use multipattern in url matcher * Add escape functions for hyperscan * Allow to optimize lua -> C transition by flattening table args * Optimize hot paths in SA plugin * Optimize rspamd_re_cache_type_from_string * Allow empty tries * Fix extraction of URLs from Subject * Allow to have different flags for different patterns in multipattern * Add common directory for hyperscan cache to config * Implement caching for hyperscan multipattern * Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev) * Allow multipattern scans to be nested for the case of hyperscan * Simplify SURBL redirector search code and avoid ac_trie * Add two way substring search algorithm * Avoid acism usage to find gtube pattern * Fix processing of empty headers * Allow to disable pthread mutexes on broken platforms * Make web interface not send password in query strings (#585) by @fatalbanana * Add maximum delay to ratelimit module * Backport fix for empty files inclusion from libucl * Fix settings id setup * Add min_learns option to classifiers * Use more clever to utf8 conversion strategy * Fix disabling of virtual symbols in the settings * Rework settings to work properly in metric-less configuration * Set the default limit for classifier * Fix ttl based expiration from LRU cache * Rework DKIM module to use OpenSSL for digests * Fix mailto urls parsing with hyperscan * Do not set obscured flag for urls starting with spaces * Fix crash on redis learn * Fix ratelimit ctime setting Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.2.3 * New DCC module (by @smfreegard) * Rework whitelist module: - Now we check different elements for different checks - MIME from for DMARC - DKIM signature domain for DKIM - SMTP from or HELO for SPF * Fix regexps results combination (*critical*) * Fix issue with expressions processing (*critical*) * Optimize strlcpy for aligned input * Add support of half-closed connection in lua_tcp * Allow to print compact json in client * Save required score in history (#581) * Allow to attach file descriptors to control commands * Allow to send descriptors from workers to main * Allow to attach fd when broadcasting to workers * Implement log pipe feature for rspamd logs analysis * Add `log_helper` worker * Add `URIBL_SBL_CSS` (by @smfreegard) * Add worker scripts functionality * Add on load hooks for rspamd_config * Add lua scripts for log_helper worker * Add generic maillist detector (#584) * Implement FANN autolearn using log_helper worker * Rework metrics configuration to allow includes * Change default value of forced removal in composite rules * Allow to use assembly version of blake2b on x86 cpu * Use less precise (but faster) clock if possible * Insert redirected URL to the urls list * Allow to get and set callback data for rspamd symbols * Add binary heap implementation * Use binary heap for expire algorithms in the hash * Use `least frequent used` expiration strategy * Allow to get mime headers from a task * Add support for mime headers in `regexp` module * Update Exim patches (by @fatalbanana) * Allow building rspamd with jemalloc * Save multipart boundaries * SA plugin changes: - Properly handle MIME headers - Fix eval:check_for_missing_to_header rule - Implement SA compatible body regexps - Use sabody rules in SA plugin * LUA API changes: - Add util.get_ticks function - Add util.stat function - Add task:get_symbols_numeric method - Add method to get number of symbols in the cache - Add lua methods to get redirected urls - Allow to get callbacks for lua symbols - Add config:set_symbol_callback function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.2.3 * New DCC module (by @smfreegard) * Rework whitelist module: - Now we check different elements for different checks - MIME from for DMARC - DKIM signature domain for DKIM - SMTP from or HELO for SPF * Fix regexps results combination (*critical*) * Fix issue with expressions processing (*critical*) * Optimize strlcpy for aligned input * Add support of half-closed connection in lua_tcp * Allow to print compact json in client * Save required score in history (#581) * Allow to attach file descriptors to control commands * Allow to send descriptors from workers to main * Allow to attach fd when broadcasting to workers * Implement log pipe feature for rspamd logs analysis * Add `log_helper` worker * Add `URIBL_SBL_CSS` (by @smfreegard) * Add worker scripts functionality * Add on load hooks for rspamd_config * Add lua scripts for log_helper worker * Add generic maillist detector (#584) * Implement FANN autolearn using log_helper worker * Rework metrics configuration to allow includes * Change default value of forced removal in composite rules * Allow to use assembly version of blake2b on x86 cpu * Use less precise (but faster) clock if possible * Insert redirected URL to the urls list * Allow to get and set callback data for rspamd symbols * Add binary heap implementation * Use binary heap for expire algorithms in the hash * Use `least frequent used` expiration strategy * Allow to get mime headers from a task * Add support for mime headers in `regexp` module * Update Exim patches (by @fatalbanana) * Allow building rspamd with jemalloc * Save multipart boundaries * SA plugin changes: - Properly handle MIME headers - Fix eval:check_for_missing_to_header rule - Implement SA compatible body regexps - Use sabody rules in SA plugin * LUA API changes: - Add util.get_ticks function - Add util.stat function - Add task:get_symbols_numeric method - Add method to get number of symbols in the cache - Add lua methods to get redirected urls - Allow to get callbacks for lua symbols - Add config:set_symbol_callback function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.2.0 * New dynamic updates plugin * Regular expressions map support * Faster radix trie algorithm * Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes) * PCRE2 support * Allow quoted and slashed keys in map * Add proper support of DNS resolvers balancing (#552) * Rework includes and configuration system for better local changes support * New keypairs framework for signing and encryption * Added support for dynamic modules and workers * Allow to dump configuration with help comments * Rework once_received module - Fix priority for `good_hosts` - If a good host has been found do not add once_received symbols - Fix priorities for strict once_received - Add ability to whitelist IP addresses * Implement support of signed maps for HTTP and file maps * Add command to sync fuzzy storage (#533) * Rework system of symbols and actions registration It is possible now to use priorities when adding symbols to metrics and override scores for symbols with lower priority with the scores with high priority. * Add auth support and db selection for redis stats * Improve composite rules application * Add ignore_received option * Fix critical issue with inconsistent resorting * Fix `all` in spf redirects * Add punycoded versions for IDN domains (#554) * Improve sorting order for symbols cache * Add lockless logging for processes management * Allow to specify flags for metric symbols * Load images height and width from style attribute (#538) * Override DNS requests limits for SPF and DKIM * Fix resetting symbols to their default values in WebUI * Improve configuration agility for redis stats * Allow to set db and password for redis in stat_convert * Import the latest libucl * LUA API changes: - Add rspamd_version function to LUA API - Add lua_cryptobox module - Add lua_map module - Add task:set_metric_action lua API method - Fix race condition in lua_tcp module - Fix a lot of issues in lua_redis module - Rework and abstract lua maps API - Add util.strlen_utf8 lua function - Add lua functions for caseless comparison - Allow optional symbols registration - Add config:add_map table form method, add regexp maps - Add task:has_urls method - Add task:has_flag method - Add html tags methods to lua_html - Add task:get_dns_req * Plugins changes: - Add support for WLBLEval SA plugin - Use caseless comparison in SA and DMARC plugins - Allow SA plugin to set scores for rspamd symbols - Add regexp maps support to multimap - Allow filenames match in multimap - Add more filters for the existing map types - Fix html images rules to reduce FP rates * New rules: - LONG_SUBJ - too long subject - MIME_BAD_ATTACHMENT - bad attachement type - RDNS_NONE - no reverse DNS record for sender's IP - Fix MISSING_MIMEOLE rule for modern OE * Many other bugfixes, memory leaks plugs thanks to: - Coverity scan - New gcc-6 warnings - valgrind manual iterations * Documentation improvements: - FAQ list: https://rspamd.com/doc/faq.html - Reworked quick start guide - Added documentation for all active modules * Other changes: - Dropped Ubuntu Vivid support - Added Ubuntu Xenial support - Rework build system for rspamd and rmilter Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.2.0 * New dynamic updates plugin * Regular expressions map support * Faster radix trie algorithm * Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes) * PCRE2 support * Allow quoted and slashed keys in map * Add proper support of DNS resolvers balancing (#552) * Rework includes and configuration system for better local changes support * New keypairs framework for signing and encryption * Added support for dynamic modules and workers * Allow to dump configuration with help comments * Rework once_received module - Fix priority for `good_hosts` - If a good host has been found do not add once_received symbols - Fix priorities for strict once_received - Add ability to whitelist IP addresses * Implement support of signed maps for HTTP and file maps * Add command to sync fuzzy storage (#533) * Rework system of symbols and actions registration It is possible now to use priorities when adding symbols to metrics and override scores for symbols with lower priority with the scores with high priority. * Add auth support and db selection for redis stats * Improve composite rules application * Add ignore_received option * Fix critical issue with inconsistent resorting * Fix `all` in spf redirects * Add punycoded versions for IDN domains (#554) * Improve sorting order for symbols cache * Add lockless logging for processes management * Allow to specify flags for metric symbols * Load images height and width from style attribute (#538) * Override DNS requests limits for SPF and DKIM * Fix resetting symbols to their default values in WebUI * Improve configuration agility for redis stats * Allow to set db and password for redis in stat_convert * Import the latest libucl * LUA API changes: - Add rspamd_version function to LUA API - Add lua_cryptobox module - Add lua_map module - Add task:set_metric_action lua API method - Fix race condition in lua_tcp module - Fix a lot of issues in lua_redis module - Rework and abstract lua maps API - Add util.strlen_utf8 lua function - Add lua functions for caseless comparison - Allow optional symbols registration - Add config:add_map table form method, add regexp maps - Add task:has_urls method - Add task:has_flag method - Add html tags methods to lua_html - Add task:get_dns_req * Plugins changes: - Add support for WLBLEval SA plugin - Use caseless comparison in SA and DMARC plugins - Allow SA plugin to set scores for rspamd symbols - Add regexp maps support to multimap - Allow filenames match in multimap - Add more filters for the existing map types - Fix html images rules to reduce FP rates * New rules: - LONG_SUBJ - too long subject - MIME_BAD_ATTACHMENT - bad attachement type - RDNS_NONE - no reverse DNS record for sender's IP - Fix MISSING_MIMEOLE rule for modern OE * Many other bugfixes, memory leaks plugs thanks to: - Coverity scan - New gcc-6 warnings - valgrind manual iterations * Documentation improvements: - FAQ list: https://rspamd.com/doc/faq.html - Reworked quick start guide - Added documentation for all active modules * Other changes: - Dropped Ubuntu Vivid support - Added Ubuntu Xenial support - Rework build system for rspamd and rmilter Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.1.2 * Fix stat_cache closing * Add checkpoints to sqlite3 learn cache * Do not recompile lua generated headers all the time * Increase number of messages learned * Fix issues with dual stack and hfilter * Disable MID checks for hfilter by default * Fix cache definitions in multiple classifier and no type * Don't crash if learn cache failed to initialize * Fix googlegroups support in maillist plugin * Rework flags LUA API: - Allow to check for a specific flag - Add `learn_spam`, `learn_ham` and `broken_headers` flags - Unify internal functions * Add `BROKEN_HEADERS` rule * Add support for forged confirmation headers (by @AdUser) * Allow `any`, `mime` and `smtp` for get_from/get_recipients * Add mime types checking plugin * Add rule to detect spammers attempts to cheat mime parsing * Rework parsing of IP addresses in configuration (better IPv6 support) * Add `util.parse_mail_address` function to LUA API * Add lua sqlite3 module * Implement synchronous redis call * Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana) * Add stat_convert command to convert stats tokens from sqlite3 to redis * Implement redis advanced lua api with pipelining * Fix memory leak on redis stat (#500) * Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana) * Fix build with custom pcre * Fix fuzzy relearning (#498) * Improve planning of asynchronous tasks * Show slow rules in log * Add warning for slow regexps * Add base32 decode/encode routines to lua util * Allow converting of learn cache from sqlite to redis * Add methods to check if a messages has from/rcpts * Improve and fix multimap plugin: - Restore 'header' maps - Add filters for headers - Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters - Add generic regexp filters * Disable reload command in rc scripts * Improve runtime CPU dispatcher for libcryptobox * Add preliminary support of digital signatures via ed25519 * Add detection for RDRAND support * Print configuration of crypto on start * A in SPF presumes AAAA lookup as well Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.1.2 * Fix stat_cache closing * Add checkpoints to sqlite3 learn cache * Do not recompile lua generated headers all the time * Increase number of messages learned * Fix issues with dual stack and hfilter * Disable MID checks for hfilter by default * Fix cache definitions in multiple classifier and no type * Don't crash if learn cache failed to initialize * Fix googlegroups support in maillist plugin * Rework flags LUA API: - Allow to check for a specific flag - Add `learn_spam`, `learn_ham` and `broken_headers` flags - Unify internal functions * Add `BROKEN_HEADERS` rule * Add support for forged confirmation headers (by @AdUser) * Allow `any`, `mime` and `smtp` for get_from/get_recipients * Add mime types checking plugin * Add rule to detect spammers attempts to cheat mime parsing * Rework parsing of IP addresses in configuration (better IPv6 support) * Add `util.parse_mail_address` function to LUA API * Add lua sqlite3 module * Implement synchronous redis call * Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana) * Add stat_convert command to convert stats tokens from sqlite3 to redis * Implement redis advanced lua api with pipelining * Fix memory leak on redis stat (#500) * Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana) * Fix build with custom pcre * Fix fuzzy relearning (#498) * Improve planning of asynchronous tasks * Show slow rules in log * Add warning for slow regexps * Add base32 decode/encode routines to lua util * Allow converting of learn cache from sqlite to redis * Add methods to check if a messages has from/rcpts * Improve and fix multimap plugin: - Restore 'header' maps - Add filters for headers - Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters - Add generic regexp filters * Disable reload command in rc scripts * Improve runtime CPU dispatcher for libcryptobox * Add preliminary support of digital signatures via ed25519 * Add detection for RDRAND support * Print configuration of crypto on start * A in SPF presumes AAAA lookup as well Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.1.0 * Incompatible change: sqlite3 and per_user behaviour: Now both redis and sqlite3 follows the common principles for per-user statistics: 1) If per-user statistics is enabled check per-user tokens ONLY 2) If per-user statistics is not enabled then check common tokens ONLY If you need old behaviour, then you'd need to use separate classifier for per-user statistics. * Implement redis statistics backend and cache * Implement autolearning for statistics * Reworked statistics architecture from scratch * Add hyperscan (https://github.com/01org/hyperscan) engine for regular expressions: - add lazy loader for hyperscan databases - rework regexp cache to have joint pcre/hyperscan scanning - implement hyperscan pre-filter support - add compilation guards for bad expressions - implement `rspamadm control recompile` command - implement hyperscan cache monitoring - slides: <https://highsecure.ru/rspamd-hyperscan.pdf> * Implement flexible task logging * Rework fuzzy worker: - it is now possible to run multiple fuzzy workers; - implement lazy writing as sqlite3 is bad at concurrent writing; - add retries for simple sql commands in fuzzy backend; - use fine-grained transactions for fuzzy; - implement new multi-pubkeys mode; - allow encrypted only storages; - rework statistics for fuzzy; - add `rspamadm control fuzzystat` command for extended statistics; - implement human readable output for the previous command; - add condition script for learning fuzzy storage; * Various fixes to SPF: - fix `redirect` records; - fix domains when parsing mx/ptr/a records in includes/redirects; - fix issues with multiple addresses in SPF records; - ignore SPF results in case of DNS failure; - adjust TTL of records when resolving subelements of SPF records; - always select `v=spf1` line if it is available - do not cache records with DNS failure in subrequests; - ignore records with temporary fails during subrequests resolving; - fix `RDNS_RC_NOREC` support; * Add clang plugin for static analysis: - implement static checks for `rspamd_printf` format strings; * Add 'allow_raw_input' option for non-mime messages * Recognize types using libmagic * Fix parsing of IPv6 received headers * Add new interface of communication between workers in rspamd * Add support for named socketpairs * Don't write URLs by default as it is too verbose * Set status for HTTP replies * Try load `rspamd.conf.override` * Implement words decaying for text parts to limit many checks * Improve support of SA rules and plugins: - add check_for_shifted_date and check_for_missing_to_header eval rules; - add 'check_relays_unparseable' support; - add `check_for_mime('mime_attachement')` function; - use new re_cache interface for all SA rules; - add support for `Mail::SpamAssassin::Plugin::MIMEHeader`; - add support of 'special' SA headers to `exists` function; - fix issue when SA metas contain other metas; - fix freemail rules; * Many fixes to the URL parser * Match any newline character in regexps * Fix resolving of upstreams and detection of poor IPv6 configurations * Parse upstreams selection algorithm from the configuration line * Add `reresolve` command to the control interface * Generate fuzzy hashes from task metadata (URLs and headers) * Add method to check if IP is local and `local_addrs` option * Implement forced timeout for delayed filters * Disable fast path of pcre-jit as it seems to be broken * Bayes fixes: - new normalizer function; - really use weights of tokens from the OSB algorithm; - restore multiple classifiers support; * Rules changes: - add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's; - improve empty image rule; - rework `FORGED_RECIPIENTS` rule; - reduce weight of `SUSPICIOUS_RECIPS`; - fix `*_NORESOLVE_MX` symbols in hfilter; - add `SUBJ_ALL_CAPS` rule with support of UTF8 - add spamhaus SBL to uribl - fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules - remove `R_TO_SEEMS_AUTO` as it generates a lot of FP; - add new Message-ID regexp for Thunderbird (by @moisseev); * Plugins changes: - allow ratelimit plugin to set symbol instead of pre-result - support IP DNS black lists for URIBL (e.g spamhaus SBL); - drop deprecated SURBL bits (by @fatalbanana) - rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana) - add `SURBL_BLOCKED` (by @fatalbanana) - add `CR_SURBL` - SURBL: allow fallthrough to default symbol (by @fatalbanana) - Settings: fix IP match (by @fatalbanana) - SURBL: add missing symbols to metric (by @fatalbanana) - allow processing images urls for SURBL - unconditionally disable SPF for authenticated users and local networks * Rework ratelimit plugin - switch to `rates` instead of old and stupid strings to setup; - check if a bucket is zero and disable the corresponding limits' - turn off all buckets by default; - check either `rcpt` or `user` buckets, not all together' - document new `rates` and `symbol` options; - inform user about what buckets are used in the configuration; * Add neural network **experimental** plugin * Add a sample script to learn neural network from rspamd logs * Add documentation strings support to rspamd: - add strings for the main configuration options; - document workers options; - add internal plugin options; - create `rspamadm confighelp` routine; - implement human readable output for the previous command; - add subtree search support; - add keyword search support; * Documentation improvements, tutorials section, statistics description * Many other minor and major bugfixes not noted here Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.1.0 * Incompatible change: sqlite3 and per_user behaviour: Now both redis and sqlite3 follows the common principles for per-user statistics: 1) If per-user statistics is enabled check per-user tokens ONLY 2) If per-user statistics is not enabled then check common tokens ONLY If you need old behaviour, then you'd need to use separate classifier for per-user statistics. * Implement redis statistics backend and cache * Implement autolearning for statistics * Reworked statistics architecture from scratch * Add hyperscan (https://github.com/01org/hyperscan) engine for regular expressions: - add lazy loader for hyperscan databases - rework regexp cache to have joint pcre/hyperscan scanning - implement hyperscan pre-filter support - add compilation guards for bad expressions - implement `rspamadm control recompile` command - implement hyperscan cache monitoring - slides: <https://highsecure.ru/rspamd-hyperscan.pdf> * Implement flexible task logging * Rework fuzzy worker: - it is now possible to run multiple fuzzy workers; - implement lazy writing as sqlite3 is bad at concurrent writing; - add retries for simple sql commands in fuzzy backend; - use fine-grained transactions for fuzzy; - implement new multi-pubkeys mode; - allow encrypted only storages; - rework statistics for fuzzy; - add `rspamadm control fuzzystat` command for extended statistics; - implement human readable output for the previous command; - add condition script for learning fuzzy storage; * Various fixes to SPF: - fix `redirect` records; - fix domains when parsing mx/ptr/a records in includes/redirects; - fix issues with multiple addresses in SPF records; - ignore SPF results in case of DNS failure; - adjust TTL of records when resolving subelements of SPF records; - always select `v=spf1` line if it is available - do not cache records with DNS failure in subrequests; - ignore records with temporary fails during subrequests resolving; - fix `RDNS_RC_NOREC` support; * Add clang plugin for static analysis: - implement static checks for `rspamd_printf` format strings; * Add 'allow_raw_input' option for non-mime messages * Recognize types using libmagic * Fix parsing of IPv6 received headers * Add new interface of communication between workers in rspamd * Add support for named socketpairs * Don't write URLs by default as it is too verbose * Set status for HTTP replies * Try load `rspamd.conf.override` * Implement words decaying for text parts to limit many checks * Improve support of SA rules and plugins: - add check_for_shifted_date and check_for_missing_to_header eval rules; - add 'check_relays_unparseable' support; - add `check_for_mime('mime_attachement')` function; - use new re_cache interface for all SA rules; - add support for `Mail::SpamAssassin::Plugin::MIMEHeader`; - add support of 'special' SA headers to `exists` function; - fix issue when SA metas contain other metas; - fix freemail rules; * Many fixes to the URL parser * Match any newline character in regexps * Fix resolving of upstreams and detection of poor IPv6 configurations * Parse upstreams selection algorithm from the configuration line * Add `reresolve` command to the control interface * Generate fuzzy hashes from task metadata (URLs and headers) * Add method to check if IP is local and `local_addrs` option * Implement forced timeout for delayed filters * Disable fast path of pcre-jit as it seems to be broken * Bayes fixes: - new normalizer function; - really use weights of tokens from the OSB algorithm; - restore multiple classifiers support; * Rules changes: - add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's; - improve empty image rule; - rework `FORGED_RECIPIENTS` rule; - reduce weight of `SUSPICIOUS_RECIPS`; - fix `*_NORESOLVE_MX` symbols in hfilter; - add `SUBJ_ALL_CAPS` rule with support of UTF8 - add spamhaus SBL to uribl - fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules - remove `R_TO_SEEMS_AUTO` as it generates a lot of FP; - add new Message-ID regexp for Thunderbird (by @moisseev); * Plugins changes: - allow ratelimit plugin to set symbol instead of pre-result - support IP DNS black lists for URIBL (e.g spamhaus SBL); - drop deprecated SURBL bits (by @fatalbanana) - rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana) - add `SURBL_BLOCKED` (by @fatalbanana) - add `CR_SURBL` - SURBL: allow fallthrough to default symbol (by @fatalbanana) - Settings: fix IP match (by @fatalbanana) - SURBL: add missing symbols to metric (by @fatalbanana) - allow processing images urls for SURBL - unconditionally disable SPF for authenticated users and local networks * Rework ratelimit plugin - switch to `rates` instead of old and stupid strings to setup; - check if a bucket is zero and disable the corresponding limits' - turn off all buckets by default; - check either `rcpt` or `user` buckets, not all together' - document new `rates` and `symbol` options; - inform user about what buckets are used in the configuration; * Add neural network **experimental** plugin * Add a sample script to learn neural network from rspamd logs * Add documentation strings support to rspamd: - add strings for the main configuration options; - document workers options; - add internal plugin options; - create `rspamadm confighelp` routine; - implement human readable output for the previous command; - add subtree search support; - add keyword search support; * Documentation improvements, tutorials section, statistics description * Many other minor and major bugfixes not noted here Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.0.0 * Rework symbols processing: - Improve sorting logic for symbols - Organize processing into multiple stages - Added asynchronous watchers for symbols - Added ability to organize dependencies between symbols * Fixed URL redirector: - Use optimized POE loop - Organize dependencies - Fix startup * New sqlite3 backend: - Allow to have per-languages and per-user statistics - Allow sqlite3 to be used as statistics backend * Store tokenizer configuration within statfiles * Improve bayes statistics: - Use headers and images metainformation in bayes - Suggest using of pre-processed tokens for statistics - Fix tokens normalization for OSB algorithm * Rewrite url parsing: - Fix numerous issues with url extraction and normalization - Fix mailto urls * Fix settings plugin to allow custom actions scores * Improve rbl plugin * Allow capturing patterns in rspamd lua regexp library * Add GTUBE support * Fix spamc legacy support * Add DKIM support to RBL module * Fix issues with multiple DKIM signatures * Fix issue if rspamd cannot create statfiles (#331) * Rework parts and task structure: - Now text_parts, parts and received are arrays - Pre-allocate arrays with some reasonable defaults - Use arrays instead of lists in plugins and checks - Remove unused fields from task structure - Rework mime_foreach callback function - Remove deprecated scan_milliseconds field * Add ip_score plugin support (not enabled by default): - Can check for asn/country and network using DNS lookups - Can store and load reputation from redis server * Improve PARTS_DIFFER rule to count merely different words * New HTML parser: - Parses HTML parts using a set of state machines - Extracts useful data and exports it to lua functions: + Styles + Images + URLs + Colors + Structure elements - Added HTML rules for some checks * New version of LUA DNS API * Table versions of many functions in LUA API * Improve rspamc client: - Print execution time - Allow executing of external commands and passing output to them - Allow mime output mode when rspamc alters message according to rspamd checks and send it to an external command or stdout * Allow scanning of local files using HTTP requests * Rework configuration system: - Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity - All modules configurations are now split in $CONFDIR/modules.d/* to simplify upgrades - Move hfilter to plugins - Allow plugins and rules to define default scores to simplify metrics setup - Include overrides for all modules to honor local/automatic parameters - Tune scores for many modules * Rework and enable DMARC plugin * Add whitelist plugin for SPF/DKIM/DMARC based whitelisting * Add some common domains to whitelists shipped with rspamd * Rework logging: - Now each log entry supports module name and a `tag`. Tag is used to identify unique objects (such as tasks) when checking log files - It is possible to turn on debugging for the specific modules - Systemd logging is fixed * Improve spamassassin plugin. - Now headers are matched more like SA - Improve support of Message-ID - Add support of ToCc header type - Fix :addr and :name in headers regexps * Resurrect rrd support code * Save controller stats between restarts * Fixed tonns of bugs * Added tonns of minor improvements and features * Added more unit tests * Create functional tests framework * Added documentation for missing modules * Added rpm/deb repositories and scripts * Updated WebUI and libucl externals Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.0.0 * Rework symbols processing: - Improve sorting logic for symbols - Organize processing into multiple stages - Added asynchronous watchers for symbols - Added ability to organize dependencies between symbols * Fixed URL redirector: - Use optimized POE loop - Organize dependencies - Fix startup * New sqlite3 backend: - Allow to have per-languages and per-user statistics - Allow sqlite3 to be used as statistics backend * Store tokenizer configuration within statfiles * Improve bayes statistics: - Use headers and images metainformation in bayes - Suggest using of pre-processed tokens for statistics - Fix tokens normalization for OSB algorithm * Rewrite url parsing: - Fix numerous issues with url extraction and normalization - Fix mailto urls * Fix settings plugin to allow custom actions scores * Improve rbl plugin * Allow capturing patterns in rspamd lua regexp library * Add GTUBE support * Fix spamc legacy support * Add DKIM support to RBL module * Fix issues with multiple DKIM signatures * Fix issue if rspamd cannot create statfiles (#331) * Rework parts and task structure: - Now text_parts, parts and received are arrays - Pre-allocate arrays with some reasonable defaults - Use arrays instead of lists in plugins and checks - Remove unused fields from task structure - Rework mime_foreach callback function - Remove deprecated scan_milliseconds field * Add ip_score plugin support (not enabled by default): - Can check for asn/country and network using DNS lookups - Can store and load reputation from redis server * Improve PARTS_DIFFER rule to count merely different words * New HTML parser: - Parses HTML parts using a set of state machines - Extracts useful data and exports it to lua functions: + Styles + Images + URLs + Colors + Structure elements - Added HTML rules for some checks * New version of LUA DNS API * Table versions of many functions in LUA API * Improve rspamc client: - Print execution time - Allow executing of external commands and passing output to them - Allow mime output mode when rspamc alters message according to rspamd checks and send it to an external command or stdout * Allow scanning of local files using HTTP requests * Rework configuration system: - Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity - All modules configurations are now split in $CONFDIR/modules.d/* to simplify upgrades - Move hfilter to plugins - Allow plugins and rules to define default scores to simplify metrics setup - Include overrides for all modules to honor local/automatic parameters - Tune scores for many modules * Rework and enable DMARC plugin * Add whitelist plugin for SPF/DKIM/DMARC based whitelisting * Add some common domains to whitelists shipped with rspamd * Rework logging: - Now each log entry supports module name and a `tag`. Tag is used to identify unique objects (such as tasks) when checking log files - It is possible to turn on debugging for the specific modules - Systemd logging is fixed * Improve spamassassin plugin. - Now headers are matched more like SA - Improve support of Message-ID - Add support of ToCc header type - Fix :addr and :name in headers regexps * Resurrect rrd support code * Save controller stats between restarts * Fixed tonns of bugs * Added tonns of minor improvements and features * Added more unit tests * Create functional tests framework * Added documentation for missing modules * Added rpm/deb repositories and scripts * Updated WebUI and libucl externals Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 0.9.0 * Add support of the fast and secure protocol level encryption: - curve25519 is used for key exchange; - chacha20/poly1305 cryptobox construction for bulk encryption; - zero latency overhead; - encrypting and balancing HTTP proxy worker * Rework expressions and create new expressions library: - aggressive optimizations based on the abstract syntax tree; - abstract expressions support (regular expressions, functions, lua modules composites and so on) - New comparision and '+' operators support - New greedy algorithm to minimize execution time of expressions and all symbols - Dynamic expressions benchmark and reoptimizations * Many improvements to the LUA API: - reworked logger module allowing to do pretty print of the most of lua types (including tables and userdata classes) - reworked lua redis and lua HTTP to support more features - added opaque type for passing large text chunks without copying - new regexp module with many auxiliary functions (e.g. `re:split`) * LuaJIT is now the default requirement for rspamd allowing to speed up lua execution by a large margin (however, plain lua is still supported) * New plugins: - spamassassin rules plugin that allows to load and re-use the most of SA rules natively - DMARC plugin that evaluates SPF and DKIM policies to the domain policies - many old plugins has been reworked to implement new features and improve stability * New aho-corasic trie implementation from @mischasan that allows to load and use hundreds of thousands of patterns with no influence on load * Support of PCRE JIT and PCRE JIT fast path modes that significantly improves the performance of regular expressions if supported by PCRE * New URLs parser and extractor: - removed legacy code that was useless for url finding - reworked algorithms of URL parsing for more precise and accurate results - added top-level-domains tree from http://publicsuffix.org - improved emails parsing - removed many phishing false positives due to TLD tree check * New statistics infrastructure: - created a separate layer of statistic library - improved OSB-Bayes by re-weighting tokens according to the original academic paper and `crm114` implementation, which reduced false positives rate significantly - created learn cache to avoid double learning of statistics and providing an efficient way to re-learn class for a message - created abstract layers for different statistics backends - implemented new tokenization algorithms with fast or secure (siphash) hashes to generate statistics features * Reworked utf8 tokenization that previously corrupted all UTF8 words (minor incompatibility with old fuzzy hashes with utf-8 symbols) * SPF module has been completely rewritten to support complex cases of `include` and `redirect` within SPF records * DKIM module now supports multiple signatures * Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the configuration file * Many hand-written HTTP clients has been replaced with the common rspamd http module * New test framework: - import lua `telescope` test framework - add unit tests for many rspamd modules and routines - create a unit test for each possible bug found - use luajit ffi for testing C code - added preliminary support of functional testing by creating tasks from lua * Randomize hash seed to avoid certain hash tables vulnerabilities * Documentation improvements: - added documentation for the vast majority of rspamd modules - added documentation for rspamd protocol - added documentation for the most of rspamd LUA extensions * Fixed tonns of bugs and memory leaks * Added tonns of minor features Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
11 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 0.9.0 * Add support of the fast and secure protocol level encryption: - curve25519 is used for key exchange; - chacha20/poly1305 cryptobox construction for bulk encryption; - zero latency overhead; - encrypting and balancing HTTP proxy worker * Rework expressions and create new expressions library: - aggressive optimizations based on the abstract syntax tree; - abstract expressions support (regular expressions, functions, lua modules composites and so on) - New comparision and '+' operators support - New greedy algorithm to minimize execution time of expressions and all symbols - Dynamic expressions benchmark and reoptimizations * Many improvements to the LUA API: - reworked logger module allowing to do pretty print of the most of lua types (including tables and userdata classes) - reworked lua redis and lua HTTP to support more features - added opaque type for passing large text chunks without copying - new regexp module with many auxiliary functions (e.g. `re:split`) * LuaJIT is now the default requirement for rspamd allowing to speed up lua execution by a large margin (however, plain lua is still supported) * New plugins: - spamassassin rules plugin that allows to load and re-use the most of SA rules natively - DMARC plugin that evaluates SPF and DKIM policies to the domain policies - many old plugins has been reworked to implement new features and improve stability * New aho-corasic trie implementation from @mischasan that allows to load and use hundreds of thousands of patterns with no influence on load * Support of PCRE JIT and PCRE JIT fast path modes that significantly improves the performance of regular expressions if supported by PCRE * New URLs parser and extractor: - removed legacy code that was useless for url finding - reworked algorithms of URL parsing for more precise and accurate results - added top-level-domains tree from http://publicsuffix.org - improved emails parsing - removed many phishing false positives due to TLD tree check * New statistics infrastructure: - created a separate layer of statistic library - improved OSB-Bayes by re-weighting tokens according to the original academic paper and `crm114` implementation, which reduced false positives rate significantly - created learn cache to avoid double learning of statistics and providing an efficient way to re-learn class for a message - created abstract layers for different statistics backends - implemented new tokenization algorithms with fast or secure (siphash) hashes to generate statistics features * Reworked utf8 tokenization that previously corrupted all UTF8 words (minor incompatibility with old fuzzy hashes with utf-8 symbols) * SPF module has been completely rewritten to support complex cases of `include` and `redirect` within SPF records * DKIM module now supports multiple signatures * Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the configuration file * Many hand-written HTTP clients has been replaced with the common rspamd http module * New test framework: - import lua `telescope` test framework - add unit tests for many rspamd modules and routines - create a unit test for each possible bug found - use luajit ffi for testing C code - added preliminary support of functional testing by creating tasks from lua * Randomize hash seed to avoid certain hash tables vulnerabilities * Documentation improvements: - added documentation for the vast majority of rspamd modules - added documentation for rspamd protocol - added documentation for the most of rspamd LUA extensions * Fixed tonns of bugs and memory leaks * Added tonns of minor features Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
11 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
12 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
12 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
12 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
12 years ago
12 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
12 years ago
12 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
12 years ago
12 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
12 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
12 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
13 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
13 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
13 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
14 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
14 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
8 years ago
  1. 3.14.3: 08 Jan 2026
  2. * [Feature] Add task registry for safe Lua task reference validation
  3. * [Feature] Add text quality analysis for PDF garbage filtering
  4. * [Feature] Implement basic PDF text extraction with UTF-16 detection
  5. * [Feature] Add extra tables API for clickhouse plugin
  6. * [Feature] Add confighelp documentation for RBL module
  7. * [Feature] WebUI: add backend API interaction error log
  8. * [Fix] Neural: by default include symbols with no flags
  9. * [Fix] Symcache: make FINE propagation deterministic
  10. * [Fix] URL: Prevent false positives from numeric IP regeneration in mailto URLs
  11. * [Fix] Settings: Allow spaces in selector regexps
  12. * [Fix] Prevent use-after-free in Redis callbacks after session cleanup
  13. * [Fix] Lua 5.4 compatibility in clickhouse and elastic plugins
  14. * [Fix] Use exact map lookup for DKIM key_table instead of glob
  15. * [Fix] Handle connection errors with io_uring backend in HTTP client
  16. * [Minor] Update public suffix list
  17. 3.14.2: 10 Dec 2025
  18. * [Feature] DMARC: Add --recheck-rua option to dmarc_report for RUA filtering at send time
  19. * [Feature] Metadata exporter: Add multipart and msgpack formatters
  20. * [Feature] Milter headers: Add remove_ar_from option for selective Authentication-Results header removal
  21. * [Fix] Security: Backport security fixes from libucl 0.9.3
  22. * [Fix] HTTP: Handle early server responses during request write
  23. * [Fix] HTTP: Only apply early response handling for HTTP clients
  24. * [Fix] MIME: Prevent splitting UTF-8 sequences in header encoding
  25. * [Fix] URL: Normalize URLs with multiple slashes between host and path
  26. * [Fix] URL: Use locale-independent patterns in URL encoding
  27. * [Fix] Milter headers: Allow default_headers_order to be configured
  28. * [Fix] Milter headers: Fix hierarchical domain matching in remove_ar_from map lookups
  29. * [Fix] Aliases: Enable plugin by default to restore plus-addressing
  30. * [Fix] DCC: Handle nil opts in plugin initialization
  31. * [Fix] Neural: Fix Lua 5.4 compatibility issues
  32. * [Fix] Reputation: Fix whitelist schema and selector-aware checking
  33. * [Fix] Logging: Fix off-by-one when at the maximum tag length
  34. * [Fix] Logging: Null-terminate the log tag when copying the header
  35. * [Fix] Prometheus: Use double type for rspamd_scan_time_average metric
  36. * [Fix] Build: FreeBSD 15 inotify build compatibility
  37. * [Fix] Build: Remove deprecated OpenSSL engine.h include
  38. * [Fix] Lua: Use 0 instead of LUA_OK for lua_pcall result comparison (Lua 5.1 compat)
  39. * [Fix] Lua: Handle Lua 5.4 require returning two values
  40. * [Fix] Lua: Use math.floor for Lua 5.4 integer division compatibility
  41. * [Fix] Lua: Improve loadstring error handling for Lua 5.4 compatibility
  42. * [Fix] Lua: Use userdata __gc for UCL objects in all Lua versions
  43. * [Fix] Lua: Use ipairs for ordered iteration in header checks
  44. * [Fix] Lua: Fix other places of Lua 5.4 rounding issues
  45. * [Fix] Performance: Avoid repeated simdutf implementation detection on each call
  46. * [Minor] Update public suffix list
  47. * [Minor] Debian: Allow skipping ASAN build
  48. * [Test] Add Lua 5.4 and cffi-lua support for testing
  49. * [Test] Fix Python 3.12+ compatibility in test scripts
  50. * [Test] Fix TCP tests for HTTP/1.1 compliance
  51. 3.14.1: 1 Dec 2025
  52. * [Feature] Composites: Add inverted index for fast composite rule evaluation
  53. * [Feature] Composites: Add bloom filter for fast negative symbol lookups
  54. * [Feature] Composites: Add statistics tracking and control protocol command
  55. * [Feature] Composites: Precompute atom types at config time
  56. * [Feature] Multimap: Add combinator option for selector rules
  57. * [Feature] SPF: Add rspamadm spf_flatten tool with macro preservation
  58. * [Feature] URL: Add deep processing architecture with C-to-Lua filter consultation
  59. * [Feature] URL: Add obfuscated URL detection to url_suspect plugin
  60. * [Feature] URL: Add rspamd_util.decode_html_entities for HTML entity decoding
  61. * [Feature] lua_shape: Add new validation library as tableshape replacement
  62. * [Feature] lua_shape: Add T.callable() type for function validation
  63. * [Feature] lua_shape: Add callable defaults support
  64. * [Feature] Whitelist: Auto-mark symbols with SYMBOL_TYPE_FINE flag
  65. * [Fix] url_suspect: Fix plugin causing massive false positives
  66. * [Fix] url_suspect: Optimize for high URL volume messages
  67. * [Fix] Network: Prevent infinite loop in split_networks_into_chunks()
  68. * [Fix] Memory: Fix leak in custom tokenizer result handling
  69. * [Fix] Composites: Fix group matchers handling in inverted index
  70. * [Fix] Composites: Improve atom polarity detection in inverted index
  71. * [Fix] Composites: Copy expression string to memory pool for Lua composites
  72. * [Fix] lua_shape: Fix transform logic and tableshape compatibility
  73. * [Fix] lua_shape: Fix registry to recursively resolve nested schemas
  74. * [Fix] lua_shape: Improve error safety
  75. * [Fix] Settings: Keep groups_*/symbols_* fields for runtime processing
  76. * [Fix] URL: Encode redirect URLs to handle unencoded spaces and special characters
  77. * [Fix] external_relay: Fix mixins and confighelp
  78. * [Fix] RBL: Fix plugin transform schemas
  79. * [WebUI] Update D3 libs with bug fixes and validation
  80. * [WebUI] Restore hover colors for symbols
  81. * [WebUI] Fix hover behavior outside status tables
  82. * [Minor] Whitelist: Use contemporary API for maps
  83. * [Minor] Migrate all plugins and libraries from tableshape to lua_shape
  84. * [Minor] Simplify configuration by removing use_*_map flags
  85. * [Minor] Add plugins registry with reworked mixins
  86. * [Test] Add comprehensive tests for URL deep processing
  87. * [Test] Isolate url_suspect tests with symbols_enabled
  88. 3.14.0: 10 Nov 2025
  89. * [Feature] Fuzzy check: Add HTML fuzzy hashing for structural similarity matching
  90. * [Feature] Fuzzy check: Add per-rule text_hashes toggle for HTML-only fuzzy rules
  91. * [Feature] Fuzzy check: Add structured checks configuration with backward compatibility
  92. * [Feature] Fuzzy storage: Implement full TCP protocol support with auto-switch
  93. * [Feature] Fuzzy check: Add TCP connection management and error handling
  94. * [Feature] URL: Add task:get_cta_urls() API for proper CTA domain extraction
  95. * [Feature] URL: Move CTA processing into dedicated module
  96. * [Feature] URL: Add url:get_hash() method for efficient deduplication without string conversion
  97. * [Feature] GPT: Add web search context support with Redis caching
  98. * [Feature] HTML: Add infrastructure for async URL rewriting with Lua bindings
  99. * [Feature] HTML: Add task:rewrite_html_urls() and task:get_html_urls() Lua API
  100. * [Feature] WebUI: Implement dark mode with theme toggle and auto detection
  101. * [Feature] Aliases: Add advanced resolution with loop detection for converging paths
  102. * [Feature] Milter: Add ESMTP argument parsing with Lua API access
  103. * [Feature] Milter: Add per-recipient ESMTP args parsing and metadata access
  104. * [Feature] Milter: Support array of positions for remove_headers operations
  105. * [Feature] Proxy: Add client IP preservation in message headers through chain
  106. * [Feature] Rspamc: Add milter.add_headers object format support to --mime
  107. * [Feature] Configwizard: Add Postfix integration wizard using postconf utility
  108. * [Feature] Build: Add comprehensive BSD workflows (FreeBSD, NetBSD, OpenBSD) with Lua version selection
  109. * [Feature] Build: Add automated code review GitHub Actions workflow with Rspamd-specific guidelines
  110. * [Feature] Build: Add Docker-based integration test suite with ASAN and real corpus
  111. * [Feature] Build: Add automatic public suffix list synchronization
  112. * [Feature] Multimap: Add support for symbols with leading numerals
  113. * [Feature] DMARC: Add Auto-Reply-To and Precedence headers to prevent out-of-office replies
  114. * [Feature] Platform: Add NetBSD memory usage tracking support
  115. * [Feature] Utilities: Add fuzzy Redis migration utility
  116. * [Feature] Bayes: Allow skipping local/authenticated mail in autolearn condition
  117. * [Feature] ARC: Add DKIM signing key API for flexible ARC signing
  118. * [Feature] Logger: Add type specifiers support for better formatting
  119. * [Feature] Heap: Add rspamd_heap_push_slot to eliminate double allocation
  120. * [Fix] DNS: Preserve req->pos during reply validation to prevent packet truncation on UDP-to-TCP retransmits
  121. * [Fix] DNS: Regenerate transaction ID before copying to TCP buffer to avoid collisions
  122. * [Fix] DNS: Fix nameserver round-robin when using /etc/resolv.conf
  123. * [Fix] DNS: Fix TCP uninitialized memory leak
  124. * [Fix] DMARC: Add batching and forced GC for Redis connections to prevent pool exhaustion
  125. * [Fix] DMARC: Validate and normalize batch_size to prevent fractional indexing and loop errors
  126. * [Fix] DMARC: Refactor reporting to use helper functions and async maps
  127. * [Fix] Allocator: Fix jemalloc/system malloc mixing in getline() to prevent crashes
  128. * [Fix] Allocator: Fix allocator mismatches in hiredis
  129. * [Fix] Allocator: Fix allocator mismatches in libucl
  130. * [Fix] Hyperscan: Use runtime version instead of compile-time for database validation
  131. * [Fix] Hyperscan: Auto-recreate invalid unserialized cache files on version mismatch
  132. * [Fix] Memory: Fix leaks in fuzzy storage khash tables
  133. * [Fix] Memory: Fix leaks in upstream address parsing
  134. * [Fix] Memory: Fix leaks in *-any address parsing
  135. * [Fix] Memory: Fix OpenSSL providers cleanup
  136. * [Fix] Memory: Fix UCL object memory leak in Lua integration
  137. * [Fix] Memory: Fix stat metadata tokenization leak
  138. * [Fix] Fuzzy TCP: Fix double-release in fuzzy_tcp_session
  139. * [Fix] Fuzzy TCP: Fix refcount leak in destructor
  140. * [Fix] Fuzzy TCP: Fix timeout handling and buffer overflow
  141. * [Fix] Fuzzy TCP: Fix endianness mismatch in framing protocol
  142. * [Fix] Fuzzy TCP: Fix race conditions and fd reuse bugs
  143. * [Fix] Fuzzy TCP: Use pure ev_timer for session timeouts
  144. * [Fix] Fuzzy TCP: Fix server replies and client event handling
  145. * [Fix] Shutdown: Keep srv events active during shutdown to track auxiliary processes
  146. * [Fix] ARC: Restore strict header ordering to comply with RFC 8617
  147. * [Fix] ARC: Add ed25519 key support
  148. * [Fix] Composites: Implement two-phase evaluation for postfilter dependencies
  149. * [Fix] Composites: Use null-terminated string for symbol lookup
  150. * [Fix] URL: Refactor extraction to prevent DoS with hash-based deduplication
  151. * [Fix] URL: Add 50k URL limit with warning for DoS protection
  152. * [Fix] URL: Skip HTML_DISPLAYED URLs in CTA detection
  153. * [Fix] URL: Fix CTA priority preservation in extract_specific_urls
  154. * [Fix] Bayes: Improve Redis server discovery
  155. * [Fix] Bayes: Only bypass learn when header value matches
  156. * [Fix] ESMTP: Robust per-recipient parsing in milter with safe cursor advance
  157. * [Fix] ESMTP: Refcount ESMTP args in proxy_session_refresh to avoid use-after-free
  158. * [Fix] ESMTP: Correct Lua stack cleanup in lua_task_get_rcpt_esmtp_args
  159. * [Fix] HTML: Correct attribute value offset calculation for URL rewriting
  160. * [Fix] HTML: Add HTML entity encoding for URL rewriting
  161. * [Fix] HTML: Fix segfault due to incorrect HTML features access
  162. * [Fix] HTML: Fix frequency-based ordering in domain hashing
  163. * [Fix] HTML: Fix shingles hash generation bugs
  164. * [Fix] HTML: Fix memory leaks in shingles generation
  165. * [Fix] HTML: Fix memory management in html_cta.process_html_links
  166. * [Fix] HTML: Fix CSS class normalization in fuzzy tokens
  167. * [Fix] HTML: Fix cache key collision between text and HTML fuzzy hashes
  168. * [Fix] OpenBSD: Fix kinfo_proc structure member names
  169. * [Fix] OpenBSD: Disable Hyperscan (not available)
  170. * [Fix] FreeBSD: Fix zstd package name
  171. * [Fix] FreeBSD: Add IGNORE_OSVERSION for package version mismatches
  172. * [Fix] NetBSD: Setup pkgin and PKG_PATH before installing packages
  173. * [Fix] NetBSD: Fix missing dependencies and package names
  174. * [Fix] BSD: Remove -j flag from ninja in all BSD workflows
  175. * [Fix] Multimap: Handle symbols with leading numerals
  176. * [Fix] Aliases: Prevent creation of malformed email addresses
  177. * [Fix] Aliases: Fix alias loop detection for converging paths
  178. * [Fix] Aliases: Fix is_local_domain to support backend objects
  179. * [Fix] Aliases: Correct to_local when no recipients present
  180. * [Fix] Aliases: Fix set_addr validation to prevent malformed addresses
  181. * [Fix] MIME: Remove Authentication-Results and anonymize envelope-from in Received headers
  182. * [Fix] Mempool: Prevent double-free in destructor cleanup
  183. * [Fix] Rspamadm: Unbreak dnstool command
  184. * [Fix] Integration tests: Fix ASAN configuration and startup diagnostics
  185. * [Minor] Replace GHashTable with khash in fuzzy_check.c and lua_textpart_get_cta_urls
  186. * [Minor] Update cache key prefix to match module name in llm_search_context
  187. * [Minor] Fix llm_search_context to follow Rspamd idioms
  188. * [Minor] Refactor llm_search_context to use lua_cache module
  189. * [Minor] Address review comments in various modules
  190. * [Minor] Fix droid usage
  191. * [Minor] Use GPT-5 Codex for code reviews
  192. * [Minor] Update libucl with automatic stack management
  193. * [Rework] Prioritize CTA URLs in redirector and Lua helpers
  194. * [Rework] RBL configuration: Add new from selectors, content_urls checks, and lower_utf8 for hashed domains
  195. * [Rework] Make Bayes learn guards configurable
  196. * [Rework] Refactor element visibility control to use Bootstrap classes
  197. * [Rework] Use postconf utility for Postfix configuration in configwizard
  198. * [Rework] Remove Lua-level HTTP header parsing in ESMTP args getters
  199. * [Rework] Add CFG_REF_* macros with debug logging for config refcounting
  200. * [Rework] Move OpenSSL providers from global to libs_ctx
  201. * [Rework] Convert heap to fully intrusive kvec-based implementation
  202. * [Rework] Add specialized pool types for long-lived and short-lived allocations
  203. * [Rework] Improve memory pool destructors with smart preallocation based on pool type
  204. * [Project] Restrict code review workflow to authorized maintainers
  205. * [Project] Add Claude Code and Cursor AI assistant configuration
  206. * [WebUI] Replace Glyphicons with FontAwesome SVG icons
  207. * [WebUI] Update CodeJar to version 4.3.0
  208. * [WebUI] Update Node.js and ESLint
  209. * [WebUI] Update D3-based visualization libs
  210. * [WebUI] Replace deprecated alert-error class with alert-danger
  211. * [WebUI] Add search syntax hint to history table filter input
  212. * [WebUI] Fix theme toggle default to auto
  213. * [WebUI] Keep classifiers list when request is skipped
  214. * [WebUI] Repopulate classifier dropdown
  215. * [WebUI] Add comment for removeEventListener
  216. * [WebUI] Fix icon rendering race condition in tab initialization
  217. * [Test] Add comprehensive Lua unit tests for HTML URL rewriting
  218. * [Test] Add unit tests for HTML URL rewriting patch engine
  219. * [Test] Add functional tests for HTML fuzzy hashing
  220. * [Test] Add ARC chain verification tests with multiple signatures
  221. * [Test] Add e2e for classifier dropdown population
  222. * [Test] Multimap symbol with leading numerals
  223. * [Test] Sync public suffix list automatically
  224. * [Test] Update JS linters
  225. * [Test] Fix integration test environment variable passing
  226. * [Test] Add detailed error output for integration test failures
  227. 3.13.2: 5 Oct 2025
  228. * [Feature] Fuzzy check: Add separate encryption keys for read and write operations
  229. * [Feature] DKIM: Add ED25519 support for DKIM signing and verification with OpenSSL version checks
  230. * [Feature] Vault: Add HashiCorp Vault KV version 2 support for DKIM key management
  231. * [Feature] MetaDefender: Add MetaDefender Cloud Lua module for SHA256 hash lookups
  232. * [Feature] LLM: Add user/domain context support for LLM-based classification with Redis-based conversation context
  233. * [Feature] DMARC: Add RUA address exclusion configuration option
  234. * [Fix] DKIM: Fix relaxed bodyhash calculation for lines with only spaces to comply with RFC 6376
  235. * [Fix] DKIM: Fix ED25519 key loading to prevent memory corruption in union handling
  236. * [Fix] HTTP maps: Enforce server-controlled refresh intervals and prevent aggressive polling
  237. * [Fix] HTTP maps: Prevent time_t overflow in expires header processing
  238. * [Fix] Once received plugin: Fix duplicate symbol addition by changing break to return
  239. * [Fix] Redis: Propagate unused Sentinel options properly
  240. * [Fix] Fuzzy check: Fix reply decryption when using separate read/write keys
  241. * [Fix] Fuzzy check: Add fallback when only one specific encryption key is set
  242. * [Fix] Fuzzy check: Fix duplicate key filtering in reply decryption
  243. * [Fix] Fuzzy ping: Allow read/write servers configuration
  244. * [Minor] Fuzzy check: Refactor encryption key selection into helper functions
  245. * [Minor] Fuzzy check: Stop early when found a correct key
  246. * [Minor] Add cursor rules for development
  247. 3.13.1: 30 Sep 2025
  248. * [Feature] Archive module: Full support for encrypted ZIP archives with ZipCrypto and AES encryption
  249. * [Feature] Archive module: Both reading and writing of AES-encrypted ZIP archives is supported
  250. * [Feature] Archive module: Updated Lua bindings for libarchive
  251. * [Feature] Encrypted maps: Support for encrypted maps to enable new distribution scenarios
  252. * [Feature] Redis TLS: Configurable TLS connections in Redis backend
  253. * [Feature] Map helpers alignment: Enforce 64-byte alignment to prevent unaligned memory access
  254. * [Feature] Enhanced CLI for secretbox with additional security test coverage
  255. * [Fix] MIME encoding: Major overhauls and multiple fixes for MIME encoding logic
  256. * [Fix] MIME encoding: Improved handling and decoding of UTF-8 in MIME headers
  257. * [Fix] Learning system: Numerous fixes to learn checks and autolearn flag handling
  258. * [Fix] Learning system: Prevention of duplicate message learning
  259. * [Fix] Learning system: Extended multiclass learning test coverage
  260. * [Fix] Critical: Fixed bug when converting zero-length strings to numbers
  261. * [Fix] Critical: Fixed XML prolog detection in lua_magic module
  262. * [Fix] Build: Fixed build issues on 32-bit platforms
  263. * [Fix] Compatibility: Improved compatibility with Lua versions above 5.1
  264. * [Fix] Empty input: Addressed issues with empty input handling in lua_magic
  265. * [Fix] Testing: Improved stability of automated testing with multiple test fixes
  266. * [Fix] Minor compatibility improvements (buffer allocation, missing cmath include)
  267. 3.13.0: 17 Sep 2025
  268. * [Conf] Add defaults
  269. * [Conf] Fix JB IDE damage
  270. * [Feature] Add a signal from main to workers for workers ready state
  271. * [Feature] Add lua_util.fold_header_with_encoding
  272. * [Feature] Add some convenience options to rspamc
  273. * [Feature] Add some more OS utility functions
  274. * [Feature] Add symbols proxy for piecewise changes
  275. * [Feature] Allow lua callback maps to be filled line by line
  276. * [Feature] Allow selectors in regexp maps expressions
  277. * [Feature] Allow to pass expression flags in the regexp plugin
  278. * [Feature] Detect part types in mime parser
  279. * [Feature] Resolve DNS nameservers names using getaddrinfo
  280. * [Fix] Bayes: Try to be bug-to-bug compatible
  281. * [Fix] Check skip_hashes for the returned hashes
  282. * [Fix] Fix DL lists initialisations
  283. * [Fix] Fix double free in the client...
  284. * [Fix] Fix end-to-end proxy compression
  285. * [Fix] Fix l= calculations again
  286. * [Fix] Fix lua state setting ambiguity
  287. * [Fix] Fix order of descriptor closing
  288. * [Fix] Fix probabilities overflow
  289. * [Fix] Fix rules setup
  290. * [Fix] Fix statfiles ordering
  291. * [Fix] Fix various corner cases and tests
  292. * [Fix] Fix whitelist options in the arc module
  293. * [Fix] GPT: Fix occasional damage
  294. * [Fix] GPT: fix processing of messages with no subject
  295. * [Fix] Prevent WebUI crash with empty RRD
  296. * [Fix] Store html attributes that are empty
  297. * [Fix] Try to fix learned order
  298. * [Fix] Use C++20 standard consistently to resolve ODR violations
  299. * [Fix] Use a more straightforward approach for learn cache
  300. * [Fix] fix error check in lua_dkim_tools.lua
  301. * [Project] Add CTA analytics engine
  302. * [Project] Add ability to create custom tokenizers for languages
  303. * [Project] Add controller learn endpoints
  304. * [Project] Add support of granular timeouts to plugins and maps
  305. * [Project] Add tests and fix stuff
  306. * [Project] Add tests for LLM provider, fix various issues with metatokens
  307. * [Project] Apply changes to bayes_expiry plugin
  308. * [Project] Create an isolated API for external tokenizers
  309. * [Project] Extract more features from HTML messages
  310. * [Project] Fix Lua API and some constexpr compatibility
  311. * [Project] Fix binary classification and lua scripts
  312. * [Project] Fix more calculation issues
  313. * [Project] Fix other classification and learning issues
  314. * [Project] Fix scoped compilation again
  315. * [Project] Fix symbols finalisation
  316. * [Project] Fix unlearn stuff
  317. * [Project] Fix various issues
  318. * [Project] Fix various other issues
  319. * [Project] Further updates
  320. * [Project] Implement backoff for upstreams revival
  321. * [Project] Implement more flexible http timeouts
  322. * [Project] Implement scoped compilation
  323. * [Project] Implement scoped regexp cache system
  324. * [Project] Multi-class classification project baseline
  325. * [Project] Rework rspamc to allow training of different neural types
  326. * [Project] Rework system of html tags to allow more tag types
  327. * [Project] Rework tokenizers initialisation
  328. * [Project] Some rework of the CTA defaults
  329. * [Project] Start implementation of the rules maps
  330. * [Project] Start to implement better revive strategy for upstreams
  331. * [Project] Store regexp rules state to avoid incomplete/orphaned rules
  332. * [Project] Support more common html attributes
  333. * [Project] Take button weight into consideration
  334. * [Project] Use re_cache scopes for maps
  335. * [Rework] Fix logger format string mismatch
  336. * [Rework] MIME detection via Lua Magic; enforce cfg in Lua task API
  337. * [Rework] Return back N-ary optimizations for arithmetic-alike expressions
  338. * [Rework] Use GLib agnostic type for words
  339. * [Rework]Refactor MIME detection via Lua Magic; enforce cfg in Lua task API
  340. * [Rules] Make bitcoin expression to use explicit flags
  341. 3.12.1: 17 Jun 2025
  342. * [Feature] Add /bayes/classifiers HTTP endpoint
  343. * [Feature] Further improvements in scheduling next checks
  344. * [Fix] Another fix for maps concurrent load
  345. * [Fix] Do not add log tag header in milter logic
  346. * [Fix] Do not explicitly add Connection header if it's there
  347. * [Fix] Fix proxy headers duplication
  348. * [Fix] Fix several issues with the lua_logger
  349. * [Fix] Make logger more graceful when dealing with format arguments
  350. * [Fix] Try to avoid incomplete writes
  351. * [Rework] Eliminate maps locking
  352. 3.12.0: 09 Jun 2025
  353. * [CritFix] In lua-ucl disable macros and file variables by default
  354. * [Feature] Add keep-alive support
  355. * [Feature] Add some convenience methods
  356. * [Feature] Add support for separate read and write servers in fuzzy check
  357. * [Feature] Allow CDB files as external maps
  358. * [Feature] Allow to specify Redis version
  359. * [Feature] Allow to specify extra headers in Rspamd proxy
  360. * [Feature] Allow to specify log tag in proxy
  361. * [Feature] Allow to specify max log tag length for all log messages
  362. * [Feature] Allow to use HTTPS when connection to backends in proxy
  363. * [Feature] Output content for all maps
  364. * [Feature] Plugin to integrate with Contextal platform
  365. * [Feature] Show all maps status
  366. * [Fix] Add fail check for cfg transform for some corner cases
  367. * [Fix] Add header with reason everytime (not only for ham) and use correct value for header
  368. * [Fix] Add null check for master_conn->up in proxy backend error handler
  369. * [Fix] Allow 'Hash' in Access-Control-Allow-Headers
  370. * [Fix] Arc: Use tonumber when comparing
  371. * [Fix] As we have replxx library, always use it
  372. * [Fix] Backport some issues from libucl
  373. * [Fix] Filter invalid domains in fuzzy extra data
  374. * [Fix] Fix maps ids
  375. * [Fix] Fix race condition in maps loading by unlocking backend on switch
  376. * [Fix] Fix static maps description passing
  377. * [Fix] Fix variable propagation (no functional change)
  378. * [Fix] Fix various issues
  379. * [Fix] Greylist: Improve body hash calculations
  380. * [Fix] Known senders: More recipients test logic
  381. * [Fix] Known senders: Use the same logic as in the replies module
  382. * [Fix] Prevent crashes when accessing upstream address in self-scan mode
  383. * [Fix] Really fix local objects filtering, sigh...
  384. * [Fix] Update default URL for openphish
  385. * [Fix] Use bundled libfmt everywhere
  386. * [Fix] Use safe parsers everywhere except configuration
  387. * [Fix] correct logic error in milter_headers.lua: skip_wanted()
  388. * [Fix] initialize ollama result table
  389. * [Fix] libmime: declare comparators const for doctest 2.4.12 compatibility
  390. * [Project] Modernize cmake
  391. * [Project] Rework OSDep
  392. * [Rework] Replies: consider all recipients and use smtp ones
  393. * [Rework] Store shared maps data separately
  394. * [Rework] Use locks/loaded per backend for all maps
  395. 3.11.1: 08 Mar 2025
  396. * [Feature] Add 'noop' redis backend for scripts running
  397. * [Feature] Add Redis caching framework
  398. * [Feature] Add UTF8 `sub` and `len` variants for rspamd_text
  399. * [Feature] Allow adding timers to task (respecting symbols)
  400. * [Feature] Allow additional categories to be defined in GPT
  401. * [Feature] Allow fine-grained control on keys permissions
  402. * [Feature] Allow individual `read_only` flag per key
  403. * [Feature] Allow multiple lua scripts for fuzzy storage
  404. * [Feature] Allow to add periodic functions in Lua API
  405. * [Feature] Allow to disable rbls from map
  406. * [Feature] Allow to hash any Lua types
  407. * [Feature] Allow to store shingles as opaque Lua data
  408. * [Feature] Cache LLM replies
  409. * [Feature] GPT: Add ollama support
  410. * [Feature] GPT: Support reason adding
  411. * [Feature] Improve prompt and use plaintext instead of JSON
  412. * [Feature] Lua_task: Allow to load data into the existing task
  413. * [Feature] More additions
  414. * [Feature] Pass shingles to Lua scripts
  415. * [Feature] Preliminary implementation of LLM based anonymizing
  416. * [Feature] Support LLM models consensus
  417. * [Feature] Try to check maps earlier if their expires is too long
  418. * [Feature] Use debug module name from caller in lua_cache
  419. * [Fix] Add timer update before timer setting
  420. * [Fix] Allow to work with no ratelimits
  421. * [Fix] Always create ratelimit buckets
  422. * [Fix] Avoid collision hacks in mempool variables hash
  423. * [Fix] Expire neural ham and spam sets so they not hangup in redis indefinitely
  424. * [Fix] Fix crash on FreeBSD when Rspamd is built without hyperscan
  425. * [Fix] Make table digests consistent
  426. * [Fix] RBL: fix use of `content_urls` and `images` inside `checks`
  427. * [Fix] Verify key type to match DKIM signature type
  428. * [Fix] connIP is not correctly added to request
  429. * [Fix] properly close multipart/related boundary when adding text footer
  430. * [Rework] GPT: Use cache framework
  431. 3.11.0: 16 Dec 2024
  432. * [Conf] Add lua.local.d folder
  433. * [Conf] Add more ways to extend Rspamd configuration
  434. * [Conf] Allow to install examples conditionally
  435. * [Conf] Fix comments in elastic.conf
  436. * [Conf] Install examples
  437. * [Conf] Use full hostnames for SURBL
  438. * [CritFix] Allow to reupload scripts when Redis is restarted
  439. * [CritFix] Fix ARC-Seal signing
  440. * [CritFix] Properly check dynamic keys in fuzzy storage
  441. * [Feature] Add LRU cache for last filled ratelimit buckets
  442. * [Feature] Add include/exclude logic for headers
  443. * [Feature] Add ratelimit util to manage last filled ratelimit buckets
  444. * [Feature] Dump current ratelimits for fuzzy keys
  445. * [Feature] GPT: Try harder to find JSON in NN reply
  446. * [Feature] Improve address rotation algorithm
  447. * [Feature] Show stats in rspamadm
  448. * [Feature] Unify displayed part selection
  449. * [Fix] Avoid null-bytes in Log-Tag header value.
  450. * [Fix] Another story about char sign
  451. * [Fix] Check DNS limits when resolving MX/PTR records
  452. * [Fix] Dmarc: Properly encode structured headers
  453. * [Fix] Do not abort when OpenSSL is broken, report that to a user
  454. * [Fix] Fix dealing with happy eyeballs in SPF
  455. * [Fix] Fix memory leak in `lua_new_text` invocations
  456. * [Fix] Fix usage of the logging function
  457. * [Fix] Load "default" provider as well as "legacy"
  458. * [Fix] Milter headers: fold before encoding
  459. * [Fix] More fixes to rfc2047 encoding
  460. * [Fix] More things to fix
  461. * [Fix] Phishing: Fix many cases of FP for the same domains
  462. * [Fix] Remove USPOOF_SINGLE_SCRIPT_CONFUSABLE from the checks
  463. * [Fix] Restore propagation of spam/ham learns
  464. * [Fix] Sigh, another fix
  465. * [Fix] Some fixes to `rspamadm ratelimit`
  466. * [Fix] Some more fixes
  467. * [Fix] Use `dkim_signing` for `sign_headers` option
  468. * [Fix] Use correct type for keylen in lua_ucl_newindex
  469. * [Fix] Use cumulative timeout when dealing with TCP connections
  470. * [Fix] Use unsigned char for ragel machines
  471. * [Project] Add routine to strip attachments
  472. * [Project] Add tool to rspamadm
  473. * [Project] Initial stuff to implement messages anonymization
  474. * [Project] Keep Changelog updated
  475. * [Project] Various fixes and `rspamadm mime strip` command
  476. * [Rework] Allow `Content-Encoding` standard header for zstd compression
  477. * [Rework] Breaking: Actualize elastic module, support Elastic 8 & OpenSearch 2, add index policy with logs retention and many more
  478. * [Rework] Multimap should use only distinct text parts for content matching
  479. * [Rework] Replace fastutf with simdutf
  480. * [Rework] Rewrite rfc2047 encoding as it was totally broken
  481. * [Rework] Update hiredis to 1.2.0
  482. 3.10.2: 21 Oct 2024
  483. * [CritFix] Fix ARC-Seal signing
  484. * [Fix] add EOF to openmetrics response in proxy and server
  485. 3.10.1: 16 Oct 2024
  486. * [Feature] Update effective_tld_names.dat by @wdhdev in #5176
  487. * [Fix] Use correct type for keylen in lua_ucl_newindex by @arkamar in #5169
  488. * [Fix] Avoid null-bytes in Log-Tag header value by @smarsching in #5179
  489. * [Fix] Do not abort when OpenSSL is broken, report that to a user by @vstakhov in #5188
  490. * [Fix] Update hiredis library removing all hacks by @vstakhov in #5167
  491. * [Fix] Remove proxy from url_redirector.conf as it not the option by @dragoangel in #5164
  492. * [Fix] Some build fixes by @vstakhov in #5189
  493. * [Fix] Some more fixes by @vstakhov in #5190
  494. 3.10.0: 30 Sep 2024
  495. * [Conf] Add SenderScore RPBL return codes
  496. * [Conf] Add SenderScore Reputationlist RBL
  497. * [Conf] Increase scores for strange things in the archives
  498. * [CritFix] The max size for signing key is actually 65 bytes for p256
  499. * [Feature] Add rspamadm secretbox command
  500. * [Feature] Add rspamd_cxx_unit_cryptobox for unit testing
  501. * [Feature] Add support for OpenSSL 3.0
  502. * [Feature] Add support for OpenSSL 3.0 for DKIM
  503. * [Feature] Add tests for rspamd_cryptobox
  504. * [Feature] Add tooling to encrypt strings in Lua
  505. * [Feature] Allow differen modes for fuzzy rules
  506. * [Feature] Allow to set negative group score limit via `min_score`
  507. * [Feature] DMARC: Implement reporting.only_domains setting
  508. * [Fix] Add workaround for the bug in OpenSSL < 3.0.8
  509. * [Fix] Another fix for Redis schema
  510. * [Fix] Another safe-guards for learning/classifying an empty message
  511. * [Fix] Apply the same workaround for signing keys
  512. * [Fix] Check message before trying to dereference pointer
  513. * [Fix] Do not install doctest stuff
  514. * [Fix] Encode headers that we send to milter add header
  515. * [Fix] Fix DCC `rep` handling
  516. * [Fix] Fix Redis scripts uploading when Redis is not ready
  517. * [Fix] Fix and rework various parts
  518. * [Fix] Fix compatibility with OSSL 1.0
  519. * [Fix] Fix glib internals by setting locale in `rspamc`
  520. * [Fix] GPT: Fix bug in condition check
  521. * [Fix] Get rid of EVP_PKEY_CTX_set1_rsa_keygen_pubexp
  522. * [Fix] Iterate over dynamic keys in fuzzy storage
  523. * [Fix] Make tostring in UCL a bit less brain-damaged
  524. * [Fix] More bogus sizes fix
  525. * [Fix] Preserve the previous behaviour of RDNS_* checks
  526. * [Fix] Rework DMARC to correctly handle spaces in DMARC records Issue: #4906
  527. * [Fix] Sign key != encryption key, omg
  528. * [Fix] Unify lua symbols registration
  529. * [Fix] Use proper keys when doing asymmetric encryption
  530. * [Project] Add API method to push unwrapped UCL object
  531. * [Project] Add more stuff to transparent UCL
  532. * [Project] Add parsing of key limits and expire date
  533. * [Project] Add ratelimit parsing for fuzzy keys
  534. * [Project] Allow manipulations with opaque UCL objects
  535. * [Project] Allow to change log tag from HTTP request
  536. * [Project] Implement expiration
  537. * [Project] Implement per-key ratelimit
  538. * [Project] Move ratelimit parsing stuff to a common library
  539. * [Project] Remove NIST (OpenSSL) mode from cryptobox
  540. * [Project] Remove NIST mode from everywhere
  541. * [Project] Rework ratelimits check
  542. * [Project] Some more fixes
  543. * [Project] Start support of MIME UTF8
  544. * [Project] Try to allow more transparent access of ucl elements
  545. * [Rework] Allow more flexible keypair encoding
  546. * [Rework] Breaking: Rewrite cfg transform and remove legacy
  547. * [Rework] Change fuzzy error symbols
  548. * [Rework] Change the logic of skipping symbols
  549. * [Rework] Clean up legacy code
  550. * [Rework] Implement new replies logic on the server's side
  551. * [Rework] Remove control block support
  552. * [Rework] Resolve rdns in a separate function
  553. * [Rework] Use __builtin_cpu_supports where possible
  554. * [Rework] Use a more straight structure for DKIM keys
  555. * [Rules] Fix some old rules
  556. 3.9.1: 23 Jul 2024
  557. * [Conf] Spf: Add R_SPF_PLUSALL symbol with some score
  558. * [Feature] Spf: Treat SPF +all in a special way
  559. * [Minor] Ensure some safety when checking weights
  560. * [Minor] Fix several issues with flag propagation
  561. * [Minor] Gpt: Improve prompt and add some conversion heursitics
  562. * [Minor] Gpt: Remove top_p reduce temperature to 0
  563. * [Minor] Gpt: Set response_format
  564. * [Minor] Gpt: Use gpt-4o-mini by default
  565. 3.9.0: 12 Jul 2024
  566. * [CritFix] Protect regexp matcher from regexps with empty patterns
  567. * [Feature] Allow adding X-CMAE-Score header
  568. * [Feature] Allow custom milter quarantine and tempfail messages
  569. * [Feature] Allow to specify minimum weight in GPT plugin
  570. * [Feature] Cloudmark: Add scores_symbols setting
  571. * [Feature] Further optimization to the hot path
  572. * [Feature] Make min_resolve_interval configurable
  573. * [Feature] Report slow synchronous rules
  574. * [Feature] Rspamc: add `files-list` option
  575. * [Feature] Support metrics command in normal/proxy workers
  576. * [Feature] Support reply in message pack format
  577. * [Feature] Use msgpack in the client
  578. * [Feature] Verify sanity of l= tag in DKIM
  579. * [Feature] milter_headers: support not removing headers
  580. * [Fix] Allow autolearn for the controller worker
  581. * [Fix] Allow spaces in DMARC records
  582. * [Fix] Allow to set 0 as number of rows to disable roll history
  583. * [Fix] Always set the unlearn flag when relearning
  584. * [Fix] Apply detection phase if fasttext could not detect language
  585. * [Fix] Backport fixes from libucl
  586. * [Fix] Backport multiple fixes from libucl
  587. * [Fix] Change expiration logic for redis_cache
  588. * [Fix] Check nconns when firing the final termination event
  589. * [Fix] Do not apply tableshape for known senders
  590. * [Fix] Do not crash if symbol is missing in the metric
  591. * [Fix] Do not read out-of-boundary when doing base64 encoding
  592. * [Fix] Do not save multipatterns to FS in certain cases
  593. * [Fix] Fix another corner case that allows candidates to be freed without init
  594. * [Fix] Fix documentation nesting
  595. * [Fix] Fix dynamic_symbols in the multimap plugin
  596. * [Fix] Fix inconsistent nesting in mime parts
  597. * [Fix] Fix parsing of maps definitions
  598. * [Fix] Fix shared memory proxying when compression is set
  599. * [Fix] Fuzzy add/delete handlers are badly broken for compression
  600. * [Fix] Honor dynamic thresholds for greylisting module
  601. * [Fix] If we have one statfile disabled we need to disable all
  602. * [Fix] Increase/decrease hash_key value for tokens depending on is_unlearn
  603. * [Fix] Libucl: Fix messagepack autodetection
  604. * [Fix] Resgore the collaterally removed line
  605. * [Fix] Unset autolearn flag if we learn explicitly
  606. * [Fix] Update `nconns` in proxy
  607. * [Fix] Update greylisting threshold as it can be easily dynamic
  608. * [Fix] Update is_spam in learned_ids when relearning
  609. * [Fix] Use explicit timeouts in the metadata exporter
  610. * [Fix] metric_exporter: avoid sending `nil` in metric values
  611. * [Project] Add GPT plugin
  612. * [Project] Add a tool to perform statistical analysis of classifiers
  613. * [Project] Finally fix slow timer behaviour
  614. * [Project] Implement fuzzy check retransmits backpressure
  615. * [Project] Improve stats processing
  616. * [Project] Reduce default window size of OSB tokenizer to 2
  617. * [Project] Rework grow factor
  618. * [Project] Start optimization of userdata hashing
  619. * [Project] Use libarchive for troublesome archives
  620. * [Rework] Allow metrics endpoint to be enabled for each scanner
  621. * [Rework] Allow multiple conditions for dkim signing when using vault/redis
  622. * [Rework] Introduce dynamic_rate_limit for ratelimits
  623. * [Rework] Serialize control commands
  624. * [Rework] Update fmt to version 11
  625. * [Rework] Use direct hashing instead of string hashing
  626. * [Rules] Added rules for detecting likely malware
  627. 3.8.2: 20 Feb 2024
  628. * [Feature] Add extraction type for `from` maps
  629. * [Feature] Allow to add templates to redis history prefix
  630. * [Feature] Implement dynamic keys map in fuzzy storage
  631. * [Feature] Lua_url: Add `to_http` method
  632. * [Feature] Support JSON logging when in syslog mode
  633. * [Fix] Deal with `Connection` and `Host` headers on proxying
  634. * [Fix] Encode headers in metadata exporter
  635. * [Fix] Fix initial maps load
  636. * [Fix] Make stat tokens allocation consistent
  637. * [Fix] Resolve issue with bayes stat in `rspamadm` mode
  638. * [Fix] Try to fix url path issue
  639. * [Rework] Breaking: Do not report module as action
  640. * [Rework] Use khash instead of glib hashes for many reasons
  641. 3.8.1: 25 Jan 2024
  642. * [Fix] Fix headers insertion in the ordered list
  643. * [Fix] Fix learn error propagation
  644. * [Fix] Fix learning with long prefixes
  645. * [Fix] Fix potential double free by expclicitly zeroing pointers
  646. * [Fix] Fix storing of the bayes tokens
  647. * [Fix] Use short comments rather than long one
  648. 3.8.0: 19 Jan 2024
  649. * [Project] Rspamadm fuzzyping command
  650. * [Project] Use Redis scripts for Bayes statistics and cache
  651. * [Project] Support JSON logging
  652. * [CritFix] - Fix reported length of logging structure
  653. * [Feature] Allow to set `max_users` for Redis bayes backend
  654. * [Feature] Escape JSON when needed
  655. * [Feature] Proxy: Allow `encrypted_only` option
  656. * [Feature] Reiterate on gtube patterns
  657. * [Feature] Support ping command in fuzzy storage
  658. * [Feature] Support suppressing DMARC reporting for particular recipients
  659. * [Feature] rbl: support disabling or replacing url_whitelist per RBL
  660. * [Fix] Another try to fix setproctitle
  661. * [Fix] Cleanup session on exit
  662. * [Fix] Do not cleanup hyperscan files unless new ones are loaded
  663. * [Fix] Fix bad access when printing pending events on failure path
  664. * [Fix] Fix issues with the raw header on header insertion/modification
  665. * [Fix] Fix overflow in fuzzy_stats command
  666. * [Fix] Fix some corner cases when parsing maps with no newlines
  667. * [Fix] Fix various issues with canonicalisation of the paths
  668. * [Fix] Fix wrong raw_len usage
  669. * [Fix] Make words selection random deterministic upon content
  670. * [Fix] Properly set config field when creating tasks from Lua
  671. * [Fix] Really fix the language detector statistical heuristic
  672. * [Fix] Set loaded variable explicitly
  673. * [Fix] dkim_signing: siging_table: lowercase before lookup
  674. * [Fix] known_senders: fix config handling
  675. * [Fix] rbl: fix `exclude_local`
  676. 3.7.5: 15 Dec 2023
  677. * Add support for loongarch
  678. * [Fix] Fix rspamadm lua logger
  679. * [Fix] Fix systemd logging
  680. * [Fix] dkim_signing: siging_table: lowercase before lookup
  681. * [Minor] Fix build with no hyperscan
  682. * [Minor] Try improve test stability
  683. 3.7.4: 14 Nov 2023
  684. * [Enhancement] Add composite rule for suspicious URLs in suspicious messages
  685. * [Fix] Another try to fix setproctitle
  686. * [Fix] Do not cleanup hyperscan files unless new ones are loaded
  687. * [Fix] Fix various issues with canonicalisation of the paths
  688. * [Fix] Properly set config field when creating tasks from Lua
  689. * [Fix] Set loaded variable explicitly
  690. * [Fix] known_senders: fix config handling
  691. * [Fix] rbl: fix `exclude_local`
  692. * [Minor] Add missing <algorithm> include for gcc 14
  693. * [Minor] Add rule for messages missing both X-Mailer and User-Agent header
  694. * [Minor] Bundle fasttext as linux distributives are just useless
  695. * [Minor] Enable fasttext on RPM based linux
  696. * [Minor] Forgot it in one more place
  697. * [Minor] Improve FREEMAIL_AFF capture rates
  698. * [Minor] Limit fasttext to amd64 only
  699. * [Minor] Properly search for fasttext include
  700. * [Minor] Treat *.zpaq attachments as archives and harmful
  701. * [Minor] Update to 3.7.4
  702. * [Minor] force_actions: set a group for symbols
  703. * [WebUI] Update map editor
  704. 3.7.3: 27 Oct 2023
  705. * [Fix] Emergency fix for the hyperscan path error
  706. 3.7.2: 26 Oct 2023
  707. * [Feature] rbl: support checking returncodes by CIDR
  708. * [Feature] rbl: support checking returncodes by regex
  709. * [Feature] rbl: support globbed return codes
  710. * [Fix] DMARC reporting: fix reporting for subdomains
  711. * [Fix] Deal with fmtlib exceptions properly
  712. * [Fix] backport fix for dlfcn.h from backward-cpp
  713. * [Rules] Blank spam detection
  714. 3.7.1: 07 Oct 2023
  715. * [CritFix] Fix leak in `gzip` function
  716. * [Feature] Add ICAP Content-Type and Filename
  717. * [Feature] Add `logging`->`task_max_elts` option
  718. * [Feature] Add utility to split string like stuff for C++ code
  719. * [Feature] Allow to set HTTP auth parameters for the maps
  720. * [Feature] Check for plugin configuration errors on `configtest`
  721. * [Feature] `known_senders` plugin
  722. * [Feature] Use backward-cpp instead of manual libunwind stuff
  723. * [Feature] rbl: support checking numeric URLs in isolation
  724. * [Fix] CMakeLists.txt remove whitespace added by linter as it makes tests fail
  725. * [Fix] Change Date: header location to conform with RFC
  726. * [Fix] Correct format pattern for RE tree tempfile name
  727. * [Fix] Correct format string for unw_word_t
  728. * [Fix] Do not accept invalid ucl object types
  729. * [Fix] Do not pollute public headers with libev internals
  730. * [Fix] Do not set output type if list application failed
  731. * [Fix] Fix `url:set_redirected` method
  732. * [Fix] Fix format string and some length issues
  733. * [Fix] Fix grammar definition for content-disposition attributes
  734. * [Fix] Fix lua schema enrichment logic for Redis params
  735. * [Fix] Fix lua stack corruption when logging large tables
  736. * [Fix] Fix merge table utility
  737. * [Fix] Fix output of non-RSA DKIM keys
  738. * [Fix] Fix some corner cases of single-host urls parsing
  739. * [Fix] Fix various issues in the `url_redirector` plugin
  740. * [Fix] MISSING_MIMEOLE: avoid matching messages from Android GMail app (#4561)
  741. * [Fix] Prevent DNSWL sabotage
  742. * [Fix] Try to fix unzip function
  743. * [Fix] rbl: really fix dependency registration when symbols_prefixes is used
  744. * [Fix] rspamadm mime: arguments beginning with letter `t`
  745. * [Rework] Breaking: return back to semver
  746. * [Rework] Move rcl logic to C++
  747. 3.6: 03 Aug 2023
  748. * [Conf] Add `one_shot` to some specific multimap rules
  749. * [Conf] Add language detection configuration
  750. * [Conf] Add missing attributes for the language detection configuration
  751. * [Conf] Remove outdated composite rules
  752. * [Feature] Add `sentinel_password` option
  753. * [Feature] Add ability to deny specific fuzzy flags by default
  754. * [Feature] Add controller endpoint to get fuzzy hashes from messages
  755. * [Feature] Add extra symbol when URL redirector reaches nested limit
  756. * [Feature] Add function to transliterate utf8 to ascii with some normalisation
  757. * [Feature] Add html parsing limit
  758. * [Feature] Add order to urls structure
  759. * [Feature] Add some missing functions to `lua_rsa` library
  760. * [Feature] Allow fuzzy workers to exchange blocked information
  761. * [Feature] Allow to have weak flags in fuzzy storage
  762. * [Feature] Allow to read options from maps in the multimap plugin
  763. * [Feature] Allow to use other methods when fasttext detection is enabled
  764. * [Feature] Count stats per key per flag
  765. * [Feature] Finish all features of dkim_keygen in Lua
  766. * [Feature] Khash: Allow static initialisation
  767. * [Feature] Maps: Add on_load support
  768. * [Feature] Preliminary implementation of dynamic composites
  769. * [Feature] Process HTML parts before text ones
  770. * [Feature] Reorganise struct rspamd_url to be 64 bytes size
  771. * [Feature] Save fuzzy ratelimit buckets
  772. * [Feature] Use in-place deflation for strings
  773. * [Feature] external_relay: add ip_map strategy
  774. * [Fix] Avoid race between config new/free by using a counter
  775. * [Fix] Do not use `rspamadm.dkim_keygen`
  776. * [Fix] Feed fasttext language model with the pre-tokenized words
  777. * [Fix] Fix `rspamd_has_only_html_part`
  778. * [Fix] Fix an old issue with order of destruction race between redis pool and lua
  779. * [Fix] Fix format string usage
  780. * [Fix] Fix parsing due to old bug revealed
  781. * [Fix] Fix parsing of the mask values that are invalid
  782. * [Fix] Ignore non-unique stop words
  783. * [Fix] Include the last character when parsing the last header with no value
  784. * [Fix] More fixes to fuzzystat
  785. * [Fix] Set proper counter
  786. * [Fix] Try harder to clean pending bucket
  787. * [Fix] Try harder to remove bad hyperscan files
  788. * [Fix] Update stats before encryption...
  789. * [Fix] dmarc gramar - allow spaces before ";"
  790. * [Fix] rbl: fix dependency registration when symbols_prefixes is used
  791. * [Fix] remove obsolete rspamd-redirector files
  792. * [Fix] test external_relay: count should always be the last rule, as it have no matching condition
  793. * [Project] Allow to register multimap symbols dynamically
  794. * [Project] Implement fasttext language detection
  795. * [Rework] Default max shots must not influence options
  796. * [Rework] Write dkim keygen tool in lua
  797. * [Rules] Add thread hijacking composite rule
  798. 3.5: 17 Mar 2023
  799. * [Conf] Clarify `timeout` in rspamd_proxy
  800. * [Conf] Fix hashbl
  801. * [Conf] Add SURBL hashbl support (WIP)
  802. * [Conf] RBL: Fix selector
  803. * [Conf] Reduce the default timeout
  804. * [CritFix] Deserialise hyperscan to the page-aligned space to prevent alignment issues
  805. * [CritFix] Fill path field in hyperscan notice command
  806. * [Feature] Add `thresholds` field to the scan result
  807. * [Feature] Add possibility to execute lua scripts for blocked fuzzy clients
  808. * [Feature] Add preliminary support of the external maps in the multimap plugin
  809. * [Feature] Allow to build a map by combining tuples of selectors
  810. * [Feature] Allow to query external maps for settings
  811. * [Feature] Allow to specify `selector_alias` in the maps definition
  812. * [Feature] Enable Mime part filters on antivirus module
  813. * [Feature] Improve ratelimit redis scripts
  814. * [Feature] Selectors: Add specific_urls_filter_map extractor
  815. * [Feature] Some rework of the selectors framework
  816. * [Fix] Add O_CREAT flag when creating a file
  817. * [Fix] Add a database check function unless we have anything from Hyperscan
  818. * [Fix] Add hyperscan databases sanity check
  819. * [Fix] Add workaround for ENOBUFS error on sending
  820. * [Fix] Do not do `lstat` when we are creating file
  821. * [Fix] Finally get rid of RSPAMD_USE_47BIT_LIGHTUSERDATA_HACK
  822. * [Fix] Fix boundaries that contain only dashes
  823. * [Fix] Fix off-by-one error in css tokenizer
  824. * [Fix] Fix url reputation plugin
  825. * [Fix] Fix usage of the Redis config schema as `extra_fields`
  826. * [Fix] Further checks for the hs_scratch_alloc
  827. * [Fix] Honor group flag for one shot
  828. * [Fix] Normalize glob paths to avoid hash table misses
  829. * [Fix] Ratelimit: Use unpack for `HMGET` return value
  830. * [Fix] Rbl: Fix helo check pipeline
  831. * [Fix] Replace broken strict_domains with phishing_exceptions
  832. * [Fix] Restore `strict_domains` support
  833. * [Fix] Return true from has_urls(true) if only emails are present
  834. * [Fix] Rework lists applications
  835. * [Fix] Set symcache item in coroutine calls
  836. * [Fix] Treat `hs_allocate_scratch` errors as non-fatal
  837. * [Fix] Treat hostnames with no dots as eSLD of their own
  838. * [Fix] received: filtering of artificial header
  839. * [Project] Add external maps support
  840. * [Rework] Breaking: Do not report soft reject in history
  841. * [Rework] Convert chartable plugin to c++ for convenience
  842. * [Rework] Use a different approach for customization of the settings
  843. * [Rules] Mid: Add MID_END_EQ_FROM_USER_PART rule
  844. 3.4: 01 Nov 2022
  845. * [CritFix] Restore compatibility with the integrations and headers alterations
  846. * [Feature] Milter_headers: Add `x-rspamd-action` routine
  847. * [Feature] Share hyperscan database among processes
  848. * [Fix] Another corner case in url parsing
  849. * [Fix] Another fix for the enable password
  850. * [Fix] Another try to fix close method in lua_tcp
  851. * [Fix] Fix additional fields in the Redis schema
  852. * [Fix] Fix emoji joiner FP
  853. * [Fix] Fix favicon.ico Content-Type header
  854. * [Fix] Fix hang when close is used
  855. * [Fix] Lua_tcp: Sigh, another try to fix `close` invocation
  856. * [Fix] Mx_check: Cache the fact of a missing MX record
  857. * [Fix] Try to fix parsing of the unencoded `>` characters in html attributes
  858. * [Fix] Try to fix the case where password == enable_password
  859. * [Project] (Re)implement hyperscan caching
  860. * [Project] Rework cleanup
  861. * [Project] Synchronize hyperscan caches via the main process
  862. * [Rework] Convert multipattern to use hyperscan tools
  863. * [Rework] Make http normalize path function a generic function
  864. * [Rework] Split locked and unlocked files, as mmap does not need flock normally
  865. * [Rework] Start movement of the hyperscan related routines into a single unit
  866. * [Rework] Store the current worker, so other libraries could use this information
  867. * [Rework] Use blocking socket for IPC between main and workers
  868. * [Rework] Use more predictable size for commands buffers
  869. * [Rules] Do not insert ONCE_RECEIVED_STRICT on RDNS missing
  870. * [Rules] Reduce score of HTTP_TO_HTTPS - subject to remove completely
  871. 3.3: 01 Oct 2022
  872. * [Conf] Add missing groups for whitelist module symbols
  873. * [CritFix] Neural: Fix keys regression after #3968
  874. * [Feature] Accept upstream in lua_tcp
  875. * [Feature] Add ability to statically maintain disabled/enabled patterns
  876. * [Feature] Add function to store upstreams for HTTP urls
  877. * [Feature] Allow augmentations set in Lua API
  878. * [Feature] Allow lua_http module to accept upstreams
  879. * [Feature] Allow to limit write access to fuzzy storage by key
  880. * [Feature] Allow to sort symbols output
  881. * [Feature] Check content for binary stuff before dumping it to Lua
  882. * [Feature] Implement symbols augmentations
  883. * [Fix] Add missing flags
  884. * [Fix] Add more sanity checks for rua in dmarc_report
  885. * [Fix] Adjust length of the fuzzy checks for short text parts
  886. * [Fix] Another try to fix add headers compatibility logic
  887. * [Fix] Another try to fix race condition in the runtime destruction
  888. * [Fix] Avoid cyclic references in symcache and fix memory leaks
  889. * [Fix] Avoid overriding IP with Sender IP
  890. * [Fix] BAD_REP_POLICIES did not trigger when message was classified as spam by Bayes
  891. * [Fix] Bind AF_UNIX DGRAM client connection to annonymous address
  892. * [Fix] Disable IPv6 lookups for Blocklist.de RBL
  893. * [Fix] Distinguish dynamic and static items
  894. * [Fix] Dkim: Ignore unknown DKIM kv pairs as stated in RFC
  895. * [Fix] Dmarc report: Use local timezone instead of GMT
  896. * [Fix] Do not exclude authenticated users from URIBL lookups
  897. * [Fix] Empty envelopes should not be emitted as arrays (json+messagepack) when populated envelopes are objects. This greatly complicates decoding in strictly typed languages.
  898. * [Fix] External_relay: Restore the originating hostname check
  899. * [Fix] Fix DKIM keys with spaces still allowing errors on invalid base64
  900. * [Fix] Fix copying of sockaddr_un addresses
  901. * [Fix] Fix crash with cname replies
  902. * [Fix] Fix dependencies propagation
  903. * [Fix] Fix iteration over milter headers
  904. * [Fix] Fix ordering when sorting symcache
  905. * [Fix] Fix reading of the cached maps
  906. * [Fix] Fix several issues with the HTTP keepalive parsing
  907. * [Fix] Fix stack smashing
  908. * [Fix] Fix synchronous auth/select in lua_redis
  909. * [Fix] Fix various symcache issues
  910. * [Fix] Ignore all (I hope) unknown DKIM signature KV pairs
  911. * [Fix] Ignore directories in RarV5 archives
  912. * [Fix] Libucl: avoid memory leak on objects merging
  913. * [Fix] Lua_tcp: Another try to fix closing logic
  914. * [Fix] Mempool: Fix alloc_array function to actually multiply nmembers by size
  915. * [Fix] Only check allowed fuzzy worker update ips for non-unix sockets
  916. * [Fix] Plug memory leak in regexp destruction with pcre2
  917. * [Fix] Properly check the original email flag
  918. * [Fix] Properly deal with `get_symbol/get_metric_symbol` ambiguity
  919. * [Fix] Properly parse expressions atoms
  920. * [Fix] Properly set `Host` in rspamd_proxy
  921. * [Fix] Rbl: Fix received positioned checks
  922. * [Fix] Remove check for a score with no symbol being registered
  923. * [Fix] Same fix for lua_tcp
  924. * [Fix] Skip cname records when processing SPF records
  925. * [Fix] Skip sending dmarc reports in no-opt mode fixes https://github.com/rspamd/rspamd/issues/4241
  926. * [Fix] Stop slow timer on task destruction
  927. * [Fix] Symcache: Do not use C style comparators in C++ sorts
  928. * [Fix] Try to avoid a corner case for `@` pattern
  929. * [Fix] Try to fix dkim reputation adjustements
  930. * [Fix] Try to fix passthrough results processing logic
  931. * [Fix] Try to fix the mess with read only flag
  932. * [Fix] Upstreams: Don't ignore revive_time config option
  933. * [Fix] Use proper format string, sigh...
  934. * [Fix] Use space category in ragel automata to resolve space characters
  935. * [Fix] Zstd: Fix compression with the new Zstd API
  936. * [Fix] milter_headers: Header fields may be inserted at wrong position.
  937. * [Project] Add experimental HTTP statistics backend
  938. * [Project] Add more methods for symbols addition
  939. * [Project] Add raii_sink file helper
  940. * [Project] Add some more methods
  941. * [Project] Add symbols processing methods
  942. * [Project] Allow `=` separated augmentations to be treated as kv pairs
  943. * [Project] Allow to extract augmentation values
  944. * [Project] Few more methods
  945. * [Project] Fix on conditions
  946. * [Project] Further efforts to make a more consistent architecture
  947. * [Project] Further rework
  948. * [Project] Further rework tracking
  949. * [Project] Further split of the code
  950. * [Project] Get rid of C style ctors/dtors
  951. * [Project] Http_stat: Notice statfiles when creating runtime
  952. * [Project] Implement dynamic items lookup and processing
  953. * [Project] Implement item finalization
  954. * [Project] Implement more methods
  955. * [Project] Implement runtime creation
  956. * [Project] Implement settings processing + some neats
  957. * [Project] Implement some conditions checks
  958. * [Project] Implement validation logic
  959. * [Project] More methods
  960. * [Project] Move runtime cache part to a separate unit
  961. * [Project] Move some more methods
  962. * [Project] Re-implement counters method
  963. * [Project] Reimplement dependencies processing
  964. * [Project] Remove obsoleted methods
  965. * [Project] Remove old code (finally)
  966. * [Project] Rework symbols execution
  967. * [Project] Some more adjustments in symbols registration
  968. * [Project] Start rewrite symcache in c++
  969. * [Project] Support augmentations with values
  970. * [Project] Symcache: Use ordered filters to avoid extra lookups
  971. * [Rework] Another movement
  972. * [Rework] Augmentations can now imply flags
  973. * [Rework] Further steps
  974. * [Rework] Further work on deps processing
  975. * [Rework] Implement cache resorting
  976. * [Rework] Isolate disable/enable logic for the configuration ucl objects
  977. * [Rework] Move item implementation to a separate header
  978. * [Rework] Multimap: Avoid prefilters usage where augmentations can be used
  979. * [Rework] Pass upstream when sending TCP requests
  980. * [Rework] Re-implement cache sorting
  981. * [Rework] Reimplement saving/loading the cache items
  982. * [Rework] Reiterate on priorities
  983. * [Rework] Rework files structure
  984. * [Rework] Rewrite rspamc in C++
  985. * [Rework] Simplify scores check and extend it to pre/post filters
  986. * [Rework] Switch minimum C++ standard version to C++20
  987. * [Rework] Try to fix the mess with types & flags
  988. * [Rework] Use another version of hash table from the same author
  989. * [Rework] Use dynamic items for calling callbacks
  990. * [Rework] Use dynamic items in the callbacks
  991. * [Rework] Use hash map for id->symbol mappings
  992. * [Rework] Use khash instead of uthash in rdns compression logic
  993. 3.2: 26 Mar 2022
  994. * [Conf] Score MIME_OBFUSCATED_ARCHIVE to 8 points
  995. * [Conf] Set one_shot for URIBL rules by default
  996. * [CritFix] Fix upstreams name resolution when there is also a port
  997. * [Feature] Add ROC feature to neural network plugin
  998. * [Feature] Add public suffic compilation utility
  999. * [Feature] Add support of Cloudmark
  1000. * [Feature] Allow hyperscan for ppc64, as vectorscan now suports it.
  1001. * [Feature] Allow to skip DNS resolution for keep-alive connections
  1002. * [Feature] Aws_s3: Allow to store large parts separately
  1003. * [Feature] BIMI: Add preliminary version of the BIMI plugin
  1004. * [Feature] JSON endpoint for querying maps
  1005. * [Feature] Lua_magic: Add a sane CSV heuristic
  1006. * [Feature] Lua_mime: Add schema for message transfer
  1007. * [Feature] Output average scan time in /stat endpoint
  1008. * [Feature] Show average scan time in `rspamc stat` output
  1009. * [Fix] Add guards to avoid race condition on TCP connection
  1010. * [Fix] Allow spaces in DKIM key records
  1011. * [Fix] Apply the similar fix to the url_reputation
  1012. * [Fix] Avoid overwriting whitelisted_signers_map
  1013. * [Fix] Backport PR from libucl
  1014. * [Fix] Clear SSL errors
  1015. * [Fix] ClickHouse cleanup of old partitions
  1016. * [Fix] Do not double call error handler on ssl errors in the timeout path
  1017. * [Fix] Do not forget to clear pointers on IOC reset
  1018. * [Fix] External_relay: Remove useless check of the map value
  1019. * [Fix] Find suspicious url encodings that could break url extraction
  1020. * [Fix] Fix HTTP(s) client timeout
  1021. * [Fix] Fix exclude flags setting
  1022. * [Fix] Fix expanding of the variables
  1023. * [Fix] Fix host header usage in lua_http
  1024. * [Fix] Fix http maps shared memory cache cleanup
  1025. * [Fix] Fix logic in HTML processing FSM
  1026. * [Fix] Fix parsing of the compound mailto urls
  1027. * [Fix] Fix processing captures from pcre2
  1028. * [Fix] Fix removing from khash
  1029. * [Fix] Fix stuctured headers pushing
  1030. * [Fix] Further fix for i386 compilation
  1031. * [Fix] Improve duplicate settings error reporting
  1032. * [Fix] Lua: task:remove_result didn't work in some cases
  1033. * [Fix] Output service parts as well
  1034. * [Fix] Phishing: Deal with phishing + redirected URL
  1035. * [Fix] Phishing: Fix finding domains in the phishing map
  1036. * [Fix] Plug memory leak by using mempool for a copied address
  1037. * [Fix] Properly find the request and the number of requested entries
  1038. * [Fix] Rbl: Fix inversed logic of the url_full_hostname
  1039. * [Fix] Read file maps if they were not pre-read during preload
  1040. * [Fix] Restrict x86_64 assembly to x86_64
  1041. * [Fix] Return a real number of recipients when dealing with aliases
  1042. * [Fix] Rework unshedule DNS request function
  1043. * [Fix] Support definition of ungrouped symbol in conf file, use group info from lua or other conf file
  1044. * [Fix] Unschedule DNS request when clearing IO channel
  1045. * [Fix] When checking for phishing, we need to convert punicode -> UTF8, not vice versa
  1046. * [Fix] lua_cfg_transform - actions without score (discard)
  1047. * [Fix] lua_cfg_transform - silly break break actions
  1048. * [Fix] ratelimit - symbol per bucket
  1049. * [Project] BIMI: Fix helper integration issues
  1050. * [Project] Further DNS over TCP architecturing
  1051. * [Project] Rdns: Add more functions for TCP based requests
  1052. * [Project] Rdns: Add preliminary reading logic for TCP channels
  1053. * [Project] Rdns: Add reaper for inactive TCP connections
  1054. * [Project] Rdns: Add timeout logic for TCP requests
  1055. * [Project] Rdns: Do not treat TCP channels failure as fatal
  1056. * [Project] Rdns: Fix TCP connection mess
  1057. * [Project] Rdns: Fix TCP stuff cleanup
  1058. * [Project] Rdns: Fix various ownership issues
  1059. * [Project] Rdns: Implement TCP writing logic
  1060. * [Project] Rdns: Initial support of TCP IO channels
  1061. * [Project] Rdns: More fixes in TCP handling
  1062. * [Project] Rdns: Restore the previous EDNS0 size
  1063. * [Project] Rdns: Send truncated replies via TCP
  1064. * [Project] Rdns: Unregister TCP requests
  1065. * [Rework] Allow to restore SSL handlers after keepalive pooling
  1066. * [Rework] Allow to set a different behaviour for actions from settings
  1067. * [Rework] Include SSL flag into keepalive hash
  1068. * [Rework] Make `rspamadm dmarc_report` default behaviour more sane
  1069. * [Rework] Mempool: Use explicit alignment
  1070. * [Rework] Rdns: Use faster and more compact hash table for DNS requests
  1071. * [Rework] Rework SSL flag operations
  1072. * [Rework] Take disabled flag into account
  1073. * [Rework] Timeouts are now global per event and not reseted by IO activity
  1074. * [Rework] Use xxh3 as a default hash and fix memory/alignment issues
  1075. * [Rules] Fix old rules to stop global functions usage
  1076. * [Rules] Fix symbol for DKIM temporary failure
  1077. * [Rules] Remove ancient and inefficient rules
  1078. * [Rules] Slightly reduce MULTIPLE_FROM score
  1079. 3.1: 01 Nov 2021
  1080. * [Feature] Add junk_threshold for autolearn
  1081. * [Feature] Add neural test command
  1082. * [Feature] Antivirus: Allow to set fake eicar patterns for testing AV engines
  1083. * [Feature] Lua_cdb: Add cdb building interface
  1084. * [Feature] Ratelimit: Add per bucket configurations
  1085. * [Feature] S3: Allow to store structured data in messagepack
  1086. * [Fix] Add concept of uncancellable events to prevent use-after-free
  1087. * [Fix] Add temporary guard to prevent linked list exploitation
  1088. * [Fix] Another rework of the ucl hashing
  1089. * [Fix] Another try to fix references safety
  1090. * [Fix] Another try to fix rspamd_text passing in the selectors
  1091. * [Fix] Avoid copy for received structure as it has raw C pointers
  1092. * [Fix] Avoid dangling reference
  1093. * [Fix] Correctly check numeric URLs in URL DNS lists
  1094. * [Fix] Delete the correct pointer type
  1095. * [Fix] Dmarc: Always lowercase domain
  1096. * [Fix] Fix compilation of the hyperscan databases with errors
  1097. * [Fix] Fix hash table lookup
  1098. * [Fix] Fix http message flag shift
  1099. * [Fix] Fix parsing of the from_hostname when it is an IP address
  1100. * [Fix] Fix parsing of the unquoted attributes in HTML
  1101. * [Fix] Fix passing of rspamd_text in selectors pipelines
  1102. * [Fix] Fix rubbish QP sequences decoding
  1103. * [Fix] Fix some complicated case with the closing tags parsing
  1104. * [Fix] Fix the case when l tag is too small
  1105. * [Fix] Html: Fix the case where only bgcolor is explicitly set
  1106. * [Fix] Libucl: Fix deletion from ucl objects
  1107. * [Fix] Namespace and add metadata for OpenMetrics, fix interleaving
  1108. * [Fix] Plug memory leak in http settings reload
  1109. * [Fix] Preserve SPF top record in the mempool variable
  1110. * [Fix] Remove aarch64 GC64 workaround
  1111. * [Fix] Remove bogus G_LIKELY
  1112. * [Fix] Spf: Do not parse non TXT DNS replies as TXT replies
  1113. * [Fix] Try to use on_connect/on_disconnect callbacks to handle internal Redis failures
  1114. * [Fix] buffer overflow in rspamc counters
  1115. * [Fix] fix static building
  1116. * [Fix] lua_scanners - message_min_words logic
  1117. * [Fix] src/lua/lua_mimepart.c: fix null dereference
  1118. * [Project] Add constant iterators
  1119. * [Project] Add helper library to handle mime strings in a more safe matter
  1120. * [Project] Add preliminary support of CDB bayes dump
  1121. * [Project] Add trim operations
  1122. * [Project] Allow mempool allocated mime strings
  1123. * [Project] Cdb: Finish backend implementation
  1124. * [Project] Cdb: Fix configuration load
  1125. * [Project] Cdb: Use shared data between cdb statfiles
  1126. * [Project] Cdb: continue statistics backend implementation
  1127. * [Project] Finish received headers rework part
  1128. * [Project] Move C++ specific declarations to C++ header
  1129. * [Project] Rework received headers parsing to C++
  1130. * [Project] Start using of the new received structure
  1131. * [Project] Start work on cdb backend
  1132. * [Rework] Further rework of the redis pool
  1133. * [Rework] Redis_pool: fix issues found
  1134. * [Rework] Rework learn and add classify condition
  1135. * [Rework] Save invisible content to a separate buffer
  1136. * [Rework] Start rewriting of the redis pool logic
  1137. * [Rules] Improve zero font rule
  1138. 3.0: 19 Aug 2021
  1139. * [Conf] Align ARC scores with DKIM scores
  1140. * [CritFix] Neural: Fix sorting application
  1141. * [Feature] Add a simple dumper for bayes tokens
  1142. * [Feature] Add lua_maps.fill_config_maps function
  1143. * [Feature] Add preliminary exporter to AWS S3
  1144. * [Feature] Add preliminary restore bayes support
  1145. * [Feature] Add race condition protection against hs_helper restarts
  1146. * [Feature] Add rspamd_utf8_strcmp utility
  1147. * [Feature] Add zstd streaming API
  1148. * [Feature] Allow to log severity level explicitly
  1149. * [Feature] Allow to save and show attachment name when inserting AV scan results
  1150. * [Feature] Allow to sort urls for Lua
  1151. * [Feature] Allow to specify different timeouts/retransmits for fuzzy rules
  1152. * [Feature] Aws_s3: Allow to compress data stored
  1153. * [Feature] CMakeLists.txt: Change check and run-test to use rspamd-test-cxx * fixes #3807
  1154. * [Feature] Dmarc_report: allow sending reports in batches
  1155. * [Feature] Fuzzy_check: Allow to disable subject when making short text hash
  1156. * [Feature] Lua_cryptobox: Add keyed ssl hash functions via HMAC
  1157. * [Feature] Lua_task: Add get_urls_filtered method
  1158. * [Feature] Make monitored checks less frequent
  1159. * [Feature] Milter_headers: Add x-rspamd-pre-result header
  1160. * [Feature] Neural: Allow to balance FP/FN for the network
  1161. * [Feature] Ppopagate monitored errors from rbl module
  1162. * [Feature] Pyzor calculate score dynamically Count - WL-Count of default_score in percent
  1163. * [Feature] Rbl: Distinguish flattened and non-flattened selectors in RBL requests
  1164. * [Feature] Re-add pyzor support
  1165. * [Feature] Settings: add ip_map check and rework structure slightly
  1166. * [Feature] Spamassassin: Allow to set the default priority for SA scores
  1167. * [Feature] Strip smtp comments from message id
  1168. * [Feature] add SYSTEM_ZSTD cmake option To use the system zstd instead on the bundled version
  1169. * [Feature] external_relay plugin
  1170. * [Feature] rspamadm clickhouse neural_train subcommand
  1171. * [Fix] #3400 milter_headers: fix inverted logic for extended_headers_rcpt
  1172. * [Fix] ASN: fix _FAIL symbol for when main symbol is disabled
  1173. * [Fix] Add a special logic for text part with no text extraction
  1174. * [Fix] Add diacritics flag for several eu languages
  1175. * [Fix] Another FSM fix to accomodate possibility of multiple consequent ?
  1176. * [Fix] Avoid curse of dynamic array referencing
  1177. * [Fix] Avoid reinitialising neural settings
  1178. * [Fix] Check remain before processing TXT records
  1179. * [Fix] Enable error multiplier on http errors
  1180. * [Fix] Finally rework parsing entities logic
  1181. * [Fix] Fix '==' parsing in the content type attributes parser
  1182. * [Fix] Fix IPv6 expansion for SPF macros
  1183. * [Fix] Fix Mozilla Message-ID detection
  1184. * [Fix] Fix an edge case in BITCOIN_ADDR rule
  1185. * [Fix] Fix brain-damaged behaviour when http request has a custom Host header
  1186. * [Fix] Fix check of limits in email address parsing
  1187. * [Fix] Fix copy&paste error and rework
  1188. * [Fix] Fix expressions logic for and/or and float values
  1189. * [Fix] Fix fuzzy retransmits
  1190. * [Fix] Fix http maps with no or invalid expires data
  1191. * [Fix] Fix last quote character parsing in the content-type state machine
  1192. * [Fix] Fix normalisation flags propagation
  1193. * [Fix] Fix overflow when appending many broken tags
  1194. * [Fix] Fix parsing of rfc2047 tokens with '?' inside
  1195. * [Fix] Fix phishing flag set
  1196. * [Fix] Fix rfc2047 embedded into rfc2231 pieces in special headers
  1197. * [Fix] Fix round-robin rotation
  1198. * [Fix] Fix searching for symbols
  1199. * [Fix] Fix storing of the regexps inside variant
  1200. * [Fix] Fix tokenization near exceptions
  1201. * [Fix] Fix visibility calculations
  1202. * [Fix] Html: Attach inline tags to the structure
  1203. * [Fix] Html: Do not treat empty tags as block tags
  1204. * [Fix] Ical: Do not extract urls from all flags using merely specific ones
  1205. * [Fix] Initialise symcache even if it cannot be loaded properly
  1206. * [Fix] Lua_fuzzy: Remove text parts check when checking image dimensions
  1207. * [Fix] Lua_maps: Fix adjustments for the map type in the complex map definitions
  1208. * [Fix] Lua_task: Fix deleted symbols in has_symbol/get_symbol
  1209. * [Fix] Move metric and symcache link from validation to the init stage
  1210. * [Fix] Oletools: Another try to fix table sorting
  1211. * [Fix] One more default behaviour fix
  1212. * [Fix] Phishing: Rework urls processing
  1213. * [Fix] RBL: was missing some config schema
  1214. * [Fix] Replies: Fix 'Reply-To' handling in task:get_reply_sender
  1215. * [Fix] Rework metrics handling
  1216. * [Fix] Save symcache on exit
  1217. * [Fix] Selectors: Filter nil elements in lists
  1218. * [Fix] Selectors: Properly fix implicit tostring for nils
  1219. * [Fix] Try to fix some broken code in DMARC reporting plugin
  1220. * [Fix] Urls: Fix processing of html urls when it comes to the flags
  1221. * [Fix] Use proper buffer length
  1222. * [Fix] Various visibility fixes
  1223. * [Fix]: ASN: dns cb func should also return in case of an error
  1224. * [Project] Add a simple css rule definition
  1225. * [Project] Add css style skeleton
  1226. * [Project] Add css syntax (adopted from ebnf)
  1227. * [Project] Add css_selectors
  1228. * [Project] Add doctest unit testing library
  1229. * [Project] Add expected library
  1230. * [Project] Add fmt library for simple string ops
  1231. * [Project] Add fu2 library to better functions abstractions
  1232. * [Project] Add hashing method
  1233. * [Project] Add parsers skeleton
  1234. * [Project] Add preliminary support of vcard parser
  1235. * [Project] Add process exceptions for invisible text
  1236. * [Project] Add some methods for css parser
  1237. * [Project] Allow static libstdc++
  1238. * [Project] Another whitespace hack
  1239. * [Project] CSS: Various fixes in the declarations and values parsing
  1240. * [Project] Cpp: Add robin-hood hash map library
  1241. * [Project] Css: Add AST debug
  1242. * [Project] Css: Add colors conversion functions
  1243. * [Project] Css: Add dimensions handling
  1244. * [Project] Css: Add display value support
  1245. * [Project] Css: Add frozen library from https://github.com/serge-sans-paille/frozen/
  1246. * [Project] Css: Add opacity support
  1247. * [Project] Css: Add parser helpers to simplify debugging
  1248. * [Project] Css: Add preliminary stylesheet support
  1249. * [Project] Css: Add rules processing functions and tests
  1250. * [Project] Css: Add simple selectors unit tests
  1251. * [Project] Css: Add some c++ unit tests
  1252. * [Project] Css: Add some debug methods
  1253. * [Project] Css: Add some debug statements for the css parser
  1254. * [Project] Css: Add some logical skeleton for declarations parser
  1255. * [Project] Css: Add url/function tokens
  1256. * [Project] Css: Allow at rules parsing
  1257. * [Project] Css: Declarations parsing logic skeleton
  1258. * [Project] Css: Enable conditional css parsing support from the HTML parser
  1259. * [Project] Css: Finish generic lexer cases
  1260. * [Project] Css: Fix HSL conversion
  1261. * [Project] Css: Fix minus parsing
  1262. * [Project] Css: Fix parser consumers nesting
  1263. * [Project] Css: Fix parsing of the qualified rules
  1264. * [Project] Css: Fix rules merging
  1265. * [Project] Css: Further fixes to lexer
  1266. * [Project] Css: Further steps to parse css colors + rework
  1267. * [Project] Css: Further work on parser's methods
  1268. * [Project] Css: Implement backlog of css tokens
  1269. * [Project] Css: Implement numbers and ident parsers
  1270. * [Project] Css: Implement simple css selectors lookup
  1271. * [Project] Css: Implement styles merging
  1272. * [Project] Css: Make debug strings json like to simplify tests
  1273. * [Project] Css: Minor adjustments
  1274. * [Project] Css: More meat to the lexer
  1275. * [Project] Css: Move some of the tests to the doctest
  1276. * [Project] Css: Projected a parser
  1277. * [Project] Css: Properties attachment logic
  1278. * [Project] Css: Remove ragel from build targets (maybe keep for reference)
  1279. * [Project] Css: Rework css block structure
  1280. * [Project] Css: Rework flags of css properties
  1281. * [Project] Css: Rework tokens structure
  1282. * [Project] Css: Several fixes + tests
  1283. * [Project] Css: Simplify checks
  1284. * [Project] Css: Simplify debug code
  1285. * [Project] Css: Start css selectors parsing logic
  1286. * [Project] Css: Start semantic parsing for rules
  1287. * [Project] Css: Start stylesheet implementation
  1288. * [Project] Css: Tidy up lambdas
  1289. * [Project] Css: rework tokeniser
  1290. * [Project] Dmarc: Add dmarc report tool (WIP)
  1291. * [Project] Dmarc: Add munging configuration
  1292. * [Project] Dmarc: Add preliminary munging logic
  1293. * [Project] Dmarc: Fix header removal
  1294. * [Project] Dmarc: Fix munging logic
  1295. * [Project] Dmarc: Use full recipient address instead of a domain map
  1296. * [Project] Dmarc: Use zlists for dmarc reports
  1297. * [Project] Dmarc_report: Add message generation logic
  1298. * [Project] Dmarc_report: Add preliminary sending support
  1299. * [Project] Fix lua bindings
  1300. * [Project] Fix xml/sgml tags processing
  1301. * [Project] Handle new modification
  1302. * [Project] Html/CSS: Add transform from a CSS rule to html block
  1303. * [Project] Html/CSS: Link html and css styles
  1304. * [Project] Html/CSS: Switch styles parsing to css parser
  1305. * [Project] Html/Css: Fix some issues found
  1306. * [Project] Html/Css: Implement visibility rules for a block
  1307. * [Project] Html: Add more tests cases and fix some more corner issues
  1308. * [Project] Html: Add rows display type support
  1309. * [Project] Html: Allow decode entities function to normalise spaces + unit tests
  1310. * [Project] Html: Another rework of the tags structure
  1311. * [Project] Html: Another try to fix unbalanced cases
  1312. * [Project] Html: Fix crossing spans
  1313. * [Project] Html: Fix parent propagation
  1314. * [Project] Html: Further rework of the html parsing stuff
  1315. * [Project] Html: Implement logic for tags pairing
  1316. * [Project] Html: Implement rawtext state machine
  1317. * [Project] Html: Insert closing tags as well :(
  1318. * [Project] Html: More fixes
  1319. * [Project] Html: More fixes
  1320. * [Project] Html: More spaces logic fixes
  1321. * [Project] Html: One more attempt to write text content
  1322. * [Project] Html: Replace \0 in html content
  1323. * [Project] Html: Rework img/a tags handling
  1324. * [Project] Html: Rework propagation method
  1325. * [Project] Html: Rework tags placement
  1326. * [Project] Html: Rework transparency logic
  1327. * [Project] Html: Support 'hidden' attribute
  1328. * [Project] Html: Try another approach to append tags content
  1329. * [Project] Html: Try to deal with bad unknown tags properly
  1330. * [Project] Lua_aws: Add canonicalisation utility
  1331. * [Project] Lua_aws: Add function to produce AWS Authorisation header
  1332. * [Project] Lua_aws: Implement request signing
  1333. * [Project] Lua_mime: Add lua_mime.modify_headers routine
  1334. * [Project] Lua_task: Add modify_header method
  1335. * [Project] Lua_task: Allow to extract modified headers
  1336. * [Project] Make unescape code public for unit testing
  1337. * [Project] More fixes for closed tags
  1338. * [Project] More fixes to calculations
  1339. * [Project] Rework API for the modified headers
  1340. * [Project] Rework html visibility rule
  1341. * [Project] Skeleton of the css library
  1342. * [Project] Start headers modification API structure
  1343. * [Project] Start working on AWS Lua API
  1344. * [Project] Use lua_mime to modify headers
  1345. * [Project] Use modified headers on dkim signing
  1346. * [Project] Use string_view to constexpr variant unpacking
  1347. * [Rework] Add composites manager concept
  1348. * [Rework] Add tags definitions
  1349. * [Rework] Allow C code to be compiled with C++ compiler
  1350. * [Rework] Clickhouse: Store url flags
  1351. * [Rework] Composites: Rewrite the composites logic
  1352. * [Rework] Composites: Start rework of the composites framework
  1353. * [Rework] Dmarc: Move check policy function to the common utils
  1354. * [Rework] Dmarc: Rework reports keys structure
  1355. * [Rework] Further work to make html content private
  1356. * [Rework] Html/CSS: Remove css C bindings as they are useless now
  1357. * [Rework] Html/CSS: Rework Lua bindings
  1358. * [Rework] Html/Css: Start rework of the html blocks
  1359. * [Rework] Html: Add images processing logic
  1360. * [Rework] Html: Add traverse function
  1361. * [Rework] Html: Another steps to get rid of gnode
  1362. * [Rework] Html: Convert to variant
  1363. * [Rework] Html: Deal with the utf_content part
  1364. * [Rework] Html: Final rework part for the html processing code
  1365. * [Rework] Html: Fix Lua bindings
  1366. * [Rework] Html: Forgot to add the internal include
  1367. * [Rework] Html: Further html urls rework
  1368. * [Rework] Html: Further rework of the tags content extraction
  1369. * [Rework] Html: Make parameters as a vector again
  1370. * [Rework] Html: Move blocks part
  1371. * [Rework] Html: Move images processing stuff
  1372. * [Rework] Html: Rework lua bindings
  1373. * [Rework] Html: Start html text extraction rework
  1374. * [Rework] Html: Start refactoring of the html tags handling
  1375. * [Rework] Html: Start removing of GNode stuff
  1376. * [Rework] Html: Start rework of the html content structure
  1377. * [Rework] Lua_magic: Try to detect text parts with 8bit characters for non-utf8 encodings
  1378. * [Rework] Move HTML url functions and rework them
  1379. * [Rework] Move and adopt entities handling logic
  1380. * [Rework] Move common and rarely used dmarc code to the library
  1381. * [Rework] Move compression routines outside of rspamd_util library
  1382. * [Rework] Move entities/tags handling
  1383. * [Rework] Phishing: Split from redirectors usage
  1384. * [Rework] Redesign html blocks propagation logic
  1385. * [Rework] Remove tag name string
  1386. * [Rework] Rename phished url to a linked url
  1387. * [Rework] Reorganize dmarc plugin and remove unsupported reporting code
  1388. * [Rework] Reputation: Use more flexible types in get/set functions
  1389. * [Rework] Require proper C++ environment for Rspamd build
  1390. * [Rework] Rework extended urls output
  1391. * [Rework] Rework tags parsing machine
  1392. * [Rework] Slightly improve old regexp API
  1393. * [Rework] Start conversion of the redis pool code to c++
  1394. * [Rework] Try to resolve failed upstreams more agressively
  1395. * [Rework] Use C++ utf8 library with unit tests to trim whitespaces
  1396. * [Rework] Use C++ version for unicode normalisation
  1397. * [Rework] Use C++ version of the lua threads pool
  1398. * [Rules] Add raw addresses to MULTIPLE_FROM options
  1399. * [Rules] Another fix to HTTP_TO_HTTPS rule
  1400. * [Rules] Do not trigger HTML_SHORT_LINK_IMG on external images
  1401. * [Rules] Extend FORGED_X_MAILER
  1402. * [Rules] Extend OLD_X_MAILER
  1403. * [Rules] Fix CTYPE_MIXED_BOGUS for text attachments
  1404. * [Rules] Fix FPs for CTYPE_MIXED_BOGUS
  1405. * [Rules] Fix HTTP_TO_HTTPS rule
  1406. * [Rules] Fix HTTP_TO_HTTPS rule
  1407. * [Rules] Fix zerofont rule (partially)
  1408. * [Rules] Micro-optimize X_PHP_EVAL
  1409. * [Rules] Reduce default weight for R_MISSING_CHARSET
  1410. 2.7: 08 Jan 2021
  1411. * [Conf] Add R_DKIM_PERMFAIL to the metric
  1412. * [CritFix] Dkim: Fix simple canonicalisation if multiple signatures are presented
  1413. * [CritFix] Fix controller paths normalisation
  1414. * [Feature] Add INVALID_DATE rule
  1415. * [Feature] Add controller endpoint for training neural
  1416. * [Feature] Add sanity checks for actions thresholds
  1417. * [Feature] Add support of '==' and '!=' in Rspamd expressions
  1418. * [Feature] Composites: Improve composite atoms parser
  1419. * [Feature] Docker: use Debian slim variant
  1420. * [Feature] Elastic: Add some missing fields
  1421. * [Feature] Extract text from img alt attributes
  1422. * [Feature] Improve charset detection logic
  1423. * [Feature] Lua_clickhouse: Add optional row callback for large selections
  1424. * [Feature] Lua_dns_resolver: Add idna_convert_utf8 method
  1425. * [Feature] Lua_mime: Add ability to do multipattern replacement
  1426. * [Feature] Lua_trie: Allow to report start of the match
  1427. * [Feature] Multimap: support adding map values as extra options
  1428. * [Feature] Neural: Move PCA learning to a subprocess
  1429. * [Feature] RBL: support matching content/image URLs only
  1430. * [Feature] RBL: support use of multiple selectors
  1431. * [Feature] Reputation: Allow to specify ip masks
  1432. * [Feature] Support SMIME signed messages container
  1433. * [Feature] Support multiple conditions for symbols
  1434. * [Feature] Support ping in milter mode
  1435. * [Feature] Support rspamd_text in selector regexps
  1436. * [Feature] Use own daemonization routine
  1437. * [Feature] Vadesecure: Implement settings_outbound feature as recommended by Vade
  1438. * [Feature] `rspamadm clickhouse` command
  1439. * [Feature] allow hyperscan for aarch64
  1440. * [Fix] Allow to set priorities between post init scripts
  1441. * [Fix] Allow to use maps for strings that are not zero terminated
  1442. * [Fix] Apply max_lua_urls limit for emails as well
  1443. * [Fix] Arc: Fix CV check on signing
  1444. * [Fix] Arc: Fix signing of the broken ARC chains
  1445. * [Fix] Clickhouse: escape carriage return
  1446. * [Fix] Composites: Allow partial match
  1447. * [Fix] Deduct type of a table methods
  1448. * [Fix] Do not load errored hyperscan database
  1449. * [Fix] Do not process links in ignored html tags
  1450. * [Fix] Fix ClamAV result for cached encrypted file (#3395)
  1451. * [Fix] Fix canonicalisation when l= tag is presented
  1452. * [Fix] Fix flag shift
  1453. * [Fix] Fix handling of skip/skip_process http flags
  1454. * [Fix] Fix html attachments checks
  1455. * [Fix] Fix issue with pushing binary formats to Lua strings
  1456. * [Fix] Fix logging for rspamadm
  1457. * [Fix] Fix off-by-one with init check
  1458. * [Fix] Fix parsing of escape characters in quoted pairs
  1459. * [Fix] Fix pushing ucl strings with \0 inside
  1460. * [Fix] Fix quoted-printable soft newlines bugged case
  1461. * [Fix] Fix settings in case actions are set to null (#3415)
  1462. * [Fix] Fix several issues with auth results producing
  1463. * [Fix] Fix smtp comments exclusion
  1464. * [Fix] Fix smtp date syntax definition
  1465. * [Fix] Fix substring search in case if srchlen == inlen
  1466. * [Fix] Fix text selectors
  1467. * [Fix] Honour `systemd` setting when logging to console (#3514)
  1468. * [Fix] Html: Add entities collisions prevention logic (e.g. for mathml entities)
  1469. * [Fix] Lua_auth_results: Quote potentially bad values in AR header
  1470. * [Fix] Multimap: Fix flags usage
  1471. * [Fix] Multimap: Fix scoring for combined maps
  1472. * [Fix] Plug GList * leak in redis pool
  1473. * [Fix] RBL: allow for multiple matches of the same label if types are different
  1474. * [Fix] Rely on libev checks for file maps
  1475. * [Fix] Restore simple dkim canonicalisation mode
  1476. * [Fix] Return MimeCharset as we work with emails...
  1477. * [Fix] Spamassassin: Fix pcre_only flags
  1478. * [Fix] Spamassassin: Preserve 'pcre_only' flag when dealing with regexp replacements
  1479. * [Fix] Try to fix GError leak
  1480. * [Fix] Try to fix a mess with settings loading by adding priorities
  1481. * [Fix] Try to move setings initialisation to a later stage
  1482. * [Fix] Use dup fd in milter handler to avoid races with the proxy
  1483. * [Fix] Use message pointer to avoid obsolete data to be cached
  1484. * [Project] Rbl: Migrate to `checks`
  1485. * [Project] Rbl: Move config code outside of the plugin
  1486. * [Project] Ressurect empty prefilters as connection filters
  1487. * [Project] Support connection filters registration from Lua
  1488. * [Rework] Add final cleanup logic
  1489. * [Rework] Add preliminary support of hyperscan caching for re maps
  1490. * [Rework] Add stale cache removal
  1491. * [Rework] Clickhouse: Improve performance
  1492. * [Rework] Distinguish between strict config test mode
  1493. * [Rework] Furhter logging improvements
  1494. * [Rework] Milter_headers: improve extended_headers_rcpt support
  1495. * [Rework] Move parsers to a separate lua library
  1496. * [Rework] Neural: Skip composite symbols
  1497. * [Rework] Rbl: Rework defaults logic
  1498. * [Rework] Some tunes to cache saving
  1499. * [Rework] Track maps origins
  1500. * [Rework] Use full crypto hash for regexp maps
  1501. * [Rules] Remove broken rule
  1502. 2.6: 30 Sep 2020
  1503. * [Conf] Add missing symbols
  1504. * [Conf] Add missing symbols
  1505. * [Conf] Fix fat-fingers typo
  1506. * [Conf] Fix wrong comment in options.inc
  1507. * [Conf] Neural: Fix the default name for max_trains
  1508. * [Conf] Register a known symbol
  1509. * [Conf] Spf: Add R_SPF_PERMFAIL symbol
  1510. * [CritFix] Arc: Fix ARC validation for chains of signatures
  1511. * [CritFix] Distinguish socketpairs between different fuzzy workers
  1512. * [CritFix] Fix IDNA dots parsing
  1513. * [CritFix] Fix test assertion method
  1514. * [CritFix] Fix usage of crypto_sign it should be crypto_sign_detached!
  1515. * [Feature] Add BOUNCE rule
  1516. * [Feature] Add controller plugins support and selectors plugin
  1517. * [Feature] Add maps query method
  1518. * [Feature] Add minimal delay to fuzzy storage
  1519. * [Feature] Add multiple base32 alphabets for decoding
  1520. * [Feature] Add preliminary support of BCH addresses
  1521. * [Feature] Add query_specific endpoint
  1522. * [Feature] Allow multiple base32 encodings in Lua API
  1523. * [Feature] Allow to specify nonces manually
  1524. * [Feature] Controller: Allow to pass query arguments to the lua webui plugins
  1525. * [Feature] Fuzzy_check: Add gen_hashes command
  1526. * [Feature] Fuzzy_check: Add weight_threshold option for fuzzy rules
  1527. * [Feature] Implement address retry on connection failure
  1528. * [Feature] Improve limits in pdf scanning
  1529. * [Feature] Initial support of subscribe command in lua_redis
  1530. * [Feature] Lua_cryptobox: Add secretbox API
  1531. * [Feature] Lua_text: Add encoding methods
  1532. * [Feature] Milter_headers: Allow to activate routines via users settings
  1533. * [Feature] PDF: Add timeouts for expensive operations
  1534. * [Feature] Preliminary maps addon for controller
  1535. * [Feature] Split pdf processing object and output object to allow GC
  1536. * [Feature] Support BLIS blas library
  1537. * [Feature] Support input vectorisation by recvmmsg call
  1538. * [Feature] Support multiple base32 alphabets
  1539. * [Feature] add queueid, uid, messageid and specific symbols to selectors [Minor] use only selectors to fill vars in force_actions message
  1540. * [Feature] allow variables in force_actions messages
  1541. * [Feature] extend lua api
  1542. * [Fix] #3249
  1543. * [Fix] Allow to adjust neurons in the hidden layer
  1544. * [Fix] Another try to fix email names parsing
  1545. * [Fix] Arc: Allow to reuse authentication results when doing multi-stage signing
  1546. * [Fix] Arc: Fix bug with arc chains verification where i>1
  1547. * [Fix] Arc: Sort headers by their i= value
  1548. * [Fix] Change neural plugin's loss function
  1549. * [Fix] Deal with double eqsigns when decoding headers
  1550. * [Fix] Default ANN names in clickhouse
  1551. * [Fix] Disable reuseport for TCP sockets as it causes too many troubles
  1552. * [Fix] Disable text detection heuristics for encrypted parts
  1553. * [Fix] Distinguish DKIM keys by md5
  1554. * [Fix] Distinguish type from flags in register_symbol
  1555. * [Fix] Dmarc: Unbreak reporting after cf2ae3292ac93da8b6e0624b48a62828a51803c9
  1556. * [Fix] Do not flag pre-result of virus scanners as least if action is reject
  1557. * [Fix] Do not use GC64 workaround on 32bit platforms, omg
  1558. * [Fix] Exclude damaged urls from html parser
  1559. * [Fix] Fix FREEMAIL_REPLYTO_NEQ_FROM_DOM
  1560. * [Fix] Fix FROM_NEQ_ENVFROM
  1561. * [Fix] Fix FWD_GOOGLE rule (#1815)
  1562. * [Fix] Fix adding of the empty archive file for gzip
  1563. * [Fix] Fix aliases in forged recipients and limit number of iterations
  1564. * [Fix] Fix authentication results insertion
  1565. * [Fix] Fix calling of methods in selectors
  1566. * [Fix] Fix clen length for hiredis...
  1567. * [Fix] Fix endless loop if broken arc chain has been found
  1568. * [Fix] Fix false - operation
  1569. * [Fix] Fix get_urls table invocation
  1570. * [Fix] Fix group based composites
  1571. * [Fix] Fix headers passing in rspamd_proxy
  1572. * [Fix] Fix incomplete utf8 sequences handling
  1573. * [Fix] Fix lua_next invocation
  1574. * [Fix] Fix lua_parse_symbol_type function logic
  1575. * [Fix] Fix multiple listen configuration
  1576. * [Fix] Fix occasional encryption of the cached data
  1577. * [Fix] Fix parsing boundaries with spaces
  1578. * [Fix] Fix passing of methods arguments
  1579. * [Fix] Fix poor man allocator algorithm
  1580. * [Fix] Fix regexp selector and add flattening
  1581. * [Fix] Fix rfc base32 encode ordering (skip inverse bits)
  1582. * [Fix] Fix rfc based base32 decoding
  1583. * [Fix] Fix sockets leak in the client
  1584. * [Fix] Fix storing of the original smtp from
  1585. * [Fix] Fix types check and types usage in lua_cryptobox
  1586. * [Fix] Fix unused results
  1587. * [Fix] Fuzzy_check: Disable shingles for short texts (really)
  1588. * [Fix] Ical: Fix identation grammar
  1589. * [Fix] Improve part:is_attachment logic
  1590. * [Fix] Mmap return value must be checked versus MAP_FAILED
  1591. * [Fix] One more fix to skip images that are not urls
  1592. * [Fix] Pdf: Support some weird objects with no newline before endobj
  1593. * [Fix] Rbl: Fix ignore_defaults in conjunction with ignore_whitelists
  1594. * [Fix] Restore support for `for` and `id` parts in received headers
  1595. * [Fix] Segmentation fault in contrib/lua-lpeg/lpvm.c on ppc64el
  1596. * [Fix] Skip spaces at the boundary end
  1597. * [Fix] Slashing fix: fix captures matching API
  1598. * [Fix] Spamassassin: Rework metas processing
  1599. * [Fix] Store reference of upstream list in upstreams objects
  1600. * [Fix] Understand utf8 in content-disposition parser
  1601. * [Fix] Unify selectors digest functions
  1602. * [Fix] Use `abs` value when checking composites
  1603. * [Fix] Use strict IDNA for utf8 DNS names + add sanity checks for DNS names
  1604. * [Fix] Use unsigned char and better support of utf8 in ragel parser
  1605. * [Fix] add missing selector_cache declaration
  1606. * [Project] Add `L` flag for regexps to save start of the match in Hyperscan
  1607. * [Project] Add `lower` method to lua_text
  1608. * [Project] Add a simple matrix Lua library
  1609. * [Project] Add implicit bitcoincash prefix
  1610. * [Project] Add linalg ffi library for prototyping
  1611. * [Project] Add methods to append data to fuzzy requests
  1612. * [Project] Add routine to call a generic lua function
  1613. * [Project] Add ssyev method interface
  1614. * [Project] Add tensors index method
  1615. * [Project] Add text:sub method
  1616. * [Project] Allow rspamd_text based selectors
  1617. * [Project] Allow to specify re_conditions for regular expressions
  1618. * [Project] Attach extensions to the binary fuzzy commands
  1619. * [Project] Bitcoin: BTC cash addresses needs some checksum validation
  1620. * [Project] Cleanup the redis script
  1621. * [Project] Convert bitcoin rules to the new regexp conditions feature
  1622. * [Project] Detect memrchr in systems that supports it
  1623. * [Project] Do not listen sockets in the main process
  1624. * [Project] Implement 'probabilistic' learn mode for ANN
  1625. * [Project] Implement BTC polymod in C as it requires 64 bit ops
  1626. * [Project] Implement bitcoin cash validation in a proper way
  1627. * [Project] Implement extensions logic for fuzzy storage
  1628. * [Project] Implement symbols insertion in multiple results mode
  1629. * [Project] Lua_text: Add method memchr
  1630. * [Project] Neural: Add PCA loading logic
  1631. * [Project] Neural: Fix PCA based learning
  1632. * [Project] Neural: Fix matrix gemm
  1633. * [Project] Neural: Further PCA fixes
  1634. * [Project] Neural: Implement PCA in learning
  1635. * [Project] Neural: Implement PCA learning
  1636. * [Project] Neural: Implement PCA on ANN forward
  1637. * [Project] Neural: Implement PCA serialisation
  1638. * [Project] Neural: Start PCA implementation
  1639. * [Project] Neural: Use C version of scatter matrix producing
  1640. * [Project] Preliminary support of lua conditions for regexps
  1641. * [Project] Preliminary usage of the reuseport
  1642. * [Project] Process composites separately for each shadow result
  1643. * [Project] Remove old code
  1644. * [Project] Rework scan result functions to support shadow results
  1645. * [Project] Rework some more functions to work with shadow results
  1646. * [Project] Some more fixes
  1647. * [Project] Start results chain implementation
  1648. * [Project] Support fun iterators on rspamd_text objects
  1649. * [Project] Support multiply, minus and divide operators in expressions
  1650. * [Project] Tensor: Move scatter matrix calculation to C
  1651. * [Rework] Allow to specify exat metric result when adding a symbol
  1652. * [Rework] Change and improve openblas detection and usage
  1653. * [Rework] Close listen sockets in main after fork
  1654. * [Rework] Further rework of lua urls extraction API
  1655. * [Rework] Lua_cryptobox: Allow to store output of the hash function
  1656. * [Rework] Lua_task: Add more methods to deal with shadow results
  1657. * [Rework] Modernize logging for expressions
  1658. * [Rework] Remove empty prefilters feature - we are not prepared...
  1659. * [Rework] Remove old FindLua module, disable lua fallback when LuaJIT is enabled
  1660. * [Rework] Rework and refactor forged recipients plugin
  1661. * [Rework] Rework expressions processing
  1662. * [Rework] Rework fuzzy commands processing
  1663. * [Rework] Rework url flags handling API
  1664. * [Rework] Rework urls extraction
  1665. * [Rework] Split operations processing and add more debug logs
  1666. * [Rework] Update zstd to 1.4.5
  1667. * [Rework] Use google-ced instead of libicu chardet as the former sucks
  1668. * [Rework] add alias util:parse_addr for util:parse_mail_address
  1669. * [Rework] get rid of util:parse_addr duplicating the util:parse_mail_address, replace where used
  1670. * [Rules] Allow prefix for bitcoin cash addresses
  1671. * [Rules] More fixes for bitcoin cash addresses decoding
  1672. * [Rules] Refactor bleach32 addresses handling
  1673. 2.5: 01 Apr 2020
  1674. * [Conf] Mark Rspamd emailbl as ignore whitelist
  1675. * [Conf] RBL: Add missing emails = true option
  1676. * [Feature] Add support for scripts in fuzzy storage
  1677. * [Feature] Arc: Add whitelisted_signers_map option
  1678. * [Feature] Implement hosts file processing
  1679. * [Feature] Neural: Introduce classes bias that allows non-equal classes learning
  1680. * [Feature] Update libev to 4.33
  1681. * [Fix] Another brain damage html standard adoptions
  1682. * [Fix] Another fix for brain damaged obs-fws state
  1683. * [Fix] Fix flags that caused force_actions failure
  1684. * [Fix] Fix logging issue
  1685. * [Fix] Fix lua symbols scores registration when config does not define scores
  1686. * [Fix] Fix opaque maps logic
  1687. * [Fix] Fix parsing of the html tags with no spaces after attributes
  1688. * [Fix] Fix some corner cases in urls parsing, add limits
  1689. * [Fix] Fix tlds extraction if custom composition rules are used
  1690. * [Fix] Fix variables replacement in mempool
  1691. * [Fix] Improve base64 detection
  1692. * [Fix] Normalize dynamic scores in ANN correctly
  1693. * [Fix] Plug memory leak introduced by #3153
  1694. * [Fix] Stat_redis_backend: Fix memory leak and simplify learn path
  1695. * [Fix] Try hard to deal with ghost workers
  1696. * [Fix] metadata_exporter default formatter
  1697. * [Rework] Change the way to extract URLs when dealing with alternative parts
  1698. * [Rework] Fix various url extraction issues
  1699. * [Rework] Re cache: Load compiled hyperscan in the main process as well
  1700. * [Rework] Re cache: Load hyperscan early
  1701. * [Rework] Rework URL structure: adjust tld part
  1702. * [Rework] Rework URL structure: host field
  1703. * [Rework] Rework URL structure: more structure optimisations
  1704. * [Rework] Rework URL structure: user field
  1705. * [Rework] URL: Another update for urls extraction logic
  1706. * [Rework] Urls: Improve query urls handling
  1707. * [Rework] Urls: adopt html related stuff
  1708. * [Rework] Urls: more rework of the urls sets
  1709. * [Rework] Urls: process query urls in HTML urls correctly
  1710. * [Rework] Urls: rework urls hash structure
  1711. * [Rework] Urls: update lua libraries
  1712. * [Rework] Use multiple search tries for different url extraction types
  1713. 2.4: 26 Feb 2020
  1714. * [CritFix] Fix parsing of the content type attributes
  1715. * [Feature] Clickhouse: Add extra columns support
  1716. * [Feature] Rbl: Add url_compose_map option for RBL rules
  1717. * [Fix] 'R' flag is for all headers regexp
  1718. * [Fix] Allow to reset settings id from Lua (e.g. because of the priority)
  1719. * [Fix] Avoid collisions in mempool variables by changing fuzzy caching logic
  1720. * [Fix] Avoid strdup usage for symbols options
  1721. * [Fix] Do not trust stat(2) it lies
  1722. * [Fix] Filter all options for symbols to have sane characters
  1723. * [Fix] Fix all headers iteration
  1724. * [Fix] Fix allowed_settings for neural
  1725. * [Fix] Fix listen socket parsing
  1726. * [Fix] Fix maps expressions evaluation
  1727. * [Fix] Fix sentinel connections leak by using async connections
  1728. * [Fix] Fix smtp message on passthrough result
  1729. * [Fix] Fix tld compositon rules
  1730. * [Fix] Fuzzy_storage: Do not check for shingles if a direct hash has been found
  1731. * [Fix] Lua_mime: Do not perform QP encoding for 7bit parts
  1732. * [Fix] Neural: Distinguish missing symbols from symbols with low scores
  1733. * [Fix] Support listening on systemd sockets by name
  1734. * [Project] Add lua_urls_compose library
  1735. * [Project] Allow to set a custom log function to the logger
  1736. * [Project] CDB maps: Start making cdb a first class citizen
  1737. * [Project] Clickhouse: Add extra columns concept
  1738. * [Project] Fix urls composition rules, add unit tests
  1739. * [Project] Unify cdb maps
  1740. * [Rework] Logger infrastructure rework
  1741. * [Rework] Refactor libraries structure
  1742. * [Rework] Rework SSL caching
  1743. * [Rework] Update snowball stemmer to 2.0 and remove all crap aside of UTF8
  1744. 2.3: 04 Feb 2020
  1745. * [Conf] SPF is no longer a C module
  1746. * [Conf] Update spamtrap map path example
  1747. * [CritFix] Fix html entities decoding
  1748. * [CritFix] Fix re cache when mix of pcre and hyperscan is used
  1749. * [Feature] Allow milter code to deal with multiple headers
  1750. * [Feature] Antivirus: Add avast support
  1751. * [Feature] Dkim_signing: Allow to sign via milter_headers
  1752. * [Feature] Implement content hashes
  1753. * [Feature] Lua_text: Add regexp split iterator method
  1754. * [Feature] Lua_text: Implement flattening of the input tables
  1755. * [Feature] Send quit command to Redis
  1756. * [Feature] Speed up is_ascii function
  1757. * [Feature] Spf: Add external_relay option
  1758. * [Fix] Avoid double escaping
  1759. * [Fix] Fix O(N^2) algorithm
  1760. * [Fix] Fix arc seal validation
  1761. * [Fix] Fix base tag processing according to stupid HTML renderer behaviour
  1762. * [Fix] Fix dealing with `\0` in ucl strings and JSON
  1763. * [Fix] Fix gpg parts misdetection
  1764. * [Fix] Fix ignored symbols exporting
  1765. * [Fix] Fix processing of numeric url's
  1766. * [Fix] Fix processing of the closed tcp connections
  1767. * [Fix] Fix regexp type check for pcre2
  1768. * [Fix] Fix urls encode function
  1769. * [Fix] Fix urls shifting when doing decode to include separators
  1770. * [Fix] Fix white on white rule and add is_leaf flag
  1771. * [Fix] Further fixes in charset detection
  1772. * [Fix] Ignore diacritics in chartable module for specific languages
  1773. * [Fix] Limit size of symbols options by max_opts_len option
  1774. * [Fix] More fixes in html tag content calculations
  1775. * [Fix] Plug memory leak in fuzzy storage
  1776. * [Fix] Process high priority settings even if settings/id has been specified
  1777. * [Fix] Select a different upstream on last retransmit
  1778. * [Fix] Treat soft hyphen as zero width space
  1779. * [Fix] Try harder to watch the lifetime of the key_stat
  1780. * [Fix] Use ipv6-mapped-ipv4 addresses in radix trie
  1781. * [Project] Add logic to break execution when processing symbols*
  1782. * [Project] Add methods to set specific content for mime parts from Lua
  1783. * [Project] Lua_content: support PDF files
  1784. * [Project] Move dns_tool to using of the rspamd_spf from FFI module
  1785. * [Project] Preliminary SPF plugin in Lua
  1786. * [Project] Show debug stat for memory pool
  1787. * [Project] Some rework about specific data that is now tagged
  1788. * [Project] Start reworking of the mempool structure
  1789. * [Rework] Allow to add userdata as symbols options
  1790. * [Rework] Change mime part specifics handling
  1791. * [Rework] Move LRU SPF cache from spf plugin
  1792. * [Rework] Rework HTML tags content attachment
  1793. * [Rework] Rework options hash structure
  1794. * [Rework] Start lua_content library
  1795. * [Rework] Stop using of uthash for http headers
  1796. * [Rework] Use faster hashing approach for memory pools variables
  1797. * [Rules] Add PDF related rules
  1798. 2.2: 19 Nov 2019
  1799. * [Conf] Antivirus: Fix the default config
  1800. * [Feature] Add verdict library in lua
  1801. * [Feature] Allow exception when choosing upstream
  1802. * [Feature] Allow to disable symbols from the metric config
  1803. * [Feature] Allow to limit maps per specific worker
  1804. * [Feature] Always validate Rspamd protocol output
  1805. * [Feature] Antivirus: Add preliminary virustotal support
  1806. * [Feature] Clickhouse: Rework Clickhouse collection logic
  1807. * [Feature] Improve base64 usage
  1808. * [Feature] Shutdown timeout is now associated with task timeout
  1809. * [Fix] #3129 Multiple classifiers on redis working incorrectly
  1810. * [Fix] Allow real upstreams configuration
  1811. * [Fix] Another try to fix slow callbacks and timers
  1812. * [Fix] Check results of write message as SSL can bork them
  1813. * [Fix] Clickhouse: Avoid potential races in collection
  1814. * [Fix] Clickhouse: Fix periodic script
  1815. * [Fix] Fail DNS upstream on each retransmit attempt
  1816. * [Fix] Fix consistent hashing when upstreams are marked inactive
  1817. * [Fix] Fix issues found
  1818. * [Fix] Fix off-by-one in retries for the proxy
  1819. * [Fix] Fix termination
  1820. * [Fix] Fix upstreams exclusion logic
  1821. * [Fix] Fix utf8 validation for symbols options and empty strings
  1822. * [Fix] Oops, fix maps reload
  1823. * [Fix] Rbl: Allow utf8 lookups for IDN domains
  1824. * [Fix] Sigh, another try to fix brain-damaged openssl
  1825. * [Project] Add fast utf8 validation library
  1826. * [Project] Use own utf8 validation instead of glib
  1827. * [Rework] Another phase of finish actions rework
  1828. * [Rework] Further cmake system rework
  1829. * [Rework] Further isolation of the controller's functions
  1830. * [Rework] Make cmake structure more modular
  1831. * [Rework] Move cmake modules to a dedicated path
  1832. * [Rework] Replace controller functions by any scanner worker if needed
  1833. * [Rework] Rework final scripts logic
  1834. * [Rework] Rewrite rspamd_str_make_utf_valid function
  1835. 2.1: 28 Oct 2019
  1836. * [Conf] Update neural.conf
  1837. * [CritFix] Fix dkim verification for multiple headers listed
  1838. * [Feature] Add support of uudecode
  1839. * [Feature] Allow to explicitly set events backend
  1840. * [Feature] Implement configurable limits for SPF lookups
  1841. * [Feature] Lua_scanners: Use lua magic for inclusion/exclusion logic
  1842. * [Feature] Multimap: Do not check files in office archives
  1843. * [Feature] Neural: Add sampling when storing training vectors
  1844. * [Feature] SPF: Allow to disable AAAA checks in configuration
  1845. * [Feature] Spf: Add limits configuration support
  1846. * [Feature] Store etag in cached HTTP maps + better logging
  1847. * [Feature] Support segwit BTC addresses, fix LTC verification
  1848. * [Feature] Support uuencoding
  1849. * [Fix] Add configurable number of threads for OpenBLAS
  1850. * [Fix] Add workaround for ragel 7 in hyperscan related maps code
  1851. * [Fix] Another fix for numeric urls parsing
  1852. * [Fix] Correct EMA time calculations
  1853. * [Fix] Do not treat archives as text
  1854. * [Fix] Do not use strdup on data extracted from lua
  1855. * [Fix] Fix a failure calcuating URL reputation.
  1856. * [Fix] Fix crash due to constructors init order
  1857. * [Fix] Fix crash on parts with no cd
  1858. * [Fix] Fix empty prefilters that require mime structures
  1859. * [Fix] Fix event loop creation
  1860. * [Fix] Fix issues sending DMARC reports.
  1861. * [Fix] Fix misprint
  1862. * [Fix] Fix saving of the file maps
  1863. * [Fix] Fix size calculations when converting from utf16
  1864. * [Fix] Fix support of disable_monitoring in rbl
  1865. * [Fix] Fix use-after-free
  1866. * [Fix] Fix zip files check to relax requirements
  1867. * [Fix] Important hiredis fixes
  1868. * [Fix] Lot's of fixes in maps check logic
  1869. * [Fix] Lua_tcp: Deal with temporary fails on write
  1870. * [Fix] Lua_tcp: Make write errors fatal and rework error handlers
  1871. * [Fix] Meta: Filter some more values
  1872. * [Fix] Neural: Add protection agains infinities
  1873. * [Fix] Oops, fix math.huge invocation
  1874. * [Fix] Plug memory leak
  1875. * [Fix] Sigh, another email to string fix
  1876. * [Fix] Try to fix another ownership race in ssl connection
  1877. * [Fix] Uuencode: Fix parsing of corrupted uuencode
  1878. * [Fix] lua_scanners - razor rename need_check function
  1879. * [Rework] Require CMake 3.9 to work, remove manual lto crap
  1880. 2.0: 11 Oct 2019
  1881. * [Conf] Add BROKEN_HEADERS_MAILLIST composite
  1882. * [Conf] Add path to greylist-whitelist-domains.inc
  1883. * [Conf] Clarify documentation in the config files
  1884. * [Conf] Introduce maps.d directories
  1885. * [Conf] Log settings id by default
  1886. * [Conf] Make LEAKED_PASSWORD_SCAM a composite rule again
  1887. * [Conf] Move all surbl/emails rules to rbl
  1888. * [Conf] Register new Spamhaus codes
  1889. * [Conf] Remove configs for deleted modules
  1890. * [Conf] Remove surbl parts, fix hash_format attribute
  1891. * [Conf] Show autolearn sample
  1892. * [Conf] Slashing: Change default stats backend to Redis
  1893. * [Conf] Surbl: Utilise new `check_emails` option
  1894. * [Conf] Update header
  1895. * [Conf] Use multi-prefixes RBLs in the default config
  1896. * [CritFix] Deal with case-sensivity in Content-Disposition parser
  1897. * [CritFix] Eliminate old endpoint
  1898. * [CritFix] Fix case sensivity when parsing Content-Type
  1899. * [CritFix] Fix loading of DKIM public keys
  1900. * [CritFix] Fix procesing of urls
  1901. * [CritFix] Fix whitelisting when both spf and dkim are required to be valid
  1902. * [CritFix] Langdet: Fix language detection where no stop words found
  1903. * [Feature] Add description to the groups
  1904. * [Feature] Add limit for number of URLs in Lua
  1905. * [Feature] Add logging of groups to the log_format
  1906. * [Feature] Add lua_smtp library
  1907. * [Feature] Add maps cache and type refinement
  1908. * [Feature] Add p0f scanner
  1909. * [Feature] Adopt emails module to use lua_maps
  1910. * [Feature] Allow options matching in composites
  1911. * [Feature] Allow selectors in rbl module
  1912. * [Feature] Allow to output group results
  1913. * [Feature] Asn: Allow to use bgpdump when NET::MRT is broken
  1914. * [Feature] Calculate tokens occurrences distribution
  1915. * [Feature] Clickhouse: Add authenticated user and settings id columns
  1916. * [Feature] Clickhouse: Store groups data
  1917. * [Feature] Clickhouse: Utilise LowCardinality feature
  1918. * [Feature] Implement Redis prefixes registration logic
  1919. * [Feature] Implement settings id propagation between deps
  1920. * [Feature] Improve AV results caching
  1921. * [Feature] Improve autolearning
  1922. * [Feature] Improve logging locking logic (remove it actually)
  1923. * [Feature] Improve settings processing
  1924. * [Feature] Langdet: Limit number of stop words to be checked
  1925. * [Feature] Libucl: Allow to sort keys in ucl objects
  1926. * [Feature] Lua_config: Extend get symbols method
  1927. * [Feature] Lua_maps: Allow static maps for key-value pairs
  1928. * [Feature] Lua_mimepart: Add function filter_words
  1929. * [Feature] Lua_selectors: Add `words` selector
  1930. * [Feature] Lua_selectors: Add sort and uniq transform functions
  1931. * [Feature] Lua_selectors: Allow table arguments for selectors
  1932. * [Feature] Lua_tcp: Add preliminary support of SSL connections
  1933. * [Feature] Lua_trie: More flexible API
  1934. * [Feature] Lua_util: Add filter_specific_url function
  1935. * [Feature] Lua_util: table_digest can now recursively traverse tables
  1936. * [Feature] Maillist: Improve detection
  1937. * [Feature] Maps: Allow caching for complex maps
  1938. * [Feature] Monitored: Support random lookups
  1939. * [Feature] Multimap: Add combined maps prototype
  1940. * [Feature] Multimap: Add dependend maps via redis keys selectors
  1941. * [Feature] Multimap: Allow multiple email addresses matches
  1942. * [Feature] Multimap: Also check detected charset when do filename checks
  1943. * [Feature] Output number of messages processed to proctitle
  1944. * [Feature] Perform clean SSL shutdown
  1945. * [Feature] Performance: Do not use base64 SIMD version for bad inputs
  1946. * [Feature] RBL: Support bit results in replies
  1947. * [Feature] RBL: Support type specific prefixes
  1948. * [Feature] Ratelimit: Consider number of SMTP recipients
  1949. * [Feature] Rbl: Add ability to check urls
  1950. * [Feature] Rbl: Add resolve_ip based RBLs
  1951. * [Feature] Rbl: Make config checks much more strict
  1952. * [Feature] Rbl: Support per-rule whitelists
  1953. * [Feature] Rbl: Support process script
  1954. * [Feature] Rbl: Support replyto addresses
  1955. * [Feature] SURBL: Allow to check email domains
  1956. * [Feature] Selectors: Add `list` generator
  1957. * [Feature] Selectors: Add `specific_urls` extractor
  1958. * [Feature] Selectors: Add flatten function
  1959. * [Feature] Selectors: Support filter_map and apply_map functions
  1960. * [Feature] Store Clickhouse data outside of lua alloc
  1961. * [Feature] Support caching for encrypted files and macros
  1962. * [Feature] Support images when extracting urls
  1963. * [Feature] Support more hyperscan flags
  1964. * [Feature] Support protocol flags
  1965. * [Feature] URL: Apply stringprep to hostnames to filter garbage
  1966. * [Feature] Upstreams: Add lazy resolving logic to all upstreams
  1967. * [Feature] Upstreams: Set noresolve flag on numeric upstreams
  1968. * [Feature] Use `scores` in apply section
  1969. * [Feature] Use maps logic from lua_maps for multimap
  1970. * [Feature] Use random monitored in rbl module
  1971. * [Feature] lua_scanners - add Razor support
  1972. * [Fix] Add another safe-guard in urls processing
  1973. * [Fix] Add debug to ssl, fixed write hangs
  1974. * [Fix] Add missing groups to C callback symbols
  1975. * [Fix] Add more checks for ghosts symbols
  1976. * [Fix] Allow to enable or add new actions via settings
  1977. * [Fix] Allow to set 0 size for spf/dkim caches
  1978. * [Fix] Another bunch of fixes towards protocol mess
  1979. * [Fix] Another fix to deal with bad URLs
  1980. * [Fix] Arc: Another bunch of fixes for arc signing
  1981. * [Fix] Arc: More arc signing fixes
  1982. * [Fix] Avoid another overflow in fpconv
  1983. * [Fix] Clickhouse: Fix quoting
  1984. * [Fix] Clickhouse: Fix retention query quoting
  1985. * [Fix] Distinguish empty and non-empty prefilters
  1986. * [Fix] Distinguish remote and local addrs parsing
  1987. * [Fix] Do not assert if length of sig is bad, just fail verification
  1988. * [Fix] Do not assert if we have broken mime boundary in the headers
  1989. * [Fix] Do not call implicit strlen to avoid issues
  1990. * [Fix] Do not count images urls when checking url regexps for compatibility
  1991. * [Fix] Do not output rbl suffix in symbol option
  1992. * [Fix] Do not use config pool to avoid issues with double reload
  1993. * [Fix] Do not use ephemeral string
  1994. * [Fix] Do not use lightuserdata for traceback
  1995. * [Fix] Do not use priority in metric registration
  1996. * [Fix] Emails: Check email sanity before testing on BL
  1997. * [Fix] Emails: Fix misprint in key name
  1998. * [Fix] Escape utf in regexp to dodge ragel/hyperscan issue
  1999. * [Fix] Extend task_timeout to postfilters stage
  2000. * [Fix] Fix ARC signing after fixing another bug in it...
  2001. * [Fix] Fix AV scan logic
  2002. * [Fix] Fix DMARC_NA behaviour in case of no valid policies
  2003. * [Fix] Fix LRU hash iteration logic
  2004. * [Fix] Fix alignment mess
  2005. * [Fix] Fix configuring symbols without scores
  2006. * [Fix] Fix disabling of the actions
  2007. * [Fix] Fix dkim signing exceptions
  2008. * [Fix] Fix embedded images linking logic
  2009. * [Fix] Fix events leak
  2010. * [Fix] Fix eviction corner case
  2011. * [Fix] Fix fuzzy image score calculation #2962
  2012. * [Fix] Fix hang in fuzzy_learn when explicit rotation is set
  2013. * [Fix] Fix headers propagation logic
  2014. * [Fix] Fix hearbeats restart issue
  2015. * [Fix] Fix history reset
  2016. * [Fix] Fix log parameter
  2017. * [Fix] Fix lua_ip_equal logic
  2018. * [Fix] Fix more issues with nested messages + tests
  2019. * [Fix] Fix normalization of non-alphabet based languages
  2020. * [Fix] Fix offsets when parsing message/rfc822 in multipart
  2021. * [Fix] Fix options in rbl symbols
  2022. * [Fix] Fix out of bound access in lua logger
  2023. * [Fix] Fix out-of-bound read in qp decode
  2024. * [Fix] Fix parent CTE propagation
  2025. * [Fix] Fix parsing of the received headers with empty part
  2026. * [Fix] Fix pending checks for events
  2027. * [Fix] Fix printing of NULL pointer with fixed length
  2028. * [Fix] Fix race condition in watcher handler
  2029. * [Fix] Fix read-after-end in quoted printable decoding
  2030. * [Fix] Fix redis sentinel support
  2031. * [Fix] Fix registry leak in case of DNS errors
  2032. * [Fix] Fix reload logic
  2033. * [Fix] Fix sending of large entries via HTTPS
  2034. * [Fix] Fix settings reload
  2035. * [Fix] Fix some more corner cases for fpconv
  2036. * [Fix] Fix trie code when there are regexps and Hyperscan is absent
  2037. * [Fix] Further fixes to printing of the FP numbers
  2038. * [Fix] Fuzzy_check: Fix timeouts
  2039. * [Fix] Grrr, fix empty ip case
  2040. * [Fix] Html: Fix processing of fjlig entity
  2041. * [Fix] Lang_det: Try better to distinguish Chinese and Japanese
  2042. * [Fix] Lua_mime: Fix reversed extensions map
  2043. * [Fix] Lua_task: Fix message-less API
  2044. * [Fix] Lua_tcp: Report connection failures
  2045. * [Fix] Lua_tcp: Various fixes and debugging improvements
  2046. * [Fix] Metadata_exporter: This plugin is idempotent not a postfilter
  2047. * [Fix] More fixes to extract_specific_urls
  2048. * [Fix] More stages fixes
  2049. * [Fix] Neural: Another bunch of fixes
  2050. * [Fix] Neural: use version in ANN key profile
  2051. * [Fix] Postpone lua state destruction to allow lua dtors to be used
  2052. * [Fix] Prefer surbl/emails rule on rbl to preserve compatibility
  2053. * [Fix] RBL: Fix behaviour of emails_domainonly
  2054. * [Fix] Ratelimit: Fix dynamic score
  2055. * [Fix] Rbl: Fix emailbl functions
  2056. * [Fix] Really fix hyperscan workaround
  2057. * [Fix] Set sanity limits for pcre2
  2058. * [Fix] Settings: Fix settings check flags
  2059. * [Fix] Sort keys when getting data from Lua when filling rules
  2060. * [Fix] Statistics: Do not query Redis tokens when there are no learns
  2061. * [Fix] Stop IO event on write finished in http connection
  2062. * [Fix] Use heuristically detected text parts data
  2063. * [Fix] Various fixes to QP encoding algorithm
  2064. * [Fix] Various fixes to SSL state machine handler
  2065. * [Fix] Various fixes to asn module
  2066. * [Fix] Workaround for empty charset in rfc2231 encoding
  2067. * [Project] Switch from torch to KANN
  2068. * [Project] Add heartbeat events
  2069. * [Project] Add preliminary support of the Kaspersky Scan Engine
  2070. * [Project] Add preliminary version of maps expressions
  2071. * [Project] Add preprocessed settings to the config structure
  2072. * [Project] Add simple forward propagation function
  2073. * [Project] Add small helpers for migration simplifications
  2074. * [Project] Allow to replace body in milter
  2075. * [Project] Bundle libev
  2076. * [Project] First refactoring step libevent->libev
  2077. * [Project] Implement syntax highlighting for Lua
  2078. * [Project] Lua_magic: Adopt lua_magic stuff in mime_types
  2079. * [Project] Remove libfann, gd and other unsupported stuff
  2080. * [Project] Remove torch
  2081. * [Project] Rework upstreams
  2082. * [Rework] Allow execution of async events when hs compiles regexps
  2083. * [Rework] Bayes expiry: eliminate `default` expiration mode
  2084. * [Rework] Dkim: Remove signing code
  2085. * [Rework] Dkim_signing: Move sign condition to dkim_signing
  2086. * [Rework] Do not lowercase all data send to ClickHouse
  2087. * [Rework] Drop url tags
  2088. * [Rework] Eliminate lua_squeeze as it has shown no improvements
  2089. * [Rework] Eliminate virtual scan time as it is useless
  2090. * [Rework] Lua core: Use lightuserdata to index classes
  2091. * [Rework] Lua_util: Another rework for extract_specific_urls
  2092. * [Rework] Migrate from ip_score to reputation
  2093. * [Rework] Move mime modification functions to lua_mime library
  2094. * [Rework] Rbl: Major whitelisting logic rework
  2095. * [Rework] Remove deprecated plugins
  2096. * [Rework] Remove log helper worker
  2097. * [Rework] Remove rspamd.classifiers.lua
  2098. * [Rework] Rename filter.h to a more sane name
  2099. * [Rework] Reorganise selectors implementation
  2100. * [Rework] Replace linenoise with replxx
  2101. * [Rework] Reputation: Remove ipnet from the ip reputation
  2102. * [Rework] Reputation: Slashing - change name of symbols
  2103. * [Rework] Rework children operations
  2104. * [Rework] Rework config reload
  2105. * [Rework] Rework expression API
  2106. * [Rework] Rework image urls processing
  2107. * [Rework] Rework initialisation to reduce static leaks count
  2108. * [Rework] Rework request headers processing
  2109. * [Rework] Slashing: Change versioning schema - move to 2.0
  2110. * [Rework] Slashing: Turn off postfilters when passthrough result is set
  2111. * [Rework] Start moving to replxx
  2112. * [Rework] Stop support of signed HTTP maps to simplify code
  2113. * [Rework] Store ASN as UInt32 in ClickHouse
  2114. * [Rework] Url_redirector: Rewrite plugin
  2115. * [Rework] Use a dedicated library for autolearn
  2116. * [Rework] Use libsodium instead of hand crafted crypto implementations
  2117. * [Rework] Use opaque structure to store a table of mime headers
  2118. * [Rules] Add dedicated bitcoin addresses filter rule
  2119. * [Rules] Add more detection to LEAKED_PASSWORD_SCAM
  2120. * [Rules] Catch LTC addresses
  2121. * [Rules] Reduce weight of RSPAMD_EMAILBL
  2122. * [Rules] Rework LEAKED_PASSWORD_SCAM rule one more time
  2123. 1.9.4: 23 May 2019
  2124. * [CritFix] Fix case sensitivity when parsing Content-Type
  2125. * [Fix] Arc: Another bunch of fixes for arc signing
  2126. * [Fix] Arc: More arc signing fixes
  2127. * [Fix] Avoid another overflow in fpconv
  2128. * [Fix] Fix ARC signing after fixing another bug in it...
  2129. * [Fix] Fix dkim signing exceptions
  2130. * [Fix] Fix some more corner cases for fpconv
  2131. * [Fix] Further fixes to printing of the FP numbers
  2132. * [Fix] Ratelimit: Fix dynamic score
  2133. 1.9.3: 13 May 2019
  2134. * [Conf] Add IP_SCORE_FREEMAIL composite rule
  2135. * [Feature] Add cryptobox method to generate dkim keypairs
  2136. * [Feature] Add fast hashes to lua cryptobox hash
  2137. * [Feature] Add least passthrough results
  2138. * [Feature] Allow oversign if exists mode
  2139. * [Feature] Clickhouse: Modernise table initial schema
  2140. * [Feature] Implement IUF interface for specific fast hashes
  2141. * [Feature] Lua_util: Allow to obfuscate different fields
  2142. * [Feature] Tune memory management in Rspamd and Lua
  2143. * [Fix] Avoid buffer overflow when printing long lua strings
  2144. * [Fix] Change the default oversigning headers to a more sane list
  2145. * [Fix] Clickhouse: Do not store digest as it is not needed now
  2146. * [Fix] Clickhouse: Fix lots of storage issues
  2147. * [Fix] Clickhouse: Support custom actions
  2148. * [Fix] Deny URLs where hostname is bogus
  2149. * [Fix] Do not blacklist mail by SPF/DMARC for local/authed users
  2150. * [Fix] Fix DoS caused by bug in glib
  2151. * [Fix] Fix UCL parsing of the multiline strings
  2152. * [Fix] Fix buffer overflow when printing small floats
  2153. * [Fix] Fix init code for servers keypairs cache
  2154. * [Fix] Fix issue with urls with no tld (e.g. IP)
  2155. * [Fix] Fix memory in arc signing logic
  2156. * [Fix] Fix memory leak in language detector during reloads
  2157. * [Fix] Fix mixed case content type processing
  2158. * [Fix] Fix processing of the ip urls in file
  2159. * [Fix] Fix use after free
  2160. * [Fix] HTML: Fix `size` attribute processing
  2161. * [Fix] Hum, it seems that 99ff1c8 was not correct
  2162. * [Fix] Lua_task: Fix task:get_from method
  2163. * [Fix] Preserve fd when mapping file to scan
  2164. * [Fix] Re-use milter_headers settings when doing arc signing
  2165. * [Fix] Set dmarc force action as least action
  2166. * [Fix] Switch to GMT
  2167. * [Fix] allow PKCS7 signatures to be text/plain, too
  2168. * [Project] Add initial version of the vault management tool
  2169. * [Project] Add vault support for DKIM and ARC signing
  2170. * [Project] Implement keys rotation in the vault
  2171. * [Project] Improve dkim keys generation for vault
  2172. * [Project] Improve keys creation in rspamadm vault
  2173. * [Rework] Move lua_worker to a dedicated unit
  2174. * [WebUI] Add URL fragments (#) support
  2175. * [WebUI] Fix AJAX request URL
  2176. 1.9.2: 16 Apr 2019
  2177. * [Conf] Allow to load users plugins from plugins.d
  2178. * [Conf] oversign openpgp and autocrypt headers
  2179. * [Feature] Add SPF FFI library for Lua
  2180. * [Feature] Add more verbosity for SPF caching
  2181. * [Feature] Antivirus: Handle encrypted files specially
  2182. * [Feature] Clickhouse: Slashing - add new fields to CH
  2183. * [Feature] Dkim_signing: Add OpenDKIM like signing_table and key_table
  2184. * [Feature] Dkim_signing: Allow to use new options as maps
  2185. * [Feature] Import fpconv library
  2186. * [Feature] Lua_maps: Allow static regexp and glob maps
  2187. * [Feature] Parse ical files
  2188. * [Feature] Rspamadm: Add dns_tool utility
  2189. * [Feature] Store SPF records digests
  2190. * [Feature] Use fpconv girsu2 implementation for printing floats
  2191. * [Fix] Clickhouse: Use integer seconds when inserting rows
  2192. * [Fix] Fix floating point printing
  2193. * [Fix] Fix processing of embedded urls
  2194. * [Fix] Lua_clickhouse: Fix CH errors processing
  2195. * [Fix] Make spf digest stable
  2196. * [Fix] Properly detect encrypted files in zip archives
  2197. * [Fix] Slashing: Store times in GMT timezone in ClickHouse
  2198. * [Rules] Add additional conditions to perform BTC checks
  2199. * [Rules] Fix pay-to-hash addresses validation
  2200. 1.9.1: 5 Apr 2019
  2201. * [Conf] Add vendor groups for symbols
  2202. * [Feature] Add `rspamadm template` command
  2203. * [Feature] Allow to add messages from settings
  2204. * [Feature] Allow unconnected DNS servers operations
  2205. * [Feature] Check limits after being set, migrate to uint64
  2206. * [Feature] Greylist: Allow to disable greylisting depending on symbols
  2207. * [Feature] Improve lua binary strings output
  2208. * [Feature] Mime_types: Implement user configurable extension filters
  2209. * [Feature] Mime_types: When no extension defined, detect it by content
  2210. * [Feature] Preprocess config files using jinja templates
  2211. * [Feature] Replies: Filter replies sender to limit whitelisting to direct messages
  2212. * [Feature] Treat all tags with HREF as a potential hyperlinks
  2213. * [Feature] Validate BTC addresses in LEAKED_PASSWORD_SCAM
  2214. * [Fix] Add crash safety for HTTP async routines
  2215. * [Fix] Another fix for Redis sentinel
  2216. * [Fix] Clickhouse: Fix table schema upload
  2217. * [Fix] Core: Fix squeezed dependencies handling for virtual symbols
  2218. * [Fix] Finally fix default parameters parsing in actions section
  2219. * [Fix] Fix ES sending logic (restore from coroutines mess)
  2220. * [Fix] Fix finishing script for clickhouse collection
  2221. * [Fix] Fix priority for regexp symbols registriation
  2222. * [Fix] Fix various issues found by PVS Studio
  2223. * [Fix] Initialize lua debugging earlier
  2224. * [Fix] Neural: Fix training
  2225. * [Fix] Rework cached Redis logic to avoid sentinels breaking
  2226. * [Fix] SURBL: Fix regression in surbl module
  2227. * [Fix] Fix double signing in the milter
  2228. * [Project] Add support of HTTP proxy in requests
  2229. * [Rework] Change lua global variables registration
  2230. * [Rework] Rework HTML content urls extraction
  2231. * [Rework] Start rework of aliasing in Rspamd
  2232. * [WebUI] Combine Scan and Learning into one tab
  2233. * [WebUI] Fix symbol score input type
  2234. * [WebUI] Show grayed out pie
  2235. * [WebUI] Update Throughput summary values dynamically
  2236. 1.9.0: 12 Mar 2019
  2237. * [Conf] Add missing includes
  2238. * [Conf] Move to options
  2239. * [Conf] Rbl: DWL is actually special whitelist
  2240. * [Conf] Relax some uribl rules
  2241. * [Conf] Remove abuse.ch
  2242. * [CritFix] Html: Entities are not valid within tag params values
  2243. * [Feature] Add `rspamadm mime sign` tool
  2244. * [Feature] Add configgraph utility
  2245. * [Feature] Add dedicated ZW spaces detection for URLs
  2246. * [Feature] Add flag to url object when visible part is url_like
  2247. * [Feature] Add method task:lookup_words
  2248. * [Feature] Add pyzor support (by crosenberg)
  2249. * [Feature] Allow to add upstream watchers to Lua API
  2250. * [Feature] Allow to set rewrite subject pattern from settings
  2251. * [Feature] Better escaping of unicode
  2252. * [Feature] Clickhouse: Allow to store subject in Clickhouse
  2253. * [Feature] Core: Add QP encoding utility
  2254. * [Feature] Core: Add libmagic detection for all parts
  2255. * [Feature] Core: Add support for gzip archives
  2256. * [Feature] Core: Allow to construct scan tasks from raw data
  2257. * [Feature] Core: Detect charset in archived files
  2258. * [Feature] Core: Ignore and mark invisible spaces
  2259. * [Feature] Core: Normalise zero-width spaces in urls
  2260. * [Feature] Core: Process data urls for images
  2261. * [Feature] Core: Relax quoted-printable encoding
  2262. * [Feature] Core: Support RFC2231 encoding in headers
  2263. * [Feature] Core: Support telephone URLs
  2264. * [Feature] Core: allow to emit soft reject on task timeout
  2265. * [Feature] DCC: Add bulkness and reputation checks to dcc
  2266. * [Feature] Elastic: Modernize plugin
  2267. * [Feature] Export visible part of url to lua
  2268. * [Feature] Fuzzy_storage: add preliminary support of rate limits
  2269. * [Feature] HTML: Specially treat data urls in HTML
  2270. * [Feature] Implement event watchers for upstreams
  2271. * [Feature] Implement includes tracing in Lua
  2272. * [Feature] Improve dkim part in configwizard
  2273. * [Feature] Lua_scanners: Add VadeSecure engine support
  2274. * [Feature] Lua_task: Add flexible method to get specific urls
  2275. * [Feature] Mime_types: Add MIME_BAD_UNICODE rule
  2276. * [Feature] Mime_types: Use detected content type as well
  2277. * [Feature] Plugins: Add preliminary version of the external services plugin
  2278. * [Feature] Query sentinel on master errors
  2279. * [Feature] Regexp: Allow local lua functions in Rspamd regexp module
  2280. * [Feature] Rspamadm: Allow to append footers to plain messages
  2281. * [Feature] Rspamadm: Allow to rewrite headers in messages
  2282. * [Feature] Selectors: Add `ipmask` processor
  2283. * [Feature] Settings: Allow hostname match
  2284. * [Feature] Settings: Allow local when selecting settings
  2285. * [Feature] Settings: Allow multiple selectors
  2286. * [Feature] Settings: Allow to inverse conditions
  2287. * [Feature] Support User-Agent in HTTP requests
  2288. * [Feature] Support ed25519 dkim keys generation
  2289. * [Feature] Try to filter bad unicode types during normalisation
  2290. * [Feature] external_services - oletools (olefy) support
  2291. * [Feature] lua_scanners - icap protocol support
  2292. * [Feature] lua_scanners - spamassassin spam scanner
  2293. * [Fix] Add filter for absurdic URLs
  2294. * [Fix] Add some more cases for Received header
  2295. * [Fix] Allow to disable/enable composite symbols
  2296. * [Fix] Arc: Use a separated list of headers for arc signing
  2297. * [Fix] Archive: Final fixes for 7z archives
  2298. * [Fix] Clickhouse: Fix database usage
  2299. * [Fix] Controller: Make save stats timer persistent
  2300. * [Fix] Core: Detect encrypted rarv5 archives
  2301. * [Fix] Core: Don't detect language twice
  2302. * [Fix] Core: Fix address rotation bug
  2303. * [Fix] Core: Fix content calculations for message parts
  2304. * [Fix] Core: Fix emails comments parsing and other issues
  2305. * [Fix] Core: Fix etags support
  2306. * [Fix] Core: Fix headers folding on the last token
  2307. * [Fix] Core: Fix iso-8859-16 encoding
  2308. * [Fix] Core: Fix log_urls flag (and encrypted logging)
  2309. * [Fix] Core: Fix part length when dealing with boundaries
  2310. * [Fix] Core: Fix parts distance calculations
  2311. * [Fix] Core: Fix processing of NDNs of certain type
  2312. * [Fix] Core: Implement logic to find some bad characters in URLs
  2313. * [Fix] Core: treat nodes with ttl properly in lru cache
  2314. * [Fix] Fix Content-Type parsing
  2315. * [Fix] Fix HTTP headers signing case
  2316. * [Fix] Fix control interface
  2317. * [Fix] Fix deletion of the duplicate headers
  2318. * [Fix] Fix emails filtering in emails module
  2319. * [Fix] Fix greylisting log message and logic
  2320. * [Fix] Fix issues with storing of the accepted addr in rspamd control
  2321. * [Fix] Fix maps object update race condition
  2322. * [Fix] Fix memor leaks and whitespace processing
  2323. * [Fix] Fix processing of null bytes in headers
  2324. * [Fix] Fix rcpt_mime and from_mime in user settings
  2325. * [Fix] Fix rfc2047 decoding for CD headers
  2326. * [Fix] Fix rfc2231 for Content-Disposition header
  2327. * [Fix] Fix setting of the subject pattern in config
  2328. * [Fix] Greylist: fix records checking
  2329. * [Fix] HTML: Another HTML comments exception fix
  2330. * [Fix] HTML: Another entities decoding logic fix
  2331. * [Fix] HTML: Fix HTML comments with many dashes
  2332. * [Fix] HTML: Fix entities in HTML attributes
  2333. * [Fix] HTML: Fix some more SGML tags issues
  2334. * [Fix] Ignore whitespaces at the end of value in DKIM records
  2335. * [Fix] MID module: Fix DKIM domain matching
  2336. * [Fix] Milter_headers: Fix remove_upstream_spam_flag and modernise config
  2337. * [Fix] Mime_parser: Fix issue with parsing of the trailing garbadge
  2338. * [Fix] Mime_parser: Fix parsing of mime parts without closing boundary
  2339. * [Fix] Multimap: Fix operating with userdata
  2340. * [Fix] Process orphaned `symbols` section
  2341. * [Fix] Rdns: Fix multiple replies in fake replies
  2342. * [Fix] Rework groups scores definitions
  2343. * [Fix] Set proper element when reading data from Sentinel
  2344. * [Fix] Set rspamd user to initialise supplementary groups on reload
  2345. * [Fix] Settings: Fix selectors usage
  2346. * [Fix] Sort data received from Sentinel to avoid constant replacing
  2347. * [Fix] groups.conf - filename typo
  2348. * [Fix] lua_scanner - oletools typos, logging
  2349. * [Fix] lua_scanners - actions and symbol_fail
  2350. * [Fix] lua_scanners - fix luacheck
  2351. * [Fix] lua_scanners - kaspersky - response with fname
  2352. * [Fix] lua_scanners - savapi redis prefix
  2353. * [Fix] tests - antivirus - fprot symbols
  2354. * [Project] Add concept of flexible actions
  2355. * [Project] Add heuristical from parser to received parser
  2356. * [Project] Add new flags to clickhouse, redis and elastic exporters
  2357. * [Project] Attach new received parser
  2358. * [Project] Fallback to callbacks from coroutines
  2359. * [Project] Implement keep-alive support in lua_http
  2360. * [Project] Lua_udp: Implement fully functional client
  2361. * [Project] Plug keepalive knobs into http connection handling
  2362. * [Project] Rspamadm: Add `modify` tool
  2363. * [Rework] Convert rspamd-server to a shared library
  2364. * [Rework] Dcc: Rework DCC plugin
  2365. * [Rework] Enable explicit coroutines symbols
  2366. * [Rework] Rework telephone urls parsing logic
  2367. * [Rework] Rewrite RBL module
  2368. * [Rework] Settings: Rework settings check
  2369. * [Rework] Slashing: Distinguish lualibdir, pluginsdir and sharedir
  2370. * [Rework] Unify task_timeout
  2371. * [Rework] Use VEX instructions in assembly, relocate
  2372. * [WebUI] Notify user if uploaded data was not learned
  2373. * [WebUI] Remove redundant condition
  2374. 1.8.3: 03 Dec 2018
  2375. * [CritFix] Make flags mutually exclusive for mime parts
  2376. * [CritFix] Strictly deny unencoded bad utf8 sequences in headers
  2377. * [Feature] Add Kaspersky antivirus support
  2378. * [Feature] Add method to get dkim results
  2379. * [Feature] Add more words regexp classes
  2380. * [Feature] Allow to choose words format in `rspamadm mime`
  2381. * [Feature] Allow to get all types of words from Lua
  2382. * [Feature] Allow to get task flags in C expressions
  2383. * [Feature] Allow to require encryption when accepting connections
  2384. * [Feature] Ignore bogus whitespaces in the words
  2385. * [Feature] Implement more strict configuration tests
  2386. * [Feature] Improve SPF results in Authentication-Results
  2387. * [Feature] Support ClickHouse database
  2388. * [Fix] Add failsafety for utf8 regexps
  2389. * [Fix] Do not trigger BROKEN_CONTENT_TYPE on innocent text parts
  2390. * [Fix] Emit error if connection has been terminated with no stop pattern
  2391. * [Fix] Fix boundaries checks in embedded messages
  2392. * [Fix] Fix double free
  2393. * [Fix] Perform policy downgrade on sample out, add tests
  2394. * [Fix] Properly escape utf8 regexps in hyperscan mode
  2395. * [Fix] Selectors - attachments args condition
  2396. * [Fix] Some fixes for raw parts
  2397. * [Fix] Treat learning errors as non-fatal
  2398. * [Fix] Use tld when looking for DKIM domains
  2399. * [Project] Words unicode structure rework
  2400. * [Project] Add preliminary Redis Sentinel support
  2401. * [Project] Improve Authentication-Results header
  2402. * [Project] Rework DKIM checks results
  2403. * [Project] Use more generalised API to produce meta words
  2404. 1.8.2: 19 Nov 2018
  2405. * [Conf] Add DWL support in the default configuration
  2406. * [Conf] Disable rspamd_update by default (again)
  2407. * [Conf] Fix configuration sample for ratelimit
  2408. * [CritFix] Disable broken url tags by default
  2409. * [CritFix] Fix \0 processing when doing RSA sign
  2410. * [CritFix] Fix adding symbols to their primary groups
  2411. * [Feature] Add `rspamadm cookie` utility
  2412. * [Feature] Add specialised functions for generating encrypted cookies
  2413. * [Feature] Add support of cookies in replies module
  2414. * [Feature] Add support of words regexps
  2415. * [Feature] Allow to add 3rd party clang plugins
  2416. * [Feature] Allow to create lua regexps from glob or plain patterns
  2417. * [Feature] Allow to set custom limits for upstream lists
  2418. * [Feature] Detect orphaned parts and attach them to message
  2419. * [Feature] Filter tokens in bayes
  2420. * [Feature] Fold b= value when doing arc sealing
  2421. * [Feature] Ignore cookies in the future and too old in the past
  2422. * [Feature] Skip stop words in statistics
  2423. * [Feature] Store stop words and allow to query them
  2424. * [Feature] Support query arguments in controller's custom commands
  2425. * [Feature] Tune upstream limits in Rspamd proxy
  2426. * [Feature] Use different callback symbols for different uribls
  2427. * [Feature] Write DKIM selector in dkim allow/reject symbols
  2428. * [Fix] Add obs_fws state support to eoh state machine
  2429. * [Fix] Add sanity check when applying mime boundaries heuristic
  2430. * [Fix] Antivirus - virus names with 0 were recognized as tables
  2431. * [Fix] Disable headernames in bayes temporarily
  2432. * [Fix] Do not allow syntax errors in include files...
  2433. * [Fix] Do not allow to merge an object with an array (or vice versa)
  2434. * [Fix] Don't perform forged recipients check for missing recipients
  2435. * [Fix] Fix DKIM based RBLs
  2436. * [Fix] Fix actrie implementation (sync from upstream), fixed OOB read
  2437. * [Fix] Fix explicit methods call in selectors
  2438. * [Fix] Fix extraction of additional parts
  2439. * [Fix] Fix finalization for internal plugins
  2440. * [Fix] Fix override_defaults function
  2441. * [Fix] Fix squeezed symbols when using settings
  2442. * [Fix] Fix urls insertion in Clickhouse module
  2443. * [Fix] Furhter fixes to ratelimits logic
  2444. * [Fix] Ignore signatures when looking for boundaries
  2445. * [Fix] Properly set learned count
  2446. * [Fix] Really fix ratelimits configuration and work
  2447. * [Fix] Remove ambigious format flag from printf
  2448. * [Fix] Restore URLs exporting in ClickHouse plugin
  2449. * [Fix] Rework bayes calculations...
  2450. * [Fix] Switch from chi-square to naive for large Fisher value
  2451. * [Fix] Treat normal password as enable password if there is no enable password
  2452. * [Fix] Use proper syntax for making DNS requests
  2453. * [Fix] Various fixes in embedded plugins
  2454. * [Project] Change fuzzy check selection logic to lua_fuzzy library
  2455. * [Project] Rework async events and symbols
  2456. * [Project] Move all metatokens in Bayes to lua_stat from C
  2457. * [WebUI] Add history rows per page control
  2458. 1.8.1: 16 Oct 2018
  2459. * [CritFix] Fix options insertion
  2460. * [CritFix] Fix words decay one more time (affects long messages)
  2461. * [CritFix] Increase default words_decay
  2462. * [CritFix] Plug memory leak in redis pool
  2463. * [Feature] Add `check_violation` feature to DKIM/ARC signing
  2464. * [Feature] Add only unique elements to Clickhouse url arrays
  2465. * [Feature] Allow `g+:` and `g-:` composite atoms
  2466. * [Feature] Allow dkim domains check in surbl
  2467. * [Feature] Allow maps with HTTP auth
  2468. * [Feature] Allow to disable actions by users settings
  2469. * [Feature] Extend whitelisting options
  2470. * [Feature] Store url object in images
  2471. * [Feature] Use verdict instead of the plain action in plugins
  2472. * [Fix] Allow to call fstring append with NULL string
  2473. * [Fix] DCC - luacheck
  2474. * [Fix] Do not load torch on each rspamadm invocation
  2475. * [Fix] Fix boundaries detection and rework stop words algorithm
  2476. * [Fix] Fix dependencies for DNS_SIGNED symbol
  2477. * [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit
  2478. * [Fix] Fix groups mess
  2479. * [Fix] Fix groups mess
  2480. * [Fix] Fix parsing address with comments
  2481. * [Fix] Fix resolving in DMARC reports
  2482. * [Fix] Fix various issues with parsing of the received headers
  2483. * [Fix] Fix watchers issue in lua_tcp when doing no resolving
  2484. * [Fix] Plug memory leak in language detector (affects reloads)
  2485. * [Fix] Remove one letter stop words
  2486. * [Fix] Slashing: backport chunk logic from libucl
  2487. * [Fix] Stop libevent from using cached time in rspamadm
  2488. * [Fix] Try to fix watchers chaining
  2489. * [Fix] Various fixes in redis sync interface
  2490. * [Fix] ip_score - respect check_authed and check_local settings from config
  2491. * [Project] Rework passthrough actions
  2492. * [Project] Clustering module
  2493. * [Rework] Always create result for a task
  2494. * [Rework] Completely rewrite DMARC checks logic
  2495. * [Rework] Rework and fix whitelist plugin
  2496. * [WebUI] Add symbols sorting buttons
  2497. * [WebUI] Change symbols order without updating history
  2498. * [WebUI] Colorize symbols
  2499. * [WebUI] Do not display password form when secure_ip is set
  2500. * [WebUI] Fix symbol description tooltips display
  2501. * [WebUI] History: add sorting by symbol score value
  2502. 1.8.0: 24 Sep 2018
  2503. * [Feature] Add arguments schemas to processors and extractors
  2504. * [Feature] Add functional selectors library
  2505. * [Feature] Add generic selector to reputation module
  2506. * [Feature] Add more ratelimits: by digest, by attachments data, by filenames
  2507. * [Feature] Add preliminary stop words detection support
  2508. * [Feature] Add pure Lua debugm function
  2509. * [Feature] Add schema validation for Redis settings
  2510. * [Feature] Add selectors combine function
  2511. * [Feature] Add some recursion protection to lua logger
  2512. * [Feature] Add support for Lua API tracing
  2513. * [Feature] Allow to apply schema to arguments
  2514. * [Feature] Allow to get dkim signing data directly from HTTP headers
  2515. * [Feature] Allow to reuse existing authentication results
  2516. * [Feature] Cache selectors results in re runtime
  2517. * [Feature] Implement new text tokenizer based on libicu
  2518. * [Feature] Integrate selectors framework to multimap
  2519. * [Feature] Relax FORGED_RECIPIENTS
  2520. * [Feature] Support (almost) all html entities
  2521. * [Feature] Support adding and deletion of recipients in the milter block
  2522. * [Feature] Support gathering HTTP body from fragments in lua_http
  2523. * [Feature] Support multi flag in regexp and glob maps
  2524. * [Feature] Support selectors in ratelimit module
  2525. * [Feature] Support selectors in settings
  2526. * [Feature] Use khash in HTML parser
  2527. * [Feature] Use pure Lua debugm function
  2528. * [Fix] Add fail-safety for destroying sessions
  2529. * [Fix] Allow to add result-less fake DNS records
  2530. * [Fix] Another try to fix race conditions on config unload
  2531. * [Fix] Call Lua callback on DNS timeouts
  2532. * [Fix] Deprecate task:inc_dns_req as it is redundant
  2533. * [Fix] Do not allow events deletions on cleanup
  2534. * [Fix] Do not try to process skipped messages
  2535. * [Fix] Fix HTTP requests with no body
  2536. * [Fix] Fix another cleanup race condition
  2537. * [Fix] Fix bug in processing of pcre regexps
  2538. * [Fix] Fix byte array allocation in the pool
  2539. * [Fix] Fix crashes on task cleanup
  2540. * [Fix] Fix dynamic buckets in ratelimits
  2541. * [Fix] Fix endless loop when waiting for Rspamd to stop
  2542. * [Fix] Fix lua_util.str_split in case of delimiters set
  2543. * [Fix] Fix more issues with watching of async events
  2544. * [Fix] Fix stop words detection and loading logic
  2545. * [Fix] Fix various corner cases for language detection
  2546. * [Fix] Fix watchers in lua_tcp
  2547. * [Fix] Fix words decay algorithm
  2548. * [Fix] Implement watchers replacement to handle nested calls
  2549. * [Fix] Save faked code into fake dns record
  2550. * [Fix] Show the proper frame when using lua_util.debugm
  2551. * [Fix] Use fake dns records in tests
  2552. * [Fix] Use unicode replacements for HTML entities
  2553. * [Fix] fixed "cannot find dependency on symbol 1" issue when using replaced symbols in spamassassin rules
  2554. * [Fix] partition_id is not available in old versions of CH
  2555. * [Project] Add implicit conversion logic to selectors
  2556. * [Project] Add initial support for selectors in regexps
  2557. * [Project] Add method concept
  2558. * [Project] Further changes in unicode operations
  2559. * [Project] Implement Clickhouse migrations
  2560. * [Project] Implement implicit conversions to userdata
  2561. * [Project] Implement insert method
  2562. * [Project] Implement selectors registration for regular expressions
  2563. * [Project] Implement selectors support in re_cache
  2564. * [Project] Improve language detector: cleanup unused files, categorize
  2565. * [Project] Migrate CH data to a fat table
  2566. * [Project] Rework selectors logic
  2567. * [Project] Start Clickhouse utilities library
  2568. * [Project] Start unicode rework
  2569. * [Project] coroutine threaded model for API calls: thread pool
  2570. * [Rework] Move phishtank to a DNS based service
  2571. * [Rework] Rework Clickhouse plugin to use the new API
  2572. * [Rework] Rework language detector
  2573. * [Rework] Rework utf content processing in text parts
  2574. * [WebUI] Add progress bar for AJAX requests
  2575. * [WebUI] Avoid errors table reinitialization
  2576. * [WebUI] Avoid history table reinitialization
  2577. * [WebUI] Avoid throughput summary table reinitialization
  2578. * [WebUI] Destroy summary table on disconnect
  2579. * [WebUI] Fix "auth" request URL
  2580. * [WebUI] Fix disabling and hiding controls on page reload
  2581. * [WebUI] Fix maps loading from neighbours
  2582. * [WebUI] Fix symbols sorting by score
  2583. * [WebUI] Fix tables destroying
  2584. * [WebUI] Fix throughput data consolidation
  2585. * [WebUI] Fix upload buttons disabling
  2586. * [WebUI] Notify user on module loading failure
  2587. * [WebUI] Update FooTable 3.1.4 -> 3.1.6
  2588. 1.7.9: 01 Aug 2018
  2589. * [CritFix] Fix caseless comparison of equal length strings
  2590. * [Feature] Add HTTP basic auth support to elastic and clickhouse plugins
  2591. * [Feature] Add SPF selector to reputation
  2592. * [Feature] Add support of the fallback backends for HTTP maps
  2593. * [Feature] Allow to print full mime structure when extracting mime data
  2594. * [Feature] Allow to split symbols in reputation plugin
  2595. * [Feature] Check attachments only on AV scanners in attachments_only mode
  2596. * [Feature] Disable all SSL checks if ssl_no_verify flag is set
  2597. * [Feature] Implement parsing of scoped IPv6 addresses
  2598. * [Feature] Improve `rspamc counters` output
  2599. * [Fix] Add sanity checks when expanding SPF macros
  2600. * [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack)
  2601. * [Fix] Avoid one extra byte writing
  2602. * [Fix] Deal with direct hash table
  2603. * [Fix] Detect empty text part as text, not HTML
  2604. * [Fix] Do not reduce map watch timeout for mixed http/file maps
  2605. * [Fix] Fix HTML part detection heuristic
  2606. * [Fix] Fix double free in redirectors cleanup
  2607. * [Fix] Fix legacy history handling in the controller
  2608. * [Fix] Fix messages insertion
  2609. * [Fix] Fix sending string method
  2610. * [Fix] Fix statconver command line arguments
  2611. * [Fix] Fixed argument checking for being null
  2612. * [Fix] Fixed issues reported by luacheck
  2613. * [Fix] Freeze updates queue when do actual storage update
  2614. * [Fix] HTTP map hash is per-backend and not per-map
  2615. * [Fix] Plug memory leak in fuzzy updates
  2616. * [Fix] Prefer 'MTA-Name' when producing authentication results
  2617. * [Fix] Replace bad unicode sequences instead of stopping on them
  2618. * [Fix] Set classifier version on learning
  2619. * [Project] Reworked ratelimits
  2620. * [Project] Apply topological sorting for symbols in Rspamd
  2621. * [Project] Remove global contexts from C modules
  2622. * [Project] Move performance critical hash tables to khash
  2623. * [WebUI] Avoid unused indexes
  2624. * [WebUI] Do not execute `on_success` callback
  2625. * [WebUI] Fix history reset for "All SERVERS" (#2346)
  2626. * [WebUI] Fix query URL for selected server
  2627. * [WebUI] Fix symbols display in legacy history,
  2628. * [WebUI] Hide symbols order selector for legacy history
  2629. * [WebUI] Refactor query functions into one
  2630. * [WebUI] Remove previously-attached event handlers
  2631. * [WebUI] Save symbols to the selected server
  2632. * [WebUI] Unify arguments of query functions
  2633. * [WebUI] Use common query functions to get graph data
  2634. * [WebUI] Use common query functions to save symbols
  2635. 1.7.8: 12 Jul 2018
  2636. * [Feature] Add more extended statistics about fuzzy updates
  2637. * [Feature] Add more non-conformant Received headers support
  2638. * [Feature] Add preliminary function to get fuzzy hashes from text in Lua
  2639. * [Feature] Allow to configure AV module rejection message
  2640. * [Feature] Implement fuzzy hashes extraction in mime tool
  2641. * [Feature] Improve WHITE_ON_WHITE rule
  2642. * [Feature] Improve integer -> string conversion
  2643. * [Feature] Reuse maps in multimap module more aggressively
  2644. * [Fix] Avoid race condition in skip map as pool lifetime is not enough
  2645. * [Fix] Eliminate all specific C plugins pools
  2646. * [Fix] Fix DKIM check rule if DNS is unavailable
  2647. * [Fix] Fix build where ucontext is defined in ucontext.h
  2648. * [Fix] Fix crash in base url handling
  2649. * [Fix] Fix descriptors leak in sqlite3 locking code
  2650. * [Fix] Fix messages quarantine
  2651. * [Fix] Fix padded numbers printing
  2652. * [Fix] Fix race condition on maps reinit
  2653. * [Fix] Fix regexp functions when no data is passed
  2654. * [Fix] Fix specific urls extraction
  2655. * [Fix] Fix styles propagation
  2656. * [Fix] Improve resetting of the limit buckets
  2657. * [Fix] Initialize sqlite3 properly
  2658. * [Fix] Work with broken resolvers in resolv.conf
  2659. * [Project] Implement HTTP maps caching
  2660. * [Project] Refresh fuzzy hashes when matched
  2661. * [Project] Add logic to deduplicate fuzzy updates queue
  2662. * [WebUI] Add missed declarations
  2663. * [WebUI] Avoid using "undefined" property
  2664. * [WebUI] Do not accept passwords containing control characters
  2665. * [WebUI] Do not redeclare variables
  2666. * [WebUI] Enable strict mode,
  2667. * [WebUI] Fix variable assignment
  2668. * [WebUI] Initialize variables at declaration
  2669. * [WebUI] Remove duplicated path from RequireJS config
  2670. * [WebUI] Remove unused block
  2671. * [WebUI] Remove unused variable
  2672. * [WebUI] Remove unused variables
  2673. * [WebUI] Use self-explanatory notation
  2674. * [WebUI] Use type-safe equality operators
  2675. 1.7.7: 02 Jul 2018
  2676. * [CritFix] Check NM part of pubkey to match it with rotating keypairs
  2677. * [CritFix] Do not overwrite PID of the main process
  2678. * [CritFix] Fix maps after reload
  2679. * [CritFix] Fix maps race conditions on reload
  2680. * [CritFix] Fix shmem leak in encrypting proxy mode
  2681. * [Feature] Add a concept of ignored symbols to avoid race conditions
  2682. * [Feature] Add ability to print bayes tokens in rspamadm mime
  2683. * [Feature] Add method to get statistical tokens in Lua API
  2684. * [Feature] Add preliminary mime stat command
  2685. * [Feature] Add rspamadm mime tool
  2686. * [Feature] Add urls extraction tool
  2687. * [Feature] Address ZeroFont exploit
  2688. * [Feature] Allow rspamadm mime to process multiple files
  2689. * [Feature] Allow to extract words in `rspamadm mime`
  2690. * [Feature] Allow to print mime part data
  2691. * [Feature] Allow to show HTML structure on extraction
  2692. * [Feature] Distinguish IP failures from connection failures
  2693. * [Feature] Improve output for mime command
  2694. * [Feature] Improve styles propagation
  2695. * [Feature] Main process crash will now cleanup all children
  2696. * [Feature] Preload file and static maps in main process
  2697. * [Feature] Print stack trace on crash
  2698. * [Feature] Process font size in HTML parser
  2699. * [Feature] Propagate content length of invisible tags
  2700. * [Feature] Read ordinary file maps in chunks to be more safe on rewrites
  2701. * [Feature] Support base tag in HTML
  2702. * [Feature] Support more size suffixes when parsing HTML styles
  2703. * [Feature] Support opacity style
  2704. * [Fix] Another fix for nested composites
  2705. * [Fix] Fill nm id in keypairs cache code
  2706. * [Fix] Fix colors alpha channel handling
  2707. * [Fix] Fix destruction logic
  2708. * [Fix] Fix double free
  2709. * [Fix] Fix maps preload logic
  2710. * [Fix] Fix nested composites process
  2711. * [Fix] Fix proxying of Exim connections
  2712. * [Fix] Fix reload crash
  2713. * [Fix] Fix rspamadm -l command
  2714. * [Fix] Update ed25519 signing schema
  2715. * [WebUI] Stop using "const" declaration
  2716. * [WebUI] Update RequireJS to 2.3.5
  2717. 1.7.6: 15 Jun 2018
  2718. * [CritFix] Fix multiple neural networks support
  2719. * [Feature] Add decryption function to keypair command
  2720. * [Feature] Add gzip compression for HTTP requests in elastic module
  2721. * [Feature] Add gzip methods to lua util
  2722. * [Feature] Add maps based on Top Level Domains
  2723. * [Feature] Add pubkey checks for dkim_signing
  2724. * [Feature] Add support of fake DNS records
  2725. * [Feature] Add tool to encrypt files
  2726. * [Feature] Allow to add symbols using settings directly
  2727. * [Feature] Allow to match private and public keys for DKIM signatures
  2728. * [Feature] Allow to set task flags via settings
  2729. * [Feature] Allow to specify fake DNS address from the config
  2730. * [Feature] Implement signatures verification using rspamadm keypair
  2731. * [Feature] Implement signing using `rspamadm keypair`
  2732. * [Feature] Improve error reporting for DKIM key access issues
  2733. * [Feature] Provide $HOSTNAME variable in UCL
  2734. * [Feature] Rework levenshtein distance computation
  2735. * [Feature] Split message parsing and processing
  2736. * [Feature] Support ED25519 DKIM signatures
  2737. * [Feature] Support encrypted configs in UCL
  2738. * [Feature] Suppress duplicate warning on very large radix tries
  2739. * [Feature] Use OSB to combine header names
  2740. * [Fix] Cleanup maps data on shutdown
  2741. * [Fix] Fix '~' behaviour in composites
  2742. * [Fix] Fix HTTP maps updates
  2743. * [Fix] Fix NIST signatures
  2744. * [Fix] Fix RFC822 comments when processing a mime address
  2745. * [Fix] Fix double free
  2746. * [Fix] Fix dynamic settings application
  2747. * [Fix] Fix for CommuniGate Pro maillist
  2748. * [Fix] Fix keypair creation method to actually create keypair...
  2749. * [Fix] Fix matching patterns with no paths
  2750. * [Fix] Fix memory leak in parsing comments
  2751. * [Fix] Fix parsing of urls with numeric password
  2752. * [Fix] Fix plugins intialisation in configwizard
  2753. * [Fix] Fix potential crash on reload
  2754. * [Fix] Fix potential race condition for a finished HTTP connections
  2755. * [Fix] Fix race-condition leak on processes reload
  2756. * [Fix] Fix signing in openssl mode
  2757. * [Fix] Free language detector structures
  2758. * [Fix] Relax alignment requirements
  2759. * [Fix] Send DMARC reports compressed
  2760. * [Fix] Try to fix leak in dmarc module
  2761. * [Fix] Try to plug memory leak in metric exporter
  2762. * [Project] Convert rspamadm subcommands to Lua
  2763. * [WebUI] Display smtp sender/recipient in history
  2764. * [WebUI] Fix elements disabling in "Symbols" tab
  2765. * [WebUI] Limit recipients list in history column to 3
  2766. * [WebUI] Match envelope and mime addresses following in arbitrary order
  2767. * [WebUI] Update column header
  2768. * [WebUI] Wrap addresses in history
  2769. 1.7.5: 18 May 2018
  2770. * [Conf] Add MSBL proposed return codes
  2771. * [Conf] Add additional groups for policies
  2772. * [CritFix] Do not use volatile Lua strings as UCL keys
  2773. * [Feature] Add ability to add fuzzy hashes to headers
  2774. * [Feature] Add function to extract most meaningful urls
  2775. * [Feature] Add rule to block mixed text and encrypted parts
  2776. * [Feature] Allow multiple groups for symbols
  2777. * [Feature] Allow to disable lua squeezing logic
  2778. * [Feature] Allow to get multipart children in Lua
  2779. * [Feature] Allow to insert multiple headers from milter headers
  2780. * [Feature] Allow to print scores in subject and further extensions
  2781. * [Feature] Be more error-prone in squeezed rules
  2782. * [Feature] Support multiple return codes in emails module
  2783. * [Feature] Use EMA for calculating averages
  2784. * [Feature] Use common jit cache for all regexps
  2785. * [Feature] support for CommuniGate Pro self-generated messages
  2786. * [Fix] Allow to have multiple values for headers as arrays
  2787. * [Fix] Do not open sockets for disabled workers
  2788. * [Fix] Fix AuthservId
  2789. * [Fix] Fix base64 folding in Lua API
  2790. * [Fix] Fix build on non-x86 platforms
  2791. * [Fix] Fix cached maps logic
  2792. * [Fix] Fix compatibility with old maps query logic
  2793. * [Fix] Fix crash if skip_map is used
  2794. * [Fix] Fix importing static maps from UCL
  2795. * [Fix] Fix parsing of unix sockets
  2796. * [Fix] Fix raw_mime regexp on HTML part with no text content
  2797. * [Fix] Fix tables logging
  2798. * [Fix] Fix vertical tab handling in libucl
  2799. * [Fix] Try to fix frequency counters
  2800. * [Fix] Use better sharding for ip_score
  2801. * [Fix] Use multiple results from SURBL DNS reply
  2802. * [Fix] When doing AV scan select a different server for retransmit
  2803. 1.7.4: 01 May 2018
  2804. * [Conf] Major stock config updates:
  2805. - Workers are now specified in a new format worker "type" { ... }
  2806. - Enable fuzzy worker to simplify local fuzzy storages configuration
  2807. - Bind all workers to localhost by default to avoid security flaws
  2808. * [Conf] Make more sane fuzzy_check default settings
  2809. * [CritFix] Fix ucl escape for bad symbols
  2810. * [Feature] Add failure symbol for AV module
  2811. * [Feature] Add lazy expiration mode for new classifier schema
  2812. * [Feature] Add preliminary version of maps stats plugin
  2813. * [Feature] Allow to block fuzzy requests from specific networks
  2814. * [Feature] Allow to change `expire` of live statistics
  2815. * [Feature] Distinguish AV failure from clean result
  2816. * [Feature] Further improvements of language detector by using khash
  2817. * [Feature] Further optimization of the lang_detection
  2818. * [Feature] Implement cluster-aware bayes expiry
  2819. * [Feature] Implement exclude patterns in rspamc
  2820. * [Feature] Implement glob maps in addition to regexp maps
  2821. * [Feature] Implement map statistics function for lua API
  2822. * [Feature] Implement stop symbols for Clickhouse collection
  2823. * [Feature] Support recipients separated by commas
  2824. * [Feature] Try harder to upload scripts to the Redis server
  2825. * [Feature] Upgrade t1ha distribution
  2826. * [Feature] use_domain_sign_inbound
  2827. * [Feature] Use scores from maps if `symbols_set` is not defined
  2828. * [Fix] Add resolving version of radix map helper
  2829. * [Fix] Check URL before adding implicit prefix
  2830. * [Fix] Do not check pid/state when using PRNG
  2831. * [Fix] Fix CentOS logrotate script for systemd
  2832. * [Fix] Fix slash + dot in urls
  2833. * [Fix] Fix systemd version of the logrotate script
  2834. * [Fix] Propagate key when import implicit array from Lua
  2835. * [Fix] Strip spaces from map keys and values
  2836. * [Fix] Try to fix a specific case when processing milter protocol
  2837. * [Fix] Try to fix crash when a tcp connection cannot be set
  2838. * [Fix] Typo use_domain_local --> use_domain_sign_local
  2839. * [Fix] Various fixes to once_received module
  2840. * [Project] Store hits counters for map elements
  2841. 1.7.3: 10 Apr 2018
  2842. * [CritFix] Plug bad memory leak in protocol reply
  2843. * [Feature] Add avx2 codec for base64
  2844. * [Feature] Add method to receive all URL flags from Lua API
  2845. * [Feature] Allow to fold headers on stop characters
  2846. * [Feature] Allow to set lua_cpath from options
  2847. * [Feature] Allow to specify custom rejection message in milter
  2848. * [Feature] Deal with unnormalised Unicode obfuscation
  2849. * [Feature] Do not detect language twice for relative parts
  2850. * [Feature] Implement oversigning feature
  2851. * [Feature] Implement silent logging level to minimize noise in logs
  2852. * [Feature] Improve URL_IN_SUBJECT rule
  2853. * [Feature] Use hashing to reduce redis attack surface
  2854. * [Fix] Add oversigning for the most important headers
  2855. * [Fix] add 'rewrite subject' to History dropdown
  2856. * [Fix] Another fix in folding algorithm
  2857. * [Fix] Do not call multimap addr for parts of addr if filter is presented
  2858. * [Fix] Do not clean hostname on generic reset
  2859. * [Fix] Do not create pid file in no-fork mode
  2860. * [Fix] Fix fold_after case to preserve multiple spaces
  2861. * [Fix] Fix folding and folding tests
  2862. * [Fix] Fix hostname usage in milter mode
  2863. * [Fix] Fix lua RSA verify and its tests
  2864. * [Fix] Fix metadata exporter send_mail backend (#2124)
  2865. * [Fix] Fix processing of '\v' in libucl
  2866. * [Fix] Fix shemaless URLs detection
  2867. * [Fix] Fix support of multiple headers in sign_header
  2868. * [Fix] Fix usage of util.parse_mail_address
  2869. * [Fix] Fix weights of dynamic squeezed rules
  2870. * [Fix] Leak from bucket before checking the burst
  2871. * [Fix] Stop using own localtime as DST could be messy in many cases
  2872. * [Fix] Treat unnormalised URLs as obscured
  2873. * [Rework] Restore leaky bucket model in ratelimit plugin
  2874. * [WebUI] Add messages total to throughput summary
  2875. * [WebUI] Add symbols order selector to history
  2876. * [WebUI] Config: Load list on demand
  2877. * [WebUI] Fix modalBody for maps that appear more than once
  2878. * [WebUI] History: Fix Tooltips on paging, filtering and sorting
  2879. * [WebUI] Remove a previously-attached event handler
  2880. * [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1
  2881. 1.7.2: 23 Mar 2018
  2882. * [Feature] Store emails in Clickhouse
  2883. * [Feature] Support single quotes in config
  2884. * [Feature] Use templates when publishing CH schema
  2885. * [Feature] Improve Docker image
  2886. * [Fix] Add rounding when printing a lot of FP variables
  2887. * [Fix] Allow to disable certain actions by assigning null to them
  2888. * [Fix] Disable results caching
  2889. * [Fix] Fix disabling of squeezed symbols
  2890. * [Fix] Fix scan time set
  2891. * [Fix] Rework logic of actions setting
  2892. * [Fix] Try to fix various Lua stack issues
  2893. * [WebUI] Add link tag for favicon.ico
  2894. * [WebUI] Display hostname:port/path in the page title
  2895. 1.7.1: 20 Mar 2018
  2896. * [CritFix] Fix lowercase comparison
  2897. * [CritFix] Timezone defines seconds WEST UTC not East
  2898. * [Feature] Add filename to log format
  2899. * [Feature] Add lua rules squeezing
  2900. * [Feature] Add related symbols analysis to rspamd_stats
  2901. * [Feature] Remove upstream `X-Spam: Yes` header by default
  2902. * [Feature] rspamd_stats: Output progress info on STDERR
  2903. * [Feature] Whitelist for emails module
  2904. * [Fix] Do not allow dependencies on self
  2905. * [Fix] Do not cache metric result
  2906. * [Fix] Do not trust all issuers as a client certificate
  2907. * [Fix] Fix dependencies in lua squeeze
  2908. * [Fix] Fix enabling/disabling squeezed rules
  2909. * [Fix] Fix enabling/disabling symbols
  2910. * [Fix] Fix external dependencies
  2911. * [Fix] Fix processing of a single compressed file
  2912. * [Fix] Fix some typos
  2913. * [Fix] Fix various modules in case of empty message
  2914. * [Fix] Handle callbacks that returns table of options
  2915. * [Fix] Improve cached action interaction
  2916. * [Fix] Make dynamic conf more NaN aware
  2917. * [Fix] Never hide actions from WebUI `configuration` tab
  2918. * [Project] Implementation of Lua rules squeezing
  2919. 1.7.0: 12 Mar 2018
  2920. * [Conf] Add bayes_expiry as explicit module
  2921. * [Conf] Adjust names and weights for neural network plugin
  2922. * [Conf] Change updates url
  2923. * [Conf] Default statistics is stored in Redis now
  2924. * [Conf] Disable fann_redis module by default
  2925. * [Conf] Fix default elastic configuration
  2926. * [Conf] Fix double quote position
  2927. * [Conf] Massive config rework for new structure of symbols and scores
  2928. * [Conf] Rename Rambler BLs as they are now Rspamd's ones
  2929. * [Conf] Use dedicated rspamd.com subdomains
  2930. * [Conf] Use more data from rspamd.com fuzzy storage
  2931. * [CritFix] Add sanity guards for badly broken HTML
  2932. * [CritFix] Another errors path handling fix
  2933. * [CritFix] Another portion of tokenization fixes
  2934. * [CritFix] Do not send reject messages after set reply
  2935. * [CritFix] Fix ARC chain verification
  2936. * [CritFix] Fix crash in milter errors handler
  2937. * [CritFix] Fix memory leak in spf caching logic
  2938. * [CritFix] Fix milter commands pipelining
  2939. * [CritFix] Fix newlines detection
  2940. * [CritFix] Fix semicolons parsing in the content type
  2941. * [CritFix] Plug memory leak in zstd protocol compression
  2942. * [Feature] Add ability to match score in force_actions module
  2943. * [Feature] Add aes-rng PRF to libottery
  2944. * [Feature] Add 'composites' debug module
  2945. * [Feature] Add concept of experimental modules
  2946. * [Feature] Add DKIM trace symbol
  2947. * [Feature] Add EBL to the default config
  2948. * [Feature] Add expected ip check for emails plugin
  2949. * [Feature] Add framework to manage Redis scripts
  2950. * [Feature] Add framing for the new reputation generic plugin
  2951. * [Feature] Add function to show plugins stat
  2952. * [Feature] Add gzip compression support for clickhouse module
  2953. * [Feature] Add gzip compression support for rspamd controller
  2954. * [Feature] Add gzip support when sending lua http requests
  2955. * [Feature] Add json output for rspamd_stats
  2956. * [Feature] Add method to do a synchronous Redis connection
  2957. * [Feature] Add method to get all content-type attributes in Lua
  2958. * [Feature] Add `-m` flag to configdump to show modules states
  2959. * [Feature] Add mime types to extensions map
  2960. * [Feature] Add more features to rescore utility
  2961. * [Feature] Add more gtube like patterns to test other spam actions
  2962. * [Feature] Add more metafunctions, improve logging
  2963. * [Feature] Add more text attributes
  2964. * [Feature] Add new configwizard command to rspamadm
  2965. * [Feature] Add new tooling for stats conversation
  2966. * [Feature] Add old groups migration tool
  2967. * [Feature] Add plugins state variable
  2968. * [Feature] Add preliminary ecdsa keys support in DKIM
  2969. * [Feature] Add preliminary support of idempotent symbols
  2970. * [Feature] Add Redis server wizard
  2971. * [Feature] Add routine to convert old style stats to a new one
  2972. * [Feature] Add some sanity checks for actions and controller
  2973. * [Feature] Add statistic convertation module to configwizard
  2974. * [Feature] Add sugestions logic to mempool allocator
  2975. * [Feature] Add support of config transform in Lua
  2976. * [Feature] Add timeout to rspamc when doing corpus test
  2977. * [Feature] Add tooling to convert bayes schemas
  2978. * [Feature] Add torch conditional to configuration
  2979. * [Feature] Add torch-decisiontree package
  2980. * [Feature] Add torch-optim contrib package
  2981. * [Feature] Add TTL autodetection
  2982. * [Feature] Add urls reputation to the reputation framework
  2983. * [Feature] Allow floating and negative values in expressions limits
  2984. * [Feature] Allow multiple CTs in full extensions map
  2985. * [Feature] Allow multiple fann rules
  2986. * [Feature] Allow randomly select User-Agent from a list
  2987. * [Feature] Allow rspamadm commands to export methods in Lua
  2988. * [Feature] Allow rule specific min_bytes in fuzzy check
  2989. * [Feature] Allow to adjust symbols scores from Lua
  2990. * [Feature] Allow to attach stat signature to messages
  2991. * [Feature] Allow to change SMTP from via milter headers
  2992. * [Feature] Allow to configure monitored
  2993. * [Feature] Allow to create directories in Lua API
  2994. * [Feature] Allow to disable torch and skip train samples for ANN
  2995. * [Feature] Allow to discard messages dynamically
  2996. * [Feature] Allow to enable/disable languages from the detector
  2997. * [Feature] Allow to generate DKIM keys from rspamadm API
  2998. * [Feature] Allow to get CPU flags from Lua
  2999. * [Feature] Allow to have high precision timestamps in logs
  3000. * [Feature] Allow to insert headers into specific position
  3001. * [Feature] Allow to limit redirector requests per task
  3002. * [Feature] Allow to load and use dynamic ANNs with torch
  3003. * [Feature] Allow to quarantine rejected messages using milter interface
  3004. * [Feature] Allow to receive signing keys from mempool vars
  3005. * [Feature] Allow to reserve elements in libucl
  3006. * [Feature] Allow to reuse signal handlers chains
  3007. * [Feature] Allow to set custom mempool variables from settings
  3008. * [Feature] Allow to set headers from settings
  3009. * [Feature] Allow to set Settings-Id for all connections
  3010. * [Feature] Allow to skip real action and add a header instead
  3011. * [Feature] Allow to skip specific hashes in fuzzy storage
  3012. * [Feature] Allow to spawn asynchronous processes from Lua
  3013. * [Feature] Allow to specify number of threads for ANN learning
  3014. * [Feature] Allow to use global lua maps in settings
  3015. * [Feature] Allow to use postfilters in composites
  3016. * [Feature] Allow to verify signatures from HTTP headers in maps
  3017. * [Feature] Antivirus: ordered pattern matches
  3018. * [Feature] Authentication-Results: support hiding usernames
  3019. * [Feature] Automatically create tables in clickhouse
  3020. * [Feature] Catch next-to-last bad extension
  3021. * [Feature] Check cached maps more frequently
  3022. * [Feature] Check groups sanity
  3023. * [Feature] Deal with obscured URLs with @ symbols
  3024. * [Feature] Enhance task:store_in_file method
  3025. * [Feature] Export password encryption routines to Redis
  3026. * [Feature] Filter nan and inf when adding scores
  3027. * [Feature] Finalize 7zip files support
  3028. * [Feature] Further improvements in language detection
  3029. * [Feature] Further improvements in language detection algorithm
  3030. * [Feature] Generic key name expansion for Redis keys
  3031. * [Feature] Hash whitelist for fuzzy_check
  3032. * [Feature] Implement bayes signatures storage
  3033. * [Feature] Implement buckets for Redis backend
  3034. * [Feature] Implement DKIM reputation adjustments
  3035. * [Feature] Implement forked workers children monitoring
  3036. * [Feature] Implement headers flags in mime parser
  3037. * [Feature] Implement l1/l2 regularization against the current weights
  3038. * [Feature] Implement manual ANN train mode
  3039. * [Feature] Implement per-user ANN support
  3040. * [Feature] Implement torch based ANN learning
  3041. * [Feature] Implement upstreams logic for clickhouse exporter
  3042. * [Feature] Import torch to Rspamd...
  3043. * [Feature] Improve allocation policy when interacting with Lua
  3044. * [Feature] Improve Lua/C interaction in history_redis
  3045. * [Feature] Improve multiple fuzzy results combining
  3046. * [Feature] Improve parsing of DKIM keys: parse algorithm
  3047. * [Feature] Improve subprocesses termination handle
  3048. * [Feature] Improve symbol type parsing in Lua API
  3049. * [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600)
  3050. * [Feature] Milter headers: support adding/removing arbitrary headers from config
  3051. * [Feature] More metatokens
  3052. * [Feature] Multimap: checking of symbol options
  3053. * [Feature] Multimap: template URL filter
  3054. * [Feature] New bayes expiry plugin
  3055. * [Feature] Periodically save rspamd stats to disk
  3056. * [Feature] Preliminary import of the elasticsearch module
  3057. * [Feature] Ratelimit: allow full addresses in whitelisted_rcpts
  3058. * [Feature] Ratelimit: support fetching limits from Redis
  3059. * [Feature] RBL: received: filtering by position & flags
  3060. * [Feature] Read global maps for lua
  3061. * [Feature] Redis settings: support checking multiple keys
  3062. * [Feature] Rework fann plugin to be a normal post-filter
  3063. * [Feature] Rework logging configuration for rspamadm case
  3064. * [Feature] Rework short hashes generation to avoid FP
  3065. * [Feature] Save real ucl types when exporting to Lua
  3066. * [Feature] Set TCP_NODELAY for milter sockets
  3067. * [Feature] Setup DKIM signing from configwizard
  3068. * [Feature] Skip certain symbols from ANN classify
  3069. * [Feature] Store plugins state
  3070. * [Feature] Support etag for HTTP maps
  3071. * [Feature] Support Expires header when using HTTP maps
  3072. * [Feature] Support sending given header multiple times in lua_http
  3073. * [Feature] Support sha512 in DKIM signatures
  3074. * [Feature] Try to detect HTML messages better
  3075. * [Feature] Use array instead of queue to reduce memory fragmentation
  3076. * [Feature] Use controller port by default when connecting to local IP
  3077. * [Feature] Use rdtsc where possible
  3078. * [Fix] Actively load skip hashes map in fuzzy storage
  3079. * [Fix] Add another workaround to display history properly
  3080. * [Fix] Add definition for old glib compatibility method
  3081. * [Fix] Add missing rspamadm control options to help
  3082. * [Fix] Add workaround for IPv6 in sendmail
  3083. * [Fix] Add workaround for system with non-XSI compatible tzset
  3084. * [Fix] Allow oversigning in DKIM signatures
  3085. * [Fix] Allow to check negative scores in force_actions
  3086. * [Fix] Allow to have negative actions limits
  3087. * [Fix] Allow to set any layers number for fann rules
  3088. * [Fix] Another fix for rdtcs
  3089. * [Fix] Another fix to lua xmlrpc
  3090. * [Fix] Another try to deal with #1998
  3091. * [Fix] Another try to fix #1998
  3092. * [Fix] Another try to fix threading in torch
  3093. * [Fix] Apply language detection when adding fuzzy hashes
  3094. * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer
  3095. * [Fix] Authentication Results: Fix SPF smtp.mail_from
  3096. * [Fix] Auth-Results: Multiple DKIM signatures
  3097. * [Fix] Avoid changing content-transfer-encoding header's value
  3098. * [Fix] Better handling of the legacy protocol
  3099. * [Fix] Check decoded headers sanity (e.g. by excluding \0)
  3100. * [Fix] Check for magic when checking for an archive
  3101. * [Fix] Cleanup mess with groups
  3102. * [Fix] Clickhouse: Insertion in the symbols table
  3103. * [Fix] Crash in URL processing
  3104. * [Fix] Deal with another case when processing exceptions
  3105. * [Fix] Deal with deeply nested messages more aggressively
  3106. * [Fix] Deal with nan and inf encoding in json/ucl
  3107. * [Fix] Deal with non-key arguments in lua_redis.exec_script
  3108. * [Fix] Deal with unknown weight
  3109. * [Fix] Deal with URLs with no slashes after protocol
  3110. * [Fix] Deal with URLs wrapped in [] in text parts
  3111. * [Fix] Deal with zero scores symbols
  3112. * [Fix] Default monitoring domain for surbl plugin
  3113. * [Fix] Delay upstream re-resolving when one upstream is defined
  3114. * [Fix] Detection of maillist optimized and fixed
  3115. * [Fix] DKIM signing: allow for auth_only to be false
  3116. * [Fix] DMARC: require report_settings for sending reports only
  3117. * [Fix] Do not allow garbadge when checking url domain
  3118. * [Fix] Do not cache SPF records with PTR elements
  3119. * [Fix] Do not constantly re-resolve failed upstreams with a single element
  3120. * [Fix] Do not crash if no words defined
  3121. * [Fix] Do not crash on empty subtype
  3122. * [Fix] Do not expose spamtrap messages to SMTP reply
  3123. * [Fix] Do not fail rbl plugin when there are no received or emails
  3124. * [Fix] Do not ignore short words
  3125. * [Fix] Do not include idempotent/nostat symbols to checksum
  3126. * [Fix] Do not override groups when converting metrics
  3127. * [Fix] Do not override unix socket group when group comes before owner
  3128. * [Fix] Do not skip the last character
  3129. * [Fix] Do not spawn too many workers by default
  3130. * [Fix] Do not stop monitored on dns errors
  3131. * [Fix] Do not stop parsing headers on bad IP header
  3132. * [Fix] Do not strip last character in the last word
  3133. * [Fix] Do not treat script content as text
  3134. * [Fix] Do not try to connect to non-supported addresses
  3135. * [Fix] Do not try to dereference last character
  3136. * [Fix] Do not try to sign unknown domains
  3137. * [Fix] Don't use whitelist/greylist maps as regexp, but as map
  3138. * [Fix] Erase unknown HTML entities
  3139. * [Fix] Exim Received header protocol parsing
  3140. * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false
  3141. * [Fix] Fix a lot of FP in chartable in mixed languages
  3142. * [Fix] Fix ANN checks
  3143. * [Fix] Fix ANN loading logic
  3144. * [Fix] Fix another tokenization issue
  3145. * [Fix] Fix autolearn parameters reading
  3146. * [Fix] Fix bad archive characters stripping
  3147. * [Fix] Fix bad extension check
  3148. * [Fix] Fix bayes schema conversion
  3149. * [Fix] Fix blacklists and DMARC in whitelist
  3150. * [Fix] Fix brain-damaged torch build system
  3151. * [Fix] Fix build on FreeBSD
  3152. * [Fix] Fix clickhouse exporter
  3153. * [Fix] Fix clickhouse schema
  3154. * [Fix] Fix comparision
  3155. * [Fix] Fix composites processing
  3156. * [Fix] Fix connecting to a unix socket in rspamadm statconvert
  3157. * [Fix] Fix couple of warnings
  3158. * [Fix] Fix crashes in the rspamd_control path
  3159. * [Fix] Fix deletion from hash
  3160. * [Fix] Fix DKIM forgeries via multiple headers
  3161. * [Fix] FIx dynamic conf plugin
  3162. * [Fix] Fix emails detection
  3163. * [Fix] Fix empty headers simple canonicalization
  3164. * [Fix] Fix empty threshold check in greylisting module
  3165. * [Fix] Fix encrypted legacy reply in fuzzy storage
  3166. * [Fix] Fix enormous scores for R_WHITE_ON_WHITE
  3167. * [Fix] Fix exceptions list in surbl
  3168. * [Fix] Fix *_EXCESS_BASE64 rules
  3169. * [Fix] Fix expire rounding
  3170. * [Fix] Fix extra hits in PCRE mode for regular expressions
  3171. * [Fix] Fix format strings
  3172. * [Fix] Fix get_content method
  3173. * [Fix] Fix groups override when defining symbols
  3174. * [Fix] Fix learned count in new schema
  3175. * [Fix] Fix learn errors propagation
  3176. * [Fix] Fix loading of per-user redis backend for statistics
  3177. * [Fix] Fix logging buffer corruption in case of repeated messages
  3178. * [Fix] Fix lua cached elements invalidation
  3179. * [Fix] Fix merging of the implicit arrays
  3180. * [Fix] Fix mime_types scoring
  3181. * [Fix] Fix multiple headers in DKIM headers list
  3182. * [Fix] Fix null callee case in clang plugin
  3183. * [Fix] Fix obscured url in format user@@example.com
  3184. * [Fix] Fix parsing of the per-user script
  3185. * [Fix] Fix priorities in rspamd_update, disable rules execution
  3186. * [Fix] Fix processing of closed tags
  3187. * [Fix] Fix processing of idempotent rules when autolearn fails
  3188. * [Fix] Fix processing of multipart parts with no headers
  3189. * [Fix] Fix processing of skip-hashes in fuzzy storage
  3190. * [Fix] Fix PTR processing in SPF
  3191. * [Fix] Fix pushing country to clickhouse asn table
  3192. * [Fix] Fix random forests module
  3193. * [Fix] Fix real IP parsing for some strange Exim received
  3194. * [Fix] Fix Redis timeout setup
  3195. * [Fix] Fix reload crash when hyperscan is enabled
  3196. * [Fix] Fix reusing of redis connection after exec
  3197. * [Fix] Fix sanity checks on macro value
  3198. * [Fix] Fix setting of path and cpath for Lua
  3199. * [Fix] Fix setting of signals when spawning a thread
  3200. * [Fix] Fix text splitting: stack overflow (too many captures)
  3201. * [Fix] Fix ticks processing
  3202. * [Fix] Fix upstream addrs updating
  3203. * [Fix] Fix urls/emails distinguishing found in queries
  3204. * [Fix] Fix user settings check
  3205. * [Fix] Fix variable increment
  3206. * [Fix] Fix various issues in stat_convert
  3207. * [Fix] F-PROT Antivirus infection string for all known occurences
  3208. * [Fix] F-PROT Antivirus: only check return code to determine infection
  3209. * [Fix] Further fixes around floating point expressions
  3210. * [Fix] Further fixes to ANN module
  3211. * [Fix] Further fixes to rescore tool
  3212. * [Fix] Further fixes to support ES 6
  3213. * [Fix] Further tokenization fixes
  3214. * [Fix] Greylisting set phase is not idempotent
  3215. * [Fix] Handle proxy copy errors
  3216. * [Fix] Header checks: Fix get_raw_header method
  3217. * [Fix] Header checks: REPLYTO_UNPARSEABLE rule
  3218. * [Fix] Kill spawned processes on termination
  3219. * [Fix] Load skip map from all processes as shared cache is unavailable
  3220. * [Fix] Lowercase HTTP headers to make them searchable from Lua
  3221. * [Fix] Lowercase words
  3222. * [Fix] Lua_http: freeing
  3223. * [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found"
  3224. * [Fix] Map absence is not an error
  3225. * [Fix] Metadata exporter: check IP sanity
  3226. * [Fix] Milter headers: custom headers: removing headers
  3227. * [Fix] Milter headers: skip_local / skip_authenticated settings
  3228. * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first
  3229. * [Fix] mime_types: fix next-to-last extension length check
  3230. * [Fix] More hacks to deal with old configs
  3231. * [Fix] Move composites second pass to the dedicated stage
  3232. * [Fix] Multimap: received: filtering of artificial header
  3233. * [Fix] Multiple fixes in torch based ANN plugins
  3234. * [Fix] Once more (#1879) fix bad extension check
  3235. * [Fix] Optimize rspamd_fstring_t reallocations
  3236. * [Fix] options.local_networks setting
  3237. * [Fix] Parse HREF urls without explicit prefix
  3238. * [Fix] Plan new event on HTTP errors
  3239. * [Fix] Plug another possible memory leak
  3240. * [Fix] Plug memory leak
  3241. * [Fix] Plug memory leak in lua_tcp
  3242. * [Fix] Plug memory leak when setting email addresses from Lua
  3243. * [Fix] Propagate learn/stat errors more precisely
  3244. * [Fix] Ratelimit: fix whitelisted_rcpts matching
  3245. * [Fix] Ratelimit: lowercase email addresses
  3246. * [Fix] RBL: received: deal with missing data (#1965)
  3247. * [Fix] Rebalance and slightly rework MX check plugin
  3248. * [Fix] Redis key expansion: EVAL: deal with strings
  3249. * [Fix] Redis script loading in DMARC; URL tags; URL reputation
  3250. * [Fix] Reject invalid bh for DKIM signatures earlier
  3251. * [Fix] Relax pem signature detection
  3252. * [Fix] Relax unicode properties requirements for chartable module
  3253. * [Fix] Remove extra noise from dkim and arc signing
  3254. * [Fix] Remove hop-by-hop headers in proxy
  3255. * [Fix] Remove incorrect method `task:set_metric_subject`
  3256. * [Fix] Replace space like characters in headers with plain space
  3257. * [Fix] Restore old style ratelimits support
  3258. * [Fix] Rework elasticsearch plugin
  3259. * [Fix] Rewriting subjects via force actions module
  3260. * [Fix] RPM postinstall
  3261. * [Fix] Sanitize IP in history redis
  3262. * [Fix] Select the correct signature when doing simple canon
  3263. * [Fix] Set CLOEXEC flag on files opened
  3264. * [Fix] Setting check_local / check_authed in plugins (#1954)
  3265. * [Fix] Settings: avoid checking invalid IP (#1981)
  3266. * [Fix] Settings: header: deal with multiple settings (#1988)
  3267. * [Fix] Skip checks if both extensions are not bad
  3268. * [Fix] Skip nostat tokens when get number of tokens
  3269. * [Fix] Some more fixes towards emails detection
  3270. * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match
  3271. * [Fix] Stop using of g_slice...
  3272. * [Fix] Switch rspamadm logging to message level
  3273. * [Fix] Symbol 'FANNR_SPAM' has its score defined..
  3274. * [Fix] Table parameter for rspamd_config:add_doc()
  3275. * [Fix] Treat 'rewrite subject' as spam action
  3276. * [Fix] Try harder in passing IPv6 addresses
  3277. * [Fix] Try harder to find rfc822 notifications
  3278. * [Fix] Try harder to find urls
  3279. * [Fix] Use decoded values when parsing mime addresses
  3280. * [Fix] Use full URL when making an HTTP request
  3281. * [Fix] Use greylisting threshold in greylisting module
  3282. * [Fix] Use n_words attribute from ngramms
  3283. * [Fix] Use raw urls when sending requests to redirector
  3284. * [Fix] Use the right boolean operator on error check
  3285. * [Fix] Use weight from map for fuzzy scoring
  3286. * [Fix] Various fixes to elastic plugin
  3287. * [Fix] Various fixes to fann_redis instantiation
  3288. * [Fix] Various improvements in language detection
  3289. * [Fix] Virus infection string for F-PROT Antivirus
  3290. * [Fix] Virus infetction string for F-PROT Antivirus
  3291. * [Fix] WebUI: use relative path for savemap (#1943)
  3292. * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule
  3293. * [Fix] Write configuration changes as UCL config
  3294. * [Project] Add detection logic for words
  3295. * [Project] Add fast debug logging infrastructure
  3296. * [Project] Add more flags to languages
  3297. * [Project] Add n-gramms data files
  3298. * [Project] Add ngramms frequencies detector
  3299. * [Project] Add random words selection logic
  3300. * [Project] Add unigramms to language detection as well
  3301. * [Project] Convert all C modules to fast debug infrastructure
  3302. * [Project] Detect some languages based on unicode script
  3303. * [Project] Enable fast debug lookup for some modules
  3304. * [Project] Enable language detector init in scanner workers
  3305. * [Project] Further improvements to language detector
  3306. * [Project] Implement logic of ngramms application
  3307. * [Project] Improve weighting in lang_detection
  3308. * [Project] Initialize language detector
  3309. * [Project] Preliminary version of ngramms based language detector
  3310. * [Project] Preliminary version of the new stat_convert
  3311. * [Project] Remove old language detector
  3312. * [Project] Rework language detection ngramms structure
  3313. * [Project] Start language detection project
  3314. * [Project] Start rework of language detection to improve quality
  3315. * [Project] Use fast debug logging check
  3316. * [Rework] Add frame for new reputation based IP score module
  3317. * [Rework] Continue stat_convert rework task
  3318. * [Rework] Implement new version of fuzzy replies
  3319. * [Rework] Improve readability of xmlrpc API
  3320. * [Rework] Kill metrics!11
  3321. * [Rework] Ratelimit module
  3322. * [Rework] Rename fann_redis to neural plugin
  3323. * [Rework] Reorganize mime_types module
  3324. * [Rework] Rework rescore utility
  3325. * [Rework] Rewrite model and learning logic for rescore
  3326. * [Rework] Run post-loads when all initialization is completed
  3327. * [Rework] Simplify lua path initialization
  3328. * [Rework] Start major stat_convert rework
  3329. * [Rework] Start mempool fragmentation reduce project
  3330. * [Rework] Start moving of fann redis to torch
  3331. * [Rework] Stop embedding rspamadm scripts into C
  3332. * [Rework] Use floating point arithmetics in Rspamd expressions
  3333. * [Rework] Use frequencies distribution in language detector
  3334. * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages
  3335. * [WebUI] Compact graph selectors
  3336. * [WebUI] Escape strings inside HTML in history
  3337. * [WebUI] Fix message count in throughput summary (#1724)
  3338. * [WebUI] Fix NaNs display on Throughput graph
  3339. * [WebUI] Migrate widgets to D3 v4
  3340. * [WebUI] Restore passwordless login support (#2003)
  3341. * [WebUI] Show symbol descriptions as tooltips in history
  3342. * [WebUI] Stop using commas in pie chart tooltips
  3343. * [WebUI] Update D3 and jQuery
  3344. * [WebUI] Update D3Evolution 1.0.0 -> 1.1.0
  3345. 1.6.6: 16 Feb 2018
  3346. * [CritFix] Add sanity guards for badly broken HTML
  3347. * [CritFix] Another errors path handling fix
  3348. * [CritFix] Fix ARC chain verification
  3349. * [CritFix] Fix crash in milter errors handler
  3350. * [Feature] Allow to insert headers into specific position
  3351. * [Feature] Allow to receive signing keys from mempool vars
  3352. * [Feature] Authentication-Results: support hiding usernames
  3353. * [Fix] Another try to deal with #1998
  3354. * [Fix] Another try to fix #1998
  3355. * [Fix] Better handling of the legacy protocol
  3356. * [Fix] Check decoded headers sanity (e.g. by excluding \0)
  3357. * [Fix] Deal with nan and inf encoding in json/ucl
  3358. * [Fix] Deal with URLs wrapped in [] in text parts
  3359. * [Fix] DKIM signing: allow for auth_only to be false
  3360. * [Fix] Do not crash on empty subtype
  3361. * [Fix] Do not fail rbl plugin when there are no received or emails
  3362. * [Fix] Do not skip the last character
  3363. * [Fix] Do not try to dereference last character
  3364. * [Fix] Do not try to sign unknown domains
  3365. * [Fix] Exim Received header protocol parsing
  3366. * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false
  3367. * [Fix] Fix bad archive characters stripping
  3368. * [Fix] Fix comparision
  3369. * [Fix] Fix connecting to a unix socket in rspamadm statconvert
  3370. * [Fix] Fix empty headers simple canonicalization
  3371. * [Fix] Fix extra hits in PCRE mode for regular expressions
  3372. * [Fix] Fix parsing of the per-user script
  3373. * [Fix] Fix processing of skip-hashes in fuzzy storage
  3374. * [Fix] Fix Redis timeout setup
  3375. * [Fix] Fix sanity checks on macro value
  3376. * [Fix] Fix text splitting: stack overflow (too many captures)
  3377. * [Fix] Fix urls/emails distinguishing found in queries
  3378. * [Fix] F-PROT Antivirus: only check return code to determine infection
  3379. * [Fix] Metadata exporter: check IP sanity
  3380. * [Fix] Multimap: received: filtering of artificial header
  3381. * [Fix] Plan new event on HTTP errors
  3382. * [Fix] Plug another possible memory leak
  3383. * [Fix] Remove hop-by-hop headers in proxy
  3384. * [Fix] Sanitize IP in history redis
  3385. * [Fix] Setting check_local / check_authed in plugins (#1954)
  3386. * [Fix] Settings: avoid checking invalid IP (#1981)
  3387. * [Fix] Try harder in passing IPv6 addresses
  3388. * [Fix] WebUI: use relative path for savemap (#1943)
  3389. * [WebUI] Fix message count in throughput summary (#1724)
  3390. * [WebUI] Fix NaNs display on Throughput graph
  3391. * [WebUI] Restore passwordless login support (#2003)
  3392. 1.6.5: 22 Oct 2017
  3393. * [CritFix] Another portion of tokenization fixes
  3394. * [CritFix] Fix memory leak in spf caching logic
  3395. * [CritFix] Fix milter commands pipelining
  3396. * [CritFix] Fix newlines detection
  3397. * [Feature] Filter nan and inf when adding scores
  3398. * [Feature] Implement headers flags in mime parser
  3399. * [Feature] Support Expires header when using HTTP maps
  3400. * [Fix] Actively load skip hashes map in fuzzy storage
  3401. * [Fix] Add workaround for IPv6 in sendmail
  3402. * [Fix] Authentication Results: Fix SPF smtp.mail_from
  3403. * [Fix] Check for magic when checking for an archive
  3404. * [Fix] Deal with another case when processing exceptions
  3405. * [Fix] Deal with URLs with no slashes after protocol
  3406. * [Fix] Do not allow garbadge when checking url domain
  3407. * [Fix] Do not ignore short words
  3408. * [Fix] Do not strip last character in the last word
  3409. * [Fix] Do not treat script content as text
  3410. * [Fix] Erase unknown HTML entities
  3411. * [Fix] Fix another tokenization issue
  3412. * [Fix] Fix DKIM forgeries via multiple headers
  3413. * [Fix] Fix emails detection
  3414. * [Fix] Fix empty threshold check in greylisting module
  3415. * [Fix] Fix enormous scores for R_WHITE_ON_WHITE
  3416. * [Fix] Fix loading of per-user redis backend for statistics
  3417. * [Fix] Fix multiple headers in DKIM headers list
  3418. * [Fix] Fix obscured url in format user@@example.com
  3419. * [Fix] Further tokenization fixes
  3420. * [Fix] Load skip map from all processes as shared cache is unavailable
  3421. * [Fix] Lowercase words
  3422. * [Fix] Milter headers: skip_local / skip_authenticated settings
  3423. * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first
  3424. * [Fix] Ratelimit: fix whitelisted_rcpts matching
  3425. * [Fix] Some more fixes towards emails detection
  3426. * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match
  3427. * [Fix] Use greylisting threshold in greylisting module
  3428. 1.6.4: 10 Sep 2017
  3429. * [Feature] Add method to get all content-type attributes in Lua
  3430. * [Feature] Add some sanity checks for actions and controller
  3431. * [Feature] Allow randomly select User-Agent from a list
  3432. * [Feature] Deal with obscured URLs with @ symbols
  3433. * [Feature] Milter headers: support adding/removing arbitrary headers from config
  3434. * [Fix] Add another workaround to display history properly
  3435. * [Fix] Add missing rspamadm control options to help
  3436. * [Fix] Auth-Results: Multiple DKIM signatures
  3437. * [Fix] Crash in URL processing
  3438. * [Fix] Default monitoring domain for surbl plugin
  3439. * [Fix] Detection of maillist optimized and fixed
  3440. * [Fix] Do not cache SPF records with PTR elements
  3441. * [Fix] Fix blacklists and DMARC in whitelist
  3442. * [Fix] Fix exceptions list in surbl
  3443. * [Fix] Fix processing of closed tags
  3444. * [Fix] Fix PTR processing in SPF
  3445. * [Fix] Lowercase HTTP headers to make them searchable from Lua
  3446. * [Fix] options.local_networks setting
  3447. * [Fix] Ratelimit: lowercase email addresses
  3448. * [Fix] Rebalance and slightly rework MX check plugin
  3449. * [Fix] Redis script loading in DMARC; URL tags; URL reputation
  3450. * [Fix] Reject invalid bh for DKIM signatures earlier
  3451. * [Fix] Remove incorrect method `task:set_metric_subject`
  3452. * [Fix] Rewriting subjects via force actions module
  3453. * [Fix] RPM postinstall
  3454. * [Fix] Treat 'rewrite subject' as spam action
  3455. * [Fix] Try harder to find urls
  3456. * [Fix] Use full URL when making an HTTP request
  3457. * [Fix] Use raw urls when sending requests to redirector
  3458. * [Fix] Use weight from map for fuzzy scoring
  3459. * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages
  3460. 1.6.3: 26 Jul 2017
  3461. * [CritFix] Fix semicolons parsing in the content type
  3462. * [Feature] Add EBL to the default config
  3463. * [Feature] Allow to configure monitored
  3464. * [Feature] Allow to skip specific hashes in fuzzy storage
  3465. * [Feature] Multimap: checking of symbol options
  3466. * [Feature] Redis settings: support checking multiple keys
  3467. * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer
  3468. * [Fix] Avoid changing content-transfer-encoding header's value
  3469. * [Fix] Don't use whitelist/greylist maps as regexp, but as map
  3470. * [Fix] Fix get_content method
  3471. * [Fix] Header checks: Fix get_raw_header method
  3472. * [Fix] Header checks: REPLYTO_UNPARSEABLE rule
  3473. * [Fix] Lua_http: freeing
  3474. * [Fix] Milter headers: custom headers: removing headers
  3475. * [Fix] Parse HREF urls without explicit prefix
  3476. * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule
  3477. * [WebUI] Escape strings inside HTML in history
  3478. 1.6.2: 08 Jul 2017
  3479. * [Conf] Remove Rambler email bl for now
  3480. * [Conf] Switch RAMBLER_URIBL to a locally managed source
  3481. * [CritFix] Switch from ragel to C for Content-Type parsing
  3482. * [Feature] Add `-e` option for lua_repl
  3483. * [Feature] Add per-domain emails normalisation rules
  3484. * [Feature] Add sessions cache to debug dangling sessions
  3485. * [Feature] Add short_text_direct_hash for fuzzy check module
  3486. * [Feature] Add text_part:get_stats function
  3487. * [Feature] Allow to add custom processing script for surbl
  3488. * [Feature] Allow to check reply-to email
  3489. * [Feature] Allow to customize spam header, remove existing spam headers
  3490. * [Feature] Allow to disable specific workers in the config
  3491. * [Feature] Allow to discard messages instead of rejection
  3492. * [Feature] Allow to specify custom delimiter in emails plugin
  3493. * [Feature] Allow to specify custom User-Agent for rspamc
  3494. * [Feature] Allow to store symbols data in Clickhouse
  3495. * [Feature] Allow to use HTTPS when connecting to Clickhouse
  3496. * [Feature] Enable sessions cache tracking for milter connections
  3497. * [Feature] Implement per-line mode in lua_repl (like `perl -p`)
  3498. * [Feature] Implement rdns-curve plugin based on rspamd cryptobox
  3499. * [Feature] Improve maps cached data lifetime
  3500. * [Feature] Improve maps checking frequency
  3501. * [Feature] Improve monitored timeouts logic
  3502. * [Feature] milter_headers: add `extended_headers_rcpt` option
  3503. * [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers
  3504. * [Feature] Milter headers: remove-header routine
  3505. * [Feature] Multimap: received filters for extracting TLDs from hostnames
  3506. * [Feature] Normalize email aliases in emails module
  3507. * [Feature] Re-add rambler email bl (as hashed list)
  3508. * [Feature] Reload file maps more frequently
  3509. * [Feature] Rework newlines strip parser one more time
  3510. * [Feature] Skip updates for messages scanned via controller
  3511. * [Feature] Split long DKIM public keys
  3512. * [Feature] Store more data when stripping newlines
  3513. * [Feature] Support SPF macros transformations
  3514. * [Feature] Support suppressing DMARC reports for some domains
  3515. * [Fix] Add missing `break` statement
  3516. * [Fix] Allow modifiers in SPF macros
  3517. * [Fix] DKIM sign tools: edge-cases around use_esld
  3518. * [Fix] Do not cache SPF records with macros
  3519. * [Fix] Do not overwrite score when setting pre-action
  3520. * [Fix] Fix comparison logic
  3521. * [Fix] Fix DKIM base64 folding for milter flagged messages
  3522. * [Fix] Fix emails module configuration
  3523. * [Fix] Fix folding for arc headers when milter interface is used
  3524. * [Fix] Fix gmail dots removal
  3525. * [Fix] Fix rspamc detection in greylist module
  3526. * [Fix] Fix some more issues with HTTP maps
  3527. * [Fix] Milter sessions can live forever
  3528. * [Fix] Normalize fuzzy probability better
  3529. * [Fix] Plug memory leak
  3530. * [Fix] RBL: Fixed hashed email address lookups
  3531. * [Fix] Try to deal with brain-damaged milter behaviour
  3532. * [Fix] Use `\n` to fold headers for milter
  3533. * [Rework] Allow to use custom callback for monitored checks
  3534. * [Rework] Further steps towards one process monitoring
  3535. * [Rework] Send health checks from a single worker
  3536. * [WebUI] Round-up throughput summary values
  3537. 1.6.1: 14 Jun 2017
  3538. * [Fix] Allow to init resolver without rspamd_config
  3539. * [Fix] Do not crash when resolver failed to initialize
  3540. * [Fix] Fix abstract context layout
  3541. * [Fix] Fix CGP helper reply parsing
  3542. * [Fix] Fix crashes when socket write errors occur
  3543. * [Fix] Fix parsing IPv6 nameservers in resolv.conf
  3544. * [Fix] Milter: Don't defer on "greylist" action
  3545. 1.6.0: 12 Jun 2017
  3546. * [Conf] Add rspamd_proxy to the default configuration set
  3547. * [Conf] Add sample arc module config
  3548. * [Conf] Do away with systemd specifics completely
  3549. * [Conf] Increase min_bytes to avoid FP
  3550. * [Conf] Remove ratelimits from default configuration
  3551. * [CritFix] Fix accepting on IPv6 sockets
  3552. * [CritFix] Fix corruption when multiple fuzzy are defined
  3553. * [CritFix] Fix learn condition in fuzzy check
  3554. * [CritFix] Fix memory leak in fuzzy check
  3555. * [CritFix] Fix memory leak in maps scheduling
  3556. * [CritFix] Paese the last character in DKIM signature correctly
  3557. * [CritFix] Zero fill sockaddr_un
  3558. * [Feature] Add ability to add doc strings by example
  3559. * [Feature] Add API to verify DKIM (and ARC) signatures
  3560. * [Feature] Add compression/decompression to proxy
  3561. * [Feature] Add count to url structure
  3562. * [Feature] Add initial support of the new protocol reply
  3563. * [Feature] Add Lua plugin spamtrap
  3564. * [Feature] Add `monitored_address` for rbls
  3565. * [Feature] Add new schema for bayes tokens
  3566. * [Feature] Add preliminary ARC support to dkim code
  3567. * [Feature] Add preliminary support of ARC signing
  3568. * [Feature] Add rules to detect bad 8bit characters in From and To
  3569. * [Feature] Add scanning support for milter protocol
  3570. * [Feature] Add support for bidirectional symbols in rspamd_stats
  3571. * [Feature] Add support for static maps
  3572. * [Feature] Add support of maps with multiple regexps matches
  3573. * [Feature] Add `text_multiplier` param
  3574. * [Feature] Add the preliminary ARC plugin
  3575. * [Feature] Add top redirector targets rank
  3576. * [Feature] Allow async events to be registered from LUA rules
  3577. * [Feature] Allow storing bayes tokens in Redis
  3578. * [Feature] Allow to exclude specific domains from mx check
  3579. * [Feature] Allow to have a stack of watcher finalisers
  3580. * [Feature] Allow to pass hostname to `-i` flag in Rspamc
  3581. * [Feature] Allow to set custom user agent in url redirector
  3582. * [Feature] Allow to use custom callback when parsing resolv.conf
  3583. * [Feature] Allow to use domain from authenticated user
  3584. * [Feature] Bayes expiry plugin
  3585. * [Feature] Check dkim sign keys for modifications
  3586. * [Feature] DKIM signing: sign_networks/local address specific use_domain settings
  3587. * [Feature] DMARC: Support excluding domains from sampling
  3588. * [Feature] Expire processing items for URL redirector aggressively
  3589. * [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option
  3590. * [Feature] Implement caching for dkim body hashes
  3591. * [Feature] Implement milter protocol scan reply
  3592. * [Feature] Improve omograph phishing detection
  3593. * [Feature] Initial support of self-scan in Rspamd proxy
  3594. * [Feature] Keep track of headers in milter interface
  3595. * [Feature] Milter headers: better controls for local/authenticated
  3596. * [Feature] Multimap: email:domain:tld filter
  3597. * [Feature] Preliminary DMARC reporting implementation
  3598. * [Feature] Reuse stemmers in the cache
  3599. * [Feature] Rework confighelp to load Lua plugins
  3600. * [Feature] Rework hfilter to use hyperscan if possible
  3601. * [Feature] Rework lua RSA API
  3602. * [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers
  3603. * [Feature] Start integration of milter support in proxy
  3604. * [Feature] Store average words length and short words count
  3605. * [Feature] Store hash of headers order and names
  3606. * [Feature] Support MTA name header
  3607. * [Feature] Support multiple types of dkim signing in Lua
  3608. * [Feature] Support numeric arguments for Redis requests
  3609. * [Feature] Use headers hash in bayes metatokens
  3610. * [Feature] Use normal resolv.conf rules of rotation in Rspamd
  3611. * [Feature] Use version 2 proto for checking messages
  3612. * [Fix] Allow to follow symlinks when safe
  3613. * [Fix] Append MX name for authentication results as required
  3614. * [Fix] Change default text multiplier from 0.5 to 2.0
  3615. * [Fix] Check min_bytes for images as well
  3616. * [Fix] Deal with 7bit charsets properly
  3617. * [Fix] Deal with 8bit characters in email addresses
  3618. * [Fix] Deal with unpaired <a> tags
  3619. * [Fix] Detect confighelp in plugins initialisation
  3620. * [Fix] Disable certain checks for utf spoof detection
  3621. * [Fix] DKIM Signing: avoid nil index when From header is missing
  3622. * [Fix] Do not add exact hashes from different parts
  3623. * [Fix] Do not check DMARC if SPF or DKIM were not checked
  3624. * [Fix] Do not check URLs that are resolved to be redirected
  3625. * [Fix] Do not set bayes probability if we don't use it
  3626. * [Fix] Do not stop on illegal unicode points - replace them
  3627. * [Fix] Fix another race condition in arc checks
  3628. * [Fix] Fix arc count logic
  3629. * [Fix] Fix ARC signing
  3630. * [Fix] Fix brain-damaged spamc protocol for now
  3631. * [Fix] Fix calling for peak functions
  3632. * [Fix] Fix couple of issues in FORWARDED rule
  3633. * [Fix] Fix CTE propagation from parent containers to children parts
  3634. * [Fix] Fix errors processing in the controller
  3635. * [Fix] Fix format string in milter
  3636. * [Fix] Fix issues in SPF macros parsing
  3637. * [Fix] Fix logging format string
  3638. * [Fix] Fix logic of cached passwords check
  3639. * [Fix] Fix lowercasing of stemmed words
  3640. * [Fix] Fix LRU elements removal
  3641. * [Fix] Fix memory leak when accepting from unix sockets
  3642. * [Fix] Fix milter connections persistence
  3643. * [Fix] Fix objects merging in UCL
  3644. * [Fix] Fix order of operations to avoid race condition
  3645. * [Fix] Fix parsing of long regexp types
  3646. * [Fix] Fix passing data to log helper when many symbols defined
  3647. * [Fix] Fix pools management for milter session
  3648. * [Fix] Fix processing of the watchers
  3649. * [Fix] Fix queue id macro in milter
  3650. * [Fix] Fix R_BAD_CTE_7BIT rule
  3651. * [Fix] Fix Redis timeout set
  3652. * [Fix] Fix REPLYTO_UNPARSEABLE rule
  3653. * [Fix] Fix setting of email address
  3654. * [Fix] Fix some more issues about duplicated fuzzy requests
  3655. * [Fix] Fix spamc support in rspamd proxy
  3656. * [Fix] Fix syntax error in spamtrap plugin
  3657. * [Fix] Fix url counts for href urls
  3658. * [Fix] Fix url handling in the protocol
  3659. * [Fix] Multimap: Received IP filters with Redis
  3660. * [Fix] Oops, fix d9d0fa5e86db2f4470d34395a233b450478b2f60
  3661. * [Fix] Parse rgb[a](x,x,x[,x]) css colors
  3662. * [Fix] Phishing: strict_domains
  3663. * [Fix] Reduce maps aggressiveness
  3664. * [Fix] Reresolve upstreams even if there is a single server there
  3665. * [Fix] Rspamadm grep: Disable Lua patterns in string search by default
  3666. * [Fix] Skip text parts when checking binary parts in fuzzy check
  3667. * [Fix] Support v2 checks in controller
  3668. * [Fix] Treat empty address as valid
  3669. * [Fix] Try harder to detect CTE
  3670. * [Fix] Try to deal with v4 mapped to v6 addresses on accept
  3671. * [Fix] Use dkim signing callback properly
  3672. * [Fix] Use non-volatile memory for storing data
  3673. * [Fix] Use static maps instead of ugly hack for radix_from_config
  3674. * [Fix] Use the same pool for related sessions
  3675. * [Rework] Continue modularisation for lua library
  3676. * [Rework] Initial milter protocol support
  3677. * [Rework] Make log pipes worker agnostic, add scanners API
  3678. * [Rework] Move authentication results generation to a separate routine
  3679. * [Rework] Move common DKIM functions to a separate lua module
  3680. * [Rework] Move global functions to a separate directory
  3681. * [Rework] Prepare dkim module for ARC checks
  3682. * [Rework] Propagate ucl variables from the command line
  3683. * [Rework] Remove multiple metrics support from Rspamd
  3684. * [Rework] Stop using name 'rmilter' for the modern protocol
  3685. * [Rework] Use LFU algorithm in LRU cache
  3686. * [Rules] Fix received TLS rules
  3687. * [Rules] Improve URL_COUNT_ODD rule
  3688. * [WebUI] Fix add header filter in history
  3689. * [WebUI] Use modern protocol for checking messages
  3690. 1.5.9:
  3691. * [Conf] Increase min_bytes to avoid FP
  3692. * [Conf] Remove ratelimits from default configuration
  3693. * [CritFix] Fix accepting on IPv6 sockets
  3694. * [CritFix] Zero fill sockaddr_un
  3695. * [Feature] Add `text_multiplier` param
  3696. * [Fix] Check min_bytes for images as well
  3697. * [Fix] Do not add exact hashes from different parts
  3698. * [Fix] Fix memory leak when accepting from unix sockets
  3699. * [Fix] Fix some more issues about duplicated fuzzy requests
  3700. * [Fix] Phishing: strict_domains
  3701. * [Fix] Skip text parts when checking binary parts in fuzzy check
  3702. * [Minor] Add the same duplicates protection for all fuzzy hashes types
  3703. * [Minor] Fix braces
  3704. * [Minor] Fix test
  3705. * [Minor] SPOOF_DISPLAY_NAME: Use all SMTP/MIME recipients
  3706. * [Minor] Validate assumed spoofed display name domains to contain a dot
  3707. 1.5.8:
  3708. * [CritFix] Fix memory leak in fuzzy check
  3709. * [CritFix] Fix memory leak in maps scheduling
  3710. * [Feature] Multimap: email:domain:tld filter
  3711. * [Fix] DKIM Signing: avoid nil index when From header is missing
  3712. * [Fix] Do not set bayes probability if we don't use it
  3713. * [Fix] Do not stop on illegal unicode points - replace them
  3714. * [Fix] Fix brain-damaged spamc protocol for now
  3715. * [Fix] Fix Redis timeout set
  3716. * [Fix] Fix spamc support in rspamd proxy
  3717. * [Fix] Multimap: Received IP filters with Redis
  3718. * [Fix] Parse rgb[a](x,x,x[,x]) css colors
  3719. * [Fix] Reresolve upstreams even if there is a single server there
  3720. * [Fix] Treat empty address as valid
  3721. * [Fix] Try harder to detect CTE
  3722. * [Fix] Try to deal with v4 mapped to v6 addresses on accept
  3723. * [Minor] Add `wsf` and `hta` bad archive extensions
  3724. * [Minor] Fix configuration option
  3725. * [Minor] Fix result parsing for SAVAPI
  3726. * [Minor] Further logging improvements
  3727. * [Minor] Improve logging of errors
  3728. * [Minor] Prevent MID_CONTAINS_FROM from firing on empty address
  3729. * [Minor] Reduce digit->number transmission penalty
  3730. * [Minor] Relax CTYPE_MISSING_DISPOSITION rule
  3731. 1.5.7:
  3732. * [CritFix] Fix corruption when multiple fuzzy are defined
  3733. * [CritFix] Fix learn condition in fuzzy check
  3734. * [Feature] Add rules to detect bad 8bit characters in From and To
  3735. * [Feature] DKIM signing: sign_networks/local address specific use_domain settings
  3736. * [Feature] Support numeric arguments for Redis requests
  3737. * [Fix] Deal with 8bit characters in email addresses
  3738. * [Fix] Fix couple of issues in FORWARDED rule
  3739. * [Fix] Fix passing data to log helper when many symbols defined
  3740. * [Fix] Fix R_BAD_CTE_7BIT rule
  3741. * [Fix] Fix REPLYTO_UNPARSEABLE rule
  3742. * [Fix] Fix setting of email address
  3743. * [Fix] Rspamadm grep: Disable Lua patterns in string search by default
  3744. * [Minor] Add Lua 5.3 workaround
  3745. * [Minor] Add lua methods to detect if a part has 8bit characters
  3746. * [Minor] Allow session-less lua dns requests
  3747. * [Minor] Allow to append greylist end time to message reported
  3748. * [Minor] Avoid `nil` table
  3749. * [Minor] Disable dkim_signing if redis is specified but not configured
  3750. * [Minor] Fix build with pcre2
  3751. * [Minor] Fix rule
  3752. * [Minor] Fix warnings
  3753. * [Minor] Format floating point number
  3754. * [Minor] Push email flags to the lua API
  3755. * [Minor] Silence some warnings
  3756. * [Minor] Silence warning
  3757. * [Minor] Try all hostname regexps to find the most significant one
  3758. * [WebUI] Fix add header filter in history
  3759. 1.5.6:
  3760. * [Feature] Add unigramms support in bayes
  3761. * [Feature] Allow configurable sign headers for DKIM
  3762. * [Feature] Allow to add unigramm metatokens from Lua
  3763. * [Feature] DKIM Signing: envelope match exception for local IPs
  3764. * [Feature] UCL: register parser variables from Lua
  3765. * [Fix] Always try to adjust filename
  3766. * [Fix] Do extra copy to ensure that original content is never touched
  3767. * [Fix] Fix SPOOF_REPLYTO rule
  3768. * [Fix] Ignore Rmilter added Received
  3769. * [Fix] More fixes for hashed email dnsbls
  3770. * [Fix] Plug memory leak in chartable module
  3771. * [WebUI] Display multiple alerts at once
  3772. 1.5.5:
  3773. * [CritFix] Fix classifier learning with Redis backend
  3774. * [CritFix] Fix issue when parsing encoded rfc822/messages
  3775. * [Feature] Add escaped version of lua_ucl import
  3776. * [Feature] Add task:headers_foreach function
  3777. * [Feature] Allow to process filenames from content type
  3778. * [Feature] Allow to query hashed emails
  3779. * [Feature] Ignore bayes with mostly metatokens or with too few text
  3780. * [Feature] Probabilistically skip metatokens
  3781. * [Feature] Retrieve all virus names from SAVAPI
  3782. * [Feature] Rework classifiers lua metatokens
  3783. * [Feature] Store headers order
  3784. * [Feature] Store text tokens inside bayes tokens
  3785. * [Feature] Use cached shingles keys
  3786. * [Fix] Add missing score normalisation for HFILTER_URL_ONLY
  3787. * [Fix] Avoid lookup in absent hash
  3788. * [Fix] Check return values from Lua functions called from C
  3789. * [Fix] Do not count sending and loading time in rspamc
  3790. * [Fix] Escape json strings for controller rejplies from Lua
  3791. * [Fix] Fix archive scans for savapi
  3792. * [Fix] Fix domain_only emails RBL
  3793. * [Fix] Fix ip_score map configuration
  3794. * [Fix] Fix JSON output for history_redis
  3795. * [Fix] Fix one character length substrings search
  3796. * [Fix] Fix parsing of non-RFC compatible Exim received
  3797. * [Fix] Fix parsing of options for workers with the same type
  3798. * [Fix] Fix processing of small tokens vectors
  3799. * [Fix] Fix rfc2047 tokenization
  3800. * [Fix] Fix typo
  3801. * [Fix] More fixes for inplace decoding
  3802. * [Fix] Try to avoid modifications of the original data
  3803. * [Fix] URL redirector: Fix call to is_redirector
  3804. * [Rework] Set token data as uint64_t instead of chars array
  3805. * [WebUI] Check if neighbours' history backend versions match
  3806. * [WebUI] Disable phrase connectors replacement in history filtering
  3807. * [WebUI] Disable phrase connectors replacement in symbols filtering
  3808. * [WebUI] Do not hide messages with bad subject, just replace it with '???'
  3809. * [WebUI] Fix error message
  3810. * [WebUI] Fix history v2 display
  3811. * [WebUI] Fix legacy history
  3812. * [WebUI] history: break To address lists on commas
  3813. * [WebUI] Increase default timeout to 20 seconds
  3814. * [WebUI] Save some history table space
  3815. 1.5.4:
  3816. * [Conf] Add history_redis default configuration
  3817. * [Feature] Add spoofed rules
  3818. * [Feature] Add URL_IN_SUBJECT rule
  3819. * [Feature] Allow to get task's subject
  3820. * [Feature] Allow to specify maximum number of shots for symbols
  3821. * [Feature] Distinguish URLs found in Subject
  3822. * [Feature] Memoize LPEG grammars
  3823. * [Feature] Parse else parts in SA rules
  3824. * [Feature] Process subject for mixed characters
  3825. * [Feature] Resolve url chains in url_redirector module
  3826. * [Feature] Stat greylisted messages as greylisted not soft-rejected
  3827. * [Feature] Support checking for redirector in Lua SURBL
  3828. * [Feature] Support tag_exists SA function
  3829. * [Feature] Work with broken rfc2047 tokens
  3830. * [Fix] Check all watcher's dependencies
  3831. * [Fix] Do not compile hyperscan with no SSSE3 support
  3832. * [Fix] Do not crash if cannot decode qp encoded part
  3833. * [Fix] Fix dependencies of DKIM when multiple signatures are found
  3834. * [Fix] Fix lists in whitelist plugin
  3835. * [Fix] Fix one-shot symbols weight calculations
  3836. * [Fix] Fix options and shots match
  3837. * [Fix] Fix order of symbol options
  3838. * [Fix] Fix parsing of dot at the end of the address
  3839. * [Fix] Fix parsing of lua table arguments
  3840. * [Fix] Fix processing of subject words
  3841. * [Fix] Fix string split memoization
  3842. * [Fix] Fix templates grammar usage
  3843. * [Fix] Fix various issues related to Lua stack manipulation
  3844. * [Fix] Force actions: Use postfilter if we have honor_action / require_action
  3845. * [Fix] Further fixes to avoid PHISHING FP
  3846. * [Fix] Preserve order of options in symbols
  3847. * [Fix] Rspamadm grep: deal with unusually-formatted logs
  3848. * [Fix] Use hostname suffix when dealing with history
  3849. * [Rework] Remove outdated SA rules
  3850. * [WebUI] Add flexible columns
  3851. * [WebUI] Add footable
  3852. * [WebUI] Add sender, recipients and subject columns
  3853. * [WebUI] Allow message-id break
  3854. * [WebUI] Fix history clustering
  3855. * [WebUI] Fix history display
  3856. * [WebUI] Fix sorting
  3857. * [WebUI] Humanize sizes
  3858. * [WebUI] Initial move towards footable
  3859. * [WebUI] Remove datatables
  3860. * [WebUI] Replace `.values` method with `.map`
  3861. * [WebUI] Rework v2 symbols display
  3862. * [WebUI] Try to normalize frequencies
  3863. * [WebUI] Unbreak WebUI
  3864. * [WebUI] Use Footable to draw Throughput summary table
  3865. 1.5.3:
  3866. * [Conf] Add composite for hacked wordpress phishing
  3867. * [CritFix] Fix base64 decoding when there are unparseable characters
  3868. * [Feature] Additional symbol metadata in metadata exporter
  3869. * [Feature] Add method to get protocol reply from Lua
  3870. * [Feature] Add symbols when tagged rcpt/sender are normalised
  3871. * [Feature] Add task:get_symbols_all() function
  3872. * [Feature] Allow multiple formats of DKIM signing key
  3873. * [Feature] Allow to cache and use flexible protocol reply
  3874. * [Feature] Allow to set one_shot flag from register_symbol
  3875. * [Feature] Allow to skip certain types of hashes when learning fuzzy
  3876. * [Feature] Cache and insert scan time into the protocol
  3877. * [Feature] Detect newlines in rspamc --mime
  3878. * [Feature] DKIM signing: support use of maps
  3879. * [Feature] Greylist: Support excluding low-scoring messages from greylisting
  3880. * [Feature] Implement lua history in controller
  3881. * [Feature] Implement redis history querying
  3882. * [Feature] Preliminary implementation of redis history plugin
  3883. * [Feature] Support using request headers in settings
  3884. * [Fix] Change default template to deal with non-ASCII characters
  3885. * [Fix] Deal with lists of maps in whitelist module
  3886. * [Fix] DKIM signing: use domain-specific signing key
  3887. * [Fix] Do not reallocate completed zstd buffer
  3888. * [Fix] Do not use local_addrs in proxy
  3889. * [Fix] Fix crash when resolver is undefined
  3890. * [Fix] Fix double free when closing lua_tcp connections
  3891. * [Fix] Fix for lua 5.3
  3892. * [Fix] Fix freeing of arrays iterators
  3893. * [Fix] Fix issue with task:get_symbol and symbols with no metric
  3894. * [Fix] Fix log line duplication in `rspamadm grep`
  3895. * [Fix] Fix memory corruption on termination
  3896. * [Fix] Fix out-of-bound access in base64 decode
  3897. * [Fix] Fix ratelimit + greylisting
  3898. * [Fix] Fix subject rewriting
  3899. * [Fix] Fix task:set_recipients function
  3900. * [Fix] Fix URI_COUNT_ODD rule
  3901. * [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule
  3902. * [Fix] Greylist: Suppress greylist action for whitelisted hosts too
  3903. * [Fix] Metadata exporter: use rule-specific settings for emails
  3904. * [Fix] Properly set missing fields in exporter
  3905. * [Fix] Proxy: max_retries option
  3906. * [Fix] RCPT_COUNT fixes
  3907. * [Fix] Rework HAS_X_PRIO rule to match symbols conventions
  3908. * [Fix] Update issues in ac-trie
  3909. * [Fix] Use optimised base64 decoding in DKIM
  3910. * [WebUI] Add preliminary v2 history parser
  3911. * [WebUI] Allow different history parsers
  3912. * [WebUI] Display symbols
  3913. * [WebUI] Rework history v2 function
  3914. 1.5.2:
  3915. * [Conf] Add default config for spamassasssin plugin
  3916. * [Conf] Add default configuration for antivirus module
  3917. * [Conf] Add dkim signing docs
  3918. * [Conf] Add mx_check default config
  3919. * [Conf] Add replies config
  3920. * [Conf] Add trie default config
  3921. * [Feature] Add heuristic to find text parts in files
  3922. * [Feature] Add rule to detect broken content type
  3923. * [Feature] Allow to extract CTE in Lua API
  3924. * [Feature] Allow to set from address for a lua_task
  3925. * [Feature] Allow to set recipients of a task from Lua
  3926. * [Feature] Enchance text_part:get_content method
  3927. * [Feature] Remove + aliases from emails
  3928. * [Feature] Support rmilter block and dkim signature in CGP helper
  3929. * [Feature] Support running event loop from Lua
  3930. * [Fix] Antivirus: use scanner-specific redis prefix
  3931. * [Fix] Couple of fixes for DKIM signing module
  3932. * [Fix] Distinguish missing and broken mandatory headers
  3933. * [Fix] Do more heuristical detection for missing CTE
  3934. * [Fix] Do not resort cache on each check
  3935. * [Fix] Fix CGP escaping
  3936. * [Fix] Fix MISSING_MIME_VERSION rule for plain messages
  3937. * [Fix] Fix parsing of cte in expressions
  3938. * [Fix] Fix partial matches in rspamadm grep
  3939. * [Fix] Fix setting class on style field
  3940. * [WebUI] Auto-switch Throughput units to `msg/min` for very low rate
  3941. * [WebUI] Update D3Evolution to 0.0.2
  3942. 1.5.1:
  3943. * [CritFix] Fix processing of stop_patterns with `\0` character
  3944. * [CritFix] Fix setting of raw key for signing
  3945. * [Fix] Fix lua exports from plugins during reload
  3946. * [Fix] Fix prefilters action scores
  3947. * [Fix] Fix symbols processing order
  3948. * [Minor] Help cmake find gthread
  3949. * [Minor] Some cmake fixes
  3950. 1.5.0:
  3951. * [Conf] Add configurations for asn, clickhouse and dcc
  3952. * [Conf] Add default config for url redirector plugin
  3953. * [Conf] Add the default config for greylist module
  3954. * [Conf] Allow to edit all local maps from WebUI by default
  3955. * [CritFix] Deal with absent headers in DKIM
  3956. * [CritFix] Do not trust remote shingles count
  3957. * [CritFix] Fix bad memory leak in TLS certificates validation
  3958. * [CritFix] Fix critical memory issues with radix maps
  3959. * [CritFix] Fix descriptors leak on reload
  3960. * [CritFix] Fix headers selection in DKIM verification
  3961. * [CritFix] Fix parsing of boundaries that end with `--`
  3962. * [CritFix] Repair PTR_ARRAY_FOREACH macro
  3963. * [Feature] Add CORS support to the controller
  3964. * [Feature] Add FROM_NAME_EXCESS_SPACE rule
  3965. * [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule
  3966. * [Feature] Add `caseless_hash` method to `lua_util`
  3967. * [Feature] Add `rip` keyword to ratelimit module
  3968. * [Feature] Add a simple benchmark for content type parsing
  3969. * [Feature] Add boundaries parsing in content type
  3970. * [Feature] Add charset detection for text parts
  3971. * [Feature] Add content disposition parser
  3972. * [Feature] Add fallback if too many updates are failing
  3973. * [Feature] Add function to convert struct tm to time using timezone
  3974. * [Feature] Add function to normalize HTTP paths
  3975. * [Feature] Add fuzzy collection plugin
  3976. * [Feature] Add fuzzy logic for images
  3977. * [Feature] Add gmime parser to mime_tool
  3978. * [Feature] Add heuristic to detect broken messages
  3979. * [Feature] Add heuristic to find displayed URLs
  3980. * [Feature] Add heuristic to process broken email addresses
  3981. * [Feature] Add images normalization
  3982. * [Feature] Add mechanism for disabling composites (Fixes #1270)
  3983. * [Feature] Add method to create regexp from a glob pattern
  3984. * [Feature] Add mime encoding manipulation routines
  3985. * [Feature] Add mime tool to explore messages
  3986. * [Feature] Add more meta tokens from received headers
  3987. * [Feature] Add neighbours option to support Rspamd cluster in WebUI
  3988. * [Feature] Add new function to parse mime addresses
  3989. * [Feature] Add new methods for lua_tcp
  3990. * [Feature] Add own headers decoding routine
  3991. * [Feature] Add own routine to generate a message id
  3992. * [Feature] Add parser for SMTP date
  3993. * [Feature] Add per-task lua cache to reuse 'heavy' objects
  3994. * [Feature] Add plugins list path in WebUI
  3995. * [Feature] Add preliminary multipart support
  3996. * [Feature] Add preliminary version of DKIM signing module
  3997. * [Feature] Add profiling support in client output
  3998. * [Feature] Add rfc2047 grammar
  3999. * [Feature] Add rfc2047 variant for QP decoding
  4000. * [Feature] Add rmilter_headers module (Fixes #1227)
  4001. * [Feature] Add sse42 version of base64 decoding
  4002. * [Feature] Add ssse3 and avx2 base64 decoders
  4003. * [Feature] Add support of libgd
  4004. * [Feature] Add the preliminary version of redirects resolver in Lua
  4005. * [Feature] Add ucl_object_iterate_full function
  4006. * [Feature] Add url encoding function
  4007. * [Feature] Allow SOA requests in lua dns
  4008. * [Feature] Allow custom parse types in lua ucl
  4009. * [Feature] Allow plugins to register webui handlers
  4010. * [Feature] Allow to add options explicitly to symbols
  4011. * [Feature] Allow to call a callback when symbol frequency is on peak
  4012. * [Feature] Allow to call redirector script from SURBL
  4013. * [Feature] Allow to create variable length dkim keys
  4014. * [Feature] Allow to have module specific options for Redis in plugins
  4015. * [Feature] Allow to pass sign key directly from Lua
  4016. * [Feature] Allow to register configuration docs from Lua API
  4017. * [Feature] Allow to return options as a table
  4018. * [Feature] Allow to set peak callbacks from Lua
  4019. * [Feature] Allow to specify custom method for a message
  4020. * [Feature] Allow to store dkim keys in Redis
  4021. * [Feature] Allow to store messages in files
  4022. * [Feature] Apply DCT using AAN for fuzzy signature
  4023. * [Feature] Avira SAVAPI support
  4024. * [Feature] Cache and simplify DCT and jpeg decode
  4025. * [Feature] Cache libicu converters
  4026. * [Feature] Detect URLs with suspicious omographs
  4027. * [Feature] Do not increase score for duplicate options
  4028. * [Feature] Do not trust CTE, check base64 and qp strictly
  4029. * [Feature] Dynamic reputation in URL reputation plugin
  4030. * [Feature] Extend redis lock when learning spawned
  4031. * [Feature] Filter non-utf chars from all decoded headers
  4032. * [Feature] Fix phishing detection for IDNA urls
  4033. * [Feature] Ignore bad symbols on base64 decoding
  4034. * [Feature] Ignore too wide elements in SPF
  4035. * [Feature] Implement fuzzy collection mode
  4036. * [Feature] Implement helo maps in multimap
  4037. * [Feature] Implement human readable buckets configuration
  4038. * [Feature] Implement min-hash shingles for DCT data from images
  4039. * [Feature] Implement new algorithm for fuzzy hashes of images
  4040. * [Feature] Implement new unicode normalizer
  4041. * [Feature] Implement quoted printable decoding
  4042. * [Feature] Implement received headers flags
  4043. * [Feature] Implement rspamdgrep tool
  4044. * [Feature] Implement sane checksum for config file
  4045. * [Feature] Implement url tags concept
  4046. * [Feature] Improve detection of omographs using libicu
  4047. * [Feature] Improve url redirector module
  4048. * [Feature] Multimap: Received header processing
  4049. * [Feature] Multiple improvements in the maps
  4050. * [Feature] New URL filters in multimap
  4051. * [Feature] Plugin to force actions on selected symbols
  4052. * [Feature] RBL module: support hashing for emails and helo RBL
  4053. * [Feature] Reuse URL tags in SURBL module
  4054. * [Feature] Rework RRD ds count, add conversion path
  4055. * [Feature] Rework surbl module to avoid extra redirector calls
  4056. * [Feature] Send config id to the WebUI
  4057. * [Feature] Simplify HTTPCrypt client support
  4058. * [Feature] Skip processing for large images
  4059. * [Feature] Start collection only mode implementation for fuzzy storage
  4060. * [Feature] Start import of the optimized base64 decode
  4061. * [Feature] Store all received headers in lua
  4062. * [Feature] Store relational order of all headers in a message
  4063. * [Feature] Support DKIM signing in Lua plugins
  4064. * [Feature] Support HTTPCrypt client in lua_http
  4065. * [Feature] Support setting SMTP message in multimap
  4066. * [Feature] Support setting metric subject from Lua
  4067. * [Feature] Support setting subject in force actions module
  4068. * [Feature] Treat v6 mapped addresses as v4 addresses
  4069. * [Feature] URL reputation plugin
  4070. * [Feature] Use Redis instead of memcached in URLs redirector
  4071. * [Feature] Use Rspamd rfc2047 decoder instead of gmime one
  4072. * [Feature] Use a different normalization for fuzzy images
  4073. * [Feature] Use normalized images in fuzzy hashes
  4074. * [Feature] Use own code for parsing of date
  4075. * [Feature] Use shingles for images fuzzying
  4076. * [Feature] Use t1ha for hashes, allow inlining
  4077. * [Feature] Use t1ha instead of metrohash and xxhash32
  4078. * [Feature] Various new features in metadata exporter module
  4079. * [Feature] rmilter_headers: authentication-results (#78)
  4080. * [Fix] Add additional check to mark redis connection inactive
  4081. * [Fix] Add packed attribute for protocol structure
  4082. * [Fix] Adopt OMOGRAPH_URL rule
  4083. * [Fix] Allow static maps
  4084. * [Fix] Allow to disable classifiers checks using settings and conditions
  4085. * [Fix] Another try to fix 0 length maps
  4086. * [Fix] Another try to fix corruption during maps reload
  4087. * [Fix] Another try to fix descriptors leak
  4088. * [Fix] Another try to fix reload and logger
  4089. * [Fix] Antivirus module: register virtual symbols for patterns
  4090. * [Fix] Avoid extensive reallocs
  4091. * [Fix] Avoid mempool leak in SA plugin on reload
  4092. * [Fix] Avoid race condition on saving cache and reload
  4093. * [Fix] Avoid reusing g_error (Fixes #1262)
  4094. * [Fix] Break pool connection on fatal redis errors
  4095. * [Fix] Check for NaN properly
  4096. * [Fix] Couple of fixes for date parsing
  4097. * [Fix] Date header timezone adjustments (#1279)
  4098. * [Fix] Deal with EOF properly
  4099. * [Fix] Decode filename in content disposition
  4100. * [Fix] Disable fuzzy images by default
  4101. * [Fix] Disable zero-copy mode for text parts to avoid crashes
  4102. * [Fix] Do not destroy session when not all finish scripts are done
  4103. * [Fix] Do not greyscale images
  4104. * [Fix] Do not leave parent-less workers processes on fatal errors
  4105. * [Fix] Do not lowercase Content-Disposition to perform decoding
  4106. * [Fix] Do not penalize characters just after numeric prefix
  4107. * [Fix] Do not refork workers that are intended to die
  4108. * [Fix] Do not set pre-result and update records for no `Queue-ID` messages
  4109. * [Fix] Do not skip post-filters when pre-filters have set some results
  4110. * [Fix] Do not stop symbols planning if async events are pending
  4111. * [Fix] Do not try to set keys for unencrypted requests in proxy
  4112. * [Fix] Encode URLs according to rfc3986
  4113. * [Fix] Encode URLs before sending them to the protocol
  4114. * [Fix] Filter bad characters from message id
  4115. * [Fix] Fix CTE detection heuristic
  4116. * [Fix] Fix Content-Type in HTTP requests
  4117. * [Fix] Fix IDN eslds phishing checks
  4118. * [Fix] Fix adding maps from config in Lua
  4119. * [Fix] Fix another reload memory issue
  4120. * [Fix] Fix argument returned on redis backend errors
  4121. * [Fix] Fix assertion in graph handling
  4122. * [Fix] Fix body trie matching
  4123. * [Fix] Fix build
  4124. * [Fix] Fix byte array expansion during toutf8 conversion
  4125. * [Fix] Fix charset normalisation
  4126. * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added
  4127. * [Fix] Fix couple of cornercases with email addresses
  4128. * [Fix] Fix couple of issues
  4129. * [Fix] Fix dependencies tracking for callback symbols
  4130. * [Fix] Fix detection of jpeg size
  4131. * [Fix] Fix errors handling in fuzzy backend initialization
  4132. * [Fix] Fix fuzzy hashes count
  4133. * [Fix] Fix globbing and convert lists to arrays in fuzzy_check
  4134. * [Fix] Fix heuristical CTE detection for QP encoding
  4135. * [Fix] Fix ignoring of bad text parts
  4136. * [Fix] Fix indexes in array access, interleave loop
  4137. * [Fix] Fix int64 -> double conversion
  4138. * [Fix] Fix invalid memory access on reload
  4139. * [Fix] Fix issues with empty updates
  4140. * [Fix] Fix issues with quoted-printable encoding
  4141. * [Fix] Fix keys names
  4142. * [Fix] Fix lots of issues in mime parser code
  4143. * [Fix] Fix lua maps load
  4144. * [Fix] Fix macro name
  4145. * [Fix] Fix mas group score calculations
  4146. * [Fix] Fix matching of the same patterns from different tries
  4147. * [Fix] Fix memory corruprtion and leak
  4148. * [Fix] Fix memory leak in HTTP maps
  4149. * [Fix] Fix memory leak in expression destroying
  4150. * [Fix] Fix memory leak in parsing of mime names
  4151. * [Fix] Fix memory leak in safe ucl iterators
  4152. * [Fix] Fix memory leak on reload in plugins
  4153. * [Fix] Fix modules reconfigure on reload
  4154. * [Fix] Fix monitored setup fro URLBLs with IP addresses
  4155. * [Fix] Fix name of var
  4156. * [Fix] Fix new rrd updates
  4157. * [Fix] Fix out of bounds access
  4158. * [Fix] Fix parsing messages with no body
  4159. * [Fix] Fix parsing of '=' character in headers
  4160. * [Fix] Fix parsing of messages with no content type
  4161. * [Fix] Fix plugins callbacks in webui
  4162. * [Fix] Fix possible memory corruption in redis pool
  4163. * [Fix] Fix probability calculations for fuzzy redis backend
  4164. * [Fix] Fix processing errors in lua_tcp
  4165. * [Fix] Fix processing of emails with name only
  4166. * [Fix] Fix processing of non-multipart messages
  4167. * [Fix] Fix processing of parts with no valid content type
  4168. * [Fix] Fix race condition in SIGUSR2 handler
  4169. * [Fix] Fix redis options parsing when no redis servers are defined
  4170. * [Fix] Fix reload and hyperscan ready event
  4171. * [Fix] Fix reload memory issue
  4172. * [Fix] Fix rra_ptr conversion
  4173. * [Fix] Fix rrd file conversion
  4174. * [Fix] Fix setting of content-type attributes
  4175. * [Fix] Fix signing headers creation in DKIM
  4176. * [Fix] Fix stddev calculations
  4177. * [Fix] Fix surbl plugin to work with composite maps
  4178. * [Fix] Fix timezones parsing
  4179. * [Fix] Fix tokens usage
  4180. * [Fix] Fix urls and emails hashes
  4181. * [Fix] Fix usage of unsafe ucl iterators
  4182. * [Fix] Fix work with broken utf8 tokens
  4183. * [Fix] Fix writing of user to roll history
  4184. * [Fix] Forgotten worker
  4185. * [Fix] Further memory leaks fixes
  4186. * [Fix] Ignore lua metatokens in bayes for now
  4187. * [Fix] Improve OMOGRAPH_URL rule
  4188. * [Fix] Lua IP from string should be invalid if parsing failed
  4189. * [Fix] Miltiple fixes to new lua_tcp, add debugging
  4190. * [Fix] More fixes for iterators cleanup
  4191. * [Fix] More fixes to logger initialization
  4192. * [Fix] More heuristic fixes for phishing detection
  4193. * [Fix] More leaks eliminated
  4194. * [Fix] More leaks...
  4195. * [Fix] More random fixes for reload...
  4196. * [Fix] Multimap: Fixes for email filters
  4197. * [Fix] Multiple fixes for fann module
  4198. * [Fix] Multiple memory corruption fixes
  4199. * [Fix] Normalize path in HTTP router
  4200. * [Fix] Plug memory leak
  4201. * [Fix] Plug memory leak in adding radix trees
  4202. * [Fix] Plug memory leak in configuration parser
  4203. * [Fix] Plug memory leak in expressions parsing during reload
  4204. * [Fix] Plug memory leak in learning fuzzy storage
  4205. * [Fix] Plug memory leak in lua_tcp
  4206. * [Fix] Plug reload leaks
  4207. * [Fix] Plug termination memory leaks
  4208. * [Fix] Really increase lock lifetime
  4209. * [Fix] Replies module: fix symbol weight
  4210. * [Fix] Restore content type params related functions
  4211. * [Fix] Set task's subject from mime subject
  4212. * [Fix] Sigh, one more reload leak
  4213. * [Fix] Simplify images shingles
  4214. * [Fix] Some more memory issues are fixed
  4215. * [Fix] Stop hardcoding of lua in C
  4216. * [Fix] Stop processing of bad parts as text parts
  4217. * [Fix] Strictly filter bad characters when emittin json
  4218. * [Fix] Strings returned from lua are ephemeral
  4219. * [Fix] Support unix sockets for lua redis
  4220. * [Fix] Try to fix issues with reloading config
  4221. * [Fix] Try to fix race condition in redis_pool
  4222. * [Fix] Use checksum to avoid intersection between different ANNs
  4223. * [Fix] Use rspamd hashes in embedded ucl
  4224. * [Fix] Use sane default rewrite subject (*** SPAM *** %s)
  4225. * [Fix] Various collection mode fixes
  4226. * [Fix] Various fixes to mime parser
  4227. * [Fix] Various reload leak fixing
  4228. * [Fix] Whitelist certain extensions from archive checks
  4229. * [Rework] Add preliminary implementation of the mime parser
  4230. * [Rework] Adopt code for the new options
  4231. * [Rework] Change logger setup interface
  4232. * [Rework] Composite configuration (#1270)
  4233. * [Rework] Finally remove gmime dependency from Rspamd
  4234. * [Rework] Further fixes to symbols frequencies
  4235. * [Rework] Implement content type parser for mime
  4236. * [Rework] Kill all InternetAddressList usages
  4237. * [Rework] Multiple fixes for symbols cache statistics
  4238. * [Rework] Refactor struct names
  4239. * [Rework] Rework images fuzzy hashes algorithm
  4240. * [Rework] Rework lua_tcp to allow TCP dialog
  4241. * [Rework] Start massive rework to get rid of gmime
  4242. * [Rework] Start new approach for multiparts parsing
  4243. * [Rework] Start rework of mime addresses
  4244. * [Rework] Start rework of symbols cache updates
  4245. * [Rework] Start switching to libicu
  4246. * [Rework] Use a special structure for stats tokens
  4247. * [Rework] Use hash tables for symbols options
  4248. * [Rework] Use libicu instead of iconv for conversions
  4249. * [Rework] Use new scheme to parse mime parts
  4250. * [WebUI] Add Access-Control-Allow-Origin for cluster management
  4251. * [WebUI] Add Throughput graph autorefreshing (#820)
  4252. * [WebUI] Add Visibility.js library
  4253. * [WebUI] Add basic cluster support to Throughput tab
  4254. * [WebUI] Add graph legend entries for new DSes
  4255. * [WebUI] Add graph tab
  4256. * [WebUI] Add neighbours RRD data consolidation
  4257. * [WebUI] Add preliminary save symbols clustering
  4258. * [WebUI] Add server selector to navbar
  4259. * [WebUI] Add soft reject to auth stats
  4260. * [WebUI] Add summary to the Throughput tab
  4261. * [WebUI] Allow to save maps on the cluster
  4262. * [WebUI] Avoid extra graph redraw and alerts glitching
  4263. * [WebUI] Be more generous with AJAX timeout
  4264. * [WebUI] Disable error ring loading in `read only` mode
  4265. * [WebUI] Enclose table header cells with `tr`s
  4266. * [WebUI] Finish interface rework
  4267. * [WebUI] Fix RRD summary pie chart position
  4268. * [WebUI] Fix `All SERVERS` graph fot just one available server
  4269. * [WebUI] Fix case when no cluster is defined
  4270. * [WebUI] Fix compatibility with non-ES6 compliant browsers
  4271. * [WebUI] Fix config ID
  4272. * [WebUI] Fix configuration page partially
  4273. * [WebUI] Fix disabled state
  4274. * [WebUI] Fix graph dataset selector initialization
  4275. * [WebUI] Fix graph selectors state resetting
  4276. * [WebUI] Fix mouse events on throughput summary table area
  4277. * [WebUI] Fix multiple JS issues
  4278. * [WebUI] Fix pie chart displaying
  4279. * [WebUI] Fix read only
  4280. * [WebUI] Fix read only2
  4281. * [WebUI] Fix retarded datatables
  4282. * [WebUI] Fix soft reject in pie chart
  4283. * [WebUI] Fix stat widgets timers multiplication on `Refresh` click
  4284. * [WebUI] Fix symbols config
  4285. * [WebUI] Fix various errors with login form
  4286. * [WebUI] Further fixes
  4287. * [WebUI] Hide learning tab in read-only mode
  4288. * [WebUI] Initial clusters support
  4289. * [WebUI] Make legend entry colours more contrast
  4290. * [WebUI] Move configuration tab to a separate module
  4291. * [WebUI] Move history tab
  4292. * [WebUI] Move symbols config as well
  4293. * [WebUI] New sec to time function
  4294. * [WebUI] Prevent multiple clicks on `Refresh`
  4295. * [WebUI] RRD summary: Hide inner labels of tiny pie sectors
  4296. * [WebUI] RRD summary: Respect undefined values
  4297. * [WebUI] Reduce font size of graph's legend
  4298. * [WebUI] Remove orphaned font duplicates
  4299. * [WebUI] Remove unused code
  4300. * [WebUI] Replace spinner with animated glyphicon
  4301. * [WebUI] Reset refresh timer on server switching
  4302. * [WebUI] Rework interface to use requirejs
  4303. * [WebUI] Rework neighbours query function
  4304. * [WebUI] Separate attributes by space
  4305. * [WebUI] Set focus to password field (#1230)
  4306. * [WebUI] Simplify neighbours table populating
  4307. * [WebUI] Start rework of modules
  4308. * [WebUI] Stop stats refreshing if the page is hidden
  4309. * [WebUI] Turn d3pie's stuff into a reusable function,
  4310. * [WebUI] Unify send data functions
  4311. * [WebUI] Update D3Evolution to 0.0.1
  4312. * [WebUI] Update d3.js
  4313. * [WebUI] Update datatables to work with the requirejs
  4314. * [WebUI] Use unified tab click event handler,
  4315. * [WebUI] clusters for the chart
  4316. * [WebUI] fix uptime
  4317. 1.4.2:
  4318. * [CritFix] Deal with absent headers in DKIM
  4319. * [CritFix] Do not trust remote shingles count
  4320. * [CritFix] Fix headers selection in DKIM verification
  4321. * [Feature] Add EXT_CSS rule
  4322. * [Feature] Add toggle for disabling SURBLs
  4323. * [Feature] Extend redis lock when learning spawned
  4324. * [Feature] Parse <link> HTML tags
  4325. * [Fix] Avoid reusing g_error (Fixes #1262)
  4326. * [Fix] Do not reset loaded ANN when learning is requested
  4327. * [Fix] Fix another issue with external deps in SA
  4328. * [Fix] Fix body trie matching
  4329. * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added
  4330. * [Fix] Fix fuzzy hashes count
  4331. * [Fix] Fix keys names
  4332. * [Fix] Fix length calculations for url encoded urls
  4333. * [Fix] Fix matching of the same patterns from different tries
  4334. * [Fix] Fix name of var
  4335. * [Fix] Fix parsing of URLs with spaces and other bad chars
  4336. * [Fix] Fix probability calculations for fuzzy redis backend
  4337. * [Fix] Fix signing headers creation in DKIM
  4338. * [Fix] Plug memory leak
  4339. * [Fix] Really fix chained SA dependencies
  4340. * [Fix] Really increase lock lifetime
  4341. * [Fix] Use checksum to avoid intersection between different ANNs
  4342. * [Fix] Use rspamd hashes in embedded ucl
  4343. * [Fix] Yet another change for testing external deps
  4344. 1.4.1:
  4345. * [Feature] ASN support in Clickhouse module
  4346. * [Feature] Add clickhouse plugin
  4347. * [Feature] Add generic tool to add universal maps for lua modules
  4348. * [Feature] Add logger.debugm to debug lua modules
  4349. * [Feature] Allow to register metrics symbols using register_symbol
  4350. * [Feature] Allow to specify prefix for fann_redis
  4351. * [Feature] Clickhouse: support different masks for IPv4/IPv6
  4352. * [Feature] Support forcing action in antivirus plugin
  4353. * [Fix] Add handling of regexp maps
  4354. * [Fix] Allow backslashes in http urls
  4355. * [Fix] Avoid mapping of empty files
  4356. * [Fix] Do not load tld file to speed up rspamadm
  4357. * [Fix] Do not resolve numeric IP addresses due to ipv6 insanity
  4358. * [Fix] Filter incorrect training data
  4359. * [Fix] Fix Fuzzyconvert tool when password or DB is given
  4360. * [Fix] Fix build with custom glib/gmime
  4361. * [Fix] Fix converting of learn count from sqlite to redis
  4362. * [Fix] Fix crashes with invalid received and task:set_from_ip
  4363. * [Fix] Fix external dependencies for SA module
  4364. * [Fix] Fix fann_redis when number of scores has been changed
  4365. * [Fix] Fix hyperscan usage for non compatible platforms
  4366. * [Fix] Fix loading of maps from UCL objects
  4367. * [Fix] Fix memory leak for task-less redis requests
  4368. * [Fix] Fix mid module with new maps syntax
  4369. * [Fix] Fix parsing of URLs with username
  4370. * [Fix] Fix re cache initialisation
  4371. * [Fix] Fix replacements to sanitize '%' character
  4372. * [Fix] Fix set and regexp like static maps
  4373. * [Fix] Fix some issues in redis settings
  4374. * [Fix] Fix static IP maps
  4375. * [Fix] Fix total learns counter for redis stats
  4376. * [Fix] Fix usage of config during reload
  4377. * [Fix] Fix various warnings and issues
  4378. * [Fix] Invalidate ANN if training data is incorrect
  4379. * [Fix] Miltiple fixes to fann_redis module
  4380. * [Fix] More fixes for URLs with backslashes
  4381. * [Fix] Properly get options for ip_score module
  4382. * [Fix] Relax requirements for Received as gmail cannot RFC
  4383. * [Fix] Remove or fix hyperscan incompatible regexps
  4384. * [Fix] Settings: correctly read redis config
  4385. * [Rework] Rework lua logger interface slightly
  4386. * [Rework] Use new maps add function
  4387. 1.4.0:
  4388. * [CritFix] Add guards for inactive redis connections
  4389. * [CritFix] Another fix for proxying files using rspamd_proxy
  4390. * [CritFix] Cleanup inactive redis connections
  4391. * [CritFix] Do not sometimes try to exec posfilters before classification
  4392. * [CritFix] Fix application of IPv6 mask
  4393. * [CritFix] Fix chunked encoding when reading messages
  4394. * [CritFix] Fix file mode for rspamd_proxy
  4395. * [CritFix] Fix hyperscan compilation on regexp change
  4396. * [CritFix] Fix issue with finding of end of lines pointers
  4397. * [CritFix] Fix iteration over headers array (introduced in 1.4)
  4398. * [CritFix] Fix processing of learned tokens count for redis backend
  4399. * [CritFix] Fix race condition in checking of cached maps
  4400. * [CritFix] Fix workers scripts by sharing workers configs
  4401. * [CritFix] Introduce raw content to text parts
  4402. * [CritFix] Plug memory leak and potential memory corruption
  4403. * [Feature] Adaptive ratelimits
  4404. * [Feature] Add ASN -> rbldnsd script for asn.rspamd.com
  4405. * [Feature] Add DMARC_NA symbol
  4406. * [Feature] Add F-Prot support to antivirus module
  4407. * [Feature] Add HTTP backend to metadata exporter
  4408. * [Feature] Add Lua API module for monitored objects
  4409. * [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols
  4410. * [Feature] Add R_DKIM_PERMFAIL symbol
  4411. * [Feature] Add R_SPF_PERMFAIL symbol
  4412. * [Feature] Add Sophos antivirus support
  4413. * [Feature] Add ZSTD compression to Lua API
  4414. * [Feature] Add `mid` Lua module
  4415. * [Feature] Add `one_param` flag for metric symbols
  4416. * [Feature] Add a generic lua classifier
  4417. * [Feature] Add a very basic interface to access workers data from on_load
  4418. * [Feature] Add ability to delete a hash by its data to fuzzy_check plugin
  4419. * [Feature] Add ability to enable/disable symbols via dynamic_conf
  4420. * [Feature] Add ability to lookup settings by key
  4421. * [Feature] Add common way to disable Lua modules
  4422. * [Feature] Add compression support to rspamd client
  4423. * [Feature] Add condition to do antiviral check
  4424. * [Feature] Add configuration for lua classifiers
  4425. * [Feature] Add configuration knobs for the errors circular buffer
  4426. * [Feature] Add decompression support in rspamd client
  4427. * [Feature] Add errors exporter to the controller
  4428. * [Feature] Add expected value for monitored DNS resources
  4429. * [Feature] Add exporter from error ringbuf to ucl
  4430. * [Feature] Add extended version for fann creation function
  4431. * [Feature] Add ffi friendly version of process_regexp function
  4432. * [Feature] Add frequency and time display to webui
  4433. * [Feature] Add fuzzy_delhash command to rspamc client
  4434. * [Feature] Add implementation of redis connections pool
  4435. * [Feature] Add latency and offline time monitoring
  4436. * [Feature] Add learning support for lua classifiers
  4437. * [Feature] Add max-size and timeout options to CGP helper
  4438. * [Feature] Add method to enable/disable symbols in config
  4439. * [Feature] Add methods to get metric's actions and symbols from Lua
  4440. * [Feature] Add mmap support to lua_text
  4441. * [Feature] Add monitored object for surbl plugin
  4442. * [Feature] Add more exceptions to surbl whitelist
  4443. * [Feature] Add more meta-tokens to bayes
  4444. * [Feature] Add neural net classifier to fann_scores module
  4445. * [Feature] Add neural net serialization/deserialization
  4446. * [Feature] Add new dynamic conf module
  4447. * [Feature] Add periodic events support for lua_config
  4448. * [Feature] Add plugin to check MX'es for the sender's domain
  4449. * [Feature] Add preliminary monitored module
  4450. * [Feature] Add preliminary support of dynamic conf updates in Redis
  4451. * [Feature] Add preliminary version of clamav plugin
  4452. * [Feature] Add redis cache to asn module
  4453. * [Feature] Add replies compression
  4454. * [Feature] Add spamhaus DROP dnsbl
  4455. * [Feature] Add support for dictionary in client compression
  4456. * [Feature] Add support for fuzzy learn and unlearn from lua
  4457. * [Feature] Add support for input encryption
  4458. * [Feature] Add support of min_learns to neural net classifier
  4459. * [Feature] Add termination callbacks for workers
  4460. * [Feature] Add user-agent for rspamc
  4461. * [Feature] Add utility to perform classifier tests
  4462. * [Feature] Add zstd compression library
  4463. * [Feature] Allow HTTPS requests in lua_http
  4464. * [Feature] Allow conditions for pre and postfilters
  4465. * [Feature] Allow custom functions for ratelimits
  4466. * [Feature] Allow for excluding messages from AV scanning based on size
  4467. * [Feature] Allow for getting worker stats from Lua
  4468. * [Feature] Allow getting task UID from Lua
  4469. * [Feature] Allow parsing of mailbox messages from the commandline
  4470. * [Feature] Allow plugins to publish their lua API via rspamd_plugins
  4471. * [Feature] Allow to compare other systems with Rspamd
  4472. * [Feature] Allow to execute Lua scripts by controller
  4473. * [Feature] Allow to have a function to set custom greylist message
  4474. * [Feature] Allow to iterate over multiple tags
  4475. * [Feature] Allow to pass extra data from plugins to log helper
  4476. * [Feature] Allow to plan new periodics at different time
  4477. * [Feature] Allow to reset hashes
  4478. * [Feature] Allow to run rspamadm lua just as a lua interpreter
  4479. * [Feature] Allow to store settings in redis
  4480. * [Feature] Allow to update dynamic conf in Redis
  4481. * [Feature] Allow to use dictionaries for compression
  4482. * [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua
  4483. * [Feature] Allow whitelisting by IP for greylisting plugin
  4484. * [Feature] Antivirus: Support whitelists & pattern-matching sig names
  4485. * [Feature] Backport pack/unpack routines from Lua 5.3
  4486. * [Feature] Check settings with equal priopities in alphabetical order
  4487. * [Feature] Compress neural net in redis
  4488. * [Feature] Consider more tags when doing WHITE_ON_WHITE rule
  4489. * [Feature] Descriptive options for DMARC failure symbols
  4490. * [Feature] Descriptive options for RBL symbols
  4491. * [Feature] Enable configuration for monitored objects
  4492. * [Feature] Execute on_load scripts with ev_base ready
  4493. * [Feature] Fann scores now uses metadata from a message
  4494. * [Feature] Implement FANN threaded learning
  4495. * [Feature] Implement classifying for lua classifiers
  4496. * [Feature] Implement finish scripts for worker processes
  4497. * [Feature] Implement monitoring for DNS resources
  4498. * [Feature] Implement real priorities for pre and post filters
  4499. * [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one
  4500. * [Feature] Module to push metadata/messages to redis pubsub
  4501. * [Feature] Monitor RBL records
  4502. * [Feature] Move fann_classifier to a separate plugin
  4503. * [Feature] Normalize all ANN inputs
  4504. * [Feature] Preliminary version of metric exporter module
  4505. * [Feature] Preserve decompression context between tasks
  4506. * [Feature] Ratelimit: Support dynamic bucket size/leak rate
  4507. * [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves
  4508. * [Feature] Remove symbols weights on composites processing
  4509. * [Feature] Return symbol scores when getting resulting symbols
  4510. * [Feature] Rework lua tcp module
  4511. * [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries
  4512. * [Feature] Rule to identify some X-PHP-Script forgeries
  4513. * [Feature] Rules for scoring Google Message-ID fixes
  4514. * [Feature] Send hashes values to reply
  4515. * [Feature] Set expire for dmarc reports
  4516. * [Feature] Stop using cymru zone as it is unstable
  4517. * [Feature] Stop using of GLists for headers, improve performance
  4518. * [Feature] Store `for` in task:get_received_headers
  4519. * [Feature] Store `for` part in received headers
  4520. * [Feature] Store enabled flag for webui session
  4521. * [Feature] Store error messages in ring buffer
  4522. * [Feature] Support compressed maps
  4523. * [Feature] Support excluding selected users from ratelimits
  4524. * [Feature] Support looking up NS records in lua_dns
  4525. * [Feature] Support modern style SURBL configuration
  4526. * [Feature] Support multiple hashes in delhash path
  4527. * [Feature] Support new messages in rspamc
  4528. * [Feature] Support requests without reads in lua_tcp
  4529. * [Feature] Support setting task message from Lua
  4530. * [Feature] Track visibility of HTML elements
  4531. * [Feature] Try to add CRLF when checking DKIM
  4532. * [Feature] Try to guess line endings when folding headers
  4533. * [Feature] Try to improve normalization function for bayes
  4534. * [Feature] Use FFI to optimize SA module
  4535. * [Feature] Use length based arguments for redis, allow lua_text as arg
  4536. * [Feature] Use more layers for fann and another normalization
  4537. * [Feature] User-defined ratelimits
  4538. * [Feature] Utility to convert fuzzy storage from sqlite to redis
  4539. * [Feature] Yield DMARC_DNSFAIL on lookup failure
  4540. * [Fix] Adopt fuzzy storage for flexible backends
  4541. * [Fix] Allow plain IP addresses in Rspamd maps
  4542. * [Fix] Another fix for brain-damaged hiredis
  4543. * [Fix] Another fix for rdns write errors
  4544. * [Fix] Another fix for rdns_make_request_full invocation
  4545. * [Fix] Another fix in DKIM canonicalization
  4546. * [Fix] Another memory leak plugged
  4547. * [Fix] Another try to deal with posix idiotizm
  4548. * [Fix] Another try to fix RDNS events processing logic
  4549. * [Fix] Avoid double frees in HEAD requests
  4550. * [Fix] Avoid extra symbols for RBLs
  4551. * [Fix] Banish table.maxn from Lua parts
  4552. * [Fix] Check for socket error before connection in lua_tcp
  4553. * [Fix] Correctly propagate redis timeouts to Lua
  4554. * [Fix] Do not add extra newline in MIME mode
  4555. * [Fix] Do not be cheated by system hiredis
  4556. * [Fix] Do not classify when a message has not enough tokens
  4557. * [Fix] Do not crash on redis errors
  4558. * [Fix] Do not distinguish NXDOMAIN and NOREC for monitored
  4559. * [Fix] Do not replan retransmits if merely one server is defined
  4560. * [Fix] Do not use headers to calculate messages digests
  4561. * [Fix] Don't force action in replies module for authenticated users/local networks
  4562. * [Fix] Explicitly ban default passwords in webui
  4563. * [Fix] Finally fix ambiguity between parsed and resolved spf elts
  4564. * [Fix] Fix 'decoded' value in task:get_header_full()
  4565. * [Fix] Fix DKIM calculations
  4566. * [Fix] Fix DKIM signing for messages with no newline at the end
  4567. * [Fix] Fix DNS request in monitored
  4568. * [Fix] Fix DNS write errors processing
  4569. * [Fix] Fix HTTP methods other than GET and POST
  4570. * [Fix] Fix PERMFAIL for v6/v4 ambiguities
  4571. * [Fix] Fix absurdic scores for HFILTER_URL_ONLY
  4572. * [Fix] Fix actions in rolling history
  4573. * [Fix] Fix actrie patterns
  4574. * [Fix] Fix applying of lua dynamic confg
  4575. * [Fix] Fix autolearning errors and redis cache
  4576. * [Fix] Fix bayes learn_condition
  4577. * [Fix] Fix build with the recent OpenSSL
  4578. * [Fix] Fix caching and compressed maps
  4579. * [Fix] Fix check plain text part
  4580. * [Fix] Fix crash on OpenBSD in `url_email_start`
  4581. * [Fix] Fix double free in SPF
  4582. * [Fix] Fix extraction of shingles from redis fuzzy storage
  4583. * [Fix] Fix false sharing for symbols in the cache
  4584. * [Fix] Fix float usage in util:get_time
  4585. * [Fix] Fix folding algorithm to deal with empty tokens
  4586. * [Fix] Fix format string
  4587. * [Fix] Fix format string usage in controller errors handling
  4588. * [Fix] Fix handling of '\0' in lua_tcp
  4589. * [Fix] Fix handling of HTTP HEAD methods
  4590. * [Fix] Fix hash creation
  4591. * [Fix] Fix hiredis stupidity
  4592. * [Fix] Fix implicit settings module settingsup
  4593. * [Fix] Fix interaction with lua GC to avoid craches
  4594. * [Fix] Fix ip_score module registration
  4595. * [Fix] Fix issue with empty messages and dkim
  4596. * [Fix] Fix issues with CGP helper
  4597. * [Fix] Fix issues with the recent SPF changes
  4598. * [Fix] Fix key name to load ANN correctly
  4599. * [Fix] Fix lua tcp module by saving `do_read` in callback data
  4600. * [Fix] Fix memory leak in client when using compression
  4601. * [Fix] Fix min_learns option
  4602. * [Fix] Fix on_finish scripts and async handlers
  4603. * [Fix] Fix options for SPF dnsfail symbol
  4604. * [Fix] Fix parsing includes and redirects in SPF
  4605. * [Fix] Fix parsing of lua comments with empty lines
  4606. * [Fix] Fix parsing of unquoted HTML attributes
  4607. * [Fix] Fix periodic events and redis
  4608. * [Fix] Fix processing of fuzzy learns from Lua
  4609. * [Fix] Fix processing of redirect in SPF includes
  4610. * [Fix] Fix processing of symbols when reject limit is reached
  4611. * [Fix] Fix refcounts when map is specified by IP
  4612. * [Fix] Fix rspamd{session} class in Lua API
  4613. * [Fix] Fix setting ratelimit key for 'ip' bucket
  4614. * [Fix] Fix some cases of TLD urls detector
  4615. * [Fix] Fix statconvert tool
  4616. * [Fix] Fix stats for backend-less classifiers
  4617. * [Fix] Fix training script for fann_redis
  4618. * [Fix] Fix variable in ann module
  4619. * [Fix] Fix various errors in lua dynamic conf plugin
  4620. * [Fix] Forget old ANN when max_usages is reached to avoid overtrain
  4621. * [Fix] Further canonicalization fixes
  4622. * [Fix] Further fixes for fann_redis prefixes
  4623. * [Fix] Handle failures for inactive pooled connections
  4624. * [Fix] Improve multimap info message
  4625. * [Fix] More fixes in ANN loading
  4626. * [Fix] More fixes to fann_redis
  4627. * [Fix] More issues in fann_redis
  4628. * [Fix] More spaces fix in DKIM signature
  4629. * [Fix] Multiple fixes to asn script, add IPv6 support
  4630. * [Fix] Multiple issues in fann_redis
  4631. * [Fix] No greylist rejected messages
  4632. * [Fix] One more attempt to fix lua_redis
  4633. * [Fix] One more check for readdir...
  4634. * [Fix] Params should be treated as a hash
  4635. * [Fix] Plug memory leak in regexp desctructor
  4636. * [Fix] Process headers only once
  4637. * [Fix] Properly handle nil values in ratelimit plugin
  4638. * [Fix] Really fix redis shingles check
  4639. * [Fix] Remove fann with incorrect layers count
  4640. * [Fix] Remove mentions of deleted include
  4641. * [Fix] Remove some incompatible functions
  4642. * [Fix] Settings: fix `authenticated` parameter (#886)
  4643. * [Fix] Skip MX check for authenticated users and local networks
  4644. * [Fix] Slightly fix ANN routines
  4645. * [Fix] Stop caching records with DNS failures
  4646. * [Fix] Treat all errors in redis_pool as fatal errors for a connection
  4647. * [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule
  4648. * [Fix] Try to avoid race condition when using rrd
  4649. * [Fix] Try to reload redis scripts if they are missing
  4650. * [Fix] Unbreak once_received skipping for local networks
  4651. * [Fix] Unlock ANN on error
  4652. * [Fix] Use memmove for overlapping regions
  4653. * [Fix] Use real size instead of displayed for core limits
  4654. * [Fix] Use the correct macro to get the size of control
  4655. * [Fix] Various fixes for errors ringbuffer
  4656. * [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails
  4657. * [Fix] mid: fix map initialization
  4658. * [Fix] mid: handle incorrect rgexps in the map
  4659. * [Rework] Add extract training data function to fann_redis
  4660. * [Rework] Add preliminary train tests
  4661. * [Rework] Add redis storage feature to fann_redis
  4662. * [Rework] Adopt fuzzy storage for abstract backend
  4663. * [Rework] Adopt plugins
  4664. * [Rework] First reiteration on fann scores
  4665. * [Rework] Implement loading/invalidating
  4666. * [Rework] Make lua_redis task agnostic
  4667. * [Rework] Make rspamd protocol messages useful
  4668. * [Rework] Massive removal of legacy code
  4669. * [Rework] More cleanup actions
  4670. * [Rework] Remove legacy code never used for classifiers
  4671. * [Rework] Remove outdated and unused lua_session module
  4672. * [Rework] Reorganize fuzzy backend structure
  4673. * [Rework] Reorganize the internal backend structure
  4674. * [Rework] Restore old fann_scores, move common parts
  4675. * [Rework] Rework and simplify rbl plugin
  4676. * [Rework] Rework parsing of DMARC records
  4677. 1.3.4:
  4678. * [Feature] ASN module; support matching ASN/country in multimap
  4679. * [Feature] Add SPF method in spf return result
  4680. * [Feature] Add Yandex and Mail.ru forwarding rules
  4681. * [Feature] Add mempool maps in multimap
  4682. * [Feature] Add rule for identifying mail sent by eval()'d PHP code
  4683. * [Feature] Add support of stub DNSSEC resolver to rdns
  4684. * [Feature] Add task:get_digest method
  4685. * [Feature] Allow for more fine-grained scoring for ip_score
  4686. * [Feature] Allow to get digest of a mime part from lua
  4687. * [Feature] Allow to print message digest in logs
  4688. * [Feature] Fold DKIM-Signature header
  4689. * [Feature] Implement encrypted logs
  4690. * [Feature] Log URLs encrypted if we have log encryption pubkey
  4691. * [Feature] Pass authenticated bit to lua
  4692. * [Feature] Read redis backend statistics configuration from global section
  4693. * [Feature] Show the exact value matched for multima symbols
  4694. * [Feature] Store task checksum
  4695. * [Fix] Avoid setting limits when required elements are missing
  4696. * [Fix] DMARC: Fix alignment checking for subdomains
  4697. * [Fix] DMARC: deal with missing and spurious spaces
  4698. * [Fix] Defer insertion of results in ip_score to avoid skewing stats
  4699. * [Fix] Disable DMARC for local/authorized mail
  4700. * [Fix] Fix handling of proxied headers in controller
  4701. * [Fix] Fix hex printing of strings
  4702. * [Fix] Fix issue with spaces in maps
  4703. * [Fix] Fix parsing of forwarded IP
  4704. * [Fix] Fix reload in some plugins and workers
  4705. * [Fix] Fix reloading on SIGHUP
  4706. * [Fix] Fix some border cases for DKIM canonicalization
  4707. * [Fix] Fix url maps
  4708. * [Fix] Make dnssec configurable option disabled by default for now
  4709. * [Fix] rspamadm statconvert: force db to be a string
  4710. * [Fix] rspamadm statconvert: use db/password for learn cache
  4711. * [Rework] Rework flags in rspamd logger
  4712. 1.3.3:
  4713. * [CritFix] Check hyperscan cache sanity before loading
  4714. * [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage)
  4715. * [Feature] Add SARBL (sarbl.org) uribl
  4716. * [Feature] Add `--search-pattern` option to rspamd_stats
  4717. * [Feature] Add some sanity check for very long from/to log elements
  4718. * [Feature] Allow to create hashes from string in a single step
  4719. * [Feature] Fix order of pre and postfilters
  4720. * [Feature] Improve lua URLs API
  4721. * [Feature] Improve message about fuzzy rules
  4722. * [Feature] Pre-calculate blake2 digest for all parts
  4723. * [Feature] Print radix duplicate keys as IP addresses
  4724. * [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf
  4725. * [Feature] Use faster hash function for fuzzy storage
  4726. * [Feature] rspamd_stats: support log directory reading
  4727. * [Fix] Add sanity check for url filters
  4728. * [Fix] Do not show rmilter section as a fake metric in rspamc
  4729. * [Fix] Fix URL filters
  4730. * [Fix] Fix a stupid mistake in util.strequal_caseless
  4731. * [Fix] Fix blake2b hash of the string "rspamd"
  4732. * [Fix] Fix filename maps filter
  4733. * [Fix] Fix finding tld in util.get_tld
  4734. * [Fix] Fix multimap content filters
  4735. * [Fix] Fix returning boolean from Lua
  4736. * [Fix] Fix returning of REDIS_NIL
  4737. * [Fix] Try to deal with multiple workers terminated
  4738. * [Fix] Use forced DNS request when calling for lua_http
  4739. * [Rework] Rework multimap filters, add redis maps
  4740. 1.3.2:
  4741. * [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL
  4742. * [Feature] Add correlations report in fuzzy stats
  4743. * [Feature] Add experimental CGP integration
  4744. * [Feature] Add method to get urls length in a text part
  4745. * [Feature] Add new methods to lua_html to access HTML tags
  4746. * [Feature] Allow all types of symbols to be added via __newindex method
  4747. * [Feature] Allow to create settings for authenticated users
  4748. * [Feature] Allow to get block content for HTML tags
  4749. * [Feature] Improve DNS failures when dealing with SPF
  4750. * [Feature] Properly implement R_WHITE_ON_WHITE rule
  4751. * [Feature] Remove old ugly rules
  4752. * [Feature] Rspamc can now add dkim signature in mime mode
  4753. * [Feature] Store content length for HTML tags
  4754. * [Feature] Support reacher set of HTML colors
  4755. * [Feature] Try to avoid FP for low contrast fonts detection
  4756. * [Fix] Add missing HTML colors
  4757. * [Fix] Add spaces to dkim signature to allow folding
  4758. * [Fix] Avoid returning NaN as score on scan
  4759. * [Fix] Decode entitles in href parts
  4760. * [Fix] Do not cache SPF records with DNS errors
  4761. * [Fix] Do not crash on cyclic depends
  4762. * [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed
  4763. * [Fix] Do not set absent hostname to "unknown"
  4764. * [Fix] Do not stress redis with KEYS command (#791)
  4765. * [Fix] Fix DMARC_BAD_POLICY symbol
  4766. * [Fix] Fix HFILTER_URL module
  4767. * [Fix] Fix HFILTER_URL_ONELINE rule
  4768. * [Fix] Fix buffering in CGP integration
  4769. * [Fix] Fix colors propagation from parent nodes
  4770. * [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey
  4771. * [Fix] Fix dependencies id sanity check
  4772. * [Fix] Fix folding for semicolon separated tokens
  4773. * [Fix] Fix largest possible TLD behaviour
  4774. * [Fix] Fix last token folding
  4775. * [Fix] Fix length calculations in white on white rule
  4776. * [Fix] Fix multiple request headers structure
  4777. * [Fix] Fix multiple values headers freeing
  4778. * [Fix] Fix parsing of background color
  4779. * [Fix] Fix printing from field in log_urls
  4780. * [Fix] Fix processing of last element of DMARC policies
  4781. * [Fix] Further fixes for HTML colors
  4782. * [Fix] Further fixes for multiple values headers
  4783. * [Fix] Further fixes for white on white rule
  4784. * [Fix] Further fixes in HTML tags parsing
  4785. * [Fix] Ignore content type/subtype case
  4786. * [Fix] Increase score of R_WHITE_ON_WHITE
  4787. * [Fix] Parse CGP envelope data
  4788. * [Fix] Propagate colors in HTML
  4789. * [Fix] Restore multiple values headers in protocol
  4790. * [Fix] Restore multiple values in headers processing
  4791. * [Fix] Some more changes to tag's content length calculations
  4792. * [Fix] Some more fixes for low contrast fonts detector
  4793. * [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..])
  4794. * [Fix] Trigger HTML_SHORT_LINK_IMG on any external image
  4795. * [Fix] rspamd_stats: remove deprecated defined(@array)
  4796. 1.3.1:
  4797. * [CritFix] Fix catena passwords validation
  4798. * [CritFix] Fix crash when the first received is faked
  4799. * [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid
  4800. * [Feature] Allow for matching hostnames in multimap (#773)
  4801. * [Feature] Allow for setting action based on DMARC disposition
  4802. * [Feature] Allow limiting of the inbound message size
  4803. * [Feature] Allow maps with multiple symbols and scores
  4804. * [Feature] Allow regexps in the emails maps
  4805. * [Feature] Allow to register metric symbols from multimap
  4806. * [Feature] Allow to reset redis tokens instead of appendig values
  4807. * [Feature] Allow to store strings in radix maps
  4808. * [Feature] Check UTF validity when there are utf regexps in a map
  4809. * [Feature] Correctly work when there is no hard reject action
  4810. * [Feature] Implement dependencies for maps
  4811. * [Fix] Another effort to unbreak sqlite locking
  4812. * [Fix] Avoid crash when closing mmapped file
  4813. * [Fix] Do not break history on NaN in required score
  4814. * [Fix] Ensure that hyperscan cache written is written properly
  4815. * [Fix] Filter NaN from scores in history
  4816. * [Fix] Fix DNSBL maps
  4817. * [Fix] Fix another locking issue in sqlite
  4818. * [Fix] Fix another locking issue with mapped files
  4819. * [Fix] Fix deadlock in mmaped file stats
  4820. * [Fix] Fix dependencies in multimap plugin
  4821. * [Fix] Fix emails module configuration
  4822. * [Fix] Fix greylist plugin (#755)
  4823. * [Fix] Fix greylisting plugin variable usage
  4824. * [Fix] Fix installed permissions for rspamd_stats
  4825. * [Fix] Fix locking in mmapped statistics
  4826. * [Fix] Fix paths in tests
  4827. * [Fix] Fix prefilter mode for multimap
  4828. * [Fix] Forgot to commit leftover changes
  4829. * [Fix] Really fix local.d includes
  4830. * [Fix] Restore selective greylisting behaviour
  4831. * [Fix] Set max size on per connection basis
  4832. * [Fix] Use temporary storage for hyperscan cache
  4833. * [Rework] Remove systemd socket activation
  4834. 1.3.0:
  4835. * [CritFix] Fix SA rawbody processing - exclude top part
  4836. * [CritFix] Fix decoding of UTF HTML entitles
  4837. * [CritFix] Fix encrypted fuzzy requests
  4838. * [CritFix] Fix leak of shared memory fds and files
  4839. * [CritFix] Fix levenshtein distance calculations
  4840. * [CritFix] Fix mime headers processing
  4841. * [CritFix] Fix parsing of URLs in texts
  4842. * [CritFix] Fix parsing of missing classes
  4843. * [CritFix] Fix redis structure by adding {NULL, NULL} member
  4844. * [CritFix] Fix some more URL detector issues
  4845. * [CritFix] Fix systemd sockets activation
  4846. * [CritFix] Fix unencrypted passwords processing in the controller
  4847. * [CritFix] Fix writing CDPs to the database
  4848. * [CritFix] Fix writing of encrypted HTTP requests
  4849. * [CritFix] Plug memory leak in dkim module
  4850. * [CritFix] Plug memory leak in headers getting code
  4851. * [CritFix] Pre-filters and post-filters were completely broken
  4852. * [CritFix] Properly support SA body regexps
  4853. * [CritFix] Really skip filters in case of pre-result set
  4854. * [CritFix] Restore the intended pre-filters behaviour
  4855. * [Rework] Adopt new maps code
  4856. * [Rework] Compile ragel sources when building rspamd
  4857. * [Rework] Finish rework for the rest of places that use HTTP
  4858. * [Rework] Fix DKIM headers canonicalization
  4859. * [Rework] Fix lua maps API
  4860. * [Rework] Import linenoise for line editing
  4861. * [Rework] Include config structure to all rcl handlers
  4862. * [Rework] Make chartable module useful
  4863. * [Rework] Move http internal structures to a private header
  4864. * [Rework] Partly fix controller
  4865. * [Rework] Remove dedicated images list
  4866. * [Rework] Rename http proxy to rspamd proxy
  4867. * [Rework] Rename mime parts structures
  4868. * [Rework] Rework HTTP code
  4869. * [Rework] Rework exceptions and newlines processing
  4870. * [Rework] Rework pre and postfilters system
  4871. * [Rework] Separate method to close backend connections
  4872. * [Rework] Start the complete maps rework
  4873. * [Rework] Use dynamically generated ragel C sources
  4874. * [Feature] Add 'blacklist' and 'strict' modes for whitelists
  4875. * [Feature] Add 'symbols_enabled' and 'groups_enabled' to settings
  4876. * [Feature] Add ESMTPSA received type
  4877. * [Feature] Add a simple script to evaluate rspamd rules in the logs
  4878. * [Feature] Add a simple tool to generate DKIM keys
  4879. * [Feature] Add a trivial heuristic for codepages
  4880. * [Feature] Add and use mumhash for non-crypto hashing
  4881. * [Feature] Add better method to check lua userdata types
  4882. * [Feature] Add better zip files search algorithm
  4883. * [Feature] Add catena PBKDF function
  4884. * [Feature] Add configuration knobs for in and out parser scripts
  4885. * [Feature] Add content filtering support to multimap
  4886. * [Feature] Add different timeouts for proxy connections
  4887. * [Feature] Add dot commands for lua REPL:
  4888. * [Feature] Add execution of lua global functions script
  4889. * [Feature] Add function for pretty printing of inet addresses
  4890. * [Feature] Add function to convert fstring_t to c string
  4891. * [Feature] Add function to create temporary shared memory mapping
  4892. * [Feature] Add function to generate random hex string
  4893. * [Feature] Add generic fucnction to parse IP maps
  4894. * [Feature] Add initial version of HTTP lua repl
  4895. * [Feature] Add learn condition to the default configuration
  4896. * [Feature] Add learn conditions for classifiers
  4897. * [Feature] Add limit for dkim signatures to be checked
  4898. * [Feature] Add locking routines for lua_util
  4899. * [Feature] Add lua API for getting info from archives
  4900. * [Feature] Add lua utility to decode URL encoding
  4901. * [Feature] Add method to copy message from http connection
  4902. * [Feature] Add mirrors feature
  4903. * [Feature] Add more algorithms for shingles generation
  4904. * [Feature] Add more domains to redirectors list
  4905. * [Feature] Add more encodingsto cryptobox hash API
  4906. * [Feature] Add more file utilities to lua_util
  4907. * [Feature] Add more functions to extract data from text parts
  4908. * [Feature] Add more methods to get headers from a task
  4909. * [Feature] Add more methods to init http message body
  4910. * [Feature] Add more options for redis config parsing function
  4911. * [Feature] Add new representation of email address
  4912. * [Feature] Add new symbols to filter bad extensions in messages
  4913. * [Feature] Add new utility methods to mimepart object
  4914. * [Feature] Add openphish support to rspamd phishing module
  4915. * [Feature] Add parsers for SMTP address in ragel
  4916. * [Feature] Add parsing of mirror hosts for fuzzy storage
  4917. * [Feature] Add parsing scripts for master connection as well
  4918. * [Feature] Add preliminary greylist plugin
  4919. * [Feature] Add preliminary phishtank support
  4920. * [Feature] Add preliminary rarv5 support
  4921. * [Feature] Add preliminary version of ssl toolbox
  4922. * [Feature] Add protection against open files limit and accepting sockets
  4923. * [Feature] Add rar v4 support
  4924. * [Feature] Add reading scripts for master connection
  4925. * [Feature] Add replies plugin
  4926. * [Feature] Add results parsing code
  4927. * [Feature] Add routines to compare and check pubkeys
  4928. * [Feature] Add simpler versions of refcounts
  4929. * [Feature] Add some time manipulation functions for lua APi
  4930. * [Feature] Add support for non-standard BATV signatures
  4931. * [Feature] Add support of address with at-domain list
  4932. * [Feature] Add support to search archives by magic
  4933. * [Feature] Add task:get_rawbody method
  4934. * [Feature] Add test to check shared memory support sanity
  4935. * [Feature] Add the initial version of LUA repl to rspamadm
  4936. * [Feature] Add throughput graph for RRD backend to WebUI
  4937. * [Feature] Add universal function to make a proper redis request
  4938. * [Feature] Add universal function to parse redis servers for plugins
  4939. * [Feature] Add util.unlink function
  4940. * [Feature] Add utility function to return random number from 0 to 1
  4941. * [Feature] Add utility method to convert ftok to C string
  4942. * [Feature] Add utility to map shared memory segments
  4943. * [Feature] Add versions to fuzzy storage
  4944. * [Feature] Add workaround for legacy clients in rspamd proxy
  4945. * [Feature] Add workaround for systems without sane shmem support
  4946. * [Feature] Add xoroshiro+ fast rng for non-crypto purposes
  4947. * [Feature] Adopt plugins for new maps API
  4948. * [Feature] Allow SPF to be checked for empty tasks
  4949. * [Feature] Allow binary patterns in lua_trie
  4950. * [Feature] Allow catena encrypted passwords in controller
  4951. * [Feature] Allow client ip match in the settings
  4952. * [Feature] Allow easy adding and overriding of fuzzy rules
  4953. * [Feature] Allow empty tasks to be processed
  4954. * [Feature] Allow hostnames in IP maps
  4955. * [Feature] Allow https maps
  4956. * [Feature] Allow multiple PBKDF types in `rspamadm pw`
  4957. * [Feature] Allow named fuzzy rules
  4958. * [Feature] Allow non zero terminated patterns in multipattern
  4959. * [Feature] Allow partial hash updates
  4960. * [Feature] Allow pipelining for redis.make_request
  4961. * [Feature] Allow sending empty requests using client
  4962. * [Feature] Allow setting fuzzy flag by symbol not by value
  4963. * [Feature] Allow setting scores and actions from lua
  4964. * [Feature] Allow shared memory simple http client
  4965. * [Feature] Allow static lua files in any parts of rspamd sources
  4966. * [Feature] Allow to change flag from fuzzy learn condition
  4967. * [Feature] Allow to check rspamd_text using maps
  4968. * [Feature] Allow to disable composite rules from settings
  4969. * [Feature] Allow to disable some modules from common redis setup
  4970. * [Feature] Allow to extract ucl_object from lua using common API
  4971. * [Feature] Allow to get settings and settings id hash from lua_task
  4972. * [Feature] Allow to have specific settings for mirrored traffic
  4973. * [Feature] Allow to open message from a shared memory segment
  4974. * [Feature] Allow to parse pubkeys from the rcl config
  4975. * [Feature] Allow to pass extradata from rspamd to rmilter
  4976. * [Feature] Allow to query storage about number of fuzzy hashes stored
  4977. * [Feature] Allow to read logs without symbols scores
  4978. * [Feature] Allow to read password from console for rspamc
  4979. * [Feature] Allow to set ciphers and CA paths in config
  4980. * [Feature] Allow to skip some initialization phases to speed up rspamadm
  4981. * [Feature] Allow underscore separated names in settings
  4982. * [Feature] Allow versioning for sqlite databases
  4983. * [Feature] Always allow to terminate rspamd
  4984. * [Feature] Better deal with backend errors
  4985. * [Feature] Better lua_redis logging
  4986. * [Feature] Configure CA path and ciphers
  4987. * [Feature] Create a dedicated parser to strip newlines
  4988. * [Feature] Deduplicate the same urls in multimap module
  4989. * [Feature] Distinguish luajit from lua
  4990. * [Feature] Do not print garbadge in --compact output
  4991. * [Feature] Dynamically detect if a CPU is incompatible with hyperscan
  4992. * [Feature] Enable forced resolving for some lua plugins
  4993. * [Feature] Enable rrd by default
  4994. * [Feature] Enable workaround for exim
  4995. * [Feature] Fix task functions to work without rspamd_config
  4996. * [Feature] Further improvements to chartable module
  4997. * [Feature] Further micro-optimizations for hashing and shingles
  4998. * [Feature] Further relax parser
  4999. * [Feature] Humanize numbers in stats widgets
  5000. * [Feature] Implement HTTPS client
  5001. * [Feature] Implement SSL support in http client
  5002. * [Feature] Implement body rules for the trie plugin
  5003. * [Feature] Implement braced regexp quantifiers
  5004. * [Feature] Implement compare scripts for mirrors results
  5005. * [Feature] Implement compare scripts setup
  5006. * [Feature] Implement composites policies
  5007. * [Feature] Implement conditional learning for classifiers
  5008. * [Feature] Implement constructing of map from UCL
  5009. * [Feature] Implement dkim signing in dkim check plugin
  5010. * [Feature] Implement fuzzy storage updates
  5011. * [Feature] Implement fuzzy updates push protocol
  5012. * [Feature] Implement https maps
  5013. * [Feature] Implement inter-process maps caching
  5014. * [Feature] Implement mapping of remote flags to local flags
  5015. * [Feature] Implement mirroring in rspamd proxy
  5016. * [Feature] Implement multi-flags fuzzy replies
  5017. * [Feature] Implement multiple-sources fuzzy storage
  5018. * [Feature] Implement order of pre/post filters
  5019. * [Feature] Implement partial deleting for multi-flags
  5020. * [Feature] Implement pipelining for redis async interface
  5021. * [Feature] Implement ragel parser for received headers
  5022. * [Feature] Implement reading of messages to shared memory
  5023. * [Feature] Implement refcount for messages
  5024. * [Feature] Implement retransmits for master connection
  5025. * [Feature] Implement zero-copy mode for HTTP reading
  5026. * [Feature] Improve SPF domain detection logic
  5027. * [Feature] Improve config:register_symbol function
  5028. * [Feature] Improve error report for type mismatch in lua
  5029. * [Feature] Improve fstrings API
  5030. * [Feature] Improve getting SMTP data from lua_task
  5031. * [Feature] Improve levenshtein distance function
  5032. * [Feature] Improve logging in proxy and add refcounts
  5033. * [Feature] Improve logging lua types
  5034. * [Feature] Improve master/slave logging
  5035. * [Feature] Improve phishing plugin
  5036. * [Feature] Improve phishtank and openphish support
  5037. * [Feature] Improve ragel build target
  5038. * [Feature] Improve statistics script
  5039. * [Feature] Initialize ssl library to use SSL connections
  5040. * [Feature] Interpolate points sent to webui
  5041. * [Feature] Limit logging of elements that could have too many items
  5042. * [Feature] Lock ANN file when loading
  5043. * [Feature] New abstract hashing API in cryptobox
  5044. * [Feature] Normalize quoted addresses
  5045. * [Feature] Now cryptobox lua API accepts rspamd text as input
  5046. * [Feature] Optimize alignment to speed up hashing
  5047. * [Feature] Parse received date and ESMTPA proto
  5048. * [Feature] Parse received timestamp
  5049. * [Feature] Pass settings id to log helper
  5050. * [Feature] Pass settings id to lua script from log helper
  5051. * [Feature] Perform files expansion on proxying
  5052. * [Feature] Preliminary implementation of fuzzy master/slave updates
  5053. * [Feature] Print userdata using tostring if possible
  5054. * [Feature] Propagate HTTP result string when using proxy
  5055. * [Feature] Properly implement unweighted round-robin algorithm
  5056. * [Feature] Reduce number of timers queries
  5057. * [Feature] Rework and improve fuzzy storage
  5058. * [Feature] Rework dns resolving API for lua, add 'forced' option
  5059. * [Feature] Rework fann module to understand settings
  5060. * [Feature] Rework listening system to allow multiple socket types per worker
  5061. * [Feature] Rework ratelimit module to set expiration
  5062. * [Feature] Save bayes probability in memory pool var
  5063. * [Feature] Save settings id hash for convenience
  5064. * [Feature] Search for SSL_set_tlsext_host_name support
  5065. * [Feature] Send DKIM signature to protocol reply
  5066. * [Feature] Show DKIM signature in rspamc client
  5067. * [Feature] Show symbols description in scan output
  5068. * [Feature] Sign message merely after DKIM check
  5069. * [Feature] Simplify machines by assuming that headers are unfolded
  5070. * [Feature] Sort symbols when displaying them in log
  5071. * [Feature] Split main connection from mirrored connections
  5072. * [Feature] Start moving to the new email address structure
  5073. * [Feature] Store HTTP headers in a hash table
  5074. * [Feature] Store more information about compressed files
  5075. * [Feature] Store raw headers value to use them in DKIM
  5076. * [Feature] Store text parts content with newlines stripped
  5077. * [Feature] Support DKIM signing
  5078. * [Feature] Support EXIF jpeg images
  5079. * [Feature] Support archive files list extraction
  5080. * [Feature] Support archives when matching patterns in multimap
  5081. * [Feature] Support premium/academic feed for openphish
  5082. * [Feature] Support rspamd_updates via https
  5083. * [Feature] Supprort FQDNs in phishing module maps
  5084. * [Feature] Try to read on fuzzy timeout to avoid fake timeouts
  5085. * [Feature] Try to select the optimal possible function for input
  5086. * [Feature] Unescape and unquote smtp addresses
  5087. * [Feature] Update fuzzy timestamp when adding value
  5088. * [Feature] Update mumhash
  5089. * [Feature] Use -flto if possible when optimizations are enabled
  5090. * [Feature] Use extended map types in lua map, unify code
  5091. * [Feature] Use file lock in logger to avoid deadlocks
  5092. * [Feature] Use generic global string split function
  5093. * [Feature] Use metrohash as well
  5094. * [Feature] Use mumhash by default for incremental hashing
  5095. * [Feature] Use mumhash for words hashing
  5096. * [Feature] Use new ip parsing API
  5097. * [Feature] Use new maps API for local addrs
  5098. * [Feature] Use new ragel parser in message parsing code
  5099. * [Feature] Use new received parser instead of old one
  5100. * [Feature] Use new redis API in DMARC plugin
  5101. * [Feature] Use new redis API in greylist plugin
  5102. * [Feature] Use new redis API in ip_score plugin
  5103. * [Feature] Use new redis API in ratelimit plugin
  5104. * [Feature] Use new redis API in replies plugin
  5105. * [Feature] Use new version of register_symbol in rspamd plugins
  5106. * [Feature] Use offset when passing shm to deal with encrypted requests
  5107. * [Feature] Use one pass to remove newlines and store their positions
  5108. * [Feature] Use rspamd specific type checks for userdata
  5109. * [Feature] Use shared memory storage for http maps
  5110. * [Feature] Use universal redis definitions in rspamd plugins
  5111. * [Feature] Various improvements in greylist module
  5112. * [Feature] Wait for sqlite if locked when switching to WAL mode
  5113. * [Fix] Add filenames sanity filtering for mime types
  5114. * [Fix] Add guards for empty parts
  5115. * [Fix] Add missing types
  5116. * [Fix] Add more priority for config file symbols registered from UCL
  5117. * [Fix] Add sanity checks when compiling regexp maps
  5118. * [Fix] Add spaces instead of newlines to the normalized content
  5119. * [Fix] Add workaround for ancient openssl
  5120. * [Fix] Add workaround for gmime CTE stupidity
  5121. * [Fix] Add workaround for hex digits
  5122. * [Fix] Adjust MISSING_MIMEOLE score
  5123. * [Fix] Adjust body/body_buf when stealing encrypted message
  5124. * [Fix] Adopt lua task API for the new email addresses structure
  5125. * [Fix] Allow for disabling DMARC reporting when Redis is configured
  5126. * [Fix] Always register openphish and phishtank virtual symbols
  5127. * [Fix] Always use shmem on linux
  5128. * [Fix] Another change of newlines policy
  5129. * [Fix] Another d3evolution update
  5130. * [Fix] Another fix for exim workaround
  5131. * [Fix] Another fix for legacy clients
  5132. * [Fix] Another fix for maps scheduling
  5133. * [Fix] Another fix for marking upstreams inactive
  5134. * [Fix] Another fix for postfilters
  5135. * [Fix] Another fix for redis timeouts
  5136. * [Fix] Avoid `table.getn` method as it has been removed in lua 5.3
  5137. * [Fix] Avoid double hashing for images
  5138. * [Fix] Avoid linking with actrie if hyperscan is enabled
  5139. * [Fix] Check copy result when sending message to mirrors
  5140. * [Fix] Cleanup message when assiging body
  5141. * [Fix] Cleanup stack from global vars
  5142. * [Fix] Correctly parse query type
  5143. * [Fix] Disable all symbols if enable_groups is found in settings
  5144. * [Fix] Disable fts as it is completely broken in bloody linux
  5145. * [Fix] Disable multiple autolearn checks
  5146. * [Fix] Disallow updates by default
  5147. * [Fix] Do not abort when cannot load a map
  5148. * [Fix] Do not check recursion for non-DNS SPF record types
  5149. * [Fix] Do not delete uninitialized events
  5150. * [Fix] Do not die if shmem_mkstemp fails
  5151. * [Fix] Do not die when no metrics defined
  5152. * [Fix] Do not even try pcre in case of regexp maps
  5153. * [Fix] Do not greylist messages if redis has failed somehow
  5154. * [Fix] Do not greylist on rejection
  5155. * [Fix] Do not leave temporary maps cached
  5156. * [Fix] Do not output meaningless errors
  5157. * [Fix] Do not send NaN in json
  5158. * [Fix] Don't mix hyperscan and pcre processing within a same task
  5159. * [Fix] Finally rework and simplify redis backend for statistics
  5160. * [Fix] Fix Exim shutdown patch
  5161. * [Fix] Fix JIT compilation for PCRE2 expressions
  5162. * [Fix] Fix JIT usage for PCRE2
  5163. * [Fix] Fix REPL output
  5164. * [Fix] Fix SMTP address parsing machine
  5165. * [Fix] Fix UTF8 mode in PCRE2
  5166. * [Fix] Fix a stupid misprint in word 'phishing'
  5167. * [Fix] Fix accepting the first update when local idx is -1
  5168. * [Fix] Fix adding maps from ucl
  5169. * [Fix] Fix adding upstream to an active queue
  5170. * [Fix] Fix and rescore R_PARTS_DIFFER logic
  5171. * [Fix] Fix body rules in SA plugin
  5172. * [Fix] Fix body start position
  5173. * [Fix] Fix border case in urls detector
  5174. * [Fix] Fix border cases for incremental hashing
  5175. * [Fix] Fix caseless uthash application
  5176. * [Fix] Fix chartable issue with starting digits
  5177. * [Fix] Fix client_ip in users settings
  5178. * [Fix] Fix compilation issue
  5179. * [Fix] Fix conditional learning
  5180. * [Fix] Fix crash on empty maps
  5181. * [Fix] Fix creating of URLs from LUA
  5182. * [Fix] Fix creating of temporary shmem segment
  5183. * [Fix] Fix creation of mmapped statfiles
  5184. * [Fix] Fix descriptors leak on shmem detaching
  5185. * [Fix] Fix detaching of shared memory segments
  5186. * [Fix] Fix detection of URLs in text parts
  5187. * [Fix] Fix directories processing for rspamc
  5188. * [Fix] Fix displaying of rewrite subject in WebUI
  5189. * [Fix] Fix dkim private keys operations
  5190. * [Fix] Fix dkim signing
  5191. * [Fix] Fix dynamic scoring of subject symbols
  5192. * [Fix] Fix email address build
  5193. * [Fix] Fix encrypted proxy requests
  5194. * [Fix] Fix errors counting in upstreams
  5195. * [Fix] Fix errors handling in the proxy
  5196. * [Fix] Fix event bases for IO events
  5197. * [Fix] Fix events handling when scheduling map wacth
  5198. * [Fix] Fix fann rewrite
  5199. * [Fix] Fix files fallback for shmem transfer
  5200. * [Fix] Fix fuzzy adding in webui
  5201. * [Fix] Fix fuzzy storage encrypted mirroring
  5202. * [Fix] Fix fuzzy storage sync replies
  5203. * [Fix] Fix handling of the same words
  5204. * [Fix] Fix inserting values to the sources list
  5205. * [Fix] Fix ipv6 mask application
  5206. * [Fix] Fix issue with missing recipients
  5207. * [Fix] Fix issues with multiple returns from lua
  5208. * [Fix] Fix learning for non-existent backend
  5209. * [Fix] Fix legacy clients support in proxy
  5210. * [Fix] Fix length calculations for shared memory segments
  5211. * [Fix] Fix listening on UDP sockets
  5212. * [Fix] Fix loading of file maps
  5213. * [Fix] Fix long regexp flags (e.g. {sa_body})
  5214. * [Fix] Fix lua compare function init
  5215. * [Fix] Fix maps descriptions
  5216. * [Fix] Fix maps locking
  5217. * [Fix] Fix max_train setup in ANN module
  5218. * [Fix] Fix memory corruption
  5219. * [Fix] Fix memory leak in unsigned maps reading
  5220. * [Fix] Fix misprints for lto usage
  5221. * [Fix] Fix more issues with scripts processing
  5222. * [Fix] Fix next-to-last extension length check
  5223. * [Fix] Fix openssl initialization
  5224. * [Fix] Fix order of arguments
  5225. * [Fix] Fix order of initialization
  5226. * [Fix] Fix parser
  5227. * [Fix] Fix parsing of binary tries
  5228. * [Fix] Fix parsing of braced IPv6 addresses
  5229. * [Fix] Fix parsing of nested braces in SMTP comments
  5230. * [Fix] Fix parsing of rarv5 archives
  5231. * [Fix] Fix periodic scheduling when a map is not modified
  5232. * [Fix] Fix possible FP in TRACKER_ID rule
  5233. * [Fix] Fix post-filters processing
  5234. * [Fix] Fix potential NULL dereference
  5235. * [Fix] Fix processing of <br> and <hr> tags
  5236. * [Fix] Fix processing of addresses in protocol
  5237. * [Fix] Fix processing of messages without received headers
  5238. * [Fix] Fix proxying issue for unconnected sessions
  5239. * [Fix] Fix proxying of the encrypted messages
  5240. * [Fix] Fix race condition with shared memory by refcounts
  5241. * [Fix] Fix ratelimit initialization
  5242. * [Fix] Fix redis set request in replies plugin
  5243. * [Fix] Fix redis timeout events handling
  5244. * [Fix] Fix redis timeouts processing logic
  5245. * [Fix] Fix refcounts in lua_redis
  5246. * [Fix] Fix results checking if no master connection is active
  5247. * [Fix] Fix return value for couple of lua functions
  5248. * [Fix] Fix round-robin selection when upstreams have no weight
  5249. * [Fix] Fix rows calculation in graph
  5250. * [Fix] Fix rspamd_redis_make_request syntax in replies plugin
  5251. * [Fix] Fix scheduling of locked map events
  5252. * [Fix] Fix scores detection
  5253. * [Fix] Fix searching for newline positions
  5254. * [Fix] Fix secure_ip setting in controller
  5255. * [Fix] Fix sending data to graph command
  5256. * [Fix] Fix setting of score for parts differ
  5257. * [Fix] Fix setting of the lua top
  5258. * [Fix] Fix setting path for lua
  5259. * [Fix] Fix setting path for phishtank
  5260. * [Fix] Fix settings application
  5261. * [Fix] Fix shm_open call as described in POSIX
  5262. * [Fix] Fix size of length in fuzzy mirror wire protocol
  5263. * [Fix] Fix smtp grammar issues
  5264. * [Fix] Fix some issues with redis API
  5265. * [Fix] Fix some issues with retries in the proxy
  5266. * [Fix] Fix stack growing
  5267. * [Fix] Fix start of body detection in DKIM
  5268. * [Fix] Fix state on timeout
  5269. * [Fix] Fix stats script
  5270. * [Fix] Fix substring search when there are '\0' in strings
  5271. * [Fix] Fix symbol name for spf soft fail
  5272. * [Fix] Fix symbol type's check
  5273. * [Fix] Fix symbols registration and execution
  5274. * [Fix] Fix the case of multiple values keys
  5275. * [Fix] Fix the default symbol names according to metric
  5276. * [Fix] Fix timeout setup on learning
  5277. * [Fix] Fix timeouts in redis cache processing
  5278. * [Fix] Fix timeouts processing in lua_redis
  5279. * [Fix] Fix upstreams interaction for rspamd proxy
  5280. * [Fix] Fix usage of rdns reply structure
  5281. * [Fix] Fix varargs loop
  5282. * [Fix] Fix whitelists and blacklists in SA rules
  5283. * [Fix] Fix write servers setup for redis
  5284. * [Fix] Fix writing of HTTP messages
  5285. * [Fix] Force rspamd to upgrade fuzzy db on opening
  5286. * [Fix] Free the correct pointer
  5287. * [Fix] Further fixes for lto and static linking
  5288. * [Fix] Further fixes for surbl extensions map
  5289. * [Fix] Further fixes in maps code
  5290. * [Fix] Further improvements to error messages in fuzzy check
  5291. * [Fix] Further tweaks to redis garbadge collection
  5292. * [Fix] Groups are now case insensitive
  5293. * [Fix] Handle log pipe read errors
  5294. * [Fix] Handle nested dependencies in SpamAssassin plugin
  5295. * [Fix] Implement new automata to skip empty lines for dkim signing
  5296. * [Fix] Improve error messages on fuzzy add
  5297. * [Fix] Improve lua redis handling
  5298. * [Fix] Improve phishing module logging
  5299. * [Fix] Improve printing of fuzzy errors
  5300. * [Fix] Improve rrd diagnostic errors
  5301. * [Fix] Improve strcase hash used in uthash
  5302. * [Fix] Include fuzzy key to distinguish storages with different keys
  5303. * [Fix] Include slave cluster name into http request
  5304. * [Fix] Include some more information about archives
  5305. * [Fix] Indicate upstream error on timeout
  5306. * [Fix] Initialize hash tables array to avoid crashes
  5307. * [Fix] Initialize parser scripts properly
  5308. * [Fix] Initialize vars to avoid warnings
  5309. * [Fix] Inverse logic for saving ANN
  5310. * [Fix] Link lpeg to rspamd lua library
  5311. * [Fix] Make extension checks case-insensitive
  5312. * [Fix] Mark expired hashes as not found and not as zero flag
  5313. * [Fix] Match archive name as well
  5314. * [Fix] More and more fixes to redis states
  5315. * [Fix] More fixes about shared memory in proxy
  5316. * [Fix] More fixes for redis refcounts
  5317. * [Fix] More fixes to end of headers detection
  5318. * [Fix] More fixes to events logic
  5319. * [Fix] More fixes to multi-flag fuzzy storage
  5320. * [Fix] More fixes to parts distance calculations
  5321. * [Fix] More guards for redis free
  5322. * [Fix] One more fix in redis destructor
  5323. * [Fix] One more try to fix redis
  5324. * [Fix] PIE is required for static build
  5325. * [Fix] Partial fix for mmap'd statistics tests
  5326. * [Fix] Plug memory leak in proxy
  5327. * [Fix] Properly detect end of headers position
  5328. * [Fix] Properly init and free session structures
  5329. * [Fix] Reduce PRECEDENCE_BULK rule weight
  5330. * [Fix] Reduce the default thresholds for learning
  5331. * [Fix] Remove Type=forking from systemd unit file (#709)
  5332. * [Fix] Remove bad FANN file to save computational resources
  5333. * [Fix] Remove event before closing of fd to avoid race conditions
  5334. * [Fix] Remove parsing of 'from' variable in redis backend
  5335. * [Fix] Remove some bad domains from whitelists
  5336. * [Fix] Repair optional dependencies
  5337. * [Fix] Reset master connection when retransmitting scan request
  5338. * [Fix] Restore ONCE_RECEIVED symbol
  5339. * [Fix] Restore compatibility with old lua API behaviour
  5340. * [Fix] Restore redis runtime state
  5341. * [Fix] Reverse options when received
  5342. * [Fix] Send updates to mirrors only if we have some changes
  5343. * [Fix] Set host attribute properly when making HTTP request from lua
  5344. * [Fix] Set terminated state before calling of async free
  5345. * [Fix] Simplify MISSING_MIMEOLE rule
  5346. * [Fix] Simplify state machine by ignoring multiple spaces
  5347. * [Fix] Skip setting RPATH for static builds
  5348. * [Fix] Slightly reduce weights of rules with high FP rate
  5349. * [Fix] Some fixes to libmagic initialization
  5350. * [Fix] Some more fixes to ratelimit plugin
  5351. * [Fix] Strip '\r\n' properly
  5352. * [Fix] Switch hashes to mumhash
  5353. * [Fix] Treat NaN values properly in graph command
  5354. * [Fix] Try to avoid FP when checking for phished URLs
  5355. * [Fix] Try to avoid recursive events deletions
  5356. * [Fix] Try to fix false positive URL detections in text parts
  5357. * [Fix] Try to fix issue in redis stats backend when task is closed
  5358. * [Fix] Try to fix proxying of stupid spamc protocol to HTTP mirrors
  5359. * [Fix] Try to fix redis crashes
  5360. * [Fix] Try to fix upstreams with one element
  5361. * [Fix] Try to handle multiline history in a more sane way
  5362. * [Fix] Unbreak build on gcc < 4.9
  5363. * [Fix] Update RPM spec/sources (#700)
  5364. * [Fix] Update d3evolution version
  5365. * [Fix] Update mumhash implementation
  5366. * [Fix] Use custom error function for pre and post filters
  5367. * [Fix] Use new postfilters and prefilters API in the plugins
  5368. * [Fix] Use non-blocking mode for systemd sockets
  5369. * [Fix] Use shared memory merely for local backends in the proxy
  5370. * [Fix] Use watchers for spf plugin
  5371. * [Fix] Varioud fixes to the maps code
  5372. 1.2.8:
  5373. * Another fix for exim workaround (#637)
  5374. * Fix unencrypted passwords processing in the controller
  5375. * Fix setting path for lua (#652)
  5376. * Fix usage of rdns reply structure (#654)
  5377. * Use file lock in logger to avoid deadlocks
  5378. * Add `application/octet-stream` mime type for `pdf` extension (by @moisseev)
  5379. * Implement new automata to skip empty lines for dkim signing (#651)
  5380. * Fix parsing of missing classes
  5381. * Clarify some rspamc arguments (by @fatalbanana)
  5382. * Correct suppress spelling
  5383. 1.2.7:
  5384. * Slightly reduce weights of rules with high FP rate
  5385. * Add workround for rspamd-1.3
  5386. * Fix possible FP in TRACKER_ID rule
  5387. * Simplify MISSING_MIMEOLE rule
  5388. * Add workaround for gmime CTE stupidity
  5389. * Fix mime headers processing
  5390. * Fix false positive URL detections in text parts
  5391. * Fix Exim shutdown patch
  5392. * Enable workaround for exim mailbox format
  5393. * Backport shingles static test
  5394. * Fix levenshtein distance calculations
  5395. * Fix max_train setup in ANN module
  5396. * Fix redis structure by adding {NULL, NULL} member
  5397. * Fix build with unmodified LibreSSL opensslv.h
  5398. * Repair optional dependencies
  5399. * Really skip filters in case of pre-result set
  5400. * Restore the intended pre-filters behaviour
  5401. * Fix ipv6 mask application
  5402. 1.2.6:
  5403. * Fix parsing of URLs in texts
  5404. * Fix creating of URLs from LUA
  5405. * Fix some more URL detector issues
  5406. * Fix unit tests
  5407. * Fix JIT compilation for PCRE2 expressions
  5408. * Fix JIT usage for PCRE2
  5409. * Fix UTF8 mode in PCRE2
  5410. * Add workaround for pre-historic compilers (#605)
  5411. * Fix and rescore R_PARTS_DIFFER logic
  5412. * Properly set lua paths for tests
  5413. * Fix SA rawbody processing - exclude top part
  5414. * Store text parts content with newlines stripped
  5415. * Properly support SA body regexps
  5416. * Fix body rules in SA plugin
  5417. * Fix setting of score for parts differ
  5418. * More fixes to parts distance calculations
  5419. - Use hashed words instead of full words for speed
  5420. - Improve levenstein distance calculations and penalise replaces
  5421. - Always return number from 0 to 1
  5422. - Use g_malloc instead of alloca
  5423. * Fix percents output in R_PARTS_DIFFER
  5424. * Plug memory leak in dkim module
  5425. * Plug minor memory leak in regexps creation
  5426. 1.2.5:
  5427. * Plug an important memory leak in headers getting code
  5428. * Remove some bad domains from whitelists
  5429. 1.2.4:
  5430. * Implement new multipattern matcher that uses hyperscan if possible
  5431. * Use mutlipattern for lua_trie code
  5432. * Add utility methods for multipattern
  5433. * Use multipattern in url matcher
  5434. * Add escape functions for hyperscan
  5435. * Allow to optimize lua -> C transition by flattening table args
  5436. * Optimize hot paths in SA plugin
  5437. * Optimize rspamd_re_cache_type_from_string
  5438. * Allow empty tries
  5439. * Fix extraction of URLs from Subject
  5440. * Allow to have different flags for different patterns in multipattern
  5441. * Add common directory for hyperscan cache to config
  5442. * Implement caching for hyperscan multipattern
  5443. * Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev)
  5444. * Allow multipattern scans to be nested for the case of hyperscan
  5445. * Simplify SURBL redirector search code and avoid ac_trie
  5446. * Add two way substring search algorithm
  5447. * Avoid acism usage to find gtube pattern
  5448. * Fix processing of empty headers
  5449. * Allow to disable pthread mutexes on broken platforms
  5450. * Make web interface not send password in query strings (#585) by @fatalbanana
  5451. * Add maximum delay to ratelimit module
  5452. * Backport fix for empty files inclusion from libucl
  5453. * Fix settings id setup
  5454. * Add min_learns option to classifiers
  5455. * Use more clever to utf8 conversion strategy
  5456. * Fix disabling of virtual symbols in the settings
  5457. * Rework settings to work properly in metric-less configuration
  5458. * Set the default limit for classifier
  5459. * Fix ttl based expiration from LRU cache
  5460. * Rework DKIM module to use OpenSSL for digests
  5461. * Fix mailto urls parsing with hyperscan
  5462. * Do not set obscured flag for urls starting with spaces
  5463. * Fix crash on redis learn
  5464. * Fix ratelimit ctime setting
  5465. 1.2.3:
  5466. * New DCC module (by @smfreegard)
  5467. * Rework whitelist module:
  5468. - Now we check different elements for different checks
  5469. - MIME from for DMARC
  5470. - DKIM signature domain for DKIM
  5471. - SMTP from or HELO for SPF
  5472. * Fix regexps results combination (*critical*)
  5473. * Fix issue with expressions processing (*critical*)
  5474. * Optimize strlcpy for aligned input
  5475. * Add support of half-closed connection in lua_tcp
  5476. * Allow to print compact json in client
  5477. * Save required score in history (#581)
  5478. * Allow to attach file descriptors to control commands
  5479. * Allow to send descriptors from workers to main
  5480. * Allow to attach fd when broadcasting to workers
  5481. * Implement log pipe feature for rspamd logs analysis
  5482. * Add `log_helper` worker
  5483. * Add `URIBL_SBL_CSS` (by @smfreegard)
  5484. * Add worker scripts functionality
  5485. * Add on load hooks for rspamd_config
  5486. * Add lua scripts for log_helper worker
  5487. * Add generic maillist detector (#584)
  5488. * Implement FANN autolearn using log_helper worker
  5489. * Rework metrics configuration to allow includes
  5490. * Change default value of forced removal in composite rules
  5491. * Allow to use assembly version of blake2b on x86 cpu
  5492. * Use less precise (but faster) clock if possible
  5493. * Insert redirected URL to the urls list
  5494. * Allow to get and set callback data for rspamd symbols
  5495. * Add binary heap implementation
  5496. * Use binary heap for expire algorithms in the hash
  5497. * Use `least frequent used` expiration strategy
  5498. * Allow to get mime headers from a task
  5499. * Add support for mime headers in `regexp` module
  5500. * Update Exim patches (by @fatalbanana)
  5501. * Allow building rspamd with jemalloc
  5502. * Save multipart boundaries
  5503. * SA plugin changes:
  5504. - Properly handle MIME headers
  5505. - Fix eval:check_for_missing_to_header rule
  5506. - Implement SA compatible body regexps
  5507. - Use sabody rules in SA plugin
  5508. * LUA API changes:
  5509. - Add util.get_ticks function
  5510. - Add util.stat function
  5511. - Add task:get_symbols_numeric method
  5512. - Add method to get number of symbols in the cache
  5513. - Add lua methods to get redirected urls
  5514. - Allow to get callbacks for lua symbols
  5515. - Add config:set_symbol_callback function
  5516. 1.2.2:
  5517. * Use HTTP Content-Type on non mime input if possible
  5518. * Save log level when compressing log messages
  5519. * Further rework of composite rules (add '^' prefix)
  5520. * Add tracking for rspamd expressions
  5521. * Store actions limits in metric result
  5522. * Fix parsing of include/redirect with many records in SPF
  5523. * Add method to disable symbols execution in the cache
  5524. * Allow to disable checks from settings
  5525. * Allow to select settings by id in HTTP query
  5526. * Find URLs with '\r' and '\n' inside href attribute
  5527. * Implement vectored mode for hyperscan (experimental)
  5528. * Improve client connection errors diagnostics
  5529. * Allow to edit new files with signtool
  5530. * Improve hashes performance on 32 bit platforms
  5531. * Fix sorting of limits
  5532. * Remove slow and unused rules `INVALID_EXIM_RECEIVED*`
  5533. * Add expression:process_traced lua method
  5534. * Allow tables in task:insert_result
  5535. * Save trace for SA metas
  5536. * Do not parse broken TLD parts in URLs
  5537. * Investigate many border cases in URLs parser
  5538. 1.2.1:
  5539. * Add list support to `mime types` module configuration (by @moisseev)
  5540. * Allow symbols params to be printed in logs
  5541. * Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN
  5542. * Fix crashes on arm32
  5543. * Do not classify message if some class is missing
  5544. * Fix cryptobox cleanup
  5545. * Remove multipart/report from bad mime types (#569)
  5546. * Improve logging for fuzzy hashes
  5547. * Show map URLs in webui
  5548. * Sort symbols in webui
  5549. 1.2.0:
  5550. * New dynamic updates plugin
  5551. * Regular expressions map support
  5552. * Faster radix trie algorithm
  5553. * Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes)
  5554. * PCRE2 support
  5555. * Allow quoted and slashed keys in map
  5556. * Add proper support of DNS resolvers balancing (#552)
  5557. * Rework includes and configuration system for better local changes support
  5558. * New keypairs framework for signing and encryption
  5559. * Added support for dynamic modules and workers
  5560. * Allow to dump configuration with help comments
  5561. * Rework once_received module
  5562. - Fix priority for `good_hosts`
  5563. - If a good host has been found do not add once_received symbols
  5564. - Fix priorities for strict once_received
  5565. - Add ability to whitelist IP addresses
  5566. * Implement support of signed maps for HTTP and file maps
  5567. * Add command to sync fuzzy storage (#533)
  5568. * Rework system of symbols and actions registration
  5569. It is possible now to use priorities when adding symbols to metrics and
  5570. override scores for symbols with lower priority with the scores with
  5571. high priority.
  5572. * Add auth support and db selection for redis stats
  5573. * Improve composite rules application
  5574. * Add ignore_received option
  5575. * Fix critical issue with inconsistent resorting
  5576. * Fix `all` in spf redirects
  5577. * Add punycoded versions for IDN domains (#554)
  5578. * Improve sorting order for symbols cache
  5579. * Add lockless logging for processes management
  5580. * Allow to specify flags for metric symbols
  5581. * Load images height and width from style attribute (#538)
  5582. * Override DNS requests limits for SPF and DKIM
  5583. * Fix resetting symbols to their default values in WebUI
  5584. * Improve configuration agility for redis stats
  5585. * Allow to set db and password for redis in stat_convert
  5586. * Import the latest libucl
  5587. * LUA API changes:
  5588. - Add rspamd_version function to LUA API
  5589. - Add lua_cryptobox module
  5590. - Add lua_map module
  5591. - Add task:set_metric_action lua API method
  5592. - Fix race condition in lua_tcp module
  5593. - Fix a lot of issues in lua_redis module
  5594. - Rework and abstract lua maps API
  5595. - Add util.strlen_utf8 lua function
  5596. - Add lua functions for caseless comparison
  5597. - Allow optional symbols registration
  5598. - Add config:add_map table form method, add regexp maps
  5599. - Add task:has_urls method
  5600. - Add task:has_flag method
  5601. - Add html tags methods to lua_html
  5602. - Add task:get_dns_req
  5603. * Plugins changes:
  5604. - Add support for WLBLEval SA plugin
  5605. - Use caseless comparison in SA and DMARC plugins
  5606. - Allow SA plugin to set scores for rspamd symbols
  5607. - Add regexp maps support to multimap
  5608. - Allow filenames match in multimap
  5609. - Add more filters for the existing map types
  5610. - Fix html images rules to reduce FP rates
  5611. * New rules:
  5612. - LONG_SUBJ - too long subject
  5613. - MIME_BAD_ATTACHMENT - bad attachment type
  5614. - RDNS_NONE - no reverse DNS record for sender's IP
  5615. - Fix MISSING_MIMEOLE rule for modern OE
  5616. * Many other bugfixes, memory leaks plugs thanks to:
  5617. - Coverity scan
  5618. - New gcc-6 warnings
  5619. - valgrind manual iterations
  5620. * Documentation improvements:
  5621. - FAQ list: https://rspamd.com/doc/faq.html
  5622. - Reworked quick start guide
  5623. - Added documentation for all active modules
  5624. * Other changes:
  5625. - Dropped Ubuntu Vivid support
  5626. - Added Ubuntu Xenial support
  5627. - Rework build system for rspamd and rmilter
  5628. 1.1.4:
  5629. * Print traceback on lua errors in lua config
  5630. * Fix leaks in lua error paths
  5631. * Improve 'R_EMPTY_IMAGE' rule
  5632. * Fix metas memoization in SA plugin
  5633. * Properly set `flag` in fuzzy replies
  5634. * Fix arguments order
  5635. * Fix issue with out-of-boundary reading
  5636. * Fix issues found by coverity
  5637. * Same result checking error found by coverity
  5638. * Fix varargs processing (found by coverity)
  5639. * Fix error in printing hex
  5640. * Reduce weights for some hfilter patterns
  5641. * Add aliases for task:get_from_ip:
  5642. - task:get_addr
  5643. - task:get_from_addr
  5644. - task:get_ip
  5645. * Rework once_received module
  5646. - Fix priority for `good_hosts`
  5647. - If a good host has been found do not add once_received symbols
  5648. - Fix priorities for strict once_received
  5649. - Add ability to whitelist IP addresses
  5650. * Fix `MISSING_MIMEOLE` rule for modern OE
  5651. * Treat meta tags as embedded tags (#501)
  5652. 1.1.3:
  5653. * Fix DSN rules when SMTP from is unavailable
  5654. * Fix statconvert routine to avoid lua module usage
  5655. * Set a sane quark for configtest to avoid NULL to be printed in logs
  5656. * Support c11 if available
  5657. * Fix parsing of ip:port strings
  5658. * Add more diagnostic for lua subr errors
  5659. * Fix task:set_from_ip lua method
  5660. * Add basic routines for digital signatures
  5661. * Add tool for digital signatures
  5662. * Add plain open file API method for atomic open
  5663. * Fix parsing nested braces inside logger vars
  5664. * Pre filters now actually skip processing
  5665. * Add pre-filter mode for multimap
  5666. * Switch to apache 2 license
  5667. 1.1.2:
  5668. * Fix stat_cache closing
  5669. * Add checkpoints to sqlite3 learn cache
  5670. * Do not recompile lua generated headers all the time
  5671. * Increase number of messages learned
  5672. * Fix issues with dual stack and hfilter
  5673. * Disable MID checks for hfilter by default
  5674. * Fix cache definitions in multiple classifier and no type
  5675. * Don't crash if learn cache failed to initialize
  5676. * Fix googlegroups support in maillist plugin
  5677. * Rework flags LUA API:
  5678. - Allow to check for a specific flag
  5679. - Add `learn_spam`, `learn_ham` and `broken_headers` flags
  5680. - Unify internal functions
  5681. * Add `BROKEN_HEADERS` rule
  5682. * Add support for forged confirmation headers (by @AdUser)
  5683. * Allow `any`, `mime` and `smtp` for get_from/get_recipients
  5684. * Add mime types checking plugin
  5685. * Add rule to detect spammers attempts to cheat mime parsing
  5686. * Rework parsing of IP addresses in configuration (better IPv6 support)
  5687. * Add `util.parse_mail_address` function to LUA API
  5688. * Add lua sqlite3 module
  5689. * Implement synchronous redis call
  5690. * Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana)
  5691. * Add stat_convert command to convert stats tokens from sqlite3 to redis
  5692. * Implement redis advanced lua api with pipelining
  5693. * Fix memory leak on redis stat (#500)
  5694. * Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana)
  5695. * Fix build with custom pcre
  5696. * Fix fuzzy relearning (#498)
  5697. * Improve planning of asynchronous tasks
  5698. * Show slow rules in log
  5699. * Add warning for slow regexps
  5700. * Add base32 decode/encode routines to lua util
  5701. * Allow converting of learn cache from sqlite to redis
  5702. * Add methods to check if a messages has from/rcpts
  5703. * Improve and fix multimap plugin:
  5704. - Restore 'header' maps
  5705. - Add filters for headers
  5706. - Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters
  5707. - Add generic regexp filters
  5708. * Disable reload command in rc scripts
  5709. * Improve runtime CPU dispatcher for libcryptobox
  5710. * Add preliminary support of digital signatures via ed25519
  5711. * Add detection for RDRAND support
  5712. * Print configuration of crypto on start
  5713. * A in SPF presumes AAAA lookup as well
  5714. 1.1.1:
  5715. * Fix duplicated XBL symbol
  5716. * Reduce log severity for ratelimit missing servers
  5717. * Fix XBL composite to avoid duplicate symbols
  5718. * Reduce weight of URL_ONLY rule due to FP rate
  5719. * Disable fuzzy hashes from the metadata for now
  5720. * Fix processing of empty messages (#486)
  5721. * Always treat DNS timeouts as temporary fail for SPF
  5722. * Fix issue with SPF double IP stack (#483)
  5723. * Use X-Forwarded-For when checking secure_ip (#488)
  5724. * Fix hash calculation for sqlite stats
  5725. * Fix memory corruption on punycode
  5726. * Fix strings allocation in punycode
  5727. * Fix error message (#491)
  5728. 1.1.0:
  5729. * Incompatible change: sqlite3 and per_user behaviour:
  5730. Now both redis and sqlite3 follows the common principles for per-user
  5731. statistics:
  5732. 1) If per-user statistics is enabled check per-user tokens ONLY
  5733. 2) If per-user statistics is not enabled then check common tokens ONLY
  5734. If you need old behaviour, then you'd need to use separate classifier
  5735. for per-user statistics.
  5736. * Implement redis statistics backend and cache
  5737. * Implement autolearning for statistics
  5738. * Reworked statistics architecture from scratch
  5739. * Add hyperscan (https://github.com/01org/hyperscan) engine for regular
  5740. expressions:
  5741. - add lazy loader for hyperscan databases
  5742. - rework regexp cache to have joint pcre/hyperscan scanning
  5743. - implement hyperscan pre-filter support
  5744. - add compilation guards for bad expressions
  5745. - implement `rspamadm control recompile` command
  5746. - implement hyperscan cache monitoring
  5747. - slides: <https://highsecure.ru/rspamd-hyperscan.pdf>
  5748. * Implement flexible task logging
  5749. * Rework fuzzy worker:
  5750. - it is now possible to run multiple fuzzy workers;
  5751. - implement lazy writing as sqlite3 is bad at concurrent writing;
  5752. - add retries for simple sql commands in fuzzy backend;
  5753. - use fine-grained transactions for fuzzy;
  5754. - implement new multi-pubkeys mode;
  5755. - allow encrypted only storages;
  5756. - rework statistics for fuzzy;
  5757. - add `rspamadm control fuzzystat` command for extended statistics;
  5758. - implement human readable output for the previous command;
  5759. - add condition script for learning fuzzy storage;
  5760. * Various fixes to SPF:
  5761. - fix `redirect` records;
  5762. - fix domains when parsing mx/ptr/a records in includes/redirects;
  5763. - fix issues with multiple addresses in SPF records;
  5764. - ignore SPF results in case of DNS failure;
  5765. - adjust TTL of records when resolving subelements of SPF records;
  5766. - always select `v=spf1` line if it is available
  5767. - do not cache records with DNS failure in subrequests;
  5768. - ignore records with temporary fails during subrequests resolving;
  5769. - fix `RDNS_RC_NOREC` support;
  5770. * Add clang plugin for static analysis:
  5771. - implement static checks for `rspamd_printf` format strings;
  5772. * Add 'allow_raw_input' option for non-mime messages
  5773. * Recognize types using libmagic
  5774. * Fix parsing of IPv6 received headers
  5775. * Add new interface of communication between workers in rspamd
  5776. * Add support for named socketpairs
  5777. * Don't write URLs by default as it is too verbose
  5778. * Set status for HTTP replies
  5779. * Try load `rspamd.conf.override`
  5780. * Implement words decaying for text parts to limit many checks
  5781. * Improve support of SA rules and plugins:
  5782. - add check_for_shifted_date and check_for_missing_to_header eval rules;
  5783. - add 'check_relays_unparseable' support;
  5784. - add `check_for_mime('mime_attachement')` function;
  5785. - use new re_cache interface for all SA rules;
  5786. - add support for `Mail::SpamAssassin::Plugin::MIMEHeader`;
  5787. - add support of 'special' SA headers to `exists` function;
  5788. - fix issue when SA metas contain other metas;
  5789. - fix freemail rules;
  5790. * Many fixes to the URL parser
  5791. * Match any newline character in regexps
  5792. * Fix resolving of upstreams and detection of poor IPv6 configurations
  5793. * Parse upstreams selection algorithm from the configuration line
  5794. * Add `reresolve` command to the control interface
  5795. * Generate fuzzy hashes from task metadata (URLs and headers)
  5796. * Add method to check if IP is local and `local_addrs` option
  5797. * Implement forced timeout for delayed filters
  5798. * Disable fast path of pcre-jit as it seems to be broken
  5799. * Bayes fixes:
  5800. - new normalizer function;
  5801. - really use weights of tokens from the OSB algorithm;
  5802. - restore multiple classifiers support;
  5803. * Rules changes:
  5804. - add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's;
  5805. - improve empty image rule;
  5806. - rework `FORGED_RECIPIENTS` rule;
  5807. - reduce weight of `SUSPICIOUS_RECIPS`;
  5808. - fix `*_NORESOLVE_MX` symbols in hfilter;
  5809. - add `SUBJ_ALL_CAPS` rule with support of UTF8
  5810. - add spamhaus SBL to uribl
  5811. - fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules
  5812. - remove `R_TO_SEEMS_AUTO` as it generates a lot of FP;
  5813. - add new Message-ID regexp for Thunderbird (by @moisseev);
  5814. * Plugins changes:
  5815. - allow ratelimit plugin to set symbol instead of pre-result
  5816. - support IP DNS black lists for URIBL (e.g spamhaus SBL);
  5817. - drop deprecated SURBL bits (by @fatalbanana)
  5818. - rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana)
  5819. - add `SURBL_BLOCKED` (by @fatalbanana)
  5820. - add `CR_SURBL`
  5821. - SURBL: allow fallthrough to default symbol (by @fatalbanana)
  5822. - Settings: fix IP match (by @fatalbanana)
  5823. - SURBL: add missing symbols to metric (by @fatalbanana)
  5824. - allow processing images urls for SURBL
  5825. - unconditionally disable SPF for authenticated users and local networks
  5826. * Rework ratelimit plugin
  5827. - switch to `rates` instead of old and stupid strings to setup;
  5828. - check if a bucket is zero and disable the corresponding limits'
  5829. - turn off all buckets by default;
  5830. - check either `rcpt` or `user` buckets, not all together'
  5831. - document new `rates` and `symbol` options;
  5832. - inform user about what buckets are used in the configuration;
  5833. * Add neural network **experimental** plugin
  5834. * Add a sample script to learn neural network from rspamd logs
  5835. * Add documentation strings support to rspamd:
  5836. - add strings for the main configuration options;
  5837. - document workers options;
  5838. - add internal plugin options;
  5839. - create `rspamadm confighelp` routine;
  5840. - implement human readable output for the previous command;
  5841. - add subtree search support;
  5842. - add keyword search support;
  5843. * Documentation improvements, tutorials section, statistics description
  5844. * Many other minor and major bugfixes not noted here
  5845. 1.0.11:
  5846. * Fix spf redirects
  5847. * Fix domains when parsing mx/ptr/a records in includes/redirects
  5848. * Fix unfolded base64 encoding
  5849. * Fix GError use-after-free
  5850. * Do not rewrite the original url when using redirector
  5851. * Fix parsing of fragment in urls
  5852. * Fix processing of HTML tags
  5853. * Improve empty image rule
  5854. * Avoid long double type
  5855. * Fix tokens weights in OSB algorithm
  5856. * Improve debugging for bayes
  5857. 1.0.10:
  5858. * Fix settings application (#416)
  5859. * Fix another issue with fixed strings
  5860. * Fix hash function invocation
  5861. * Use the proper string for make_dns_request in lua_http
  5862. * Fix scan time output
  5863. * Update webui:
  5864. - fix labels for greylisting
  5865. - fix dimension of scan time
  5866. 1.0.9:
  5867. * Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password
  5868. * Support passwords longer than 64 symbols
  5869. 1.0.8:
  5870. * Add function to traverse AST atoms
  5871. * Allow dependencies on rspamd symbols for SA metas
  5872. * Fix memory corruption when timeout is removed in fuzzy check
  5873. * Fix encrypted fuzzy add processing
  5874. * Avoid use-after-free in controller session destructor
  5875. * Use session pool instead of task pool in fuzzy check
  5876. * Fix assembly in i386 mode (#413, #412)
  5877. 1.0.7:
  5878. * Plugged memory leaks in internet address object & html parser
  5879. * Fixed static build
  5880. * Fixed multiple sigchld processing
  5881. * Fixed deletion of signal events after event processing loop
  5882. * Fixed build on ARM (#404 - reported by @Gottox)
  5883. * Fixed setting the default mask for SPF.
  5884. * Fixed sanitisation of HTTP query values
  5885. * Fixed parsing of the last header in encrypted HTTP messages
  5886. * Additions and fixes for test suite & benchmarks
  5887. * Added openssl aes-256-gcm support to libcryptobox & HTTP server
  5888. * Implemented support for starting multiple HTTP servers
  5889. * Implemented batch accept in HTTP server
  5890. * Added module to get data from HTTP headers (#285 - reported by @msimerson)
  5891. * Added `rspamadm control` command
  5892. * Added ability to sort counters output.
  5893. * Added ability to specify custom headers for rspamc client
  5894. * Fix architecture detection
  5895. * Converted history storage to the UCL format
  5896. * Allow flexible number of rows in history
  5897. * Fix action badges in WebUI
  5898. * Add universal cryptobox hash API
  5899. * Migrated to the optimized blake2b implementation adopted from Andrew Moon
  5900. * Allow explicit loading of specific modules
  5901. * Always load settings module
  5902. * Allow to add symbols from settings
  5903. * Fix double free in the controller fuzzy learn command
  5904. * Avoid endless loop when cannot open sqlite db
  5905. * Updated libucl
  5906. 1.0.6:
  5907. * Fix build on i386
  5908. * Update CentOS7 service file patch (by @fatalbanana)
  5909. * Fix path to rspamadm in Debian init script (by @fatalbanana)
  5910. * Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD
  5911. * Fix portability issues
  5912. * Use cryptobox chacha for libottery
  5913. * Better support of 32 bit builds
  5914. * Fix header name tokens setup
  5915. * Fix levenstein distance method for words
  5916. * Add workaround for old libevent (#400)
  5917. * Fix microseconds in termination timer
  5918. * Fix some more issues with fixed strings
  5919. * Explicitly test CPU instructions even after CPUID call
  5920. * Do not check out of boundary memory
  5921. * Do not output broken emails
  5922. * Fix unknown symbols registration
  5923. * Handle SIGILL using longjmp
  5924. * Block signals when exiting event loop
  5925. * Fix incorrect allocation size
  5926. * Slightly optimize alignment
  5927. * Restore rspamd -t for compatibility
  5928. * Add more sanity checks for emails
  5929. 1.0.5:
  5930. * Add rspamd control interface:
  5931. - support `stat` command to get runtime stats of rspamd workers
  5932. - support `reload` command to reload runtime elements (e.g. sqlite3 databases)
  5933. * Rework curve25519 library for modular design:
  5934. - add Sandy2x implementation by Tung Chou
  5935. - fix CPU detection for variables loading assembly
  5936. - add testing for curve25519 ECDH
  5937. * New fixed strings library
  5938. * Add `R_SUSPICIOUS_IMAGES` rule
  5939. * Enable mmap in sqlite3
  5940. * Use new strings in the HTTP code
  5941. * Improve google perftools invocation
  5942. * Improve performance profiling in http test
  5943. * Reorganize includes to reduce namespace pollution
  5944. * Allow specific sections printing in configdump command
  5945. * Rework workers signals handlers to be chained if needed
  5946. * Update socketpair utility function
  5947. * Add control_path option for rspamd control protocol
  5948. * Fix ownership when listening on UNIX sockets
  5949. * Rework signals processing in main
  5950. * Remove extra tools from rspamd (they live in rspamadm now)
  5951. * Remove global rspamd_main
  5952. * Add global timeout for the overall task processing (8 seconds by default)
  5953. * Sanitize NULL values for fuzzy backend
  5954. * Store NM between encrypt/decrypt
  5955. * Add textpart:get_words_count method
  5956. * Fix generic DNS request in lua
  5957. * Tune hfilter weights
  5958. * Add support of IPv6 in hfilter
  5959. * Fix parsing of HTTP headers with IP addresses
  5960. * Sync with the recent libucl
  5961. * Various minor bugfixes
  5962. 1.0.4:
  5963. * Add configdump routine to rspamadm
  5964. * Implement retransmits for fuzzy_check plugin
  5965. * Fix events processing for learning anf checking fuzzy hashes
  5966. * Avoid dependency on unneeded and uncompatible glib include
  5967. * Add `historyreset` command to the controller
  5968. * Fix loading of tokenizer config from dump (#389)
  5969. * Add sorting hints for the history
  5970. * Allow custom lua scripts for users/languages extraction (#388)
  5971. * Do not add FORGED_RECIPIENTS when 'To' is missing (#387)
  5972. * Do not add R_UNDISC_RCPT when 'To' is missing (#387)
  5973. * Add encryption to fuzzy check plugin
  5974. * Add encryption for fuzzy storage
  5975. * Add new epoch for encrypted fuzzy request
  5976. * Add encryption for `rspamd.com` storage
  5977. * Remove gmime processing for LDA mode as it is deadly broken
  5978. * Add routine to find end of headers position in mime messages
  5979. * Fix LDA headers folding
  5980. * Init libraries in rspamc client as well to avoid locale issues
  5981. * Avoid collision with locally installed includes
  5982. * Allocate and free memory with the same allocator in rspamadm (#385)
  5983. * Preserve expired fuzzy hashes counter
  5984. * Improvements in webui:
  5985. - Add favicon.ico
  5986. - Rework history table
  5987. - Fix sorting for the history
  5988. - Migrate to bootstrap 3 and jquery 2
  5989. - Fix css bugs
  5990. - Add glyphicons
  5991. - Add reset history
  5992. - Improve history buttons
  5993. - Redraw graph to avoid display issues
  5994. - Webui is now MIT licensed to match licensing policy of rspamd
  5995. 1.0.3:
  5996. * Fix piechart clean slice (#380)
  5997. * Fix controller crashes when GString is reallocated (#381)
  5998. * Correctly set locale before start
  5999. * Set C locale for numeric values
  6000. * Add rspamadm routine:
  6001. - add `pw` command to manage passwords
  6002. - add `help` command for displaying help
  6003. - add `configtest` command to check configuration files
  6004. - add `keypair` command for generating encryption keys
  6005. - add `fuzzy_merge` routine to merge fuzzy sqlite databases
  6006. - add a simple manual page for rspamadm
  6007. * Allow metric registration for composite expressions
  6008. * Add strict mode for configtest
  6009. * Add logger counters
  6010. * Save and show learned messages count (#383)
  6011. * Add `no_stat` flag
  6012. * Add `task:set_flag` and `task:get_flags` (#382)
  6013. * Enable foreign keys in sqlite3
  6014. * Remove orphaned shingles from fuzzy storage
  6015. * Optimize synchronization steps for fuzzy storage
  6016. * Allow delayed conditions registration
  6017. * Add lua API for conditions registering
  6018. 1.0.2:
  6019. * Fix critical bug in webui that prevents password from being sent
  6020. * Rework webui view:
  6021. - Switch to d3.js for graphs
  6022. - Improve piechart look
  6023. - Rework colors for piechart
  6024. - Fix layout for symbols
  6025. - Fix refresh button
  6026. * Add descriptions for whitelist maps
  6027. * Fix build on arm (#379)
  6028. * Fix issue with the last element in the radix trie
  6029. * Add more tests for radix trie algorithm
  6030. * Allow to extract URLs from query strings of other URLs (#361)
  6031. * Initialize rrd fields before writing to file
  6032. * Fix double free if no password has been specified
  6033. 1.0.1:
  6034. * Add writing to rrd from the controller
  6035. * Fixed lots of bugs in rrd code
  6036. * Adopt new DNS API in hfilter plugin (by @AlexeySa)
  6037. * Allow only one controller process to manage rrd file
  6038. * Set event base for fuzzy calls
  6039. * Improve fuzzy IO errors logging
  6040. * Add rra extraction function to rrd library
  6041. * Add graph handler to the controller
  6042. * Cache correct passwords to avoid too high CPU usage when working with webui
  6043. * Controller sockets are owned by router do not export them to task
  6044. * Optimize logging by skipping hash table search if it's empty
  6045. * Fix loading issue with broken statfiles
  6046. * Print assertions from glib to rspamd logger
  6047. * Load legacy `lua/rspamd.local.lua`
  6048. * Update webui with some fixes to learning and scanning
  6049. 1.0.0:
  6050. * Rework symbols processing:
  6051. - Improve sorting logic for symbols
  6052. - Organize processing into multiple stages
  6053. - Added asynchronous watchers for symbols
  6054. - Added ability to organize dependencies between symbols
  6055. * Fixed URL redirector:
  6056. - Use optimized POE loop
  6057. - Organize dependencies
  6058. - Fix startup
  6059. * New sqlite3 backend:
  6060. - Allow to have per-languages and per-user statistics
  6061. - Allow sqlite3 to be used as statistics backend
  6062. * Store tokenizer configuration within statfiles
  6063. * Improve bayes statistics:
  6064. - Use headers and images metainformation in bayes
  6065. - Suggest using of pre-processed tokens for statistics
  6066. - Fix tokens normalization for OSB algorithm
  6067. * Rewrite url parsing:
  6068. - Fix numerous issues with url extraction and normalization
  6069. - Fix mailto urls
  6070. * Fix settings plugin to allow custom actions scores
  6071. * Improve rbl plugin
  6072. * Allow capturing patterns in rspamd lua regexp library
  6073. * Add GTUBE support
  6074. * Fix spamc legacy support
  6075. * Add DKIM support to RBL module
  6076. * Fix issues with multiple DKIM signatures
  6077. * Fix issue if rspamd cannot create statfiles (#331)
  6078. * Rework parts and task structure:
  6079. - Now text_parts, parts and received are arrays
  6080. - Pre-allocate arrays with some reasonable defaults
  6081. - Use arrays instead of lists in plugins and checks
  6082. - Remove unused fields from task structure
  6083. - Rework mime_foreach callback function
  6084. - Remove deprecated scan_milliseconds field
  6085. * Add ip_score plugin support (not enabled by default):
  6086. - Can check for asn/country and network using DNS lookups
  6087. - Can store and load reputation from redis server
  6088. * Improve PARTS_DIFFER rule to count merely different words
  6089. * New HTML parser:
  6090. - Parses HTML parts using a set of state machines
  6091. - Extracts useful data and exports it to lua functions:
  6092. + Styles
  6093. + Images
  6094. + URLs
  6095. + Colors
  6096. + Structure elements
  6097. - Added HTML rules for some checks
  6098. * New version of LUA DNS API
  6099. * Table versions of many functions in LUA API
  6100. * Improve rspamc client:
  6101. - Print execution time
  6102. - Allow executing of external commands and passing output to them
  6103. - Allow mime output mode when rspamc alters message according to rspamd
  6104. checks and send it to an external command or stdout
  6105. * Allow scanning of local files using HTTP requests
  6106. * Rework configuration system:
  6107. - Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity
  6108. - All modules configurations are now split in $CONFDIR/modules.d/* to
  6109. simplify upgrades
  6110. - Move hfilter to plugins
  6111. - Allow plugins and rules to define default scores to simplify metrics
  6112. setup
  6113. - Include overrides for all modules to honor local/automatic parameters
  6114. - Tune scores for many modules
  6115. * Rework and enable DMARC plugin
  6116. * Add whitelist plugin for SPF/DKIM/DMARC based whitelisting
  6117. * Add some common domains to whitelists shipped with rspamd
  6118. * Rework logging:
  6119. - Now each log entry supports module name and a `tag`. Tag is used to
  6120. identify unique objects (such as tasks) when checking log files
  6121. - It is possible to turn on debugging for the specific modules
  6122. - Systemd logging is fixed
  6123. * Improve spamassassin plugin.
  6124. - Now headers are matched more like SA
  6125. - Improve support of Message-ID
  6126. - Add support of ToCc header type
  6127. - Fix :addr and :name in headers regexps
  6128. * Resurrect rrd support code
  6129. * Save controller stats between restarts
  6130. * Fixed tonns of bugs
  6131. * Added tonns of minor improvements and features
  6132. * Added more unit tests
  6133. * Create functional tests framework
  6134. * Added documentation for missing modules
  6135. * Added rpm/deb repositories and scripts
  6136. * Updated WebUI and libucl externals
  6137. 0.9.10:
  6138. * Do not dereference null pointer on learning.
  6139. * Fix some extreme cases in BAYES.
  6140. * Add a workaround to avoid bad HTML messages breaking.
  6141. * Build with -O2 flags by default.
  6142. * Add constraints to limit DNS requests count per task.
  6143. * Add workaround for SURBL DNS flood.
  6144. * Set error if rspamd cannot learn anything.
  6145. 0.9.9:
  6146. * Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana)
  6147. * Import updated public suffix list (by @fatalbanana)
  6148. * Remove debug message
  6149. * Fix settings (by @fatalbanana)
  6150. * Remove duplicated symbol registration
  6151. * Use WAL for fuzzy storage
  6152. * RBL fixes (by @fatalbanana):
  6153. - silence errors;
  6154. - yield unknown results from RBLs;
  6155. - fix scoring for DNSWL;
  6156. - fix use of RBL name as symbol;
  6157. - ignore RBL names that would not be yielded;
  6158. * Support captures in regular expressions
  6159. * Add captures support to lua_regexp
  6160. * Support dist on FreeBSD and Darwin
  6161. * Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana)
  6162. * Multiple fixes to URL detection:
  6163. - support port definition;
  6164. - fix query and path recognition;
  6165. - fix parsing of multiple slashes in URL;
  6166. - fix parsing query just after port;
  6167. - fix path field in `url:to_table` method;
  6168. - improve support of IP based URLs.
  6169. * Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana)
  6170. * Add GTUBE support
  6171. * Ignore User header in SA mode
  6172. 0.9.8:
  6173. * Fix critical bug in bayes classifier (#305)
  6174. * Fix critical bug in RBL module (by @fatalbanana)
  6175. * Fix and rework settings plugin.
  6176. * Fix get_all_opts for a case of non-iterable options.
  6177. * Use tld for redirector's matching.
  6178. 0.9.7:
  6179. * Add whitelist_exception setting to RBL module (by @fatalbanana)
  6180. * Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by
  6181. @fatalbanana)
  6182. * Fix extreme cases in bayes classifier.
  6183. * Fix parsing of urls with '?' at the end of hostname.
  6184. * Update interface.
  6185. * Fix number of issues with webui interaction.
  6186. * Fix saving maps.
  6187. * Allow user@ and @domain matches in multimap.
  6188. * Fix issues with bounces From processing.
  6189. * Fix abs/fabs misuse.
  6190. * Fix builds on suse and arch linux distributions.
  6191. 0.9.6:
  6192. * Fix memory leak if mime cannot be parsed.
  6193. * Fix dkim cache expiration.
  6194. * Fix issues with redirector HTTP response.
  6195. * Fix abnormal connection closing with certains messages with a high score
  6196. (issue #296)
  6197. * Fix redirector installation.
  6198. * Use specific POE loop for some systems.
  6199. * Fix number of issues in URL redirector.
  6200. * Fix selecting URLs for sending to redirector.
  6201. 0.9.5:
  6202. * Avoid double free when extending HTTP message.
  6203. * Fix double free if multiple classifiers are defined.
  6204. * Fix misprint in spamassassin plugin.
  6205. * Fix cpuid invocation on i386.
  6206. * Fix ownership issues for zero-copy decode.
  6207. * Allow __len metamethod on rspamd{text}.
  6208. * Add base64 decoding lua utility.
  6209. * Fix build on FreeBSD
  6210. * Skip spaces at the beginning of mime messages.
  6211. * DBL_ABUSE_REDIR should not have significant weight.
  6212. * Allow to split by lua_regexp rspamd{text} objects.
  6213. * Allow to specify custom stop pattern for lua_tcp.
  6214. 0.9.4:
  6215. * Fix critical bugs in tokenization algorithm
  6216. * Write unit tests for tokenization
  6217. * Add documentation for lua_tcp
  6218. * Switch off legacy tokenization by default.
  6219. * Fix critical bugs in words normalization
  6220. * Add lua bindings to tokenizer.
  6221. * Implement storing of HTTP headers inside task
  6222. * Add lua API to accerss HTTP headers data
  6223. * Implemented base64 encoding suitable for MIME
  6224. * Use caseless hash and equal functions for HTTP request headers.
  6225. * Improve debian architectures support (by @dottedmag)
  6226. 0.9.3:
  6227. * Revert incorrect regexp change that broke the default rules
  6228. * Fix lua_tcp module
  6229. 0.9.2:
  6230. * Fix error on spawning unique workers.
  6231. * Add preliminary version of generic LUA TCP requests API.
  6232. * Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
  6233. * Fix fuzzy mime strings with only type.
  6234. * Improve thunderbird sanity checks.
  6235. * Fix critical bug on matching regular expressions.
  6236. * Make hiredis optional dependency.
  6237. * Fix multiple bugs in daemon reloading
  6238. 0.9.1:
  6239. * Restore utf8 validation for regular expressions to avoid crashes
  6240. * Fix symbols displaying in the interface
  6241. * Add symbol groups to the interface
  6242. * Fix maps ID parsing in the controller
  6243. * Add multimap and regexp modules documentation
  6244. * Backport fixes from libucl
  6245. * Fix debian package (by @dottedmag)
  6246. * Rework XXH32 invocations
  6247. 0.9.0:
  6248. * Add support of the fast and secure protocol level encryption:
  6249. - curve25519 is used for key exchange;
  6250. - chacha20/poly1305 cryptobox construction for bulk encryption;
  6251. - zero latency overhead;
  6252. - encrypting and balancing HTTP proxy worker
  6253. * Rework expressions and create new expressions library:
  6254. - aggressive optimizations based on the abstract syntax tree;
  6255. - abstract expressions support (regular expressions, functions, lua modules
  6256. composites and so on)
  6257. - New comparison and '+' operators support
  6258. - New greedy algorithm to minimize execution time of expressions and
  6259. all symbols
  6260. - Dynamic expressions benchmark and reoptimizations
  6261. * Many improvements to the LUA API:
  6262. - reworked logger module allowing to do pretty print of the most of lua
  6263. types (including tables and userdata classes)
  6264. - reworked lua redis and lua HTTP to support more features
  6265. - added opaque type for passing large text chunks without copying
  6266. - new regexp module with many auxiliary functions (e.g. `re:split`)
  6267. * LuaJIT is now the default requirement for rspamd allowing to speed up lua
  6268. execution by a large margin (however, plain lua is still supported)
  6269. * New plugins:
  6270. - spamassassin rules plugin that allows to load and re-use the most of
  6271. SA rules natively
  6272. - DMARC plugin that evaluates SPF and DKIM policies to the domain policies
  6273. - many old plugins has been reworked to implement new features and improve
  6274. stability
  6275. * New aho-corasic trie implementation from @mischasan that allows to load and
  6276. use hundreds of thousands of patterns with no influence on load
  6277. * Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
  6278. the performance of regular expressions if supported by PCRE
  6279. * New URLs parser and extractor:
  6280. - removed legacy code that was useless for url finding
  6281. - reworked algorithms of URL parsing for more precise and accurate results
  6282. - added top-level-domains tree from http://publicsuffix.org
  6283. - improved emails parsing
  6284. - removed many phishing false positives due to TLD tree check
  6285. * New statistics infrastructure:
  6286. - created a separate layer of statistic library
  6287. - improved OSB-Bayes by re-weighting tokens according to the original
  6288. academic paper and `crm114` implementation, which reduced false positives
  6289. rate significantly
  6290. - created learn cache to avoid double learning of statistics and providing
  6291. an efficient way to re-learn class for a message
  6292. - created abstract layers for different statistics backends
  6293. - implemented new tokenization algorithms with fast or secure (siphash)
  6294. hashes to generate statistics features
  6295. * Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
  6296. incompatibility with old fuzzy hashes with utf-8 symbols)
  6297. * SPF module has been completely rewritten to support complex cases of
  6298. `include` and `redirect` within SPF records
  6299. * DKIM module now supports multiple signatures
  6300. * Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
  6301. configuration file
  6302. * Many hand-written HTTP clients has been replaced with the common rspamd
  6303. http module
  6304. * New test framework:
  6305. - import lua `telescope` test framework
  6306. - add unit tests for many rspamd modules and routines
  6307. - create a unit test for each possible bug found
  6308. - use luajit ffi for testing C code
  6309. - added preliminary support of functional testing by creating tasks from lua
  6310. * Randomize hash seed to avoid certain hash tables vulnerabilities
  6311. * Documentation improvements:
  6312. - added documentation for the vast majority of rspamd modules
  6313. - added documentation for rspamd protocol
  6314. - added documentation for the most of rspamd LUA extensions
  6315. * Fixed tonns of bugs and memory leaks
  6316. * Added tonns of minor features
  6317. 0.8.3:
  6318. * Various critical fixes in distribution (by @dottedmag and @fatalbanana)
  6319. * Fixed bugs in url detector to parse certain patterns
  6320. * Add default host and helo for a client
  6321. * Some sanity checks for tokenizer and classifier
  6322. * Reiterate on systemd support
  6323. * Fix missing symbol registration
  6324. * Add support of spamc compatible output
  6325. * Filter double-dots in rbl.lua validate_dns (by @fatalbanana)
  6326. * Update ucl submodule due to critical bugfix
  6327. 0.8.2:
  6328. * Create fuzzy db if it does not exist
  6329. * Fix: Centos init script: configtest() (by @AlexeySa)
  6330. * Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana)
  6331. * Update Exim patch (by @fatalbanana)
  6332. * Fix processing of unix sockets.
  6333. * Allow applying settings to authenticated users (by @fatalbanana)
  6334. * Make settings priorities work as documented (by @fatalbanana)
  6335. * Fix race condition in symbols planner
  6336. * Add DNSWL_BLOCKED symbol (by @fatalbanana)
  6337. * Make Exim pass usernames to rspamd (by @fatalbanana)
  6338. * Update RBL module (by @fatalbanana):
  6339. - fix indentation;
  6340. - collapse loops;
  6341. - avoid calling for un-needed information;
  6342. - allow disabling RBLs for authenticated users
  6343. * once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana)
  6344. * hfilter.lua: Add exclusion for authenticated users (by @AlexeySa)
  6345. * Updates to hfilter rules (by @AlexeySa)
  6346. * Set empty <> user or addr for msgs without FROM (by @eneq123)
  6347. * Fix: attempt to index field '?' (a nil value) (by @eneq123)
  6348. * Fix: if not exist Date-header (by @AlexeySa)
  6349. * Add task:get_content() method.
  6350. * rbl.lua: Ignore private IP space (by @fatalbanana)
  6351. * Allow to check radix maps from lua by rspamd{ip}
  6352. * Make local exclusions configurable per-RBL (by @fatalbanana)
  6353. * Add rspamd_config:radix_from_config() (by @fatalbanana)
  6354. * Support emails dnsbl in rbl (by @fatalbanana)
  6355. * Complete rework of url extraction logic
  6356. * Allow customizations for unix sockets. (fixes #182)
  6357. * Set lua path according to rspamd settings.
  6358. * Import lua-functional for plugins stuff.
  6359. * Completely rewrite multimap plugin in functional style.
  6360. * Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186)
  6361. * Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana)
  6362. * Add lowercase utility for utf8 strings.
  6363. * Various fixes to build system
  6364. * Updated debian configuration infrastructure (by @dottedmag)
  6365. 0.8.1:
  6366. * Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana)
  6367. * Remove whitelist.lua from RPM file list (by @fatalbanana)
  6368. * Make Exim pass hostnames to rspamd (by @fatalbanana)
  6369. * Fix building on Fedora (by @fatalbanana)
  6370. * Add toggle for disabling installation of systemd units on Linux (by @fatalbanana)
  6371. * Fix double format rounding that caused output corruption (reported by @fatalbanana)
  6372. * Revert broken change for destructors ordering that led to memory corruption
  6373. * Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev)
  6374. * Fix build on SunOS (by @wiedi)
  6375. * Fix multiple crashes on broken DKIM DNS records
  6376. * Fix critical issue with composites weights removing
  6377. * Fix memory corruption in composites processing code
  6378. * Ignore non-SPF TXT records when parsing SPF includes
  6379. 0.8.0:
  6380. * New fuzzy check logic:
  6381. - use shingles algorithm for fuzzy matching
  6382. - use blake2 instead of md5 for larger output space
  6383. - combine fuzzy and strict matching
  6384. - allow to organize private storages by means of keys
  6385. - preserve compatibility with previous versions
  6386. * New fuzzy storage:
  6387. - use sqlite instead of own memory based hash tables
  6388. - rework commands interface
  6389. - add conversion from the old format
  6390. - add fuzzy match by shignles
  6391. - support old rspamd versions
  6392. * Add lemmatizing for words used in fuzzy hashes that allows to improve match
  6393. quality by using of the first forms of all words
  6394. * Rework language detection
  6395. * Fix several critical bugs, memory leaks and deadlocks:
  6396. - memory leak in HTML nodes parsing
  6397. - deadlock in logger code
  6398. - deadlock in signals processing
  6399. - crashes in fuzzy_storage
  6400. - crashes in tokenizers if the input was empty
  6401. * Import new libucl with several bugfixes and improvements
  6402. * Support listening on ipv6 addresses only
  6403. * Fix macro expansion in SPF module
  6404. * Several bugfixes in DKIM module
  6405. * Add load headers support for mime parts to the lua API
  6406. * Add documentation for:
  6407. - workers in general
  6408. - fuzzy_storage worker
  6409. - fuzzy_check plugin
  6410. - mimepart and textpart lua API modules
  6411. 0.7.6:
  6412. * Apply boundary fix for dkim simple canonization
  6413. * Fix ping command
  6414. * Return nil if header was not found in lua_task
  6415. * Fix hang in upstreams revive logic
  6416. * Decode entitles when normalizing HTML parts
  6417. * Fix logic of finding URLs in HTML parts
  6418. * Do not include \0 into length of text when performing conversion to utf8
  6419. * Fix raw vs parsed reperesentations
  6420. Raw parts are now:
  6421. - decoded b64/qp, but *NOT* converted to utf-8
  6422. Processed parts are now:
  6423. - converted to UTF-8
  6424. - normalized if needed (e.g. HTML tags are stripped)
  6425. * Rework DKIM canonization to line based
  6426. * Fix fuzzy hashes adding
  6427. * Use more specific hash function for fuzzy
  6428. * Fix leaking of iconv descriptors
  6429. * Fix PTR resolving in lua resolver
  6430. * Rework spf module.
  6431. - Copy data to memory pool as cached record might be destroyed causing
  6432. freed memory being passed to the protocol output (use after free)
  6433. - Allow SPF_NEUTRAL policy to be handled separately
  6434. - Add R_SPF_NEUTRAL to the default config
  6435. * Rework `register_symbols` function
  6436. * Allow to disable components of hfilter
  6437. 0.7.5:
  6438. * Fix owner when creating folder /run/rspamd (by @sfirmery)
  6439. * Fix IP validity checks
  6440. * Decode URLs obtained from HTML tags
  6441. * Fix crash with unweighted upstreams
  6442. * Stop processing headers in parts
  6443. * Set sockaddr.sa_family properly when connectig to upstreams
  6444. * Fix reload issues in surbl and fuzzy_check (reported by @citrin)
  6445. * Fix timeouts in redirector
  6446. * Improve lua errors reporting
  6447. * Fix lua closures processing in libucl
  6448. * Rework calling of lua functions from regexp module
  6449. * Choose raw regexp for raw headers
  6450. * Rework conversion to utf since glib one is broken
  6451. * Ignore SGML style tags in html
  6452. * Fix old bug with non-capturing https urls
  6453. * Fix memory corruption on fuzzy reload (reported by @citrin)
  6454. * Fix percents display in rspamc
  6455. * Fix buffer update for DKIM
  6456. * Do not validate utf for raw headers
  6457. 0.7.4:
  6458. * Fix build under *BSD
  6459. * Detect HAN unicode script
  6460. * Implement language detection heuristic for text parts
  6461. * Fix time output in history
  6462. * Improve piechart coloring
  6463. * Fix \r\n conversion in DKIM module (reported by @citrin)
  6464. * Try to detect systems with no IPv6 support
  6465. * Fix multiple/single values in use settings (reported by @citrin)
  6466. * Rework IP addresses in upstreams:
  6467. - Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
  6468. upstreams (since the support of ipv6 is poor in many OSes and
  6469. environments)
  6470. - Free IP list on upstream destruction
  6471. - Add test cases for addresses selection
  6472. - Allow adding of free form IP addresses to upstreams
  6473. * Fix endianness in lua_radix search (reported by @citrin)
  6474. * Soft shutdown should also set wanna_die flag (reported by @citrin)
  6475. * Stop use-after-free in event loop termination
  6476. * Fix processing of very short messages in DKIM (reported by @citrin)
  6477. * Detect systems without shared mutexes
  6478. * Fix issues with PTR and MX elements in SPF parser (reported by @citrin)
  6479. 0.7.3:
  6480. * New upstreams code:
  6481. - simplify upstreams API;
  6482. - unify strings parsing in upstreams definition;
  6483. - add configuration options for the upstreams;
  6484. - for failed upstreams re-resolve their addresses;
  6485. - use all resolved addresses for an upstream (round-robin);
  6486. - implement stable hashing and use it by default for upstreams;
  6487. - add unit test for upstreams module.
  6488. * Rework signals processing in all rspamd workers:
  6489. - signals are now processed in the event loop;
  6490. - implement the most common signal handlers for all workers;
  6491. - add callbacks for workers specific signal handlers
  6492. * Fix critical issue with fuzzy storage:
  6493. Fuzzy stroage could not save any hashes on termination due to bugged
  6494. signals handling
  6495. * Fix roll history IP storage
  6496. * Rework ipv4/ipv6 handling in parsing addresses:
  6497. - turn off support of IPV6_V6ONLY socket option;
  6498. - create ipv6 socket prior to ipv4 one to handle systems with v6/v4
  6499. sockets enabled (Linux)
  6500. * Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
  6501. * Remove nszones.com fake RBL (by @citrin)
  6502. * Fix upstreams interaction for fuzzy_check
  6503. * Verify spf PTR records (reported by @citrin)
  6504. * Fix spf MX records parsing
  6505. * Add compatibility for old libevent (by @yellowman)
  6506. * Sync bugfixes from libucl
  6507. 0.7.2:
  6508. * Convert all maps to the compressed radix trie
  6509. * Allow IPv6 addresses in IP maps
  6510. * Remove dynamic items support from symbols cache
  6511. * Allow hex encoded output of strings
  6512. * Fix bug with control connections count
  6513. * Process fuzzy weight correctly (reported by @fatalbanana)
  6514. * Remove extra reference retain of http connection on error
  6515. * Remove deprecated options from the default config
  6516. * Add `one_shot` attr to metric's symbols
  6517. * Doc: add documentation for metrics
  6518. * Add Upstart job to debian packaging (by @CameronNemo)
  6519. * Config: improve SURBL symbols descriptions (by @citrin)
  6520. * Config: reflect SURBL changes (by @citrin):
  6521. - Outblaze removed, malware moved to separate list:
  6522. http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
  6523. * Fix C modules initialization on restart
  6524. * Treat single IP as a single IP in radix lists (reported by @citrin)
  6525. * Do not touch file and core limits if not asked explicitly (reported by @citrin)
  6526. * Improve logging for fuzzy errors
  6527. * Block SIGPIPE for HTTP writing
  6528. * Doc: update manual pages
  6529. * Fix HTTP connection termination
  6530. * Reduce default number of parallel requests to 8
  6531. * Sync with libucl include features
  6532. 0.7.1:
  6533. * Fix typo in stat output.
  6534. * Fix issues with includes crossing with the system includes
  6535. * Restore testing framework
  6536. * Add radix trie test suite
  6537. * Implement new path-compressed radix trie.
  6538. - The performance benefit over the old algorithm is about 1.5 times.
  6539. - Memory usage is significantly lower as well.
  6540. - Now radix trie can accept any IPv4/IPv6 values
  6541. * Various improvements to the memory pools code
  6542. * Fix writing reply to a client when no filters are defined
  6543. * Write base32 encoded fuzzy
  6544. * Fix 'soft reject' action
  6545. * Fix rspamd reload and modules reconfiguration
  6546. * Fix subject rewriting for the default subject
  6547. * Fix states for processing task and pre-filters
  6548. * Fix issues with connection closing
  6549. * Fix crashes in rdns
  6550. * Fix ratelimit pre-filter
  6551. * Update exim patch.
  6552. - Update to the recent exim version
  6553. - Strip extra leading src/ from the patch
  6554. - Remove sendfile since it was broken
  6555. - Fix rspamd spam report for exim
  6556. * Improve documentation
  6557. 0.7.0:
  6558. * Use HTTP protocol for all operatiosn
  6559. * Webui worker is now removed and controller works as webui
  6560. * Allow to serve static files via controller by option `static_dir`
  6561. * Rspamd interface is now a part of rspamd
  6562. * Rspamc client has been rewritten to use HTTP and non-blocking mode
  6563. allowing to start multiple operations simultaneously (see `-n` option)
  6564. * Lua API was completely reworked to satisfy modern standards of LUA:
  6565. * Module `lua-message` was removed
  6566. * Reduced number of superglobals registered by rspamd
  6567. * Many functions has been redesigned
  6568. * Symbols registration is now more convenient
  6569. * Users settings has been rewritten as lua plugin
  6570. * Reworked headers system as gmime's based one misses many headers and is
  6571. very slow to get headers values
  6572. * Reorganized code and removed embedded jannsson by using UCL for all json
  6573. parsing
  6574. * Migrated to `librdns` for DNS resolving that improves concurrency for
  6575. DNS requests significantly
  6576. * Fixed tonns of bugs in MIME processing
  6577. * Improved metrcis and default symbol's weights
  6578. * Added new RBL's
  6579. * Fixed a number of issues in the modules
  6580. * Removed several memory leaks found
  6581. * Fix unicode processing
  6582. * Fix fuzzy checking for unicode parts
  6583. * Significantly improve documentation and especially LUA API docs
  6584. * See migration notes at https://rspamd.com/doc/migration.html
  6585. 0.6.8:
  6586. * Controller now listen for localhost and not for 127.0.0.1 by default
  6587. * Allow FCrDNS-style RBL lookups (by @fatalbanana)
  6588. * Reduce threshold for parts_differ function.
  6589. * Fix hostname lookup for rdns rbl (by @AlexeySa)
  6590. * Fix HFILTER_URL_ONELINE to reduce false positive rate.
  6591. * Fix whitelist module.
  6592. * Allow override system predefined settings without touching system ones
  6593. by .try_include macro (by @andrejzverev)
  6594. * Check for [ip.address]-style HELO and suppress lookups. (by
  6595. @fatalbanana)
  6596. * Optimize hfilter (by @AlexeySa)
  6597. * Fix issue with random numbers generator in dns.
  6598. * Use more clever time values to setup entropy.
  6599. * Synced with the recent libucl.
  6600. 0.6.7:
  6601. * Use ChaCha20 for DNS generator (more secure DNS id)
  6602. * Unknown symbols now has zero weight and not 1.0
  6603. * Fix fuzzy hashes expire time
  6604. * Fix critical issue in statfiles on FreeBSD 9 (and some other platforms)
  6605. * Add .include_map macro to ucl parser
  6606. * Update libucl
  6607. * Fix headers end detection for DKIM module
  6608. * Fix a bug in received headers parser
  6609. * Validate IP addresses before pushing them to lua
  6610. * Start new documentation project
  6611. * Fixed tonns of other minor bugs
  6612. * Start to prepare for 0.7 with HTTP protocol and new settings
  6613. 0.6.6:
  6614. * Removed issue with BUFSIZ limitation in the controller output
  6615. * Simplify logging symbols escaping
  6616. * Adjusted weights for several rules
  6617. * Improve spamhaus rbl support
  6618. * Removed PBL for received headers checks
  6619. * Added hfilter module that performs various HELO and IP checks.
  6620. * Rspamd can now be reloaded by HUP signal
  6621. * Fuzzy storage should expire hashes properly
  6622. * Build system has been reworked for better supportof pkg-config
  6623. * Various minor bugfixes
  6624. 0.6.5:
  6625. * Fixed critical bug in DNS resolver, introduced in 0.6.4
  6626. * Improved multimap and rbl plugins to skip
  6627. * Add dns_sockets option for tuning sockets per server in DNS resolver
  6628. * Improved packages for rspamd
  6629. 0.6.4:
  6630. * Added io channels for DNS request to balance load and reduce id
  6631. collisions chance
  6632. * Fixed a bug in SPF filter that may cause core dump in specific
  6633. circumstances
  6634. * FIxed default config for rbl module
  6635. * It is possible to get a list of rspamc commands with their descriptions
  6636. * Added SORBS bl to the default config
  6637. * 2tld file for surbl module has been significantly extended
  6638. * Perl modules has been removed from the code.
  6639. * Fixed an issue in libucl when parsing macros
  6640. 0.6.3:
  6641. * Fixed issues with DNS:
  6642. - labels decompression algorithm was fixed;
  6643. - added resolve_mx to lua API;
  6644. - fixed modules that use DNS.
  6645. * Lua modules once_received and emails reworked for new resolver API and UCL.
  6646. * Debian package was polished.
  6647. * Fixed a bug in fuzzy_check module that prevents correct processing messages
  6648. without valid parts.
  6649. 0.6.2:
  6650. * Fuzzy check module has been reworked:
  6651. - now fuzzy_check operates with a group of rules, that define which
  6652. servers sre to be checked;
  6653. - it is possible to specify read_only groups to avoid learning errors;
  6654. - turn fuzzy_check to one_shot mode permanently;
  6655. - fuzzy_check module configuration is now incompatible with the previous
  6656. versions.
  6657. * Imported bugfixes from libucl.
  6658. * Fixed whitelist plugin.
  6659. * Fixed statfiles resizing.
  6660. * Improved logging initialization order.
  6661. * Fixed race condition in the controller worker.
  6662. 0.6.1:
  6663. * Critical bugfixes:
  6664. - fixed build system;
  6665. - fixed in_class setting in bayes learning;
  6666. 0.6.0:
  6667. * Use UCL instead xml for configuration (https://github.com/vstakhov/libucl)
  6668. * Fix statistics module normalization
  6669. * Rework the many modules for the new configuration:
  6670. - surbl has incompatible configuration;
  6671. - fuzzy_check has incompatible configuration;
  6672. - multimap has now new configuration;
  6673. - received_rbl is removed and replaced by rbl module.
  6674. * Removed deprecated options:
  6675. - statfile_pool_size;
  6676. - action and required/reject score for a metric.
  6677. * Simplify build system and unify configuration for all platforms.
  6678. * Improved URL detector (reduced false positive rate).
  6679. * Lua 5.2 is now the default and fully supported version.
  6680. * Tons of bugfixes and minor improvements.
  6681. 0.5.6:
  6682. * Fix bug with counters incrementing in rolling history
  6683. * Detect expl and exp2l as some systems do not have it
  6684. * Support input streams without Content-Length
  6685. * Implement counters output via rspamc and controller interface
  6686. * Fix bug with udp sockets in fuzzy storage
  6687. 0.5.5:
  6688. * New bayes normalizator based on inverse chi-square function
  6689. * Various fixes to fuzzy storage
  6690. * Allow update fuzzy storage only from specific IP addresses
  6691. * Better support of IPv6 and address selection algorithms
  6692. * Add CentOS spec file
  6693. 0.5.4:
  6694. * Fixed issues with diff algorithm
  6695. * Added support of RRD statistics
  6696. * Add webui worker for interface interaction
  6697. * Fix a lot of issues with dynamic conf
  6698. * Fix critical memory leak in settings code
  6699. * Improve stability of the system
  6700. 0.5.3:
  6701. * Added dynamic options
  6702. * Added advanced metaclassfication
  6703. * Added RESTfull API for controller
  6704. * Improved hashing algorithms
  6705. * Various fixes for rspamc client:
  6706. - allow interacting with unix sockets
  6707. - librspamdclient major cleanup
  6708. - bayes is now default classifier
  6709. 0.5.2:
  6710. * Added lua bindings for:
  6711. - basic mime parts, that allows checkign attachements for example;
  6712. - DNS resolver;
  6713. * Existing lua bindings now works without task object allowing to use them
  6714. in custom code.
  6715. * Threads system was reworked to avoid global lua interpreter lock.
  6716. * DKIM module now converts all line endings to CRLF how opendkim does.
  6717. * URL detector is now more accurate for text parts.
  6718. * Several critical bugs and memory leaks were fixed.
  6719. 0.5.1:
  6720. * Added lua worker type to handle network connections in lua.
  6721. * Added lua bindings for async_session, IO dispatcher, memory_pool, and
  6722. worker.
  6723. * Composites can now uses other composites in expressions.
  6724. * Fixes for debian package and for FreeBSD >= 9.1
  6725. * Add support of gmime-2.6 if gmime-2.4 is not found.
  6726. * Improve url detection and phishing detection.
  6727. * Add lua mime_part library to get an access to all message part
  6728. attributes (like filename, length, type).
  6729. 0.5.0:
  6730. * Added SMTP lightweight balancing proxy with XCLIENT support.
  6731. * Added lua bindings for upstreams objects and API.
  6732. * New pre-filters are implemented to support initial checking for messages.
  6733. * Added ratelimit plugin that uses redis protocol to store data.
  6734. * Added ipv6 support to spf and some other modules.
  6735. * Unbreak spf plugin.
  6736. * Allow options with the same name be threated as list.
  6737. * DKIM plugin an parsing code was added.
  6738. * Separate build system to put logic in several shared libraries.
  6739. * Many bugfixes.
  6740. 0.3.2:
  6741. * Add error handling for regexps
  6742. * Fix quit command in controller interface
  6743. * Write symbols weights to rspamc output
  6744. * Improve logic of selecting rspamc version
  6745. * Do not try to parse broken DNS replies
  6746. * Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin)
  6747. * Output message id in rspamc reply
  6748. * Fix inserting composite symbol
  6749. * Fix output of log line
  6750. * Document composites
  6751. * Add logging for fuzzy checks
  6752. * Add logging for learning
  6753. * Improve logic of learning messages - do not learn more than specific threshold
  6754. * Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file
  6755. * Do not output control characters if output is not terminal
  6756. * Fix some logic errors in learning
  6757. * Consider lua plugins errors as fatal configuration errors
  6758. * Fix writing message id during fuzzy_add command
  6759. * Display weight of symbols correctly
  6760. * Fixes to winnow learning
  6761. * One more try to improve accuracy of winnow algorithm
  6762. * Add bayesian classifier (initial version)
  6763. * Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1
  6764. * Some fixes to fuzzy hashes expiration:
  6765. * Fix assertion while look up value in NULL hash (found by cirtin)
  6766. * Fix normalization for systems that have not tanhl function
  6767. * Ignore rfc822 group addresses
  6768. * Move images library to core rspamd
  6769. * Add lua api to access images properties
  6770. * Add post filters to lua API - filters that would be called after all message's processing
  6771. * Add ability to check for specified symbol in task results from lua
  6772. * Add ability to check for metric's results from lua
  6773. * Add ability to learn specified statfile form lua
  6774. * Add ability to extract filename and size of images from lua
  6775. * Fix assertion while extracting internet address
  6776. * Fix races in fuzzy storage
  6777. * Make spf parser case insensitive
  6778. * Add ability to check hashes of selected mime types
  6779. * Add ability to set minimum size in bytes for mime types
  6780. * Add ability to set minimum dimensions for images
  6781. * Assume all text/* content types as text/plain
  6782. * Fix getting data wrapper for gmime24
  6783. * Many fixes to statfile synchronization system
  6784. * Fixed statfile pool initialization and synchronization with disk
  6785. * Prepare 0.3.2
  6786. * Fix Mail::Rspamd::Config for new rspamd features
  6787. * Use Mail::Rspamd::Config in rspamc client
  6788. * Write user's name to rspamd log
  6789. * Prepare rspamd build infrastructure for creating rpm and deb packages
  6790. * Fix depends
  6791. * Add start script for linux systems
  6792. * Fix shared usage of statfiles
  6793. * Add invalidation of statfiles in case of learning, so now statfiles
  6794. * This should fix shared usage of statfile pool by several processes
  6795. * Fix misprint (reported by az)
  6796. * Fix stupid error when all checks can be done in a single pass
  6797. * New trie based url scanner (based on libcamel)
  6798. * Small fixes to rspamd perl client
  6799. * Write fuzzy hashes info to log
  6800. * Add trie interface to lua api
  6801. * Explain sample config and cleanup it
  6802. 0.3.1:
  6803. * Add modules documentation
  6804. * Continue implementing smtp proxy
  6805. * Implement new learning system, now rspamd should be much more intelligent while learning messages
  6806. * Convert statistic sums to use long double for counters
  6807. * Use hyperbolic tangent for internal normalizer
  6808. * In classify normalize result after comparing, not before
  6809. * New symbols sorter
  6810. * Fix strict aliasing while compiling with optimization
  6811. * Fix tanhl detection for platforms that have not implementation of it
  6812. * Remove several compile warnings
  6813. * Add experimental support of dynamic rules to regexp module
  6814. * Document views configuration
  6815. * Several fixes to documentation
  6816. * Add more logic for dynamic rules
  6817. * Add documentation for dynamic rules
  6818. * Add ability to make negations in networks in dynamic rules
  6819. * Clean up cache items correctly
  6820. * Implement basic SMTP dialog:
  6821. * Implement interaction with smtp upstream (with support of XCLIENT)
  6822. * Check messages received via smtp proxy
  6823. * Add support for sendfile in io dispatcher
  6824. * Fix issues with compatibility of worker_task and smtp proxy
  6825. * Proxy DATA command
  6826. * Fix SMTP
  6827. * Change metric logic
  6828. * Completely remove lex/yacc readers for config
  6829. * Make common sense of metric/action and symbols
  6830. * Sync changes with all plugins
  6831. * Incorrectly removed in previous commit
  6832. * Fix misprint (by Andrej Zverev)
  6833. * announce the "password" keyword in usage list
  6834. * Implement initial version of greylisting triplets storage
  6835. * Fix issues with smtp worker
  6836. * Fix QUIT command in SMTP worker
  6837. * Some fixes about new metrics system (may be incomplete)
  6838. * Get weights of symbol from default metric for symbols cache
  6839. * Fix setting task->from/task->rctp in smtp client
  6840. * Copy from and rcpt correctly
  6841. * Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed)
  6842. * Fix smtp data input
  6843. * Fix misprint
  6844. * Add limit of maximum allowed smtp session errors
  6845. * New logic of SURBL module:
  6846. * Use system mkstemp(3) on systems where it is available as glib implementation
  6847. * Try to fix memmove issues in io dispatcher
  6848. * Remove debug from SURBL module
  6849. * Rewrite buffered input for line policy (again)
  6850. * Fix issue with links that are ip addresses in numeric form in surbl
  6851. * On Darwin use BSD style sendfile definition
  6852. * Reorganize platform specific knobs in CMakeLists
  6853. * Use gettimeofday on systems that have not clock_getres
  6854. * Use ftime for dns trans id generation on systems without clock_getres
  6855. * Darwin sendfile(2) support
  6856. * TIMEDB->TIMEB
  6857. * More to previous commit
  6858. * Pass env from main() arguments instead of platform specific global environ
  6859. * Fix compatibility issues
  6860. * Fix -lintl detection
  6861. * Init some variables to avoid problems
  6862. * Remove garbadge (gnome terminal sucks)
  6863. * Add more information about why we drop smtp connection
  6864. * Fix mkstemp call
  6865. * Send to upstream QUIT command at the end of session
  6866. * Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early
  6867. * Fix states in smtp dialog
  6868. * Use rspamd_snprintf instead of libc one
  6869. * Fix URLS command
  6870. * Fix reconfigure process of surbl module
  6871. * Fix destroying smtp session (unmap memory and do not delete pool early)
  6872. * Delete pool after using its variables
  6873. * Delay timer must be registered in async session to correctly handle connection termination
  6874. * Register dns requests in session too
  6875. * Make session before registering events
  6876. * Remove events in handlers
  6877. * Add ability to set filters for smtp worker for each smtp stage
  6878. * Add very initial version of DNS resolver (many things to be done)
  6879. * Announce weights and sync
  6880. * Fix few typo
  6881. * Understand short names of facility in logging config
  6882. * Add ability to make whitelist for spf checks
  6883. * Misprint != -> ==
  6884. * Handle lua tag in way that it is not required to write additional text:
  6885. * Strip all starting whitespace symbols from xml texts
  6886. * Fix stupid bug in calculating buffer length while reading file maps
  6887. * Add resolv.conf parsing into dns.c
  6888. * Fix microseconds<->milliseconds conversions
  6889. * Take callback argument in Mail::Rspamd::Client for processing files and directories
  6890. * Print results if rspamc is called for a directory
  6891. * Fix stupid error with surbl module reconfig (another one, blame me)
  6892. * Do not show duplicate urls in url header
  6893. * Fix detection of numeric urls (reported by citrin)
  6894. * Write real time of message's scan to log (not only virtual)
  6895. * Fix chartable module in utf mode
  6896. * Fix parsing of some broken urls
  6897. * Add ability to test regexp with 'T' flag
  6898. * Write more code for DNS resolver:
  6899. * Make DNS resolver working
  6900. * Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality
  6901. * Write DNS resolver tests
  6902. * Fix issues with memory_pool mutexes and with creating of statfiles
  6903. * Forgotten in previous commit
  6904. * Add support for parsing SPF and SRV records
  6905. * Fix PTR parsing
  6906. * Add tests
  6907. * Make SURBL module to use rspamd dns module
  6908. * Several fixes to DNS logic
  6909. * Remove evdns and use only rspamd resolver
  6910. * Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table
  6911. * Fix resolving in smtp module
  6912. * Init events before configuring resolver in smtp worker
  6913. * Set resolver inside task
  6914. * Fix reload signal (reported by citrin)
  6915. * Some improvements to redirector
  6916. * Call has_forked method to inform POE about fork
  6917. * Fix lua DNS code
  6918. * Decompress labels in DNS packets more strictly
  6919. * Fix some problems with TXT records
  6920. * Try to fix removing of DNS events
  6921. * Do not insert unparsed RR's into reply
  6922. * Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action
  6923. * Fix check_smtp_data function
  6924. * Add ability to make views by recipient
  6925. * Add ability to set metric's action from config file
  6926. * Fix bug with writing garbadge if message has no urls or no messages
  6927. * Fix bug with incorrect behaviour of compare_parts_distance function
  6928. * Add ability to assign several actions to one metric
  6929. * Report action in rspamc protocol
  6930. * Mail::Rspamd::Client and rspamc can now understand Action header too
  6931. * Write action to log as well
  6932. * Make valgrind happy about comparing symbols
  6933. * Add more debug to comparing parts distance function
  6934. * Write action even if message has no symbols
  6935. * Make improvements to HTML entities decoder: now it replaces entities with common characters and
  6936. * Add -d option to force debug output
  6937. * Assume 7bit as default transfer encoding
  6938. * Do not overwrite lua plugins and configs if they already exists in target directory
  6939. * Improve logging
  6940. * Write queue id to log
  6941. * Remove test messages from cmake
  6942. * Reopen log file by USR1 signal
  6943. * Add reopenlog method to FreeBSD rc script
  6944. * Adopt foreach for cmake 2.6
  6945. * Fix to rc script
  6946. * Do not try to resolve names with several dots in a row
  6947. * Fix surbl request formatting for ip addresses
  6948. * Handle cases of broken requests
  6949. * Fix problems with parsing compressed names
  6950. * Fix TXT records parsing
  6951. * Fix expanding spf macros that may fail in rare cases
  6952. * Fix another error with early task destroying
  6953. * Handle empty from header
  6954. * Improve reopenlog command in rc script
  6955. * Strip trailing whitespace characters in Mail::Rspamd::Client
  6956. * Use ungreedy match to strip trailing whitespaces
  6957. * Stupid error in calculation compressed label length
  6958. * Some optimizations to client library
  6959. * Do not compare empty parts
  6960. * Empty and non-empty parts are always different
  6961. * Save in regexp cache the whole regexp with header name (if exists) and with flags
  6962. * Add rspamd_log variable to lua plugins to access logging functions
  6963. * Each part in rspamd task now can have parent part
  6964. * Check for parts distance only for multipart/alternative subparts
  6965. * Do not check attachements even if they are text (but attached as file)
  6966. * Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second)
  6967. * Add ability to turn on debug for specific symbols
  6968. * Add ability to configure dns timeouts and dns retransmits in config file
  6969. * More debug
  6970. * Fix extracting arguments in lua logger interface
  6971. * Turn off debug_ip during reload if it was disabled by new config
  6972. * Improve lua logging
  6973. * Pre-init symbols cache when rereading config
  6974. * Fix lua representing of invalid ip (nil, not 255.255.255.255)
  6975. * Fix R_TO_SEEMS_AUTO rule (by citrin)
  6976. * Add multimap lua plugin
  6977. * Fix some multimap issues
  6978. * Try to save images hashes to fuzzy storage to stop some annoying spammers
  6979. * Allocate some more bytes for read buffer to avoid incorrect behavoiur
  6980. * Add ability to check dns black lists by multimap module
  6981. * Add multimap documentation
  6982. * Fix labels parsing
  6983. * Another try to save regexps in cache correctly
  6984. * Improve test logs for regexps
  6985. * Fix parsing txt records to avoid reading of uninitialized data
  6986. * Fix error with writing symbols cache file
  6987. * Fix error while working in utf mode when raw regexps was not created properly
  6988. * Do not add extra byte while converting text to utf
  6989. * Add error handling for regexps