Commit graph

52 commits

Author SHA1 Message Date
Christian Hesse
e2d3f0f073 fw-addr-lists: delay on possible scripting subsystem crash
This happens in :convert when a list is way too large.

Let's use $LogPrintOnce here. If the scripting subsystem really crashes
the message will be purged from $LogPrintOnceMessages anyway (as all
global variables are lost).
2025-06-11 17:46:09 +02:00
Christian Hesse
ce129ee441 fw-addr-lists: for IPv6 the CIDR is always expected 2025-05-22 10:13:52 +02:00
Christian Hesse
b807fc9e90 fw-addr-lists: adopt changes in wording...
... to match the changes from previous commit.
2025-05-21 22:12:10 +02:00
Christian Hesse
142b0760b0 fw-addr-lists: :do ... on-error=... -> :onerror ... do=... 2025-05-21 22:05:35 +02:00
Christian Hesse
9e70bca30b fw-addr-lists: check for global config to be ready 2025-05-08 09:51:08 +02:00
Christian Hesse
4b92181f2e fw-addr-lists: fail if global functions do not become ready 2025-05-08 09:51:07 +02:00
Christian Hesse
0e93d8ca66 fw-addr-lists: use :onerror for outer block 2025-05-06 09:54:42 +02:00
Christian Hesse
f0e6cbcfe1 fw-addr-lists: get branch from calculated checksum
The addresses were spread very uneven before.

Let's calculate a checksum, and take the first two characters of that.
The addresses are now spread evenly on 256 branches (0x00 to 0xff).
2025-03-11 15:51:10 +01:00
Christian Hesse
d71ea804b0 fw-addr-lists: two characters for branch
Using one character for IPv4 is ok (1 to 9), but IPv6 global unicase
(2000::/3) results in just two different characters (2 and 3).

So let's use first two characters...
2025-03-11 14:20:50 +01:00
Christian Hesse
e148df9e57 fw-addr-lists: put addresses into "branches"...
... effectively adding another layer and some complexity, but:
The addresses are sorted inside the array, and sorting less addresses in
a branch saves a lot of processing power. So this is a lot faster now...
2025-03-11 14:20:44 +01:00
Christian Hesse
2f55bfaf00 fw-addr-lists: strip cidr for host addresses
This makes sure the addresses match later when we read them from
address-list for renew.
2025-03-11 14:19:10 +01:00
Christian Hesse
ea6de35699 fw-addr-lists: do not clean up
Cleanup is important on renew (so the script does not attempt to re-add),
but we do not care here.
2025-03-11 14:13:55 +01:00
Christian Hesse
fb343c99e3 fw-addr-lists: put timeout into variable 2025-03-06 22:59:31 +01:00
Christian Hesse
6d718ec987 fw-addr-lists: use $LogPrintVerbose ...
... to reduce debug output and speed up execution.
2025-03-06 15:59:44 +01:00
Christian Hesse
39c5aeda13 fw-addr-lists: use short url rsc.eworm.de 2025-01-29 14:14:26 +01:00
Christian Hesse
9e3729c279 update copyright for 2025 2025-01-02 00:04:06 +01:00
Christian Hesse
a7878d664f fw-addr-lists: do not fail on invalid json data 2024-12-17 13:21:19 +01:00
Christian Hesse
923a6385bf fw-addr-lists: use $ExitError to indicate unintentional error 2024-12-09 11:49:50 +01:00
Christian Hesse
794525b706 fw-addr-lists: require RouterOS 7.16
Actually the requirement bumped with the change in commit:

* 084c246ef0
  fw-addr-lists: simplify looping lines
2024-11-27 10:00:27 +01:00
Christian Hesse
084c246ef0 fw-addr-lists: simplify looping lines
With `:deserialize` the **record** separator is always a new line. The
property `delimiter=` is a **field** reparator, so you can parse a lines
into an array.

We do not want (or need) that, so use new line as field separator. This
will result in an array with just one element, and we use that.

Also convert the data to line feed explicitly, just to be sure.
2024-11-06 22:42:40 +01:00
Christian Hesse
d23d05f2ea fw-addr-lists: handle JSON format from spamhaus.org
Closes: https://github.com/eworm-de/routeros-scripts/issues/79
2024-09-11 10:48:45 +02:00
Christian Hesse
6fbafe76ba bump RouterOS requirement for all scripts and modules...
... now that global-functions requires RouterOS 7.14 anyway.
2024-07-16 13:50:22 +02:00
Christian Hesse
c4d2ea19dd fw-addr-lists: break long lines 2024-04-07 22:51:24 +02:00
Christian Hesse
19c52ed2a7 fw-addr-lists: show count of active addresses 2024-04-07 22:51:24 +02:00
Christian Hesse
491d85000d fw-addr-lists: human readable numbers for counts 2024-04-07 22:45:54 +02:00
Christian Hesse
70cc1ff53b fw-addr-lists: rework log messages, always include list name 2024-04-07 22:45:34 +02:00
Christian Hesse
af6556bdba fw-addr-lists: add debug message on successful download 2024-04-07 22:44:34 +02:00
Christian Hesse
4b6d0c02f1 fw-addr-lists: try with less regexp matches 2024-04-07 22:21:10 +02:00
Christian Hesse
c3809f240d fw-addr-lists: use $FetchHuge 2024-04-07 22:21:10 +02:00
Christian Hesse
4df1468e25 global-functions: rename $FetchUserAgent -> $FetchUserAgentStr
... to make sure the function does not clash with the variable we had before,
as this causes issue with news and changes notification.
2024-03-29 11:11:17 +01:00
Christian Hesse
9ec8b9c03f fw-addr-lists: one more interation on download failure...
... with even more delay. This script is called with long interval from
scheduler, so should not be an issue.
2024-03-26 08:37:07 +01:00
Christian Hesse
eb7919c1d8 global-functions: generate user agent string in function...
... and add the caller and my shortened url.
2024-03-16 23:10:47 +01:00
Christian Hesse
6715696ba1 fw-addr-lists: switch to $LogPrint 2024-03-12 20:37:57 +01:00
Christian Hesse
19802c0b69 global-functions: $LogPrintOnce: drop support for exit
This is not widely adopted or used, so let's just drop it - no
compatibility.
2024-03-12 20:37:57 +01:00
Christian Hesse
c1362f54e5 fw-addr-lists: drop main function, use :do with on-error 2024-03-12 15:27:15 +01:00
Christian Hesse
1e8918fdaa global-functions: $ScriptLock: do not exit from global function 2024-03-12 15:27:15 +01:00
Christian Hesse
480ad0c196 fw-addr-lists: move code into function 2024-03-04 17:04:45 +01:00
Christian Hesse
62790ae091 fw-addr-lists: add support for IPv6 2024-02-29 14:21:58 +01:00
Christian Hesse
0125f102b4 fw-addr-lists: rename variable 2024-02-29 14:21:58 +01:00
Christian Hesse
31966479dc fw-addr-lists: update wording 2024-02-29 14:21:58 +01:00
Christian Hesse
1687e2780f fw-addr-lists: get timeout from loop 2024-02-29 14:21:58 +01:00
Christian Hesse
afd779c0bb fw-addr-lists: use :jobname to get script name 2024-01-30 00:52:21 +01:00
Christian Hesse
4b8854946d fw-addr-lists: use prepared user-agent string with fetch 2024-01-19 13:25:22 +01:00
Christian Hesse
a7619a5119 global-functions: $LogPrintOnce: support exit 2024-01-18 10:19:22 +01:00
Christian Hesse
9a73fc526f update copyright for 2024 2024-01-01 15:25:25 +01:00
Christian Hesse
f9528f0ac5 fw-addr-lists: warn on possible truncation
... as fetch truncates data at about 64kB, reported in SUP-132297.
2023-12-13 15:57:28 +01:00
Christian Hesse
aba4770395 fw-addr-lists: support timeout per list
This works with something like this:

    :global FwAddrLists {
      "allow"={
        { url="https://eworm.de/ros/fw-addr-lists/allow";
          cert="E1"; timeout=1w };
      };
      ...
    }

All urls for one named list should have the same timeout! With different
timeout values and identical addresses the behavior is besically undefined,
depending on order.
2023-11-30 13:51:57 +01:00
Christian Hesse
d4e5194a65 fw-addr-lists: rework the retry logic
... with increasing delay.
2023-10-19 21:17:03 +02:00
Christian Hesse
50429a0ad8 fw-addr-lists: no (mixed) capitalization in message 2023-09-06 11:35:02 +02:00
Christian Hesse
eb3b4d169e fw-addr-lists: retry to download on failure 2023-06-23 09:49:34 +02:00