Просмотр исходного кода

Normalize blacklist IP parsing and improve sorting (#5219)

Improve IP extraction and normalization in the blacklist generation script.

Update the grep pattern from ^(?:… to (?:… so it no longer requires the IP address to start at the beginning of the line, allowing matches embedded in files or lines with leading characters.

Enhance IPv4 normalization by stripping leading zeros from each octet while preserving any CIDR suffix using an updated sed expression.

Switch from sed -r to sed -E for better portability across systems.

Finally, replace the slower sort -n | sort -mu pipeline with a single sort -Vu, which correctly sorts dotted IPv4 addresses and removes duplicates in one pass.

Fixes #5213
sahsanu 1 месяц назад
Родитель
Сommit
b4bd2dbce0
1 измененных файлов с 2 добавлено и 2 удалено
  1. 2 2
      install/common/firewall/ipset/blacklist.sh

+ 2 - 2
install/common/firewall/ipset/blacklist.sh

@@ -22,7 +22,7 @@ for i in "${BLACKLISTS[@]}"; do
 	IP_TMP=$(mktemp)
 	((HTTP_RC = $(curl -L --connect-timeout 10 --max-time 10 -o "$IP_TMP" -s -w "%{http_code}" "$i")))
 	if ((HTTP_RC == 200 || HTTP_RC == 302 || HTTP_RC == 0)); then # "0" because file:/// returns 000
-		command grep -Po '^(?:\d{1,3}\.){3}\d{1,3}(?:/\d{1,2})?' "$IP_TMP" | sed -r 's/^0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)$/\1.\2.\3.\4/' >> "$IP_BLACKLIST_TMP"
+		command grep -Po '(?:\d{1,3}\.){3}\d{1,3}(?:/\d{1,2})?' "$IP_TMP" | sed -E 's/^0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)(.*)$/\1.\2.\3.\4\5/' >> "$IP_BLACKLIST_TMP"
 	elif ((HTTP_RC == 503)); then
 		echo >&2 -e "\\nUnavailable (${HTTP_RC}): $i"
 	else
@@ -31,5 +31,5 @@ for i in "${BLACKLISTS[@]}"; do
 	rm -f "$IP_TMP"
 done
 
-sed -r -e '/^(0\.0\.0\.0|10\.|127\.|172\.1[6-9]\.|172\.2[0-9]\.|172\.3[0-1]\.|192\.168\.|22[4-9]\.|23[0-9]\.)/d' "$IP_BLACKLIST_TMP" | sort -n | sort -mu
+sed -E -e '/^(0\.0\.0\.0|10\.|127\.|172\.1[6-9]\.|172\.2[0-9]\.|172\.3[0-1]\.|192\.168\.|22[4-9]\.|23[0-9]\.)/d' "$IP_BLACKLIST_TMP" | sort -Vu
 rm -f "$IP_BLACKLIST_TMP"