feat: add backend, admin console and live monitoring
Integrate tunasync management, SQLite authentication and content settings, real monitoring and per-repository pull speeds. Add Chinese-only announcements and mirror help, editable related links, admin layout updates, deployment tools and tests.
This commit is contained in:
@@ -0,0 +1,100 @@
|
||||
# MirrorOUC 部署
|
||||
|
||||
当前预览:`http://100.69.2.2:8080`,后台:`/admin`。这是单 Worker 适配版本,使用服务器现有的 tunasync Manager、Worker 和并行 rsync 配置,不重新安装或替换同步引擎。
|
||||
|
||||
## 运行结构
|
||||
|
||||
| 服务 | 位置与用途 |
|
||||
| --- | --- |
|
||||
| Nginx 预览 | 只监听 `100.69.2.2:8080`;原有 80 端口站点保持原配置 |
|
||||
| mirrorouc.service | `mirrorouc` 普通用户,API 与静态文件监听 `127.0.0.1:8081` |
|
||||
| mirrorouc-config.service | root 受限配置写入器,Unix socket 只允许 `mirrorouc` 组访问 |
|
||||
| mirrorouc-metrics.timer | 每 10 秒采集 Linux CPU/内存、NFS `/data` I/O 和磁盘容量 |
|
||||
| mirrorouc-counters.service | 添加独立 nftables 计数表,只计数,不修改已有规则、不丢弃数据包 |
|
||||
| mirrorouc-speeds.service | 每约 2 秒读取 TCP 接收计数,按仓库汇总并行连接 |
|
||||
| tunasync | 继续使用已有 Manager `127.0.0.1:12345` 与 Worker `srv-software-mirror` |
|
||||
|
||||
代码发布目录 `/opt/mirrorouc/releases/<release>`;`/opt/mirrorouc/current` 指向当前版本。业务数据库和上传图片保存在本地磁盘 `/var/lib/mirrorouc/`,不要放在 NFS 上。监控历史位于其 `metrics/history.json`,保留最近 24 小时;刚启动时没有完整 24 小时历史。
|
||||
|
||||
## 首次安装/更新
|
||||
|
||||
本地 Node.js 22.18+ 与 Go 1.27+:
|
||||
|
||||
```sh
|
||||
npm ci
|
||||
npm run check:i18n
|
||||
npm run check:monitoring
|
||||
npm run build
|
||||
cd backend
|
||||
go test ./...
|
||||
go vet ./...
|
||||
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -o ../mirrorouc .
|
||||
```
|
||||
|
||||
把 `mirrorouc`、`dist/` 和 `deploy/` 放进新的 `/opt/mirrorouc/releases/<release>/`,确认二进制可执行。在 Debian 服务器上执行:
|
||||
|
||||
```sh
|
||||
python3 /opt/mirrorouc/releases/<release>/deploy/test_config_helper.py
|
||||
python3 /opt/mirrorouc/releases/<release>/deploy/test_job_speeds.py
|
||||
sh /opt/mirrorouc/releases/<release>/deploy/install-metrics.sh
|
||||
sh /opt/mirrorouc/releases/<release>/deploy/install-preview.sh /opt/mirrorouc/releases/<release>
|
||||
systemctl status mirrorouc mirrorouc-config mirrorouc-metrics.timer
|
||||
curl http://127.0.0.1:8081/healthz
|
||||
nginx -t
|
||||
```
|
||||
|
||||
安装脚本针对当前服务器路径。迁移到其他服务器时,应先调整 `nginx-preview.conf`、环境文件、配置写入器的固定路径与 Manager 地址。脚本不会清理旧发布目录或删除镜像文件。
|
||||
|
||||
首次安装会创建管理员 `admin`,随机密码写入 root 才能读取的 `/etc/mirrorouc/initial-admin-password`。本次部署的本地凭据文件为被 Git 忽略的 `.local/deployment/admin-credentials.json`。后台「账户」可修改当前密码,修改后所有旧会话失效。此密码文件不会跟随密码修改而更新。
|
||||
|
||||
环境配置文件 `/etc/mirrorouc/mirrorouc.env` 不随发布覆盖,权限 0600。正式启用 HTTPS 后将 `MIRROROUC_SECURE_COOKIES=true` 并重启服务;当前 Tailnet 预览通过 HTTP 访问,故为 false。登录 Cookie 为 HttpOnly、SameSite=Strict,有效期 12 小时;写入接口另需会话 CSRF Token 与同源请求。
|
||||
|
||||
## 配置发布和权限
|
||||
|
||||
Web 服务不能直接写 `/etc/tunasync`。配置写入器固定只接受 `/etc/tunasync/worker.conf` 和 `/etc/tunasync/mirrors.d/*.conf`:
|
||||
|
||||
- 全局只可修改 concurrent、interval、retry、timeout。
|
||||
- 已有镜像只可修改 upstream、interval、retry、timeout。自定义 command、rsync_options、provider 等字段必须保留。
|
||||
- 新增只支持 rsync/two-stage-rsync,文件名固定为 `mirrorouc-<name>.conf`;不接受任意 Shell 命令。
|
||||
- 编辑前后均验证 TOML、数值、上游地址和文件内容哈希。产生冲突时拒绝覆盖。
|
||||
- 先生成可阅读的完整文件前后预览,再由管理员确认应用;应用失败恢复原文件并尝试重新加载。
|
||||
- SQLite 保留变更版本与审计记录;应用过的变更可以恢复至变更前,若文件已被其他修改覆盖则拒绝恢复。
|
||||
- 写入器保留待完成事务日志 `/var/lib/mirrorouc-config/pending.json`;若进程在写入中退出,下次启动恢复原文件。若恢复失败则拒绝继续写入,需要运维检查该文件与 Worker 状态。
|
||||
|
||||
Manager 返回成功表示 reload 命令已提交,并不等于 Worker 已完成重载,需查看 Worker 服务和任务状态。已有仓库禁用后用「开始同步」重新启动;tunasync 不提供单独的 enable 命令。
|
||||
|
||||
## 监控口径
|
||||
|
||||
HTTPS/rsync 使用 TCP 443/873 的 IPv4、IPv6 收发字节计数,双向合计,排除回环接口。8080/80 端口的 HTTP 预览流量不计入 HTTPS,也不展示 HTTP 曲线。没有经过这些端口的流量不会被推算进对应协议。没有有效采样时显示等待采集。
|
||||
|
||||
`/status` 同步明细的「拉取速度」来自 Linux TCP_INFO 的 `bytes_received`,通过 `ss -tinpeH` 读取,关联 `tunasync-worker.service` 的进程与仓库目标路径,并追溯 git/ssh 等辅助进程的父进程。多个 rsync 子进程共享同一个 socket 时按 inode/socket cookie 去重,多个并行连接求和;约每 2 秒采样并刷新,单位 B/s、KB/s、MB/s、GB/s。只计算 TCP 接收,不把上传量、NFS I/O 或磁盘写入速度当作拉取速度。
|
||||
|
||||
这是采样窗口的接收速率,采样间建立并关闭的短连接可能漏记,不作为精确流量计费数据。首次采样不使用连接生存期累计值冒充瞬时速度。仅同步中的仓库显示数值;有效采样中的零表示当时没有收到数据,未同步/采样失败/数据过期时显示「—」。数据超过 15 秒即失效。相关原始字段见 [Linux TCP_INFO](https://github.com/torvalds/linux/blob/master/include/uapi/linux/tcp.h) 和 [iproute2 ss](https://github.com/iproute2/iproute2/blob/main/misc/ss.c)。
|
||||
|
||||
NFS I/O 来自 `/proc/self/mountstats` 的 `/data` 挂载,磁盘容量来自 statvfs;CPU、内存来自 `/proc`。采样历史仅覆盖采集器启用后的时间,不补造旧数据。图表最多展示 241 个采样点,均值、当前值、峰值从当前窗口全部原始有效采样计算。
|
||||
|
||||
镜像文件由 Nginx 直接提供,API 不读取整个仓库;镜像目录响应设置 CSP sandbox 和 nosniff,避免仓库中的 HTML/脚本获得管理页面同源权限。
|
||||
|
||||
## 备份与回退
|
||||
|
||||
日常备份包括 SQLite、`uploads/`、`/etc/mirrorouc/` 和 `/etc/tunasync/`。数据库运行时使用 SQLite backup API 或 `sqlite3 ... '.backup ...'`;直接复制数据库文件前先停止 `mirrorouc.service`,同时处理 WAL 文件。备份文件包含账户哈希、会话和可能带内部地址的配置,应限制读取权限。
|
||||
|
||||
应用版本回退:将 `/opt/mirrorouc/current` 指回旧发布目录,再 `systemctl restart mirrorouc`。本次新增数据库表使用 `CREATE TABLE IF NOT EXISTS`;未来若出现不兼容数据库迁移,需要同时恢复对应数据库备份。
|
||||
|
||||
只撤下预览时,删除 `/etc/nginx/sites-enabled/mirrorouc-preview`,通过 `nginx -t` 后 reload Nginx,并停止 `mirrorouc.service` 与 `mirrorouc-config.service`。这不会修改原有 80 端口镜像站点。停止新增采集器不会影响 tunasync;若需要删除计数表,只删除 `inet mirrorouc_monitor`,不要 flush 全局 nftables 规则。
|
||||
|
||||
## 验证范围
|
||||
|
||||
`backend/*_test.go` 使用临时数据库与假的 Manager,覆盖鉴权、角色、CSRF、草稿隔离、并发冲突、任务提交、配置应用/恢复、上游不可用缓存、图片校验及监控统计。`test_config_helper.py` 独立验证写入白名单、自定义命令保留、注入拒绝与失败恢复,不操作真实同步配置。
|
||||
|
||||
本次部署核验真实 API、镜像目录、日志、监控与后台草稿保存。没有为验证效果而停止、重启仓库,或应用修改后的生产同步参数。
|
||||
|
||||
## 统一中文内容升级
|
||||
|
||||
公告的 title/body 和镜像的 help 已改为单个字符串,所有界面语言共用中文内容。旧版本的语言映射在服务启动时以事务迁移:优先保留 zh-CN,其次 zh;没有中文键的历史记录保留原内容供管理员整理。原始 JSON 保存在 SQLite 的 shared_content_backups 表,迁移会递增记录 revision,避免旧编辑页面覆盖新内容;重复启动不会重复迁移。
|
||||
|
||||
此次内容格式升级需要在回退旧应用时同时恢复升级前的数据库备份。部署前使用 SQLite backup API 生成一致性快照,备份目录仅 root 可读。迁移不修改镜像介绍、站点文案或同步配置。
|
||||
|
||||
## 首页相关链接
|
||||
|
||||
`settings/site` 的 `relatedLinks` 独立控制首页侧栏,与页脚 `links` 分开。旧设置缺少该字段时,接口补充原有的三个入口,不改写数据库。显式保存空数组会隐藏该栏目;仍打开旧版表单的客户端提交时,若未携带该字段,后端保留当前已保存的相关链接。条目支持 `kind=help` 的站内使用帮助与 `kind=url` 的 HTTP(S)/站内绝对路径;不接受脚本、data URL、协议相对地址或反斜杠路径。名称支持多语言,数组顺序即显示顺序。
|
||||
@@ -0,0 +1,124 @@
|
||||
#!/usr/bin/python3
|
||||
"""Sample per-socket TCP receive counters and attribute worker processes to mirrors."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
PROC = Path('/proc')
|
||||
MIRRORS = '/data/mirrors/'
|
||||
OUTPUT = Path('/var/lib/mirrorouc/metrics/job-speeds.json')
|
||||
NAME = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9_.+-]{0,127}$')
|
||||
|
||||
def mirror_name(value):
|
||||
if not value.startswith(MIRRORS):
|
||||
return None
|
||||
name = value[len(MIRRORS):].split('/')[0]
|
||||
return name if NAME.fullmatch(name) and name not in ('.', '..') else None
|
||||
|
||||
def process_jobs(proc=PROC):
|
||||
jobs = {}
|
||||
parents = {}
|
||||
for entry in proc.iterdir():
|
||||
if not entry.name.isdigit():
|
||||
continue
|
||||
try:
|
||||
if '/tunasync-worker.service' not in (entry / 'cgroup').read_text():
|
||||
continue
|
||||
pid = int(entry.name)
|
||||
status = (entry / 'stat').read_text().rsplit(') ', 1)[1].split()
|
||||
parents[pid] = int(status[1])
|
||||
arguments = (entry / 'cmdline').read_bytes().decode(errors='replace').split('\0')
|
||||
candidates = {name for argument in arguments if (name := mirror_name(argument))}
|
||||
if not candidates:
|
||||
name = mirror_name(os.readlink(entry / 'cwd'))
|
||||
if name:
|
||||
candidates.add(name)
|
||||
if len(candidates) == 1:
|
||||
jobs[pid] = candidates.pop()
|
||||
except (OSError, ValueError, IndexError):
|
||||
continue
|
||||
# git/ssh/curl helpers may inherit a job's parent without its destination argument.
|
||||
for pid in parents:
|
||||
ancestor = pid
|
||||
for _ in range(16):
|
||||
if ancestor in jobs:
|
||||
jobs[pid] = jobs[ancestor]
|
||||
break
|
||||
ancestor = parents.get(ancestor, 0)
|
||||
if not ancestor:
|
||||
break
|
||||
return jobs
|
||||
|
||||
def parse_sockets(output, jobs):
|
||||
sockets = {}
|
||||
header = None
|
||||
for line in output.splitlines():
|
||||
if line and not line[0].isspace():
|
||||
header = line
|
||||
continue
|
||||
if header is None:
|
||||
continue
|
||||
identity = re.search(r'\bino:(\d+)\s+sk:([a-fA-F0-9]+)', header)
|
||||
if not identity:
|
||||
continue
|
||||
pids = [int(value) for value in re.findall(r'\bpid=(\d+)', header)]
|
||||
names = {jobs[pid] for pid in pids if pid in jobs}
|
||||
received = re.search(r'\bbytes_received:(\d+)', line)
|
||||
# The same inherited socket can list several PIDs: count its inode/cookie once.
|
||||
sockets[':'.join(identity.groups())] = {'bytes': int(received.group(1)) if received else 0, 'job': names.pop() if len(names) == 1 else None}
|
||||
header = None
|
||||
return sockets
|
||||
|
||||
def rates(current, previous, active, elapsed):
|
||||
result = {name: {'bytesPerSecond': 0.0, 'connections': 0} for name in active}
|
||||
if previous is None or elapsed <= 0 or elapsed > 15:
|
||||
return {}
|
||||
for identity, socket in current.items():
|
||||
name = socket['job']
|
||||
if name is None:
|
||||
continue
|
||||
old = previous.get(identity)
|
||||
delta = socket['bytes'] - old['bytes'] if old else socket['bytes']
|
||||
if delta < 0: # A reset/reused socket must never produce a negative rate/spike.
|
||||
continue
|
||||
value = result.setdefault(name, {'bytesPerSecond': 0.0, 'connections': 0})
|
||||
value['bytesPerSecond'] += delta / elapsed
|
||||
value['connections'] += 1
|
||||
return result
|
||||
|
||||
def write(value):
|
||||
fd, temporary = tempfile.mkstemp(prefix='.speeds-', dir=OUTPUT.parent)
|
||||
try:
|
||||
with os.fdopen(fd, 'w') as stream:
|
||||
json.dump(value, stream, allow_nan=False, separators=(',', ':'))
|
||||
os.chmod(temporary, 0o640)
|
||||
os.replace(temporary, OUTPUT)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
|
||||
def run():
|
||||
previous = None
|
||||
last = time.monotonic()
|
||||
while True:
|
||||
start = time.monotonic()
|
||||
try:
|
||||
jobs = process_jobs()
|
||||
result = subprocess.run(['/usr/bin/ss', '-tinpeH', 'state', 'established'], capture_output=True, text=True, timeout=4, check=True, env={**os.environ, 'LC_ALL': 'C'})
|
||||
current = parse_sockets(result.stdout, jobs)
|
||||
sampled = time.monotonic()
|
||||
elapsed = sampled - last
|
||||
values = rates(current, previous, set(jobs.values()), elapsed)
|
||||
write({'updatedAt': round(time.time() * 1000), 'interval': elapsed, 'available': previous is not None and elapsed <= 15, 'jobs': values})
|
||||
previous, last = current, sampled
|
||||
except (OSError, ValueError, subprocess.SubprocessError):
|
||||
previous = None
|
||||
write({'updatedAt': round(time.time() * 1000), 'interval': 2, 'available': False, 'jobs': {}})
|
||||
time.sleep(max(.1, 2 - (time.monotonic() - start)))
|
||||
|
||||
if __name__ == '__main__':
|
||||
run()
|
||||
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/python3
|
||||
"""Read Linux/NFS counters. A separate passive nftables table supplies protocol bytes."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
ROOT = Path(os.environ.get('MIRROROUC_METRICS_ROOT', '/var/lib/mirrorouc/metrics'))
|
||||
ROOT.mkdir(parents=True, exist_ok=True)
|
||||
path = ROOT / 'history.json'
|
||||
try:
|
||||
history = json.loads(path.read_text())
|
||||
except (OSError, ValueError):
|
||||
history = {'samples': []}
|
||||
|
||||
now = time.time()
|
||||
cpu = list(map(int, Path('/proc/stat').read_text().splitlines()[0].split()[1:9]))
|
||||
mem = {line.split(':')[0]: int(line.split()[1]) * 1024 for line in Path('/proc/meminfo').read_text().splitlines() if len(line.split()) > 1}
|
||||
nfs = {}
|
||||
for block in Path('/proc/self/mountstats').read_text().split('device '):
|
||||
if ' mounted on /data ' not in block:
|
||||
continue
|
||||
for line in block.splitlines():
|
||||
values = line.strip().split()
|
||||
if values and values[0] in ('READ:', 'WRITE:'):
|
||||
nfs[values[0][:-1]] = list(map(int, values[1:]))
|
||||
|
||||
counters = {}
|
||||
try:
|
||||
result = subprocess.run(['/usr/sbin/nft', '-j', 'list', 'counters', 'table', 'inet', 'mirrorouc_monitor'], capture_output=True, timeout=4, check=True)
|
||||
for item in json.loads(result.stdout).get('nftables', []):
|
||||
counter = item.get('counter', {})
|
||||
if counter.get('name') in ('https4', 'https6', 'rsync4', 'rsync6'):
|
||||
counters[counter['name']] = counter['bytes']
|
||||
except (OSError, ValueError, subprocess.SubprocessError):
|
||||
pass
|
||||
|
||||
previous = history.get('previous')
|
||||
sample = {'time': round(now * 1000)}
|
||||
if previous:
|
||||
elapsed = max(.1, now - previous['time'])
|
||||
delta = [max(0, a - b) for a, b in zip(cpu, previous['cpu'])]
|
||||
total = sum(delta)
|
||||
if total:
|
||||
sample.update(user=(delta[0] + delta[1]) / total * 100, system=(delta[2] + delta[5] + delta[6] + delta[7]) / total * 100, wait=delta[4] / total * 100, idle=delta[3] / total * 100)
|
||||
for key, value in counters.items():
|
||||
old = previous.get('counters', {}).get(key)
|
||||
if old is not None and value >= old:
|
||||
sample[key] = (value - old) * 8 / elapsed / 1e9
|
||||
for operation, prefix in [('READ', 'read'), ('WRITE', 'write')]:
|
||||
a, b = nfs.get(operation), previous.get('nfs', {}).get(operation)
|
||||
if a and b and a[0] >= b[0]:
|
||||
sample[prefix + 'Ops'] = (a[0] - b[0]) / elapsed / 1000
|
||||
byte_index = 4 if operation == 'READ' else 3
|
||||
sample[prefix + 'Bytes'] = max(0, a[byte_index] - b[byte_index]) / elapsed / 1e9
|
||||
cached = max(0, mem.get('Cached', 0) + mem.get('SReclaimable', 0) - mem.get('Shmem', 0))
|
||||
free, buffers, total_mem = mem['MemFree'], mem.get('Buffers', 0), mem['MemTotal']
|
||||
used = max(0, total_mem - free - cached - buffers)
|
||||
sample.update(used=used / 2**30, cached=cached / 2**30, buffers=buffers / 2**30, free=free / 2**30)
|
||||
if 'user' in sample:
|
||||
history['samples'].append(sample)
|
||||
|
||||
disks = []
|
||||
for mount in ['/data', '/']:
|
||||
try:
|
||||
disk = os.statvfs(mount)
|
||||
disks.append({'name': mount, 'total': disk.f_blocks * disk.f_frsize / 1e12, 'used': (disk.f_blocks - disk.f_bfree) * disk.f_frsize / 1e12})
|
||||
except OSError:
|
||||
pass
|
||||
history.update(previous={'time': now, 'cpu': cpu, 'nfs': nfs, 'counters': counters}, updatedAt=round(now * 1000), memoryTotal=mem['MemTotal'] / 2**30, disks=disks)
|
||||
history['samples'] = [point for point in history['samples'] if point['time'] >= (now - 86400) * 1000][-8640:]
|
||||
fd, temporary = tempfile.mkstemp(dir=ROOT, prefix='.metrics-')
|
||||
try:
|
||||
with os.fdopen(fd, 'w') as output:
|
||||
json.dump(history, output, allow_nan=False, separators=(',', ':'))
|
||||
output.flush()
|
||||
os.fsync(output.fileno())
|
||||
os.chmod(temporary, 0o640)
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
@@ -0,0 +1,178 @@
|
||||
#!/usr/bin/python3
|
||||
"""Root-only configuration writer, reachable only through a group-restricted Unix socket.
|
||||
|
||||
Independently verifies allowed TOML changes; never executes submitted commands.
|
||||
"""
|
||||
import copy
|
||||
import glob
|
||||
import grp
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import socketserver
|
||||
import tempfile
|
||||
import tomllib
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
ROOT = '/etc/tunasync'
|
||||
MAIN = ROOT + '/worker.conf'
|
||||
INCLUDES = ROOT + '/mirrors.d'
|
||||
SOCKET = '/run/mirrorouc-config/control.sock'
|
||||
JOURNAL = '/var/lib/mirrorouc-config/pending.json'
|
||||
IDENTIFIER = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9_.+-]{0,127}$')
|
||||
|
||||
def number(value, lower, upper):
|
||||
return type(value) is int and lower <= value <= upper
|
||||
|
||||
def upstream(value):
|
||||
if not isinstance(value, str) or len(value) >= 2048 or re.search(r'[\s`;$|<>\\\"\']', value):
|
||||
return False
|
||||
parsed = urllib.parse.urlsplit(value)
|
||||
return parsed.scheme in ('rsync', 'https', 'http') and bool(parsed.hostname) and parsed.username is None
|
||||
|
||||
def sync_numbers(table):
|
||||
return all(key not in table or number(table[key], low, high) for key, low, high in [('interval', 1, 525600), ('retry', 0, 20), ('timeout', 30, 1209600)])
|
||||
|
||||
def stripped_mirrors(doc):
|
||||
result = copy.deepcopy(doc)
|
||||
mirrors = result.get('mirrors', [])
|
||||
assert isinstance(mirrors, list)
|
||||
for mirror in mirrors:
|
||||
for field in ('upstream', 'interval', 'retry', 'timeout'):
|
||||
mirror.pop(field, None)
|
||||
return result
|
||||
|
||||
def validate_new(doc, path):
|
||||
assert set(doc) == {'mirrors'} and len(doc['mirrors']) == 1
|
||||
mirror = doc['mirrors'][0]
|
||||
assert set(mirror) <= {'name', 'provider', 'upstream', 'interval', 'retry', 'timeout', 'stage1_profile'}
|
||||
assert IDENTIFIER.fullmatch(mirror['name']) and mirror['name'] not in ('.', '..')
|
||||
assert path == INCLUDES + '/mirrorouc-' + mirror['name'] + '.conf'
|
||||
assert mirror['provider'] in ('rsync', 'two-stage-rsync')
|
||||
assert upstream(mirror['upstream']) and sync_numbers(mirror)
|
||||
assert 'stage1_profile' not in mirror or mirror['stage1_profile'] == 'debian'
|
||||
|
||||
def validate(path, before, after):
|
||||
assert path == MAIN or (os.path.dirname(path) == INCLUDES and os.path.basename(path).endswith('.conf'))
|
||||
assert not os.path.islink(path)
|
||||
old = tomllib.loads(before) if before is not None else None
|
||||
new = tomllib.loads(after) if after is not None else None
|
||||
assert old is not None or new is not None
|
||||
if path == MAIN:
|
||||
assert old is not None and new is not None
|
||||
assert number(new['global'].get('concurrent'), 1, 64) and sync_numbers(new['global'])
|
||||
for doc in (old, new):
|
||||
for field in ('concurrent', 'interval', 'retry', 'timeout'):
|
||||
doc['global'].pop(field, None)
|
||||
assert old == new
|
||||
elif old is None or new is None:
|
||||
validate_new(new if new is not None else old, path)
|
||||
else:
|
||||
assert stripped_mirrors(old) == stripped_mirrors(new)
|
||||
for previous, current in zip(old.get('mirrors', []), new.get('mirrors', [])):
|
||||
if previous != current:
|
||||
assert upstream(current['upstream']) and sync_numbers(current)
|
||||
|
||||
def read(path):
|
||||
try:
|
||||
with open(path, encoding='utf-8', newline='') as stream:
|
||||
return stream.read()
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
|
||||
def atomic(path, value):
|
||||
if value is None:
|
||||
os.unlink(path)
|
||||
return
|
||||
mode = os.stat(path).st_mode & 0o777 if os.path.exists(path) else 0o644
|
||||
fd, temporary = tempfile.mkstemp(prefix='.mirrorouc-', dir=os.path.dirname(path))
|
||||
try:
|
||||
with os.fdopen(fd, 'w', encoding='utf-8', newline='') as stream:
|
||||
stream.write(value)
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
os.chmod(temporary, mode)
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
|
||||
def reload_worker():
|
||||
worker = tomllib.loads(read(MAIN))['global']['name']
|
||||
body = json.dumps({'worker_id': worker, 'mirror_id': '', 'cmd': 'reload', 'args': [], 'options': {}}).encode()
|
||||
request = urllib.request.Request('http://127.0.0.1:12345/cmd', data=body, headers={'Content-Type': 'application/json'})
|
||||
with urllib.request.urlopen(request, timeout=10) as response:
|
||||
assert response.status == 200
|
||||
|
||||
def signature():
|
||||
pattern = tomllib.loads(read(MAIN)).get('include', {}).get('include_mirrors', '')
|
||||
assert pattern == INCLUDES + '/*.conf'
|
||||
names = sorted(set([MAIN] + glob.glob(pattern)))
|
||||
digest = hashlib.sha256()
|
||||
for path in names:
|
||||
assert not os.path.islink(path)
|
||||
raw = read(path).encode()
|
||||
digest.update(str(len(path.encode())).encode() + b':' + path.encode() + str(len(raw)).encode() + b':' + raw)
|
||||
return digest.hexdigest()
|
||||
|
||||
def restore(changes):
|
||||
for path, change in changes.items():
|
||||
if change['before'] is None and not os.path.exists(path):
|
||||
continue
|
||||
atomic(path, change['before'])
|
||||
reload_worker()
|
||||
|
||||
def apply(request):
|
||||
changes = request['files']
|
||||
assert isinstance(changes, dict) and 0 < len(changes) <= 20
|
||||
for path, change in changes.items():
|
||||
validate(path, change['before'], change['after'])
|
||||
if read(path) != change['before']:
|
||||
return 'CONFLICT'
|
||||
if signature() != request['baseHash']:
|
||||
return 'CONFLICT'
|
||||
# Write-ahead journal survives a helper/process restart during a multi-file update.
|
||||
atomic(JOURNAL, json.dumps(changes))
|
||||
try:
|
||||
for path, change in changes.items():
|
||||
atomic(path, change['after'])
|
||||
reload_worker()
|
||||
except Exception:
|
||||
try:
|
||||
restore(changes)
|
||||
except Exception:
|
||||
return 'CONFIG_ROLLBACK_FAILED'
|
||||
os.unlink(JOURNAL)
|
||||
return 'CONFIG_APPLY_FAILED'
|
||||
os.unlink(JOURNAL)
|
||||
return ''
|
||||
|
||||
class Handler(socketserver.StreamRequestHandler):
|
||||
def handle(self):
|
||||
self.connection.settimeout(5)
|
||||
try:
|
||||
raw = self.rfile.readline(2 * 1024 * 1024 + 1)
|
||||
assert len(raw) <= 2 * 1024 * 1024 and raw.endswith(b'\n')
|
||||
if os.path.exists(JOURNAL):
|
||||
code = 'CONFIG_ROLLBACK_FAILED'
|
||||
else:
|
||||
code = apply(json.loads(raw))
|
||||
except (AssertionError, KeyError, TypeError, ValueError):
|
||||
code = 'INVALID_INPUT'
|
||||
except Exception:
|
||||
code = 'CONFIG_APPLY_FAILED'
|
||||
self.wfile.write(json.dumps({'code': code}).encode() + b'\n')
|
||||
|
||||
if __name__ == '__main__':
|
||||
if os.path.exists(JOURNAL):
|
||||
with open(JOURNAL) as stream:
|
||||
restore(json.load(stream))
|
||||
os.unlink(JOURNAL)
|
||||
if os.path.exists(SOCKET):
|
||||
os.unlink(SOCKET)
|
||||
with socketserver.UnixStreamServer(SOCKET, Handler) as server:
|
||||
os.chown(SOCKET, 0, grp.getgrnam('mirrorouc').gr_gid)
|
||||
os.chmod(SOCKET, 0o660)
|
||||
server.serve_forever()
|
||||
@@ -0,0 +1,5 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
if ! /usr/sbin/nft list table inet mirrorouc_monitor >/dev/null 2>&1; then
|
||||
/usr/sbin/nft -f /etc/mirrorouc/protocol-counters.nft
|
||||
fi
|
||||
@@ -0,0 +1,23 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cd "$(dirname "$0")"
|
||||
if ! id mirrorouc >/dev/null 2>&1; then
|
||||
useradd --system --home-dir /var/lib/mirrorouc --shell /usr/sbin/nologin mirrorouc
|
||||
fi
|
||||
install -d -o mirrorouc -g mirrorouc -m 0700 /var/lib/mirrorouc
|
||||
install -d -o root -g mirrorouc -m 0750 /var/lib/mirrorouc/metrics
|
||||
install -d -o root -g root -m 0755 /usr/local/lib/mirrorouc
|
||||
install -d -o root -g root -m 0755 /etc/mirrorouc
|
||||
install -m 0644 collect_metrics.py /usr/local/lib/mirrorouc/collect_metrics.py
|
||||
install -m 0644 collect_job_speeds.py /usr/local/lib/mirrorouc/collect_job_speeds.py
|
||||
install -m 0755 ensure-counters.sh /usr/local/lib/mirrorouc/ensure-counters.sh
|
||||
install -m 0644 protocol-counters.nft /etc/mirrorouc/protocol-counters.nft
|
||||
install -m 0644 mirrorouc-counters.service mirrorouc-metrics.service mirrorouc-metrics.timer mirrorouc-speeds.service /etc/systemd/system/
|
||||
if ! /usr/sbin/nft list table inet mirrorouc_monitor >/dev/null 2>&1; then
|
||||
/usr/sbin/nft -c -f /etc/mirrorouc/protocol-counters.nft
|
||||
fi
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now mirrorouc-counters.service mirrorouc-metrics.timer
|
||||
systemctl start mirrorouc-metrics.service
|
||||
systemctl enable --now mirrorouc-speeds.service
|
||||
systemctl restart mirrorouc-speeds.service
|
||||
@@ -0,0 +1,46 @@
|
||||
#!/bin/sh
|
||||
# Run as root: sh deploy/install-preview.sh /opt/mirrorouc/releases/<release>
|
||||
set -eu
|
||||
release=$(readlink -f "$1")
|
||||
case "$release" in /opt/mirrorouc/releases/*) ;; *) echo 'Invalid release path' >&2; exit 1;; esac
|
||||
test -x "$release/mirrorouc"
|
||||
test -f "$release/dist/index.html"
|
||||
deploy="$release/deploy"
|
||||
id mirrorouc >/dev/null 2>&1 || useradd --system --home-dir /var/lib/mirrorouc --shell /usr/sbin/nologin mirrorouc
|
||||
install -d -m 0700 -o mirrorouc -g mirrorouc /var/lib/mirrorouc
|
||||
install -d -m 0755 /etc/mirrorouc /usr/local/lib/mirrorouc
|
||||
install -d -m 0700 /var/lib/mirrorouc-config
|
||||
install -m 0755 "$deploy/config-helper.py" /usr/local/lib/mirrorouc/config-helper.py
|
||||
install -m 0644 "$deploy/mirrorouc.service" "$deploy/mirrorouc-config.service" /etc/systemd/system/
|
||||
if [ ! -f /etc/mirrorouc/mirrorouc.env ]; then
|
||||
umask 077
|
||||
cat > /etc/mirrorouc/mirrorouc.env <<'EOF'
|
||||
MIRROROUC_LISTEN=127.0.0.1:8081
|
||||
MIRROROUC_DATA=/var/lib/mirrorouc
|
||||
MIRROROUC_STATIC=/opt/mirrorouc/current/dist
|
||||
MIRROROUC_MANAGER=http://127.0.0.1:12345
|
||||
MIRROROUC_WORKER=srv-software-mirror
|
||||
MIRROROUC_WORKER_CONFIG=/etc/tunasync/worker.conf
|
||||
MIRROROUC_LOGS=/var/log/tunasync
|
||||
MIRROROUC_METRICS=/var/lib/mirrorouc/metrics/history.json
|
||||
MIRROROUC_CONFIG_SOCKET=/run/mirrorouc-config/control.sock
|
||||
MIRROROUC_SECURE_COOKIES=false
|
||||
EOF
|
||||
fi
|
||||
# Leave current tunasync configurations, ownership and permissions intact.
|
||||
if [ ! -s /var/lib/mirrorouc/mirrorouc.db ]; then
|
||||
umask 077
|
||||
runuser -u mirrorouc -- "$release/mirrorouc" -data /var/lib/mirrorouc -init-admin admin > /etc/mirrorouc/initial-admin-password
|
||||
fi
|
||||
ln -sfn "$release" /opt/mirrorouc/current.new
|
||||
mv -Tf /opt/mirrorouc/current.new /opt/mirrorouc/current
|
||||
install -m 0644 "$deploy/nginx-preview.conf" /etc/nginx/sites-available/mirrorouc-preview
|
||||
ln -sfn /etc/nginx/sites-available/mirrorouc-preview /etc/nginx/sites-enabled/mirrorouc-preview
|
||||
nginx -t
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now mirrorouc-config.service
|
||||
systemctl restart mirrorouc-config.service
|
||||
systemctl enable mirrorouc.service
|
||||
systemctl restart mirrorouc.service
|
||||
systemctl reload nginx
|
||||
echo 'MirrorOUC preview installed at http://100.69.2.2:8080'
|
||||
@@ -0,0 +1,25 @@
|
||||
[Unit]
|
||||
Description=MirrorOUC restricted tunasync configuration writer
|
||||
After=tunasync-manager.service
|
||||
|
||||
[Service]
|
||||
ExecStart=/usr/bin/python3 /usr/local/lib/mirrorouc/config-helper.py
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
RuntimeDirectory=mirrorouc-config
|
||||
RuntimeDirectoryMode=0755
|
||||
StateDirectory=mirrorouc-config
|
||||
StateDirectoryMode=0700
|
||||
UMask=0077
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/etc/tunasync /var/lib/mirrorouc-config /run/mirrorouc-config
|
||||
ProtectKernelTunables=true
|
||||
ProtectControlGroups=true
|
||||
RestrictSUIDSGID=true
|
||||
RestrictAddressFamilies=AF_UNIX AF_INET
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,9 @@
|
||||
[Unit]
|
||||
Description=MirrorOUC passive protocol byte counters
|
||||
After=nftables.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/lib/mirrorouc/ensure-counters.sh
|
||||
RemainAfterExit=yes
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,13 @@
|
||||
[Unit]
|
||||
Description=MirrorOUC read-only host metrics collection
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=root
|
||||
Group=mirrorouc
|
||||
ExecStart=/usr/bin/python3 /usr/local/lib/mirrorouc/collect_metrics.py
|
||||
UMask=0027
|
||||
NoNewPrivileges=true
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=/var/lib/mirrorouc/metrics
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
@@ -0,0 +1,8 @@
|
||||
[Unit]
|
||||
Description=Collect MirrorOUC metrics every 10 seconds
|
||||
[Timer]
|
||||
OnBootSec=10
|
||||
OnUnitActiveSec=10
|
||||
AccuracySec=1
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -0,0 +1,22 @@
|
||||
[Unit]
|
||||
Description=MirrorOUC per-mirror upstream TCP receive rates
|
||||
After=tunasync-worker.service
|
||||
|
||||
[Service]
|
||||
User=root
|
||||
Group=mirrorouc
|
||||
ExecStart=/usr/bin/python3 /usr/local/lib/mirrorouc/collect_job_speeds.py
|
||||
Restart=on-failure
|
||||
RestartSec=3
|
||||
UMask=0027
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=/var/lib/mirrorouc/metrics
|
||||
ProtectKernelTunables=true
|
||||
ProtectControlGroups=true
|
||||
RestrictAddressFamilies=AF_UNIX AF_NETLINK AF_INET AF_INET6
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,26 @@
|
||||
[Unit]
|
||||
Description=MirrorOUC web and administration API
|
||||
After=network.target tunasync-manager.service mirrorouc-config.service
|
||||
Wants=mirrorouc-config.service
|
||||
|
||||
[Service]
|
||||
User=mirrorouc
|
||||
Group=mirrorouc
|
||||
WorkingDirectory=/opt/mirrorouc/current
|
||||
EnvironmentFile=/etc/mirrorouc/mirrorouc.env
|
||||
ExecStart=/opt/mirrorouc/current/mirrorouc
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
UMask=0077
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/lib/mirrorouc
|
||||
ProtectKernelTunables=true
|
||||
ProtectControlGroups=true
|
||||
RestrictSUIDSGID=true
|
||||
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,22 @@
|
||||
server {
|
||||
listen 100.69.2.2:8080;
|
||||
server_name 100.69.2.2;
|
||||
client_max_body_size 5m;
|
||||
|
||||
location /mirrors/ {
|
||||
alias /data/mirrors/;
|
||||
autoindex on;
|
||||
autoindex_exact_size off;
|
||||
autoindex_localtime on;
|
||||
add_header X-Content-Type-Options nosniff always;
|
||||
add_header Content-Security-Policy "sandbox; default-src 'none'; style-src 'unsafe-inline'; img-src 'self' data:" always;
|
||||
}
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:8081;
|
||||
proxy_set_header Host $http_host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_read_timeout 35s;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
# Passive accounting only; no existing rules or tables are flushed.
|
||||
table inet mirrorouc_monitor {
|
||||
counter https4 { }
|
||||
counter https6 { }
|
||||
counter rsync4 { }
|
||||
counter rsync6 { }
|
||||
chain account {
|
||||
meta nfproto ipv4 tcp sport 443 counter name https4
|
||||
meta nfproto ipv4 tcp dport 443 tcp sport != 443 counter name https4
|
||||
meta nfproto ipv6 tcp sport 443 counter name https6
|
||||
meta nfproto ipv6 tcp dport 443 tcp sport != 443 counter name https6
|
||||
meta nfproto ipv4 tcp sport 873 counter name rsync4
|
||||
meta nfproto ipv4 tcp dport 873 tcp sport != 873 counter name rsync4
|
||||
meta nfproto ipv6 tcp sport 873 counter name rsync6
|
||||
meta nfproto ipv6 tcp dport 873 tcp sport != 873 counter name rsync6
|
||||
}
|
||||
chain input {
|
||||
type filter hook input priority 300; policy accept;
|
||||
iifname != "lo" jump account
|
||||
}
|
||||
chain output {
|
||||
type filter hook output priority 300; policy accept;
|
||||
oifname != "lo" jump account
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
spec = importlib.util.spec_from_file_location('helper', os.path.join(os.path.dirname(__file__), 'config-helper.py'))
|
||||
helper = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(helper)
|
||||
|
||||
class ValidationTests(unittest.TestCase):
|
||||
def test_preserves_custom_commands(self):
|
||||
before = '[[mirrors]]\nname="debian"\nprovider="rsync"\nupstream="rsync://old/debian"\ncommand="/usr/local/bin/parallel-rsync"\n'
|
||||
after = before.replace('old/debian', 'new/debian') + 'interval=60\nretry=2\ntimeout=120\n'
|
||||
helper.validate(helper.INCLUDES + '/debian.conf', before, after)
|
||||
with self.assertRaises(AssertionError):
|
||||
helper.validate(helper.INCLUDES + '/debian.conf', before, after.replace('parallel-rsync', 'other-command'))
|
||||
|
||||
def test_blocks_other_global_fields_and_paths(self):
|
||||
before = '[global]\nname="worker"\nconcurrent=12\ninterval=60\nretry=2\ntimeout=120\n'
|
||||
helper.validate(helper.MAIN, before, before.replace('concurrent=12', 'concurrent=8'))
|
||||
for path, after in [('/etc/shadow', before), (helper.MAIN, before.replace('worker', 'other')), (helper.MAIN, before.replace('concurrent=12', 'concurrent=0'))]:
|
||||
with self.assertRaises(AssertionError):
|
||||
helper.validate(path, before, after)
|
||||
|
||||
def test_new_mirrors_and_injection(self):
|
||||
value = '[[mirrors]]\nname="test"\nprovider="rsync"\nupstream="rsync://mirror/test"\ninterval=60\nretry=0\ntimeout=120\n'
|
||||
helper.validate_new(helper.tomllib.loads(value), helper.INCLUDES + '/mirrorouc-test.conf')
|
||||
for changed in [value + 'command="evil"\n', value.replace('rsync://mirror/test', 'rsync://mirror/`cmd`'), value.replace('retry=0', 'retry=-1')]:
|
||||
with self.assertRaises(AssertionError):
|
||||
helper.validate_new(helper.tomllib.loads(changed), helper.INCLUDES + '/mirrorouc-test.conf')
|
||||
|
||||
def test_reload_failure_restores_files(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
path = root + '/worker.conf'
|
||||
before = '[global]\nname="worker"\nconcurrent=12\ninterval=60\nretry=2\ntimeout=120\n'
|
||||
helper.atomic(path, before)
|
||||
changes = {path: {'before': before, 'after': before.replace('concurrent=12', 'concurrent=8')}}
|
||||
with patch.object(helper, 'MAIN', path), patch.object(helper, 'JOURNAL', root + '/pending.json'), patch.object(helper, 'signature', return_value='base'), patch.object(helper, 'reload_worker', side_effect=[OSError('unavailable'), None]):
|
||||
self.assertEqual(helper.apply({'files': changes, 'baseHash': 'base'}), 'CONFIG_APPLY_FAILED')
|
||||
self.assertEqual(helper.read(path), before)
|
||||
self.assertFalse(os.path.exists(root + '/pending.json'))
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,40 @@
|
||||
import importlib.util
|
||||
import os
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
spec = importlib.util.spec_from_file_location('speeds', os.path.join(os.path.dirname(__file__), 'collect_job_speeds.py'))
|
||||
speeds = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(speeds)
|
||||
|
||||
class SpeedTests(unittest.TestCase):
|
||||
def test_sockets_deduplicate_shared_pids_and_separate_repos(self):
|
||||
data = '0 0 10.0.0.1:123 10.0.0.2:873 users:(("rsync",pid=1,fd=3),("rsync",pid=2,fd=3)) uid:9 ino:20 sk:abc <->\n\t cubic bytes_sent:4 bytes_received:1000\n0 0 10.0.0.1:124 10.0.0.2:873 users:(("rsync",pid=3,fd=3)) uid:9 ino:21 sk:abd <->\n\t cubic bytes_received:2000\n'
|
||||
current = speeds.parse_sockets(data, {1: 'debian', 2: 'debian', 3: 'ubuntu'})
|
||||
result = speeds.rates(current, {'20:abc': {'bytes': 400}, '21:abd': {'bytes': 1000}}, {'debian', 'ubuntu'}, 2)
|
||||
self.assertEqual(result['debian'], {'bytesPerSecond': 300, 'connections': 1})
|
||||
self.assertEqual(result['ubuntu']['bytesPerSecond'], 500)
|
||||
|
||||
def test_parallel_and_new_connections_add_without_startup_spike(self):
|
||||
current = {'1:a': {'bytes': 1000, 'job': 'debian'}, '2:b': {'bytes': 500, 'job': 'debian'}, '3:c': {'bytes': 9000, 'job': None}}
|
||||
self.assertEqual(speeds.rates(current, None, {'debian'}, 2), {})
|
||||
result = speeds.rates(current, {'1:a': {'bytes': 800}}, {'debian', 'ubuntu'}, 2)
|
||||
self.assertEqual(result['debian']['bytesPerSecond'], 350)
|
||||
self.assertEqual(result['ubuntu']['bytesPerSecond'], 0)
|
||||
self.assertEqual(speeds.rates(current, {}, {'debian'}, 20), {})
|
||||
|
||||
def test_process_ancestry_and_path_boundaries(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
for pid, parent, command in [(1, 0, ['/usr/bin/python', '/worker']), (2, 1, ['rsync', '/data/mirrors/debian/']), (3, 2, ['ssh', 'host']), (4, 0, ['rsync', '/data/mirrors/ubuntu/'])]:
|
||||
path = Path(root) / str(pid); path.mkdir()
|
||||
(path/'stat').write_text(f'{pid} (process) S {parent} 0 0')
|
||||
(path/'cgroup').write_text('0::/system.slice/tunasync-worker.service' if pid != 4 else '0::/system.slice/other.service')
|
||||
(path/'cmdline').write_bytes('\0'.join(command).encode())
|
||||
jobs = speeds.process_jobs(Path(root))
|
||||
self.assertEqual(jobs, {2: 'debian', 3: 'debian'})
|
||||
self.assertIsNone(speeds.mirror_name('/data/mirrors-other/secret'))
|
||||
self.assertIsNone(speeds.mirror_name('/data/mirrors/../secret'))
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user