APT Update Error: Public Key is N/A

问题背景

为了保持系统更新，笔者每隔一周会更新一次软件源。今天在尝试更新时，发现 APT 无法获取 MySql 的 Apt repository url，报错如下：

图 1: 公钥未找到，无法验签

Err:4 http://repo.mysql.com/apt/debian bullseye InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 467B942D3A79BD29

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://repo.mysql.com/apt/debian bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 467B942D3A79BD29
W: Failed to fetch http://repo.mysql.com/apt/debian/dists/bullseye/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 467B942D3A79BD29
W: Some index files failed to download. They have been ignored, or old ones used instead.

简短回答

TL;DR

从问题的关键词来看，原因就是我们缺少对应的公钥，无法验证该软件源的签名。本着缺啥就补啥的修复思路，重新从公钥服务器（Key Server）获取 Public Key 即可。

Key Servers

已知的几个 Key Server 可供下载：

这些 Key Server 一般带有 Web 版的搜索界面，可以先在网页端查找库中是否存有我们想要的 Key ID。这个 ID 可以理解为该证书与其它证书的区别性特征，即 Key Fingerprint。

图 2: Ubuntu Key Server

笔者先从报错信息中拿到 Key ID，在上述几个 Server 都进行了搜索，但只有 Ubuntu Key Server 找到了匹配的结果。

须注意：搜索的关键词若为十六进制的 Key ID，则需在开头加上 0x，比如 0x467B942D3A79BD29 ，不然系统当作普通字符串去搜索了。

图 3: Key ID Search Result

从 KS 获取公钥

我们将上面能找到匹配结果的 Key Server 和 Key ID 分别填到 apt-key 的命令行关键字参数后，开始下载公钥。相关的参数说明如下：

adv：意指 advanced options，这些高级选项将会被传递给 gpg 命令处理，即用于下载 key。
--keyserver：其中 Key Server 的 URL 是使用独有的 OpenPGP HTTP Keyserver Protocol (HKP) 协议类型，其以 hkp// 开头；类似于 http 与 https 的关系，hkp 也有受 SSL/TLS 加密保护的 hkps（HKP over TLS）。
--recv-keys：要获取的 Key ID

图 4: 获取公钥并加入 trusted 证书库

$ sudo apt-key adv --keyserver hkps://keyserver.ubuntu.com --recv-keys 0x467B942D3A79BD29

Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
Executing: /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.1.sh --keyserver hkps://keyserver.ubuntu.com --recv-keys 0x467B942D3A79BD29
gpg: key 467B942D3A79BD29: public key "MySQL Release Engineering <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1

查看已导入的 Key

从下面命令的输出可以看到，我们成功导入一个有效期更长的 MySql 公钥，另一个旧公钥临近过期了（2022-02-16），再过半个月就失效了，想必这就是 MySQL 维护者 Oracle 更换 APT 包公钥的原因。

$ sudo apt-key list

/etc/apt/trusted.gpg
--------------------
pub   dsa1024 2003-02-03 [SCA] [expires: 2022-02-16]
      A4A9 4068 76FC BD3C 4567  70C8 8C71 8D3B 5072 E1F5
uid           [ unknown] MySQL Release Engineering <[email protected]>

pub   rsa4096 2021-12-14 [SC] [expires: 2023-12-14]
      859B E8D7 C586 F538 430B  19C2 467B 942D 3A79 BD29
uid           [ unknown] MySQL Release Engineering <[email protected]>
sub   rsa4096 2021-12-14 [E] [expires: 2023-12-14]

/etc/apt/trusted.gpg.d/debian-archive-bullseye-automatic.gpg
------------------------------------------------------------
...

重新 APT Update

在将缺少的公钥导入到系统信任库中后，笔者重新执行 apt update 这次没有报错，可以下载更新，可以验证签名，可以安装更新。Bingo~ 🎉

图 5: 正常更新软件源索引

$ sudo apt update
Hit:1 http://repo.mysql.com/apt/debian bullseye InRelease
Hit:2 http://deb.debian.org/debian bullseye InRelease
Hit:3 http://deb.debian.org/debian bullseye-updates InRelease
Hit:4 http://security.debian.org bullseye-security InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
9 packages can be upgraded. Run 'apt list --upgradable' to see them.

更好的答案

Deprecation warnings

细心的读者可能注意到，在执行 apt-key 获取公钥时，出现了这行警告，提示该命令已经弃用，建议我们直接使用 gpg 去管理 /etc/apt/trusted.gpg.d/ 下的密钥文件。

Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).

在弃用 APT key 命令之前，我们先看看它背后帮我们做了些什么工作。在切换到 gpg 命令后，这些工作中哪部分是需要我们自己手动做的。

APT Key 背后原理

首先，利用 which 命令定位一下 apt-key 命令的出处：

$ which apt-key
/usr/bin/apt-key

拷贝一份出来，查看其内部实现，它是一个 8 百多行的 shell 脚本。它本身支持的命令比较多，我们直奔解析 adv 高级选项的相关代码。

解析命令行参数

# command 变量，它的值就是第一个命令行参数
command="$1"
# 必要参数如未传递，显示本命令的用法
if [ -z "$command" ]; then
    usage
    exit 1
fi
shift

# 若是请求非 `帮助` 或 `验证` 的话，准备 GPG 要用到的主目录。
if [ "$command" != 'help' ] && [ "$command" != 'verify' ]; then
    prepare_gpg_home
fi

case "$command" in
    add)
        ;;
    del|rm|remove)
        ;;
    update)
        ;;
    net-update)
        ;;
    list|finger*)
        ;;
    export|exportall)
        ;;
    # 解析将要传递给 gpg 的高级选项
    adv*)
        warn_on_script_usage
        # 环境准备：配置环境变量，文件系统等
        setup_merged_keyring
        # 准备执行：将待执行的命令和参数，通过 echo 输出到终端
        aptkey_echo "Executing: $GPG" "$@"
        # 实际执行：从 Key Server 获取公钥
        aptkey_execute "$GPG" "$@"
        # 将变更（若有）合并到本地的证书库
        merge_back_changes
        ;;
    verify)
        ;;
    help)
        usage
        ;;
    *)
        usage
        exit 1
        ;;
esac

实际执行的代码为 aptkey_execute "$GPG" "$@"。其中 $@ 表示的是剩余的命令行参数，在我们的例子中就是后面两个具名参数 --keyserver hkps://keyserver.ubuntu.com --recv-keys 0x467B942D3A79BD29，那么 $GPG 代表什么呢？

$GPG 是什么？

# now tell gpg that it shouldn't try to maintain this trustdb file
echo "#!/bin/sh
exec '$(escape_shell "${GPG_EXE}")' --ignore-time-conflict --no-options --no-default-keyring \\
--homedir '$(escape_shell "${GPGHOMEDIR}")' --no-auto-check-trustdb --trust-model always \"\$@\"" > "${GPGHOMEDIR}/gpg.0.sh"
GPG_SH="${GPGHOMEDIR}/gpg.0.sh"
# GPG 临时可执行文件.0
GPG="$GPG_SH"

# ... skip some lines of code ...

setup_merged_keyring() {
    # FORCE_KEYID 字符串变量的长度 > 0？
    if [ -n "$FORCED_KEYID" ]; then
        merge_all_trusted_keyrings_into_pubring
        FORCED_KEYRING="${GPGHOMEDIR}/forcedkeyid.gpg"
        TRUSTEDFILE="${FORCED_KEYRING}"
        echo "#!/bin/sh
exec sh '($(escape_shell "${GPG}")' --keyring '$(escape_shell "${TRUSTEDFILE}")' \"\$@\"" > "${GPGHOMEDIR}/gpg.1.sh"
        GPG="${GPGHOMEDIR}/gpg.1.sh"
        # ignore error as this "just" means we haven't found the forced keyid and the keyring will be empty
        import_keyring_into_keyring '' "$TRUSTEDFILE" "$FORCED_KEYID" || true
    # FORCE_KEYID 字符串变量的长度 = 0？(✅匹配我们的命令行输入)
    elif [ -z "$FORCED_KEYRING" ]; then
        merge_all_trusted_keyrings_into_pubring
        if [ -r "${GPGHOMEDIR}/pubring.gpg" ]; then
            cp -a "${GPGHOMEDIR}/pubring.gpg" "${GPGHOMEDIR}/pubring.orig.gpg"
        else
           touch "${GPGHOMEDIR}/pubring.gpg" "${GPGHOMEDIR}/pubring.orig.gpg"
        fi
        # 构造临时可执行文件.1
        echo "#!/bin/sh
exec sh '$(escape_shell "${GPG}")' --keyring '$(escape_shell "${GPGHOMEDIR}/pubring.gpg")' \"\$@\"" > "${GPGHOMEDIR}/gpg.1.sh"
        GPG="${GPGHOMEDIR}/gpg.1.sh"
    else
        TRUSTEDFILE="$(dearmor_filename "$FORCED_KEYRING")"
        create_new_keyring "$TRUSTEDFILE"
        echo "#!/bin/sh
exec sh '$(escape_shell "${GPG}")' --keyring '$(escape_shell "${TRUSTEDFILE}")' \"\$@\"" > "${GPGHOMEDIR}/gpg.1.sh"
        GPG="${GPGHOMEDIR}/gpg.1.sh"
    fi
}

从上面的定义可知，GPG 先是赋值为 /tmp-path/gpg.0.sh，后面指向 /tmp-path/gpg.1.sh。而后者又调用了前者。

结合终端输出

还记得上面 aptkey_echo "Executing: $GPG" "$@" 有记录日志到终端吗？摘选出来：

Executing: /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.1.sh --keyserver hkps://keyserver.ubuntu.com --recv-keys 0x467B942D3A79BD29

结合其终端输出，可知 $GPG 最终指向 /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.1.sh 这个临时文件。笔者尝试想把查看这些临时文件的内容，但是 apt-key 命令自带清理机制——执行完毕它们就自动删除了。自动删除临时文件的代码写得很棒👍，留在下一节分析🔬。

截留临时 SH 文件

为了截留这些临时可执行文件，最容易想到的方法就是在变动前各拷贝一份。笔者选在 setup_merged_keyring 调用的前、后时刻，分别保存 $GPG 变量指向的临时文件的内容，并记录如下：

$ cat /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.1.sh
#!/bin/sh
exec sh '/tmp/apt-key-gpghome.4UClNKT1Z2/gpg.0.sh' --keyring '/tmp/apt-key-gpghome.4UClNKT1Z2/pubring.gpg' "$@"

$ cat /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.0.sh
#!/bin/sh
exec 'gpg' --ignore-time-conflict --no-options --no-default-keyring \
--homedir '/tmp/apt-key-gpghome.4UClNKT1Z2' --no-auto-check-trustdb --trust-model always "$@"

以上两个嵌套调用的 sh 便是 apt-key 实际执行的 GPG 调用命令。

合并变更的 Keys

merge_back_changes() {
    if [ -n "$FORCED_KEYRING" ]; then
        # if the keyring was forced merge is already done
        if [ "$FORCED_KEYRING" != "$TRUSTEDFILE" ]; then
            mv -f "$FORCED_KEYRING" "${FORCED_KEYRING}~"
            export_key_from_to "$TRUSTEDFILE" "$FORCED_KEYRING" --armor
        fi
        return
    fi
    # -s 判断目标文件是否存在，且文件大小 > 0
    if [ -s "${GPGHOMEDIR}/pubring.gpg" ]; then
        # merge all updated keys
        foreach_keyring_do 'merge_keys_into_keyrings' "${GPGHOMEDIR}/pubring.gpg"
    fi
    # look for keys which were added or removed
    get_fingerprints_of_keyring "${GPGHOMEDIR}/pubring.orig.gpg" > "${GPGHOMEDIR}/pubring.orig.keylst"
    get_fingerprints_of_keyring "${GPGHOMEDIR}/pubring.gpg" > "${GPGHOMEDIR}/pubring.keylst"
    comm -3 "${GPGHOMEDIR}/pubring.keylst" "${GPGHOMEDIR}/pubring.orig.keylst" > "${GPGHOMEDIR}/pubring.diff"
    # key isn't part of new keyring, so remove
    cut -f 2 "${GPGHOMEDIR}/pubring.diff" | while read key; do
        if [ -z "$key" ]; then continue; fi
        foreach_keyring_do 'remove_key_from_keyring' "$key"
    done
    # key is only part of new keyring, so we need to import it
    cut -f 1 "${GPGHOMEDIR}/pubring.diff" | while read key; do
        if [ -z "$key" ]; then continue; fi
        import_keyring_into_keyring '' "$TRUSTEDFILE" "$key"
    done
}

临时可执行文件的生成

# gpg needs (in different versions more or less) files to function correctly,
# so we give it its own homedir and generate some valid content for it later on
create_gpg_home() {
    # for cases in which we want to cache a homedir due to expensive setup
    if [ -n "$GPGHOMEDIR" ]; then
        return
    fi
    if [ -n "$TMPDIR" ]; then
        # tmpdir is a directory and current user has rwx access to it
        # same tests as in apt-pkg/contrib/fileutl.cc GetTempDir()
        if [ ! -d "$TMPDIR" ] || [ ! -r "$TMPDIR" ] || [ ! -w "$TMPDIR" ] || [ ! -x "$TMPDIR" ]; then
            unset TMPDIR
        fi
    fi
    GPGHOMEDIR="$(mktemp --directory --tmpdir 'apt-key-gpghome.XXXXXXXXXX')"
    CURRENTTRAP="${CURRENTTRAP} cleanup_gpg_home;"
    trap "${CURRENTTRAP}" 0 HUP INT QUIT ILL ABRT FPE SEGV PIPE TERM
    if [ -z "$GPGHOMEDIR" ]; then
        apt_error "Could not create temporary gpg home directory in $TMPDIR (wrong permissions?)"
        exit 28
    fi
    chmod 700 "$GPGHOMEDIR"
}

（未完待续）