Chapter 9. 高级打包

Table of Contents

9.1. Historical perspective
9.2. Current trends
9.3. Note on build system
9.4. 持续集成
9.5. 自举
9.6. 编译加固
9.7. 可重现的构建
9.8. Substvar
9.9. 库软件包
9.10. 多体系结构
9.11. Debian 二进制软件包的拆分
9.12. 拆包的场景和例子
9.13. Multiarch library path
9.14. Multiarch header file path
9.15. Multiarch *.pc file path
9.16. 库符号
9.17. Library package name
9.18. 库变迁
9.19. binNMU 安全
9.20. 调试信息
9.21. -dbgsym package
9.22. debconf

Let’s describe advanced topics on Debian packaging.

Let me oversimplify historical perspective of Debian packaging practices focused on the non-native packaging.

Debian was started in 1990s when upstream packages were available from public FTP sites such as Sunsite. In those early days, Debian packaging used Debian source format currently known as the Debian source format 1.0:

  • The Debian source package ships a set of files for the Debian source package.

    • package_version.orig.tar.gz : symlink to or copy of the upstream released file.
    • package_version-revision.diff.gz : One big patch for Debian modifications.
    • package_version-revision.dsc : package description.
  • Several workaround approaches such as dpatch, dbs, or cdbs were deplyoed to manage multiple topic patches.

The modern Debian source format 3.0 (quilt) was invented around 2008 (see ProjectsDebSrc3.0):

  • The Debian source package ships a set of files for the Debian source package.

    • package_version.orig.tar.?z : symlink to or copy of the upstream released file.
    • package_version-revision.debian.tar.?z : tarball of debian/ for Debian modifications.

      • The debian/source/format file contains 3.0 (quilt).
      • Optional multiple topic patches are stored in the debian/patches/ directory.
    • package_version-revision.dsc : package description.
  • The standarized approach to manage multiple topic patches using quilt(1) is deployed for the Debian source format 3.0 (quilt).

Most Debian packages adopted the Debian source formats 3.0 (quilt) and 3.0 (native).

Now, the git(1) is popular with upstream and Debian developers. The git and its associated tools are important part of the modern Debian packaging workflow. This modern workflow involving git will be mentioned later in Chapter 10, Packaging with git.

Current Debian packaging practices and their trends are moving target. See:

  • Debian Trends” — Hints for De facto standard of Debian practices

    • Build systems: dh
    • Debian source format: 3.0 (quilt)
    • VCS: git
    • VCS Hosting: salsa
    • Rules-Requires-Root: adopted, fakeroot
    • Copyright format: DEP-5
  • debhelper-compat-upgrade-checklist(7) manpage — Upgrade checklist for debhelper
  • DEP - Debian Enhancement Proposals — Formal proposals to enhance Debian

You can also search entire Debian source code data by yourself, too.

Auto-generated files of the build system may be found in the released upstream tarball. These should be regenerated when Debian package is build. E.g.:

  • dh $@ --with autoreconf should be used in the debian/rules if Autotools (autoconf + automake) are used.

Some modern build system may be able to download required source codes and binary files from arbitrary remote hosts to satisfy build requirements. Don’t use this download feature. The official Debian package is required to be build only with packages listed in Build-Depends: of the debian/control file.

The dh_auto_test(1) command is a debhelper command that tries to automatically run the test suite provided by the upstream developer during the Debian package building process.

The autopkgtest(1) command can be used after the Debian package building process. It tests generated Debian binary packages in the virtual environment using the debian/tests/control RFC822-style metadata file as continuous integration (CI). See:

您可以在 Debian 系统上探索使用不同的持续集成系统。

Debian cares about supporting new ports or flavours. The new ports or flavours require bootstrapping operation for the cross-build of the initial minimal native-building system. In order to avoid build-dependency loops during bootstrapping, the build-dependency needs to be reduced using the DEB_BUILD_PROFILES environment variable.

See Debian wiki: BuildProfileSpec.

[Tip]Tip

If a core package foo build depends on a package bar with deep build dependency chains but bar is only used in the test target in foo, you can safely mark the bar with <!nocheck> in the Build-depends of foo to avoid build loops.

The compiler hardening support spreading for Debian jessie (8.0) demands that we pay extra attention to the packaging.

您应当详细阅读下列参考内容。

debmake 命令会向 debian/rules 文件中按需添加 DEB_BUILD_MAINT_OPTIONSDEB_CFLAGS_MAINT_APPENDDEB_LDFLAGS_MAINT_APPEND 的项目(参见 Chapter 5, Simple packagingdpkg-buildflags(1))。

为了做到软件包可重现的构建,这里给出一些相关的建议。

dpkg-genbuildinfo(1) 生成的控制文件 source-name_source-version_arch.buildinfo 记录了构建环境信息。参见 deb-buildinfo(5)

debian/control 也定义了软件包的依赖关系,其中变量替换机制”(substvar)的功能可以用来将软件包维护者从跟踪(大多数简单的)软件包依赖的重复劳动中解放出来。请参见 deb-substvars(5)。

debmake 命令支持下列变量替换指令:

  • ${misc:Depends},可用于所有二进制软件包
  • ${misc:Pre-Depends},可用于所有 multiarch 软件包
  • ${shlibs:Depends},可用于所有含有二进制可执行文件或库的软件包
  • ${python:Depends},可用于所有 Python 软件包
  • ${python3:Depends},可用于所有 Python3 软件包
  • ${perl:Depends},用于所有 Perl 软件包
  • ${ruby:Depends},用于所有 Ruby 软件包

For the shared library, required libraries found simply by objdump -p /path/to/program | grep NEEDED are covered by the shlib substvar.

For Python and other interpreters, required modules found simply looking for lines with import, use, require, etc., are covered by the corresponding substvars.

对其它没有部署属于自己范畴内的变量替换机制的情况,misc 变量替换占位符通常用来覆盖对应的依赖替换需求。

对 POSIX shell 程序来说,并没有简单的办法来验证其依赖关系,substvar 的变量替换也无法自动得出它们的依赖。

对使用动态加载机制,包括 GObject introspection 机制的库和模块来说,现在没有简单的方法可以检查依赖关系,变量替换机制也无法自动推导出所需的依赖。

打包软件库需要您投入更多的工作。下面有一些打包软件库的提醒和建议:

在打包共享库软件之前,请查阅:

如需研究其历史背景,请参见:

Debian wheezy(7.0,2013年5月)在 dpkgapt 中引入了对跨架构二进制软件包安装的多架构支持(特别是 i386 架构和 amd64 架构,但也支持其它的组合),这部分内容值得我们额外关注。

您应当详细阅读下列参考内容。

多架构支持使用三元组(<triplet>)的值,例如 i386-linux-gnux86_64-linux-gnu;它们出现在共享链接库的安装路径中,例如 /usr/lib/<triplet>/,等等。

  • 三元组 <triplet> 的值由 debhelper 脚本隐式提前设置好,软件包维护者无需担心。
  • 不过,在 debian/rules 文件中用于 override_dh_* 目标的三元组 <triplet> 值需要由维护者手动进行显式设置。三元组 <triplet> 的值可由 $(DEB_HOST_MULTIARCH) 变量在 debian/rules 文件中获取到,具体方法如下:

    DEB_HOST_MULTIARCH = $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)
    ...
    override_dh_install:
    	mkdir -p package1/lib/$(DEB_HOST_MULTIARCH)
    	cp -dR tmp/lib/. package1/lib/$(DEB_HOST_MULTIARCH)

参见:

对行为良好的构建系统来说,对 Debian 二进制包的拆分可以由如下方式实现。

  • 为所有二进制软件包在 debian/control 文件中创建对应的二进制软件包条目。
  • 在对应的 debian/二进制软件包名.install 文件中列出所有文件的路径(相对于 debian/tmp 目录)。

请查看本指南中相关的例子:

An intuitive and flexible method to create the initial template debian/control file defining the split of the Debian binary packages is accommodated with the -b option. See Section 15.2, “debmake -b.

对于下面这样的上游源代码示例,我们在这里给出使用 debmake 处理时一些典型的 multiarch 软件包拆分的场景和做法:

  • 一个软件库源码 libfoo-1.0.tar.gz
  • 一个软件工具源码 bar-1.0.tar.gz,软件由编译型语言编写
  • 一个软件工具源码 baz-1.0.tar.gz,软件由解释型语言编写
二进制软件包类型Architecture:Multi-Arch:软件包内容

libfoo1

lib*

any

same

共享库,可共同安装

libfoo-dev

dev*

any

same

共享库头文件及相关开发文件,可共同安装

libfoo-tools

bin*

any

foreign

运行时支持程序,不可共同安装

libfoo-doc

doc*

all

foreign

共享库文档

bar

bin*

any

foreign

编译好的程序文件,不可共同安装

bar-doc

doc*

all

foreign

程序的配套文档文件

baz

script

all

foreign

解释型程序文件

Debian policy requires to comply with the Filesystem Hierarchy Standard (FHS), version 3.0”, with the exceptions noted in File System Structure.

The most notable exception is the use of /usr/lib/<triplet>/ instead of /usr/lib<qual>/ (e.g., /lib32/ and /lib64/) to support a multiarch library.


对基于 Autotools 且由 debhelper (compat>=9)管理的软件包来说,这些路径设置已由 dh_auto_configure 命令自动处理。

对于其它使用不支持的构建系统的软件包,您需要按照下面的方式手动调整安装路径。

  • If ./configure is used in the override_dh_auto_configure target in debian/rules, make sure to replace it with dh_auto_configure -- while re-targeting the install path from /usr/lib/ to /usr/lib/$(DEB_HOST_MULTIARCH)/.
  • 请在 debian/foo.install 文件中将所有出现的 /usr/lib/ 字符串替换为 /usr/lib/*/

所有启用多架构的软件包安装至相同路径的文件必须内容完全相同。您必须小心处理,避免数据字节序或者压缩算法等等问题带来的文件内容差异。

位于默认路径 /usr/lib//usr/lib/<triplet>/ 的共享库可被自动加载。

对位于其它路径的共享库,必须使用 pkg-config 命令设置 GCC 选项 -l 以正确进行加载。

在支持多架构的 Debian 系统上,GCC 默认会同时包含、使用 /usr/include//usr/include/<triplet>/ 下的头文件。

如果头文件不在这些路径中,必须使用 pkg-config 命令设置 GCC 的 -I 参数以使得“#include <foo.h>”正常工作。


为库文件使用 /usr/lib/<triplet>/软件包名/ 路径可帮助上游维护者对使用 /usr/lib/<triplet> 的多架构系统和使用 /usr/lib<qual>/ 的双架构系统使用相同的安装脚本。[19]

使用包含 packagename 的文件路径也使得在同一系统上同时安装多个架构的开发库成为可能。

packagename 用来获取系统上已安装库的信息。它在 *.pc 文件中存储配置参数,用来设置 GCC 的 -I-l 选项。


Debian lenny(5.0,2009年5月)中引入的 dpkg 符号支持可以帮助我们管理同一共享链接库软件包的向后 ABI 兼容性(backward ABI compatibility)。二进制软件包中的 DEBIAN/symbols 文件提供了每个符号及其对应的最小版本号。

一个极其简化的软件库打包流程大概如下所示。

  • Extract the old DEBIAN/symbols file of the immediate previous binary package with the dpkg-deb -e command.

    • 或者,mc 命令也可以用来解压得到 DEBIAN/symbols 文件。
  • 将其复制为 debian/binarypackage.symbols 文件。

    • 如果这是第一次打包的话,可以只创建一个空文件。
  • 构建二进制软件包。

    • 如果 dpkg-gensymbols 命令警告添加了新的符号的话:

      • Extract the updated DEBIAN/symbols file with the dpkg-deb -e command.
      • 将其中的 Debian 修订版本号,例如 -1,从文件中去除。
      • 将其复制为 debian/binarypackage.symbols 文件。
      • 重新构建二进制软件包。
    • 如果 dpkg-gensymbols 命令不报和新链接符号有关的警告:

      • 您已完成了共享库的打包工作。

如需了解详细信息,您应当阅读下列第一手参考资料。

  • 8.6.3 The symbols system” of the Debian Policy Manual
  • dh_makeshlibs(1) manapage
  • dpkg-gensymbols(1) manapage
  • dpkg-shlibdeps(1) manapage
  • deb-symbols(5) manapage

您也应当查看:

[Tip]Tip

For C++ libraries and other cases where the tracking of symbols is problematic, follow 8.6.4 The shlibs system of the Debian Policy Manual, instead. Please make sure to erase the empty debian/binarypackage.symbols file generated by the debmake command. For this case, the DEBIAN/shlibs file is used.

我们考虑 libfoo 这个库的上游 tarball 源码压缩包的名字从 libfoo-7.0.tar.gz 更新为了 libfoo-8.0.tar.gz,同时带有一次 SONAME 大版本的跳跃(并因此影响了其它软件包)。

库的二进制软件包将必须从 libfoo7 重命名为 libfoo8 以保持使用 unstable 套件的系统上所有依赖该库的软件包在上传了基于 libfoo-8.0.tar.gz 的新库后仍然能够正常运行。

[Warning]Warning

如果这个二进制库软件包没有得到更名,许多使用 unstable 套件的系统上的各个依赖该库的软件包会在新的库包上传后立刻破损,即便立刻请求进行 binNMU 上传也无法避免这个问题。由于种种原因,binNMU 不可能在上传后立刻开始进行,故无法缓解问题。

-dev 软件包必须遵循以下命名规则:

[Tip]Tip

如果包内数据文件编码方案有所变化(如,从 latin1 变为 utf-8),该场景应比照 API 变化做类似的考虑与处理。

参见 Section 9.9, “库软件包”

When you package a new library package version which affects other packages, you must file a transition bug report against the release.debian.org pseudo package using the reportbug command with the ben file and wait for the approval for its upload from the Release Team.

发行团队提供了变迁跟踪系统。参见 变迁(Transition)

[Caution]Caution

请确保您按照 Section 9.17, “Library package name” 的描述正确地对二进制软件包进行了重命名。

A binNMU” is a binary-only non-maintainer upload performed for library transitions etc. In a binNMU upload, only the Architecture: any packages are rebuilt with a suffixed version number (e.g. version 2.3.4-3 will become 2.3.4-3+b1). The Architecture: all packages are not built.

The dependency defined in the debian/control file among binary packages from the same source package should be safe for the binNMU. This needs attention if there are both Architecture: any and Architecture: all packages involved in it.

  • Architecture: any package: depends on Architecture: any foo package

    • Depends: foo (= ${binary:Version})
  • Architecture: any package: depends on Architecture: all bar package

    • Depends: bar (= ${source:Version})
  • Architecture: all package: depends on Architecture: any baz package

    • Depends: baz (>= ${source:Version}), baz (<< ${source:Version}.0~)

The Debian package is built with the debugging information but packaged into the binary package after stripping the debugging information as required by Chapter 10 - Files” of the Debian Policy Manual.

参见

调试信息由 dh_strip 命令的默认行为自动打包并进行剥离。所分离得到的调试软件包名具有 -dbgsym 的后缀。

  • debian/rules 文件不应显式包括 dh_strip
  • 编辑 debian/control 文件,在 Build-Depends 中写入 debhelper-compat (>=13),同时移除 Build-Depends 中对 debhelper 的依赖。

debconf 软件包可以帮助我们在下列两种情况下配置软件包:

  • debian-installer(Debian 安装器)预安装时进行非交互式配置。
  • interactively from the menu interface (dialog, gnome, kde, …​)

    • 软件包安装时:由 dpkg 命令调用
    • 对已安装软件包:由 dpkg-reconfigure 命令调用

软件包安装时的所有用户交互都必须由这里的 debconf 系统进行处理,下列配置文件对这个过程进行控制。

  • debian/binarypackage.config

    • 这是 debconf config 脚本,用于向用户询问对于配置软件包必需的问题。
  • debian/binarypackage.template

    • 这是 debconf templates(模板)文件,用于向用户询问对于配置软件包必需的问题。

These debconf files are called by package configuration scripts in the binary Debian package

  • DEBIAN/binarypackage.preinst
  • DEBIAN/binarypackage.prerm
  • DEBIAN/binarypackage.postinst
  • DEBIAN/binarypackage.postrm

See dh_installdebconf(1), debconf(7), debconf-devel(7) and 3.9.1 Prompting in maintainer scripts” in the Debian Policy Manual.



[17] 该文档是在 symbols 文件被引入之前写成的。

[18] 第六章 - 开发(-DEV)软件包中,存在强烈的使用含有 SONAME 版本号的 -dev 软件包名而非仅使用 -dev 作为名称的偏好,但前 ftp-master 成员(Steve Langasek)对此有不同意见。请注意该文档在 multiarch 系统和 symbols 引入之前写成,可能有一定程度的过时。

[19] This path is compliant with the FHS. Filesystem Hierarchy Standard: /usr/lib : Libraries for programming and packages states Applications may use a single subdirectory under /usr/lib. If an application uses a subdirectory, all architecture-dependent data exclusively used by the application must be placed within that subdirectory.