简要记录如何编写spec文件来打rpm包。

安装 (1)

以CentOS-7为例：

1
2

yum install -y rpmdevtools.noarch rpmlint.noarch
rpmdev-setuptree

Spec文件的结构 (2)

首先，假设要打多个rpm包，例如

• lua-5.3.6-1.el7.x86_64.rpm
• lua-devel-5.3.6-1.el7.x86_64.rpm
• lua-static-5.3.6-1.el7.x86_64.rpm

第一个叫main package。打单个rpm包(main package)是打多个包的特例，不比细说。

Spec文件主要包含：

• 元信息：
  • Name, Version, Release, License, URL, Source0, Source1, …, Patch0, Patch1, …, BuildRequires
  • Summary, Group, Provides, Requires
  • 每个package的元信息写在%package ${package-name}的下面，main package除外，可以认为默认是main的元信息。
  • 上面我故意把元信息分层2部分(Name开头的一行和Summary开头的一行)。通常，只有main包含第1部分，其它package不须重复，因为一般情况下多个package是一个工程编译构建出来的，源代码(Source)、Patch、编译依赖(BuildRequires)是共同的。
• 编译前准备(%prep段)：一般是解压并进入源代码目录(%setup命令)，安装patch(%patch0命令)，也可以添加一些自定义操作(用bash写即可)；
• 编译(build段)：前一步已经进入源代码目录，现在开始编译。一般linux程序的编译步骤是./configure && make(用bash写即可)。Rpmbuild工具也提供了一个%configure命令，应该是调用./configure。
• 安装到buildroot(%install段)：
  • 通常是make install DESTDIR=%{buildroot}
  • 注意，这不是rpm -hiv，也不是在目标机器上执行。从命令make install也可以看出来，是在编译机器上是把编译产生的binaries, headers, libraries, docs, scripts等文件拷贝到buildroot(通常是/root/rpmbuild/BUILDROOT/{name}-{release}.{arch})目录。
• 定义每个package包含哪些文件(%files段)：例如main应该包含binaries和一些libraries, scripts及docs；devel package应该包含headers和libraries等等。注意：上一步install拷贝的文件要被全覆盖，否则报错。
• 安装前脚本(%pre段)：在目标机器上执行。执行rpm -hiv时拷贝文件之前的动作。
• 安装后脚本(%post段)：在目标机器上执行。执行rpm -hiv时拷贝文件之后的动作。例如，安装可执行包之后把binaries的目录添加到PATH；安装devel package之后执行ldconfig建立libraries索引；
• 删除前脚本(preun段)：在目标机器上执行。执行rpm -e时删除文件之前的动作。
• 删除后脚本(%postun段)：在目标机器上执行。执行rpm -e时删除文件之后的动作。通常是%post段的反操作，例如把binaries的目录从PATH删除；删除devel package之后再次执行ldconfig更新索引；
• 清理(%clean段): 通常是rm -rf ${buildroot}.
• 修改日志(%changelog段)：每个release的更新记录；

Spec skeleton:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

#%package foo declaration neglected
Name:           foo
Version:        1.0
Release:        1%{?dist}
Summary:        Stupid foo
Group:          Development
License:        MIT
URL:            http://www.foo.org/
Source0:        foo-%{version}.tar.gz
Patch0:         foo-%{version}-bugfix.patch
BuildRequires:  bar-devel buz-devel
#package foo provides foo-1.0
Provides:       lua = 1.0

#should be "%description foo", but package name ("foo") neglected
%description
Foo is a super stupid foo project.

%prep
%setup -q
%patch0 -p1

%build
./configure
make -j30

%install
make install DESTDIR=%{buildroot}

#should be "%files foo", but package name ("foo") neglected
%files
/usr/local/bin/*
/usr/local/libexec/*
/usr/local/share/*

#should be "%pre foo", but package name ("foo") neglected
%pre

#should be "%post foo", but package name ("foo") neglected
%post

#should be "%preun foo", but package name ("foo") neglected
%preun

#should be "%postun foo", but package name ("foo") neglected
%postun

#package declaration
%package devel
Summary:        Development files for %{name}
Group:          System Environment/Libraries
Requires:       %{name} = %{version}-%{release}
Requires:       pkgconfig

%description devel
This package contains development files for %{name}.

%files devel
/usr/local/include/*
/usr/local/lib/lib*.so

%pre devel

%post devel
ldconfig

%preun devel

%postun devel
ldconfig

%clean
rm -rf %{buildroot}

%changelog

案例 (3)

简要说明linux虚拟化套件中的一些术语，特别是qemu, KVM, hypervisor；以及它们的区别与联系。

Qemu和kvm的区别与联系 (1)

最初qemu (Quick Emulator)是一个系统模拟器，它可以在一种architecture的系统上模拟另一种architecture的另一种系统。不但模拟CPU，还模拟各种外设。其中CPU的模拟主要是通过指令翻译的方式，所以速度比较慢。

KVM是后来产生的。

KVM is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V). It consists of a loadable kernel module, kvm.ko, that provides the core virtualization infrastructure and a processor specific module, kvm-intel.ko or kvm-amd.ko.

可见，KVM只在Linux系统上，并且只支持特定的CPU。可以查看CPU是否支持KVM:

1

cat /proc/cpuinfo | egrep "vmx|svm" | uniq

The presence of the vmx (for Intel) or svm (for AMD) flags indicate that your CPU supports the virtualization extensions.

KVM只靠这两个内核模块也不能创建完整的虚拟机，所以KVM团队就把qemu fork过来，作为用户态的部分，一起实现了虚拟机(qemu模拟系统的其他组件如磁盘等；KVM模拟CPU内存网络)。

后来，fork的分支又merge到qemu主分支中去，就形成现在的局面：

• 从KVM的角度看，它的用户态部分在qemu里；qemu和KVM一起虚拟出完整的guest；
• 从qemu的角度看，它有一个virtualization模式，就是使用KVM（或Xen）来模拟CPU内存网络，实现加速。qemu通过KVM的用户态部分访问KVM的内核态部分。KVM或Xen叫做hypervisor。

另外：qemu-kvm是过时的命令，被qemu-system-x86_64 -enable-kvm取代。

这里有一个操作过程，虽然有点老，但可以看出qemu的使用方式。
https://subscription.packtpub.com/book/virtualization_and_cloud/9781788294676/1

Qemu的3种模式 (2)

Full-system emulation (2.1)

Qemu emulates a full system, including one or several processors and various peripherals.

启动方式：

• qemu-system-x86_64
• qemu-system-i386
• qemu-system-aarch64
• qemu-system-ppc
• qemu-system-arm
• …

User-mode emulation (2.2)

Qemu is able to invoke a Linux executable compiled for a (potentially) different architecture by leveraging the host system resources。例如在x86_64的系统上运行为PowerPC编译的binaray。

Virtualization (2.3)

Similar with “full-system emulation”, but if the target architecture matches the host CPU, this mode may still benefit from a significant speedup by using a hypervisor like KVM or Xen。有的地方也把它归到”Full-system emulation”模式里，也叫做：full system emulation with the hardware acceleration support provided by hypervisors like KVM and Xen.

启动方式：

• qemu-system-x86_64 -enable-kvm (-enable-kvm等价于-accel kvm)

KVM需要：1. host操作系统是Linux；2. host的processor是x86_64（Intel-VT或者AMD-V）。使用Virtualization模式要求guest和host的architecture相同。也就是说，使用qemu+kvm的方式的话，host必须是”Linux on x86_64 (Intel-VT or AMD-V)”，guest必须是x86_64的VM，可以是x86_64的Linux，也可以是x86_64的Windows。

Hypervisor (3)

Hypervisor was originally called a virtual machine monitor or VMM. The physical hardware that a hypervisor runs on is typically referred to as a host machine, whereas the VMs the hypervisor creates and supports are collectively called guest machines. A hypervisor enables the host hardware to operate multiple VMs independent of each other and share abstracted resources among those VMs.

There are two types of hypervisors: Type 1 and Type 2 hypervisors. Both hypervisor varieties can virtualize common elements such as CPU, memory and networking, but based on its location in the stack, the hypervisor virtualizes these elements differently. The main difference between Type 1 vs. Type 2 hypervisors is that Type 1 runs on bare metal and Type 2 runs on top of an operating system.

• Type 1 hypervisors: A Type 1 hypervisor runs directly on the host machine’s physical hardware, and it’s referred to as a bare-metal hypervisor. The Type 1 hypervisor doesn’t have to load an underlying OS. With direct access to the underlying hardware and no other software – such as OSes and device drivers – to contend with for virtualization, Type 1 hypervisors are regarded as the most efficient and best-performing hypervisors available for enterprise computing. Examples:

  • ESXi hypervisor (included in VMware vSphere);
  • Microsoft Hyper-V;
  • KVM: The KVM hypervisor enables admins to convert a Linux kernel into a hypervisor and has direct access to hardware along with any VMs hosted by the hypervisor. Features include live migration, scheduling and resource control. Every VM is implemented as a regular Linux process, scheduled by the standard Linux scheduler, with dedicated virtual hardware like a network card, graphics adapter, CPU(s), memory, and disks.
  • Xen hypervisor;
  • Oracle VM;
  • Citrix Hypervisor

• Type 2 hypervisors: A Type 2 hypervisor is typically installed on top of an existing OS. It is sometimes called a hosted hypervisor because it relies on the host machine’s preexisting OS to manage calls to CPU, memory, storage and network resources. Examples:

  • Oracle VM VirtualBox;
  • VMware Workstation Pro: on Windows and Linux systems
  • VMware Fusion: on Mac;
  • Qemu;

Although the purpose and goals of Type 1 and Type 2 hypervisors are identical, the presence of an underlying OS with Type 2 hypervisors introduces unavoidable latency; all of the hypervisor’s activities and the work of every VM has to pass through the host OS. Also, any security flaws or vulnerabilities in the host OS could potentially compromise all of the VMs running above it.

可见，从Hypervisor的分类方式上看，qemu是type 2 hypervisor，KVM是Type 1 hypervisor。但从定义上看，Type 1 hypervisor不需要Operating System，而KVM却需要Linux系统，如何理解呢？我读到一些这样的描述：

• KVM installs natively on all Linux distributions and turns underlying physical servers into hypervisors so that they can host multiple, isolated virtual machines (VMs);
• The KVM hypervisor enables admins to convert a Linux kernel into a hypervisor;
• KVM plugs directly into the kernel’s code and allows it to function as a hypervisor;
• KVM converts Linux into a type-1 (bare-metal) hypervisor. All hypervisors need some operating system-level components—such as a memory manager, process scheduler, input/output (I/O) stack, device drivers, security manager, a network stack, and more—to run VMs. KVM has all these components because it’s part of the Linux kernel.
• KVM is part of Linux. Linux is part of KVM.

KVM把Linux变成了bare-metal hypervisor。或者说，使用KVM生产虚拟机时，KVM变成主体，Linux就变成a part of KVM，KVM通过Linux的能力管理内存、CPU及其它设备。这样KVM（包扩Linux）就像ESXi一样，是一个Type 1 hypervisor（虽然它看起来还像一个普通Linux系统）。所以，这种情况下，不能认为KVM就是两个Linux内核模块，相反，Linux是KVM的一部分。KVM（包括Linux）变成Linux hypervisor。

KVM allows Linux to function as a hypervisor, so a host machine can run multiple isolated virtual environments called guests. KVM basically provides Linux with hypervisor capabilities. This means that the hypervisor components such as memory manager, scheduler, network stack, etc. are provided as part of the Linux kernel. The VMs are regular Linux processes scheduled by a standard Linux scheduler with dedicated virtual hardware such as network adapters.

简要记录Python3中的metaclass。注意本文是基于new-style class的。Python3中所有class都是new-style class，Python2则不同：Python2.2以前根本不支持new-style class，而从Python2.2开始，虽然支持但需要显示地声明。

一切皆对象 (1)

在Python中，一切都是对象，比如: int, 3, 自定义类的实例foo，以及自定义类Foo，所以下面的语句都打印True:

1
2
3
4

print(isinstance(3, object))
print(isinstance(int, object))
print(isinstance(foo, object))
print(isinstance(Foo, object))

Foo是对象，那么它的class是什么呢？答案是metaclass：默认情况下，Foo是metaclass type的实例，我们也可以自定义metaclass，见第2节。

如图所示:

• foo is an instance of class Foo.
• Foo is an instance of the type metaclass.
• type is also an instance of the type metaclass, so it is an instance of itself.

自定义metaclass (2)

先看这段代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

#!/usr/bin/python3

class Meta(type):
    def __init__(self, *args, **kwargs):
        print('Meta.__init__ enter')
        super().__init__(*args, **kwargs)

    def __new__(cls, *args, **kwargs):
        print('Meta.__new__ enter')
        return super().__new__(cls, *args, **kwargs)

    def __call__(self, *args, **kwargs):
        print('Meta.__call__ enter')
        return super().__call__(*args, **kwargs)


class Foo:
    def __init__(self):
        print('Foo.__init__ enter')
        super().__init__()

    def __new__(cls):
        print('Foo.__new__ enter')
        return super().__new__(cls)

    def __call__(self, *args, **kwargs):
        print('Foo.__call__ enter')


class Bar(Foo, metaclass=Meta):
    def __init__(self):
        print('Bar.__init__ enter')
        super().__init__()

    def __new__(cls):
        print('Bar.__new__ enter')
        return super().__new__(cls)

    def __call__(self, *args, **kwargs):
        print('Bar.__call__ enter')


if __name__ == '__main__':
    print('================== main enter ==================')
    foo = Foo()
    bar = Bar()

用图形表示：

当python解析器遇见Foo()语句时：

• 调用Foo的metaclass的__call__，即调用type.__call__，注意：不是`Foo.call.
• Foo.__call__调用：
  • Foo.__new__
  • Foo.__init__

当python解析器遇见Bar()语句时：

• 调用Bar的metaclass的__call__，即调用Meta.__call__，注意：不是`Bar.call.
• Meta.__call__调用：
  • Bar.__new__
  • Bar.__init__

注意区分object-class的从属关系和class-class的继承关系：

• 找__call__时，沿着object-class的从属关系往上走。
• 找__new__和__init__时才进入继承领域，沿着class-class的继承关系往上走。

所以，上面代码的输出是：

1
2
3
4
5
6
7
8
9
10
11

Meta.__new__ enter
Meta.__init__ enter
================== main enter ==================
                     <--- Foo()，这里调用type.__call__
Foo.__new__ enter
Foo.__init__ enter
Meta.__call__ enter  <--- Bar()
Bar.__new__ enter
Foo.__new__ enter    //继承，调用parent-class的__new__
Bar.__init__ enter
Foo.__init__ enter   //继承，调用parent-class的__init__

上面重点解释了foo和bar的创建过程。但不是说一切皆对象吗？Foo和Bar本身是怎么创建的呢？注意main enter之前还有两行输出，它们就是创建Bar的过程，其实和创建实例的过程一样：

• 先调用Bar的class的metaclass的__call__：沿着object-class的从属关系往上走，Bar的class是Meta(就是Bar的metaclass)，Meta的metaclass是type，所以这一步调用type.__call__，什么也没有打印。
• type.__call__调用：
  • Meta.__new__
  • Meta.__init__

这里注意class的class就是它的metclass；这应该也是meta一词的来源。
另外，Foo的的metaclass是type，我们没有机会在它里面添加打印语句，所以Foo的创建什么都没打印。

通过type动态创建class (3)

下面两种方式是等价的：

1
2
3
4
5
6
7
8
9
10
11

def plus(self, i):
    self.m = self.m + i
    print(self.m)


def minus(self, i):
    self.m = self.m - i
    print(self.m)


Foo = type('Foo', (), {'add': plus, 'sub': minus})

1
2
3
4
5
6
7
8
9
10

class Foo:
    m = 100

    def add(self, i):
        self.m = self.m + i
        print(self.m)

    def sub(self, i):
        self.m = self.m - i
        print(self.m)

详细介绍eBPF的实现原理。

本文简要介绍一下mpstat命令的使用，并补充一些CPU中断知识。

详细梳理Basic Paxos算法，试图找出从Paxos到Raft的演进过程。

Version代表LevelDB数据库的一个固化状态；VersionEdit代表对Version的修改或编辑操作；VersionSet维护一系列Version。

LevelDB中Table是一个比较复杂的结构。Block负责有序kv-pair的存储、查询及迭代；Table利用Block构造上一层的结构，包含Data Block, Index Block, Filter Block等，并管理这些Block之间的关系。本篇记录这些琐碎细节。

Linux中用户态程序总是preemptible的，内核使用clock tick中断用户态程序切换到别的线程，不用等待用户态程序主动放弃CPU。但在Kernel Preemption被引入之前，一个线程进入内核态，不放弃CPU也不返回用户态就能一直占用CPU。直到linux 2.6才引入Kernel Preemption。本文的主要目的是介绍Linux的三种Kernel Preemption模式。但介绍Voluntary Preemption的时候，也需要把might_sleep搞清楚。

本文基于linux kernel 3.19.8版本梳理一下读文件的过程，从vfs开始，到发送请求给block层结束。