Python 旋风之旅 中文翻译版

Python 官方教程的开头是这样写的：“Python 是一门既容易上手又强大的编程语言。”随着大数据、机器学习与人工智能的兴起，Python 语言正在受到越来越广泛的关注和应用。对爱好Python 的人来说，人生苦短，Python 当歌！简洁轻松的语法，开箱即用的模块，强大快乐的社区，总可以快速构建出简单高效的解决方案。对于希望快速入门 Python 语言的读者们来说，市面上众多像字典一般厚厚的 Python 书籍常常让人感到头皮发麻。为了让对 Python 语言感兴趣的读者能快速入门这门语言，而非过分纠结于Python 细枝末节之处，我们选择并翻译了这本Python 教程：A Whirlwind Tour of Python。

这本仅 100 余页的小册子是一本精简的 Python 编程入门教程，介绍了 Python 语言的核心特性以及数据科学领域内一些 Python 第三方扩展包的实际应用，目的是让熟悉其他编程语言的数据科学家快速学习 Python。本书适合从未接触过 Python 语言但对其他编程语言有一定了解的读者，建议读者跟随本书亲自动手完成每一个例子。

本书翻译力求在原版基础上做到精炼、全面、准确地介绍 Python 语言。在书中难于理解的部分我们添加了译者注，希望能便于读者理解。

在线版可以从这里下载。

Beginning Level

Python Crash Course: A Hands-On Project Based Introduction to Programming

By Eric Matthes, No Starch Press, Inc., 2016

《Python 编程：从入门到实践》

袁国忠译，人民邮电出版社，2016

Learn Python 3 The Hard Way

By Zed A. Shaw, Addison-Wesley, 2017

《“笨办法”学 Python（第 3 版）》

王巍巍译，人民邮电出版社，2014

Python Pocket Reference (5th Edition)

By Mark Lutz, O'Reilly Media, 2014

《Python 袖珍指南（第 5 版）》

侯荣涛译，中国电力出版社，2015

Python Essential Reference (4th Edition)

By David M. Beazley, Addison-Wesley Professional, 2009

《Python 参考手册（第 4 版）》

谢俊、杨越、高伟译，人民邮电出版社，2010

Functional Programming in Python

By David Mertz, O'Reilly Media, 2015

Intermediate Level

Python Cookbook (3rd Edition)

By David M. Beazley and Brian K. Jones, O'Reilly Media, 2013

《Python Cookbook 中文版（第 3 版）》

陈舸译，人民邮电出版社，2015

Introducing Python Modern Computing in Simple Packages

By Bill Lubanovic, O'Reilly Media, 2014

《Python 语言及其应用》

丁嘉瑞、梁杰、禹常隆译，人民邮电出版社，2016

Effective Python: 59 Specific Ways to Writer Better Python

By Brett Slatkin, Addison-Wesley, 2015

《Effective Python：编写高质量 Python 代码的 59 个有效方法》

爱飞翔译，机械工业出版社，2016

Advanced Level

Fluent Python

By Luciano Ramalho, O'Reilly Media, 2015

《流畅的Python》

安道、吴珂译，人民邮电出版社，2017

High Performance Python

By Micha Gorelick and Ian Ozsvald, O'Reilly Media, 2014

《Python 高性能编程》

胡世杰、徐旭彬译，人民邮电出版社，2017

Python in Specialized Applications

Computer Vision

Programming Computer Vision with Python

By Jan Erik Solem, O'Reilly Media, 2012

《Python 计算机视觉编程》

朱文涛、袁勇译，人民邮电出版社，2014

Deep Learning with Python

By Francois Chollet, Manning Publications, 2017

Data Science

Web Scraping with Python (2nd Edition)

By Ryan Mitchell, O'Reilly Media, 2018

《Python 网络数据采集》

陶俊杰、陈小莉译，人民邮电出版社，2016

Python Data Science Handbook: Tools and Techniques for Developers

By Jake VanderPlas, O'Reilly Media, 2016

《Python 数据科学手册》

陶俊杰、陈小莉译，人民邮电出版社，2018

Python for Data Analysis (2nd Edition)

By William McKinney, O'Reilly Media, 2017

《利用 Python 进行数据分析》

唐学韬译，机械工业出版社，2013

Finance

Python for Finance: Analyze Big Financial Data

Yves Hilpisch, O'Reilly Media, 2014

《Python 金融大数据分析》

姚军译，人民邮电出版社，2015

Scientific Computation

Effective Computation in Physics (Python)

By Anthony Scopatz and Kathryn D. Huff, O'Reilly Media, 2015

《Python 物理学高效计算》

孙波翔译，人民邮电出版社，2018

MIT 6.828: JOS Lab

Lab 5: File System, Spawn and Shell

Questions

Answer the following questions:

1. Do you have to do anything else to ensure that this I/O privilege setting is saved and restored properly when you subsequently switch from one environment to another? Why?

1. Nothing else needs to be done. When switching from the file system environment to another environment, the two trap frames are completely separate. If the file system environment forks, then the child would inherit the ELFLAGS register value and hence the I/O privilege.

MIT 6.828: JOS Lab

Lab 4: Preemptive Multitasking

Part A: Multiprocessor Support and Cooperative Multitasking

Questions

Answer the following questions:

1. Compare kern/mpentry.S side by side with boot/boot.S. Bearing in mind that kern/mpentry.S is compiled and linked to run above KERNBASE just like everything else in the kernel, what is the purpose of macro MPBOOTPHYS? Why is it necessary in kern/mpentry.S but not in boot/boot.S? In other words, what could go wrong if it were omitted in kern/mpentry.S?
   Hint: recall the differences between the link address and the load address that we have discussed in Lab 1.

1. The MPBOOTPHYS macro is needed because mpentry.S is linked at high addresses but gets loaded by boot_aps() at the low address MPENTRY_ADDR. The bootloader doesn't need a macro like this because it is linked and loaded at the same low address (0x00007c00).

Questions

2. It seems that using the big kernel lock guarantees that only one CPU can run the kernel code at a time. Why do we still need separate kernel stacks for each CPU? Describe a scenario in which using a shared kernel stack will go wrong, even with the protection of the big kernel lock.

2. During a trap/interrupt, the trapframe is pushed onto the stack without holding the kernel lock. Say CPU 1 enters the kernel on a system call and while it is in the kernel, CPU 2 attempts to enter the kernel on a timer interrupt. CPU 2 can't enter the kernel, it will be spinning at the lock in trap(). However, it will have pushed its trap frame on top of the trap frame already pushed by CPU 1. This of course means that when CPU 1 returns to user mode, it will pop off CPU 2's frame and return in that environment instead of its own.

Questions

3. In your implementation of env_run() you should have called lcr3(). Before and after the call to lcr3(), your code makes references (at least it should) to the variable e, the argument to env_run. Upon loading the %cr3 register, the addressing context used by the MMU is instantly changed. But a virtual address (namely e) has meaning relative to a given address context--the address context specifies the physical address to which the virtual address maps. Why can the pointer e be dereferenced both before and after the addressing switch?
4. Whenever the kernel switches from one environment to another, it must ensure the old environment's registers are saved so they can be restored properly later. Why? Where does this happen?

3. Because all environment page directories share certain mappings. The envs array is allocated and mapped to UENVS in mem_init() and those mappings are then copied to all new page directories in env_setup_vm().
4. It's a like the caller/callee save calling convention except here it is for interrupting the whole process. Processes keep temporary data and variables in registers and the process assumes that when it returns, all those values will still be there. The registers are saved on the user environment's stack as part of the trapframe constructed by the int instruction and the code in alltraps. To restore the state of a new process, JOS uses the env_pop_tf() function, which switches first to the new process' stack and the pops all the registers in place.

MIT 6.828: JOS Lab

Lab 3

Part A: User Environments and Exception Handling

Questions

Answer the following questions:

1. What is the purpose of having an individual handler function for each exception/interrupt? (i.e., if all exceptions/interrupts were delivered to the same handler, what feature that exists in the current implementation could not be provided?)
2. Did you have to do anything to make the user/softint program behave correctly? The grade script expects it to produce a general protection fault (trap 13), but softint's code says int $14. Why should this produce interrupt vector 13? What happens if the kernel actually allows softint'sint $14 instruction to invoke the kernel's page fault handler (which is interrupt vector 14)?

1. To maintain privilege isolation between kernel and user programs. If all interrupts were delivered to the same handler, there would be no way to assign different permissions to different handlers.
2. No. softint produces interrupt 13 because it doesn't have permissions to invoke a page fault. And thus the processor catches that mismatch and then raises a general protection fault.

Part B: Page Faults, Breakpoints Exceptions, and System Calls

Questions

3. The break point test case will either generate a break point exception or a general protection fault depending on how you initialized the break point entry in the IDT (i.e., your call to SETGATE from trap_init). Why? How do you need to set it up in order to get the breakpoint exception to work as specified above and what incorrect setup would cause it to trigger a general protection fault?
4. What do you think is the point of these mechanisms, particularly in light of what the user/softint test program does?

3. Because the permission set for the break point exception is user. If set to kernel, the CPU will raise a general protection fault.
4. The point of these mechanisms is to restrict the influence user level code can have on the kernel. Users can set break points but can not directly manipulate virtual memories.

MIT 6.828: JOS Lab

Lab 2

Questions

1. Assuming that the following JOS kernel code is correct, what type should variable x have, uintptr_t or physaddr_t?

   mystery_t x;
   char* value = return_a_pointer();
   *value = 10;
   x = (mystery_t) value;
   

The type of x will have to be uintptr_t. Postulating that the code is correct, since the code snippet dereferences the pointer and writes to it, return_a_pointer() can not return a physical address bypassing the MMU translation.

Questions

2. What entries (rows) in the page directory have been filled in at this point? What addresses do they map and where do they point? In other words, fill out this table as much as possible:

   Entry	Base Virtual Address	Points to (logically):
   1023	?	Page table for top 4MB of phys memory
   1022	?	?
   .	?	?
   .	?	?
   .	?	?
   2	0x00800000	?
   1	0x00400000	?
   0	0x00000000	[see next question]

3. We have placed the kernel and user environment in the same address space. Why will user programs not be able to read or write the kernel's memory? What specific mechanisms protect the kernel memory?

4. What is the maximum amount of physical memory that this operating system can support? Why?

5. How much space overhead is there for managing memory, if we actually had the maximum amount of physical memory? How is this overhead broken down?

6. Revisit the page table setup in kern/entry.S and kern/entrypgdir.c. Immediately after we turn on paging, EIP is still a low number (a little over 1MB). At what point do we transition to running at an EIP above KERNBASE? What makes it possible for us to continue executing at a low EIP between when we enable paging and when we begin running at an EIP above KERNBASE? Why is this transition necessary?

2. Three major sections of the linear address space have so far been mapped: (1) the pages data structure, (2) the kernel stack, (3) the space above KERNBASE.

3. Isolation in the virtual address space is achieved by the permission bits of both page directory and page table entries. Those permission bits, located in the bottom 12 bits of every page table/directory entry, are checked by hardware.

   To be more specific, pages are assigned either one of two privilege levels: supervisor or user. The current level is related to the CPL (current privilege level). A CPL of 0, 1 or 2 is equivalent to supervisor and a CPL of 3 is equivalent to user level. When executing in supervisor mode, all pages are accessible but when executing in user mode only other user mode pages are accessible.

4. 4GB, which is the maximum number of bytes addressable using 32 bits.

5. (1) storing the page directory and page tables, (2) storing the pages array, and (3) having a chunk of memory under the kernel stack not mapped so as to trigger a page fault in case the stack goes over.

6. The transition to a high EIP happens with the jump to the relocated tag. In entrygdir.c, virtual address [0, 4MB) has been mapped to physical address [0, 4MB). Therefore, it is viable that we continue executing at a low EIP after enabling paging. The transition is necessary because the rest of the kernel is linked at high addresses.

MIT 6.828: JOS Lab

Lab 1

Part 2: The Bootloader

Exercise 3:

Take a look at the lab tools guide, especially the section on GDB commands. Even if you're familiar with GDB, this includes some esoteric GDB commands that are useful for OS work.

Set a breakpoint at address 0x7c00, which is where the boot sector will be loaded. Continue execution until that breakpoint. Trace through the code in boot/boot.S, using the source code and the disassembly file obj/boot/boot.asm to keep track of where you are. Also use the x/i command in GDB to disassemble sequences of instructions in the boot loader, and compare the original boot loader source code with both the disassembly in obj/boot/boot.asmand GDB.

Trace into bootmain() in boot/main.c, and then into readsect(). Identify the exact assembly instructions that correspond to each of the statements in readsect(). Trace through the rest of readsect() and back out into bootmain(), and identify the begin and end of the for loop that reads the remaining sectors of the kernel from the disk. Find out what code will run when the loop is finished, set a breakpoint there, and continue to that breakpoint. Then step through the remainder of the boot loader.

Be able to answer the following questions:

• At what point does the processor start executing 32-bit code? What exactly causes the switch from 16- to 32-bit mode?
• What is the last instruction of the boot loader executed, and what is the first instruction of the kernel it just loaded?
• Where is the first instruction of the kernel?
• How does the boot loader decide how many sectors it must read in order to fetch the entire kernel from disk? Where does it find this information?

1. Switching to 32-bit protected mode requires two prerequisites: (1) global descriptor table needs to be loaded so that it is possible to switch to a code segment that supports 32 bits and (2) the first bit of the CR0 register needs to be enabled.

   The exact instruction starting executing 32-bit code is a long jump to a different code segment:0x7c2d: ljmp $0x8,$0x7c32.

2. The last instruction of the boot loader is a call to the kernel after loading it into memory:

   ((void (*)(void)) (ELFHDR->e_entry))();
   

   The first instruction executed by the kernel is:

   0x10000c: movw $0x1234,0x472
   

3. The first instruction of the kernel lies in 0x10000c (load address).

4. Through objdump -x obj/kern/kernel it is possible to figure out how many sections and programs in the ELF header of the kernel.

Exercise 4:

Read about programming with pointers in C. The best reference for the C language is The C Programming Language by Brian Kernighan and Dennis Ritchie (known as 'K&R'). We recommend that students purchase this book (here is an Amazon Link) or find one of MIT's 7 copies.

Read 5.1 (Pointers and Addresses) through 5.5 (Character Pointers and Functions) in K&R. Then download the code for pointers.c, run it, and make sure you understand where all of the printed values come from. In particular, make sure you understand where the pointer addresses in lines 1 and 6 come from, how all the values in lines 2 through 4 get there, and why the values printed in line 5 are seemingly corrupted.

There are other references on pointers in C (e.g., A tutorial by Ted Jensen that cites K&R heavily), though not as strongly recommended.

Warning: Unless you are already thoroughly versed in C, do not skip or even skim this reading exercise. If you do not really understand pointers in C, you will suffer untold pain and misery in subsequent labs, and then eventually come to understand them the hard way. Trust us; you don't want to find out what "the hard way" is.

The only point worth mentioning is that3[c] is the syntactic sugar for *(c + 3).

Exercise 5:

Trace through the first few instructions of the boot loader again and identify the first instruction that would "break" or otherwise do the wrong thing if you were to get the boot loader's link address wrong. Then change the link address in boot/Makefrag to something wrong, run make clean, recompile the lab with make, and trace into the boot loader again to see what happens. Don't forget to change the link address back and make clean again afterward!

The first erroneous instruction is the long jump switching from 16 bits to 32 bits caused by inconsistency of the load address and the link address.

Exercise 6:

We can examine memory using GDB's x command. The GDB manual has full details, but for now, it is enough to know that the command x/Nx ADDR prints N words of memory at ADDR. (Note that both 'x's in the command are lowercase.) Warning: The size of a word is not a universal standard. In GNU assembly, a word is two bytes (the 'w' in xorw, which stands for word, means 2 bytes).

Reset the machine (exit QEMU/GDB and start them again). Examine the 8 words of memory at 0x00100000 at the point the BIOS enters the boot loader, and then again at the point the boot loader enters the kernel. Why are they different? What is there at the second breakpoint? (You do not really need to use QEMU to answer this question. Just think.)

When entering the bootloader, nothing has yet been loaded in 0x0010000, so that memory is empty. Then after the bootloader loading the kernel, 0x0010000 and onwards contain the first several instructions of the kernel.

Part 3: The Kernel

Exercise 7:

Use QEMU and GDB to trace into the JOS kernel and stop at the movl %eax, %cr0. Examine memory at 0x00100000 and at 0xf0100000. Now, single step over that instruction using the stepi GDB command. Again, examine memory at 0x00100000 and at 0xf0100000. Make sure you understand what just happened.

What is the first instruction after the new mapping is established that would fail to work properly if the mapping weren't in place? Comment out the movl %eax, %cr0 in kern/entry.S, trace into it, and see if you were right.

Prior to enabling paging, 0x00100000 consists kernel instructions while 0xf0100000 remains empty. After paging is enabled, virtual addresses in the range 0xf0000000 through 0xf0400000 have been translated into physical addresses 0x00000000 through 0x00400000. Thus, 0xf0100000 points to the same memory as 0x00100000.

If commenting out the instruction enabling paging, the following jump fails:

mov	$relocated, %eax
jmp	*%eax

Exercise 8:

We have omitted a small fragment of code - the code necessary to print octal numbers using patterns of the form "%o". Find and fill in this code fragment.

Be able to answer the following questions:

1. Explain the interface between printf.c and console.c. Specifically, what function does console.c export? How is this function used by printf.c?

2. Explain the following from console.c:

   if (crt_pos >= CRT_SIZE) {
       int i;
       memmove(crt_buf, crt_buf + CRT_COLS, (CRT_SIZE - CRT_COLS) * sizeof(uint16_t));
       for (i = CRT_SIZE - CRT_COLS; i < CRT_SIZE; i++)
           crt_buf[i] = 0x0700 | ' ';
       crt_pos -= CRT_COLS;
   }
   

3. For the following questions you might wish to consult the notes for Lecture 2. These notes cover GCC's calling convention on the x86.

   Trace the execution of the following code step-by-step:

   int x = 1, y = 3, z = 4;
   cprintf("x %d, y %x, z %d\n", x, y, z);
   

   • In the call to cprintf(), to what does fmt point? To what does ap point?
   • List (in order of execution) each call to cons_putc, va_arg, and vcprintf. For cons_putc, list its argument as well. For va_arg, list what ap points to before and after the call. For vcprintf list the values of its two arguments.

4. Run the following code.

   unsigned int i = 0x00646c72;
   cprintf("H%x Wo%s", 57616, &i);
   

   What is the output? Explain how this output is arrived at in the step-by-step manner of the previous exercise. Here's an ASCII table that maps bytes to characters.

   The output depends on that fact that the x86 is little-endian. If the x86 were instead big-endian what would you set i to in order to yield the same output? Would you need to change 57616 to a different value?

   Here's a description of little- and big-endian and a more whimsical description.

5. In the following code, what is going to be printed after 'y='? (note: the answer is not a specific value.) Why does this happen?

   cprintf("x=%d y=%d", 3);
   

6. Let's say that GCC changed its calling convention so that it pushed arguments on the stack in declaration order, so that the last argument is pushed last. How would you have to change cprintf or its interface so that it would still be possible to pass it a variable number of arguments?

1. printf.c and console.c interface via the cputchar() function. printf.c wraps cputchar() with the putch() function that increments the count argument passed in in addition to calling out to cputchar().cputchar() is responsible for putting a single character to the console and that is also the purpose of putch().

2. When crt_pos > CRT_SIZE, then the last character written is outside the current display buffer. The loop shifts back the buffer 80 columns, essentially printing a new empty line on the screen.

3. fmt points to the format string, namely x %d, y %x, z %d\n, and ap points to a va_list.

4. The output is He110 World. The pointer i points to the sequence of bytes: 0x72 0x6c 0x64 0x00 and the interpretation of these bytes as characters is rld\0.

   If the machine is big endian then the i variable should be changed to 0x726c6400. But 57616 would not need to change since its value fits in one byte.

5. The value should be the 4 bytes on the stack.

6. va_start, va_arg and va_end macros need to be changed to decrease the point every time they fetching a new argument from the stack.

Exercise 9:

Determine where the kernel initializes its stack, and exactly where in memory its stack is located. How does the kernel reserve space for its stack? And at which "end" of this reserved area is the stack pointer initialized to point to?

Setting up the stack is through the following instructions:

movl	$0x0,%ebp
movl	$(bootstacktop),%esp

bootstacktop is a label in entry.S to a variable of size KSTKSIZE. (The space is allocated via the assembler .space directive.) $ESP is set to the top (highest address) end of the reserved space, namely 0xf0110000. Therefore the stack starts at 0xf0110000 and ends at 0xf0110000 - 4 * 4096 = 0xf010c000.

Exercise 10:

To become familiar with the C calling conventions on the x86, find the address of the test_backtrace function in obj/kern/kernel.asm, set a breakpoint there, and examine what happens each time it gets called after the kernel starts. How many 32-bit words does each recursive nesting level of test_backtrace push on the stack, and what are those words?

Note that, for this exercise to work properly, you should be using the patched version of QEMU available on the tools page or on Athena. Otherwise, you'll have to manually translate all breakpoint and memory addresses to linear addresses.

By examing obj/kern/kernel.asm, the following operations change the stack:

1. According to x86-32 register saving conventions, $ebp is a callee-saved register, and thus it needs to be saved prior to changing its value.

   push %ebp
   mov %esp, %ebp
   

2. According to x86-32 register saving conventions, $ebx is also a callee-saved register.

   push %ebx
   

3. The following instruction reserves the space of 0x14 bytes to save temporories including the arguments of the callee function.

   sub $0x14, %esp
   

4. When calls to another function, the call instruction implicitly push $eip register on the stack.

   call f01000dd <test_backtrace>
   

In summary, words of $4 + 4 + 20 + 4 = 32$ bytes have been pushed on the stack in each recursive nesting call of test_backtrace.

1. Concurrency Models #0: Introduction
2. Concurrency Models #1: Parallel Algorithms

Preface

Concurrency is nothing new, but it becomes one of hottest topics among sub-disciplines of CS nowadays. It is also the key that unlocks responsiveness, fault tolerance, efficiency, and simplicity in a system. Next semester (Fall 2017) I will share some thoughts on concurrency models in Google Camp, and I plan to write a series of tutorials in my blog simultaneously. Related lecture slides can be found here.

This series of tutorials will focus mainly on concurrency programming theories and practices.

Prerequisites

Code examples will be drawn from a number of programming languages. To make use of a specific concurrency framework, it requires a specific language. Because the theme is about concurrency, there are going to be some aspects of these languages that I will use without explaining in detail. In order to fully understand them, you may required to have basic knowledge of syntax and features of the following programming languages:

• Java
• C++
• Clojure (or Haskell)
• Elixir (preferred Erlang)
• Python
• Go

Besides, basic knowledge about OS is preferred as well.

Topics to be Covered

I am going to cover some approaches that are already mainstream in the concurrent world, and also reveal knowledge and theory behind them. To be specific, topics below will be discussed sequentially:

1. Parallel Algorithms
2. Multithreading
3. Asynchronous Programming
4. Communicating Sequential Processes
5. Functional Programming
6. Actors
7. The Lambda Architecture

Concurrency: Beyond Multiple Cores

Let’s get down to business. The presence of concurrency models originates in what is so called “multicore crisis”. Multicore crisis is found of the fact that microprocessors have stopped getting faster every 18 months. Instead of gaining faster clock speed, Moore’s law continues to deliver more transistors per chip, and thus we have computers with increasingly cores.

The days of waiting for faster hardware is (long) gone. Therefore, to make software systems that perform efficiently, you need to incorporate concurrency into your system designs.

Concurrent or Parallel?

The most common question to ask is what is the difference between concurrency and parallelism. Actually, they refer to different things, though they are often used interchangeably.

When we execute a program, we create a process. A sequential program has a single thread of control, while a concurrent program has multiple threads of control. A single computer can have multiple processes running at once. If that machine has a single processor, then the illusion of multiple processes running at once is just that: an illusion. Thus, if a machine has more than a single processor, then true parallelism can occur: you can have \(N\) processes running simultaneously on a machine that has \(N\) processors.

In summary:

• A concurrent program has multiple logical threads of control. These threads may or may not run in parallel.
• A parallel program potentially runs more quickly than a sequential program by executing different parts of the computation in parallel. It may or may not have more than one logical thread of control.

An alternative way of thinking about it is that concurrency is part of the problem domain—multiple events can happen at the same time. Parallelism, by contrast, is an aspect of the solution domain—to make programs faster, it is necessary to design a program such that computations occur simultaneously.

Features of Concurrency

Concurrency is great more than just exploiting parallelism. As I have said before, concurrency is the key that unlocks responsiveness, fault tolerance, efficiency, and simplicity in a system.

Responsiveness

Concurrency is the key to responsive systems. Responsiveness becomes a more and more indispensable quantity of a software. Basically, it means that we do not block waiting for a time-consuming task, such as the I/O operation. In the server side, concurrency enables a web server handling requests from enormous clients, and guarantees that a single slow request does not hold up others. In the client side, the main/GUI thread of the process will not be blocked and should be free to handle user events.

Efficiency

The concept of efficiency for concurrency can be characterized both in terms of probability and in terms of time. A process is more efficient than others if it is able to perform certain computations with a greater probability or by taking a lesser time. Concurrency brings efficiency because it utilizes the computational resources of the system.

When we talk about efficiency, we are not just dealing with speed. Efficiency should also weigh the CPU and memory overhead and the cost to ensure data consistency. For example, if an application benefits from concurrency but require an elaborate and computationally expensive process to ensure data consistency, then the entire strategy should be evaluated again.

Fault Tolerance

Concurrency enables resilient, or fault-tolerant, through independence and fault detection. Independence is important because a failure in one task should not be able to bring down another, and also a power outage at one site should not result in global downtime. And fault detection is critical so that when a task fails, a separate task is notified so that it can take remedial action. As comparison, sequential software can never be as resilient as concurrent software.

Simplicity

It is quite controversial for programmers that concurrency brings simplicity to programs, especially for those who suffer from debugging multithreaded programs. Generally, the sequential expression is considered to be simple and reliable, whereas the expression of concurrency is perceived to be complex, non-deterministic and difficult. However, the perception of the simplicity of sequentiality and the complexity of concurrency is an illusion. When written in the right language with the right tools, a concurrent solution is usually simpler and clearer than its sequential equivalent version.

Why Worry about Concurrency?

“Concurrency is hard and I have only ever needed single-threaded programs. Why should I care
about it?” A newbie always argues that. However, we cannot indulge in such fantasy.

With the frequent changing environment, multicore computers and clusters are on the rise to fulfill the needs. And growth rates for chip clock speed are flat, thus you can not wait for another 18 months for a twice speed-up anymore. Instead, the chips are going to be wider: more cores, wider memory bus, more memory. Under such circumstances, a single-threaded application is not going to see any significant performance gains from new devices. In contrast, software will only see performance gains if it is designed to do more work in parallel, as the number of available processors increase. So, the application’s computations must be amenable to parallelization. It must be possible to break its work into tasks that can run at the same time with no need to coordinate with each other.

Moreover, concurrent programming is becoming hard to ignore. Lots of application domains in which concurrency is the norm.

Drawbacks of Concurrency

While concurrency is widespread, it is error prone. Programmer strained for single-threaded programming face unfamiliar problems, such as synchronization, race conditions, deadlocks and “memory barriers”.

Parallel Architecture

Within one single machine, there are many tiers of parallelism.

• Bit-Level Parallelism: The history of computing moves from 8- to 16-, 32-, and now 64-bit architectures. A 64-bit computer can process 32-bit numbers faster than a 16-bit computer.
• Instruction-Level Parallelism: Modern CPUs are highly parallel, using techniques like pipelining, out-of-order execution, and speculative execution. Those techniques will have a profound impact on the behaviors of programs.
• Data Parallelism: Data-parallel (SIMD, for “single instruction, multiple data”) architectures are capable of performing the same operations on a large quantity of data in parallel. For instance, GPUs can process lots of data in parallel with a single instruction.
• Task-Level Parallelism: The architecture of multiple processors lets multiple threads of control execute simultaneously.

Software Architecture Design Choices

When designing a modern application, we now have to ask:

• How many machines are involved?
• What software components will be deployed on each machine?
• For each component, does it need concurrency? If so, how will we achieve that concurrency? Via multiple threads, multiple processes or both?

Case Study: Google Chrome

Google Chrome is now a popular web browser, who has one process per tab (multi-process) and multiple threads handling loading of content within each tab (multi-threaded).

Some advantages of such a design involve stability, speed and security.

• Stability: Single-process, multi-threaded browsers are vulnerable to having a crash in one tab bring down the entire browser.
• Speed: Multi-process browsers can be more responsive due to OS support.
• Security: Exploits in single-process browsers are easier if malware loaded in one tab can grab information contained in another tab. It is much harder to grab information across processes.

References

• Paul Butcher: Seven Concurrency Models in Seven Weeks. The Pragmatic Programmers, LLC. 2014.7.

Slides accompanying with this section can be found here.

跌跌撞撞地走进了人生的第20个年头，在同济大学的时光也已经过去了快两年，终于可以不那么厚着脸皮地说，我也要进入一段崭新的生活了。

过去的一年里，时常思考自己的未来和方向，却事实上浪费了大把的时光，也许因为是倦怠，也许是欠缺深刻的自省，总之因为种种原因并没有做到当初设想的最好。记得在我20岁生日即将到来的晚上，我要做点什么、写点什么，结果一直拖到了现在。我还记得许下的愿望，因为总是操一些闲心，希望20岁以后要留更多的时间给自己。那么，明确了自己的方向，就坚定地走下去。

随着年岁增长，感觉到自己的强迫症和完美主义一直在加剧。不知道是不是因为心有不甘，但是事实的确是这样了，也经常给团队工作带来一些不愉快。Siri 倒是调侃“SXKDZ 出品，必属极品”，又想了一下，好像也没什么不对。所以，就继续坚持吧！只要付出时间和心血的事情，都要尽全力做到最好。

在新的一年里，我也打算开始我的学术之旅。上次毛哥导师来的时候介绍了人生赢家们的故事，也许对我而言暂时是遥不可及的梦想（内心还是并不希望太早进入工业界吧……），但是不管出于出国申请的压力还是拓宽知识面的需求，科研学术之路都要开始走下去。祝我好运。

在我即将20岁高龄的时候，我想祝愿自己：

1. 让自己过得开心；
2. 不荒废时光；
3. 身体健康。

感谢一路以来默默关心、支持我的人们。谢谢你们。“但行好事，莫问前程”。希望我能尽全力，做到最好。

SXKDZ

2017.5.22

As recently as the 1980s, it was normal for "real-world" programmers to laugh at the idea of structured programming languages with such arcane features as "while" loops, as opposed to the jump instructions of assembly languages. How could a program written in a structured language possibly perform well enough, compared to hand-written assembly language? Those crazy academics are at it again, with their nutty theories. Of course, today the abstraction of structured control flow is ubiquitous. Data abstraction was another that seemed esoteric at first but has become completely standard now. Serious programmers today will laugh at anyone who uniformly writes off these techniques that seemed like theoretical distractions in 1980.

当时间倒转到上世纪80年代，对于“真实世界”的程序员来说，嘲笑有着像“while”循环那样晦涩难懂概念的结构化程序设计语言是再正常不过的事了。与手写的汇编语言相比，用一种结构化语言写出的程序有可能表现更加优异吗？那些疯狂的学者再一次坚持他们怪异的理论。当然，今天对结构化控制语句的抽象无处不在。数据抽象是另外一个在最初看上去难以理解，但现在已经成为普遍标准的例子。严肃的程序员在当下会毫不留情的嘲笑那些始终坚持对1980年代被视为理论上不务正业的技巧视而不见的人们了。

Thanks: Foreverbell

This week, I presented a speech on the going computational thinking. This will be the last general-purpose presentation I held in undergraduate school.

I still remember that when I was in the primary school, I was the student who always remained silent. Until the middle stage of junior high school, I was able to convince myself to give a public speech. When I prepare a speech, I always write down verbatim speech script, and carry them together with me when I stand on the platform, in case that I will forget what to say. Nevertheless, this time I dare to say, I performed a good job without such a speech draft.

Life is changing progressively. Big data, machine leaning, artificial intelligence, virtual reality and so many computational techniques are bringing comprehensive changes to our daily life.

As I said in the beginning of the presentation, everyone in this digital world needs to update himself / herself iteratively.

No matter what industry you are in, computing is ubiquitous.

Computational thinking is like mathematical thinking, to help you better understand the digital world.

In the era of digital economy, do you want to be a consumer or a creator?

One way to iterate ourselves is to learn computational thinking and understand programming. So that, everyone can become the creator of the digital age, and even the leader.

Here are the presentation slides:

1 前言

1.1 中文分词问题

中文分词(Chinese Word Segmentation) 指的是将一个汉字序列切分成一个个单独的单词，属于自然语言处理的范畴．对于中文搜索引擎而言，分词是必不可少的一个重要环节．在搜索引擎响应用户的搜索请求时，最重要的并不是呈现出所有网页结果（因为数量太过庞大），而是将与用户输入的内容最相关的内容排列在最前，这称为相关度排序．没有分词技术的出现，计算机并不会认识用户输入的句子中哪些是词语，这样搜索引擎也就无法工作．

我们知道，对于英文来说，单词是自然地以空格作为分节符，而中文却并不如此，即使有句子、段落之间的划分，我们还是无法直接找到词语与词语之间的分界符．从历史原因上分析，这是因为古代汉语除了专有名词，词语以单音词居多，并不需要特别的分词书写，而现代汉语中复音词居多，一个字不再等同于一个词．对此直观的感受是，翻译一篇文言文的字数明显多于原文．

1.2 引入：拼写纠正与贝叶斯定理

如今Word等文字处理软件都能够对用户输入不存在的单词进行纠正，比如用户输入了thew，那么他真正表达的可能是the或者they等，对拼写进行纠正就运用了贝叶斯定理．这个问题用形式化的语言描述：记\(h_{i}\)为对用户真正想输入的单词进行的假设(hypothesis)，这些假设\(h_1, h_2, \dots, h_n\)都属于一个有限而且离散的空间\(H\)（因为单词的总数是有限的）；\(D\)为用户实际输入的单词（观测数据），那么这个问题就变为求

$$
P(h_i|D)
$$

概率中最大的一个．
运用一次贝叶斯公式，可以得到：

$$
P(h_i | D) = \frac{P(h_i) \cdot P(D | h_i)}{P(D)}
$$

对于不同的具体假设\(h_1, h_2, \dots, h_n\)，\(P(D)\)都是一样的，因此在比较不同的\(P(h_i | D)\)时可以忽略它，即只需要知道：

$$
P(h_i | D) \propto P(h_i) \cdot P(D | h_i)
$$

这个公式的抽象意义是：对于给定观测数据，一个假设的好坏取决于这个假设本身独立的可能性大小（先验概率, prior）和这个假设生成观测到数据的可能性大小（似然, likelihood）的乘积．具体到thew这个例子上来说，用户实际想输入the的可能性大小取决于the本身在词汇表中被使用的可能性（频繁程度）大小（先验概率）和想输入the却输入成thew的可能性大小（似然）的乘积．

2 贝叶斯定理在分词问题中的应用

2.1 中文分词形式化描述

分词问题形式化的描述是：令\(X\)为字串（句子），\(Y\)为词串（一种特定的分词假设），问题的解即为使得\(P(Y|X)\)最大的\(Y\)．使用一次贝叶斯公式：

$$
P(Y | X) \propto P(Y) \cdot P(X | Y)
$$

用自然语言来说就是这种分词方式（词串）的可能性乘以这个词串生成句子的可能性．进一步可以看到：可以近似地将\(P(X | Y)\)看作是恒等于\(1\)的，因为任何一种分词方式都可以精确地生成一个句子。因此只要寻找一种分词方式使得词串的概率最大．

2.2 \(N\)元语言模型与局限

对于一个词串\(W_1, W_2, \dots , W_n\)，如何计算其可能性呢？根据联合概率公式展开：

$$
P(W_1, W_2, \dots, W_n) = P(W_1) \cdot P(W_2 | W_1) \cdot \ldots \cdot P(W_n | W_{n - 1}, \dots, W_1)
$$

于是，通过一系列的条件概率（等号右边）的乘积来求整个联合概率．注意到等号右边最后一项，\(W_n\)这个单词依赖于它前\(n - 1\)个单词，称为\(N\)元语言模型(\(N\)-gram)．

对于这类问题，有一些缺陷：

1. 参数空间过大，不可能实用化：
   在依靠概率统计模型中的分词技术是依赖语料库的数据得出分词方式的可能性，然而语料库再大也无法统计出有说服力的\(P(W_n | W_{n - 1}, W_{n - 2}, \dots, W_1)\)值．
2. 数据稀疏（也称高维诅咒）：
   当数据空间维度增加时，空间的体积提高太快而可用数据变得稀疏．通常情况下，为了获得在统计学上正确并且有可靠的结果，用来支撑这一结果所需要的数据量通常随着维数的提高而呈指数级增长．而且，在组织和搜索数据时也有赖于检测对象区域，这些区域中的对象通过相似度属性而形成分组．然而在高维空间中，所有的数据都很稀疏，从很多角度看都不相似，因而平常使用的数据组织策略变得极其低效．

2.3 马尔科夫假设

为了缓解这些缺陷，可以假设句子中一个词的出现仅仅依赖于它前面有限的\(k\)个词［\(k\)一般不超过\(3\)，如果仅依赖于前面一个词，就称为\(2\)元语言模型(\(2\)-gram)，\(2\)元语言模型也称马尔科夫假设或**“有限地平线”假设**，同理有\(3\)元语言模型(\(3\)-gram) 等］．

引入马尔科夫假设后，可以得到：

$$
P(W_1, W_2, \dots, W_n) = P(W_1) \cdot P(W_2 | W_1) \cdot P(W_3 | W_2) \cdot \ldots \cdot P(W_n | W_{n - 1})
$$

这样统计\(P(W_i | W_{i - 1})\)就不再受到数据稀疏问题的困扰了．

如何保证其中任意一个单词出现的概率只受到它前面的\(2\)个或\(3\)个单词的影响呢？别说\(3\)个，有时候一个单词的概率受到上一句话的影响都是绝对可能的．那么为什么这个假设在实际中的表现却不比决策树差呢？有人对此提出了一个理论解释，并且建立了什么时候朴素贝叶斯的效果能够等价于非朴素贝叶斯的充要条件．这个解释的核心就是：有些独立假设在各个分类之间的分布都是均匀的，所以对于似然的相对大小不产生影响；即便不是如此，也有很大的可能性各个独立假设所产生的消极影响或积极影响互相抵消，最终导致结果受到的影响不大．

事实上，统计机器学习方法所统计的东西往往处于相当表层(shallow)的层面，在这个层面机器学习只能看到一些非常表面的现象．越是往表层去，世界就越是繁复多变．从机器学习的角度来说，特征(feature) 就越多，成百上千维度都是可能的．特征一多，数据会变得稀疏，以至于无法使用．而人类的观察水平显然比机器学习的观察水平要更深入一些，为了避免数据稀疏我们不断地发明各种装置（最典型就是显微镜）来帮助我们直接深入到更深层的事物层面去观察更本质的联系，而不是在浅层对表面现象作统计归纳．

2.4 分词案例：朴素贪婪方法与贝叶斯定理的比较

下面通过一个例子来介绍贝叶斯定理与朴素贪婪法分词相比的优点．假设需要分词的句子是“上网站联盟加入我们”．

按照从左到右的贪婪方法分词（即最长匹配机械分词，通过“查字典”得到每次最长能够划分的单词），那么“上网”首先会被划分出来，接着对于剩下的部分“站联盟”，最理想的情况是划分为“站”和“联盟”，最后是“加入”和“我们”．这样的情况下，这个句子就被分词为“上网| 站| 联盟| 加入| 我们”，这显然不是正确的结果．

利用贝叶斯定理，不妨设\(X\)为句子“上网站联盟加入们”，\(Y_i\)为一种分词方案，比如“上 | 网站联盟 | 加入 | 我们”等，需要找到一个\(Y_i\)使得\(P(Y | X)\)最大．假设现在有一个庞大的语料库，包含大量常见单词在文章中的出现次数．那么对于上述贪婪法的分词结果：首先分析“上网”–“站”这一单词对，显然这一对单词对不符合中文的习惯，故其概率为0（通常情况下在统计语言模型中不会设置为0，而是按照平滑的原则设为一个足够小的值），同理，对于“站”–“联盟”也是一样．如果更换一种划分，按照“上| 网站| 联盟”，“上”–“网站”这个词组经常出现，能够在统计文本中找到它的概率，而“网站”–“联盟”也是如此，这样贝叶斯定理就能使得后面这一种更符合中文习惯和直觉的分词方案胜出．

3 优化：动态规划与贝叶斯定理

3.1 时间复杂度分析

下面进一步分析这个问题的具体操作．应用贝叶斯定理后，需要做的就是枚举句子的各种划分，然后在语料库中寻找各划分的统计概率，进而求出在\(2\)元语言模型下概率最大的切分，即最终的分词结果．然而，对于一个长度为\(n\)的句子（即含有\(n\)个汉字），尝试各种不同切分的时间复杂度为\(O(2^n)\)（因为\(n\)个汉字之间有\(n - 1\)个空隙，每个空隙都可以选择分割或者不分割）．对于这样一个指数级的时间复杂度，仅32个汉字的划分就需要进行20亿次计算，一般的计算机很难在理想的时间内得出结果．

3.2 建模：动态规划

对这个过程进行观察，一个词串\(Y\)的概率\(P(Y)\)等于分出第一个词first的概率\(P(first)\)乘以剩下的词串remain的概率\(P(remain)\)：

$$
P(Y) = P(first)
\cdot P(remain)
$$

这个性质对于求解它的子问题remain的划分同样适用，即满足子问题重用性，也就是说可以通过递归解决．进一步看，发现：

$$
P(Y)_{\mathrm{max}} = P(first) \cdot P(remain)_{\mathrm{max}}
$$

也就是说，整个问题的最优解由子问题的最优解构成，满足最优子结构(optimal sub-structure)．

满足以上两个性质，整个算法可以使用动态规划(dynamic programming) 建模．

3.3 具体过程以及时间复杂度分析

具体的做法，仍然以“上网站联盟加入我们”为例．在求解子问题“网站联盟加入我们”时，已经得到它的最优解是“网站 | 联盟 | 加入 | 我们”．那么当求解“上网站联盟加入我们”时，如果尝试在“上”和“网站”之间切分，得“上”“网站联盟加入我们”这两块，只需要“上”这一划分的概率．而“网站联盟加入我们”划分的概率最大值已经求解过了，直接返回，和“上”的概率相乘，得到其中一个候选结果(candidate)．在这一个递归层面继续枚举，求出概率最大的分词方案(optimal candidate)，返回上一层，如此迭代．一般地，动态规划问题都可以通过自底向上(bottom-up) 和记忆化(memoization) 两个策略进行优化，在此不再赘述．特别地，设定最长的单词不会超过\(L\)，那么函数递归的效率为\(O(nL)\)，加上每一次调用递归函数时，需要进行\(O(L)\)次的切分枚举，所以整个算法的复杂度为\(O(nL^2)\)．

如上所述，整个问题的复杂度从指数级降低到了多项式级，算法的效率提高了不少．事实上，这也正是维特比算法(Viterbi algorithm)的实现，用于记录子问题最优解的表叫做维特比表(Viterbi table)．

参考文献

[1] 数学之美番外篇：平凡而又神奇的贝叶斯方法，刘未鹏

[2] 情人节浪漫呈献——贝叶斯与中文分词，李志健

0x00: 背景

最近承接了学校一个内部搜索引擎的项目，学校提供了一台 ThinkServer RD430 机架式服务器，配置如下：

CPU: Xeon E5-2540 @ 2.40GHz x 2

Memory: 92GB

Hard Disk: 24TB

考虑到实际存储需求，需要组一个小集群来跑数据库以提高性能。我选择 VMWare vSphere 6.0 作为宿机操作系统（因为学校提供了正版），下面介绍一下部署的过程。

感谢 swx 和 2老师 的帮助。

0x01: 安装 VMWare ESXi 系统

从正规渠道获得 VMWare ESXi 系统并安装到服务器上。首先遇到的一个麻烦是这个系统不支持我校服务器的 KVM 键盘（似乎是没有驱动？），拿了一个 USB 键盘才可以使用。

接下来是分区的问题。根据 swx 的建议，将一小部分空间划给 ESXi 系统，剩下的空间做一个 RAID 1，虚拟机镜像放在 RAID 上，其他 ISO 等放在非 RAID 上。

安装好 ESXi 系统后即可登录 Web 端进行管理。对于 ESXi 6.0 系统，vSphere Client 很多功能不支持（只支持到 5.0），根据这几天的体验，Web 端速度明显不如客户端，差评。

0x02: 安装软路由 pfSense

由于学院网络限制，这个服务器只接了一根网线并分配了一个固定的 IP 地址。但是 ESXi 的管理界面本身需要占用一个，而其他虚拟机就没有办法获得 IP，因此需要安装一个软路由，作为网关组成一个子网。需要注意的是，由于手滑等一些不可抗力的原因，软路由的配置很可能被改坏，这样的话就无法访问 ESXi 的管理界面。这里可以将 ESXi 的管理界面 IPv4 交给软路由，IPv6 仍然由他自己占着，这样的话 IPv4 玩坏了还可以通过 IPv6 进行补救。

我们选择的软路由系统是基于 FreeBSD 的 pfSense。在安装这个系统时给它配置两块虚拟网卡，其中一块接入一个 VMWare 的虚拟交换机 Public，宿机的物理网卡也接入这个交换机；另一个网卡接入另外一个交换机 Private，其他所有虚拟机也接入这个交换机。

然后开始配置 pfSense。这里只能通过另一台子网下的虚拟机来访问 pfSense 的管理界面。我们再配置一台虚拟机，选择一个 Xubuntu 的 LiveCD，通过 LiveCD 的浏览器访问 pfSense 的管理界面，它的默认用户名是admin，密码是pfsense。

在管理界面中开启 WAN，Public 所接的虚拟网卡端口设置为 WAN，并且 pfSense 监听那个 IP。这样 pfSense 就成为一个网关，物理 IP 的所有流量都经过它。接着开启 LAN，Private 所接的虚拟网卡端口设为 LAN，于是其他虚拟机的流量也经过了 pfSense，并且它们组成一个子网。

最后 ESXi 系统接入 Private 交换机，让它监听一个内网 IP，于是 ESXi 流量也经过 pfSense 了。最后在 pfSense 中设置 NAT 将 443 端口转发到 ESXi 所在的内网 IP，于是就可以通过 443 端口访问 ESXi 了。

注意到 pfSense 自己的管理端口默认 80，可以改动为其他的数值来访问它。