08 - Distributed OSes: Plan 9 & LegoOS

Plan 9 From Bell Labs

Motivation

• Unix is not a good fit:
• trends: workstations instead of mainframe, many unix workstations
• new technology: graphics & networking (which is not considered in unix design, thus poorly integrated)
• administering & management distributed unix workstations is difficult
• not designed for distributed settings

Plan 9 Overview

• Benifits of above structure:
• dedicated CPU & File server for computing and storage.
• better sharing: can access information in other machines via network
• easier upgrade: just need to upgrade several servers, cost is amortized by the amounts of terminals.

Unified Abstraction — treating everything as files

• interact with objects by a file-like interface

• device: keyboard, mouse, display, console, networking. control / transfer data with device by performing I/O on their associated files.

• extends to distributed settings: access remote services by file interfaces.

• process: /proc

Namespace

• users could construct private local namespace, export to remote server when using remote services

• global conventions for easier use.

• mount, bind, unmount; union directory

9P Protocol

• network protocol

• accessing everything as files

• similar to HTTP protocol today: do not interpret domain logic

Storage Hierarchy

• memory: for caching

• disk

• WORM (Write-Once-Read-Many)
• least reliable
• need a lot of memory capacity
• slow to r/w, takes long time to repopulate disks.

Processes

• rfork: fork with fine-grained sharing mechanisms

• Alef programming language: language support for parellel programming, has influence on the Go programming lanugage.

Summary

• unified abstraction: everything is a file, unified interface

• 9P protocol

• local namespaces with global conventions (still useful today)

LegoOS: A Disseminated, Distributed OS for Hardware Resource Disaggregation

Monolithic Server & Hardware Disaggregation

• Monolithic Server: a server being the unit of deployment.
• poor resource utilization
• poor isolation: a part of server fails may affect others in the same server
• poor elasticity: hard to adjust hardware
• poort domain-specific hardware support: hard to add new domain-specific hardware

• Hardware Disaggregration: break monolithic servers to disaggregated, network-attached hardware components
• fine-grained isolation: easier failure handling → just switch to another components of the same type. (+)
• improve resource utilization (+)
• easier supporting / upgrading specialized hardware (+)
• improve resouce elasticity (+)
• lose some performance because of communication overhead (-)

LegoOS Design

• pComp(processor components), mComp (memory components), sComp (storage components)

• each components has a monitor (in general sense, can be called as a manager or controller also.)

Resource Management

• 2 level approach: global manager do coarse-grained management decision, component controller do fine-grained decision

• example: Memory management
• global decisions at vRegion granularity
• each mComponent manage its vRegions

Performance Optimization

• small memory as cache on pComponents to accelerate memory access

Implementation

• No disaggregated hardware now: emulate with commodity server by limiting the resource usage

Evaluation

• evaluate impact of network communication

• metrics: slowdown of new model
• ~2x for small working set

• micro + macro benchmarks

Ideas applied today

• disaggregated storage is the most common

Summary

• splitkernel architecture: disaggregated OS

• extreme point in design space: split hardware components completely (one step further than just split larger disks and more powerful cpu to servers, as in V, Sprite and P9)