Click to edit Master subtitle style
Trends in Distributed File System
3/20/13
Trends in Distributed File Systems
Hardware
Costs continue to drop at an amazing rate
Rapidly dropping memory costs make it possible to have every larger data bases in main memory
Servers could have entire data set in main memory
Still a cache, so coherence problems 3/20/13 with stable storage still exist
Trends Hardware
Write-Once Read Mostly (WORM)
CD-ROM burners Excellent backup and archiving method Jukeboxes add a level to memory hierarchy
CD - Disk - Main Memory
Increasingly cheap Still fairly slow
3/20/13
Huge capacity networks
Trends Hardware
Specialized hardware for sophisticated systems
Real-time support Distributed synchronization and control Consider modest FPGA assets in a workstation which could be made to do many things
FPGA synergy
Distributed locking and cache block 3/20/13 invalidation
Trends Scaling
Distributed system size strongly affects algorithm choice
Working well for 100 machines means nothing for 10K
Centralized algorithms do not scale well
Often distributed ones do not either
Distributed mutual exclusion
Partitioning and hierarchic organization 3/20/13 often helps
Trends Scaling
Data structures become important with scaling
Linear search easiest and fastest for 10 Self abuse for even 100 Remember break even N for different algorithms
Strict semantics are harder to implement as systems scale
Design Principle: use weakest semantics 3/20/13 that make sense
Trends Wide Area Networking
Virtually all distributed system research has been done in the context of LANs
Considerable changes with WAN context Latency Loss Cost Interaction
3/20/13
WAN access of major economic
Trends Wide Area Networking
Commercialization will create many changes
Service providers have no coherent pricing model IP phones Consistent interface
Economies of scale
Ubiquitous Mobile Environments
Anywhere 3/20/13
Trends Mobility
Network addressing is a big challenge - Mobile IP
May be transparent to distributed computing level
Often seen as highly variable communication bandwidth
Isolated Wireless Wired
3/20/13
Trends Mobility
Rapidly Deployable Radio Network RDRN
Wireless end-user and network nodes Steerable communication beams Self-organizing network structure
Management software is clearly distributed Interesting distributed system issues
Election: 3/20/13
DNS server
Trends Fault Tolerance
Most systems are not fault tolerant
But the general population expects things to work Phone system IP phones? Hardware Communication infrastructure Software
Requires considerable redundancy
Data 3/20/13
Trends Fault Tolerance
Down-times and periodic crashes will become less and less acceptable as computers spread to non-specialists and into commodity functions
ATM machines Microwaves Phone system (IP mode)
3/20/13
Expectations/abilities/costs are not well balanced
Trends Multi-Media
Current data files are rarely more than a few MB
MM files can exceed GB Compression clearly popular because of this and has a fundamental affect on network requirements and economics Significant affect on network traffic Perhaps also on file systems
Video-on-demand
3/20/13
Trends Virtual Environments
Many observers current killer application candidate
Still no pricing model, so how do you make money?
Extremely challenging distribution issues
Communication patterns are dynamic Small messages Fine temporal scale
3/20/13
Summary
Distributed file systems are central to many of the most popular uses of distributed systems Transparency is desirable but often hard to achieve
How do distributed semantics differ from local
Subtly Intermittently (race conditions)
3/20/13
Name spaces must be adapted in
Summary
Session semantics and immutable files are attractive because of performance advantages
Useful in many situations but importantly different Porting poses particular challenges to semantics
Sometimes depends on unpublished behavior
3/20/13
Transactions are attractive
Summary
Stateless servers attractive
Simple and efficient for many applications
WWW
Caching has a huge affect on
Performance Complexity Overhead
3/20/13
Replication and fault tolerance will
