Jun 29

Usenix 2004: Monday notes

Novell: Open Source and Proprietary Software

  • Myths:
  • OSS will destroy the software industry: Breaking vendor lock-in increases competition
  • OSS is substandard, unprofessional, etc.: 60-70% of OSS programmers are professional developers
  • There is no money in OSS:
  • Providing customer satisfaction is profitable
  • Many counterexamples of OSS + proprietary value-add
  • Success in open source
    • Collaboration / customization
    • Customer choice - forces competition on quality, future options
    • Rising value-line - staying ahead of commoditization
    • Novell’s strategy
      • “Trustee of Open Source”
        • Provide distribution for key markets to ensure quality, docs, support, etc.
        • Support Novell maintainers on OSS projects
        • Won’t compete directly with OSS projects
        • Coexistence
          • Focus on value-adds rather than competition / substitution
          • Set and adhere to standards
          • Ecosystem
            • Adjusting pricing, shifting to OSS support / integration
            • Building partnerships with developers
            • Q&A
              • Q: Why isn’t Novell Open-Sourcing everything?
                A: That decision is based on community and customer interest, code quality, etc. If it’s crufty or nobody is interested it may not be worth the cost.

                • Automating System and Storage Configuration

                The CHAMPS system

                Dr. Keller akexk@us.ibm.com

                • MS Project for system administration
                • Underlying goal: try to exploit parallelism where possible
              • Workflow:
                • Administrator submits request for change
                • Scheduler builds task graph to determine dependencies
                • Planner / Scheduler assigns tasks to resources
                  • Prioritize profits, costs, downtime, etc.

                    • Autonomics in System Configuration

                    Paul Anderson dcspaul@inf.ed.ac.uk

                    • “Fabric”: hardware + software + configuration
                    • Autonomic computing: sysadmin moves up from details to policies (analogous to using a compiler)
                    • Goals
                    • Declarative specification
                    • Allows you to monitor conformity, triggers changes
                    • Generic enough to allow the system to change implementation details (e.g. when fail-over changes the list of servers providing a service)
                  • Trust: sysadmins need to be able to rely on the autonomic system before delegating authority
                  • Transparency: useful explanations for actions
                  • Challenges:
                    • Range of software to be configured & formats required
                    • Most software not designed with dynamic reconfiguration
                    • LCFG (http://www.lcfg.org/)
                      • Uses components to handle daemons (translating spec to config, updating service config, etc.)
                      • Problems:
                        • centralized config (See http://www.gridweaver.org/ for a possible answer)
                        • language forces over-detailed specifications and lacks support high-level relationships between machines

                        • Q&A Q: “Am I being automated out of a job?” & difficulty of creating config files A: Config requires detailed knowledge of systems, developer support, etc. e.g. it’s a long way off Q: Is there a standard format for expressing dependencies, config A: Yes but it’s an ongoing project Q: “What is the nature of cost/value tradeoffs” A: Alex: We have constraints, costs can be calculated from state change Paul: LCFG is below that issue - cost/value controls the spec Q: How mature are CHAMPS and LCFG A:

                          Alex: CHAMPS is 10 months away from being production-ready
Paul: LCFG is in use for all config on 1000+ systems plus various other European DataGrid sites

                          • Drowning in a Sea of Data

                          Email Prioritization (HP labs)

                          • QoS for email - simply keep track of good and total counts for n servers (for low values of n) and prioritize into high and low priority processing
                          • Systems start out as junk, must earn their way out
                          • Augment with greylisting and tempfailing (request retry on first sender:source_ip:recipient tuple)

                          Redundancy Elimination Within Large Collections of Files

                          • Goal: stop storing duplicate data across an entire volume
                          • Problems: still being developed, non-trivial performance overhead, and questions about general applicability

                          System Administration: The Big Picture

                          The Human Big Picture

                          Tom Limoncelli, Independent Consultant

                          • Summary: Nobody is sure what we’re supposed doing and we don’t know how to measure what we are doing

                          Large Storage

                          Autonomic Policy-based Storage Management

                          • Storage allocation is largely not a technical problem
                          • Department/group-level duplication of effort
                        • Requirements
                          • Analyze current environment
                          • Translate business requirements to tech requirements
                          • Plan the infrastructure based on policy
                            • Manual planning is hard
                            • Deploy
                            • Validate
                            • Monitor
                            • Refine

                              • Experiences with Large Storage Environments (“Inside an inode, no one can hear you scream”)

                              • Scale does matter - storage capacity is outpacing the tools for managing it
                              • Challenges:
                              • Integrity checking at every point (transfer, storage, backup, app-level (“Did my app really export all of the data?”)
                              • Space management on multiple machines, volumes, etc.
                              • Replication and varying levels of storage (primary, secondary, tertiary, etc.)
                              • Dealing with previous points with non-trivial data-sets
                            • Getting data in
                              • Increasing requirement for incremental delivery, compression
                              • Encryption / access control are becoming extremely important
                              • Synchronizing with multiple, off-site backup sites
                              • Replication
                                • Integrity / backup supplement
                                • Performance
                                • Failover / disaster recovery
                                • Conclusions
                                  • Managing data tools is surprisingly hard
                                  • New ways of thinking about data required:
                                    • Is directly implemented
                                    • Can be independently audited
                                    • Supplies a context monitoring and managing data
                                    • cuscus’s FSM approach seems to be a fairly effective solution
                                    • Q&A
                                      • Tremendous range of inputs - numerous formats (some already compressed, some not at all), the sizes vary by ~12 orders of magnitude.

                                        • SCO Debate

                                        • Reasons to be suspicious:
                                        • Doubtful that SCO actually does have the control over IBM-owned code which they claim
                                        • Even if they did, Caldera (SCO’s subsidiary) was distributing the same code under the GPL for years and actively contributed code to the Linux kernel
                                        • Dennis Ritchie: they had a non-commercial use license
                                      • Why?
                                        • Microsoft gave them $16.6 million
                                        • Stock pump-and-dump
                                        • Get-rich-quick
                                        • The Real Threats
                                          • Intellectual property claims could be real next time around
                                            • Tainted contributions
                                            • Inadvertent infringement on software patents
                                            • Employee intellectual-property contract clauses (“We own everything you do”)
                                            • Deliberate sabotage attempts
                                            • Long discussion of IP law involving several cases of stolen code, a vigorous defense of the old Santa Cruz Operation, etc. Basic summary: get an IP lawyer and don’t take chances. Even if you’re innocent and remove contributed code as soon as you are made aware of the problem, you may still be liable.
                                            • FreeBSD project member: what are our liabilities if someone posts copyrighted to our public bug database?
                                              • Netcom was sued because a Usenet post containing copyrighted Scientology material was copied on their new server. Because they didn’t immediately investigate and take it down after being notified, the judge ruled that they were a participant.
                                              • DMCA “Safe Harbor” should protect anyone who removes code promptly

                                              • Q: What other helpful tips would you suggest?
                                                A: Demonstrate good will consistently - it will help immensely if you end up in front of a judge Dan: The real danger is current IP law - SCO is a merely the current menace to opensource. Bradley Kuhn: OSS is on a collision course with proprietary IP law. We certainly don’t want to see copyrighted code being stolen - if there is something in Linux, it should be taken out. We have to learn how to deal with IP issues and try to minimize the risks. Jonathan Zittrain: “I don’t hate copyright. The law here is in bad shape … hurting us along almost any dimension you choose to mention.”
                                                Get your employer to okay GPL contributions, even for work done in your spare time.
                                                We’re going to get test-cases to test the GPL in court.
                                                “I’m actually delighted that it’s IBM because they’re not going to let themselves get out-spent or out-lawyered” Grim prospects of a key layer of Internet infrastructure being claimed by companies on intellectual property grounds now that everyone is realizing the value of the Internet.