Miscellaneous INFOSEC

Taxonomy upgrade extras: 

Von Neumann Explains Computers, 1946

Jon von Neumann

John von Neumann published the earliest and most influential discussions of how to build electronic digital computers. His earliest known publication on computer design was actually a draft report he created in June, 1945, that was widely distributed among the infant community of computer developers. 

On May 15, 1946, von Neumann gave a detailed talk about the general principles of computer design. He was addressing the US Navy's Mathematical Computing Advisory Panel, a group that oversaw the Navy's computing R&D progams. Like the earlier draft, this talk was quickly transcribed, mimeographed, and distributed. The resulting paper was titled "The Principles of Large-Scale Computing Machines."

My father was about to be discharged from the Navy as this was taking place. He had spent World War II working on classified electronics in Boston. The Navy offered him a civilian job overseeing computer research projects. Thus, a copy of von Neumann's "Principles" paper ended up in his archives. 

I have made a PDF copy of this paper and posted it here. This is a sort of companion to Whirlwind block diagrams I posted earlier. 

 

Here is a rough chronology of von Neumann's earliest reports on computer design:

  • June 30, 1945 - First Draft of a Report on the EDVAC - a technical description of how to build a stored-program computer. While von Neumann produced the report and it carries his name, the contents are based on discussions within the ENIAC project about the design of a stored-program computer. 
  • May 15, 1946 - "The Principles of Large-Scale Computing Machines"
  • June 28, 1946 - Preliminary discussion of the logical design of an electronic computing instrument
  • July 8-August 30, 1946 - The Moore School Lectures - a lecture series given by members of the ENIAC project, including von Neumann, to an invited audience, including my father.

The first formal publication von Neumann produced on computer design was the Preliminary discussion of the logical design of an electronic computing instrument, co-authored with Arthur Burks and Herman Goldstine, on June 28, 1946. A year earlier he created and circulated the First Draft of a Report on the EDVAC, which might not have been intended as a formal publication, but was widely circulated. Between these, von Neumann gave his talk to the Navy, which only appeared in mimeograph form before being published in von Neumann's Collected Works in 1963. Nancy Stern also produced a copy of this paper and published it in the Annals of the History of Computing in 1981 (vol. 3 no. 3). 

Wordpress tag: 
Post category: 

Textbook Materials

Elementary Infosec cover

Draft supporting materials for the textbook

Visit the textbook site eisec.us for the latest supporting materials and study guides.

Draft materials are provided below:

About the Book

Elementary Infosec cover

Jones & Bartlett Learning, November, 2011.

CNSS LogoThe only textbook verified by the US Government to conform fully to the Committee on National
Security Systems' national training standard for information security professionals (NSTISSI 4011)

ISBN-10: 0763761419

This comprehensive, accessible Information Security text is ideal for the one-term, undergraduate college course. The text integrates risk assessment and security policy throughout the text, since security systems work best at achieving goals they are designed to meet, and security policy ties real-world goals to security mechanisms. Early chapters in the text discuss individual computers and small LANS, while later chapters deal with distributed site security and the Internet. Cryptographic topics follow the same progression, starting on a single computer and evolving to Internet-level connectivity. Mathematical concepts throughout the text are defined and tutorials with mathematical tools are provided to ensure students grasp the information at hand.

See below for sample contents. Sample chapters are also available from the publisher's site.

Table of Contents

Preface

1: Security From the Ground Up

1.1. The Security Landscape

1.2. Process Example: Bob’s Computer

1.4. Identifying Risks

1.5. Prioritizing Risks

1.6. Ethical Issues in Security Analysis

1.7. Security Example: Aircraft Hijacking

1.8. Resources

2: Controlling a Computer

2.1. Computers and Programs

2.2. Programs and Processes

2.3. Buffer Overflow and The Morris Worm

2.4. Access Control Strategies

2.5. Keeping Processes Separate

2.6. Security Policy and Implementation

2.7. Security Plan: Process Protection

2.8. Resources

3: Controlling Files

3.1. The File System

3.2. Executable Files

3.3. Sharing and Protecting Files

3.4. Security Controls for Files

3.5. File Security Controls

3.6. Patching Security Flaws

3.7. Process Example: The Horse

3.8. Chapter Resources

4: Sharing Files

4.1. Controlled Sharing

4.2. File Permission Flags

4.3. Access Control Lists

4.4. Microsoft Windows ACLs

4.5. A Different Trojan Horse

4.6. Phase Five: Monitoring The System

4.7. Chapter Resources

5: Storing Files

5.1. Phase Six: Recovery

5.2. Digital Evidence

5.3. Storing Data on a Hard Drive

5.4. FAT: An Example File System

5.5. Modern File Systems

5.6. Input/Output and File System Software

5.7. Resources

6: Authenticating People

6.1. Unlocking a Door

6.2. Evolution of Password Systems

6.3. Password Guessing

6.4. Attacks on Password Bias

6.5. Authentication Tokens

6.6. Biometric Authentication

6.7. Authentication Policy

6.8. Resources

7: Encrypting Files

7.1. Protecting the Accessible

7.2. Encryption and Cryptanalysis

7.3. Computer-Based Encryption

7.4. File Encryption Software

7.5. Digital Rights Management

7.6. Resources

8: Secret and Public Keys

8.1. The Key Management Challenge

8.2. The Reused Key Stream Problem

8.3. Public-key Cryptography

8.4. RSA: Rivest-Shamir-Adleman

8.5. Data Integrity and Digital Signatures

8.6. Publishing Public Keys

8.7. Resources

9: Encrypting Volumes

9.1. Securing a Volume

9.2. Block Ciphers

9.3. Block Cipher Modes

9.4. Encrypting a Volume

9.5. Encryption in Hardware

9.6. Managing Encryption Keys

9.7. Resources

10: Connecting Computers

10.1. The Network Security Problem

10.2. Transmitting Information

10.3. Putting Bits on a Wire

10.4. Ethernet: A Modern LAN

10.5. The Protocol Stack

10.6. Network Applications

10.7. Resources

11: Networks of Networks

11.1. Building Information Networks

11.2. Combining Computer Networks

11.3. Talking Between Hosts

11.4. Internet Addresses in Practice

11.5. Network Inspection Tools

11.6. Resources

12: End-to-End Networking

12.1. “Smart” Versus “Dumb” Networks

12.2. Internet Transport Protocols

12.3. Names on the Internet

12.4. Internet Gateways and Firewalls

12.5. Long Distance Networking

12.6. Resources

13: Enterprise Computing

13.1. The Challenge of Community

13.2. Management Processes

13.3. Enterprise Issues

13.4. Enterprise Network Authentication

13.5. Contingency Planning

13.6. Resources

14: Network Encryption

14.1. Communications Security

14.2. Crypto Keys on a Network

14.3. Crypto Atop the Protocol Stack

14.4. Network Layer Cryptography

14.5. Link Encryption on 802.11 Wireless

14.6. Encryption Policy Summary

14.7. Resources

15: Internet Services and Email

15.1. Internet Services

15.2. Internet Email

15.3. Email Security Problems

15.4. Enterprise Firewalls

15.5. Enterprise Point of Presence

15.6. Resources

16: The World Wide Web

16.1. Hypertext Fundamentals

16.2. Basic Web Security

16.3. Dynamic Web Sites

16.4. Content Management Systems

16.5. Ensuring Web Security Properties

16.6. Resources

17: Governments and Secrecy

17.1. Secrecy In Government

17.2. Classifications and Clearances

17.3. National Policy Issues

17.4. Communications Security

17.5. Data Protection

17.6. Trustworthy Systems

Preface

The goal of this textbook is to introduce college students to information security. Security often involves social and organizational skills as well as technical understanding. To solve practical security problems, we must balance real-world risks and rewards against the cost and bother of available security techniques. The text uses continuous process improvement to integrate these elements.

Security is a broad field. Some students may excel in the technical aspects, while others may shine in the more social or process-oriented aspects. Many successful students fall between these poles. The text offers opportunities for all types of students to excel.

Introducing Technology

If we want a solid understanding of security technology, we must look closely at the strengths and weaknesses of underlying information technology itself. This requires a background in computer architecture, operating systems, and computer networking. It’s hard for a typical college student to achieve breadth and depth in these subjects and still have time to really study security.

Instead of leaving a gap in students’ understanding, this book provides introductions to essential technical topics. Chapter 2 explains the basics of computer operation and instruction execution. This prepares students for a description of process separation and protection, which illustrates the essential role of operating systems in enforcing security.

Chapter 5 introduces file systems and input/output in modern operating systems. This lays a foundation for forensic file system analysis. It also shows students how a modern operating system organizes a complex service. This sets the stage for Chapter 10’s introduction to computer networking and protocol software.

Introducing Continuous Process Improvement

The text organizes security problem-solving around a six-phase security process. Chapter 1 introduces the process as a way of structuring information about a security event, and presents a simple approach to risk analysis. Chapter 2 introduces security policies as a way to state security objectives, and security controls as a way to implement a policy. Subsequent chapters introduce system monitoring and incident response as ways to assess system security and improve it.

Each step in the process builds on earlier steps. Each step also provides a chance to assess how well our work addresses our security needs. This is the essence of continuous process improvement.

In order to give students an accurate view of process improvement, the text introduces document structures that provide cross references between different steps of the process. We use elements of each earlier phase to construct information in the following phase, and we often provide a link back to earlier data to ensure complete coverage. While this may seem like nit-picking in some cases, it allows mastery of essential forms of communication in the technical and professional world.

Intended Audience

When used as a textbook, the material is intended for lower division undergraduates, or for students in a two-year community college program. Students should have completed high school mathematics. Typical students should have completed an introductory computing or programming course.

Instructors may want to use this book for either a one or two semester course. A one semester course would usually cover one chapter a week; the instructor may want to combine a couple of earlier chapters or skip the final chapter. Some institutions may find it more effective to teach the material over a full year. This gives the students more time to work with the concepts and to cover all topics in depth.

Following the style of my earlier books, this text focuses on diagrams and practical explanations to present fundamental concepts. This makes the material clearer to all readers and makes it more accessible to the math-phobic reader. Many concepts, particularly in cryptography, can be clearly presented in either a diagram or in mathematical notation. This text uses both, with a bias towards diagrams.

Many fundamental computing concepts are wildly abstract. This is also true in security, where we sometimes react to illusions perceived as risks. To combat this, the text incorporates a series of concrete examples played out by characters with names familiar to those who read cryptographic papers: Bob, Alice, and Eve. They are joined by additional classmates named Tina and Kevin, since different people have different security concerns.

Curriculum Standards

The material in this text fulfills curriculum requirements published by the US government and the Association for Computing Machinery (ACM). In particular, the text covers all required topics for training information systems security professionals under the Information Assurance Courseware Evaluation Program (NSTISSI #4011) established by the US National Security Agency (NSA). The text also provides substantial coverage of the required topics for training senior system managers (CNSSI #4012) and for system administrators (CNSSI #4013).

The text also covers the core learning outcomes for information security education published in the ACM’s “IT 2008” curricular recommendations for Information Technology education. As a reviewer and contributor to the published recommendations, the author is familiar with its guidelines.

Students who are interested in becoming a Certified Information System Security Professional (CISSP) may use this book as a study aid for the examination. All key areas of the CISSP Common Body of Knowledge are covered in this text. Certification requires four or five years of professional experience in addition to passing the exam.

Teaching Security

Information security is a fascinating but abstract subject. This text introduces students to real-world security problem solving, and incorporates security technology into the problem-solving process. There are lots of “how to” books that explain how to harden a computer system.

Many readers and most students need more than a “how to” book. They need to decide which security measures are most important, or how to trade off between alternatives. Such decisions often depend on what assets are at stake and what risks the computer’s owner are willing to take.

The practitioner’s most important task is the planning and analysis that helps choose and justify a set of security measures. In my own experience, and in that of my most capable colleagues, security systems succeed when we anticipate the principal risks and design the system to address them. In fact, once we have identified what we want for security (our policy requirements), we don’t need security gurus to figure out what we get (our implementation). We just need capable programmers, testers, and administrators.

Teaching Aids

As the chapter unfolds, we encounter certain key terms indicated in bold italics. These highlight essential terms that students may encounter in the information security community. Successful students will recognize these terms.

The Resources section at the end of each chapter lists the key terms and provides review and exercises. The review questions help students confirm that they have absorbed the essential concepts. Some instructors may want to use these as recitation or quiz questions. The problems and exercises give students the opportunity to solve problems based on techniques presented in the text.

Additional Reading Materials

Here is an incomplete collection of reading materials associated with the textbook. Visit the Elementary Infosec site for a complete set of reading and study materials

Locating copies of on-line articles and papers

Some links lead to on-line articles published by professional societies like the ACM (Association for Computing Machinery) or IEEE (Institute of Electrical and Electronic Engineers). Serious computing experts often join one or both of these, and sign up for electronic library subscriptions. Many college and university libraries may also provide free access to these for students and faculty.

1: Security From the Ground Up

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

Security Overview - Recent Events

The following blogs provide readable reports and commentary on information security.

Bruce Schneier (of Schneier on Security) coined the term security theater in his book Beyond Fear (New York: Copernicus Books, 2003).

Continuous Improvement

ASQ (the American Society for Quality, which has now gone international) provides tutorials on basic quality concepts including continuous improvement.

In Japan, the term kaizen embodies the continuous improvement process.

Systems Engineering

During World War II, military enterprises poured vast resources into various techincal projects, notably radar, codebreaking, and the atomic bomb. Those successes encouraged the peacetime military to pursue other large scale technical projects. A typical project would start with a very large budget and a vague completion date a few years in the future. But in practice, many of these projects vastly exceeded both the budget and the predicted time table.

A handful of projects, notably the Polaris Missile project, achieved success while adhering closely to their initial budget and schedule estimates. Pressure on defense budgets led the US DOD to identify features of the successful projects so they might be applied to future work. This was the genesis of systems engineering. The DOD's Defense Aquisition University has produced an introduction to systems engineering (PDF format) in the defense community.

The International Council on Systems Engineering (INCOSE) provides a more general view of systems engineering. NASA also provides on-line training materials on their Space Systems Engineering site.

Basic Principles of Information Security

Chapter 1 introduces the first three of eight basic principles of information security.

  1. Continuous Improvement
  2. Least Privilege
  3. Defense in Depth

Different authorities present different lists of principles. International standards bodies, including NIST in the US, tend to produce very general lists of principles, reflecting notions such as "be safe," "keep records," and other generalizations (for example, see NIST's SP 800-14: "Generally Accepted Principles and Practices for Securing Information Technology Systems"). These principles represent basic truths about security, but few are stated in a way that helps one make security decisions.

Saltzer and Schroeder produced a now-classic list based on experience with the Multics time sharing system in the 1970s: "The Protection of Information in Computer Systems," Proceedings of the IEEE 63, 9 (September, 1975). Some of these principles reflect features of the Multics system while others reflect some well-known shortcomings with most systems of that time. Copies exist online at Saltzer's own web site and at the University of Virginia.

There is also a Cryptosmith blog post that compares the textbook's list of principles with those in Saltzer and Schroeder.

Microsoft's Technet archives also contain a well-written summary of principles titled "10 Immutable Laws of Security," followed with "10 Immutable Laws of Security Administration."

High-Level Security Analysis

A high-level security analysis provides a brief summary of a security situation at a given point of time. The next section provides a checklist for writing a high-level security analysis.

Risk Assessment

The risk assessment processes noted in the textbook are all avaliable online:

Ethical Issues

CERT has published the policy it follows when disclosing vulnerabilities. Changes to Microsoft's disclosure policy yield news coverage. Microsoft publishes its policy in MS Word "docx" format.

Several news sources including the BBC reported on Michael Lynn's canceled talk on Cisco router vulnerabilities at the 2005 Black Hat conference.

Aircraft Hijacking

The fundamental reference for everything related to the events of September 11, 2001, is the Final Report of the National Commission on Terrorist Attacks Upon the United States, a.k.a. "The 9/11 Commission Report," published in 2004.

Following 9/11, the BBC published a brief history of airline hijackings. The 2003 Centennial of Flight web site provides a more general summary of violent incidents in aviation security. In 2007, New York Magazine published a more detailed hijacking time-line in conjunction with breaking news on the D. B. Cooper hijacking case. The US FBI web site contains a lot of information about the D. B. Cooper case, including a 2007 update.

VideoHere is a 30 second video clip of traditional aircraft hijacking. The History.com web site provides several videos of the 9/11 attack from different vantage points on its Witness to 9/11 web page.

Checklist for a High-Level Security Analysis

The textbook presents a high-level security analysis as a short writing exercise that summarizes a security situation. The analysis generally describes a situation at a particular point in time. For example, the 9/11 discussion in the textbook describes air travel security before 9/11. The analysis describes the six phases of the security process:

  • Assets
  • Risks
  • Policy
  • Implementation
  • Monitoring
  • Recovery

Here is a checklist of the basic properties of a high-level analysis:

  1. Is the analysis at least seven paragraphs long?
  2. Is the analysis too long? A typical analysis should be no more than 14 paragraphs long unless it addresses a really unusual subject area that requires extra explanation.
  3. Do the initial paragraphs introduce the scenario?
  4. Are there six headings following the introductory paragraph, one per phase?
  5. Is there at least one paragraph discussing each phase?

Note that a complete security plan will also cover the six phases, but it is not limited to this length. A complete plan covers each phase thoroughly.

2: Controlling a Computer

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

Computer Internals

Video Here is a 26-minute video of middle school students visiting a "walk through" computer to learn the basics of computer operation. Taken from the PBS TV series "Newton's Apple," 1990. Although the technology is over 20 years old, the fundamental components remain the same, except for speed speed, size, and capacity.

There are, of course, countless images and videos available through on-line searching that show specific elements of computer systems.

Binary Numbers and Hexadecimal Notations

VideoStudents who have not yet studied these topics in detail will want to visit web sites that provide an introduction to binary and hex. YouTube user Ryan of Aberdeen has created a video tutorial (9 minutes). There are also written tutorials:

The Morris worm

A faculty member at NC State University maintains a site that provides an overview of the Morris worm.

Eugene Spafford (aka spaf) wrote a report describing the worm, its operations, and its effects (PDF), shortly after the incident.

Eichin and Rochlis of MIT published a report of the worm incident from the MIT perspective (PDF). This was presented at the IEEE Symposium on Security and Privacy the following year.

In 1990, Peter Denning published a book that brought together several papers on the Morris worm and other security issues emerging at that time, titled Computers Under Attack: Intruders, Worms and Viruses.

Spafford also maintains an archive of worm-related information at Purdue University.

Open Design

Auguste Kerckhoffs' original paper on cryptographic system design recommended that cryptographic systems be published and that secrecy should reside entirely in a secret key. The paper was published in French in 1883. Portions of Kerckhoffs' paper are available on-line including partial English translations.

Claude Shannon's "Communication Theory of Secrecy Systems" (Bell System Technical Journal, vol. 28, no. 4, 1949) contains his assumption "the enemy knows the system being used" (italics his). Bell Labs provides general information about Shannon's work and publications.

Eric Raymond published a famous essay on the benefits of open design and of sharing program source code in general, called "The Cathedral and The Bazaar." This essay has inspired many members of the Open Source community.

Program and Process Security

Butler Lampson introduced the access matrix in his 1971 paper, "Protection." Lampson has posted a copy of his paper on-line in several formats.

 

 

Identifying Motherboard Features

This is, unfortunately, a lot harder than it seems. As RAM has grown smaller and I/O grown more complex, motherboard components have changed dramatically in size and appearance. Here are suggestions on identifying key features in older and newer motherboards.

  • CPU - The CPU is generally covered by a large, louvered object called a heat sink, and is often held in place by a lever or catch. The heat sink is often glued to the top of the CPU. In some cases you must remove the heat sink to remove the CPU, while in other cases they are removed as a single unit. Higher performance CPUs may have fans or liquid cooling systems attached to the heat sink.
  • RAM - these have resided on daughterboards for many years now. The daughterboards usually - but not always - plug directly into the motherboard. Each daughterboard will typically be long and thin to handle a row of individual RAM circuits. As a motherboard gets older and the RAM becomes more dense, there are fewer and fewer RAM circuits on each daughterboard. 
  • I/O circuits - the latest CPUs have special-purpose I/O circuits that are as large and power-hungry as the CPUs themselves. These may often carry their own heat sinks. Unlike the CPUs, though, a motherboard is often built around the I/O circuits. Thus, the I/O circuits are built in to the motherboard, while the CPU is added by the computer builder. In Figure 2.1 of the textbook (page 44), the large, unmarked metal object in the center of the motherboard is the heat sink for an I/O circuit.
  • I/O connectors - these are the easiest items to identify. They always mate with standardized cables to power supplies or to peripheral devices. There are often PCI connectors that accept daughterboards with specialized I/O circuits. There is often an additional high-performance connector for attaching a graphics circuit.
  • BIOS ROM - this is hard to spot on the newest devices because ROM circuits have grown so tiny. On old motherborads, the BIOS appears as a distinct row of ROM chips. On the latest, there may be only one or two chips interspersed among others on the board.

The most reliable way to identify a motherboard's contents is to locate a copy of its installation manual. These are usually posted on the web by the motherboard's manufacturer. Most boards clearly include the manufacturer's name and the board's model number.

If the manufacturer and model number aren't obvious, it may be possible to identify the motherboard using Google Images. Enter the word "motherboard" as a search term along with other textual items on the board. Compare the images displayed with the color and layout of the motherboard in question. Keep in mind everything should match when you find the correct board. Missing or misplaced features indicate that the boards don't match. A popular motherboard may appear many times, but most images will lead to pages that indicate the manufacturer and model. In some cases, the image may lead directly to the manufacturer's own pages.

3: Controlling Files

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

Computer Viruses

Security consultant Fred Cohen performed much of the pioneering analysis of computer viruses. His web site contains several useful articles on virus technology. Even though some of the material is 30 years old, the basic technical truths remain unchanged.

Some anti-virus vendors provide summaries of current anti-virus and malware activities:

Modern Malware

Here is a list of malware briefly described in the textbook, plus links to in-depth reports on each one. Check recent news: security experts occasionally make progress in eradicating one or another of these, but the botnets sometimes recover. Many of these are PDFs.

VideoVideos: Ralph Langer, a German expert in control systems security, gave a TED talk describing Stuxnet (~11 minutes). Bruce Dang of Microsoft also gave a detailed presentation about Stuxnet (75 minutes) at a conference.

Access Rights and Capabilities

Butler Lampson introduced the access matrix in his 1971 paper, "Protection" (PDF). Lampson has posted a copy on-line in several formats.

Although most modern systems use resource-oriented permissions to control access rights, there are a few cases that use capabilities, which associate rights with active entities like programs and users. Jack Dennis and Earl Van Horn of MIT introduced the notion of capabilities in their 1965 report "Programming Semantics for Multiprogrammed Computers," which was published in Communications of the ACM in 1966.

Marc Stiegler has posted an interesting introduction to capability based security that ties it to other important security concepts. The EROS OS project has also posted an essay that explains capability-based security. For a thorough coverage of capability based architecture circa 1984, see Henry Levy's book Capability-Based Computer Systems. He has posted it on-line.

Microsoft has posted an article that describes access control on Windows files and on the Windows "registry," a special database of system-specific information.

State Diagrams

Electrical engineers have relied on state diagrams for decades to help design complicated circuits. The technique is also popular with some software engineers, though it rarely finds its way into courses on computer programming. Any properly-constructed state diagram may be translated into a state table that provides the same information in a tabular form. Tony Kuphaldt's free on-line textbook Lessons in Electric Circuits explains state machines in the context of electric circuits in Volume IV, Chapter 11: Sequential Circuits-Counters.

Upper-lever computer science students may encounter state diagrams in a course on automata theory in which they use such diagrams to represent deterministic finite automata. Such mechanisms can handle the simplest type of formal language, a regular grammar. Most people encounter regular grammars as regular expressions, an arcane syntax used to match text patterns when performing complicated search-and-replace operations in text editors.

Students introduced to modern structured design techniques using the Unified Modeling Language (UML) often use state machine diagrams or state charts (a diagram in table form). On-line tutorials about UML state machines appear at Kennesaw State University and the Agile Modeling web site.

Tracking and Managing Security Flaws

In the US, there are several organizations that track and report on information security vulnerabilities. Many of these organizations provide email alerts and other data feeds to keep subscribers up to date on emerging vulnerabilities. Some organizations provide their services to particular communities (e.g. government or military organizations, or customers of a vendor's products) while others provide reports to the public at large.

The SANS Internet Storm Center also provides a variety of on-line news feeds and reports, as well as a continuously-updated "Infocon Status" to indicate unusual changes in the degree of malicious activity on the Internet. Click on the image below to visit the Internet Storm Center for further information on current vulnerabilities and malicious Internet activity.

Internet Storm Center Infocon Status

In 2000, Arbaugh, Fithen, and McHugh wrote an article describing a life-cycle model of information security vulnerabilities titled "Windows of Vulnerability: A Case Study Analysis", (IEEE Computer 33, December 2000). The authors have posted a copy of the article online (PDF).

The Trojan Horse

The library at Stanford posted a brief history of the Trojan War. Although Homer's Iliad tells the story of the Trojan War, it says very little about the Greek trickery that led to the city's fall. The story is more the province of Virgil's Aeneid.

Trojan horse by Henri Motte, Paris, 1873

In the 1970s, Guy Steele at MIT started collecting bits of jargon used in the computer community. This yielded "The Jargon File," which Steele maintained for several years until it was passed on to Eric Raymond. According to the Jargon File, the term Trojan horse entered the computing lexicon via Dan Edwards of MIT and the NSA.

US-CERT has published a two-page guide on how to deal with a Trojan horse or virus infection on a computer (PDF).

4: Sharing Files

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

File Permission Flags - Unix Style

There are numerous on-line tutorials on Unix and/or Linux file permissions, including ones provided by:

VideoSeveral people have also posted videos explaining file permissions,
including thedangercobra, theurbanpenguin, and elithecomputerguy.

Access Control Lists

ACLs first appeared in the Multics timesharing system, as described in the paper "A General-Purpose File System For Secondary Storage," by R. Daley and Peter Neumann (Proc. 1965 Fall Joint Computer Conference) and on the Multicians web site.

Since ACLs could provide very specific access restrictions, they became recommended features of high-security systems. When the US DOD developed the "Trusted Computer System Evaluation Criteria," (PDF) (a.k.a. the TCSEC or Orange Book) ACLs were an essential feature of higher security systems. Modern security products are evaluated against the Common Criteria.

POSIX ACLs

While traditional Unix systems did not have ACLs, more advanced versions of Unix incorporated them, partly to meet high security requirements like those in the Orange Book. This led to the development of POSIX ACLs as part of a proposed POSIX 1003.1e standard. The standards effort was abandoned, but several Unix-based systems did incorporate POSIX ACLs. Here are examples:

Mac OS-X ACLs

The ACL user interface on Mac OS-X is very simple. In fact, the OS-X ACLs are based on POSIX ACLs and may incorporate more spohisticated settings and inheritances than we see in the Finder's "Information" display. These features are available through special ACL options of the chmod shell command. One developer has produced an application called Sandbox that provides a more extensive GUI for managing the ACLs.

Windows ACLs

It can be challenging to find accurate online information about Windows ACLs, because the computer-based access controls are often confused with network-based access controls. The MS Developer Network provides general information about ACLs.

Researchers at CMU evaluated the Windows XP version of ACLs in a series of experiments documented in "Improving user-interface dependability through mitigation of human error," Intl J. Human-Computer Studies 63 (2005) 25-50, by Maxion and Reeder.

5: Storing Files (uc)

Under Construction

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

TBD

5: Memory Sizes: kilo mega giga tera peta exa

Here is a summary of memory size names and their corresponding address sizes. Many people memorize this type of information naturally through working with computer technology over time or during a professional career.

Names for large numbers

Practicing with Quizlet

If you want to memorize these values, visit the Quizlet page. The page tests your knowledge of the smaller sizes (K, M, G, T), how these sizes are related (i.e. a terabyte is a thousand billion bytes), and how they relate to memory sizes (a TB needs an address approximately 40 bits long).

Address Size Shortcut

Here is a simple shortcut for estimating the number of bits required to address storage of a given size.

103 ~ 210

To put this into practice, we do the following:

  1. Count the number of sets of three zeros (the "thousands") in the storage size.
  2. Multiply the number of thousands by ten

Let's work out an example with a terabyte: a trillion-byte memory.

  1. In a trillion (1,000,000,000,000) there are 4 sets of three zeroes
  2. Multiply 4 by 10, and we get approximately 40 bits.

6: Authenticating People (tbd)

UNDER CONSTRUCTION

Cryptographic hash functions

Cryptographers develop new hash functions every few years because cryptanalysts and mathematicians find weaknesses in the older ones. Valerie Aurora provides a graphic illustration of this.

7: Encrypting Files (tbd)

TBD

8: Secret and Public Keys (tbd)

TBD

9: Encrypting Volumes (tbd)

TBD

10: Connecting Computers (tbd)

TBD

11: Networks of Networks (tbd)

TBD

12: End-to-End Networking (uc)

Under Construction

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

TBD

13: Enterprise Computing (uc)

Under Construction

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

TBD

14: Network Encryption (uc)

Under Construction

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

TBD

15: Internet Services and Email (uc)

Under Construction

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

TBD

16: The World Wide Web (uc)

Under Construction

Study Acronyms and Terms

Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.

TBD

17: Governments and Secrecy (tbd)

TBD

Appendix: Alternative Terms and Concepts

This section of the textbook provides details not otherwise addressed by the main text.

A Comprehensive Model of Information System Security

Two educational standards in information system security refer to closely-related models of information system security. First, we have a US government training standard:

Second, we have an academic curriculum standard:

Detailed Chapter Outlines

1: Security From the Ground Up

1.1 The Security Landscape

Not Just Computers Any More

1.1.1 Making Security Decisions

1.1.2 The Security Process

1.1.3 Continuous Improvement: A Basic Principle

The Roots of Continuous Improvement

1.2 Process Example: Bob’s Computer

Assets

Risks

Policy

Implementation

Monitoring

Recovery

Bob’s Response

1.3 Assets and Risk Assessment

Identifying Risks

Fine Points of Terminology

Hackers

1.3.1 What Are We Protecting?

Identifying Goals

Identifying Assets

1.3.2 Security Boundaries

Least Privilege: A Second Basic Principle

Example: Boundaries in a Dorm

Analyzing the Boundary

The Insider Threat

1.3.3 Security Architecture

Defense In Depth: A Third Basic Principle

1.3.4 Risk Assessment Overview

1.4 Identifying Risks

1.4.1 Threat Agents

1.4.2 Security Properties, Services, and Attacks

1.5 Prioritizing Risks

1.5.1 Example: Risks to Alice’s Laptop

Step 1: Identify Computing Assets

Step 2: Identify Threat Agents and Potential Attacks

Step 3: Estimate the Likelihood of Individual Attacks

Step 4: Estimate the Impact of Attacks over Time

Step 5: Calculate the Impact of Each Attack

1.5.2 Other Risk Assessment Processes

NIST Recommendation

OCTAVE Process

1.6 Ethical Issues in Security Analysis

Laws, Regulations, and Codes of Conduct

1.6.1 Searching for Vulnerabilities

1.6.2 Sharing or Publishing Vulnerabilities

1.7 Security Example: Aircraft Hijacking

1.7.1 Hijacking: A High-Level Analysis

Assets

Risks

Policy

Implementation

Monitoring

Recovery

1.7.2 September 11, 2001

1.8 Resources

1.8.1 Review Questions

1.8.2 Exercises and Problems

2: Controlling a Computer

2.1 Computers and Programs

The Motherboard

2.1.1 Input/Output

Parallel Versus Serial Wiring

The BIOS

2.1.2 Program Execution

Separating Data and Control

2.1.3 Procedures

The Stack

Buffers

2.2 Programs and Processes

2.2.1 Switching Between Processes

Observing Active Processes

2.2.2 The Operating System

I/O System

Security

2.3 Buffer Overflow and The Morris Worm

The ‘finger’ program

2.3.1 The ‘finger’ Overflow

Exploiting ‘finger’

The Shellcode

The Worm Released

2.3.2 Security Alerts

Establishing CERT

2.4 Access Control Strategies

Islands

Vaults

2.4.1 Puzzles and Patterns

Open Design: A Basic Principle

Cryptography and Open Design

Pattern-based Access Control

Biometrics

2.4.2 Chain of Control: Another Basic Principle

Controlling the BIOS

Subverting the Chain of Control

2.5 Keeping Processes Separate

Separation Mechanisms

Program Modes

RAM Protection

User Identities

Evolution of Personal Computers

Security on Personal Computers

Operating System Security Features

2.5.1 Sharing a Program

Access Matrix

Access Rights

2.5.2 Sharing Data

2.6 Security Policy and Implementation

Constructing Alice’s Security Plan

Writing a Security Policy

2.6.1 Analyzing Alice’s Risks

2.6.2 Constructing Alice’s Policy

2.6.3 Alice’s Security Controls

Alice’s Backup Procedure

2.7 Security Plan: Process Protection

Policy for Process Protection

Functional Security Controls

The Dispatcher’s Design Description

The Design Features

The Dispatching Procedure

Security Controls for Process Protection

2.8 Resources

2.8.1 Review Questions

2.8.2 Problems and Exercises

3: Controlling Files

3.1 The File System

File and Directory Path Names

3.1.1 File Ownership and Access Rights

File Access Rights

Initial File Protection

3.1.2 Directory Access Rights

3.2 Executable Files

3.2.1 Execution Access Rights

Types of Executable Files

3.2.2 Computer Viruses

Virus Infection

Malicious Viruses

3.2.3 Macro Viruses

3.2.4 Modern Malware: A Rogue’s GAllery

Waledac

Conficker, also called Downadup

Pushdo/Cutwail

ZeuS

Stuxnet

3.3 Sharing and Protecting Files

Objectives

Threats

Risks

3.3.1 Policies for Sharing and Protection

Underlying System Policy

User Isolation Policy

User File Sharing Policy

3.4 Security Controls for Files

3.4.1 Deny by Default: A Basic Principle

The opposite of Deny by Default

3.4.2 Managing Access Rights

3.4.3 Capabilities

Capabilities in Practice

Resource-oriented Permissions

3.5 File Security Controls

3.5.1 File Permission Flags

System and Owner Access Rights in Practice

3.5.2 Security Controls to Enforce Bob’s Policy

3.5.3 States and State Diagrams

Information States

3.6 Patching Security Flaws

The Patching Process

Security Flaws and Exploits

Windows of Vulnerability

3.7 Process Example: The Horse

3.7.1 Troy: A High-Level Analysis

3.7.2 Analyzing the Security Failure

3.8 Chapter Resources

3.8.1 Review Questions

3.8.2 Exercises

4: Sharing Files

4.1 Controlled Sharing

Tailored File Security Policies

Bob’s Sharing Dilemma

4.1.1 Basic File Sharing on Windows

4.1.2 User Groups

Administrative Groups

4.1.3 Least Privilege and Administrative Users

Administration by Regular Users

User Account Control on Windows

4.2 File Permission Flags

4.2.1 Permission Flags and Ambiguities

4.2.2 Permission Flag Examples

Security Controls for the File Sharing Policy

4.3 Access Control Lists

Multics ACLs

Modern ACL Implementations

4.3.1 POSIX ACLs

4.3.2 Macintosh OS-X ACLs

4.4 Microsoft Windows ACLs

4.4.1 Denying Access

Determining Access Rights

Building Effective ACLs

4.4.2 Default File Protection

Inherited Rights

Dynamic ACLs

Moving and Copying Files

4.5 A Different Trojan Horse

A Trojan Horse Program

Transitive Trust: A Basic Principle

4.6 Phase Five: Monitoring The System

Catching an intruder

4.6.1 Logging Events

A Log Entry

The Event Logging Mechanism

Detecting Attacks by Reviewing the Logs

4.6.2 External Security Requirements

Laws, Regulations, and Industry Rules

External Requirements and the Security Process

4.7 Chapter Resources

4.7.1 Review Questions

4.7.2 Exercises

5: Storing Files

5.1 Phase Six: Recovery

Incidents and Damage

Compromised Systems

5.1.1 The Aftermath of an Incident

Digital Forensics

Fault and Due Diligence

5.1.2 Legal Disputes

Legal Systems

Resolving a Legal Dispute

5.2 Digital Evidence

The Fourth Amendment

Legal Concepts

5.2.1 Collecting Legal Evidence

Collecting Evidence at The Scene

Securing the Scene

Documenting the Scene

5.2.2 Digital Evidence Procedures

Authenticating a Hard Drive

5.3 Storing Data on a Hard Drive

Magnetic Recording and Tapes

Hard Drive Fundamentals

5.3.1 Hard Drive Controller

5.3.2 Hard Drive Formatting

High Level Format

Fragmentation

Quick Format

Flash Drives

5.3.3 Error Detection and Correction

Parity Checking

Checksums

Cyclic Redundancy Checks

Error Correcting Codes

5.3.4 Hard Drive Partitions

Partitioning to Support Older Drive Formats

Partitioning in Modern Systems

Partitioning and Fragmentation

Hiding Data with Partitions

5.3.5 Memory Sizes and Address Variables

Address, Index, and Pointer Variables

Memory Size Names and Acronyms

Estimating the Number of Bits

5.4 FAT: An Example File System

Volume Layout

5.4.1 Boot Blocks

The FAT

5.4.2 Building Files from Clusters

Cluster Storage

An Example FAT File

FAT Format Alternatives

5.4.3 FAT Directories

Long File Names

Deleting Files

Undeleting a File

5.5 Modern File Systems

File System Design Goals

Conflicting File System Objectives

5.5.1 Unix File System

Inodes

Directories

5.5.2 Apple’s HFS Plus

5.5.3 Microsoft’s NTFS

5.6 Input/Output and File System Software

Device Independence

The Hourglass

File System Software

Programming Assumptions

5.6.1 Software Layering

An Example

Layering Logic

Abstraction

5.6.2 A Typical I/O Operation

Part A: Call the operating system

Part B: OS constructs the I/O operation

Part C: The driver starts the actual I/O device

Part D: The I/O operation ends

5.6.3 Security and I/O

Restricting the devices themselves

Restricting Parameters in I/O Operations

File Access Restrictions

5.7 Resources

5.7.1 Review Questions

5.7.2 Problems/Exercises

6: Authenticating People

Chapter Outline

 

6.1 Unlocking a Door

6.1.1 Authentication Factors

Two-Factor Authentication

Three-Factor Authentication

6.1.2 Threats and Risks

Risks

Attack Strategy: Low Hanging Fruit

6.2 Evolution of Password Systems

Password Hashing

Procedure Diagrams

Password Hashing in Practice

6.2.1 One-way Hash Functions

Modern Hash Functions

A Cryptographic Building Block

6.2.2 Sniffing Credentials

6.3 Password Guessing

DOD Password Guideline

Interactive Guessing

Network-based Guessing

Off-line Password Cracking

6.3.1 Password Search Space

6.3.2 Truly Random Password Selection

6.3.3 Cracking Speeds

6.4 Attacks on Password Bias

Bias and Entropy

6.4.1 Biased Choices and Average Attack Space

Average Attack Space

Biased Password Selection

Measuring Likelihood, not Certainty

Making Independent Guesses

Example: 4-digit luggage lock

6.4.2 Estimating Language-Based Password Bias

Klein’s Password Study

6.5 Authentication Tokens

Passive Authentication Tokens

6.5.1 Challenge-Response Authentication

Implementing Challenge-Response

Another Cryptographic Building Block

The Nonce

Direct Connect Tokens

6.5.2 One-time Password Tokens

A Token’s Search Space

Average Attack Space

Attacking One-Time Password Tokens

Guessing a Credential

6.5.3 Token Vulnerabilities

6.6 Biometric Authentication

6.6.1 Biometric Accuracy

6.6.2 Biometric Vulnerabilities

6.7 Authentication Policy

6.7.1 Weak and Strong Threats

Weak Threats

Strong Threats

Effect of Location

6.7.2 Policies for Weak Threat Environments

A Household Policy

A Workplace Policy: Passwords Only

A Workplace Policy: Passwords and Tokens

6.7.3 Policies for Strong and Extreme Threats

Passwords Alone for Strong Threats

Passwords Plus Biometrics

Passwords Plus Tokens

Constructing the Policy

6.7.4 Password Selection and Handling

Simple Passwords

Strong But Memorable Passwords

The Strongest Passwords

6.8 Resources

6.8.1 Review Questions

6.8.2 Problems/Exercises

7: Encrypting Files

Chapter Outline

 

7.1 Protecting the Accessible

7.1.1 Process Example: The Encrypted Diary

7.1.2 Encryption Basics

Categories of Encryption

A Process View of Encryption

Shared Secret Keys

Effective Encryption

7.1.3 Encryption and Information States

Illustrating Policy with a State Diagram

Proof of Security

7.2 Encryption and Cryptanalysis

7.2.1 The Vignère Cipher

7.2.2 Electromechanical Encryption

7.3 Computer-Based Encryption

The Data Encryption Standard

The Advanced Encryption Standard

Predicting Cracking Speeds

7.3.1 Exclusive Or: A Crypto Building Block

7.3.2 Stream Ciphers: Another Building Block

Generating a Key Stream

An Improved Key Stream

7.3.3 Key Stream Security

RC4 Biases

Pseudo-Random Number Generators

The Effects of Ciphertext Errors

7.3.4 The One-time Pad

Soviet Espionage and One-time pads

Modular Arithmetic

Practical One-time pads

7.4 File Encryption Software

7.4.1 Built-in File Encryption

7.4.2 Encryption Application Programs

Ensuring Secure File Encryption

Protecting the Secret Key

7.4.3 Erasing a Plaintext File

Risks That Demand Overwriting

Preventing Low Level Data Recovery

Erasing Optical Media

7.4.4 Choosing a File Encryption Program

Software Security Checklist

File Encryption Security Checklist

Cryptographic Product Evaluation

7.5 Digital Rights Management

Policy Dilemmas

The DVD Content Scrambling System

7.6 Resources

7.6.1 Review Questions

7.6.2 Problems/Exercises

8: Secret and Public Keys

Chapter Outline

 

8.1 The Key Management Challenge

Cryptonets

Levels of Risk

Key Sharing Procedures

8.1.1 Rekeying

Cryptoperiods

Distributing New Keys

Public-Key Cryptography

8.1.2 Using Text for Encryption Keys

Taking Advantage of Longer Passphrases

Software Checklist for Key Handling

8.1.3 Key Strength

Operating Recommendations

8.2 The Reused Key Stream Problem

8.2.1 Avoiding Reused Keys

Changing the Internal Key

Combining the Key with a Nonce

Software Checklist for Internal Keys Using Nonces

8.2.2 Key Wrapping: Another Building Block

Multiple Recipients

Key Wrapping and Cryptoperiods

Key “Splitting”

Software Checklist for Wrapped Keys

8.2.3 Separation of Duty: A Basic Principle

Separation of Duty with Encryption

8.2.4 DVD Key Handling

8.3 Public-key Cryptography

Attacking Public Keys

8.3.1 Sharing a Secret: Diffie-Hellman

Perfect Forward Secrecy

Variations of Diffie-Hellman

8.3.2 Diffie-Hellman: The Basics of the Math

8.3.3 Elliptic Curve Cryptography

8.4 RSA: Rivest-Shamir-Adleman

Digital Signatures

RSA Applications

8.4.1 Encapsulating Keys with RSA

8.4.2 An Overview of RSA Mathematics

Brute Force Attacks on RSA

The Original Challenge

The Factoring Problem

Selecting a Key Size

Other Attacks on RSA

8.5 Data Integrity and Digital Signatures

8.5.1 Detecting Malicious Changes

One-way Hash Functions

Birthday Attacks

8.5.2 Detecting a Changed Hash Value

Keyed Hash

8.5.3 Digital Signatures

Non-Repudiation

8.6 Publishing Public Keys

8.6.1 Public-Key Certificates

Certificate Authorities

8.6.2 Chains of Certificates

Certificate Hierarchy

Web of Trust

Self-Signed Certificates

Trickery with Certificates

8.6.3 Authenticated Software Updates

8.7 Resources

8.7.1 Review Questions

8.7.2 Problems/Exercises

9: Encrypting Volumes

Chapter Outline

 

9.1 Securing a Volume

9.1.1 Risks To Volumes

Eavesdropping

Discarded Hard Drives

9.1.2 Risks and Policy Trade-offs

Identifying Critical Data

Policy for Unencrypted Volumes

Policy for Encrypted Volumes

9.2 Block Ciphers

Building a Block Cipher

The Effect of Ciphertext Errors

9.2.1 Evolution of DES and AES

DES and Lucifer

Triple DES

The Development of AES

AES Finalists

9.2.2 The RC4 Story

Export Restrictions

RC4 Leaking, Then Cracking

Lessons Learned

9.2.3 Qualities of Good Encryption Algorithms

Explicitly designed for encryption

Security does not rely on its secrecy

Available for analysis

Subjected to analysis

No practical weaknesses

Cryptographic evaluation

Choosing an Encryption Algorithm

9.3 Block Cipher Modes

9.3.1 Stream Cipher Modes

Weaknesses

Ciphertext Errors

Counter Mode

9.3.2 Cipher Feedback Mode

Ciphertext Errors

9.3.3 Cipher Block Chaining

Ciphertext Errors

9.4 Encrypting a Volume

Choosing a Cipher Mode

Hardware Versus Software

9.4.1 Volume Encryption in Software

Files as Encrypted Volumes

9.4.2 Adapting an Existing Mode

Drive Encryption with Counter Mode

Constructing the Counter

An Integrity Risk

Drive Encryption with CBC Mode

Integrity Issues with CBC Encryption

9.4.3 A “Tweakable” Encryption Mode

9.4.4 Residual Risks

Untrustworthy encryption

Encryption integrity

Looking for plaintext

Data Integrity

9.5 Encryption in Hardware

Recycling the Drive

9.5.1 The Drive Controller

Security Boundary

Drive Formatting

9.5.2 Drive Locking and Unlocking

9.6 Managing Encryption Keys

Key Generation

Rekeying

9.6.1 Key Storage

Working key storage in hardware

Persistent Key Storage

Protected storage

Key Wrapping

Managing removable keys

9.6.2 Booting an Encrypted Drive

Pre-boot authentication

BIOS integration

Disk-based authentication

Automatic reboot

9.6.3 Residual Risks to Keys

Intercepted Passphrase

Intercepted Keys

Eavesdrop on the encryption process

Sniffing keys from swap files

Cold boot attack

Recycled Password Attack

The “Master Key” Risk

9.7 Resources

9.7.1 Review Questions

9.7.2 Problems/Exercises

10: Connecting Computers

Chapter Outline

 

10.1 The Network Security Problem

10.1.1 Basic Network Attacks and Defenses

Example: Sharing Eve’s Printer

Policy Statements

Potential Controls

10.1.2 Physical Network Protection

Protecting External Wires

10.1.3 Host and Network Integrity

Network Worms

Botnets

Botnets in Operation

Fighting Botnets

The Insider Threat

10.2 Transmitting Information

10.2.1 Message Switching

10.2.2 Circuit Switching

10.2.3 Packet Switching

Mix and Match Network Switching

10.3 Putting Bits on a Wire

Synchronous versus Asynchronous Links

10.3.1 Wireless Transmission

Frequency, Wavelength, and Bandwidth

AM and FM Radio

Frequency sharing

Radio Propagation and Security

10.3.2 Transmitting Packets

Network Efficiency and Overhead

Acknowledgement Protocol

10.3.3 Recovering a Lost Packet

10.4 Ethernet: A Modern LAN

Today’s Ethernet

Wired

Optical

Wireless

10.4.1 Wiring a Small Network

Network Cables

10.4.2 Ethernet Frame Format

LAN Addressing

Address Format

MAC Addresses and Security

10.4.3 Finding Host Addresses

Addresses from Keyboard Commands

Addresses from Mac OS

Addresses from Microsoft Windows

10.4.4 Handling Collisions

Wireless Collisions

Wireless Retransmission

10.5 The Protocol Stack

10.5.1 Relationships Between Layers

10.5.2 The OSI Protocol Model

The Orphaned Layers

10.6 Network Applications

Servers

Peer-to-Peer

Network Applications and Information States

10.6.1 Resource Sharing

10.6.2 Data and File Sharing

Delegation: A Security Problem

10.7 Resources

10.7.1 Review Questions

10.7.2 Problems/Exercises

11: Networks of Networks

Chapter Outline

 

11.1 Building Information Networks

Network Topology: Evolution of the Phone Network

11.1.1 Point-to-Point Network

11.1.2 Star Network

11.1.3 Bus Network

11.1.4 Tree Network

11.1.5 Mesh

11.2 Combining Computer Networks

Traversing Computer Networks

The Internet Emerges

11.2.1 Hopping Between Networks

Routing Internet Packets

Counting Hops

11.2.2 Evolution of Internet Security

Protecting the ARPANET

Early Internet Attacks

Early Internet Defenses

11.2.3 Internet Structure

Autonomous Systems

Routing Security

International Rerouting

Starting an ISP

11.3 Talking Between Hosts

Socket Interface

Port Numbers

Socket Addresses

Socket API capabilities

11.3.1 IP Addresses

IP Version 6

11.3.2 IP Packet Format

Packet Fragmentation

11.3.3 Address Resolution Protocol

The ARP Cache

11.4 Internet Addresses in Practice

Network Masks

IPv4 Address Classes

11.4.1 Addresses, Scope, and Reachability

11.4.2 Private IP Addresses

Assigning Private IP Addresses

Dynamic Host Configuration Protocol

11.5 Network Inspection Tools

Wireshark

11.5.1 Wireshark Examples

Address Resolution Protocol

IP Header

11.5.2 Mapping a LAN with nmap

The Nmap Network Mapper Utility

Use Nmap with Caution

11.6 Resources

11.6.1 Review Questions

11.6.2 Problems/Exercises

12: End-to-End Networking

Chapter Outline

 

12.1 “Smart” Versus “Dumb” Networks

The End-to-End Principle

12.2 Internet Transport Protocols

User Datagram Protocol

End-to-End Transport Protocols

12.2.1 Transmission Control Protocol

Sequence and Acknowledgement Numbers

Window Size

TCP Connections

Connection Timeout

12.2.2 Attacks on Protocols

Internet Control Message Protocol

Ping Attacks

Redirection Attacks

TCP/IP Attacks

12.3 Names on the Internet

The Name Space

12.3.1 Domain Names in Practice

Using a Domain Name

12.3.2 Looking Up Names

DNS Lookup

12.3.3 DNS Protocol

Resolving a Domain Name Via Redirection

DNS Redundancy

12.3.4 Investigating Domain Names

12.3.5 Attacking DNS

Cache Poisoning

DOS Attacks on DNS Servers

Botnets

DOS Attacks and DNS Resolvers

DNS Security Improvements

12.4 Internet Gateways and Firewalls

12.4.1 Network Address Translation (NAT)

Configuring DHCP and NAT

12.4.2 Filtering and Connectivity

Packet Filtering

Inbound Connections

12.4.3 Software-based Firewalls

12.5 Long Distance Networking

12.5.1 Older technologies

Analog broadcast networks

Circuit-switched telephone systems

Analog-based digital networks

Microwave networks

Analog two-way radios

12.5.2 Mature technologies

Dedicated digital network links

Cell Phones

Cable TV

12.5.3 Evolving technologies

Optical fiber networks.

Bidirectional satellite communications

12.6 Resources

12.6.1 Review Questions

12.6.2 Problems/Exercises

13: Enterprise Computing

Chapter Outline

 

13.1 The Challenge of Community

13.1.1 Companies and Information Control

Reputation: Speaking with One Voice

Companies and Secrecy

Accountability

Need To Know

13.1.2 Enterprise Risks

Insiders and Outsiders

13.1.3 Social Engineering

Thwarting Social Engineering

13.2 Management Processes

13.2.1 Security Management Standards

Evolution of Management Standards

13.2.2 Deployment Policy Directives

Risk Acceptance

13.2.3 Management Hierarchies and Delegation

Profit Centers and Cost Centers

Implications for Information Security

13.2.4 Managing Information Resources

Managing Information Security

13.2.5 Security Audits

Compliance Audits

Security Scans

13.2.6 Information Security Professionals

Information Security Training

College Degrees

Information Security Certification

13.3 Enterprise Issues

Education, Training, and Awareness

13.3.1 Personnel Security

Employee Clearances

Employee Life Cycle

Employee Roles

Administrators and Separation of Duty

Partial Insiders

13.3.2 Physical Security

Power Management

Information System Protection

Environmental Management

13.3.3 Software Security

Software Development Security

Configuration Management

Repeatability and Traceability

Formalized Coding Activities

Avoiding Risky Practices

Software-based access controls

13.4 Enterprise Network Authentication

Local Authentication

13.4.1 Direct Authentication

13.4.2 Indirect Authentication

Ticket-based Authentication

Service-based Authentication

Redirected Authentication

Properties of Indirect Authentication

13.4.3 Off-Line Authentication

13.5 Contingency Planning

13.5.1 Data Backup and Restoration

Full Versus Partial Backups

File-Oriented Synchronized Backups

File-Oriented Incremental Backups

Image Backups

RAID as Backup

13.5.2 Handling Serious Incidents

Examining a Serious Attack

13.5.3 Disaster Preparation and Recovery

Business Impact Analysis

Recovery Strategies

Contingency Planning Process

13.6 Resources

13.6.1 Review Questions

13.6.2 Problems/Exercises

14: Network Encryption

Chapter Outline

 

14.1 Communications Security

Cryptographic Protections

14.1.1 Crypto By Layers

Link Layer Encryption and 802.11 Wireless

Network Layer Encryption and IPsec

Socket Layer Encryption with SSL/TLS

Application Layer Encryption with S/MIME or PGP

Layer-Oriented Variations

14.1.2 Administrative and Policy Issues

Sniffing Protection

Traffic Filtering

Automatic Encryption

Internet Site Access

End-to-end Crypto

Keying

14.2 Crypto Keys on a Network

The Default Keying Risk

Key Distribution Objectives

Key Distribution Strategies

Key Distribution Techniques

14.2.1 Manual Keying: A Building Block

14.2.2 Simple Rekeying

Self-Rekeying

New Keys Encrypted With Old

14.2.3 Secret-Key Building Blocks

Key Wrapping

Key Distribution Center (KDC)

Shared Secret Hashing

14.2.4 Public-Key Building Blocks

Secret Sharing with Diffie-Hellman

Wrapping a Secret with RSA

14.2.5 Public-Key versus Secret-Key Exchanges

Choosing Secret-Key Techniques

Choosing Public-Key Techniques

14.3 Crypto Atop the Protocol Stack

Privacy Enhanced Mail

Pretty Good Privacy

Adoption of Secure Email and Application Security

14.3.1 Transport Layer Security - SSL and TLS

The World Wide Web

Netscape Software

Secure Sockets Layer/Transport Layer Security

14.3.2 SSL Handshake Protocol

14.3.3 SSL Record Transmission

Message Authentication Code

Data Compression

Application Transparency and End-to-End Crypto

14.4 Network Layer Cryptography

Components of IPsec

14.4.1 The Encapsulating Security Payload (ESP)

Tunnel Mode

Transport Mode

ESP Packet Format

14.4.2 Implementing a VPN

Private IP Addressing

IPsec Gateways

Bundling Security Associations

14.4.3 Internet Key Exchange (IKE) Protocol

14.5 Link Encryption on 802.11 Wireless

Wireless Defenses

WEP

Wi-Fi Protected Access: WPA and WPA2

14.5.1 Wireless Packet Protection

Encryption processing

Integrity processing

Decryption and Validation

14.5.2 Security Associations

Establishing the Association

Establishing the Keys

14.6 Encryption Policy Summary

Prevent Sniffing

Apply Encryption Automatically

Additional Issues

14.7 Resources

14.7.1 Review Questions

14.7.2 Problems/Exercises

15: Internet Services and Email

Chapter Outline

 

15.1 Internet Services

Traditional Internet Applications

15.2 Internet Email

Message Formatting Standards

Message Headers

The To: Field

The From: Field

Additional Headers

MIME Formatting

15.2.1 Email Protocol Standards

Mailbox Protocols

POP3: An Example

Email Delivery

Port Number Summary

15.2.2 Tracking an Email

#1: From UC123 to USM01

#2: From USM01 to USM02

#3: From USM02 to MMS01

#4: From MMS01 to MMS02

15.2.3 Forging an Email Message

Authenticated Email

15.3 Email Security Problems

Connection-based attacks

15.3.1 Spam

Classic Financial Fraud

Evolution of Spam Prevention

MTA Access Restriction

Filtering on Spam Patterns

15.3.2 Phishing

Tracking a Phishing Attack

15.3.3 Email Viruses and Hoaxes

Virus Execution

Email Chain Letters

Virus Hoax Chain Letters

15.4 Enterprise Firewalls

Evolution of Internet Access Policies

A Simple Internet Access Policy

15.4.1 Controlling Internet Traffic

15.4.2 Traffic Filtering Mechanisms

Session Filtering

Application Filtering

15.4.3 Implementing Firewall Rules

Example of Firewall Security Controls

Additional Firewall Mechanisms

Firewall Rule Proliferation

15.5 Enterprise Point of Presence

Internet Service Providers

The DMZ

Intrusion Detection

Intrusion Prevention Systems

Data Loss Prevention Systems

15.5.1 POP Topology

Single Firewall Topology

Bastion Host Topology

Three-Legged Firewall

Dual Firewalls

15.5.2 Attacking an Enterprise Site

Protocol Attacks

Tunnelling

National Firewalls

15.5.3 The Challenge of Real-Time Media

15.6 Resources

15.6.1 Review

15.6.2 Problems/Exercises

16: The World Wide Web

Chapter Outline

 

16.1 Hypertext Fundamentals

Web Standards

Formatting: Hypertext Markup Language

Hypertext Links

Cascading Style Sheets

Hypertext Transfer Protocol

Retrieving data from other files or sites

16.1.1 Addressing Web Pages

Email URLs

Hosts and Authorities

Path Name

Default Web Pages

16.1.2 Retrieving a Static Web Page

Building a Page from Multiple Files

Web Servers and Statelessness

Web Directories and Search Engines

Web Crawlers

Crime via Search Engine

16.2 Basic Web Security

Client Policy Issues

Policy Motivations and Objectives

Internet Policy Directives

Strategies to Manage Web Use

Traffic Blocking

Monitoring

Training

The Tunneling Dilemma

Firewalling HTTP Tunnels

16.2.1 Security for Static Web Sites

16.2.2 Server Authentication

Mismatched Domain Name: May be Legitimate

Untrusted Certificate Authority: Difficult to Verify

Expired Certificate: Possibly Bogus, Probably Not

Revoked Certificate: Always Bogus

Invalid Digital Signature: Always Bogus

16.2.3 Server Masquerades

Bogus Certificate Authority

Misleading Domain Name

Stolen Private Key

Tricked certificate authority

16.3 Dynamic Web Sites

Web Forms and POST

16.3.1 Scripts on the Web

Scripting Languages

Client Side Scripts

Client Scripting Risks

“Same Origin” Policy

Sandboxing

16.3.2 States and HTTP

Browser Cookies

16.4 Content Management Systems

16.4.1 Database Management Systems

Structured Query Language

Enterprise Databases

Database Security

16.4.2 Password Checking: A CMS Example

Logging In to a Web Site

An Example Login Process

16.4.3 Command Injection Attacks

A Password-Oriented Injection Attack

Inside the Injection Attack

Resisting Web Site Command Injection

Input Validation

16.5 Ensuring Web Security Properties

Web Confidentiality

Serve confidential data

Collect confidential data

Web Integrity

16.5.1 Web Availability

High Availability

Continuous Operation

Continuous Availability

16.5.2 Web Privacy

Client anonymity

Anonymous proxies

Private browsing

16.6 Resources

16.6.1 Review

16.6.2 Problems/Exercises

17: Governments and Secrecy

Chapter Outline

 

17.1 Secrecy In Government

Hostile Intelligence Services

Classified Information

17.1.1 The Challenge of Secrecy

The Discipline of Secrecy

Secrecy and Information Systems

Exposure and Quarantine

17.1.2 Information Security and Operations

Intelligence and Counterintelligence

Information Operations

Operations Security

17.2 Classifications and Clearances

Classification Levels

Other Restrictions

Legal Basis for Classification

Minimizing the Amount of Classified Information

17.2.1 Security Labeling

Sensitive But Unclassified

17.2.2 Security Clearances

17.2.3 Classification Levels in Practice

Working with classified information

Higher levels have greater restrictions

17.2.4 Compartments and Other Special Controls

Sensitive Compartmented Information

SCI Clearances

Example of SCI processing

Special Access Programs

Special Intelligence Channels

Enforcing Access to Levels and Compartments

17.3 National Policy Issues

Legal elements

Federal Information Security Management Act

NIST Standards and Guidance for FISMA

Personnel roles and responsibilities

Threats, vulnerabilities, and countermeasures

17.3.1 Facets of National System Security

Physical Security

Communications Security

Information Security

Life Cycle Procedures

17.3.2 Security Planning

System Life Cycle Management

Security System Training

17.3.3 Certification and Accreditation

NIST Risk Management Framework

17.4 Communications Security

Key Leakage Through Spying

17.4.1 Cryptographic Technology

Classic Type 1 Crypto Technology

17.4.2 Crypto Security Procedures

Two-Person Integrity

Controlled Cryptographic Items

Key Management Processes

Data Transfer Device

Electronic Key Management System

17.4.3 Transmission Security

Electronic Warfare

Spread Spectrum

17.5 Data Protection

Media Handling

Media Sanitization and Destruction

17.5.1 Protected Wiring

17.5.2 TEMPEST

TEMPEST Zones

TEMPEST Red/Black

TEMPEST Separation

17.6 Trustworthy Systems

Reference Monitor

High Assurance

Trusted Systems Today

National Standards

17.6.1 Integrity of Operations

Nuclear Operations

Positive Control

Force Surety

Achieving Nuclear High Assurance

17.6.2 Multilevel Security

Rule- and Identity-Based Access Control

Covert Channels

Other Multilevel Security Problems

17.6.3 Computer Modes of Operation

Dedicated Mode

System High Mode

Compartmented or Partitioned Mode

Multilevel Mode

17.7 Resources

17.7.1 Review Questions

17.7.2 Problems/Exercises

Appendix: Alternative Security Terms and Concepts

This provides brief articles on certain topics relevant to CNSS training standards.

  • Application dependent guidance
  • Attribution of information
  • Comprehensive model of information system security
  • Configuration management of programming standards and control
  • Critical and essential workloads
  • Threats to systems
  • Information identification
  • Network security privileges (class, node)
  • Object reuse
  • Power emergency shutdown
  • Procedural controls on documentation, logs, and journals
  • Public switched network
  • Public versus private networks
  • Remanence
  • Segregation of duties
  • Threats and threat agents
  • Transmission security countermeasures

Drupal

This is a collection of pages describing experiences I've had with Drupal. 

Basics of Views Bulk Operations (VBO)

After reading and re-reading several descriptions of how to do bulk changes to a Drupal site, the Views Bulk Operation (VBO) mechanism sounded most promising. It did, however, take another couple of hours of poking at explanations to figure out how THAT works.

The Views mechanism allows the site manager to create a page or a block that displays information retrieved from the Drupal database. It's mostly intended to pull data out of nodes and display the data as a table. 

The VBO mechanism uses Views to perform database operations on nodes in the database. To use it, we create a page that, instead of simply retrieving data from the database, applies a "bulk operation" to the data. 

Here is how I used it, step-by-step:

1. Start up Views and add a new view

Give the view a name, and select "Create a page." You'll need the page in order to execute your bulk operations. Click "Continue and Edit."

2. Allow entry of fields

By default, a view will display a block of "content" text. We need to change it to display fields  of data. To do this we click on Format: Settings in the leftmost column under Page details.

Click on Force using fields, and apply.

3. Add a field to do bulk operations.

Click on the Add button to the right of the term Fields. This presents a list of fields, including two marked "Bulk Operations." Select a bulk operation.

4. Set Access: Permission so that non-privileged users can't run it.

Click on the Permission link and make sure no non-privileged users can run this view.

5. Save the whole thing to execute later

Views will display a "preview" of the operation just below the settings area. I misunderstood the preview and thought it would allow me to run the bulk operation right there. You have to save the thing, probably with a menu entry, and execute it from the menu.

6. Bring up a page on the site, and select the menu entry to run the operation.

The operation may demand details from you about the settings to change and the nodes to affect. Choose the appropriate inputs and execute the thing.

In context, I see why it works this way. From a usability standpoint, it looks like nonsense.

Migration to Drupal

Drupal Icon

After spending a few years with WordPress, I decided to migrate to Drupal. I installed WordPress in December, 2007, and replaced it in February, 2011.

Existing users had to recover their passwords, since ther was no clean way to use WordPress-encrypted passwords on a Drupal site. I also had the site off-line for part of the day while I replaced the WordPress software with the Drupal software.

My main reason for the migration is Drupal's "book" feature. Drupal makes it easy to structure short articles into a long, hierarchically-structured narrative. This is essentially how I write books anyway. This makes it easier to present complex topics built from a series of short articles.

For example, the site contains this long presentation on multilevel security. The presentation consists of several sections and subsections. The text began as a long article, and I broke it into pieces that I hooked together manually using links. The subsections should each be a separate article. Instead, I kept each section as a single article. And I had to hand-craft the navigation between sections.

The site also contains a detailed discussion of stream ciphers and the risk of reused key streams. At present the discussion consists of several separate articles that are interconnected via hyperlinks. This makes it hard to add details to the discussion.

For example, I need to post a discussion of the folly of using statistical tests to prove the effectiveness of a stream cipher's key stream. Believe it or not, there are graduate students who seriously believe this is a valid technique. Where do I put this explanation, and how do I hook it in with the existing articles? I have to fiddle with the navigation by hand in WordPress.

Drupal Security

Drupal also supports a more flexible authentication and access control discipline. I'm skeptical that this will provide better security - it's harder to mark everything correctly when you have finer-grained permissions, especially when still learning about the system.

As before, visitors need to set up an account to post comments, but don't need an account to read the site's public contents.

Instructor Discussions

Jones and Bartlett, the publisher of the upcoming Elementary Information Security, will be providing a protected area to distribute materials to course instructors who use the textbook. This would include lecture notes, figures, and answers to problem sets. So I doubt I'll need to set up such a mechanism on this site. However, I would like to encourage discussion of the text and the topics, and how best to present them to students. This may involve a mailing list or an on-line forum. I'll see how things develop.

Wordpress tag: 
Post category: 

Overview of Drupal Migration

The migration did not go without a few hitches, but it went as smoothly as might be expected for such a thing.

My inexperience with Apache's .htaccess files caused an unnecessary delay and a lot of "500 Internal Server Error" messages.

The process began with a Drupal clone of my WordPress site. I created it on a separate domain hosted by GoDaddy, my ISP. Once the clone reached a sufficient degree of done-ness, I removed the WordPress files and installed a fresh Drupal install atop it. The installation was actually performed by GoDaddy's "Host Connection" feature. Then I added the custom files I needed (graphics, themes, and plug-in modules). Once all those parts were in place, I copied the clone's Drupal database.

Migration Objectives

As I approached the migration, I realized that there were certain things I wanted to achieve:

  • Existing links and URLs should continue to work. I was especially careful about links identified by Google. One of my earliest experiences with migrating a web site was at the hands of AT&T, when they briefly owned our local cable company. They made a hideous mess of it, breaking every link I'd managed to collect.
  • The RSS feed for site articles should continue to work. I initially assumed this would happen by default. I'm glad I tested it: Drupal's feed normally links to a specific XML document while my older WordPress feed went to a path.
  • Preserve active users as much as possible. I doubted the move would be transparent to active users, but I did the best I could to cushion the impact.
  • Provide the same general features with Drupal blogging that I relied upon in WordPress. This was a huge challenge. WordPress values ease of use, while Drupal seems to value traditional defaults, even if they arose in ancient times.
  • Do as much as possible through WYSIWYG interfaces. Modern sites thrive on CSS and PHP, neither of which I know very well. I expect to learn a lot more about both in the coming months.

Migration Process

The migration involved the following steps which I'll discuss in other postings:

  • Migrate my existing content from WordPress to Drupal. This looks easy, but Drupal default behaviors actually make it difficult.
  • Ensure that Drupal provides the same text handling and post handling features as WordPress. Again, Drupal's default behaviors make this difficult.
  • Migrate the user community. This was separate from migrating the rest of the site. Meanwhile, I needed to suspend user activities on the Cryptosmith site while I actually performed the migration.

Note that these steps took place before the actual migration - I did all this to the cloned Drupal site while the WordPress Cryptosmith site remained on-line. The actual site migration involved these activities:

  • Provide a "Cryptosmith is Off-Line" page while moving from one hosting package to the other. This was easy, though it made me unrealistically confident about my ability to edit .htaccess files. 
  • Use Godaddy's hosting features to replace the WordPress site with the cloned Drupal site. This took two or three false starts due to .htaccess errors.

I expect there will be a few more topics to cover as the site evolves.

Wordpress tag: 

Migrating Blog Entries from Wordpress to Drupal

This process looks deceptively simple. WordPress happily exports all entries into a nicely formatted XML file. Drupal has a "WordPress Import" module that appears to do a comfortable import. What could possibly go wrong?

Well, in Drupal, everything comes down to a question of surprising choices for defaults. At least, if you are expecting ease of use, the default choices seem surprising.

I performed about a half-dozen imports before I was finally satisfied with the results. No doubt a Drupal expert would have nailed it the first time. But that's the problem: someone who imports another site may be the least likely to be an expert. In my case, the import is my first significant experience with Drupal. And it goes badly.

My specific problem was that the default import format discarded most of my paragraph breaks.

Input Formats

Drupal has a configuration feature called "input formats." Each format consists of three parts: a set of filters to select, configurations for those filters, and the order of those filters. One filter discards all HTML tags that aren't on an approved list. Another filter converts newlines into line or paragraph breaks. The default input formats did not automatically convert the newlines.

After a couple of attempts, I realized that there were these things called input filters, and that they should have been performing the conversion. Eventually I even realized that there was a special WordPress input format. However, the WordPress format proved to be more trouble than aid.

  • The WordPress format remains invisible until after you first try to import WordPress content. Not very helpful. I suppose there might be a way to make the secret format appear before I need it, but I never figured it out. I fell back on the expedient of importing a skeleton file of WordPress content first, and imported the serious content afterward.
  • The WordPress format doesn't actually select automatic line break conversion. I'm not sure why, since this is default behavior on WordPress.

The last thing I wanted to do was to break things at the start of my Drupal experience, so I was cautious about changing default configurations. In fact, I think I would have saved myself a world of effort if I had simply enabled the Line Break filter.

Instead, used a text editor to manually update the paragraph headers in the input text. This took about a half hour, which was a bit quicker than reinstalling a clean Drupal system and repeating the import.

I actually ended up performing this conversion twice. Once I used a global change to add paragraph breaks, but that ran into trouble with breaks that were inside paragraphs. A global change with "<br />" would have been a smarter choice.

The Probable Right Answer

If I had it to do over again, I would have enabled the Line Break Converter in the WordPress input format. I have no idea why the thing isn''t activated by default. The documentation suggests that it should do the right thing.

 

Wordpress tag: 

Blogging in Drupal

WordPress is well designed for blogging. I got used to the TinyMCE editor and easy-to-reach features to import graphics when using WordPress. I also got used to less sophisticated things like paragraph breaks and section subheadings. And I like the email alert when there's something to moderate.

I was appalled to discover that these things are omitted by default in Drupal.

A Litany of Ease-of-Use Oversights

Notice how I stuck an HTML header on the previous line. I like to use these things to break up a lengthy missive to make it easier to read. Drupal omits such things from formatted text by default.

Notice how I just started a second paragraph to address a second issue: and paragraphs, too, are considered unnecessary by Drupal's default configuration. When I talk about the "default" here I speak of "Filtered HTML." Web managers don't want regular users to enter "unfiltered" HTML because it opens the site to such things as "cross site scripting" attacks. On the other hand, we end up with hard-to-read text if our list of permitted HTML is too short.

Drupal's problems aren't limited to the defaults allowed for HTML, but they provide a good start to the list:

  • Filtered contents don't include conventional line or paragraph breaks.
  • Filtered contents don't include subsection headers, like HTML H2, H3, and so on.
  • Filtered contents don't include tables (I'm still fiddling with this one).
  • There's no built-in WYSIWYG editor - you have to install a special module and then you can install the editor software.
  • Once you install the TinyMCE editor, the default configuration provides the user with no toolbar or menu for configuring the text. You must configure such things yourself after installation.
  • Drupal does not provide an easy mechanism to upload images. That requires a separate, optional module even after you take the time to install the WYSIWYG modules and libraries.
  • There is no built-in mechanism to send alerts when it's time to moderate a comment or other new content. In fact, I couldn't find a module that obviously added this feature.

How I Fixed Them

First, I installed the WYSIWYG editor module.

To install modules in Drupal, you generally use FTP. I'm on a Macintosh and I've decided I like CyberDuck. It supports WYSIWYG uploading, downloading, and synchronization. I used to rely on DreamWeaver for synchronization, which is like killing a gnat with a hammer. Even with a nice donation, CyberDuck is much cheaper and easier to live with than DreamWeaver.

The WYSIWYG editor module does not, by itself include editors. It provides the framework within which you can install editors.

Since I like TinyMCE, I installed it as a Drupal library, which is the format that's compatible with the editor module. I simply went to the TinyMCE web site and downloaded their standard package, which is a bunch of Javascript.

Then, I fixed the WYSIWYG editor to work the way I wanted

Off the shelf, TinyMCE provides just about no functions. So you have to burrow into the "Wysiwyg profiles" under the Drupal Site Configuration.

To enable the editor, you first assign it to an input format. Then, the input format gives you a way to configure the editor.

Click on the format's "Configure" link. Then select "Buttons and plugins." This displays all of the features that may appear on the editor's GUI. The features appear as checkboxes.

Some checkboxes appear as plaintext and some appear with underlined links. I generally enable everything that's in plaintext, which is the first half of the list (about 20 boxes). I generally check all of them, plus "context menu" and "HTML block format." I also check "Teaser Break" which appears in plaintext at the bottom of the ilst.

Now, if you want to use the same editor with any other input formats - trust me, you do - then you must repeat the 20-checkbox procedure for each input format. (At least, this is the default behavior. As with most annoyances in Drupal, there may be a module to fix it, assuming you have the patience to look for one and use it. 

Then, I added image uploading

In keeping with Drupal's design philosophy, everything is optional. By itself, the WYSIWYG framework and TinyMCE allow you to create text that includes the usual, desirable elements. You can even include graphics with your text. Well, you can as long as you have a URL that points to your image.

If you want to download a particular image to go with your text, that's another story. All together now:

For that, you need a different module

I installed the IMCE module for this purpose. This provides a convenient little button on the image insertion dialog so that you can upload the image and retrieve its new URL.

Again, however, we must go back to the WYSIWYG configuration and enable IMCE for each input format.

I added paragraphs and heading support to "formatted HTML"

The only really restrictive input format is Formatted HTML, and it's generally too restrictive by default. I revised it to allow paragraph breaks, line breaks, and headings. To do this, I simply added the tags I wanted: <p>, <br>, <h2>, <h3>, and <h4>.

Finally, I added anti-spam support

Oddly enough, the AntiSpam module seems to be the only thing that knows how to contact me when it's time to moderate new content. There are various other notifier modules, but they're primarily intended to tell subscribers when new content has been published. If input is awaiting moderation, it's unpublished and these modules do nothing.

Unfortunately, the anti-spam module doesn't do the perfect job of notifying me. I get emails for every spam posting as well as every legitimate one.

Rant

Why isn't more of this done correctly by default?

Wordpress tag: 

Migrating a WordPress User List to Drupal

I'm always annoyed when I register for a web site only to have my user ID mysteriously disappear. The "scouting.org" web site has recreated itself about four times in the past decade. Each time has led to re-registration by the entire user community.

Therefore I decided to make a strong effort to retain my user community while migrating my site. The easy part was to contact those who provided email addresses and tell them what was happening. The hard part was to deal with passwords.

Step 1: Export the WordPress User List

The first step was to export the user database from WordPress. For this I installed the AmR Users plug-in produced by one AnMari, a WordPress developer. This plug-in provides an interface to construct tailored lists of registered users. I configured a list to display user IDs, "readable" names, email addresses, URLs, and comment counts. Unfortunately I could not display OpenID credentials, so I didn't have a clean way to incorporate them into the Drupal user database.

Another problem: I couldn't transfer user passwords. The passwords were safely stored in a hashed format. In a perfect world, both Drupal and WordPress would have used an identical hashing mechanism. Perhaps the mechanism could have been a standard PHP function used by both packages. In fact, each rolled their own. There were hacks that might have allowed using WordPress hashes on Drupal, but that would have required PHP coding.

A perfect world would have also allowed me to transfer the OpenID credentials. All I should have needed to do is store the appropriate information in the Drupal user database, and everything would have worked.

But that's not how things are. Maybe things will move more smoothly in a few more years.

Step 2: Clean up the user list

Like most webmasters, I get a lot of spam comments. I relied on Akismet to identify them on my WordPress site. I continue to use that service on my Drupal blog via the AntiSpam module.

Akismet killed thousands of bogus comments on Cryptosmith. To post those comments, the spammers had to create user accounts. So I had a lot of bogus accounts. I sorted the wheat from the chaff by retaining only those users whose comments had actually been kept.

I also had to delete a bunch of escaped quotations that the exported AmR User list included in its CSV format.

And I deleted users who hadn't provided email addresses. After all, existing users couldn't retrieve lost passwords without a working email address.

Step 3: Import the user list to Drupal

I installed the "User Import" module to do this. Now, this may simply reflect my ignorance of Drupal, since there seem to be some very general-purpose import mechanisms. It may be that users are a special case of "nodes" or "taxonomies" or some other basic Drupal thing, and that I could import a list of users as a special case of one of those.

In any case, the User Import module accepted the CSV from AmR User, after I massaged it with Excel and a text editor.

Drupal may - or may not - automaticaly send email to new users as soon as they are activated. I chose not to send such emails, since I imported the user list into my Drupal clone system while the main Cryptosmith site was still running WordPress. The automated message would have arrived too soon. Instead, I had a chance to review the imports and make sure that they looked correct. I also tested a few imported users.

Step 4: Tell the users about the transition

Shortly before I shut down the WordPress site, I sent an email to everyone who provided an email address to the site. The message announced the change, and assured everyone that the site contents would still be available without logging in first. I also explained that I would retain login names for users who had posted comments.

Then I performed the transition. I'll talk about that experience elsewhere.

Once the transition was finished, I sent an email to all registered users. Since the updated site was based on Drupal, I needed to install a module to send the email. I chose the "Mass Contact" module. Basic Drupal includes a "Contact" module that provides a web page for contacting the site administrator or other users (depending on how it's configured). The Mass Contact module provides a similar interface to contact specific classes of site users. I configured it to send email to all site users.

Surprising features of Mass Contact

Those of you who received the email were subjected to some of the, well, surprising implications of emailing from Drupal.

  • The Mass Contact module, working with a WYSIWYG editor, does best at producing HTML email.
  • I have this prejudice in favor of raw text email. Mass Contact allows you to select raw text email.
  • You can't really produce raw text with a WYSIWYG editor. The editor naturally inserts HTML tags to indicate the required formatting. If the only formatting consists of newlines to produce line and paragraph breaks, then you still get the tags to produce those breaks.

Thus, my bias in favor of raw text email produced a jumble of text and HTML tags in my first attempt. The Mass Contact also uses some weird scheduling discipline so that it doesn't send too much email at once, and this produces delays in email transmission. In some cases you have to wait a while to see the results of a Mass Contact email, because they aren't always transmitted immediately.

Wordpress tag: 

Taking a Site Solidly Down

If you visited Cryptosmith during the afternoon of February 5, you may have seen this:

Cryptosmith Site Down

This appeared while I was removing WordPress files from the site and inserting Drupal files. The "Site Down" display was controlled by the ".htaccess" file stored in the site's root directory. As soon as Drupal stored a new .htaccess file, links were redirected to Drupal's scripts.

I constructed the Site Down page from raw HTML and a JPEG of the Cryptosmith logo. Such pages are easy to build.

The HTML File

To begin with, here is the HTML text for such  a page:

<html>
<head>
<title>Cryptosmith</title>
</head>
<body>
<p align="middle"><img src="cslogo.JPG" /></p>
<h1 align="middle">SITE DOWN</h1>
<p align="middle">We are making major changes to the hosting software. The site
should return to service in a few hours. For more information, contact the
<a href=mailto:webmaster@cryptosmith.com>Webmaster</a></p>
</body>
</html>

The working files for this display consist of the "index.html" flile containing the above contents, a logo image refered to in the "<img src=" tag, and the .htaccess file.

The .htaccess File

The .htaccess file gives the Apache web server some specific instructions on how to respond to URLs. Those of us who rent server space from others must use .htaccess files to adjust the server's behavior. Those who actually run their own server may put this information directly in the Apache configuration file (it's more efficient that way).

If we put the following into the .htaccess file and store it in the web site's root directory, it will direct everything to the index.html page:

#
# Apache .htaccess setting for the 1-page web site
#

# Don't show directory listings for URLs which map to a directory.
Options -Indexes

# point Apache at the directory/site's home page.
DirectoryIndex index.html

# If the URL points elsewhere in the site, take it back to the home page.
ErrorDocument 404 /index.html

In the above text, the Apache server ignores the lines starting with a "#" since they are comments. Each comment explains the line following it. The "Options -Indexes" line disables directory listings: no matter how complex the site might be, visitors can't retrieve anything unless the explicit URL points to an actual file. The "DirectoryIndex" line directs Apache to the "index.html" file, which contains the contents listed earlier.

In most CMS-based web sites, including those run by WordPress, a URL path either leads to a PHP script (a file with the ".php" suffix) or it's interpreted by a PHP script at the root level. These systems use a .htaccess file that leads to an "index.php" file instead of a .html file.

When we replace the CMS .htaccess file with our own, we redirect paths to lead to "index.html." However, we've only provided one such file at the root level. The third Apache statement above, "ErrorDocument," redirects all nonexistent pages (404 errors) to our solitary home page.

The "500 Internal Server Error"

Treat all .htaccess files with respect. Any syntax errors may yield the "500 Internal Server Error" when you try to visit the web site from a browser. While adjusting the Drupal site configuration I managed to stick an error in my .htaccess file, and this yielded a couple of hours of misdirected work. The mistake probably came down to an extra space in a URL specification somewhere.

The easiest way to eliminate a 500 Server Error is to delete your .htaccess file. That should allow a complete URL (one that includes the file name) to work on your site. Then go back and try to fix the .htaccess file. See if the 500 error reappears.

Wordpress tag: 

Moving a Site on GoDaddy

I created the new Drupal-based site in parallel with the existing Wordpress site. Once the Drupal site seemed solid and provided the necessary content, I installed it over the old Wordpress site.

Why GoDaddy?

I moved my hosting from a local provider to GoDaddy about five years ago. The local provider found it easier to provide virtual hosts to customers who did their own host management. I prefer shared hosting since there's less management. The local provider provided shared-hosting support via email - I'd ask for something via email and it would happen in the next few hours.

GoDaddy offered the whole package: domain registration, SSL certificates, shared hosting, and even some package installation, via a web interface. So I no longer had to change domain aliasing or host configuration via email.

I'm currently evaluating Drupal performance under GoDaddy. Page handling seems excessively slow, especially when logged in. The customer service people suggest trying their new "4GH' service. This apparently hosts the site on scalable multiprocessors.

The Transition Process

The transition process moved the Drupal prototype site onto my main Cryptosmith domain. This was the cheapest and easiest way for me to do it.

  1. I created the prototype Drupal site. This contained a copy of every file I wanted to have on the new account. I created a subdirectory on Cryptosmith and assigned a full .COM domain name to it. Then I told GoDaddy to install Drupal there. Once installed, I migrated the posts and users from the Wordpress site. Then I configured Drupal to mimic Wordpress blogging features.
  2. I did a complete backup of the Drupal site and its database. I gave the backed-up database a special name so I'd recognize it when moving the site.
  3. I took the Wordpress site offline. I used a hand-crafted web page and .htaccess file to do this, since I knew the Wordpress files would be disappearing part way through the process.
  4. I told GoDaddy to delete the Wordpress site. This really only deleted the database. I have several separate GoDaddy hosting accounts and had originally hosted Wordpress on a different account. The Wordpress software on Cryptosmith used the database on a different account and used a no-longer-existing domain name.
  5. I deleted the remaining Cryptosmith files by hand.
  6. I told GoDaddy to install Drupal on Cryptosmith. This provided a skeletal Drupal site that GoDaddy could identify and track with its built-in host support.
  7. I copied the prototype Drupal files atop the skeleton files in Cryptosmith.
  8. I restored the prototype Drupal database atop the skeleton database in Cryptosmith.
  9. I fixed the .htaccess file and the site went live.

What Really Happened

Like true love, the course of migration rarely runs smooth.

I used Cyberduck to upload my web sites to my desktop, and Cyberduck choked when it hit access-restricted files. It's annoying to perform a large-scale file copy only to have it die tragically part way through the process. I had to restart the process and carve around the access-restricted files. I'm not exactly sure why GoDaddy has stuck a few files in my site directory over which I don't have control, but that's what I found.

Once I had copied the needed files over to the Cryptosmith domain, I was greeted with a series of "500 Internal Server Error" messages. Instead of immediately searching help fora for obvious causes of such a message, i repeated my migration steps a few times, hoping to eliminate the problem through brute force.

The typical cause of "500 Internal Server Error" is a flaw in a .htaccess file. Once I went back to an earlier .htaccess file and took more care with my editing, the problem went away and the new site went live.