Visit the textbook site eisec.us for the latest supporting materials and study guides.
Draft materials are provided below:
Jones & Bartlett Learning, November, 2011.
The only textbook verified by the US Government to conform fully to the Committee on National
Security Systems' national training standard for information security professionals (NSTISSI 4011)
This comprehensive, accessible Information Security text is ideal for the one-term, undergraduate college course. The text integrates risk assessment and security policy throughout the text, since security systems work best at achieving goals they are designed to meet, and security policy ties real-world goals to security mechanisms. Early chapters in the text discuss individual computers and small LANS, while later chapters deal with distributed site security and the Internet. Cryptographic topics follow the same progression, starting on a single computer and evolving to Internet-level connectivity. Mathematical concepts throughout the text are defined and tutorials with mathematical tools are provided to ensure students grasp the information at hand.
See below for sample contents. Sample chapters are also available from the publisher's site.
1.1. The Security Landscape
1.2. Process Example: Bob’s Computer
1.4. Identifying Risks
1.5. Prioritizing Risks
1.6. Ethical Issues in Security Analysis
1.7. Security Example: Aircraft Hijacking
2.1. Computers and Programs
2.2. Programs and Processes
2.3. Buffer Overflow and The Morris Worm
2.4. Access Control Strategies
2.5. Keeping Processes Separate
2.6. Security Policy and Implementation
2.7. Security Plan: Process Protection
3.1. The File System
3.2. Executable Files
3.3. Sharing and Protecting Files
3.4. Security Controls for Files
3.5. File Security Controls
3.6. Patching Security Flaws
3.7. Process Example: The Horse
3.8. Chapter Resources
4.1. Controlled Sharing
4.2. File Permission Flags
4.3. Access Control Lists
4.4. Microsoft Windows ACLs
4.5. A Different Trojan Horse
4.6. Phase Five: Monitoring The System
4.7. Chapter Resources
5.1. Phase Six: Recovery
5.2. Digital Evidence
5.3. Storing Data on a Hard Drive
5.4. FAT: An Example File System
5.5. Modern File Systems
5.6. Input/Output and File System Software
6.1. Unlocking a Door
6.2. Evolution of Password Systems
6.3. Password Guessing
6.4. Attacks on Password Bias
6.5. Authentication Tokens
6.6. Biometric Authentication
6.7. Authentication Policy
7.1. Protecting the Accessible
7.2. Encryption and Cryptanalysis
7.3. Computer-Based Encryption
7.4. File Encryption Software
7.5. Digital Rights Management
8.1. The Key Management Challenge
8.2. The Reused Key Stream Problem
8.3. Public-key Cryptography
8.4. RSA: Rivest-Shamir-Adleman
8.5. Data Integrity and Digital Signatures
8.6. Publishing Public Keys
9.1. Securing a Volume
9.2. Block Ciphers
9.3. Block Cipher Modes
9.4. Encrypting a Volume
9.5. Encryption in Hardware
9.6. Managing Encryption Keys
10.1. The Network Security Problem
10.2. Transmitting Information
10.3. Putting Bits on a Wire
10.4. Ethernet: A Modern LAN
10.5. The Protocol Stack
10.6. Network Applications
11.1. Building Information Networks
11.2. Combining Computer Networks
11.3. Talking Between Hosts
11.4. Internet Addresses in Practice
11.5. Network Inspection Tools
12.1. “Smart” Versus “Dumb” Networks
12.2. Internet Transport Protocols
12.3. Names on the Internet
12.4. Internet Gateways and Firewalls
12.5. Long Distance Networking
13.1. The Challenge of Community
13.2. Management Processes
13.3. Enterprise Issues
13.4. Enterprise Network Authentication
13.5. Contingency Planning
14.1. Communications Security
14.2. Crypto Keys on a Network
14.3. Crypto Atop the Protocol Stack
14.4. Network Layer Cryptography
14.5. Link Encryption on 802.11 Wireless
14.6. Encryption Policy Summary
15.1. Internet Services
15.2. Internet Email
15.3. Email Security Problems
15.4. Enterprise Firewalls
15.5. Enterprise Point of Presence
16.1. Hypertext Fundamentals
16.2. Basic Web Security
16.3. Dynamic Web Sites
16.4. Content Management Systems
16.5. Ensuring Web Security Properties
17.1. Secrecy In Government
17.2. Classifications and Clearances
17.3. National Policy Issues
17.4. Communications Security
17.5. Data Protection
17.6. Trustworthy Systems
The goal of this textbook is to introduce college students to information security. Security often involves social and organizational skills as well as technical understanding. To solve practical security problems, we must balance real-world risks and rewards against the cost and bother of available security techniques. The text uses continuous process improvement to integrate these elements.
Security is a broad field. Some students may excel in the technical aspects, while others may shine in the more social or process-oriented aspects. Many successful students fall between these poles. The text offers opportunities for all types of students to excel.
If we want a solid understanding of security technology, we must look closely at the strengths and weaknesses of underlying information technology itself. This requires a background in computer architecture, operating systems, and computer networking. It’s hard for a typical college student to achieve breadth and depth in these subjects and still have time to really study security.
Instead of leaving a gap in students’ understanding, this book provides introductions to essential technical topics. Chapter 2 explains the basics of computer operation and instruction execution. This prepares students for a description of process separation and protection, which illustrates the essential role of operating systems in enforcing security.
Chapter 5 introduces file systems and input/output in modern operating systems. This lays a foundation for forensic file system analysis. It also shows students how a modern operating system organizes a complex service. This sets the stage for Chapter 10’s introduction to computer networking and protocol software.
Introducing Continuous Process Improvement
The text organizes security problem-solving around a six-phase security process. Chapter 1 introduces the process as a way of structuring information about a security event, and presents a simple approach to risk analysis. Chapter 2 introduces security policies as a way to state security objectives, and security controls as a way to implement a policy. Subsequent chapters introduce system monitoring and incident response as ways to assess system security and improve it.
Each step in the process builds on earlier steps. Each step also provides a chance to assess how well our work addresses our security needs. This is the essence of continuous process improvement.
In order to give students an accurate view of process improvement, the text introduces document structures that provide cross references between different steps of the process. We use elements of each earlier phase to construct information in the following phase, and we often provide a link back to earlier data to ensure complete coverage. While this may seem like nit-picking in some cases, it allows mastery of essential forms of communication in the technical and professional world.
When used as a textbook, the material is intended for lower division undergraduates, or for students in a two-year community college program. Students should have completed high school mathematics. Typical students should have completed an introductory computing or programming course.
Instructors may want to use this book for either a one or two semester course. A one semester course would usually cover one chapter a week; the instructor may want to combine a couple of earlier chapters or skip the final chapter. Some institutions may find it more effective to teach the material over a full year. This gives the students more time to work with the concepts and to cover all topics in depth.
Following the style of my earlier books, this text focuses on diagrams and practical explanations to present fundamental concepts. This makes the material clearer to all readers and makes it more accessible to the math-phobic reader. Many concepts, particularly in cryptography, can be clearly presented in either a diagram or in mathematical notation. This text uses both, with a bias towards diagrams.
Many fundamental computing concepts are wildly abstract. This is also true in security, where we sometimes react to illusions perceived as risks. To combat this, the text incorporates a series of concrete examples played out by characters with names familiar to those who read cryptographic papers: Bob, Alice, and Eve. They are joined by additional classmates named Tina and Kevin, since different people have different security concerns.
The material in this text fulfills curriculum requirements published by the US government and the Association for Computing Machinery (ACM). In particular, the text covers all required topics for training information systems security professionals under the Information Assurance Courseware Evaluation Program (NSTISSI #4011) established by the US National Security Agency (NSA). The text also provides substantial coverage of the required topics for training senior system managers (CNSSI #4012) and for system administrators (CNSSI #4013).
The text also covers the core learning outcomes for information security education published in the ACM’s “IT 2008” curricular recommendations for Information Technology education. As a reviewer and contributor to the published recommendations, the author is familiar with its guidelines.
Students who are interested in becoming a Certified Information System Security Professional (CISSP) may use this book as a study aid for the examination. All key areas of the CISSP Common Body of Knowledge are covered in this text. Certification requires four or five years of professional experience in addition to passing the exam.
Information security is a fascinating but abstract subject. This text introduces students to real-world security problem solving, and incorporates security technology into the problem-solving process. There are lots of “how to” books that explain how to harden a computer system.
Many readers and most students need more than a “how to” book. They need to decide which security measures are most important, or how to trade off between alternatives. Such decisions often depend on what assets are at stake and what risks the computer’s owner are willing to take.
The practitioner’s most important task is the planning and analysis that helps choose and justify a set of security measures. In my own experience, and in that of my most capable colleagues, security systems succeed when we anticipate the principal risks and design the system to address them. In fact, once we have identified what we want for security (our policy requirements), we don’t need security gurus to figure out what we get (our implementation). We just need capable programmers, testers, and administrators.
As the chapter unfolds, we encounter certain key terms indicated in bold italics. These highlight essential terms that students may encounter in the information security community. Successful students will recognize these terms.
The Resources section at the end of each chapter lists the key terms and provides review and exercises. The review questions help students confirm that they have absorbed the essential concepts. Some instructors may want to use these as recitation or quiz questions. The problems and exercises give students the opportunity to solve problems based on techniques presented in the text.
Here is an incomplete collection of reading materials associated with the textbook. Visit the Elementary Infosec site for a complete set of reading and study materials.
Some links lead to on-line articles published by professional societies like the ACM (Association for Computing Machinery) or IEEE (Institute of Electrical and Electronic Engineers). Serious computing experts often join one or both of these, and sign up for electronic library subscriptions. Many college and university libraries may also provide free access to these for students and faculty.
Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.
The following blogs provide readable reports and commentary on information security.
In Japan, the term kaizen embodies the continuous improvement process.
During World War II, military enterprises poured vast resources into various techincal projects, notably radar, codebreaking, and the atomic bomb. Those successes encouraged the peacetime military to pursue other large scale technical projects. A typical project would start with a very large budget and a vague completion date a few years in the future. But in practice, many of these projects vastly exceeded both the budget and the predicted time table.
A handful of projects, notably the Polaris Missile project, achieved success while adhering closely to their initial budget and schedule estimates. Pressure on defense budgets led the US DOD to identify features of the successful projects so they might be applied to future work. This was the genesis of systems engineering. The DOD's Defense Aquisition University has produced an introduction to systems engineering (PDF format) in the defense community.
The International Council on Systems Engineering (INCOSE) provides a more general view of systems engineering. NASA also provides on-line training materials on their Space Systems Engineering site.
Chapter 1 introduces the first three of eight basic principles of information security.
Different authorities present different lists of principles. International standards bodies, including NIST in the US, tend to produce very general lists of principles, reflecting notions such as "be safe," "keep records," and other generalizations (for example, see NIST's SP 800-14: "Generally Accepted Principles and Practices for Securing Information Technology Systems"). These principles represent basic truths about security, but few are stated in a way that helps one make security decisions.
Saltzer and Schroeder produced a now-classic list based on experience with the Multics time sharing system in the 1970s: "The Protection of Information in Computer Systems," Proceedings of the IEEE 63, 9 (September, 1975). Some of these principles reflect features of the Multics system while others reflect some well-known shortcomings with most systems of that time. Copies exist online at Saltzer's own web site and at the University of Virginia.
There is also a Cryptosmith blog post that compares the textbook's list of principles with those in Saltzer and Schroeder.
A high-level security analysis provides a brief summary of a security situation at a given point of time. The next section provides a checklist for writing a high-level security analysis.
The risk assessment processes noted in the textbook are all avaliable online:
The fundamental reference for everything related to the events of September 11, 2001, is the Final Report of the National Commission on Terrorist Attacks Upon the United States, a.k.a. "The 9/11 Commission Report," published in 2004.
Following 9/11, the BBC published a brief history of airline hijackings. The 2003 Centennial of Flight web site provides a more general summary of violent incidents in aviation security. In 2007, New York Magazine published a more detailed hijacking time-line in conjunction with breaking news on the D. B. Cooper hijacking case. The US FBI web site contains a lot of information about the D. B. Cooper case, including a 2007 update.
The textbook presents a high-level security analysis as a short writing exercise that summarizes a security situation. The analysis generally describes a situation at a particular point in time. For example, the 9/11 discussion in the textbook describes air travel security before 9/11. The analysis describes the six phases of the security process:
Here is a checklist of the basic properties of a high-level analysis:
Note that a complete security plan will also cover the six phases, but it is not limited to this length. A complete plan covers each phase thoroughly.
Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.
Here is a 26-minute video of middle school students visiting a "walk through" computer to learn the basics of computer operation. Taken from the PBS TV series "Newton's Apple," 1990. Although the technology is over 20 years old, the fundamental components remain the same, except for speed speed, size, and capacity.
There are, of course, countless images and videos available through on-line searching that show specific elements of computer systems.
Students who have not yet studied these topics in detail will want to visit web sites that provide an introduction to binary and hex. YouTube user Ryan of Aberdeen has created a video tutorial (9 minutes). There are also written tutorials:
A faculty member at NC State University maintains a site that provides an overview of the Morris worm.
Eugene Spafford (aka spaf) wrote a report describing the worm, its operations, and its effects (PDF), shortly after the incident.
Eichin and Rochlis of MIT published a report of the worm incident from the MIT perspective (PDF). This was presented at the IEEE Symposium on Security and Privacy the following year.
In 1990, Peter Denning published a book that brought together several papers on the Morris worm and other security issues emerging at that time, titled Computers Under Attack: Intruders, Worms and Viruses.
Spafford also maintains an archive of worm-related information at Purdue University.
Auguste Kerckhoffs' original paper on cryptographic system design recommended that cryptographic systems be published and that secrecy should reside entirely in a secret key. The paper was published in French in 1883. Portions of Kerckhoffs' paper are available on-line including partial English translations.
Claude Shannon's "Communication Theory of Secrecy Systems" (Bell System Technical Journal, vol. 28, no. 4, 1949) contains his assumption "the enemy knows the system being used" (italics his). Bell Labs provides general information about Shannon's work and publications.
Eric Raymond published a famous essay on the benefits of open design and of sharing program source code in general, called "The Cathedral and The Bazaar." This essay has inspired many members of the Open Source community.
Butler Lampson introduced the access matrix in his 1971 paper, "Protection." Lampson has posted a copy of his paper on-line in several formats.
This is, unfortunately, a lot harder than it seems. As RAM has grown smaller and I/O grown more complex, motherboard components have changed dramatically in size and appearance. Here are suggestions on identifying key features in older and newer motherboards.
The most reliable way to identify a motherboard's contents is to locate a copy of its installation manual. These are usually posted on the web by the motherboard's manufacturer. Most boards clearly include the manufacturer's name and the board's model number.
If the manufacturer and model number aren't obvious, it may be possible to identify the motherboard using Google Images. Enter the word "motherboard" as a search term along with other textual items on the board. Compare the images displayed with the color and layout of the motherboard in question. Keep in mind everything should match when you find the correct board. Missing or misplaced features indicate that the boards don't match. A popular motherboard may appear many times, but most images will lead to pages that indicate the manufacturer and model. In some cases, the image may lead directly to the manufacturer's own pages.
Visit the Quizlet web site to find study aids for the chapter's acronyms and terms by following the links below. Many students prefer to study the acronyms first.
Security consultant Fred Cohen performed much of the pioneering analysis of computer viruses. His web site contains several useful articles on virus technology. Even though some of the material is 30 years old, the basic technical truths remain unchanged.
Some anti-virus vendors provide summaries of current anti-virus and malware activities:
Here is a list of malware briefly described in the textbook, plus links to in-depth reports on each one. Check recent news: security experts occasionally make progress in eradicating one or another of these, but the botnets sometimes recover. Many of these are PDFs.
Videos: Ralph Langer, a German expert in control systems security, gave a TED talk describing Stuxnet (~11 minutes). Bruce Dang of Microsoft also gave a detailed presentation about Stuxnet (75 minutes) at a conference.
Butler Lampson introduced the access matrix in his 1971 paper, "Protection" (PDF). Lampson has posted a copy on-line in several formats.
Although most modern systems use resource-oriented permissions to control access rights, there are a few cases that use capabilities, which associate rights with active entities like programs and users. Jack Dennis and Earl Van Horn of MIT introduced the notion of capabilities in their 1965 report "Programming Semantics for Multiprogrammed Computers," which was published in Communications of the ACM in 1966.
Marc Stiegler has posted an interesting introduction to capability based security that ties it to other important security concepts. The EROS OS project has also posted an essay that explains capability-based security. For a thorough coverage of capability based architecture circa 1984, see Henry Levy's book Capability-Based Computer Systems. He has posted it on-line.
Microsoft has posted an article that describes access control on Windows files and on the Windows "registry," a special database of system-specific information.
Electrical engineers have relied on state diagrams for decades to help design complicated circuits. The technique is also popular with some software engineers, though it rarely finds its way into courses on computer programming. Any properly-constructed state diagram may be translated into a state table that provides the same information in a tabular form. Tony Kuphaldt's free on-line textbook Lessons in Electric Circuits explains state machines in the context of electric circuits in Volume IV, Chapter 11: Sequential Circuits-Counters.
Upper-lever computer science students may encounter state diagrams in a course on automata theory in which they use such diagrams to represent deterministic finite automata. Such mechanisms can handle the simplest type of formal language, a regular grammar. Most people encounter regular grammars as regular expressions, an arcane syntax used to match text patterns when performing complicated search-and-replace operations in text editors.
Students introduced to modern structured design techniques using the Unified Modeling Language (UML) often use state machine diagrams or state charts (a diagram in table form). On-line tutorials about UML state machines appear at Kennesaw State University and the Agile Modeling web site.
In the US, there are several organizations that track and report on information security vulnerabilities. Many of these organizations provide email alerts and other data feeds to keep subscribers up to date on emerging vulnerabilities. Some organizations provide their services to particular communities (e.g. government or military organizations, or customers of a vendor's products) while others provide reports to the public at large.
The SANS Internet Storm Center also provides a variety of on-line news feeds and reports, as well as a continuously-updated "Infocon Status" to indicate unusual changes in the degree of malicious activity on the Internet. Click on the image below to visit the Internet Storm Center for further information on current vulnerabilities and malicious Internet activity.
In 2000, Arbaugh, Fithen, and McHugh wrote an article describing a life-cycle model of information security vulnerabilities titled "Windows of Vulnerability: A Case Study Analysis", (IEEE Computer 33, December 2000). The authors have posted a copy of the article online (PDF).
The library at Stanford posted a brief history of the Trojan War. Although Homer's Iliad tells the story of the Trojan War, it says very little about the Greek trickery that led to the city's fall. The story is more the province of Virgil's Aeneid.
In the 1970s, Guy Steele at MIT started collecting bits of jargon used in the computer community. This yielded "The Jargon File," which Steele maintained for several years until it was passed on to Eric Raymond. According to the Jargon File, the term Trojan horse entered the computing lexicon via Dan Edwards of MIT and the NSA.
US-CERT has published a two-page guide on how to deal with a Trojan horse or virus infection on a computer (PDF).
There are numerous on-line tutorials on Unix and/or Linux file permissions, including ones provided by:
ACLs first appeared in the Multics timesharing system, as described in the paper "A General-Purpose File System For Secondary Storage," by R. Daley and Peter Neumann (Proc. 1965 Fall Joint Computer Conference) and on the Multicians web site.
Since ACLs could provide very specific access restrictions, they became recommended features of high-security systems. When the US DOD developed the "Trusted Computer System Evaluation Criteria," (PDF) (a.k.a. the TCSEC or Orange Book) ACLs were an essential feature of higher security systems. Modern security products are evaluated against the Common Criteria.
While traditional Unix systems did not have ACLs, more advanced versions of Unix incorporated them, partly to meet high security requirements like those in the Orange Book. This led to the development of POSIX ACLs as part of a proposed POSIX 1003.1e standard. The standards effort was abandoned, but several Unix-based systems did incorporate POSIX ACLs. Here are examples:
The ACL user interface on Mac OS-X is very simple. In fact, the OS-X ACLs are based on POSIX ACLs and may incorporate more spohisticated settings and inheritances than we see in the Finder's "Information" display. These features are available through special ACL options of the chmod shell command. One developer has produced an application called Sandbox that provides a more extensive GUI for managing the ACLs.
It can be challenging to find accurate online information about Windows ACLs, because the computer-based access controls are often confused with network-based access controls. The MS Developer Network provides general information about ACLs.
Researchers at CMU evaluated the Windows XP version of ACLs in a series of experiments documented in "Improving user-interface dependability through mitigation of human error," Intl J. Human-Computer Studies 63 (2005) 25-50, by Maxion and Reeder.
Here is a summary of memory size names and their corresponding address sizes. Many people memorize this type of information naturally through working with computer technology over time or during a professional career.
If you want to memorize these values, visit the Quizlet page. The page tests your knowledge of the smaller sizes (K, M, G, T), how these sizes are related (i.e. a terabyte is a thousand billion bytes), and how they relate to memory sizes (a TB needs an address approximately 40 bits long).
Here is a simple shortcut for estimating the number of bits required to address storage of a given size.
103 ~ 210
To put this into practice, we do the following:
Let's work out an example with a terabyte: a trillion-byte memory.
Cryptographers develop new hash functions every few years because cryptanalysts and mathematicians find weaknesses in the older ones. Valerie Aurora provides a graphic illustration of this.
This section of the textbook provides details not otherwise addressed by the main text.
Two educational standards in information system security refer to closely-related models of information system security. First, we have a US government training standard:
Second, we have an academic curriculum standard:
1.1 The Security Landscape
Not Just Computers Any More
1.1.1 Making Security Decisions
1.1.2 The Security Process
1.1.3 Continuous Improvement: A Basic Principle
The Roots of Continuous Improvement
1.2 Process Example: Bob’s Computer
1.3 Assets and Risk Assessment
Fine Points of Terminology
1.3.1 What Are We Protecting?
1.3.2 Security Boundaries
Least Privilege: A Second Basic Principle
Example: Boundaries in a Dorm
Analyzing the Boundary
The Insider Threat
1.3.3 Security Architecture
Defense In Depth: A Third Basic Principle
1.3.4 Risk Assessment Overview
1.4 Identifying Risks
1.4.1 Threat Agents
1.4.2 Security Properties, Services, and Attacks
1.5 Prioritizing Risks
1.5.1 Example: Risks to Alice’s Laptop
Step 1: Identify Computing Assets
Step 2: Identify Threat Agents and Potential Attacks
Step 3: Estimate the Likelihood of Individual Attacks
Step 4: Estimate the Impact of Attacks over Time
Step 5: Calculate the Impact of Each Attack
1.5.2 Other Risk Assessment Processes
1.6 Ethical Issues in Security Analysis
Laws, Regulations, and Codes of Conduct
1.6.1 Searching for Vulnerabilities
1.6.2 Sharing or Publishing Vulnerabilities
1.7 Security Example: Aircraft Hijacking
1.7.1 Hijacking: A High-Level Analysis
1.7.2 September 11, 2001
1.8.1 Review Questions
1.8.2 Exercises and Problems
2.1 Computers and Programs
Parallel Versus Serial Wiring
2.1.2 Program Execution
Separating Data and Control
2.2 Programs and Processes
2.2.1 Switching Between Processes
Observing Active Processes
2.2.2 The Operating System
2.3 Buffer Overflow and The Morris Worm
The ‘finger’ program
2.3.1 The ‘finger’ Overflow
The Worm Released
2.3.2 Security Alerts
2.4 Access Control Strategies
2.4.1 Puzzles and Patterns
Open Design: A Basic Principle
Cryptography and Open Design
Pattern-based Access Control
2.4.2 Chain of Control: Another Basic Principle
Controlling the BIOS
Subverting the Chain of Control
2.5 Keeping Processes Separate
Evolution of Personal Computers
Security on Personal Computers
Operating System Security Features
2.5.1 Sharing a Program
2.5.2 Sharing Data
2.6 Security Policy and Implementation
Constructing Alice’s Security Plan
Writing a Security Policy
2.6.1 Analyzing Alice’s Risks
2.6.2 Constructing Alice’s Policy
2.6.3 Alice’s Security Controls
Alice’s Backup Procedure
2.7 Security Plan: Process Protection
Policy for Process Protection
Functional Security Controls
The Dispatcher’s Design Description
The Design Features
The Dispatching Procedure
Security Controls for Process Protection
2.8.1 Review Questions
2.8.2 Problems and Exercises
3.1 The File System
File and Directory Path Names
3.1.1 File Ownership and Access Rights
File Access Rights
Initial File Protection
3.1.2 Directory Access Rights
3.2 Executable Files
3.2.1 Execution Access Rights
Types of Executable Files
3.2.2 Computer Viruses
3.2.3 Macro Viruses
3.2.4 Modern Malware: A Rogue’s GAllery
Conficker, also called Downadup
3.3 Sharing and Protecting Files
3.3.1 Policies for Sharing and Protection
Underlying System Policy
User Isolation Policy
User File Sharing Policy
3.4 Security Controls for Files
3.4.1 Deny by Default: A Basic Principle
The opposite of Deny by Default
3.4.2 Managing Access Rights
Capabilities in Practice
3.5 File Security Controls
3.5.1 File Permission Flags
System and Owner Access Rights in Practice
3.5.2 Security Controls to Enforce Bob’s Policy
3.5.3 States and State Diagrams
3.6 Patching Security Flaws
The Patching Process
Security Flaws and Exploits
Windows of Vulnerability
3.7 Process Example: The Horse
3.7.1 Troy: A High-Level Analysis
3.7.2 Analyzing the Security Failure
3.8 Chapter Resources
3.8.1 Review Questions
4.1 Controlled Sharing
Tailored File Security Policies
Bob’s Sharing Dilemma
4.1.1 Basic File Sharing on Windows
4.1.2 User Groups
4.1.3 Least Privilege and Administrative Users
Administration by Regular Users
User Account Control on Windows
4.2 File Permission Flags
4.2.1 Permission Flags and Ambiguities
4.2.2 Permission Flag Examples
Security Controls for the File Sharing Policy
4.3 Access Control Lists
Modern ACL Implementations
4.3.1 POSIX ACLs
4.3.2 Macintosh OS-X ACLs
4.4 Microsoft Windows ACLs
4.4.1 Denying Access
Determining Access Rights
Building Effective ACLs
4.4.2 Default File Protection
Moving and Copying Files
4.5 A Different Trojan Horse
A Trojan Horse Program
Transitive Trust: A Basic Principle
4.6 Phase Five: Monitoring The System
Catching an intruder
4.6.1 Logging Events
A Log Entry
The Event Logging Mechanism
Detecting Attacks by Reviewing the Logs
4.6.2 External Security Requirements
Laws, Regulations, and Industry Rules
External Requirements and the Security Process
4.7 Chapter Resources
4.7.1 Review Questions
5.1 Phase Six: Recovery
Incidents and Damage
5.1.1 The Aftermath of an Incident
Fault and Due Diligence
5.1.2 Legal Disputes
Resolving a Legal Dispute
5.2 Digital Evidence
The Fourth Amendment
5.2.1 Collecting Legal Evidence
Collecting Evidence at The Scene
Securing the Scene
Documenting the Scene
5.2.2 Digital Evidence Procedures
Authenticating a Hard Drive
5.3 Storing Data on a Hard Drive
Magnetic Recording and Tapes
Hard Drive Fundamentals
5.3.1 Hard Drive Controller
5.3.2 Hard Drive Formatting
High Level Format
5.3.3 Error Detection and Correction
Cyclic Redundancy Checks
Error Correcting Codes
5.3.4 Hard Drive Partitions
Partitioning to Support Older Drive Formats
Partitioning in Modern Systems
Partitioning and Fragmentation
Hiding Data with Partitions
5.3.5 Memory Sizes and Address Variables
Address, Index, and Pointer Variables
Memory Size Names and Acronyms
Estimating the Number of Bits
5.4 FAT: An Example File System
5.4.1 Boot Blocks
5.4.2 Building Files from Clusters
An Example FAT File
FAT Format Alternatives
5.4.3 FAT Directories
Long File Names
Undeleting a File
5.5 Modern File Systems
File System Design Goals
Conflicting File System Objectives
5.5.1 Unix File System
5.5.2 Apple’s HFS Plus
5.5.3 Microsoft’s NTFS
5.6 Input/Output and File System Software
File System Software
5.6.1 Software Layering
5.6.2 A Typical I/O Operation
Part A: Call the operating system
Part B: OS constructs the I/O operation
Part C: The driver starts the actual I/O device
Part D: The I/O operation ends
5.6.3 Security and I/O
Restricting the devices themselves
Restricting Parameters in I/O Operations
File Access Restrictions
5.7.1 Review Questions
6.1 Unlocking a Door
6.1.1 Authentication Factors
6.1.2 Threats and Risks
Attack Strategy: Low Hanging Fruit
6.2 Evolution of Password Systems
Password Hashing in Practice
6.2.1 One-way Hash Functions
Modern Hash Functions
A Cryptographic Building Block
6.2.2 Sniffing Credentials
6.3 Password Guessing
DOD Password Guideline
Off-line Password Cracking
6.3.1 Password Search Space
6.3.2 Truly Random Password Selection
6.3.3 Cracking Speeds
6.4 Attacks on Password Bias
Bias and Entropy
6.4.1 Biased Choices and Average Attack Space
Average Attack Space
Biased Password Selection
Measuring Likelihood, not Certainty
Making Independent Guesses
Example: 4-digit luggage lock
6.4.2 Estimating Language-Based Password Bias
Klein’s Password Study
6.5 Authentication Tokens
Passive Authentication Tokens
6.5.1 Challenge-Response Authentication
Another Cryptographic Building Block
Direct Connect Tokens
6.5.2 One-time Password Tokens
A Token’s Search Space
Average Attack Space
Attacking One-Time Password Tokens
Guessing a Credential
6.5.3 Token Vulnerabilities
6.6 Biometric Authentication
6.6.1 Biometric Accuracy
6.6.2 Biometric Vulnerabilities
6.7 Authentication Policy
6.7.1 Weak and Strong Threats
Effect of Location
6.7.2 Policies for Weak Threat Environments
A Household Policy
A Workplace Policy: Passwords Only
A Workplace Policy: Passwords and Tokens
6.7.3 Policies for Strong and Extreme Threats
Passwords Alone for Strong Threats
Passwords Plus Biometrics
Passwords Plus Tokens
Constructing the Policy
6.7.4 Password Selection and Handling
Strong But Memorable Passwords
The Strongest Passwords
6.8.1 Review Questions
7.1 Protecting the Accessible
7.1.1 Process Example: The Encrypted Diary
7.1.2 Encryption Basics
Categories of Encryption
A Process View of Encryption
Shared Secret Keys
7.1.3 Encryption and Information States
Illustrating Policy with a State Diagram
Proof of Security
7.2 Encryption and Cryptanalysis
7.2.1 The Vignère Cipher
7.2.2 Electromechanical Encryption
7.3 Computer-Based Encryption
The Data Encryption Standard
The Advanced Encryption Standard
Predicting Cracking Speeds
7.3.1 Exclusive Or: A Crypto Building Block
7.3.2 Stream Ciphers: Another Building Block
Generating a Key Stream
An Improved Key Stream
7.3.3 Key Stream Security
Pseudo-Random Number Generators
The Effects of Ciphertext Errors
7.3.4 The One-time Pad
Soviet Espionage and One-time pads
Practical One-time pads
7.4 File Encryption Software
7.4.1 Built-in File Encryption
7.4.2 Encryption Application Programs
Ensuring Secure File Encryption
Protecting the Secret Key
7.4.3 Erasing a Plaintext File
Risks That Demand Overwriting
Preventing Low Level Data Recovery
Erasing Optical Media
7.4.4 Choosing a File Encryption Program
Software Security Checklist
File Encryption Security Checklist
Cryptographic Product Evaluation
7.5 Digital Rights Management
The DVD Content Scrambling System
7.6.1 Review Questions
8.1 The Key Management Challenge
Levels of Risk
Key Sharing Procedures
Distributing New Keys
8.1.2 Using Text for Encryption Keys
Taking Advantage of Longer Passphrases
Software Checklist for Key Handling
8.1.3 Key Strength
8.2 The Reused Key Stream Problem
8.2.1 Avoiding Reused Keys
Changing the Internal Key
Combining the Key with a Nonce
Software Checklist for Internal Keys Using Nonces
8.2.2 Key Wrapping: Another Building Block
Key Wrapping and Cryptoperiods
Software Checklist for Wrapped Keys
8.2.3 Separation of Duty: A Basic Principle
Separation of Duty with Encryption
8.2.4 DVD Key Handling
8.3 Public-key Cryptography
Attacking Public Keys
8.3.1 Sharing a Secret: Diffie-Hellman
Perfect Forward Secrecy
Variations of Diffie-Hellman
8.3.2 Diffie-Hellman: The Basics of the Math
8.3.3 Elliptic Curve Cryptography
8.4 RSA: Rivest-Shamir-Adleman
8.4.1 Encapsulating Keys with RSA
8.4.2 An Overview of RSA Mathematics
Brute Force Attacks on RSA
The Original Challenge
The Factoring Problem
Selecting a Key Size
Other Attacks on RSA
8.5 Data Integrity and Digital Signatures
8.5.1 Detecting Malicious Changes
One-way Hash Functions
8.5.2 Detecting a Changed Hash Value
8.5.3 Digital Signatures
8.6 Publishing Public Keys
8.6.1 Public-Key Certificates
8.6.2 Chains of Certificates
Web of Trust
Trickery with Certificates
8.6.3 Authenticated Software Updates
8.7.1 Review Questions
9.1 Securing a Volume
9.1.1 Risks To Volumes
Discarded Hard Drives
9.1.2 Risks and Policy Trade-offs
Identifying Critical Data
Policy for Unencrypted Volumes
Policy for Encrypted Volumes
9.2 Block Ciphers
Building a Block Cipher
The Effect of Ciphertext Errors
9.2.1 Evolution of DES and AES
DES and Lucifer
The Development of AES
9.2.2 The RC4 Story
RC4 Leaking, Then Cracking
9.2.3 Qualities of Good Encryption Algorithms
Explicitly designed for encryption
Security does not rely on its secrecy
Available for analysis
Subjected to analysis
No practical weaknesses
Choosing an Encryption Algorithm
9.3 Block Cipher Modes
9.3.1 Stream Cipher Modes
9.3.2 Cipher Feedback Mode
9.3.3 Cipher Block Chaining
9.4 Encrypting a Volume
Choosing a Cipher Mode
Hardware Versus Software
9.4.1 Volume Encryption in Software
Files as Encrypted Volumes
9.4.2 Adapting an Existing Mode
Drive Encryption with Counter Mode
Constructing the Counter
An Integrity Risk
Drive Encryption with CBC Mode
Integrity Issues with CBC Encryption
9.4.3 A “Tweakable” Encryption Mode
9.4.4 Residual Risks
Looking for plaintext
9.5 Encryption in Hardware
Recycling the Drive
9.5.1 The Drive Controller
9.5.2 Drive Locking and Unlocking
9.6 Managing Encryption Keys
9.6.1 Key Storage
Working key storage in hardware
Persistent Key Storage
Managing removable keys
9.6.2 Booting an Encrypted Drive
9.6.3 Residual Risks to Keys
Eavesdrop on the encryption process
Sniffing keys from swap files
Cold boot attack
Recycled Password Attack
The “Master Key” Risk
9.7.1 Review Questions
10.1 The Network Security Problem
10.1.1 Basic Network Attacks and Defenses
Example: Sharing Eve’s Printer
10.1.2 Physical Network Protection
Protecting External Wires
10.1.3 Host and Network Integrity
Botnets in Operation
The Insider Threat
10.2 Transmitting Information
10.2.1 Message Switching
10.2.2 Circuit Switching
10.2.3 Packet Switching
Mix and Match Network Switching
10.3 Putting Bits on a Wire
Synchronous versus Asynchronous Links
10.3.1 Wireless Transmission
Frequency, Wavelength, and Bandwidth
AM and FM Radio
Radio Propagation and Security
10.3.2 Transmitting Packets
Network Efficiency and Overhead
10.3.3 Recovering a Lost Packet
10.4 Ethernet: A Modern LAN
10.4.1 Wiring a Small Network
10.4.2 Ethernet Frame Format
MAC Addresses and Security
10.4.3 Finding Host Addresses
Addresses from Keyboard Commands
Addresses from Mac OS
Addresses from Microsoft Windows
10.4.4 Handling Collisions
10.5 The Protocol Stack
10.5.1 Relationships Between Layers
10.5.2 The OSI Protocol Model
The Orphaned Layers
10.6 Network Applications
Network Applications and Information States
10.6.1 Resource Sharing
10.6.2 Data and File Sharing
Delegation: A Security Problem
10.7.1 Review Questions
11.1 Building Information Networks
Network Topology: Evolution of the Phone Network
11.1.1 Point-to-Point Network
11.1.2 Star Network
11.1.3 Bus Network
11.1.4 Tree Network
11.2 Combining Computer Networks
Traversing Computer Networks
The Internet Emerges
11.2.1 Hopping Between Networks
Routing Internet Packets
11.2.2 Evolution of Internet Security
Protecting the ARPANET
Early Internet Attacks
Early Internet Defenses
11.2.3 Internet Structure
Starting an ISP
11.3 Talking Between Hosts
Socket API capabilities
11.3.1 IP Addresses
IP Version 6
11.3.2 IP Packet Format
11.3.3 Address Resolution Protocol
The ARP Cache
11.4 Internet Addresses in Practice
IPv4 Address Classes
11.4.1 Addresses, Scope, and Reachability
11.4.2 Private IP Addresses
Assigning Private IP Addresses
Dynamic Host Configuration Protocol
11.5 Network Inspection Tools
11.5.1 Wireshark Examples
Address Resolution Protocol
11.5.2 Mapping a LAN with nmap
The Nmap Network Mapper Utility
Use Nmap with Caution
11.6.1 Review Questions
12.1 “Smart” Versus “Dumb” Networks
The End-to-End Principle
12.2 Internet Transport Protocols
User Datagram Protocol
End-to-End Transport Protocols
12.2.1 Transmission Control Protocol
Sequence and Acknowledgement Numbers
12.2.2 Attacks on Protocols
Internet Control Message Protocol
12.3 Names on the Internet
The Name Space
12.3.1 Domain Names in Practice
Using a Domain Name
12.3.2 Looking Up Names
12.3.3 DNS Protocol
Resolving a Domain Name Via Redirection
12.3.4 Investigating Domain Names
12.3.5 Attacking DNS
DOS Attacks on DNS Servers
DOS Attacks and DNS Resolvers
DNS Security Improvements
12.4 Internet Gateways and Firewalls
12.4.1 Network Address Translation (NAT)
Configuring DHCP and NAT
12.4.2 Filtering and Connectivity
12.4.3 Software-based Firewalls
12.5 Long Distance Networking
12.5.1 Older technologies
Analog broadcast networks
Circuit-switched telephone systems
Analog-based digital networks
Analog two-way radios
12.5.2 Mature technologies
Dedicated digital network links
12.5.3 Evolving technologies
Optical fiber networks.
Bidirectional satellite communications
12.6.1 Review Questions
13.1 The Challenge of Community
13.1.1 Companies and Information Control
Reputation: Speaking with One Voice
Companies and Secrecy
Need To Know
13.1.2 Enterprise Risks
Insiders and Outsiders
13.1.3 Social Engineering
Thwarting Social Engineering
13.2 Management Processes
13.2.1 Security Management Standards
Evolution of Management Standards
13.2.2 Deployment Policy Directives
13.2.3 Management Hierarchies and Delegation
Profit Centers and Cost Centers
Implications for Information Security
13.2.4 Managing Information Resources
Managing Information Security
13.2.5 Security Audits
13.2.6 Information Security Professionals
Information Security Training
Information Security Certification
13.3 Enterprise Issues
Education, Training, and Awareness
13.3.1 Personnel Security
Employee Life Cycle
Administrators and Separation of Duty
13.3.2 Physical Security
Information System Protection
13.3.3 Software Security
Software Development Security
Repeatability and Traceability
Formalized Coding Activities
Avoiding Risky Practices
Software-based access controls
13.4 Enterprise Network Authentication
13.4.1 Direct Authentication
13.4.2 Indirect Authentication
Properties of Indirect Authentication
13.4.3 Off-Line Authentication
13.5 Contingency Planning
13.5.1 Data Backup and Restoration
Full Versus Partial Backups
File-Oriented Synchronized Backups
File-Oriented Incremental Backups
RAID as Backup
13.5.2 Handling Serious Incidents
Examining a Serious Attack
13.5.3 Disaster Preparation and Recovery
Business Impact Analysis
Contingency Planning Process
13.6.1 Review Questions
14.1 Communications Security
14.1.1 Crypto By Layers
Link Layer Encryption and 802.11 Wireless
Network Layer Encryption and IPsec
Socket Layer Encryption with SSL/TLS
Application Layer Encryption with S/MIME or PGP
14.1.2 Administrative and Policy Issues
Internet Site Access
14.2 Crypto Keys on a Network
The Default Keying Risk
Key Distribution Objectives
Key Distribution Strategies
Key Distribution Techniques
14.2.1 Manual Keying: A Building Block
14.2.2 Simple Rekeying
New Keys Encrypted With Old
14.2.3 Secret-Key Building Blocks
Key Distribution Center (KDC)
Shared Secret Hashing
14.2.4 Public-Key Building Blocks
Secret Sharing with Diffie-Hellman
Wrapping a Secret with RSA
14.2.5 Public-Key versus Secret-Key Exchanges
Choosing Secret-Key Techniques
Choosing Public-Key Techniques
14.3 Crypto Atop the Protocol Stack
Privacy Enhanced Mail
Pretty Good Privacy
Adoption of Secure Email and Application Security
14.3.1 Transport Layer Security - SSL and TLS
The World Wide Web
Secure Sockets Layer/Transport Layer Security
14.3.2 SSL Handshake Protocol
14.3.3 SSL Record Transmission
Message Authentication Code
Application Transparency and End-to-End Crypto
14.4 Network Layer Cryptography
Components of IPsec
14.4.1 The Encapsulating Security Payload (ESP)
ESP Packet Format
14.4.2 Implementing a VPN
Private IP Addressing
Bundling Security Associations
14.4.3 Internet Key Exchange (IKE) Protocol
14.5 Link Encryption on 802.11 Wireless
Wi-Fi Protected Access: WPA and WPA2
14.5.1 Wireless Packet Protection
Decryption and Validation
14.5.2 Security Associations
Establishing the Association
Establishing the Keys
14.6 Encryption Policy Summary
Apply Encryption Automatically
14.7.1 Review Questions
15.1 Internet Services
Traditional Internet Applications
15.2 Internet Email
Message Formatting Standards
The To: Field
The From: Field
15.2.1 Email Protocol Standards
POP3: An Example
Port Number Summary
15.2.2 Tracking an Email
#1: From UC123 to USM01
#2: From USM01 to USM02
#3: From USM02 to MMS01
#4: From MMS01 to MMS02
15.2.3 Forging an Email Message
15.3 Email Security Problems
Classic Financial Fraud
Evolution of Spam Prevention
MTA Access Restriction
Filtering on Spam Patterns
Tracking a Phishing Attack
15.3.3 Email Viruses and Hoaxes
Email Chain Letters
Virus Hoax Chain Letters
15.4 Enterprise Firewalls
Evolution of Internet Access Policies
A Simple Internet Access Policy
15.4.1 Controlling Internet Traffic
15.4.2 Traffic Filtering Mechanisms
15.4.3 Implementing Firewall Rules
Example of Firewall Security Controls
Additional Firewall Mechanisms
Firewall Rule Proliferation
15.5 Enterprise Point of Presence
Internet Service Providers
Intrusion Prevention Systems
Data Loss Prevention Systems
15.5.1 POP Topology
Single Firewall Topology
Bastion Host Topology
15.5.2 Attacking an Enterprise Site
15.5.3 The Challenge of Real-Time Media
16.1 Hypertext Fundamentals
Formatting: Hypertext Markup Language
Cascading Style Sheets
Hypertext Transfer Protocol
Retrieving data from other files or sites
16.1.1 Addressing Web Pages
Hosts and Authorities
Default Web Pages
16.1.2 Retrieving a Static Web Page
Building a Page from Multiple Files
Web Servers and Statelessness
Web Directories and Search Engines
Crime via Search Engine
16.2 Basic Web Security
Client Policy Issues
Policy Motivations and Objectives
Internet Policy Directives
Strategies to Manage Web Use
The Tunneling Dilemma
Firewalling HTTP Tunnels
16.2.1 Security for Static Web Sites
16.2.2 Server Authentication
Mismatched Domain Name: May be Legitimate
Untrusted Certificate Authority: Difficult to Verify
Expired Certificate: Possibly Bogus, Probably Not
Revoked Certificate: Always Bogus
Invalid Digital Signature: Always Bogus
16.2.3 Server Masquerades
Bogus Certificate Authority
Misleading Domain Name
Stolen Private Key
Tricked certificate authority
16.3 Dynamic Web Sites
Web Forms and POST
16.3.1 Scripts on the Web
Client Side Scripts
Client Scripting Risks
“Same Origin” Policy
16.3.2 States and HTTP
16.4 Content Management Systems
16.4.1 Database Management Systems
Structured Query Language
16.4.2 Password Checking: A CMS Example
Logging In to a Web Site
An Example Login Process
16.4.3 Command Injection Attacks
A Password-Oriented Injection Attack
Inside the Injection Attack
Resisting Web Site Command Injection
16.5 Ensuring Web Security Properties
Serve confidential data
Collect confidential data
16.5.1 Web Availability
16.5.2 Web Privacy
17.1 Secrecy In Government
Hostile Intelligence Services
17.1.1 The Challenge of Secrecy
The Discipline of Secrecy
Secrecy and Information Systems
Exposure and Quarantine
17.1.2 Information Security and Operations
Intelligence and Counterintelligence
17.2 Classifications and Clearances
Legal Basis for Classification
Minimizing the Amount of Classified Information
17.2.1 Security Labeling
Sensitive But Unclassified
17.2.2 Security Clearances
17.2.3 Classification Levels in Practice
Working with classified information
Higher levels have greater restrictions
17.2.4 Compartments and Other Special Controls
Sensitive Compartmented Information
Example of SCI processing
Special Access Programs
Special Intelligence Channels
Enforcing Access to Levels and Compartments
17.3 National Policy Issues
Federal Information Security Management Act
NIST Standards and Guidance for FISMA
Personnel roles and responsibilities
Threats, vulnerabilities, and countermeasures
17.3.1 Facets of National System Security
Life Cycle Procedures
17.3.2 Security Planning
System Life Cycle Management
Security System Training
17.3.3 Certification and Accreditation
NIST Risk Management Framework
17.4 Communications Security
Key Leakage Through Spying
17.4.1 Cryptographic Technology
Classic Type 1 Crypto Technology
17.4.2 Crypto Security Procedures
Controlled Cryptographic Items
Key Management Processes
Data Transfer Device
Electronic Key Management System
17.4.3 Transmission Security
17.5 Data Protection
Media Sanitization and Destruction
17.5.1 Protected Wiring
17.6 Trustworthy Systems
Trusted Systems Today
17.6.1 Integrity of Operations
Achieving Nuclear High Assurance
17.6.2 Multilevel Security
Rule- and Identity-Based Access Control
Other Multilevel Security Problems
17.6.3 Computer Modes of Operation
System High Mode
Compartmented or Partitioned Mode
17.7.1 Review Questions
This provides brief articles on certain topics relevant to CNSS training standards.
This is a collection of pages describing experiences I've had with Drupal.
After reading and re-reading several descriptions of how to do bulk changes to a Drupal site, the Views Bulk Operation (VBO) mechanism sounded most promising. It did, however, take another couple of hours of poking at explanations to figure out how THAT works.
The Views mechanism allows the site manager to create a page or a block that displays information retrieved from the Drupal database. It's mostly intended to pull data out of nodes and display the data as a table.
The VBO mechanism uses Views to perform database operations on nodes in the database. To use it, we create a page that, instead of simply retrieving data from the database, applies a "bulk operation" to the data.
Here is how I used it, step-by-step:
1. Start up Views and add a new view
Give the view a name, and select "Create a page." You'll need the page in order to execute your bulk operations. Click "Continue and Edit."
2. Allow entry of fields
By default, a view will display a block of "content" text. We need to change it to display fields of data. To do this we click on Format: Settings in the leftmost column under Page details.
Click on Force using fields, and apply.
3. Add a field to do bulk operations.
Click on the Add button to the right of the term Fields. This presents a list of fields, including two marked "Bulk Operations." Select a bulk operation.
4. Set Access: Permission so that non-privileged users can't run it.
Click on the Permission link and make sure no non-privileged users can run this view.
5. Save the whole thing to execute later
Views will display a "preview" of the operation just below the settings area. I misunderstood the preview and thought it would allow me to run the bulk operation right there. You have to save the thing, probably with a menu entry, and execute it from the menu.
6. Bring up a page on the site, and select the menu entry to run the operation.
The operation may demand details from you about the settings to change and the nodes to affect. Choose the appropriate inputs and execute the thing.
In context, I see why it works this way. From a usability standpoint, it looks like nonsense.
After spending a few years with WordPress, I decided to migrate to Drupal. I installed WordPress in December, 2007, and replaced it in February, 2011.
Existing users had to recover their passwords, since ther was no clean way to use WordPress-encrypted passwords on a Drupal site. I also had the site off-line for part of the day while I replaced the WordPress software with the Drupal software.
My main reason for the migration is Drupal's "book" feature. Drupal makes it easy to structure short articles into a long, hierarchically-structured narrative. This is essentially how I write books anyway. This makes it easier to present complex topics built from a series of short articles.
For example, the site contains this long presentation on multilevel security. The presentation consists of several sections and subsections. The text began as a long article, and I broke it into pieces that I hooked together manually using links. The subsections should each be a separate article. Instead, I kept each section as a single article. And I had to hand-craft the navigation between sections.
The site also contains a detailed discussion of stream ciphers and the risk of reused key streams. At present the discussion consists of several separate articles that are interconnected via hyperlinks. This makes it hard to add details to the discussion.
For example, I need to post a discussion of the folly of using statistical tests to prove the effectiveness of a stream cipher's key stream. Believe it or not, there are graduate students who seriously believe this is a valid technique. Where do I put this explanation, and how do I hook it in with the existing articles? I have to fiddle with the navigation by hand in WordPress.
Drupal also supports a more flexible authentication and access control discipline. I'm skeptical that this will provide better security - it's harder to mark everything correctly when you have finer-grained permissions, especially when still learning about the system.
As before, visitors need to set up an account to post comments, but don't need an account to read the site's public contents.
Jones and Bartlett, the publisher of the upcoming Elementary Information Security, will be providing a protected area to distribute materials to course instructors who use the textbook. This would include lecture notes, figures, and answers to problem sets. So I doubt I'll need to set up such a mechanism on this site. However, I would like to encourage discussion of the text and the topics, and how best to present them to students. This may involve a mailing list or an on-line forum. I'll see how things develop.
The migration did not go without a few hitches, but it went as smoothly as might be expected for such a thing.
My inexperience with Apache's .htaccess files caused an unnecessary delay and a lot of "500 Internal Server Error" messages.
The process began with a Drupal clone of my WordPress site. I created it on a separate domain hosted by GoDaddy, my ISP. Once the clone reached a sufficient degree of done-ness, I removed the WordPress files and installed a fresh Drupal install atop it. The installation was actually performed by GoDaddy's "Host Connection" feature. Then I added the custom files I needed (graphics, themes, and plug-in modules). Once all those parts were in place, I copied the clone's Drupal database.
As I approached the migration, I realized that there were certain things I wanted to achieve:
The migration involved the following steps which I'll discuss in other postings:
Note that these steps took place before the actual migration - I did all this to the cloned Drupal site while the WordPress Cryptosmith site remained on-line. The actual site migration involved these activities:
I expect there will be a few more topics to cover as the site evolves.
This process looks deceptively simple. WordPress happily exports all entries into a nicely formatted XML file. Drupal has a "WordPress Import" module that appears to do a comfortable import. What could possibly go wrong?
Well, in Drupal, everything comes down to a question of surprising choices for defaults. At least, if you are expecting ease of use, the default choices seem surprising.
I performed about a half-dozen imports before I was finally satisfied with the results. No doubt a Drupal expert would have nailed it the first time. But that's the problem: someone who imports another site may be the least likely to be an expert. In my case, the import is my first significant experience with Drupal. And it goes badly.
My specific problem was that the default import format discarded most of my paragraph breaks.
Drupal has a configuration feature called "input formats." Each format consists of three parts: a set of filters to select, configurations for those filters, and the order of those filters. One filter discards all HTML tags that aren't on an approved list. Another filter converts newlines into line or paragraph breaks. The default input formats did not automatically convert the newlines.
After a couple of attempts, I realized that there were these things called input filters, and that they should have been performing the conversion. Eventually I even realized that there was a special WordPress input format. However, the WordPress format proved to be more trouble than aid.
The last thing I wanted to do was to break things at the start of my Drupal experience, so I was cautious about changing default configurations. In fact, I think I would have saved myself a world of effort if I had simply enabled the Line Break filter.
Instead, used a text editor to manually update the paragraph headers in the input text. This took about a half hour, which was a bit quicker than reinstalling a clean Drupal system and repeating the import.
I actually ended up performing this conversion twice. Once I used a global change to add paragraph breaks, but that ran into trouble with breaks that were inside paragraphs. A global change with "<br />" would have been a smarter choice.
If I had it to do over again, I would have enabled the Line Break Converter in the WordPress input format. I have no idea why the thing isn''t activated by default. The documentation suggests that it should do the right thing.
WordPress is well designed for blogging. I got used to the TinyMCE editor and easy-to-reach features to import graphics when using WordPress. I also got used to less sophisticated things like paragraph breaks and section subheadings. And I like the email alert when there's something to moderate.
I was appalled to discover that these things are omitted by default in Drupal.
Notice how I stuck an HTML header on the previous line. I like to use these things to break up a lengthy missive to make it easier to read. Drupal omits such things from formatted text by default.
Notice how I just started a second paragraph to address a second issue: and paragraphs, too, are considered unnecessary by Drupal's default configuration. When I talk about the "default" here I speak of "Filtered HTML." Web managers don't want regular users to enter "unfiltered" HTML because it opens the site to such things as "cross site scripting" attacks. On the other hand, we end up with hard-to-read text if our list of permitted HTML is too short.
Drupal's problems aren't limited to the defaults allowed for HTML, but they provide a good start to the list:
To install modules in Drupal, you generally use FTP. I'm on a Macintosh and I've decided I like CyberDuck. It supports WYSIWYG uploading, downloading, and synchronization. I used to rely on DreamWeaver for synchronization, which is like killing a gnat with a hammer. Even with a nice donation, CyberDuck is much cheaper and easier to live with than DreamWeaver.
The WYSIWYG editor module does not, by itself include editors. It provides the framework within which you can install editors.
Off the shelf, TinyMCE provides just about no functions. So you have to burrow into the "Wysiwyg profiles" under the Drupal Site Configuration.
To enable the editor, you first assign it to an input format. Then, the input format gives you a way to configure the editor.
Click on the format's "Configure" link. Then select "Buttons and plugins." This displays all of the features that may appear on the editor's GUI. The features appear as checkboxes.
Some checkboxes appear as plaintext and some appear with underlined links. I generally enable everything that's in plaintext, which is the first half of the list (about 20 boxes). I generally check all of them, plus "context menu" and "HTML block format." I also check "Teaser Break" which appears in plaintext at the bottom of the ilst.
Now, if you want to use the same editor with any other input formats - trust me, you do - then you must repeat the 20-checkbox procedure for each input format. (At least, this is the default behavior. As with most annoyances in Drupal, there may be a module to fix it, assuming you have the patience to look for one and use it.
In keeping with Drupal's design philosophy, everything is optional. By itself, the WYSIWYG framework and TinyMCE allow you to create text that includes the usual, desirable elements. You can even include graphics with your text. Well, you can as long as you have a URL that points to your image.
If you want to download a particular image to go with your text, that's another story. All together now:
For that, you need a different module
I installed the IMCE module for this purpose. This provides a convenient little button on the image insertion dialog so that you can upload the image and retrieve its new URL.
Again, however, we must go back to the WYSIWYG configuration and enable IMCE for each input format.
The only really restrictive input format is Formatted HTML, and it's generally too restrictive by default. I revised it to allow paragraph breaks, line breaks, and headings. To do this, I simply added the tags I wanted: <p>, <br>, <h2>, <h3>, and <h4>.
Oddly enough, the AntiSpam module seems to be the only thing that knows how to contact me when it's time to moderate new content. There are various other notifier modules, but they're primarily intended to tell subscribers when new content has been published. If input is awaiting moderation, it's unpublished and these modules do nothing.
Unfortunately, the anti-spam module doesn't do the perfect job of notifying me. I get emails for every spam posting as well as every legitimate one.
Why isn't more of this done correctly by default?
I'm always annoyed when I register for a web site only to have my user ID mysteriously disappear. The "scouting.org" web site has recreated itself about four times in the past decade. Each time has led to re-registration by the entire user community.
Therefore I decided to make a strong effort to retain my user community while migrating my site. The easy part was to contact those who provided email addresses and tell them what was happening. The hard part was to deal with passwords.
The first step was to export the user database from WordPress. For this I installed the AmR Users plug-in produced by one AnMari, a WordPress developer. This plug-in provides an interface to construct tailored lists of registered users. I configured a list to display user IDs, "readable" names, email addresses, URLs, and comment counts. Unfortunately I could not display OpenID credentials, so I didn't have a clean way to incorporate them into the Drupal user database.
Another problem: I couldn't transfer user passwords. The passwords were safely stored in a hashed format. In a perfect world, both Drupal and WordPress would have used an identical hashing mechanism. Perhaps the mechanism could have been a standard PHP function used by both packages. In fact, each rolled their own. There were hacks that might have allowed using WordPress hashes on Drupal, but that would have required PHP coding.
A perfect world would have also allowed me to transfer the OpenID credentials. All I should have needed to do is store the appropriate information in the Drupal user database, and everything would have worked.
But that's not how things are. Maybe things will move more smoothly in a few more years.
Akismet killed thousands of bogus comments on Cryptosmith. To post those comments, the spammers had to create user accounts. So I had a lot of bogus accounts. I sorted the wheat from the chaff by retaining only those users whose comments had actually been kept.
I also had to delete a bunch of escaped quotations that the exported AmR User list included in its CSV format.
And I deleted users who hadn't provided email addresses. After all, existing users couldn't retrieve lost passwords without a working email address.
I installed the "User Import" module to do this. Now, this may simply reflect my ignorance of Drupal, since there seem to be some very general-purpose import mechanisms. It may be that users are a special case of "nodes" or "taxonomies" or some other basic Drupal thing, and that I could import a list of users as a special case of one of those.
In any case, the User Import module accepted the CSV from AmR User, after I massaged it with Excel and a text editor.
Drupal may - or may not - automaticaly send email to new users as soon as they are activated. I chose not to send such emails, since I imported the user list into my Drupal clone system while the main Cryptosmith site was still running WordPress. The automated message would have arrived too soon. Instead, I had a chance to review the imports and make sure that they looked correct. I also tested a few imported users.
Shortly before I shut down the WordPress site, I sent an email to everyone who provided an email address to the site. The message announced the change, and assured everyone that the site contents would still be available without logging in first. I also explained that I would retain login names for users who had posted comments.
Then I performed the transition. I'll talk about that experience elsewhere.
Once the transition was finished, I sent an email to all registered users. Since the updated site was based on Drupal, I needed to install a module to send the email. I chose the "Mass Contact" module. Basic Drupal includes a "Contact" module that provides a web page for contacting the site administrator or other users (depending on how it's configured). The Mass Contact module provides a similar interface to contact specific classes of site users. I configured it to send email to all site users.
Those of you who received the email were subjected to some of the, well, surprising implications of emailing from Drupal.
Thus, my bias in favor of raw text email produced a jumble of text and HTML tags in my first attempt. The Mass Contact also uses some weird scheduling discipline so that it doesn't send too much email at once, and this produces delays in email transmission. In some cases you have to wait a while to see the results of a Mass Contact email, because they aren't always transmitted immediately.
If you visited Cryptosmith during the afternoon of February 5, you may have seen this:
This appeared while I was removing WordPress files from the site and inserting Drupal files. The "Site Down" display was controlled by the ".htaccess" file stored in the site's root directory. As soon as Drupal stored a new .htaccess file, links were redirected to Drupal's scripts.
I constructed the Site Down page from raw HTML and a JPEG of the Cryptosmith logo. Such pages are easy to build.
To begin with, here is the HTML text for such a page:
<p align="middle"><img src="cslogo.JPG" /></p>
<h1 align="middle">SITE DOWN</h1>
<p align="middle">We are making major changes to the hosting software. The site
should return to service in a few hours. For more information, contact the
The working files for this display consist of the "index.html" flile containing the above contents, a logo image refered to in the "<img src=" tag, and the .htaccess file.
The .htaccess file gives the Apache web server some specific instructions on how to respond to URLs. Those of us who rent server space from others must use .htaccess files to adjust the server's behavior. Those who actually run their own server may put this information directly in the Apache configuration file (it's more efficient that way).
If we put the following into the .htaccess file and store it in the web site's root directory, it will direct everything to the index.html page:
# Apache .htaccess setting for the 1-page web site
# Don't show directory listings for URLs which map to a directory.
# point Apache at the directory/site's home page.
# If the URL points elsewhere in the site, take it back to the home page.
ErrorDocument 404 /index.html
In the above text, the Apache server ignores the lines starting with a "#" since they are comments. Each comment explains the line following it. The "Options -Indexes" line disables directory listings: no matter how complex the site might be, visitors can't retrieve anything unless the explicit URL points to an actual file. The "DirectoryIndex" line directs Apache to the "index.html" file, which contains the contents listed earlier.
In most CMS-based web sites, including those run by WordPress, a URL path either leads to a PHP script (a file with the ".php" suffix) or it's interpreted by a PHP script at the root level. These systems use a .htaccess file that leads to an "index.php" file instead of a .html file.
When we replace the CMS .htaccess file with our own, we redirect paths to lead to "index.html." However, we've only provided one such file at the root level. The third Apache statement above, "ErrorDocument," redirects all nonexistent pages (404 errors) to our solitary home page.
Treat all .htaccess files with respect. Any syntax errors may yield the "500 Internal Server Error" when you try to visit the web site from a browser. While adjusting the Drupal site configuration I managed to stick an error in my .htaccess file, and this yielded a couple of hours of misdirected work. The mistake probably came down to an extra space in a URL specification somewhere.
The easiest way to eliminate a 500 Server Error is to delete your .htaccess file. That should allow a complete URL (one that includes the file name) to work on your site. Then go back and try to fix the .htaccess file. See if the 500 error reappears.
I created the new Drupal-based site in parallel with the existing Wordpress site. Once the Drupal site seemed solid and provided the necessary content, I installed it over the old Wordpress site.
I moved my hosting from a local provider to GoDaddy about five years ago. The local provider found it easier to provide virtual hosts to customers who did their own host management. I prefer shared hosting since there's less management. The local provider provided shared-hosting support via email - I'd ask for something via email and it would happen in the next few hours.
GoDaddy offered the whole package: domain registration, SSL certificates, shared hosting, and even some package installation, via a web interface. So I no longer had to change domain aliasing or host configuration via email.
I'm currently evaluating Drupal performance under GoDaddy. Page handling seems excessively slow, especially when logged in. The customer service people suggest trying their new "4GH' service. This apparently hosts the site on scalable multiprocessors.
The transition process moved the Drupal prototype site onto my main Cryptosmith domain. This was the cheapest and easiest way for me to do it.
Like true love, the course of migration rarely runs smooth.
I used Cyberduck to upload my web sites to my desktop, and Cyberduck choked when it hit access-restricted files. It's annoying to perform a large-scale file copy only to have it die tragically part way through the process. I had to restart the process and carve around the access-restricted files. I'm not exactly sure why GoDaddy has stuck a few files in my site directory over which I don't have control, but that's what I found.
Once I had copied the needed files over to the Cryptosmith domain, I was greeted with a series of "500 Internal Server Error" messages. Instead of immediately searching help fora for obvious causes of such a message, i repeated my migration steps a few times, hoping to eliminate the problem through brute force.
The typical cause of "500 Internal Server Error" is a flaw in a .htaccess file. Once I went back to an earlier .htaccess file and took more care with my editing, the problem went away and the new site went live.