Server+ Certification Prep

Eighteen pages that define the exam

The document that defines this course is eighteen pages long. CompTIA's Server+ Certification Exam Objectives for exam SK0-005, document version 5.0, spends its first three pages on the test itself, the next eleven on 25 numbered objectives, three more on acronyms, and the last page on equipment for practice. That last page is shorter than most people expect. It asks for a computer capable of virtualization, some cables, a USB flash drive, a server operating system, virtualization software and antivirus. A rack, a UPS, a KVM switch, a network switch and a storage device are marked as ideal but not necessary.

So the exam's own authors expect you to prepare mostly inside virtual machines, which is exactly how this course works. This lesson covers three things you need before any server topic: what the exam is and how it is scored, how to read the objectives so you know what kind of question each one produces, and how to build the lab you will use in nearly every lesson that follows.

The score is scaled, which means 750 is not 75 percent of questions. CompTIA converts your raw performance onto the 100 to 900 scale, and it does not publish the conversion. Treat the practice questions in this course as a check on understanding, not as a predictor of a scaled score.

Four domains, weighted unevenly

The objectives are grouped into four domains, and CompTIA states how much of the examination each one represents. The weights matter for planning. If you have forty hours left, spending ten of them on rack rails would be a poor trade.

The item counts in the fourth column are simple arithmetic on the published percentages, rounded. CompTIA publishes weights, not question counts, and a form may hold fewer than 90 items, so read them as proportions. Notice that Domain 1 has only three objectives but they are long ones: 1.1 alone covers racking, power, network cabling, chassis types and every major component inside the box. Domain 2 has eight objectives and the largest weight, which is why it gets thirteen lessons here.

Key idea: Server Administration and Troubleshooting together make up 58 percent of the exam. Hardware facts matter, but most of your marks come from running servers and fixing them.

Reading an objective by its first verb

Every objective starts with a verb phrase, and the phrase tells you what kind of question it will produce. In SK0-005, twelve objectives begin "Given a scenario," eight begin "Explain" and five begin "Summarize."

Under each objective sits a block of bullet points: RAID levels, file systems, commands, backup types. CompTIA's document says plainly that these lists are examples and not a complete list of what can appear. A question may mention a related tool or term that is not bulleted. The bullets show you the territory; they are not a fence around it.

Some questions are performance-based questions, usually called PBQs. CompTIA's explainer lists Server+ among the exams that use simulation PBQs: a mock-up of a tool or screen, such as a terminal, a configuration page or a diagram to complete. Simulations have a reset button, you can skip one and come back to it, your work is saved as you go, and partial credit is possible. CompTIA does not say which objectives the simulations are drawn from. A task-based item needs a task, though, and the "Given a scenario" objectives are the ones written as tasks, so those are where hands-on practice pays most.

Which version you are studying for

CompTIA has started work on the next exam. Its "Exam objectives under development" page lists draft Server+ V6 objectives for an exam numbered SK0-006, in a draft document numbered version 1.1 and dated 2026. In that draft, the number of questions, the length of the test and the domain weights are all marked "TBD." The passing score is shown as 750. The draft reorganizes the content into five domains: Planning and Deployment; Configuration and Administration; Security and Compliance; Data Center Operations; and Performance Management and Troubleshooting. It adds objectives such as using AI tools in server management workflows and a separate objective on patch management. It also prints the troubleshooting method as recommended practice while stating that the method itself will not be tested.

On 27 October 2026 neither the draft nor CompTIA's Server+ page gave a launch date for SK0-006. This course follows SK0-005, the exam a learner starting now can actually book. Most of what you learn here carries straight across: the draft still covers hardware, storage, operating system installation, high availability, virtualization, scripting, hardening, backups and troubleshooting. Before you book, open CompTIA's Server+ page. If a launch date has appeared, check which exam you are buying and compare your study notes with that version's objectives.

Policies to settle before you book

• Identification. CompTIA's ID policy requires two original, unexpired IDs. The primary one must be government-issued and carry your name and a recent, recognizable photo; the secondary needs at least your name plus a signature or photo.
• Candidates aged 17 and under. Since 23 June 2026, a parent or guardian must give consent through CompTIA Central before a minor registers. In return, minors no longer need to be accompanied at the test center.
• Retakes. There is no waiting period between a first and second attempt. Before a third or later attempt you must wait at least 14 calendar days from your last one. Every attempt is paid at full price, and you cannot retake a passed exam under the same code without CompTIA's consent.
• Renewal. Server+ used to be a "Good for Life" certification. CompTIA's help center states that it moved into the Continuing Education (CE) program effective 1 October 2025. People who already held it were given the CE version and must renew by 1 October 2027. CE certifications run for three years from the date they were earned or last renewed. Some older CompTIA pages still leave Server+ off the CE list, so check your own CompTIA account rather than a forum post.
• Price. This course states no exam price. CompTIA's Server+ page showed none when this lesson was written; check the current price, and any student discount, when you buy.
• Study materials. The objectives document warns that candidates who use "brain dumps," meaning leaked real exam questions, can have certifications revoked and be suspended from testing. Every question in this course was written for it.

CompTIA and Server+ are trademarks of CompTIA. CompTIA does not endorse or sponsor this course. The lessons paraphrase CompTIA's objectives in their own words; the official PDF linked in the Sources is always the authority, and no course, this one included, can promise you a pass.

Building the lab, step by step

Here is the lab built once, end to end. Every later lesson assumes it exists, and many add a disk, a network adapter or a service to it.

1. Check the host. You need a 64-bit computer with hardware virtualization (Intel VT-x or AMD-V) switched on in its firmware settings. Plan memory for what will run at once. Microsoft gives 2 GB as the minimum for Windows Server 2025 with the Desktop Experience and recommends 4 GB, while Canonical lists 1.5 GB for Ubuntu Server. With 16 GB in the host you can run both lab servers together; with 8 GB, run one at a time.
2. Install the hypervisor. Download Oracle VirtualBox 7.2 from virtualbox.org. The platform packages are free under the GNU General Public License version 3, there are builds for Windows, Intel and Apple-silicon Macs, Linux and Solaris, and no account is needed. You do not need the separately licensed Extension Pack for anything in this course.
3. Download a Linux server. Rocky Linux is a free rebuild of Red Hat Enterprise Linux (RHEL), which matters because RHEL-family servers use SELinux and firewalld zones, both named in the objectives. Ubuntu Server LTS is a good second choice; Canonical's current long-term release, 26.04.1, comes with five years of free security and maintenance updates. Neither needs an account.
4. Download the Windows Server 2025 evaluation. Microsoft's Evaluation Center offers 64-bit ISO and VHD downloads of the Standard and Datacenter editions after you fill in a registration form. The evaluation runs for 180 days, and it must be activated over the internet within the first 10 days or it starts shutting itself down. If you are under 18, involve a parent or guardian before you give your contact details to any registration form.
5. Create one network for the lab. In VirtualBox, create a NAT Network and attach both VMs to it. The table below, taken from VirtualBox's own access matrix, shows why that mode fits.
6. Create and install the VMs. Use the allocations in the second table, install each operating system, apply all updates, and activate Windows.
7. Take a snapshot. Name it something like "clean install." A VirtualBox snapshot saves the VM's settings and the complete state of its virtual disks, plus its memory if the VM is running. Restoring it throws away every change made since. Take a fresh snapshot before every lab step that could break something.

Two details from Microsoft's hardware requirements are worth knowing before you start. Windows Server 2025 needs a processor with SSE4.2 and POPCNT, so very old hosts cannot run it. And setup in a VM fails at the bare 1,024 MB minimum; Microsoft advises giving the VM at least 1,280 MB for installation. The 4 GB above avoids both problems, and 64 GB leaves room above Microsoft's absolute minimum of 32 GB for updates and roles.

When one input changes the build

Now run the same procedure on two different computers and watch the result change.

An older Windows laptop. Red Hat lists the x86-64-v3 feature level as the baseline for RHEL 10 on Intel and AMD processors, and Rocky Linux 10 rebuilds RHEL 10. On a processor that predates that level, the Rocky 10 installer stops at boot. The fix is not more memory or a different network mode. Install Rocky Linux 9 instead: RHEL 9's baseline is the older x86-64-v2 level, and Rocky still supports release 9.

An Apple-silicon Mac. VirtualBox has an Apple-silicon build, but those Macs run ARM64 guests. Rocky Linux and Ubuntu both publish ARM64 installers, so the Linux half of the lab works. Microsoft's evaluation page offers no ARM64 build of Windows Server, so the Windows labs need an x64 PC. If you have none, follow those labs by reading: each one lists the commands and the results to expect, and the Linux labs cover the same concepts.

Remember: the procedure stays the same, but the right operating system release depends on the host's processor. When a VM refuses to boot, read the hardware requirement before you change anything else.

Studying so it lasts

In 2013 John Dunlosky and four colleagues reviewed the evidence on ten study techniques. Practice testing and distributed practice, meaning study spread out over time, were the only two to earn their highest utility rating. Rereading and highlighting, the two techniques students report using most, earned their lowest. For this course that translates into three habits:

• Answer each lesson's ten questions without looking back, then read every explanation, including those for questions you got right.
• Return to the questions you missed two or three days later, and again a week after that.
• Do the lab. A command you have typed and seen fail is easier to recall than one you have only read.

Common misconceptions

• "Server+ is good for life, so it never needs renewing." That was true until 1 October 2025. Server+ is now in CompTIA's Continuing Education program.
• "You need a rack of real servers to prepare." CompTIA's own equipment list puts racks, UPS units and KVM switches in the ideal-but-optional column. A few things, such as fitting rails or reading a server's front-panel lights, cannot be practiced in a VM; for those, this course uses vendor manuals and says so.
• "The bullet points are the whole syllabus." The objectives document says its examples are not exhaustive. Learn the ideas well enough to recognize a tool or term you have not seen listed.
• "The newest objectives document is the one to study." The SK0-006 objectives are a draft with no launch date. Study the exam you will sit.

What to carry forward

• SK0-005: up to 90 questions in 90 minutes, multiple choice plus simulation PBQs, 750 needed on a 100 to 900 scale.
• Weights: hardware 18 percent, administration 30, security and disaster recovery 24, troubleshooting 28.
• "Given a scenario" objectives ask you to act; "Explain" and "Summarize" objectives ask you to understand and recognize.
• A draft SK0-006 exists with no launch date; this course teaches SK0-005.
• Server+ moved into the CE program on 1 October 2025; the retake rule is no wait before attempt two, 14 days before attempt three.
• Your lab: VirtualBox, a Rocky Linux VM, a Windows Server 2025 evaluation VM on a shared NAT Network, and a snapshot before every risky change.

Sources

1. CompTIA. (2019). CompTIA Server+ certification exam objectives: Exam number SK0-005 (Document version 5.0). comptiacdn.azureedge.net
2. CompTIA. (n.d.). CompTIA Server+ [Certification and exam details page]. Retrieved 27 October 2026. comptia.org
3. CompTIA. (2026). DRAFT CompTIA Server+ SK0-006 V6 certification exam draft objectives (Document version 1.1), listed on Exam objectives under development. comptia.org
4. CompTIA. (n.d.). Performance-based questions explained. comptia.org
5. CompTIA. (n.d.). Candidate ID policy and CompTIA certification retake policy. comptia.org (ID policy); comptia.org (retake policy)
6. CompTIA. (2025). Server+ and Project+ transition to CE certifications [Help center article]. help.comptia.org
7. Oracle. (2025). Oracle VirtualBox user guide, release 7.2: Virtual networking (access matrix) and Snapshots. docs.oracle.com; virtualbox.org (downloads and licenses)
8. Microsoft. (2025). Hardware requirements for Windows Server. Microsoft Learn. learn.microsoft.com; and Windows Server 2025 evaluation, Microsoft Evaluation Center. microsoft.com
9. Red Hat. (2025). Red Hat Enterprise Linux 10.0 release notes: Architectures; and Red Hat Enterprise Linux 9.0 release notes: Architectures. docs.redhat.com (RHEL 10); docs.redhat.com (RHEL 9)
10. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., and Willingham, D. T. (2013). Improving students' learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4-58. pubmed.ncbi.nlm.nih.gov

A 27-kilogram box on a loading dock

A Cisco UCS C220 M8 rack server arrives in a carton. Cisco's installation guide warns that, fully loaded, it can weigh up to 60 pounds, about 27 kilograms. Lifting it takes at least two people or a mechanical lift. Its destination is a 42U APC NetShelter SX cabinet in a small server room down the corridor. Following that one server from the dock to a slot in the rack touches nearly every term in objective 1.1. Power and cables are the exceptions, and they get the next lesson.

Start with the lift itself. The UK Health and Safety Executive's guide Manual handling at work (INDG143, revision 4, 2020) gives filter weights below which a lift is usually low-risk. The highest value in its chart is 25 kg for men and 16 kg for women. Even those apply only to a load held close to the body, between knuckle and elbow height. For a load starting at mid-lower-leg height, such as a carton on the floor, the figures fall to 10 kg and 7 kg, or less with arms extended.

Twisting the torso, or lifting as a team, takes you outside the simple filter altogether and calls for a fuller assessment. A 27 kg server lifted from the floor is far outside every box on the chart, which is why Cisco asks for two people or a lift. HSE is clear that its filters are not safe limits, only a line between low-risk tasks and ones that need more thought.

The point: server lifting rules are not etiquette. A loaded 1U server weighs more than the heaviest low-risk lift in HSE's chart. Plan two people or a lift, clear the route first, and keep the load close and the back untwisted.

Units, rails and the 19-inch standard

Cisco's guide states the rack requirement precisely: a standard 19-inch (48.3 cm) wide, four-post EIA rack whose posts meet the ANSI/EIA-310-D standard. Two-post relay racks that hold network switches by their front ears are not enough for a deep, heavy server, which needs support at front and rear. The height of rack equipment is measured in rack units, written U or RU, and Cisco gives the size of one unit as 1.75 inches (44.45 mm). Servers, switches and shelves are built in whole multiples of it.

The cabinet's enclosure size is quoted separately from its usable units. APC's manual describes NetShelter SX cabinets for EIA/ECA-310, 19-inch (483 mm) equipment, in widths of 600, 700, 750 and 800 mm. The AR3100 model is 42U, 1991 mm tall, 600 mm wide and 1070 mm deep. Wider cabinets leave room at the sides for cable management. Depth matters because a server, its rails and the cable management arm behind it must all fit between the doors.

The NetShelter SX's vertical rails have square holes. APC numbers every third hole to mark the middle of a U space, so you count units rather than holes. Equipment screws into cage nuts clipped into those holes. APC's manual is specific about orientation. Insert cage nuts from inside the cabinet with their ears gripping the sides of the square hole, never the top and bottom; a nut fitted vertically can let equipment fall. Cisco's slide rails accept square 9.6 mm holes, round 7.1 mm holes and two threaded hole sizes. Rail kits are model-specific, so check them before the server arrives.

Choosing and fitting the rail kit

A rail kit is the pair of rails, and the brackets that fit them, that hold a server in the rack. The important distinction is fixed versus sliding.

• Fixed (static) rails support the server but do not extend. To open the lid you must unplug every cable and pull the whole server out.
• Sliding rails, usually ball-bearing, let the server roll forward on its rails while still mounted and cabled, so a technician can open the lid and swap memory or a fan. Cisco's C220 M8 rails install without tools and adjust to rack post spacing of 24 to 36 inches (610 to 914 mm).
• A cable management arm (CMA) is a hinged tray fixed behind a sliding server. It folds out as the server slides forward, so the power and network cables travel with it instead of being pulled out of their sockets. Cisco's CMA for this server adds 5.4 inches (137.4 mm) behind the chassis, bringing the total installed length to 35.2 inches (894 mm), which is exactly the kind of number that decides whether a rear door will close.

Cisco also gives a warning that explains why rails matter beyond convenience. Stacking servers directly on one another without rail kits blocks the vents on top of them. The result can be overheating, faster fans and higher power use.

Planning the elevation

A rack elevation is a drawing or table showing what sits in each U, front and rear. Writing one before anything is mounted forces the decisions that are painful to change later. Three rules from the vendors' manuals drive most of it.

1. Heaviest at the bottom. APC's manual tells you to install the heaviest equipment first and toward the bottom so the cabinet does not become top-heavy. Cisco's guide says the same: load from the bottom up, and if a server is the only unit in a rack, mount it at the bottom. A rack-mount UPS, full of batteries, is usually the heaviest item and belongs low.
2. Stabilize before you extend. APC warns that the cabinet is easily tipped. It must be secured to the floor before equipment is installed, and nothing should be slid out on its rails until about 158 kg (350 lb) of equipment sits in the bottom of the cabinet or a stabilizer plate or bolt-down brackets are fitted. Even then, only one piece of equipment should be extended at a time. Cisco adds that any stabilizers a rack has must be installed before a unit is mounted or serviced.
3. Place things where people use them. A 1U console drawer, which folds a monitor, keyboard and touchpad into one rack unit, gets used standing up, so it goes near the middle of the rack at working height rather than at the top or near the floor. Power distribution units (PDUs) usually mount vertically in the rear channels, which APC calls 0U accessory positions because they use no rack units at all.

Weight has a second limit: the floor. APC rates the NetShelter SX for 1,705 kg (3,750 lb) standing on its leveling feet, but only 1,021 kg (2,250 lb) rolling on its casters. The rolling rating covers only moving the cabinet into position. When you move it, the feet must be up and you push only from the front or rear, never from the side. A raised floor has its own ratings from its maker: Tate, for example, sells its ConCore 1000 panel for a total load of 4.4 kN, about 450 kg. A fully loaded cabinet can weigh far more than one panel is built to carry, so the floor rating belongs in the plan alongside the U count.

The point: an elevation is a weight plan and an airflow plan as much as a space plan. Heaviest equipment goes low, nothing slides out until the rack is stabilized, and the floor must be rated for the total.

Cooling management: air goes in the front

Cisco's guide states that airflow in the C220 M8 runs from front to back, the arrangement almost every rack server uses. A guide from the Department of Energy explains what a room should do with that fact. It is the Best Practices Guide for Energy-Efficient Data Center Design, revised in July 2024 by Otto Van Geet and David Sickinger of the National Renewable Energy Laboratory.

In that design, racks stand in rows with their fronts facing each other across a cold aisle and their backs facing each other across a hot aisle. Cooled air is delivered only to the cold aisles and warm air is drawn back only from the hot aisles. The guide adds that vacant equipment slots must be blocked off on the intake side and cable openings in raised floors and ceilings sealed as tightly as possible. Otherwise hot exhaust recirculates to the front and is drawn in again.

The plates that block vacant slots are blanking panels. An empty 1U gap looks harmless, but it is a hole between the hot aisle and the cold one, and the servers next to it breathe their own exhaust.

The guide also says where temperature counts: at the IT equipment's air intakes, not on a wall behind the equipment. It cites ASHRAE's 2021 Thermal Guidelines for Data Processing Environments, fifth edition, whose recommended range for inlet air is 18 to 27 degrees Celsius (64.4 to 80.6 degrees Fahrenheit). Running hotter is not free. The guide gives an example of a server whose fan speed rose 30 percent when its inlet air went from 77 to 91 degrees Fahrenheit. Fan power rises with the cube of fan speed, so the fans' power draw more than doubled.

Cables are part of cooling too. The same guide warns that cable congestion under a raised floor can sharply reduce airflow and create hot spots. It recommends a cable management strategy for the whole air path, including regular removal of abandoned cables.

Tower, rack mount or blade

Objective 1.1 names three chassis types. They package the same kinds of components very differently.

Cisco's X9508 data sheet shows what a blade-style chassis shares. It has six power supply bays supporting N+1, N+2 and grid redundancy, four hot-swappable fans, and two fabric modules that connect every node upstream. Cisco describes the design as combining the density and efficiency of blade servers with the expandability of rack servers. The sharing is the point and also the risk. Eight compute nodes need only the chassis's cabling, but they all depend on it.

Common misconceptions

• "1U means one inch." One rack unit is 1.75 inches, 44.45 mm. A 2U server is 3.5 inches tall.
• "Empty rack spaces help cooling by letting air move." They let hot exhaust flow back to the intake side. Blanking panels exist to close them.
• "Put the heaviest server at chest height so it is easy to service." Heavy equipment goes at the bottom, where it keeps the cabinet stable; APC and Cisco both say so.
• "A rack on casters can be pushed from any side." APC's manual says to push only from the front or rear, with the feet raised, and to put weight in the bottom first.
• "A blade is just a thin rack server." A blade or compute node depends on its enclosure for power, cooling and network connections, which a rack server provides for itself.

Recap

• One rack unit is 1.75 in (44.45 mm); servers need a four-post, 19-inch EIA-310 rack.
• Check rails against the rack's hole type and post depth; sliding rails plus a cable management arm allow service without unplugging.
• Plan the elevation: heaviest equipment at the bottom, stabilize before extending, one device out at a time, console at working height, PDUs in the 0U rear channels.
• Check the floor: static and rolling ratings differ, and raised-floor panels have their own limits.
• Air enters at the front: hot and cold aisles, blanking panels, sealed cable openings, and inlet temperature in ASHRAE's recommended 18 to 27 degree range.
• Tower for a small office, rack mount for most rooms, blades for dense, centrally managed estates that accept a shared chassis.
• Heavy lifts take two people or a lift.

Sources

1. Cisco Systems. (2025). Cisco UCS C220 M8 server installation and service guide: Installing the server (rack requirements, rails, cable management arm, airflow, lifting). cisco.com
2. APC by Schneider Electric. (2020). NetShelter SX cabinet: Installation and customization manual (990-2183J-001). download.se.com
3. Van Geet, O., and Sickinger, D. (2024). Best practices guide for energy-efficient data center design (Rev. July 2024). National Renewable Energy Laboratory for the U.S. Department of Energy Federal Energy Management Program. energy.gov
4. Health and Safety Executive. (2020). Manual handling at work: A brief guide (INDG143, rev. 4). hse.gov.uk
5. Cisco Systems. (n.d.). Cisco UCS X-Series modular system data sheet (UCS X9508 chassis). cisco.com
6. Dell Technologies. (n.d.). PowerEdge T560 spec sheet. delltechnologies.com
7. Tate. (n.d.). ConCore 1000 raised access floor panel. tateglobal.com
8. ASHRAE Technical Committee 9.9. (2021). Thermal guidelines for data processing environments (5th ed.). ASHRAE. Cited here through the U.S. Department of Energy guide above.

Two power supplies, one point of failure

A Cisco UCS C220 M8 can hold two hot-swappable power supplies, and Cisco describes the second one as the way to get 1+1 redundancy. Picture a new server fitted with two identical 1,600 W supplies. During installation, both cords go into the same power distribution unit (PDU), on the same 120 V circuit. The server boots, both supply lights are green, and the monitoring page reports redundant power. What is wrong with this installation?

There are two problems, and neither shows on the status page. First, the redundancy stops at the PDU. If that PDU fails or its circuit breaker trips, both supplies lose input at the same moment and the server goes down. Second, Cisco's specifications say this 1,600 W supply is limited to 800 W when it runs on low-line input of 100 to 127 V. Its full rating needs 200 to 240 V. Each supply must be able to carry the whole server alone if the other fails, so this server now has an 800 W budget, not a 1,600 W one.

The fix follows from the diagnosis. Plug the two supplies into two different PDUs, fed from two different circuits, at the voltage the supplies are rated for. The rest of this lesson builds that rule outward, from the supply to the utility, and then does the same for the network cables.

Why this matters: a redundant component protects you only as far as the first thing it shares with its twin. Trace every path back until nothing is shared.

The power chain, link by link

The Department of Energy's data center guide lists the usual electrical path. It runs from the utility service through switchboards and switchgear, alternate sources such as a backup generator, paralleled UPSs and PDUs for redundancy, and conditioning equipment. Each link can fail, and each has its own form of redundancy.

The words in that table are exam vocabulary. Redundant power means more than one supply in the server. Separate circuits means those supplies draw from different breakers, so one trip cannot take both. Separate providers means the building itself has more than one source, so a utility outage is carried by another feed or a generator. A UPS fills the seconds between a utility failure and the generator taking the load, or gives servers time to shut down cleanly.

There is a catch in A and B feeds. If A fails, B carries the entire rack. Each side must therefore be sized to carry the full load alone, so in normal running each side should be loaded to well under its capacity.

Choosing a UPS: three designs

Schneider Electric's White Paper 1, The Different Types of UPS Systems (version 10, 2024), classifies UPS designs using the IEC 62040-3 standard. The exam's word is simply UPS, but questions turn on how each design behaves when the power fails.

WP1 also describes a newer hybrid conversion design for dense edge sites. And many online UPSs offer an efficient bypass or "eco" mode; the DOE guide notes that it can lift efficiency toward 99 percent. In that mode, WP1 explains, the UPS behaves more like a line-interactive unit, with a short transfer on failure.

A real example shows the second sizing trap. APC's Smart-UPS SMT1500, a line-interactive tower model discontinued in 2024, is sold as a 1500 VA unit. Its data sheet rates it at 1440 VA and 1000 W. Volt-amperes (VA) are apparent power, the product of voltage and current; watts are the real power the load consumes. The two differ for most electronic loads, and a UPS must stay within both ratings. Its data sheet also gives a transfer time of 4 ms typical and 8 ms maximum, and a battery life of three to five years.

So size a UPS from the watts the equipment actually draws, read from a metered PDU or the server's own power reading, and keep headroom. A load of 1,150 W on that unit overloads it even though 1,150 is less than 1,440. Plan battery replacement as routine maintenance too.

PDUs and the A/B feed

A rack PDU is a power strip built for racks. They range from basic units that only distribute power, through metered ones that display the current drawn, to switched units that can turn individual outlets on and off remotely. APC's NetShelter switched rack PDU APDU9941 is a typical data center model. It mounts in the 0U rear channel, takes 200 or 208 V input at 30 A, and offers 21 C13 and 3 C19 outlets. It meters amps, volts, kilowatts and kilowatt-hours, and is managed over a Gigabit Ethernet network card.

Switching earns its keep when a hung server will not respond to anything else. You can cut and restore its outlet from your desk. Metering earns its keep in planning: if the A side already carries most of what one feed can supply, the rack has no real redundancy left.

Connectors you will meet

A rack power cord is named for its two ends. The usual server cord is a C13-to-C14 jumper. Its C13 end plugs into the supply's C14 inlet, and its C14 end plugs into one of the PDU's C13 outlets. A high-wattage supply with a C20 inlet needs a C19-to-C20 cord and a PDU with C19 outlets. Read the inlet on the power supply first, then choose the cord and outlet that match it. Cisco adds one rule worth remembering: do not mix power supply types or wattages in one server. Both supplies must be identical.

Network cabling: redundant paths and the right medium

Redundant networking applies the power lesson to data. A server with two network ports cabled to the same switch has the same weakness as two supplies on one PDU. Cable them to two different switches, and Lesson 18 shows how teaming makes the pair act as one link. The Network+ course (CS 215) teaches media and transceivers in depth; what follows is the part server installers use daily.

Those reaches come from Intel's cable requirements for its X540 10GBASE-T adapters. Fiber is the other choice. The Fiber Optic Association's reference describes single-mode fiber with a core of about 9 microns, used with lasers at 1310 and 1550 nm. Multimode fiber has a 50 or 62.5 micron core. Single-mode has lower loss and avoids the modal dispersion that limits multimode distance, so it reaches much farther; multimode pairs with cheaper short-reach optics. Connectors differ too. SC is a snap-in, push-pull connector with a 2.5 mm ferrule. LC is a small form factor connector with a 1.25 mm ferrule, half the SC's size, which is why it dominates crowded switch and server panels. MPO carries 12 or 16 fibers in a row for parallel optics.

Servers and switches take pluggable transceivers, so one port can be fitted for whichever medium the link needs. The SFP carries Gigabit Ethernet, the SFP+ carries 10 Gb, and the QSFP+ carries 40 Gb as four 10 Gb lanes.

Read the table for the patterns the exam uses. Inside a rack, a passive DAC is the cheap, simple 10 Gb link. Across a data center, multimode SR optics do the job. Between buildings, single-mode LR optics are the answer. An SR module will not work on single-mode fiber, and an LR module expects single-mode, so the module, the fiber type and the connector must all match.

Cable management that survives the next change

• Keep the airflow clear. The DOE guide warns that cable congestion reduces cooling airflow and creates hot spots. It recommends removing abandoned cables as a routine.
• Let cables move with the server. Route power and network cables through the cable management arm, with enough slack to slide the server out.
• Do not crush or kink fiber. The FOA notes that fiber attenuation is sensitive to the stress of bending too tightly. Use hook-and-loop straps rather than tight plastic ties, and follow the cable maker's bend radius.
• Separate and label. Run power on one side of the rack and data on the other, and label both ends of every cable with what it connects. The label is what makes Lesson 21's documentation possible.
• Color with purpose. Many sites color A and B power cords differently, so a mistake like this lesson's opening one is visible at a glance.

Common misconceptions

• "Two power supplies make the server immune to power failures." Only if they are fed from independent PDUs, circuits and sources. Two supplies on one PDU share a single point of failure.
• "A 1500 VA UPS can carry 1,500 W." APC's SMT1500 is rated 1440 VA but only 1000 W. The watt rating is usually the binding one.
• "Every UPS switches over instantly." Standby and line-interactive units take several milliseconds to transfer; only a double conversion online unit in online mode has no transfer at all.
• "Category 6 carries 10 Gb to 100 m." Intel gives 55 m for 10GBASE-T on Category 6. Category 6A reaches 100 m.
• "Any 10 Gb optic works on any fiber." SR optics are built for multimode and LR optics for single-mode; mismatching them fails or performs badly.

What you now know

• Redundancy works only as far as paths stay separate: two supplies, two cords, two PDUs, two circuits, independent sources.
• Check a supply's voltage rating: Cisco's 1,600 W unit gives only 800 W on 100-127 V.
• Standby UPS for a desktop, line interactive for a few servers, double conversion online for critical loads with 0 ms transfer.
• Size a UPS by watts and VA, keep headroom, and replace batteries on schedule.
• C14 and C20 are supply inlets; C13 and C19 are the matching cord ends and PDU outlets.
• Copper: Category 6A for 10 Gb to 100 m. Fiber: multimode with SR optics for shorter runs, single-mode with LR optics for distance; LC for most ports, MPO for parallel 40 Gb.
• SFP 1 Gb, SFP+ 10 Gb, QSFP+ 40 Gb; DAC for short in-rack links.

Sources

1. Spitaels, J., Zhang, L., and Lin, P. (2024). The different types of UPS systems (White Paper 1, version 10). Schneider Electric Energy Management Research Center. se.com
2. Cisco Systems. (2025). Cisco UCS C220 M8 server installation and service guide: Overview and Technical specifications (power supplies, inlets, input ranges). cisco.com (specifications); cisco.com (overview)
3. APC by Schneider Electric. (n.d.). APC Smart-UPS SMT1500 product data sheet and NetShelter switched rack PDU APDU9941. se.com (SMT1500); se.com (APDU9941)
4. Van Geet, O., and Sickinger, D. (2024). Best practices guide for energy-efficient data center design (Rev. July 2024). National Renewable Energy Laboratory for the U.S. Department of Energy. energy.gov
5. Intel. (n.d.). Cable requirements for Intel Ethernet converged network adapter X540 series. intel.com
6. Cisco Systems. (2026). Cisco 10GBASE SFP+ modules data sheet; and (2022) Cisco 40GBASE QSFP modules data sheet. cisco.com (SFP+); cisco.com (QSFP)
7. Fiber Optic Association. (n.d.). Fiber optic connector identifier and The FOA reference for fiber optics: Optical fiber. thefoa.org (connectors); thefoa.org (fiber)

Two servers, one table

Cisco's UCS C220 M8 fits two Intel Xeon 6 processors, with up to 86 cores each, into a single rack unit. Dell's PowerEdge T560 stands on the floor as a 4.5U tower and takes up to six GPUs. Both are two-socket servers built from the same families of parts. Put their specifications side by side and the components that objective 1.1 names, from the CPU to the hardware compatibility list, sort themselves into questions you can answer.

Read the table row by row and each row becomes a design question. How many cores do the workloads need? How much memory, and in what arrangement? How many cards, of what kind? And is every part supported together? The rest of this lesson works through those questions in that order.

Processors: sockets, cores and the memory each one owns

A socket is the physical mount for one processor package. Each package holds many cores, independent processing units, and most server processors can run two hardware threads per core. A two-socket C220 M8 with 86-core processors therefore offers 172 cores to its operating system. More cores help workloads that run many tasks at once, such as virtual machines and busy web servers. Higher clock speed helps single-threaded work that cannot be split.

Socket and core counts matter beyond performance. Some software is licensed per core or per socket, which Lesson 22 covers, so a server with more cores than its workload needs can cost more to license than to buy.

Each processor also owns its own memory. On the C220 M8, Cisco's population guide shows eight memory channels per CPU, A through H, each with two DIMM sockets. A core can read memory attached to the other processor, but more slowly. The Linux kernel's documentation calls this arrangement NUMA, non-uniform memory access. CPUs reading memory on their own cell get faster access and higher bandwidth than reads from a remote cell. A hypervisor that keeps a virtual machine's processors and memory on the same node avoids that penalty.

What matters here: sockets, cores, memory channels and NUMA nodes are linked. Adding a second CPU adds cores and memory channels together, and software performs best when it uses memory local to the cores it runs on.

Memory: ECC, registered modules and population rules

Server memory differs from desktop memory in two ways. First, it checks itself. Microsoft's hardware requirements for Windows Server call for ECC (error-correcting code) memory or similar technology on physical hosts. ECC stores extra check bits so the memory controller can correct a single-bit error and detect worse ones. Second, it is usually registered. Supermicro's paper on server memory compares the module types by what they buffer.

That paper dates from 2011 and describes DDR3, but the principle holds for DDR5: buffering reduces the electrical load each module puts on the channel, so a channel can carry more memory. It is also why the types are not interchangeable. Dell's T560 accepts registered ECC DDR5 DIMMs only.

Where the modules go matters as much as which modules you buy. Cisco's rules for the C220 M8 are typical of the genre:

• Populate 1, 2, 4, 6, 8, 12 or 16 DIMMs per CPU; Cisco does not recommend 5, 7, 9, 10 or 11.
• Fill the blue sockets (A1 onward) first, then the black ones.
• Configure the DIMMs for CPU 1 and CPU 2 identically.
• Run all DIMMs on a socket at the same speed, or the processor runs at the lowest one.
• Never leave a socket empty: fit a DIMM or a DIMM blank, which keeps the airflow correct.
• Do not mix certain types in a channel, and use only Cisco-supported DIMMs.

The speed row of the comparison table shows the trade-off behind these rules. A C220 M8 runs its memory at 6400 MT/s with one DIMM per channel, but at 5200 MT/s with two. Filling every slot buys capacity at the cost of bandwidth. Eight DIMMs per CPU, one per channel, is often the balanced choice.

Buses: PCI Express lanes and generations

Expansion cards connect to the processor over PCI Express (PCIe), a serial bus built from lanes. A slot is described by its lane count, such as x4, x8 or x16, and by its generation. Microchip's summary of the generations shows each one doubling the per-lane rate.

A 100 Gb network card moves about 12.5 GB/s in each direction. That fits a Gen4 x16 slot comfortably, but would saturate a Gen3 x8 slot, which offers only about 8 GB/s. Matching the card to the slot is arithmetic, and the exam can ask it. CompTIA's acronym list also names PCI and PCI-X, the older parallel buses that PCIe replaced; you will meet them only in very old servers.

Rack servers fit cards on risers, small boards that turn slots sideways so cards lie flat in a short chassis. That is why server cards are described by height and length. The C220 M8's risers take either three half-height cards or two full-height cards, each up to three-quarters length. A card that does not fit the riser's dimensions does not fit the server, whatever its slot type.

Expansion cards, GPUs and the ports on the back

• Network cards. Many servers keep a dedicated slot for networking: Cisco's mLOM socket takes cards with four 10/25/50 Gb ports or two 40/100/200 Gb ports, and also accepts OCP 3.0 cards, an industry standard form factor.
• Storage controllers. RAID controllers manage arrays of internal drives (Lesson 6). Host bus adapters (HBAs) connect a server to external storage, such as a Fibre Channel SAN (Lesson 7).
• Boot devices. Dell's BOSS-N1 card mirrors two M.2 NVMe SSDs in hardware RAID, so the operating system boots from a redundant pair instead of a data drive.
• GPUs. Graphics processors now do much more than graphics. They accelerate AI inference and training, video work and virtual desktops. They need wide slots, power and cooling, which is why the T560 supports six and the 1U C220 M8 only three single-wide cards.

The ports on the chassis are the server's interfaces. The C220 M8's rear panel has a dedicated 1 Gb management port for its baseboard management controller, an RS-232 serial port on an RJ-45 connector, a VGA port and two USB 3.0 ports. On the front, a single KVM connector takes a breakout cable with USB, VGA and serial plugs, so a technician with a crash cart can plug in at the front of the rack. Lesson 8 explains what each of these is for.

The hardware compatibility list

A hardware compatibility list (HCL) records which combinations a vendor has tested and will support. Each major platform publishes one:

• Microsoft's Windows Server Catalog lists systems and components that have earned the Certified for Windows Server badge, which Microsoft says also means they are fully supported in Hyper-V.
• The Red Hat Ecosystem Catalog lists servers, components such as storage controllers and network interfaces, and infrastructure certified to run Red Hat platforms.
• Broadcom's VMware Hardware Compatibility Guide does the same for VMware's hypervisor.
• Server makers publish their own rules as well. Cisco, for example, does not support third-party DIMMs in the C220 M8.

The HCL is checked in two directions. Before buying, confirm that the server and every card you plan to add are listed for the operating system or hypervisor version you will install. Before upgrading the operating system, confirm the existing hardware is still listed for the new version. A part that works but is not listed may still leave you without vendor support when something fails.

Handling parts: static and shutdown rules

Components are damaged by electrostatic discharge (ESD) long before you would feel a spark. Cisco's service guide calls for an ESD wrist strap or other grounding device, such as a grounded mat, and for handling components only by their carrier edges. The same guide separates the parts you may change with the server running from those you may not. SAS and SATA drives, fan modules and one of two redundant power supplies are hot-swappable. DIMMs and PCIe cards require the server to be shut down and unplugged. Lesson 9 returns to hot-swap work in detail.

Common misconceptions

• "Filling every memory slot always makes a server faster." On the C220 M8, two DIMMs per channel drops memory speed from 6400 to 5200 MT/s, and unbalanced counts are not recommended. More slots filled means more capacity, not necessarily more speed.
• "Any DDR5 module will work in a DDR5 server." Servers require specific types: the T560 takes registered ECC DIMMs only, and vendors restrict mixing and third-party modules.
• "Two CPUs give every program twice the memory at the same speed." Each CPU owns its memory channels; remote memory is slower, as the kernel's NUMA documentation explains.
• "If a part works, the HCL does not matter." The HCL defines what the vendor will support. Unlisted parts can leave you without a fix when a failure is blamed on them.

The short version

• Sockets hold processors; cores do the work; each processor owns its memory channels, and remote (NUMA) memory is slower.
• Servers use ECC memory, usually registered DIMMs; follow the vendor's population rules for counts, slot order and matching.
• PCIe lanes and generations set a slot's bandwidth: roughly 1, 2 and 4 GB/s per lane each way for Gen3, Gen4 and Gen5.
• Risers set the size of card a rack server can take; GPUs need wide slots, power and cooling.
• Check the HCL for every server, card and OS or hypervisor version, before buying and before upgrading.
• Wear an ESD strap, handle parts by their edges, and shut down before changing DIMMs or cards.

Sources

1. Cisco Systems. (2025). Cisco UCS C220 M8 server installation and service guide: Overview; Servicing the server (memory population guidelines, ESD, hot-swap components). cisco.com (overview); cisco.com (servicing)
2. Dell Technologies. (n.d.). PowerEdge T560 spec sheet. delltechnologies.com
3. Super Micro Computer. (2011, October). Memory solutions: Using LRDIMMs for mission-critical server and storage solutions. supermicro.com
4. Microchip Technology. (n.d.). PCI Express technology [PCIe generation table]. microchip.com
5. The kernel development community. (n.d.). What is NUMA? Linux kernel documentation. docs.kernel.org
6. Microsoft. (2025). Hardware requirements for Windows Server (memory: ECC). Microsoft Learn. learn.microsoft.com; and Windows Server Catalog. windowsservercatalog.com
7. Red Hat. (n.d.). Red Hat certified hardware. Red Hat Ecosystem Catalog. catalog.redhat.com; and Broadcom. (n.d.). VMware hardware compatibility guide. compatibilityguide.broadcom.com

Three confident answers that are wrong

Solidigm sells two data center SSDs built on the same flash and the same PCIe 4.0 interface. Its D7-P5520 is rated for up to 1 drive write per day over five years; its D7-P5620 is rated for up to 3. Ask a room of new administrators which one belongs under a busy database's transaction log and many will say it does not matter, because "SSDs are all fast." That answer is wrong, and so are two others you will hear in the same conversation: that a "4 TB" drive gives you 4 TB, and that every SSD runs at the same speed. This lesson takes each wrong answer apart and finds where the reasoning slips. Along the way it covers every drive and interface objective 1.2 names.

Wrong answer one: "any SSD will do"

Start with what makes an SSD different from a hard disk. A hard disk drive (HDD) stores data on spinning magnetic platters and moves a head across them. An SSD stores data in NAND flash, with no moving parts. Flash has one weakness a platter does not: each cell survives only a limited number of program and erase cycles. That is why SSD makers publish an endurance rating, usually in drive writes per day (DWPD) over the warranty period, or as total terabytes written (TBW).

Now do the arithmetic the "any SSD" answer skipped. Suppose the transaction log writes 7.5 TB a day and you choose a 3.84 TB D7-P5520, rated at 1 DWPD for five years.

1. Daily writes as a fraction of capacity: 7.5 / 3.84 = about 1.95 drive writes per day.
2. The rating allows 1 per day for 5 years, so the endurance budget is roughly 5 x 1 = 5 drive-years of writes at that rate.
3. At 1.95 per day, the budget is spent in about 5 / 1.95 = 2.6 years, well inside the warranty period.

The 3 DWPD D7-P5620 changes the result. At 1.95 drive writes a day it stays inside its rating for the full five years. Solidigm's brief shows the trade: the higher-endurance drive comes in slightly smaller capacities (1.6, 3.2, 6.4 and 12.8 TB against 1.92, 3.84, 7.68 and 15.36 TB), and is rated for faster random writes (up to 390,000 against 220,000 4K IOPS). Read-intensive drives fit content that is written once and read many times, such as a file archive or a web content store. Mixed-use and write-intensive drives fit logs, databases and caches.

Endurance shapes design as well as purchasing. Microsoft's description of the Storage Spaces Direct cache says that, in an all-flash pool, the cache absorbs and coalesces writes so capacity drives see less traffic and last longer. For that reason it recommends higher-endurance, write-optimized drives for the cache, while capacity drives may reasonably have lower endurance.

Remember: an SSD's speed is only half its specification. Match its drive-writes-per-day rating to the workload's daily writes, or the drive wears out early.

Where hard disks still win, and how fast they turn

Hard disks remain the cheapest way to store large amounts of data, which is why Microsoft supports hybrid pools that put HDDs behind flash. Their weakness is mechanical delay. To read a block, the head must move to the right track, which is seek time, and then wait for the block to rotate under it, which is rotational latency. On average the wait is half a revolution, so it follows directly from spindle speed.

Every figure in the table comes from one formula: 60,000 ms divided by the RPM gives one revolution, and half of that is the average wait. Even the fastest disk waits about 2 ms per random read before any data moves, which is why random input/output operations per second (IOPS) on HDDs are counted in hundreds while SSDs count them in hundreds of thousands. Sequential reads, where the head stays put, are where disks still perform respectably.

Hybrid storage combines the two media. A solid-state hybrid drive puts a small flash cache inside one disk. At larger scale, a hybrid pool pairs separate flash and disk drives. In Microsoft's Storage Spaces Direct, the fastest drive type is chosen automatically as cache. When flash caches for HDDs it caches both reads and writes, giving flash-like latency for hot data, while the HDDs supply the capacity.

Wrong answer two: "every SSD runs at the same speed"

Two SSDs with identical flash can differ several-fold in speed because of the interface between the drive and the server. Seagate's technology paper on enterprise interfaces explains the three in use.

Seagate's paper makes the key point about SATA: its command overhead was harmless for hard disks but becomes significant when an SSD can move a 4 KB block in microseconds. NVMe removes that bottleneck by putting the drive on PCIe itself. A Gen4 x4 NVMe drive such as the D7-P5520 is specified at up to 7,100 MB/s for sequential reads. A single 6 Gb/s SATA link cannot carry even a tenth of that.

Compatibility between SAS and SATA runs one way, and the exam likes to test it. KIOXIA's 2023 brief on 24G SAS states that SAS backplanes support SATA drives, so SATA SSDs and HDDs can connect to SAS backplanes, host bus adapters or RAID controllers, and the two can share drive bays. 24G SAS also remains compatible with SAS-3, SAS-2 and 6 Gb/s SATA devices. The SAS-only features do not come along, though: a SATA drive in a SAS bay still has one port, not two.

Objective 1.2 also lists interfaces for external and embedded storage.

• eSATA is SATA for external enclosures. Today you will mostly meet it on older equipment.
• USB carries external drives, installation media and, in some servers, an internal boot stick.
• SD cards were long used as embedded boot media for hypervisors. Broadcom's guidance for VMware ESXi now marks SD and USB boot as not recommended and says newer servers that use them as the boot device will not certify. It recommends a persistent local device such as an M.2, SATA, SAS or NVMe SSD with adequate endurance instead. That is the job Dell's BOSS card from Lesson 4 does.

Wrong answer three: "a 4 TB drive gives 4 TB"

Drive makers count in decimal units. KIOXIA's brief spells it out: a terabyte is 1,000,000,000,000 bytes, but an operating system counts in powers of two, where its "TB" is 2⁴⁰ bytes, so it shows less capacity. The unambiguous names come from the binary prefixes the IEC approved in December 1998. NIST's summary gives kibi (2¹⁰), mebi (2²⁰), gibi (2³⁰) and tebi (2⁴⁰).

Work an example. A drive sold as 8 TB holds 8 x 10¹² bytes. Divide by 2⁴⁰, which is 1,099,511,627,776, and you get about 7.28 TiB. Windows will report that drive as roughly 7.27 TB, using the decimal symbol for a binary quantity. Nothing is missing; the units changed. The gap grows with size, from 2.4 percent at the kilo level to about 9 percent at tera.

Real capacity planning starts there and adds four more allowances:

1. Protection overhead. RAID or mirroring consumes raw capacity; Lesson 6 calculates how much.
2. File system and free-space headroom. Volumes that run nearly full slow down and leave no room for snapshots or emergencies.
3. Growth. Measure how fast data grows each month and plan for the hardware's whole service life, not just today.
4. Performance, not just space. Count the IOPS and endurance the workload needs. Several small drives can give more IOPS than one large one, and flash sized only for space may wear out early.

Common misconceptions

• "SSDs never wear out." Flash has finite endurance, which is why makers rate drives in DWPD or TBW and offer read-intensive and mixed-use models.
• "A 15,000 RPM disk is as fast as an SSD for random reads." It still waits about 2 ms per read for the platter to turn, before any seek time.
• "SAS and SATA drives are interchangeable." SAS infrastructure accepts SATA drives, but a SATA drive gains no SAS features such as a second port.
• "The OS is hiding part of my new drive." The drive's label is decimal and the OS reports in binary units; 8 TB shows as about 7.28 TiB.
• "SD cards are a fine hypervisor boot device." Broadcom now advises against SD and USB boot devices for ESXi and recommends durable local SSDs.

Putting it together

• Choose flash by endurance as well as speed: compare the workload's daily writes with the drive's DWPD rating.
• HDD rotational latency is half a revolution: about 4.17 ms at 7,200 RPM, 3 ms at 10,000 and 2 ms at 15,000.
• Hybrid pools use flash to cache reads and writes for disks that provide the capacity.
• SATA is cheap and single-ported; SAS is full duplex, dual-ported and accepts SATA drives; NVMe puts flash on PCIe for the highest IOPS.
• eSATA and USB serve external storage; SD and USB boot media are falling out of favor for hypervisors.
• Plan capacity in the units the OS uses, then add protection overhead, headroom, growth and performance.

Sources

1. Solidigm. (2022). Solidigm D7-P5520 and D7-P5620 product brief. solidigm.com
2. Microsoft. (2025). Understanding the storage pool cache in Azure Local and Windows Server clusters. Microsoft Learn. learn.microsoft.com
3. Seagate Technology. (n.d.). Enterprise SSD interface comparisons (Technology paper TP625). seagate.com
4. KIOXIA. (2023, March). Introducing the 24G SAS interface (PM7 series technical brief, rev. 1.1). kioxia.com
5. NVM Express. (n.d.). About NVM Express. nvmexpress.org
6. Broadcom. (n.d.). SD card/USB boot device revised guidance (Knowledge base article 317631). knowledge.broadcom.com
7. National Institute of Standards and Technology. (n.d.). Definitions of the SI units: The binary prefixes. physics.nist.gov

Six drives and a decision

Seagate's data sheet for the Exos X18 describes an 18 TB, 7,200 RPM hard disk that reads at up to 270 MB/s. It also specifies at most one unrecoverable sector for every 10¹⁵ bits read. Suppose a file server arrives with six of them. Lay them out one way and you get 108 TB that dies with any single drive. Lay them out another way and you get 72 TB that survives two failures. RAID, a redundant array of independent disks, is the set of layouts in between. This lesson walks the decision through step by step for those six disks, then changes one input and watches the answer change.

Step 1: know the layouts

The Storage Networking Industry Association (SNIA) defines the standard levels by how they place data. RAID 5 stores stripes of n data blocks and one parity block across n+1 devices, with parity interleaved across all of them. RAID 6 uses parity-based protection that lets stored data survive any two device failures. RAID 10 creates a striped device (RAID 0) over a set of mirrored devices (RAID 1), and survives one failure in each mirrored set. JBOD is simply "just a bunch of disks": drives presented individually or joined end to end, with no protection.

The minimums come from the parity arithmetic. Red Hat's documentation notes that RAID 5 needs at least three disks, and the mdadm manual observes that a RAID 6 needs one more drive than a matching RAID 5. Classic RAID 10 is built from at least two mirrored pairs, so four drives, although Linux's md driver can lay out its own raid10 across three.

The last column needs an explanation, because it decides performance. A mirror writes each block twice. RAID 5 must keep parity correct, so a small write that changes one block takes four disk operations: read the old data, read the old parity, write the new data and write the new parity. RAID 6 maintains two parity blocks, so the same small write takes six. Reads cost the same everywhere; writes are where parity pays its bill.

Step 2: work the six-drive example

The TiB column applies Lesson 5: divide decimal bytes by 2⁴⁰. For a file server holding mostly large files that are read far more often than written, the choice narrows to RAID 5 or RAID 6. RAID 5 offers 18 TB more space. To see why that is not the end of the analysis, look at what happens on the day a drive fails.

Step 3: follow a failure through to the rebuild

When one member fails, the array keeps serving data in a degraded state, reconstructing missing blocks from parity on every read. The Linux kernel's md documentation exposes this directly: its degraded counter shows how many devices an array is missing, and it describes a hot spare as a working device that is not yet a full member. When a member fails, md builds the spare in to replace it; the mdadm manual adds that a spare added to a degraded array starts rebuilding immediately. A hot spare therefore shortens the most dangerous period, the time between failure and full redundancy, because nobody has to drive to the site first.

The rebuild itself is long. Rebuilding an 18 TB member means writing 18 TB, and at the drive's best sequential rate of 270 MB/s that takes at least 18 TB / 270 MB/s, about 66,700 seconds or 18.5 hours. A rebuild that competes with normal traffic takes longer.

Now the decisive arithmetic. To rebuild the failed member of the six-drive RAID 5, the controller must read every block on the five survivors: 90 TB, which is 7.2 x 10¹⁴ bits. At Seagate's specified limit of one unrecoverable sector per 10¹⁵ bits, that is about 0.72 expected read errors during a single rebuild. The specification is a maximum, and real drives usually do better, but a design should not depend on luck. In a degraded RAID 5, an unreadable sector on a surviving drive has no remaining redundancy to repair it. In RAID 6, the second parity block can still reconstruct it.

In short: with large hard disks, the risk that matters is not two drives failing on the same day but a read error during an 18-hour rebuild. That is why big HDD arrays use RAID 6, and why hot spares and regular scrubs matter.

Scrubs are the preventive half. The md documentation describes a check action that reads all blocks and verifies redundancy, and a repair action that also fixes mismatches. Hardware controllers offer the same idea under names such as patrol read or consistency check. Running one regularly finds bad sectors while the array still has the redundancy to fix them.

Change one input: six SSDs for a database

Replace the hard disks with six 3.84 TB SAS SSDs, and replace the file server with a database that makes many small random writes. Two numbers move. KIOXIA specifies its PM7 SAS SSDs with an unrecoverable bit-error rate of 1 in 10¹⁷, a hundred times better than the hard disk's 10¹⁵, and the arrays are far smaller, so rebuild risk shrinks. Meanwhile the write cost column becomes the deciding factor, because every small write in RAID 6 costs six operations against two in RAID 10. Parity writes also add flash wear, which Lesson 5 showed is finite. For this workload, RAID 10 wins: 11.52 TB usable, two operations per write, and the fastest rebuild, because a failed member is rebuilt by copying its mirror rather than computing parity from every survivor.

Two other workloads complete the picture. A pair of drives that only needs to boot an operating system takes RAID 1. Scratch space for a render farm, rebuilt from source if lost, can take RAID 0 for speed and full capacity.

Hardware RAID, software RAID and pass-through

Red Hat's storage documentation distinguishes three kinds of RAID. Hardware RAID is managed by a controller independently of the host, so the operating system sees each array as an ordinary disk. Software RAID implements the levels in the kernel's block device code; in RHEL that is md, managed with mdadm, which Red Hat prefers because it is hardware independent. Firmware RAID is software RAID configured from a firmware menu so the system can boot from the array. Windows Server's software equivalent is Storage Spaces, which pools drives and offers simple (striped), mirror (two-way or three-way) and parity spaces.

A hardware controller's selling point is its cache. With write-back caching, the controller confirms a write as soon as it reaches cache memory, which makes small writes fast. The kernel's documentation states the danger plainly: a volatile write-back cache signals completion before data has reached non-volatile storage. If power fails, those writes vanish. Controllers therefore protect the cache. Cisco's guide for the C220 M8 describes an optional Supercap module that provides about three years of backup for the write-back cache's memory in a sudden power loss, and controller cards with flash-backed write cache (FBWC). Cisco also sets its controller's default write policy to write-through, which confirms writes only once they reach the drives, and lets administrators choose write-back.

The third option is no RAID at all in the controller. An HBA, or a RAID controller in pass-through or JBOD mode, hands raw disks to the operating system so software such as Storage Spaces, md or ZFS can manage redundancy itself. Two rules from Cisco's guide are worth remembering: do not mix controller types in one server, and direct-attached NVMe drives cannot be controlled by its RAID controller at all, which leaves NVMe redundancy to software.

RAID is not a backup

Every layout above copies changes to all members immediately. Delete a file, let ransomware encrypt a share, or corrupt a database, and RAID faithfully stores the damage with full redundancy. RAID protects availability against drive failure. Backups, covered in Lesson 29, protect data against everything else.

Common misconceptions

• "RAID 10 survives any two drive failures." It survives one failure per mirrored pair. Lose both drives of one pair and the array is gone.
• "RAID 5 is always the best value." It gives the most space of the protected levels, but its four-operation small writes and its exposure during long rebuilds make it a poor fit for write-heavy work and large hard disks.
• "Write-back cache is always safe on a server." Only when the cache is protected by a battery, Supercap or flash; an unprotected volatile cache loses acknowledged writes in a power cut.
• "A mirrored array means we do not need backups." RAID copies deletions and corruption as faithfully as good data.
• "JBOD is a RAID level with protection." JBOD offers none; a failed drive loses its data.

Where this leaves us

• Capacity: RAID 0 N x C; RAID 1 C; RAID 5 (N - 1) x C; RAID 6 (N - 2) x C; RAID 10 N / 2 x C.
• Failures survived: RAID 5 one, RAID 6 any two, RAID 10 one per mirrored pair, RAID 0 and JBOD none.
• Small-write cost: 2 operations for mirrors, 4 for RAID 5, 6 for RAID 6.
• Degraded arrays rebuild onto a hot spare or replacement; long HDD rebuilds plus unrecoverable read errors favor RAID 6.
• Scrub regularly with check or patrol read so bad sectors are found while redundancy remains.
• Hardware RAID offloads parity and caches writes; protect write-back cache or use write-through. HBAs and pass-through mode leave redundancy to software such as md, Storage Spaces or ZFS.
• RAID is availability, not backup.

Sources

1. Storage Networking Industry Association. (n.d.). RAID 5, RAID 6, RAID 10 and JBOD, in SNIA online dictionary. snia.org (RAID 5); snia.org (RAID 6); snia.org (RAID 10); snia.org (JBOD)
2. Red Hat. (n.d.). Managing RAID. In Red Hat Enterprise Linux 9: Managing storage devices. docs.redhat.com
3. The kernel development community. (n.d.). RAID arrays (md) and Explicit volatile write back cache control. Linux kernel documentation. docs.kernel.org (md); docs.kernel.org (write-back cache)
4. Brown, N., and contributors. (n.d.). mdadm(8): Manage MD devices aka Linux software RAID. Linux manual page. man7.org
5. Microsoft. (2025). Storage Spaces overview. Microsoft Learn. learn.microsoft.com
6. Cisco Systems. (2025). Cisco UCS C220 M8 server installation and service guide: Storage controller considerations. cisco.com
7. Seagate Technology. (2020). Exos X18 data sheet (DS2045-1). seagate.com
8. KIOXIA. (2023, March). Introducing the 24G SAS interface (PM7 series technical brief, rev. 1.1). kioxia.com

A name for a disk that lives somewhere else

RFC 7143, the consolidated iSCSI standard published in April 2014, gives an example of a storage name: iqn.2001-04.com.example:storage:diskarrays-sn-a8675309. It reads like a domain name turned inside out, and that is roughly what it is. Behind it sits a disk that a server will treat as its own, even though the disk lives in another box across the network. This lesson follows that idea from the beginning: why servers stopped keeping all their storage inside the chassis, and the two very different ways they now share it.

Start with the server you met in Lessons 4 to 6. Its drives sit in its own bays, behind its own RAID controller. That is direct-attached storage (DAS), and it has a limit you notice the day you build a second server. Each server's storage belongs to that server alone. When three virtualization hosts need to run the same virtual machines, or a cluster needs to fail a database over from one node to another, the data must live somewhere every node can reach. That somewhere is shared storage, and it comes in two forms.

File or block: the question that sorts everything

The Storage Networking Industry Association draws the line in its dictionary. Network attached storage (NAS) is a storage device that connects to a network and provides file access services to clients, using file access protocols such as NFS and SMB. A storage area network (SAN) is a network whose primary purpose is moving data between computer systems and storage devices; SNIA notes the term is usually, though not necessarily, associated with block services.

The middle row is the one to hold on to. With NAS, the storage device owns the file system and clients ask for files by name. With a SAN, the storage device hands out raw blocks and each server builds its own file system on them. Two servers that format and write the same SAN volume without a cluster-aware file system will corrupt it, because each believes it owns the disk.

The upshot: NAS shares files and keeps the file system on the storage side; SAN shares blocks and leaves the file system to each server.

NAS protocols: NFS and SMB

NFS, the Network File System, is the native file-sharing protocol of Unix and Linux. RFC 7530 specifies version 4. It notes that NFS servers have historically used port 2049, and that a client using that registered port does not need the older RPC binding service, the portmapper, to find the server. That simplifies firewall rules: one port, 2049, instead of a range.

SMB, Server Message Block, is the file-sharing protocol of Windows. Microsoft describes it as a protocol for reading, writing and updating files on a remote server over TCP/IP, and it negotiates a dialect when a connection opens. SMB 3.1.1 is supported from Windows 10 version 1607 and Windows Server 2016 onward. Modern SMB is much more than a file-copy protocol: it offers encryption and signing, RDMA transport (SMB Direct) for data center links, transparent failover and scale-out file servers. Hyper-V can even keep virtual machine disks on SMB 3 shares.

The exam objectives also say CIFS, the Common Internet File System. Microsoft's MS-CIFS specification describes it as a cross-platform protocol that lets clients use file and print services on servers. It is the old-generation name you will still meet in exam questions, product menus and Linux's mount -t cifs. The thing to know is the history around it. SMB version 1 was superseded by SMB 2 and later from 2007 and publicly deprecated by Microsoft in 2014. Since Windows 10 and Windows Server version 1709 it is no longer installed by default, and Windows Server 2019 and later ship without it. Microsoft strongly recommends against reinstalling it because of known security issues with ransomware and malware. An old NAS that speaks only SMB 1 needs a firmware update, not a weaker client.

iSCSI: SCSI commands carried inside TCP

Back to the name that opened the lesson. iSCSI carries the same SCSI block commands a server sends to a local disk, wrapped in TCP/IP so they can cross an ordinary Ethernet network. RFC 7143 names the two ends. The initiator is the client, a server that wants a disk. The target is the storage system that provides it. Each logical unit inside a target has an address called a logical unit number (LUN), which is why storage administrators talk about presenting a LUN to a host.

• Names. Every initiator and target has an iSCSI name. The common form is the iSCSI qualified name (IQN): iqn., a year and month, a reversed domain name, then a colon and a free-form label, as in the example above. Linux keeps its own initiator name in /etc/iscsi/initiatorname.iscsi.
• Port. IANA registers TCP port 3260 for iSCSI targets, and RFC 7143's own diagrams show targets listening there.
• Access control. Red Hat's targetcli tool uses access control lists (ACLs) to grant each initiator, by IQN, access to specific LUNs. An initiator not on the list cannot log in to that LUN.
• Authentication. RFC 7143 requires every compliant initiator and target to implement CHAP authentication, and allows mutual authentication, in which the target also proves itself to the initiator.

In practice the initiator side takes three commands on Linux. iscsiadm -m discovery -t st -p followed by the target's address lists the targets on offer. iscsiadm -m node -T followed by a target's IQN and -l logs in. The LUN then appears as a new disk, such as /dev/sdb, ready for a file system. Two habits keep it reliable. Enable the iscsid and iscsi services so sessions return after a reboot, and mount the file system with the _netdev option in /etc/fstab, so the system waits for the network before mounting it. Windows Server has the same pieces built in: the iSCSI Initiator on every edition and an iSCSI Target Server role for providing LUNs.

Because iSCSI shares the Ethernet network, good practice keeps it on its own VLAN or physical network, with jumbo frames where the whole path supports them. Lesson 13 covers VLANs from the server's side.

Fibre Channel and FCoE

Before Ethernet was fast enough for storage, SANs were built on Fibre Channel (FC). SNIA describes it as a serial I/O interconnect supporting several protocols, including FCP for SCSI and NVMe, over switched, point-to-point and arbitrated loop topologies, at speeds from 1 to 128 Gb/s. The INCITS T11 committee standardizes it. A server joins an FC SAN through a Fibre Channel host bus adapter, and every port and node carries a World Wide Name: a WWPN for each port and a WWNN for each node, both in CompTIA's acronym list. FC networks are dedicated to storage, which buys predictability at the cost of a separate set of switches, cables and skills.

FC SANs control who sees what in two layers.

1. Zoning, in the fabric switches. SNIA defines it as subdividing a SAN into disjoint zones; nodes outside a zone are invisible to nodes inside it. A test server zoned away from the production array cannot even discover it.
2. LUN masking, in the storage array. Even inside a zone, the array presents each LUN only to the WWPNs listed for it, the same job targetcli's ACLs do for iSCSI initiator names.

FCoE, Fibre Channel over Ethernet, sits between the two worlds. SNIA defines it as encapsulating Fibre Channel frames inside Ethernet frames, so FC traffic can travel over an Ethernet network. The appeal is one set of cables and adapters, often converged network adapters, carrying both LAN and SAN traffic, while the storage stays Fibre Channel at heart.

Choosing a shared storage design

Common misconceptions

• "NAS and SAN are the same thing with different marketing." NAS serves files and runs the file system; a SAN serves blocks and each server runs its own file system.
• "Two servers can safely mount the same iSCSI LUN like a share." Not without a cluster-aware file system. Two hosts writing an ordinary file system on one LUN will corrupt it.
• "Zoning and LUN masking are the same control." Zoning in the switches decides which ports can see each other; LUN masking in the array decides which hosts are offered each LUN.
• "If an old device needs SMB 1, just enable SMB 1." Microsoft strongly advises against it for security reasons; update or replace the device.
• "iSCSI needs special network hardware." It runs over ordinary TCP/IP Ethernet, ideally on a separate VLAN.

Pulling it together

• DAS belongs to one server; NAS shares files over NFS (port 2049) or SMB; a SAN shares blocks over iSCSI, Fibre Channel or FCoE.
• SMB 3.x is the modern Windows protocol; CIFS is the older name, and SMB 1 is deprecated and absent by default.
• iSCSI: initiators log in to targets, which present LUNs; names use the IQN format; targets listen on TCP 3260; ACLs and CHAP control access.
• Fibre Channel runs on dedicated fabrics with HBAs and WWNs; zoning isolates ports, LUN masking restricts LUNs.
• FCoE carries Fibre Channel frames inside Ethernet frames.

Sources

1. Storage Networking Industry Association. (n.d.). Network attached storage, Storage area network, Fibre Channel, Fibre Channel over Ethernet and Zoning, in SNIA online dictionary. snia.org (NAS); snia.org (SAN); snia.org (Fibre Channel); snia.org (FCoE); snia.org (zoning)
2. Chadalapaka, M., Satran, J., Meth, K., and Black, D. (2014). Internet Small Computer System Interface (iSCSI) protocol (consolidated) (RFC 7143). RFC Editor. rfc-editor.org
3. Haynes, T., and Noveck, D. (Eds.). (2015). Network File System (NFS) version 4 protocol (RFC 7530). RFC Editor. rfc-editor.org
4. Internet Assigned Numbers Authority. (n.d.). Service name and transport protocol port number registry (port 3260, iscsi-target). iana.org
5. Microsoft. (2025). What is SMB file sharing for Windows and Windows Server? and SMBv1 not installed by default in Windows Server and Windows. Microsoft Learn. learn.microsoft.com (SMB overview); learn.microsoft.com (SMBv1)
6. Microsoft. (2025). [MS-CIFS]: Common Internet File System (CIFS) protocol (Protocol revision 31.0). Microsoft Open Specifications. learn.microsoft.com
7. Red Hat. (n.d.). Configuring an iSCSI target and Configuring an iSCSI initiator. In Red Hat Enterprise Linux 9: Managing storage devices. docs.redhat.com (target); docs.redhat.com (initiator)

A frozen server, 400 kilometers away

Suppose a monitoring alert arrives at three in the morning: a database server in a colocation facility 400 kilometers away has stopped answering. SSH times out. Remote Desktop fails. Nobody from your team is on site. What can you still do?

Everything you normally use to manage that server runs through its operating system, and the operating system is the thing that has stopped. To make progress you need four abilities that do not depend on it. You need to see the console screen, to learn whether the system panicked, hung or is waiting at a prompt. You need to press the power and reset buttons. You need to boot from a rescue image if the disk will not start. And you need to read the hardware's own logs to rule out a failed component. Objective 1.3 calls the set of tools that provide these out-of-band management, and the rest of this lesson builds them up from that need.

In-band and out-of-band

In-band management travels through the server's operating system: SSH, Remote Desktop, PowerShell remoting and web-based admin tools, which Lesson 17 covers. Out-of-band management uses a separate path that keeps working when the OS does not. In servers that path is the baseboard management controller (BMC). DMTF's Redfish specification describes a BMC as typically an independent microprocessor or system-on-chip, with its own firmware, that carries out out-of-band monitoring and management. The NSA and CISA's 2023 guidance on BMCs adds the detail that matters at three in the morning: BMC firmware runs from the moment power is applied, so it functions even when the server is shut down, and it does not need the OS or hypervisor to boot.

Every major server vendor builds one in under its own name. CompTIA's acronym list names three: Dell's iDRAC (Integrated Dell Remote Access Controller), HPE's iLO (Integrated Lights-Out) and Cisco's CIMC (Cisco Integrated Management Controller). Many have a dedicated network port: the C220 M8 from Lesson 4 has a 1 Gb management port on its rear panel, separate from the server's data ports.

Worth holding on to: the BMC is a separate small computer inside the server with its own network port, firmware and power. That independence is what makes out-of-band management possible, and also what makes the BMC dangerous if it is left unprotected.

The protocols behind the console: IPMI and Redfish

For two decades, scripts and management tools talked to BMCs through IPMI, the Intelligent Platform Management Interface. A command-line tool such as ipmitool can read sensors, read the event log and switch power over the network. IPMI is now frozen. Intel's IPMI page carries a joint message from the specification's promoters, Dell, HPE, NEC and Intel, issued in June 2020: no further updates to the IPMI specification are planned or should be expected. The promoters encourage a move to a more modern interface and name DMTF's Redfish as one example.

Redfish is DMTF's standard, specified in DSP0266 (version 1.21.0 is dated August 2024). It uses RESTful interface semantics: a client sends ordinary HTTP requests, normally over HTTPS, and the service answers with resources represented as JSON. Everything hangs from a single service root, /redfish/v1/, and clients discover the rest by following links. A GET to /redfish/v1/Systems lists the computer systems a BMC manages; POSTing a reset action to a system restarts it. Because Redfish is plain HTTPS and JSON, the same script works across vendors and fits naturally into automation tools.

Securing the controller that controls everything

A BMC can do anything a person standing at the server can do, including reflash firmware and reinstall the OS. CISA's 2013 alert on IPMI, TA13-207A, explained why that worried defenders. IPMI passwords were saved in clear text. Many interfaces used default or no passwords, or weak encryption. An option called "cipher 0", enabled by default on many devices, allowed authentication to be bypassed. Root access to IPMI meant complete control over hardware, software and firmware. CISA recommended restricting IPMI traffic, usually UDP port 623, to trusted internal networks, preferably a management VLAN, with strong unique passwords, encryption where possible, cipher 0 disabled and anonymous logins off.

The NSA and CISA returned to the subject in June 2023 with Harden Baseboard Management Controllers. Its actions are a checklist worth memorizing:

1. Protect credentials. Change default BMC credentials as soon as possible, give administrators unique accounts, and never expose default credentials to the internet or an untrusted network segment.
2. Separate the network. Put BMC ports on an isolated administrative VLAN, limit which endpoints can reach them, and limit or block their internet access.
3. Harden the configuration using vendor guidance.
4. Check for updates routinely. Monthly or quarterly, because BMC updates arrive separately; in the sheet's own words, OS patch tools do not deliver BMC updates.
5. Monitor integrity and scan BMC firmware periodically.

The sheet also explains why this matters: security tools running in the OS, such as anti-malware and intrusion detection, cannot see or stop a compromised BMC.

IP KVM: remote consoles without a BMC

Some equipment has no BMC: older servers, appliances, workstations used as servers. An IP KVM switch fills the gap. It connects to the keyboard, video and mouse ports of several machines, and makes each console available to administrators over the network. The capability is the same as a BMC's virtual console, provided by an external box instead of a chip on the motherboard. Like a BMC, it belongs on the management network, behind strong credentials.

Local hardware administration

Sometimes you are standing in front of the rack. Objective 1.3 lists the tools for that case too.

• KVM switch. One keyboard, monitor and mouse shared by many servers, often as the 1U console drawer from Lesson 2, switching between machines at a keypress.
• Crash cart. A wheeled cart with a monitor, keyboard, mouse, cables and often a laptop with a serial adapter, rolled up to whichever server needs attention. Cisco's C220 M8 has a front-panel KVM connector whose breakout cable provides USB, VGA and serial plugs, so the cart can connect at the front of the rack.
• Serial connectivity and console connections. Many servers and almost all network devices offer a serial console; the C220 M8's rear panel has an RS-232 port on an RJ-45 connector. A console cable from a laptop gives a text console that works with no network and no graphics. Linux can send its console to a serial port with a kernel option such as console=ttyS0,115200n8; the kernel documentation notes that several console options can be given at once and that 115200 is the highest baud rate. Windows Server offers Emergency Management Services (EMS) for the same purpose, enabled with bcdedit /ems {current} on, with the port and speed set by bcdedit /emssettings.
• Virtual administration console. For virtual machines, the hypervisor provides the console: the VirtualBox window you have used since Lesson 1, or the console tab of an enterprise hypervisor's management interface. It is the virtual equivalent of a crash cart.

Back to the frozen server

With a BMC on a management VLAN, the three-in-the-morning call becomes a short procedure.

1. Connect to the management network over the VPN, and sign in to the BMC with your own account.
2. Open the virtual console. Read the screen: a kernel panic, a stop error or a hung prompt each points somewhere different.
3. Check the hardware event log for memory, power or temperature faults before blaming software.
4. Capture what you see, then use power control: a graceful shutdown if the OS responds, a reset or power cycle if it does not.
5. If it will not boot, attach a rescue ISO through virtual media and boot from it to repair the disk or recover data.
6. Record what you found and did; Lesson 31's method requires it.

Common misconceptions

• "If the OS is down, only a site visit will help." A BMC runs independently of the OS, from the moment power is applied, and provides console, power and media remotely.
• "Out-of-band means the BMC needs no security; it is not on the main network." It needs more, because it controls everything: unique credentials, an isolated management VLAN and regular firmware updates.
• "Windows Update or dnf will keep the BMC patched." NSA and CISA state plainly that OS patch tools do not deliver BMC updates.
• "IPMI is the current standard." Its promoters announced in 2020 that no further updates are planned; Redfish is the actively developed successor.
• "A serial console is obsolete." It works with no network and no graphics, which is exactly when you need it.

Summing up

• In-band management uses the OS; out-of-band management uses a separate path, the BMC, which works when the OS is down or the server is off.
• BMC features map to the objectives: remote console, remote power on/off, remote drive access (virtual media), monitoring and firmware updates.
• IPMI (usually UDP 623) is frozen; Redfish offers HTTPS and JSON under /redfish/v1/.
• Secure BMCs: change defaults, unique accounts, an administrative VLAN, no internet exposure, routine firmware checks.
• IP KVM gives remote consoles to machines without BMCs; KVM switches, crash carts, serial consoles and hypervisor consoles cover local work.

Sources

1. National Security Agency and Cybersecurity and Infrastructure Security Agency. (2023, June). Harden baseboard management controllers (Cybersecurity Information Sheet U/OO/164464-23, ver. 1.0). media.defense.gov
2. Cybersecurity and Infrastructure Security Agency. (2013, revised 2016). Risks of using the Intelligent Platform Management Interface (IPMI) (Alert TA13-207A). cisa.gov
3. DMTF. (2024). Redfish specification (DSP0266, version 1.21.0). dmtf.org
4. Intel. (n.d.). Intelligent Platform Management Interface (IPMI) [includes the June 2020 joint message from the IPMI promoters]. intel.com
5. DMTF. (n.d.). Redfish Mockup Server [Open-source tool, BSD 3-Clause license]. GitHub. github.com
6. The kernel development community. (n.d.). Linux serial console. Linux kernel documentation. docs.kernel.org
7. Microsoft. (2025). Boot parameters to enable EMS redirection. Microsoft Learn. learn.microsoft.com
8. CompTIA. (2019). CompTIA Server+ certification exam objectives: Exam number SK0-005 (Document version 5.0), objective 1.3 and acronym list. comptiacdn.azureedge.net

Firmware is software you cannot see

In March 2026 Microsoft added a warning to its Secure Boot guidance for hardware makers. The Secure Boot certificates it issued in 2011 begin expiring in June 2026, and devices need updated certificates to keep verifying trusted boot software. It is a reminder that a server carries a whole layer of software below the operating system: the system firmware, the BMC firmware from Lesson 8, and firmware inside every RAID controller, network card and drive. That layer needs maintenance like any other. This lesson walks through two maintenance jobs end to end, a firmware upgrade and a hot-swap drive replacement. Each time it then changes one input and shows how the right procedure changes with it.

BIOS and UEFI: what runs before the OS

When a server powers on, its system firmware tests the hardware, finds a boot device and hands control to an operating system loader. For decades that firmware was the BIOS, the basic input/output system. Modern servers use UEFI, the Unified Extensible Firmware Interface, though "BIOS" survives as the everyday name for the setup screens. The differences matter when you install an OS and when you secure one.

Microsoft's GPT FAQ adds two details worth remembering. GPT keeps primary and backup partition tables, with CRC32 checksums, so a damaged table can be recovered. And Windows does not support booting UEFI from an MBR disk. An OS installed in one mode boots in that mode afterwards, so choose before you install. Lesson 12 returns to partition styles.

Secure Boot, in Microsoft's words, helps make sure a device boots using only software trusted by its manufacturer. At startup the firmware checks the signature of each piece of boot software, including UEFI drivers on add-in cards, EFI applications and the operating system loader, and boots only if the signatures are valid. The firmware stores the trusted signers in a signature database and the revoked ones in a separate revocation list, and Microsoft requires protection against rolling firmware back to older versions. Server firmware setup also holds the settings you will touch most often:

• Boot order, and whether network (PXE) boot is enabled; Lesson 27 locks this down.
• Virtualization extensions such as Intel VT-x and VT-d, needed by hypervisors (Lesson 1 checked them on your own PC).
• TPM, the trusted platform module. Cisco's C220 M8 supports TPM 2.0, fitted with a one-way screw, and Cisco does not support field replacement of it.
• Setup and boot passwords, covered in Lesson 24.
• Power and performance profiles, which trade energy use against latency.

Two hardware details complete the picture. Settings and the real-time clock are kept alive by a coin cell; Cisco specifies a CR2032 and warns that an incorrectly replaced battery can explode. And most boards have a jumper or switch to clear these settings. Cisco warns that clearing CMOS removes any customized settings and might result in data loss, and provides a separate switch to clear a forgotten BIOS password.

Procedure 1: a firmware upgrade on a production server

Firmware updates fix bugs, close security holes and add support for new drives and cards. They also carry a risk no software update has: an interrupted flash can leave a component unable to start. NIST's Platform Firmware Resiliency Guidelines (SP 800-193, May 2018) organize the defense around three goals: protect the firmware from unauthorized change, detect changes that happen, and recover rapidly and securely. Its central tenet for updates is that only authentic, authorized images may be applied, using an authenticated update mechanism based on digital signatures. Here is the procedure for a BIOS and storage-controller update on a rack server.

1. Read the release notes and record current versions from the BMC's inventory page. Note fixes, known issues and any required order of updates.
2. Check compatibility. Cisco, for example, says storage controller firmware must be verified for compatibility with the current Cisco IMC and BIOS versions, and has standalone servers update controller firmware with its Host Upgrade Utility, which upgrades components as a tested set. Check the OS drivers and the HCL for the new versions too.
3. Get the package from the vendor and confirm its signature or published checksum. Never flash an image from a forum or file-sharing site.
4. Raise a change request and schedule a maintenance window; Lesson 21 covers change management. On a cluster, move workloads off the node first.
5. Protect the power. Both supplies connected, on UPS-backed feeds. Do not power-cycle during the flash.
6. Export or note the firmware settings so they can be restored if an update resets them.
7. Apply the update, in the vendor's order, then reboot as instructed.
8. Verify: new versions shown, POST clean, arrays healthy, OS and applications working, no new events in the BMC log.
9. Document versions, date and results.

Change one input: the update is for the BMC itself rather than the BIOS. Now the plan can be lighter. NSA and CISA note that some servers can restart the BMC independently of the OS or hypervisor, so a BMC update may need no server reboot at all. Their guidance also reminds you that OS patching never delivers these updates, so they need a calendar of their own. The opposite change, a firmware downgrade to escape a bug, can be blocked outright. NIST describes implementations that check version numbers during an update, and Microsoft requires Secure Boot platforms to protect against firmware rollback. Read the vendor's downgrade instructions before you need them.

So what?: firmware updates are routine maintenance with an unforgiving failure mode. Signed images, a compatibility check, steady power and a rollback plan turn them from a gamble into a procedure.

Procedure 2: replacing a failed drive while the server runs

Cisco's service guide for the C220 M8 sorts components into three groups, and the groups decide the procedure.

Now the drive. The RAID controller has reported that the drive in slot 3 of a RAID 6 array has failed.

1. Confirm the failure and the slot in the controller utility or the BMC. Confirm that only one drive has failed and that the array is degraded but online.
2. Light the locator LED for slot 3 from the BMC or controller software, so you pull the right drive. Pulling a healthy drive from a degraded RAID 5 destroys the array.
3. Prepare the replacement: same interface and type, and the same or larger capacity, in the right carrier. Wear an ESD strap.
4. Release the latch, slide the failed drive out and insert the replacement until the latch closes.
5. Watch the rebuild start, automatically or after you assign the new drive, and follow it to completion. Lesson 6 explained why the array is exposed until then.
6. Handle the old drive by policy. It still holds data; Lesson 28 covers retention and destruction.

Change one input: the failed drive is an NVMe SSD. It is hot-plug, not hot-swap, so step 4 gains a prerequisite. Stop I/O and take the drive offline in the operating system before you pull it, because Cisco does not support surprise removal of NVMe drives. Change another: the part is a fan. Fans are hot-swappable, but Cisco warns not to run the server for more than one minute with a fan module removed, so the replacement must be in your hand before the old one comes out. And another: a power supply has failed in a server that has only one. There is no redundant supply to carry the load, so the replacement needs a shutdown and a maintenance window.

The objective also lists cages and cards. A drive cage is the frame and backplane that hold the bays; whether it can be serviced live depends on the design, so check the service guide. Cards are not hot-swappable on the C220 M8; like DIMMs, they wait for a shutdown.

Common misconceptions

• "Hot-swap and hot-plug mean the same thing." Cisco distinguishes them: hot-swap parts need no OS preparation, while hot-plug NVMe drives must be taken offline first.
• "A redundant power supply means any supply can be pulled at any time." Only when another supply is carrying the load; with a single supply, replacement needs a shutdown.
• "Newer firmware can always be rolled back." Platforms that enforce rollback protection may refuse older images; plan before you flash.
• "UEFI and BIOS mode are interchangeable after installation." The OS boots in the mode it was installed in, and Windows UEFI boot needs a GPT disk.
• "Clearing CMOS only resets the clock." It removes all customized firmware settings.

Looking back

• UEFI replaces BIOS: GPT boot disks, an EFI System Partition, Secure Boot signature checks and rollback protection.
• Firmware settings include boot order, virtualization extensions, TPM, passwords and power profiles; a CR2032 coin cell keeps them, and clearing CMOS erases them.
• Firmware upgrades: release notes, compatibility, signed vendor images, change control, protected power, saved settings, vendor order, verification and documentation.
• Hot-swap: SAS/SATA drives, fans, redundant supplies. Hot-plug: NVMe after taking it offline. Shutdown: DIMMs, cards, a sole power supply.
• Locate before you pull, replace like with like, and watch the rebuild finish.

Sources

1. Microsoft. (2026). Secure boot [Guidance for OEMs, including the 2011 certificate expiry notice]. Microsoft Learn. learn.microsoft.com
2. Microsoft. (2021). Boot to UEFI mode or legacy BIOS mode; and (2017) Windows and GPT FAQ. Microsoft Learn. learn.microsoft.com (UEFI or BIOS); learn.microsoft.com (GPT FAQ)
3. Regenscheid, A. (2018). Platform firmware resiliency guidelines (NIST Special Publication 800-193). National Institute of Standards and Technology. csrc.nist.gov
4. Cisco Systems. (2025). Cisco UCS C220 M8 server installation and service guide: Servicing the server (hot-swap and hot-plug components, fans, RTC battery, CMOS and password switches, TPM) and Storage controller considerations (firmware compatibility, Host Upgrade Utility). cisco.com (servicing); cisco.com (storage controllers)
5. National Security Agency and Cybersecurity and Infrastructure Security Agency. (2023, June). Harden baseboard management controllers (Cybersecurity Information Sheet U/OO/164464-23). media.defense.gov

Two instructions that decide the install

Windows Server 2025 will not install on a processor that lacks two instructions its predecessors never listed, SSE4.2 and POPCNT. Microsoft's hardware requirements page adds them to the long-standing basics: a 1.4 GHz 64-bit processor with NX and DEP support and second-level address translation. A server that ran Windows Server 2016 happily can fail at the first screen of 2025. This lesson follows one installation from that first check to the moment the new server joins the network, in the order the decisions actually have to be made.

The server in this story is a new rack server for a small company. It will run the file and print services Lesson 14 describes. Nothing about it is exotic, which is the point: every server installation passes through the same six steps.

Step one: requirements and compatibility

Every operating system publishes minimum requirements. They are floors, not targets. Microsoft says so directly: it cannot give useful recommended figures because deployments vary so widely, and it advises test deployments to find what a role really needs.

Two footnotes from Microsoft catch people in labs. A virtual machine given the bare minimum of 1,024 MB of memory fails during setup; Microsoft advises at least 1,280 MB for the installation, which can be reduced afterwards. And servers with more than 16 GB of memory need extra disk space for paging, hibernation and dump files.

Requirements describe the processor, memory and disk. The hardware compatibility list from Lesson 4 describes everything else: whether this exact server model, RAID controller and network card are certified for this OS version. Check both before you choose the version, not after the installer fails.

The core of it: minimum requirements decide whether an OS can install; the HCL decides whether the vendor will support the result. Both are checked before the media is even made.

Step two: GUI or core

Windows Server offers two installation options. Server with Desktop Experience is the familiar graphical server. Server Core, in Microsoft's description, is a minimal installation with a smaller disk footprint and therefore a smaller attack surface, because it has a smaller code base. It has no desktop by design. Server Manager, Event Viewer, Disk Management and the Services console are not available locally. Instead you manage it with the command line and PowerShell, with the SConfig menu, or remotely with Remote Server Administration Tools or Windows Admin Center.

Linux servers make the same choice differently. Server installations usually leave out the graphical desktop and are managed over SSH. The trade-off is the same on both platforms: less installed software means fewer updates and fewer things to attack, at the price of needing command-line and remote-management skills. For a file server managed from administrators' workstations, Server Core is a sound default.

Step three: bare metal or virtual

A bare metal installation puts the OS directly on the physical server. A virtualized installation puts it inside a virtual machine on a hypervisor, as you did in Lesson 1. Most new server workloads today run virtualized, and Lesson 19 explains why. The decision changes what you check. On bare metal, the HCL, firmware and drivers for real hardware matter. In a VM, the hypervisor presents standard virtual hardware, so the questions become the VM's size and the hypervisor's support for the guest OS. The VM must still meet the guest's requirements; Microsoft's 1,280 MB setup note exists precisely because VMs are often built too small.

Step four: media and method

Red Hat's installation guide shows a choice inside the choice. Its DVD ISO can serve both as the boot device and as the installation source for all the software packages. Its much smaller Boot ISO contains only what is needed to boot and start the installer, so it needs a separate installation source for packages, such as a network repository. For a server room with no internet access, the DVD ISO on a USB stick is the self-contained option.

Step five: drivers

Sometimes the installer starts and cannot see the disks. The usual cause is a storage controller newer than the drivers on the installation media. Both platforms provide a way in.

• Red Hat says a driver update should only be performed if an unsupported driver prevents the installation from completing. It offers three types. An automatic update loads drivers from a device labeled OEMDRV that is connected when installation starts. An assisted update has the installer prompt you to locate a driver update. A manual update uses the inst.dd=location boot option to point at a driver image or package.
• Windows Setup runs inside Windows PE, which needs boot-critical drivers to reach the local disk and network. Microsoft documents adding them through an answer file that lists driver paths during the windowsPE configuration pass; Lesson 11 builds such a file.

Use drivers from the server vendor's support site for this model and OS version, matched to the firmware you installed in Lesson 9.

Step six: patches, utilities and the first connection

The installer has finished, and the server is at its most vulnerable: an OS image that may be months old, with every vulnerability discovered since. NIST's Guide to General Server Security (SP 800-123) states that known vulnerabilities should be corrected before the OS hosts a server or is exposed to untrusted users. It gives two ways to protect a new server while it is patched. Keep it disconnected, or on an isolated build network, until patches are installed and configuration is done. Or put it on a VLAN or segment that allows only the traffic needed for patching and configuration. Only then move it to its production network.

The same guide adds two habits. Test patches on an identically configured server before applying them to production, because patches can cause unexpected problems. And let servers download patches automatically if you like, but do not let them install automatically, so testing can happen first.

Then add the applications and utilities the server needs, and only those. Vendor management agents link the OS to the BMC; Dell's T560, for example, lists an iDRAC Service Module. Add monitoring and backup agents and the server role itself. NIST's broader principle, covered in Lesson 27, is to install the minimum set of services and remove what is not required.

The server's first day

Put together, the small company's file server went like this. The model was checked against the Windows Server Catalog and its processor against the 2025 requirements. Server Core was chosen, and Windows was installed through the BMC's virtual media because the server sat in a locked room across town. The RAID controller's driver was added through Setup, and the server was patched on an isolated build VLAN. The vendor agent and the file server role followed, and only then was the server moved to the production VLAN. Each step depended on the one before it, which is why the order matters as much as the steps.

Common misconceptions

• "Minimum requirements are what the server needs." They are floors for installation. Microsoft advises test deployments to size real roles.
• "If an old server ran the previous Windows Server, it will run the next one." Windows Server 2025 adds SSE4.2 and POPCNT to the processor requirements.
• "Server Core cannot be managed with graphical tools." It has no local desktop, but RSAT and Windows Admin Center manage it remotely.
• "Patch the server once it is on the production network." NIST advises patching on an isolated build network or restricted VLAN first.
• "The Boot ISO and the DVD ISO are interchangeable." The Boot ISO needs a separate package source; the DVD ISO carries the packages itself.

What to remember

• Check minimum requirements and the HCL before choosing an OS version; Windows Server 2025 needs SSE4.2 and POPCNT, RHEL 10 needs x86-64-v3.
• Server Core: smaller footprint and attack surface, no local desktop, managed by PowerShell, SConfig, RSAT or Windows Admin Center.
• Bare metal depends on the HCL and drivers; virtual depends on VM sizing and hypervisor support.
• Media: optical, USB, network (PXE or HTTP boot), remote (virtual media), embedded; Red Hat's DVD ISO is self-contained, its Boot ISO is not.
• Drivers: Red Hat's automatic, assisted and manual (inst.dd) updates; Windows boot-critical drivers through answer files.
• Patch on an isolated network before production, test patches first, add only needed utilities.

Sources

1. Microsoft. (2025). Hardware requirements for Windows Server. Microsoft Learn. learn.microsoft.com
2. Microsoft. (2025). What is Server Core? Microsoft Learn. learn.microsoft.com
3. Red Hat. (n.d.). Interactively installing RHEL from installation media (Red Hat Enterprise Linux 10): disk and memory requirements, boot media options, updating drivers during installation. docs.redhat.com
4. Canonical. (n.d.). Get Ubuntu Server [Download page with system requirements]. ubuntu.com
5. Microsoft. (2017). Add device drivers to Windows during Windows Setup. Microsoft Learn. learn.microsoft.com
6. Scarfone, K., Jansen, W., and Tracy, M. (2008). Guide to general server security (NIST Special Publication 800-123). National Institute of Standards and Technology. csrc.nist.gov
7. Broadcom. (n.d.). SD card/USB boot device revised guidance (Knowledge base article 317631). knowledge.broadcom.com

The plan that sounds efficient

Microsoft's FAQ on GPT disks contains a blunt instruction: do not make a sector-by-sector copy of a GPT disk, because the disk and partition GUIDs will no longer be unique, and "this must never happen." Keep that sentence in mind, because the most natural plan for deploying twenty servers walks straight into it. Build one virtual machine perfectly, patch it, copy its disk twenty times and power on the copies. Lesson 10 installed one server by hand. This lesson starts from that copy-it-twenty-times answer and traces exactly where it fails, then builds the methods that work.

Where the copies collide

An installed operating system is full of values that are supposed to identify one machine. A raw copy duplicates every one of them.

The failure is not that the copies crash. They boot fine. The trouble arrives later and looks unrelated: odd domain behavior, two servers answering for one MAC address, monitoring that merges several machines into one. That is why the copy-it plan feels safe and is not.

Bottom line: a deployment image must be generalized. Every value that identifies one machine is removed or reset, so each copy creates its own on first boot.

Windows done properly: generalize, capture, answer

Microsoft's instructions are explicit. Before you deploy a Windows image you must generalize it, which removes computer-specific information such as installed drivers and the computer SID. That applies even when the target hardware is similar to the original. The tool is Sysprep:

1. Boot the reference machine into audit mode, then customize it: drivers, settings, applications and updates.
2. Run sysprep.exe /generalize /oobe /shutdown. If the result will be deployed as a VHD on the same kind of hypervisor, Microsoft adds the /mode:vm option.
3. Capture the shut-down disk as an image with DISM, Microsoft's imaging tool, and deploy that image to new machines, which start at the first-run (OOBE) screens.

Sysprep can be run up to 1,001 times on one image for Windows Server 2012 and later, after which the image must be rebuilt.

New machines still need answers: computer name, product key, domain, time zone. An answer file supplies them, which is what makes an installation unattended. Microsoft documents several routes. Save the file as Autounattend.xml at the root of a USB flash drive and Windows Setup finds it automatically. Or start Setup with /unattend: followed by a file name. Answer files are built with Windows System Image Manager. Microsoft adds a security warning: Setup caches the answer file in the %WINDIR%\Panther folder, and an answer file containing domain passwords or product keys is a risk, so delete the cached copy before handing the machine over.

Slipstreaming means building updates into the installation image itself, so every new installation starts patched. With DISM it is three commands. Dism /Mount-Image opens the image file in a folder. Dism /Image:, the folder and /Add-Package /PackagePath: with an update file add the packages. Dism /Unmount-Image /Commit saves the result. A slipstreamed image shortens the patch step on the isolated build network from Lesson 10.

Linux done properly: Kickstart and a clean machine ID

Red Hat's equivalent of an answer file is a Kickstart file. Red Hat describes Kickstart as installing RHEL from the parameters in a configuration file, useful for deploying the same configuration on many systems. Three details make it practical:

• Every manual installation writes its choices to /root/anaconda-ks.cfg, a ready-made starting point. Red Hat warns that the file contains information about users and passwords, so handle copies carefully.
• The ksvalidator tool, from the pykickstart package, checks a Kickstart file's syntax and deprecated options. Red Hat is clear that it cannot guarantee the installation will succeed, and it does not check the %pre, %post and %packages sections.
• The installer finds the file through the inst.ks= boot option, pointing to a file shared over HTTP, HTTPS, NFS or FTP, often from the same server that provides PXE or UEFI HTTP boot.

For Linux images that will be copied, the machine-id manual gives the rule: in a generic image the file should be missing or empty, so a fresh ID is generated at boot. An empty file is the recommended form.

Clones, templates and physical images

Hypervisors make copying cheap, so they also make it easy to copy carelessly. VirtualBox's manual describes two kinds of clone. A full clone copies all disk images to the new VM's folder and can operate fully without the source. A linked clone creates differencing disks based on the source VM's disks, which saves space but keeps the clone dependent on the source. When cloning, VirtualBox by default generates new MAC addresses for all network adapters, which its manual calls the best option when the source and the clone must work on the same network.

A template is a generalized VM kept only to be copied. Enterprise hypervisors let you mark a VM as a template so it cannot be started by accident; in VirtualBox you simply keep a generalized VM powered off and clone it. Template deployment is then clone, customize through an answer file or Kickstart, and boot. A physical clone applies the same idea to hardware. An imaging tool captures a generalized system's disk to a file and applies it to other servers of the same model, avoiding the sector copy the GPT FAQ forbids.

P2V: moving a physical server into a VM

Sometimes the problem runs the other way: an old physical server must become a virtual machine. That is P2V, physical to virtual; CompTIA's acronym list also names V2V and V2P for the other directions. Microsoft's Sysinternals tool Disk2vhd shows how it works and where it bites.

• It creates VHD or VHDX files of physical disks for use in Hyper-V, and it can run while the system is online, using Windows' volume snapshot capability for consistent point-in-time copies.
• It creates one virtual disk per physical disk, preserves the partitioning and copies only the volumes you select.
• On first boot in the VM, Windows detects the new virtual hardware and installs drivers that are present; otherwise you add the hypervisor's integration components.
• Do not attach the VHD on the same system that created it if you intend to boot from it. Windows gives it a new disk signature to avoid a collision, and because the boot configuration refers to the disk by signature, the VM then cannot find its boot disk.
• BitLocker-encrypted volumes are not supported until they are decrypted.

Licensing travels with the server too. Disk2vhd's own page notes that OEM copies of older Windows versions may not be transferred to a virtual hard drive under Microsoft's licensing terms. Check the license before the migration, as Lesson 22 explains.

Common misconceptions

• "Identical hardware means the image needs no generalizing." Microsoft says you still have to generalize to remove PC-specific information, even for similar hardware.
• "A cloned VM is a new machine." Only if its identities (SID or machine ID, MAC addresses, name and address) are made unique.
• "ksvalidator proves a Kickstart file works." It checks syntax and deprecated options only.
• "Answer files are harmless text." They can hold passwords and keys, and Setup caches them on the installed system.
• "P2V is just a disk copy." Drivers, disk signatures, encryption and licensing all change when a server becomes virtual.

The takeaway

• Never deploy raw copies: generalize Windows with Sysprep, empty /etc/machine-id in Linux images, and let the hypervisor generate new MAC addresses.
• Answer files (Autounattend.xml, or /unattend) and Kickstart files (inst.ks=) make installations unattended; protect the secrets they contain.
• Slipstream updates into images with DISM: mount, add packages, unmount with commit.
• Full clones stand alone; linked clones depend on their source; templates are generalized VMs kept for cloning.
• P2V tools such as Disk2vhd convert running physical servers to virtual disks; watch disk signatures, BitLocker and licensing.

Sources

1. Microsoft. (2021). Sysprep (generalize) a Windows installation; and (2017) Windows and GPT FAQ. Microsoft Learn. learn.microsoft.com (Sysprep); learn.microsoft.com (GPT FAQ)
2. Microsoft. (2021). Windows Setup automation overview; and (2017) Add or remove packages offline using DISM. Microsoft Learn. learn.microsoft.com (Setup automation); learn.microsoft.com (DISM)
3. Red Hat. (n.d.). Automatically installing RHEL (Red Hat Enterprise Linux 10): Kickstart, anaconda-ks.cfg, ksvalidator and inst.ks. docs.redhat.com
4. systemd project. (n.d.). machine-id(5). Linux manual page. man7.org
5. Oracle. (n.d.). Oracle VirtualBox user manual, chapter 1: Cloning virtual machines. virtualbox.org
6. Russinovich, M. (2021). Disk2vhd (Sysinternals). Microsoft Learn. learn.microsoft.com

Three layers between a disk and a file

Microsoft's documentation on Windows disks now carries a short note: for all uses except mirrored boot volumes, dynamic disks are deprecated, and data that needs resilience against drive failure belongs on Storage Spaces. It is a small sign of how much the layers under a server's files have changed. Between a raw disk and a saved file sit three of them. A partition table divides the disk. A volume manager, optionally, combines or reshapes the pieces. A file system organizes files inside a volume. Objective 2.1 names choices at all three layers, and this lesson compares them layer by layer, with a table at each.

Layer one: MBR or GPT

The table follows Microsoft's description of the two styles. In practice GPT is the default for anything new: every disk larger than 2 TB, every UEFI boot disk and every disk that might grow. MBR remains for legacy BIOS systems and old tools. Microsoft's MBR2GPT tool converts a system disk from MBR to GPT when a server moves from BIOS to UEFI.

The point: GPT removes MBR's two hard limits, four partitions and 2 TB, and adds redundancy. Choose MBR only when legacy BIOS boot demands it.

Layer two: managing volumes

A plain partition is fixed in place: one contiguous stretch of one disk. Volume managers lift that restriction.

Red Hat's LVM documentation gives the three terms you need. A physical volume (PV) is a partition or whole disk designated for LVM. A volume group (VG) is a collection of PVs. A logical volume (LV) is the usable storage device carved from the group. The everyday commands follow the layers: pvcreate, then vgcreate, then lvcreate. When the volume needs more room, add a disk to the group with vgextend and grow the volume and its file system in one step with lvextend --resizefs. Because the file system now sits on a logical volume rather than a partition, growing it never requires the free space to be next door on the same disk.

Layer three: file systems compared

Every row comes from its maintainer's documentation: Microsoft for NTFS and ReFS, the kernel documentation for ext4, Red Hat for XFS, OpenZFS for ZFS and Broadcom for VMFS. XFS is not on CompTIA's list, but you will meet it on every RHEL-family server, so it is included.

Reading the table: four decisions

1. The boot volume. Windows boots from NTFS; Microsoft lists ReFS as not bootable. Linux boots from XFS or ext4.
2. Virtual machine storage. ESXi hosts sharing a SAN LUN format it VMFS, the cluster file system that lets several hosts use it at once. Hyper-V on Storage Spaces Direct is where Microsoft recommends ReFS, for its block cloning and integrity features. Microsoft adds one exception: for Cluster Shared Volumes on SAN storage, format with NTFS to use Direct I/O.
3. Change over time. If a volume might ever need to shrink, avoid XFS and ReFS, which cannot. If it will grow, put it on LVM or Storage Spaces so growth does not depend on adjacent free space.
4. Silent corruption. Only ReFS (with integrity streams) and ZFS checksum data and can repair it from a redundant copy. On a large archive where a flipped bit could go unnoticed for years, that matters.

Cluster size, the allocation unit, is the last decision. Microsoft's NTFS table shows it sets the maximum volume size: 4 KB clusters allow 16 TB, 64 KB clusters 256 TB and 2 MB clusters 8 PB. Microsoft also recommends 64 KB clusters with large file record segments for volumes holding very large VHDX files or using Data Deduplication.

Common misconceptions

• "Dynamic disks are the Windows way to build software RAID." Microsoft has deprecated them for everything except mirrored boot volumes; Storage Spaces replaces them.
• "ReFS is simply a newer NTFS." It drops features NTFS has, including booting, disk quotas and shrinking, in exchange for integrity and scale.
• "Any Linux file system can be shrunk later." XFS can only grow. Plan sizes, or use ext4 where shrinking might be needed.
• "VMFS is a Windows-style file system for ESXi." It is a cluster file system designed so many hosts can share one datastore safely.
• "A 2 TB limit means old hardware." It is MBR's limit; the same disk initialized as GPT can use all of its capacity.

What to carry forward

• Partition tables: MBR, four partitions and 2 TB; GPT, 128 partitions in Windows, beyond 2 TB, redundant tables, required for UEFI boot.
• Volume managers: basic disks; dynamic disks (deprecated); Storage Spaces; LVM (PV, VG, LV); ZFS pools with RAID-Z.
• NTFS: Windows default, ACLs, quotas, bootable. ReFS: integrity and scale, not bootable. ext4 and XFS: Linux journaling file systems, XFS cannot shrink. ZFS: copy-on-write with checksums. VMFS: cluster file system for ESXi datastores.
• Cluster size sets NTFS's maximum volume size.

Sources

1. Microsoft. (2025). Basic and dynamic disks. Microsoft Learn. learn.microsoft.com
2. Microsoft. (2025). NTFS overview and Resilient File System (ReFS) overview. Microsoft Learn. learn.microsoft.com (NTFS); learn.microsoft.com (ReFS)
3. Microsoft. (2017). Windows and GPT FAQ. Microsoft Learn. learn.microsoft.com
4. Red Hat. (n.d.). Configuring and managing logical volumes (Red Hat Enterprise Linux 10). docs.redhat.com
5. The kernel development community. (n.d.). ext4 general information. Linux kernel documentation. docs.kernel.org
6. OpenZFS. (n.d.). Copy-on-write and RAIDZ. OpenZFS documentation. openzfs.github.io (copy-on-write); openzfs.github.io (RAIDZ)
7. Broadcom. (n.d.). vSphere VMFS datastore concepts and operations (vSphere 8.0 storage). techdocs.broadcom.com

A server that answers ping but not users

Suppose a new file server, fs02, has just been built from the Server Core install in Lesson 10. From the administrator's desk it answers ping by IP address. Users on another floor, in another subnet, cannot open \\fs02\projects, and one of them reports that the server's address looked like 169.254.something yesterday. Three separate faults are hiding behind those symptoms, each in a different part of objective 2.2. This lesson finds them one at a time, then fixes the server properly.

Network+ (CS 215) teaches subnetting, routing and switching in depth, and this lesson assumes that background. Here the view is from the server's chair: what it needs configured so that clients everywhere can find and use it.

Fault one: an address that came from nowhere

An address beginning 169.254 is a diagnosis in itself. RFC 3927 reserves 169.254/16 for IPv4 link-local addresses. A host with no other configuration picks one at random from 169.254.1.0 to 169.254.254.255, a behavior Windows calls APIPA, automatic private IP addressing. The RFC says such addresses are only for cases where stable, routable addresses are not available, such as when DHCP cannot be reached. So yesterday fs02 asked for a DHCP lease, got no answer and assigned itself an address that no router will forward. Clients in other subnets could never reach it.

The deeper fault is that a file server was relying on DHCP at all. Clients find servers by name and address, firewall rules and DNS records point at them, and an address that changes breaks all of those. Servers therefore get static addresses, or DHCP reservations that always hand the same address to the server's MAC address. Lesson 25 of Network+ covers DHCP itself.

The addresses themselves usually come from the private ranges RFC 1918 set aside in 1996: 10.0.0.0 to 10.255.255.255 (10/8), 172.16.0.0 to 172.31.255.255 (172.16/12) and 192.168.0.0 to 192.168.255.255 (192.168/16). They are reachable inside the organization and are translated at the edge when they need the internet.

Microsoft's documentation for New-NetIPAddress adds two details worth knowing. Adding a static address to an interface that was using DHCP disables DHCP on it automatically. And the default gateway must be in the same subnet as the address, which brings us to the second fault.

Fault two: a gateway in the wrong subnet

When fs02 was finally given a static address, someone typed its gateway as 10.20.6.1 while the server sat in 10.20.5.0/24. Ping from the administrator's desk worked, because that desk was in the same subnet and needed no router. Every client in another subnet failed, because fs02 had no usable route back to them. The default gateway is the router a host sends everything not on its own subnet to, and it must be reachable directly on the local subnet. Pinging from the same subnet proves nothing about routing; test from a client in another subnet too.

IPv6 adds its own layer. RFC 4291 requires every interface to have at least one link-local address, in the FE80::/10 prefix, which is why ipconfig or ip addr shows an fe80 address even on a server you never configured for IPv6. A server in a dual-stack network needs its routable IPv6 address, gateway and DNS settings checked as carefully as its IPv4 ones.

MAC addresses and VLANs

Every network adapter, physical or virtual, has a MAC address, the 48-bit layer 2 address switches use to deliver frames. Servers care about MACs in three places: DHCP reservations are keyed to them, switch port security can restrict which MACs a port accepts, and cloned VMs must not share them, as Lesson 11 showed.

VLANs divide one switched network into separate broadcast domains, and servers often live on their own. There are two ways to put a server on one. Plug it into a switch access port assigned to that VLAN, and the server needs no VLAN configuration at all. Or connect it to a port that carries several VLANs with 802.1Q tags, and configure a VLAN interface on the server. On Linux, Red Hat's example is nmcli connection add type vlan con-name vlan10 dev enp1s0 vlan.id 10, with the prerequisite that the switch is configured to support VLAN tags. A mismatch, such as a tagged server on an untagged port, produces a link light and no connectivity, which makes VLAN mistakes slow to spot.

Fault three: a name nobody can resolve

The last fault was the reason users could not type \\fs02. Clients find servers by name, and fs02 had never been registered in DNS. A fully qualified domain name (FQDN) names a host completely, such as fs02.corp.example.com: the host label followed by its full domain. DNS stores the mappings as resource records.

Before DNS, every host carried the whole list in a file. The hosts file (/etc/hosts on Linux, a file called hosts under the Windows system folder) still exists. The hosts(5) manual describes it as a static table for hostname lookup that DNS has superseded, but which is still used during boot when DNS is not running and on very small isolated sites. On Linux, the hosts line of /etc/nsswitch.conf sets the order of sources, which are queried in turn until a result is found, so where "files" comes first, a forgotten /etc/hosts entry quietly overrides DNS. Use the hosts file for a temporary test or an isolated lab, and remove the entry afterwards.

Firewall ports: open what the role needs

A host firewall should block everything except the services the server actually provides. IANA's port registry is the authority for the numbers.

On Linux with firewalld, the iSCSI lesson's pattern applies to any port: firewall-cmd --permanent --add-port= with the port and protocol, then firewall-cmd --reload; named services can be added with --add-service. On Windows, Windows Defender Firewall rules are managed with New-NetFirewallRule or the firewall console. fs02 needed only TCP 445 for clients, plus management ports from the administrators' subnet.

Back to fs02

1. Give it a static address in its subnet, or a DHCP reservation keyed to its MAC, so 169.254 never appears again.
2. Set the gateway to the router in its own subnet and test from a client in a different subnet.
3. Confirm the switch port's VLAN matches the server's configuration.
4. Register A and PTR records for fs02.corp.example.com, add a CNAME if users need a friendly alias, and remove any test entries from hosts files.
5. Allow TCP 445 from client subnets and management ports only from administrators.

Why this matters: most "server is down" calls on a new server are configuration, not failure. Address, gateway, VLAN, name and port each have to be right, and checking them in that order finds the fault fastest.

Common misconceptions

• "A 169.254 address means the network is fine but slow." RFC 3927 link-local addresses mean the host got no other configuration, typically because DHCP was unreachable.
• "If ping works, routing works." Ping from the same subnet never touches the gateway; test from another subnet.
• "Servers should use DHCP like everything else." Servers need stable addresses: static, or a DHCP reservation.
• "The hosts file is obsolete." It is still read, and depending on the lookup order it can override DNS without anyone noticing.
• "Opening all ports to trusted subnets is harmless." Each open port is a service to attack; open only what the role needs, from where it is needed.

Recap

• Servers use static addresses or reservations; 169.254.x.x (APIPA, RFC 3927) means DHCP failed.
• RFC 1918 private ranges: 10/8, 172.16/12, 192.168/16. IPv6 interfaces always have an FE80::/10 link-local address.
• The default gateway must be in the server's own subnet; test from other subnets.
• VLANs: access port and no server config, or a tagged port plus a VLAN interface on the server.
• DNS: A, AAAA, PTR, CNAME, MX and SRV records; FQDN = host plus full domain; the hosts file can override DNS.
• Open only the ports the role needs, using IANA's registry as the reference.

Sources

1. Rekhter, Y., Moskowitz, B., Karrenberg, D., de Groot, G. J., and Lear, E. (1996). Address allocation for private internets (RFC 1918). RFC Editor. rfc-editor.org
2. Cheshire, S., Aboba, B., and Guttman, E. (2005). Dynamic configuration of IPv4 link-local addresses (RFC 3927). RFC Editor. rfc-editor.org
3. Hinden, R., and Deering, S. (2006). IP version 6 addressing architecture (RFC 4291). RFC Editor. rfc-editor.org
4. Mockapetris, P. (1987). Domain names: Implementation and specification (RFC 1035); Thomson, S., et al. (2003). DNS extensions to support IP version 6 (RFC 3596); and Gulbrandsen, A., Vixie, P., and Esibov, L. (2000). A DNS RR for specifying the location of services (DNS SRV) (RFC 2782). RFC Editor. rfc-editor.org (RFC 1035); rfc-editor.org (RFC 3596); rfc-editor.org (RFC 2782)
5. Microsoft. (n.d.). New-NetIPAddress (NetTCPIP module). Microsoft Learn. learn.microsoft.com
6. Red Hat. (n.d.). Configuring and managing networking (Red Hat Enterprise Linux 10): static IP settings and VLAN tagging with nmcli. docs.redhat.com
7. Linux man-pages project. (n.d.). hosts(5) and nsswitch.conf(5). man7.org (hosts); man7.org (nsswitch.conf)
8. Internet Assigned Numbers Authority. (n.d.). Service name and transport protocol port number registry. iana.org

One day to turn off printing

On 13 July 2021 the Cybersecurity and Infrastructure Security Agency issued Emergency Directive 21-04. It gave every US federal civilian agency until 11:59 p.m. the next day to stop and disable the Windows Print Spooler service on all Active Directory domain controllers. The vulnerability, CVE-2021-34527, let a remote authenticated attacker run code with SYSTEM privileges through the spooler. Domain controllers were the urgent target because they hold the keys to the whole domain, and almost none of them needed to print anything. The spooler was simply running because it ran by default.

That directive is the shortest possible argument for this lesson's first idea: each server should do the jobs it was built for and nothing else. Follow a growing company as its single do-everything server is split into roles, and the rest of objective 2.3's first half falls into place: what each role needs, how servers join the directory, and how you learn what normal looks like.

Six roles and what each one leans on

A small company often starts with one server that is domain controller, file server, print server and database host at once. It works until it does not. One role's load slows the others, one role's vulnerability exposes them all, and every patch reboots everything. Splitting roles across servers, physical or virtual, lets each be sized, secured and maintained for its own job.

"Requirements" for a role means more than hardware. Each role brings ports to open, service accounts to create, backups to schedule and an availability target to meet. A database might need a cluster (Lesson 18) while a print server can tolerate an hour's outage. Write those requirements down before building the server, because they decide its size, its network placement and its recovery plan.

What matters here: separate roles so each can be sized, secured and patched on its own, and run nothing a role does not need. The 2021 Print Spooler directive showed the cost of a default service left running on the most sensitive servers in the network.

Directory connectivity: joining the domain

Most organizations manage identity centrally, and the most common directory is Active Directory Domain Services (AD DS). Microsoft describes a directory as a hierarchical structure that stores information about objects on a network: user and computer accounts, servers, printers and shared resources. AD DS integrates security through sign-in authentication and access control, so users get one username and password across the network. Domain controllers hold the directory and replicate every change to the others in the domain. Joining a server to the domain is what lets domain users sign in to it and lets Group Policy manage it.

On Windows, the join is one cmdlet. Microsoft's example is Add-Computer -DomainName Domain01 -Restart. One detail changed recently: since security hardening in August 2024, if you name a specific domain controller with the -Server parameter, you must give its FQDN, such as DC1.contoso.com.

Linux servers join AD too. Red Hat's documentation recommends SSSD for identity and authentication, configured by realmd. realm discover ad.example.com finds the domain and realm join ad.example.com joins it. Red Hat lists the ports the Linux server must reach on the domain controllers:

Two hidden dependencies cause most failed joins. The first is DNS: clients find domain controllers through DNS records, so a server pointed at the wrong DNS server, such as a public resolver, cannot find the domain at all. The second is time. Kerberos uses timestamps to stop replay attacks, so client and domain controller clocks must agree. Microsoft's default "maximum tolerance for computer clock synchronization" is 5 minutes. A server whose clock drifts further cannot authenticate, which is why Red Hat's checklist says to verify that both systems' clocks are synchronized. Lesson 34 returns to clock skew as a troubleshooting cause.

Baselining: what normal looks like

A server is "slow" only compared with something. A baseline is that something: measurements of the server's normal performance, taken under normal load, kept so that later readings can be compared with them. Without one, a CPU at 70 percent might be a crisis or an ordinary Monday.

Collect a baseline after the server is built and its role is in real use, and cover a full business cycle: the end of the month for a finance database, Monday morning for a file server, the evening peak for a web server. Record at least the four resources every later lesson uses: processor, memory, disk and network.

• Windows. Performance Monitor's data collector sets record counters over time; from the command line, logman create counter creates a collector, with -c naming counters such as \Processor(_Total)\% Processor Time, -si setting the sample interval, and -max or -rf bounding the log.
• Linux. The sysstat package's sar collects, reports and saves system activity. sar -u 2 5 reports CPU every two seconds five times; -r covers memory, -d block devices and -n DEV network interfaces. Daily data files are kept by default in /var/log/sa, which makes last Tuesday's numbers available when someone asks what changed.

Store the baseline in the server's documentation alongside its role, configuration, owners, dependencies and ports. Objective 2.3 lists baselining, documentation and performance metrics together for a reason: a baseline nobody can find is no baseline at all. Lesson 16 turns these measurements into thresholds and alerts, and Lesson 21 places them in the wider documentation set.

The company, split up

After the split, the company runs two domain controllers with the Print Spooler disabled, a file server, a print server, a database server on fast mirrored storage and a web server in front of it. Each one is joined to the domain, synchronized to the same time source and documented with its role, ports and baseline. None of them is larger than its job needs, and a fault or a patch on one leaves the others running.

Common misconceptions

• "Default services are harmless if nobody uses them." The 2021 directive targeted a default service that most domain controllers never used.
• "A server can join the domain as long as it can ping a domain controller." It also needs DNS that resolves the domain, the directory ports open and a clock within the Kerberos tolerance.
• "Only Windows servers can join Active Directory." Red Hat documents joining RHEL with realmd and SSSD.
• "A baseline is a single reading taken on installation day." It is a set of measurements under real, representative load, covering the role's busy periods.

What you now know

• Separate server roles and run only what each needs; CISA's ED 21-04 ordered the Print Spooler off domain controllers within a day.
• Print, database, file, web, application and messaging roles differ in what they lean on and which ports they use (631 ipp, 1433, 445, 2049, 80/443, 25/587/993).
• AD DS stores accounts and resources and provides sign-in and access control; join Windows with Add-Computer and Linux with realm join.
• Directory connectivity needs correct DNS, ports 53, 389, 88, 464 and 3268, and clocks within Kerberos's 5-minute default tolerance.
• Baselines record normal CPU, memory, disk and network over a business cycle, using Performance Monitor or logman, and sar; keep them with the documentation.

Sources

1. Cybersecurity and Infrastructure Security Agency. (2021, July 13). Emergency Directive 21-04: Mitigate Windows Print Spooler service vulnerability. cisa.gov
2. Microsoft. (2025). Active Directory Domain Services overview. Microsoft Learn. learn.microsoft.com
3. Microsoft. (n.d.). Add-Computer (Microsoft.PowerShell.Management). Microsoft Learn. learn.microsoft.com
4. Microsoft. (2017). Maximum tolerance for computer clock synchronization [Security policy setting reference]. Microsoft Learn. learn.microsoft.com
5. Red Hat. (n.d.). Integrating RHEL systems directly with Windows Active Directory (Red Hat Enterprise Linux 10). docs.redhat.com
6. Microsoft. (2017). logman create counter. Microsoft Learn. learn.microsoft.com; and Godard, S. (n.d.). sar(1), sysstat. Linux manual page. man7.org
7. Internet Assigned Numbers Authority. (n.d.). Service name and transport protocol port number registry (ipp 631, submission 587, imaps 993, ms-sql-s 1433). iana.org

Two administrators, one storage pool

The lvmthin(7) manual page, which documents thin provisioning in Linux's Logical Volume Manager, ends its description of a full thin pool with a warning: the outcome "can result in file system damage that requires repair." Two administrators reading that sentence can draw opposite conclusions. One says thin provisioning is too dangerous for anything important. The other says the warning only describes what happens when nobody is watching. Both are experienced, and both are partly right. This lesson sets out the evidence on each side, says what settles the argument, and then applies the same approach to a second old dispute: how big swap and page files should be. Along the way it covers the rest of objective 2.3's storage tasks.

First, the terms. Thick provisioning reserves the full size of a volume when it is created: a 2 TB volume takes 2 TB of the pool on day one. Thin provisioning gives the volume its full logical size but consumes physical space only as data is written. A pool of 10 TB might back thin volumes adding up to 25 TB, on the bet that they will not all fill at once.

The case for thin

• It is everywhere. Microsoft lists thin provisioning as a feature of Storage Spaces virtual disks, allocating capacity dynamically as data grows rather than up front. LVM's thin volumes, lvmthin(7) explains, can have a virtual size much larger than the available physical storage.
• Space is used, not reserved. Most volumes run far below their allocated size for most of their lives. Thin provisioning lets the organization buy disks when data arrives, not when a volume is created.
• It combines with data reduction. Red Hat's VDO layer provides inline deduplication, compression and thin provisioning together. Microsoft's deduplication figures show why that matters: typical savings of 30 to 50 percent on user documents and 80 to 95 percent on virtualization libraries such as ISOs and virtual hard disks.

The case for thick

• A full pool fails badly. lvmthin(7) describes the default: when a thin pool runs out of data space, writes are queued for up to 60 seconds while the pool waits to be extended. If space is not added, the queued writes return errors to the application. In the alternative mode, writes fail immediately. Either way, the manual warns of file system damage.
• The failure is invisible from inside. Red Hat warns that because VDO is thin-provisioned, you can run out of physical space even when the file system still shows free room. The results include read-only file systems and ENOSPC, "no space left on device", errors.
• It fails everyone at once. Every thin volume in the pool shares the same physical space, so one runaway log file can stop the database next to it.

Remember: thin provisioning moves the risk from wasted space to running out. It is safe only when someone, or something, is watching physical usage.

What settles it

The two positions agree on the facts; they differ on whether monitoring will be done. The vendors' own guidance resolves it by making monitoring part of the design.

1. Watch physical usage, not file system free space. lvmthin(7) says the pool should be extended before either its data or metadata usage, shown as Data% and Meta% by lvs, reaches 100 percent. Red Hat recommends scripting a check of VDO's available physical space and alerting when use crosses a threshold.
2. Automate extension where you can. LVM's thin_pool_autoextend_threshold extends a pool when usage reaches a set percentage, with a minimum of 50, and thin_pool_autoextend_percent sets how much it grows each time. Autoextend needs free space in the volume group to draw on.
3. Thick-provision what must never stop. Database logs, boot volumes and anything whose sudden write errors would corrupt data deserve guaranteed space.
4. Set an over-commitment limit. Decide how far logical sizes may exceed physical capacity, and treat crossing it as a purchasing trigger.

A second dispute: how big should swap be?

Generations of administrators learned that swap, or the Windows page file, should be twice the size of RAM. The rule made sense when servers had 512 MB of memory. Current guidance replaces it with something more specific.

Red Hat adds that distributing swap across multiple storage devices improves swap performance. Microsoft's guidance for Windows says page file sizing is unique to each system and cannot be generalized, because it depends on two things. The first is the commit limit: RAM plus all page files is the ceiling on memory programs can commit, so peak commit charge must fit under it. The second is crash dumps: when the system stops with an error, it writes memory to disk through the page file or a dedicated dump file. A complete memory dump needs a page file of at least the size of RAM plus 257 MB. By default, Windows manages page files itself, growing them when commit charge passes 90 percent of the limit, up to three times RAM or 4 GB, whichever is larger.

So the old rule loses: a 512 GB database server with a 1 TB page file wastes space, while a small VM with no page file cannot capture a crash dump. Size swap from the vendor's table, size page files from the dump setting and the measured peak commit, and keep the dump requirement in mind before shrinking anything. Scratch space, the temporary working area some applications write and discard, belongs on fast local storage too. Because it is rebuilt rather than restored, it is normally left out of backups.

The routine: format, partition, provision

Whatever the provisioning model, adding storage to a running server follows the same path:

1. Connect the storage: a new local disk, a LUN presented over iSCSI or Fibre Channel (Lesson 7), or a share from a NAS.
2. Detect it: Disk Management or Get-Disk on Windows, lsblk on Linux after a rescan.
3. Partition it, GPT for anything new (Lesson 12), or hand the whole disk to LVM or Storage Spaces.
4. Format it with the file system and cluster size the workload needs.
5. Mount it or assign a drive letter or mount point, persistently.
6. Apply controls: quotas, compression or deduplication, monitoring thresholds, and a backup job.

Quotas, compression and deduplication

Quotas stop one user or project from filling a shared volume. NTFS supports per-user disk quotas. Windows Server's File Server Resource Manager (FSRM) adds folder quotas, quota templates and auto-apply quotas that cover every new subfolder. Microsoft's examples show the two styles. One is a 200 MB limit on personal folders with a warning email at 180 MB. The other is a flexible 500 MB group quota that is temporarily extended to 520 MB when reached. On Linux, xfs_quota sets a soft limit (bsoft), which may be exceeded for a grace period and starts a timer, and a hard limit (bhard), which cannot be exceeded; further writes fail.

Compression trades CPU time for space. NTFS offers file system compression, and Windows deduplication can optionally compress the chunks it stores. Deduplication stores repeated data once. Microsoft's table of typical savings is the best guide to where it pays: 30 to 50 percent for user documents, 70 to 80 percent for deployment shares, 80 to 95 percent for virtualization libraries and 50 to 60 percent for a general file share. Data that is already compressed or encrypted gains little. On RHEL, VDO provides both inline, with the physical-space monitoring caveat above.

Common misconceptions

• "Thin volumes show their real free space." The file system shows logical free space; the pool's physical space can run out first.
• "Swap should always be twice the RAM." Red Hat recommends at least 4 GiB for systems with 8 to 64 GiB of RAM; Microsoft sizes page files from crash dump settings and peak commit.
• "Removing the page file always improves performance." It lowers the commit limit and can prevent crash dumps from being written.
• "Deduplication saves the same everywhere." Savings range from 30 percent on user documents to 95 percent on virtualization libraries.
• "A soft quota is a hard quota that warns first." A soft limit can be exceeded for a grace period; only a hard limit blocks writes.

The short version

• Thick provisioning reserves space; thin provisioning allocates on write and must be monitored by physical usage, with autoextend and alerts.
• A full LVM thin pool queues writes for 60 seconds by default and then errors them; file systems can be damaged.
• Swap: Red Hat's table, such as at least 4 GiB for 8 to 64 GiB of RAM. Page files: crash dump setting plus peak commit; a complete dump needs RAM plus 257 MB.
• New storage: connect, detect, partition, format, mount, then apply quotas, data reduction, monitoring and backup.
• Quotas: NTFS per-user quotas, FSRM folder quotas and templates, xfs_quota soft and hard limits. Deduplication saves most on virtualization libraries and deployment shares.

Sources

1. LVM2 project. (n.d.). lvmthin(7): LVM thin provisioning. Linux manual page. man7.org
2. Red Hat. (n.d.). Deduplicating and compressing logical volumes on RHEL; and Managing storage devices, section "Recommended system swap space" (Red Hat Enterprise Linux 10). docs.redhat.com (VDO); docs.redhat.com (swap)
3. Microsoft. (2026). How to determine the appropriate page file size for 64-bit versions of Windows. Microsoft Learn. learn.microsoft.com
4. Microsoft. (2025). Data Deduplication overview; and Storage Spaces overview. Microsoft Learn. learn.microsoft.com (deduplication); learn.microsoft.com (Storage Spaces)
5. Microsoft. (2017). Quota management (File Server Resource Manager). Microsoft Learn. learn.microsoft.com
6. Linux XFS project. (n.d.). xfs_quota(8). Linux manual page. man7.org

A busy number that is not a problem

The manual page for iostat, the Linux disk statistics tool, carries a warning that surprises many administrators. Its %util column shows how much of the time a device had requests in progress. Values near 100 percent mean saturation for a device that handles one request at a time. For devices that serve requests in parallel, such as RAID arrays and modern SSDs, the manual says the number does not reflect their performance limits. An alert built on %util alone would page someone every night about an SSD array that is nowhere near its limit. Monitoring is full of numbers like that, which look alike and mean different things. This lesson compares them side by side, then compares the two families of logs, Windows and Linux, the same way.

What to measure: the resource table

Two rows in that table are easy to confuse. Capacity asks how full something is; utilization asks how busy it is. A file server's volume can be 92 percent full while its disks sit idle, which is a capacity problem: order space. A database volume can be 30 percent full while its disks queue requests all day, which is a utilization problem: add IOPS. CompTIA lists "capacity vs. utilization" as a pair because the fixes are different.

Memory has the same trap. Microsoft notes that hard page faults include reading parts of program files and memory-mapped files, not only the page file. A high paging counter by itself is therefore not proof of a memory shortage. Pair it with low available memory before concluding that the server needs RAM.

In short: read each metric for what it actually measures. Latency and queue length show a disk struggling, and free space shows a disk filling. Only the baseline from Lesson 14 tells you whether a number is unusual for this server.

Thresholds and alerting

A threshold turns a metric into an alert. Good thresholds share three features.

1. They come from the baseline. A CPU that normally runs at 60 percent needs a different alert from one that normally idles at 5.
2. They have duration. Alert on CPU above the threshold for fifteen minutes, not for one sample, or brief spikes will train people to ignore the pager.
3. They come in levels. A warning goes to a queue for the next working day, while a critical alert wakes someone. Disk capacity suits this well, with a warning while there is time to order space and a critical alert before writes start failing.

Thresholds also apply to trends. Free space falling by a fixed amount each week can be projected forward, so the alert reads "full in 30 days" instead of "full now".

Logs compared: Windows and Linux

RFC 5424's severity scale is worth memorizing in order, because exam questions and log filters both use the numbers. Severity 0, Emergency, means the system is unusable. Severity 3 is Error, 4 is Warning, 6 is Informational and 7 is Debug.

The log life cycle: from configuration to rotation

NIST published its Guide to Computer Security Log Management (SP 800-92) in September 2006. It describes log management infrastructure as the hardware, software, networks and media used to generate, transmit, store, analyze and dispose of log data. Objective 2.3's list of event-log tasks maps onto that life cycle.

• Configuration. Decide what is logged before you need it. Microsoft's guidance for Windows Event Forwarding lists a minimum audit policy, for example success and failure for logons and for user account management. Events that were never generated cannot be collected later.
• Shipping. Copy logs off the server, so an attacker who clears local logs or a failed disk cannot erase the record. Windows Event Forwarding reads chosen events and forwards them to a Windows Event Collector. In a domain the connection is mutually authenticated and encrypted with Kerberos, and subscriptions can be pushed by the source or pulled by the collector. One caveat matters: the client's local event log is WEF's buffer, and if it overwrites events while the collector is unreachable, the collector is not told that anything was lost. On Linux, rsyslog forwards syslog messages to a central server.
• Alerting. A central system can correlate events across servers and raise alerts, which Microsoft describes as a security event manager's strength. Lesson 26 covers SIEM in more depth.
• Reporting. Scheduled summaries, such as failed logons per day or disk growth per month, turn logs into management information.
• Retention. Keep logs as long as policy, law or regulation requires; NIST notes that logs often must be preserved for legal or regulatory reasons, and moved to archival storage for that. Locally, Windows offers a choice when a log is full. With retention mode true, existing events are kept and incoming events are discarded; with it false, new events overwrite the oldest. Auto-backup saves the full log before starting again.
• Rotation. NIST defines log rotation as closing a log file and opening a new one when the first is considered complete, on a schedule or at a size limit. On Linux, logrotate does it: daily or weekly sets the schedule, size rotates by size, rotate sets how many old files to keep, compress gzips them, maxage removes old ones, missingok and notifempty handle missing or empty logs, and a postrotate script can tell the service to reopen its log.

One more requirement runs under all of these: time. NIST lists inconsistent timestamps as a core log management problem, because analysis across servers depends on events lining up. It recommends synchronizing every logging host's clock to a common time source. A three-minute skew can make an attacker's steps appear out of order.

Common misconceptions

• "100 percent %util means the disk is maxed out." Not for RAID arrays or SSDs that serve requests in parallel; watch wait times and queue length.
• "A full disk and a busy disk are the same problem." One is capacity, the other utilization; they need different fixes.
• "Heavy paging always means too little RAM." Hard page faults also come from reading program and memory-mapped files; check available memory too.
• "Forwarded logs can never be lost." If the local log overwrites events while the collector is unreachable, Windows Event Forwarding loses them without notice.
• "Retaining logs and rotating logs conflict." Rotation keeps files manageable; retention decides how long rotated files and archives are kept.

Putting it together

• Measure CPU, memory, disk IOPS and latency, capacity, network and uptime, each with the right counter or tool.
• Capacity is how full; utilization is how busy. iostat's %util understates parallel devices.
• Thresholds come from baselines, need duration and come in warning and critical levels; trend capacity forward.
• Windows event logs and Linux syslog and journal differ in tools, but share the job; RFC 5424 severities run from 0 Emergency to 7 Debug.
• Configure what is logged, ship it centrally, alert and report on it, retain it as required and rotate it with logrotate or event log size and retention settings.
• Synchronize clocks so logs from different servers line up.

Sources

1. Godard, S. (n.d.). iostat(1) and sar(1), sysstat. Linux manual pages. man7.org (iostat); man7.org (sar)
2. Microsoft. (2026). How to determine the appropriate page file size for 64-bit versions of Windows [performance counters section]. Microsoft Learn. learn.microsoft.com
3. Kent, K., and Souppaya, M. (2006). Guide to computer security log management (NIST Special Publication 800-92). National Institute of Standards and Technology. csrc.nist.gov
4. Gerhards, R. (2009). The syslog protocol (RFC 5424). RFC Editor. rfc-editor.org
5. Microsoft. (2025). Use Windows Event Forwarding to help with intrusion detection. Microsoft Learn. learn.microsoft.com
6. Microsoft. (2022). wevtutil. Microsoft Learn. learn.microsoft.com
7. logrotate project. (n.d.). logrotate(8). Linux manual page. man7.org

Eight is the number that matters

Robocopy, the Windows tool for copying file data, reports its result as an exit code, and Microsoft's documentation draws one line through the list: any value of 8 or greater means at least one failure during the copy. Codes from 0 to 7 describe successful outcomes, such as files copied or extra files found at the destination. A migration script that treats every nonzero code as an error will raise false alarms, and one that ignores codes will report success over missing files. This lesson walks through a real data migration with robocopy step by step. It then changes the destination to Linux and shows what changes with it, and finishes with the interfaces administrators use to reach servers at all.

Procedure 1: moving a department's share to a new server

The engineering share on an old Windows file server, fs01, holds 1.8 TB in 900,000 files. It must move to fs02 with every permission intact and as little downtime as possible. The plan uses two passes: a long pre-seeding copy while users keep working, then a short final pass during the cutover window.

1. Prepare. Create the destination folder on fs02, confirm free space, and run a list-only rehearsal with /L, which lists what would be copied without copying, deleting or time-stamping anything.
2. Pre-seed with full fidelity. Run robocopy \\fs01\engineering E:\engineering /E /COPYALL /DCOPY:DAT /ZB /MT:16 /R:2 /W:5 /LOG:C:\logs\eng-pass1.log. Each option has a job:
   • /E copies subdirectories, including empty ones.
   • /COPYALL copies all file information, equivalent to /COPY:DATSOU: data, attributes, time stamps, NTFS ACLs, owner and auditing information. The default, /COPY:DAT, would silently drop the permissions.
   • /DCOPY:DAT keeps directory time stamps as well as data and attributes.
   • /ZB uses restartable mode and switches to backup mode if access is denied, so the copy can read files whose ACLs would otherwise block it.
   • /MT:16 copies with 16 threads. The default is 8 and the range 1 to 128; Microsoft suggests logging to a file for better performance when multithreading.
   • /R:2 /W:5 retries a failed file twice, five seconds apart. The defaults are one million retries, 30 seconds apart, enough to stall a migration for weeks on one locked file.
3. Check the exit code and the log. Values below 8 mean no failures; read the log for skipped or mismatched files.
4. Cut over. In the maintenance window, stop changes on fs01 by making the share read-only, then run a final pass with /MIR instead of /E. /MIR mirrors the tree: it is /E plus /PURGE, so it also deletes destination files that no longer exist in the source. That is exactly what a final sync needs, and exactly why /MIR pointed the wrong way round destroys data.
5. Switch users over, through a DNS alias or DFS namespace if one exists, and keep fs01 read-only until the new server has run cleanly for an agreed period.

The upshot: a migration is a copy plus proof. The options decide what survives the copy, and the log and exit code prove whether it did.

Robocopy's multithreading is the objective's "fast copy" in practice, with two related options. /J uses unbuffered I/O, recommended for very large files, and /COMPRESS requests network compression where available. For a migration across a slow WAN link, the inter-packet gap option /IPG does the opposite job, leaving bandwidth for users, but it cannot be combined with /MT.

Procedure 2: change the destination to Linux

Now the same project data must go to a Linux analytics server. Three things change.

• The protocol. SCP copies files between hosts over SSH. The OpenSSH manual states that it uses the same authentication and provides the same security as an SSH login session, and that since OpenSSH 9.0 it uses the SFTP protocol for transfers by default. A recursive copy that preserves times and modes is scp -r -p followed by the source and the user@host:path destination. -P, capital, sets a non-standard SSH port, because lowercase -p already means preserve.
• The permissions. NTFS ACLs, with their owners and inheritance, have no direct equivalent in Linux's owner, group and mode bits, or in POSIX ACLs. A disparate-OS transfer moves the data; the permissions must be designed again on the destination and checked, not assumed.
• The verification. With no robocopy log to read, verify by comparing checksums of a sample of files on both ends, for example with Get-FileHash on Windows and sha256sum on Linux, before declaring the move complete.

Data going in and data going out

Objective 2.3 lists two directions: infiltration and exfiltration. Data coming into an environment, such as a vendor's dataset, installation media or a user's upload, can carry malware, so it should arrive through a controlled path and be scanned before it reaches production servers. Data leaving is the bigger worry. NIST's glossary, citing SP 800-53 Revision 5, defines exfiltration as the unauthorized transfer of information from a system. Every legitimate transfer tool, robocopy, SCP and file shares among them, is also an exfiltration tool in the wrong hands. That is why transfers should use encrypted protocols, run under accounts that can reach only the data they need, and be logged. Lesson 26 adds data loss prevention.

Reaching the server: four administrative interfaces

Remote Desktop. Microsoft's guidance is cautious: enabling Remote Desktop opens a port, so enable it only on trusted networks. For access from outside, it recommends a VPN rather than exposing the port. It also explains Network Level Authentication (NLA): users must authenticate before a remote session is established, which reduces the risk from unauthorized users and malicious software. Keep NLA on.

SSH. Beginning with Windows Server 2025, Microsoft installs OpenSSH by default; you enable it in Server Manager under Remote SSH Access, or with Start-Service sshd. Access can be limited to members of the OpenSSH Users group. On Linux and Windows alike, the server's sshd_config controls who may sign in and how. The OpenSSH defaults allow both password and public-key authentication, and set PermitRootLogin to prohibit-password, which blocks password logins for root. Hardening means switching PasswordAuthentication to no once every administrator uses keys, and naming permitted accounts with AllowUsers or AllowGroups.

Web interfaces. Red Hat describes its web console as a web-based graphical interface for managing and monitoring the system, listening by default on TCP port 9090. You enable it with systemctl enable --now cockpit.socket, open it in firewalld with the cockpit service, and sign in at https:// followed by the host name and :9090. A web console is convenient precisely because anyone with a browser can reach it, which is why it belongs on the management network.

Common misconceptions

• "Any nonzero robocopy exit code means the copy failed." Codes below 8 report successful outcomes; 8 or above means at least one failure.
• "Robocopy copies permissions by default." Its default is /COPY:DAT; permissions, owner and auditing need /COPYALL or the matching flags.
• "/MIR is just a thorough copy." It also deletes destination files missing from the source.
• "SCP is obsolete and insecure." It runs over SSH with the same authentication and security, and modern OpenSSH carries it over the SFTP protocol.
• "RDP is safe on the internet if the password is strong." Microsoft advises enabling it only on trusted networks and using a VPN for outside access, with NLA on.

Where this leaves us

• Robocopy migrations: /E or /MIR, /COPYALL for permissions, /ZB for backup mode, /MT for speed, short /R and /W, /LOG, and exit codes below 8 for success.
• Disparate-OS transfers: SCP over SSH (SFTP by default since OpenSSH 9.0), redesigned permissions and checksum verification.
• Infiltration brings risk in; exfiltration is the unauthorized transfer of information out. Encrypt, restrict and log transfers.
• Interfaces: console, RDP on 3389 with NLA, SSH on 22 with keys, web consoles such as Cockpit on 9090; keep them on management networks.

Sources

1. Microsoft. (2025). Robocopy. Microsoft Learn. learn.microsoft.com
2. OpenBSD project. (n.d.). scp(1) and sshd_config(5). OpenBSD manual pages. man.openbsd.org (scp); man.openbsd.org (sshd_config)
3. National Institute of Standards and Technology. (n.d.). Exfiltration, in CSRC glossary (citing NIST SP 800-53 Rev. 5). csrc.nist.gov
4. Microsoft. (2025). Get started with OpenSSH Server for Windows. Microsoft Learn. learn.microsoft.com
5. Microsoft. (2025). Enable Remote Desktop on your PC. Microsoft Learn. learn.microsoft.com
6. Red Hat. (n.d.). Managing systems in the RHEL web console (Red Hat Enterprise Linux 10). docs.redhat.com

Two designs, one argument

Microsoft's overview of failover clustering describes two configurations and declines to pick a winner. In an active-active cluster, all nodes work at once and share the load. In an active-passive cluster, some nodes run the workload while others wait on standby to take over. Microsoft says the choice depends on an organization's needs and how critical the application is. Engineers argue about it constantly, and both sides have real evidence. This lesson lays out each position, finds the question that settles it, and then covers the rest of objective 2.4: heartbeats, quorum, failover and failback, patching, load balancing and NIC teaming.

First, the shared vocabulary. A failover cluster, in Microsoft's words, is a group of independent computers, called nodes, working together to raise the availability and scalability of applications called clustered roles. If one or more nodes fail, the remaining nodes take over the workload automatically: that is failover. Clusters monitor node health with heartbeat signals, often over a dedicated private network kept separate from client traffic.

The case for active-active

• Nothing sits idle. Microsoft describes active-active as maximizing resource utilization, because every node handles part of the workload, which also improves responsiveness and throughput.
• A failure costs capacity, not service. If one node fails, Microsoft notes, the rest continue to handle the workload; overall capacity may drop for a while, but the system stays operational.
• Maintenance can be invisible. For workloads that run on several nodes at once, such as Hyper-V with live migration or file servers with SMB Transparent Failover, Microsoft says Cluster-Aware Updating can patch the cluster with no impact on service availability.

The case for active-passive

• It is simpler. Microsoft calls active-passive often easier to implement and maintain, with standby nodes ready to take over the failed node's duties.
• Performance stays predictable. Because the standby node carries nothing until it is needed, a failover lands the workload on a server with its full capacity free.
• Not every application can run in two places. Many applications, especially those that assume they own their data, run safely on only one node at a time. For them, active-passive is not a preference but the only option.

The price is the one active-active supporters point to: standby hardware that may not make full use of its resources during normal operation.

What settles it: can the survivors carry the load?

Work the numbers on a two-node active-active cluster where each node runs at 70 percent of its capacity. When one node fails, the survivor must carry both workloads: 140 percent of what it can deliver. The cluster has not failed, but the service has. To survive a node loss, an active-active cluster of two nodes must run each node below 50 percent. With four nodes, each must stay below 75 percent so that three can absorb the fourth's load. That reserve is the same idle capacity active-passive is criticized for. It is simply spread across every node instead of parked on one.

So the question that settles the argument is not which design is better, but two narrower ones. Can the application run on several nodes at once? And has the capacity plan reserved enough headroom for the survivors? If the application cannot run in two places, the answer is active-passive. If it can, active-active is efficient only when monitoring (Lesson 16) keeps every node below the failover threshold.

Worth holding on to: high availability is a capacity calculation. After the worst failure you plan for, the remaining nodes must still carry the whole workload.

Quorum, failover and failback

Heartbeats can fail because a node died, or because the network between nodes broke while both kept running. In the second case, each half might believe the other is dead and start the same workload: a split-brain. Microsoft's clusters prevent this with quorum, the minimum number of votes from cluster members needed for the cluster to run. Only the side holding a majority of votes keeps running roles. Quorum models such as Node Majority, or Node and Disk Majority, which adds a witness vote, decide how votes are counted.

Failback is what happens after the failed node returns: does the workload move back? VMware's storage path policies show the two answers side by side. Its Fixed policy returns to the preferred path when that path becomes available again: it fails back. Its Most Recently Used (MRU) policy stays on the new path, and VMware notes it does not revert when the original path returns. Failing back automatically restores the intended layout but causes a second brief interruption. Staying put avoids that, at the cost of an unbalanced cluster until someone moves things back in a maintenance window.

Patching a cluster properly

Objective 2.4 lists proper patching procedures under clustering, because patching is the most common planned failover. Microsoft's Cluster-Aware Updating (CAU) automates the sequence, and the same steps apply when you do it by hand on any platform:

1. Put one node into maintenance mode.
2. Move its clustered roles to other nodes.
3. Install the updates and any dependencies.
4. Restart if needed.
5. Bring the node out of maintenance mode.
6. Restore the clustered roles to it.
7. Move on to the next node.

Microsoft notes that for many roles this still causes a planned failover and a brief interruption for clients, while continuously available workloads see none. CAU can run in self-updating mode, as a clustered role on a schedule, or in remote-updating mode from a coordinator outside the cluster.

Fault tolerance: components versus servers

The two levels combine rather than compete: component redundancy keeps individual servers up, and server-level redundancy keeps the service up when a server goes down anyway.

Load balancing: spreading requests across servers

A load balancer presents one address to clients and spreads requests across a pool of servers. Hardware load balancers are dedicated appliances; software load balancers run on ordinary servers or VMs. NGINX, a widely used software load balancer, documents the core methods:

• Round robin, the default, hands requests to each server in turn, with optional weights to send more to stronger servers.
• Least connections sends each new request to the server with the fewest active connections.
• IP hash sends a given client to the same server each time, which keeps a user's session on one machine unless that server fails.

NGINX also performs passive health checks: after a set number of failed attempts within a timeout, it marks a server as failed, stops sending it requests for a while and then probes it again. The objective's most recently used appears in a different kind of balancing, choosing among multiple paths to storage, where VMware makes MRU the default for most active-passive arrays.

NIC teaming: redundancy for the network link

Microsoft describes NIC teaming, which it also calls load balancing and failover (LBFO), as placing several network adapters in a team for two purposes: bandwidth aggregation and traffic failover. Linux's bonding driver offers the same choices as modes:

The bond checks each link's state at an interval set by miimon, in milliseconds; the kernel documentation suggests 100 as a starting point. For pure failover, active-backup is the safe default because it needs nothing from the switch. For more bandwidth, link aggregation needs matching switch configuration, and connecting the team to two different switches for redundancy requires switches that support aggregation across both.

Common misconceptions

• "Active-active means a failure has no effect." The survivors must absorb the failed node's load; without headroom, the service degrades or fails.
• "Active-passive wastes half the money." Its standby capacity is the same reserve an active-active cluster must keep, just concentrated in one place.
• "Clustering removes the need for patching windows." Rolling updates still fail roles over between nodes, briefly interrupting all but continuously available workloads.
• "Any NIC team doubles bandwidth." Active-backup gives failover only; aggregation modes need switch support.
• "Redundant power supplies make a server highly available." They protect against one component; a motherboard or OS failure still needs server-level redundancy.

Pulling it together

• Active-active uses every node but needs headroom; active-passive is simpler and suits applications that run on one node only. Size so the survivors can carry the load.
• Heartbeats detect failures; quorum prevents split-brain; failback returns workloads after recovery, as VMware's Fixed policy does and MRU does not.
• Patch clusters node by node: maintenance mode, move roles, update, restart, return roles, next node.
• Component redundancy protects parts; server-level redundancy protects the service.
• Load balancing: round robin (default), least connections, IP hash for persistence, health checks to drop failed servers.
• NIC teaming: active-backup for failover without switch changes; 802.3ad (LACP) aggregation with switch support.

Sources

1. Microsoft. (2025). Failover clustering [overview]. Microsoft Learn. learn.microsoft.com
2. Microsoft. (2021). Cluster-Aware Updating overview. Microsoft Learn. learn.microsoft.com
3. Broadcom. (n.d.). VMware path selection plug-ins and policies (vSphere 8.0 storage). techdocs.broadcom.com
4. NGINX. (n.d.). Using nginx as HTTP load balancer. nginx.org
5. Microsoft. (2016). NIC Teaming overview (Windows Server 2012 R2). Microsoft Learn (archived). learn.microsoft.com
6. The kernel development community. (n.d.). Linux Ethernet bonding driver HOWTO. Linux kernel documentation. docs.kernel.org

The same idea, built two ways

NIST's Guide to Security for Full Virtualization Technologies (SP 800-125, January 2011) describes two forms of the technology this course's lab runs on. In bare metal virtualization, also called native virtualization, the hypervisor runs directly on the hardware with no host operating system, and can even be built into firmware. In hosted virtualization, the hypervisor runs on top of a host operating system. NIST adds the observation that sorts them in practice: servers are most often virtualized with bare metal hypervisors, desktops with hosted ones. The VirtualBox lab from Lesson 1 is hosted virtualization on your own computer. A Hyper-V or ESXi server in a data center is bare metal. This lesson compares the two, then compares how VMs get resources, how they connect to networks, and how cloud models extend the idea beyond one building.

Hosts, guests and hypervisors

VirtualBox's manual gives the two basic terms. The host operating system runs on the physical computer where the hypervisor is installed; the guest operating system runs inside a virtual machine. On a bare metal hypervisor, the host role belongs to the hypervisor and its small management layer.

NIST makes two security points that matter to administrators. First, the security of the whole virtual infrastructure relies on the virtualization management system, so administrator access to it must be restricted and protected. Second, guest tools that let a guest reach host files or the clipboard are useful, but can also become a path between guests and hosts that should not exist.

Management interfaces are how administrators drive the hypervisor. On Hyper-V that means Hyper-V Manager and PowerShell cmdlets such as New-VMSwitch. On VMware it means the vSphere Client and vCenter. On VirtualBox it means the VirtualBox Manager window and the VBoxManage command line. Like BMCs in Lesson 8, they belong on the management network.

Resources: allocation and overprovisioning

Microsoft's Hyper-V Dynamic Memory shows how memory overprovisioning is controlled rather than left to chance. Each VM gets five settings. Startup RAM is what it boots with. Minimum RAM is the floor it is never reduced below once running, and maximum RAM is the ceiling it may grow to. The memory buffer is a percentage of headroom added above what the VM is using. The memory weight decides which VMs get memory first when there is not enough for all. Microsoft notes the benefit is higher consolidation, especially for idle or lightly loaded VMs such as pooled virtual desktops. It also says what happens at the limit: when host memory is oversubscribed, Hyper-V relies on paging inside the guest operating system. Paging moves memory out to disk, which is far slower than RAM.

So what?: overprovisioning is a bet that not every VM will need its full allocation at the same moment. It raises consolidation and lowers cost, and it fails as contention exactly when demand peaks, so monitor host CPU, memory and datastores, not just the guests.

Scalability comes in two directions. Scale up gives one VM more vCPUs or memory; Hyper-V, for example, lets you increase a running VM's maximum memory. Scale out adds more VMs behind a load balancer, the pattern from Lesson 18. Virtualization makes both quick, which is a large part of why servers are now mostly virtual.

Virtual networking compared

A virtual switch connects vNICs to each other and, optionally, to the physical network. The two platforms you have met name their options differently but offer the same choices.

Microsoft's documentation adds two server-grade features. An external switch can optionally let the management OS share the same physical adapter. Virtual switches and VM adapters can also carry VLAN IDs: access mode for one VLAN, trunk mode for several. That ties virtual networks into the VLAN design from Lesson 13. Microsoft also notes the NAT design's appeal: it conserves addresses and lets several VMs use the same internal port numbers.

Cloud models compared

NIST's Definition of Cloud Computing (SP 800-145, September 2011), by Peter Mell and Timothy Grance, is the definition the industry still cites. It lists five essential characteristics: on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service. Virtualization is what makes resource pooling and elasticity possible, which is why NIST SP 800-125 calls full virtualization a core enabling technology for cloud infrastructure as a service.

Service models describe who manages what. In infrastructure as a service (IaaS), NIST says, the consumer provisions processing, storage and networks and can run any software, including operating systems. The consumer controls the OS, storage and applications, but not the underlying infrastructure. In platform as a service (PaaS) the consumer deploys applications without managing the servers or OS. In software as a service (SaaS) the consumer simply uses the provider's application. A server administrator's work is closest to IaaS: the VM is yours to patch, harden and back up.

Common misconceptions

• "A Type 2 hypervisor is just a slower Type 1." The difference is architecture: a hosted hypervisor depends on a host OS, which suits desktops and labs; servers usually run bare metal.
• "Overprovisioning means wasting resources." It means allocating more than physically exists, betting on unequal peaks; it saves money until peaks coincide.
• "Bridged and NAT networking are the same from the LAN's view." Bridged VMs appear as separate hosts on the LAN; NAT hides them behind the host's address.
• "In the cloud, the provider patches my servers." In IaaS, NIST places control of the operating system with the consumer.
• "Hybrid cloud means any company using two providers." NIST's hybrid cloud binds distinct clouds with technology for data and application portability.

Summing up

• Bare metal hypervisors run on the hardware (servers); hosted ones run on a host OS (desktops and labs). Hosts run hypervisors; guests run inside VMs.
• Protect management interfaces: Hyper-V Manager and PowerShell, vSphere, VirtualBox Manager and VBoxManage.
• Allocate vCPUs, memory, disk images and vNICs; overprovision with care and watch the host for contention.
• Dynamic Memory uses startup, minimum and maximum RAM, a buffer and a weight.
• Networking: bridged or external for direct LAN access, NAT to hide VMs, host-only or internal, and private for VM-only networks; VLAN IDs carry virtual networks into the physical design.
• NIST cloud: five characteristics; IaaS, PaaS and SaaS; private, public, hybrid and community deployment.

Sources

1. Scarfone, K., Souppaya, M., and Hoffman, P. (2011). Guide to security for full virtualization technologies (NIST Special Publication 800-125). National Institute of Standards and Technology. csrc.nist.gov
2. Mell, P., and Grance, T. (2011). The NIST definition of cloud computing (NIST Special Publication 800-145). National Institute of Standards and Technology. csrc.nist.gov
3. Microsoft. (2025). Create and configure a virtual switch with Hyper-V. Microsoft Learn. learn.microsoft.com
4. Microsoft. (2016). Hyper-V Dynamic Memory overview (Windows Server 2012 R2). Microsoft Learn (archived). learn.microsoft.com
5. Oracle. (2025). Oracle VirtualBox user guide, release 7.2: Introduction (host and guest) and Virtual networking. docs.oracle.com; virtualbox.org

Why servers get scripts

Monday morning brings three jobs. Forty new staff need accounts. A web service on twelve servers must be checked every hour. And every new virtual machine must get the same agent and settings on its first boot. Each job can be done by hand, but by hand it is slow, and the fortieth account is rarely made exactly like the first. A script is a text file of commands that a shell or interpreter runs in order. It does the same steps the same way every time, and the file itself records what was done.

The Server+ objectives ask you to read and reason about short scripts in four languages, not to become a programmer. This lesson is a procedure in six steps: choose the language, store data, make decisions, repeat work, comment and test, and attach the script to the event that should run it. Try every example in your lab VMs, never on a production server.

Step 1: Choose the script type

The operating system usually decides. Linux servers speak Bash. Windows servers speak PowerShell, with Batch and VBScript still found in older automation.

Two facts shape the choice on Windows. In October 2023, Microsoft announced that it would gradually deprecate VBScript. Its May 2024 timeline makes VBScript an optional feature that is on by default in Windows 11 version 24H2, off by default around 2027, and later removed from future versions of Windows. Microsoft points to PowerShell as the way forward. Batch remains, and Windows Setup still looks for a batch file at the end of installation, as Step 6 shows. For new work, the practical answer is Bash on Linux and PowerShell on Windows.

Step 2: Store data in variables

A variable is a named place to keep a value. The objectives name three data types: integers (whole numbers such as 90), strings (text such as web01) and arrays (ordered lists, such as the names of twelve servers). The languages treat them differently.

# Bash: variables hold text; arithmetic treats them as integers
name=web01                     # a string (no spaces around =)
limit=90                       # used as an integer below
servers=(web01 web02 db01)     # an indexed array, numbered from 0
total=$(( limit + 5 ))         # $(( )) does integer arithmetic: 95

# PowerShell: values keep their type
$name = 'web01'                          # a string
[int]$limit = 90                         # declared as an integer
$servers = @('web01', 'web02', 'db01')   # an array; $servers[0] is web01
$total = $limit + 5                      # 95

REM Batch: everything is text, and set /a evaluates arithmetic
set name=web01
set /a total=90+5

The Bash 5.3 manual page confirms that Bash arrays are one-dimensional and zero-based, so the first server is element 0. PowerShell's documentation says the same of its arrays, and adds that one array can hold items of the same type or of different types. Batch has no array type, so batch files loop over lists of words instead.

Environment variables are values the operating system hands to every program it starts: the search path, the computer name, the user's home folder. Scripts read them instead of hard-coding values. In Bash, the export command marks a variable so that programs the script starts inherit it. In cmd.exe, the set command displays, sets or removes environment variables. In PowerShell, $Env:Name reads or sets one. Microsoft notes that such a change lasts only for the current process unless it is saved at User or Machine scope.

Step 3: Decide with conditionals and comparators

A conditional runs commands only when a test is true, using if and else. The test uses a comparator. One trap catches everyone once: in Bash, Batch and PowerShell alike, the > character redirects output into a file. So each shell spells its comparisons with letters.

The Bash manual describes -eq, -ne, -lt, -le, -gt and -ge as arithmetic operators for integers, and = as a string comparison. Microsoft's documentation makes the matching points for the other two shells. PowerShell string comparisons ignore case unless you add a c, as in -ceq. And cmd.exe compares as numbers when both sides contain only digits.

#!/bin/bash
# Warn when the root file system is more than 90 percent full.
limit=90
used=$(df --output=pcent / | tail -1 | tr -dc '0-9')
if [ "$used" -gt "$limit" ]; then
  echo "WARNING: / is $used% full on $(hostname)"
else
  echo "OK: / is $used% full"
fi

# PowerShell: start the print spooler if it is not running, and say so.
$svc = Get-Service -Name Spooler
if ($svc.Status -ne 'Running') {
    Start-Service -Name Spooler
    Write-Output "Started Spooler on $Env:COMPUTERNAME"
}

Step 4: Repeat with loops

A loop repeats commands. It can run once per item in a list (for and foreach), as long as a test stays true (while), or until a test becomes true (until).

# Bash: once per server name
for s in web01 web02 db01; do
  echo "Checking $s"
done

# PowerShell: once per item in an array
foreach ($s in $servers) {
    Write-Output "Checking $s"
}

REM Batch: once per log file
for %%f in (D:\Logs\*.log) do echo Found %%f

Microsoft's for documentation explains the doubled percent sign: use one % at the command prompt and two inside a batch file. Loops that wait need a way out. This bootstrap fragment waits for a database service, but gives up after a minute instead of hanging forever:

tries=0
until systemctl is-active --quiet postgresql || [ "$tries" -ge 12 ]; do
  sleep 5
  tries=$(( tries + 1 ))
done

The systemctl is-active command returns exit code 0 when the unit is active, and --quiet stops it printing the state. The counter, an integer tested with -ge, is what guarantees the loop ends.

Step 5: Comment, test and run safely

• Comment the why. Start each script with a header: what it does, who owns it and when it last changed. Comment the reasons for odd steps, not every obvious line. Bash ignores everything from a word beginning with # to the end of the line. PowerShell uses # for a line and <# ... #> for a block, and block comments cannot be nested. Batch uses REM, and Microsoft notes that a batch comment cannot contain a redirection or pipe character.
• Preview before you change. Many PowerShell commands accept -WhatIf, which describes what the command would do instead of doing it.
• Know the execution policy. On Windows Server, PowerShell's effective default is RemoteSigned. Scripts written locally run, but scripts downloaded from the internet must be signed by a trusted publisher or unblocked. Microsoft is explicit that execution policy is not a security boundary. It prevents accidents, not attacks, because a user can simply type a script's contents at the prompt.
• Test on a snapshot. Run a new script in a lab VM, roll back, and run it again. Check exit codes: Bash keeps the last one in $?, and cmd.exe in %ERRORLEVEL%.
• Keep secrets out. Never type a password into a script file. Prompt for it, as the account example below does, or use a secrets store.

The core of it: a script is a procedure written so precisely that a computer can follow it. Choose the language the server speaks, keep values in variables, test with the right comparator, give every loop a way out, and comment for the next administrator.

Step 6: Attach the script to the right event

The objectives list common scripted jobs. When a script runs, and with whose rights, matters as much as the code.

Three details from Microsoft's guidance matter in practice. When several scripts are assigned to one Group Policy Object, they run in the order you set. A logon script that needs administrator rights, such as one that installs a driver, will fail for ordinary users, so that job belongs in a computer startup script. And Windows Setup does not check SetupComplete.cmd's exit codes, so a bootstrap script must log its own results.

# Create local accounts from new-staff.csv (columns: Name, FullName).
# The temporary password is typed once at a prompt, never stored in the file.
$password = Read-Host -AsSecureString 'Temporary password'
Import-Csv .\new-staff.csv | ForEach-Object {
    New-LocalUser -Name $_.Name -FullName $_.FullName -Password $password
}

# Bash: one account per line of new-staff.txt (run as root)
while read -r user; do
  useradd -m "$user"
done < new-staff.txt

Common misconceptions

• "Execution policy keeps malicious scripts off the server." Microsoft calls it defense in depth, not a security boundary. It prevents accidents.
• "The > sign means greater than in any script." In Bash tests, Batch and PowerShell it redirects output to a file. Use -gt or GTR.
• "PowerShell's -eq is case-sensitive." String comparisons ignore case unless you use -ceq.
• "A logon script can do anything a startup script can." Logon scripts run with the user's rights; startup scripts run as Local System.
• "VBScript is a safe choice for new automation." Microsoft is phasing it out, with removal planned for future Windows versions.
• "A waiting loop always finishes." Without a counter or timeout, a loop waiting for a failed service runs forever.

Looking back

• Bash for Linux and PowerShell for Windows; Batch and VBScript survive in older automation, and VBScript is being retired.
• Comments: # in Bash and PowerShell (with <# #> for PowerShell blocks), REM in Batch and an apostrophe in VBScript.
• Environment variables: $HOME in Bash, %COMPUTERNAME% in Batch and $Env:COMPUTERNAME in PowerShell.
• Data types are integers, strings and zero-based arrays; the constructs are variables, conditionals, comparators and loops.
• Comparators are spelled with letters (-gt, GTR) because > redirects output.
• Match the task to the event: startup and shutdown scripts run as Local System, logon scripts as the user, and bootstrap scripts once at first boot.

Sources

1. Ramey, C., and Fox, B. (2025). bash(1): GNU Bourne-Again SHell manual page (Bash 5.3). Linux man-pages project. man7.org
2. Microsoft. (n.d.). about_Comparison_Operators, about_Environment_Variables and about_Execution_Policies. PowerShell documentation, Microsoft Learn. Comparison operators; Environment variables; Execution policies
3. Microsoft. (n.d.). if, rem, set and for. Windows Commands reference, Microsoft Learn. if; rem; set; for
4. Microsoft. (2024, May 22). VBScript deprecation: Timelines and next steps. Windows IT Pro Blog. techcommunity.microsoft.com
5. Microsoft. (n.d.). Using startup, shutdown, logon, and logoff scripts in Group Policy (Windows Server 2012 R2). Microsoft Learn (archived). learn.microsoft.com
6. Microsoft. (n.d.). Add a custom script to Windows Setup. Microsoft Learn. learn.microsoft.com
7. cloud-init project. (n.d.). cloud-init documentation. docs.cloud-init.io; systemd project. (n.d.). systemctl manual page. freedesktop.org

The 2 a.m. page

At 2:04 a.m., the on-call administrator at a regional distribution company is woken by an alert. A server called WH-APP-02 is down, and the warehouse scanners on the night shift cannot look up orders. Over the next few hours she will need answers to a string of questions. Which physical machine is WH-APP-02? Whose is it, and is it under warranty? How do its parts come out? What else depends on it? How fast must it be back, and how much data can the business lose? Each answer is either in the documentation or it is not, and that difference sets how long the outage lasts. This lesson follows her night, one question at a time.

Which box is it? Inventory and labels

The monitoring system knows a hostname. The data center holds metal. The bridge between them is the asset inventory: one record per device, with its make, model, serial number and asset tag. Useful records add the fields that matter at 2 a.m.: room, rack and rack units, owner, hostname, IP and management addresses, purchase or lease details, warranty end date and support contract.

The Center for Internet Security puts this first among its Critical Security Controls. Control 1 asks organizations to actively manage all enterprise assets, servers included, whether physical, virtual, remote or in the cloud, so they know everything that must be monitored and protected. NIST's practice guide on IT asset management (SP 1800-5, September 2018) adds the other half. A physical record can tell you where a computer is, but not what software it runs or whether it is vulnerable, so a good inventory ties the physical and the virtual together.

The inventory says WH-APP-02 is a 2U server in rack B4, units 20 and 21, asset tag 004127. Labeling turns that record into something she can check with a flashlight. The asset tag is a durable label, often with a barcode, fixed where it stays visible in the rack. Hostname labels go on the front and the back, because servers are worked on from both sides. Pulling the wrong server turns one outage into two. Cables are labeled at both ends, as in Lesson 3. She matches the tag on the server to the record before she touches anything.

Whose box is it? Warranty, leases and the life cycle

The record shows the server is leased, not owned, with the lease ending next March. Its warranty includes four-hour onsite parts replacement. Both facts change what she does. The warranty means a replacement part can arrive before morning at no cost, if she calls the vendor instead of raiding the spares shelf. The lease means the server belongs to the lessor and must go back in an agreed condition.

Those details matter for data as well as money. NIST's media sanitization guidelines (SP 800-88 Revision 2, September 2025) note that control of storage media changes when it is returned from a lease, donated, resold or recycled. A failed drive exchanged under warranty that will not come back is out of the organization's control. So the company's policy, recorded in the inventory, is to keep failed drives from this server and sanitize them in-house. Lesson 28 covers sanitization in detail.

End of life is often set by software, not hardware. WH-APP-02 runs Windows Server 2016, and Microsoft's lifecycle page lists its extended support ending in January 2027. Under Microsoft's Fixed Lifecycle Policy, extended support includes security updates at no additional cost. After it ends, security updates require Microsoft's paid Extended Security Update program, where one is offered. So the server's real retirement date is already on the calendar. For disposal, the U.S. Environmental Protection Agency encourages electronics recyclers to be certified to one of two accredited standards, R2 or e-Stewards, which gives a team a simple test when choosing a recycler.

How does it come apart? Manuals, diagrams and baselines

The vendor's service manual for the model shows how to release the drive carrier and reseat the RAID controller, with the handling precautions she would otherwise have to guess. The team keeps the manual for every model in the inventory, matched by model number, because manuals differ between generations of the same product line. The rest of what she needs is in the team's own documents.

Baselines come in two kinds. The performance baseline from Lesson 14 records normal CPU, memory, disk and network figures, so "back to normal" can be measured rather than felt. A configuration baseline, as NIST SP 800-128 defines it, is a formally reviewed and agreed set of specifications for a system at a point in time. It can be changed only through change control, and it is the basis for future builds. That is why a rebuilt server can match the one that failed.

How fast, and how much loss? The numbers behind the plan

Her runbook opens with numbers from the company's business impact analysis (BIA). NIST's contingency planning guide (SP 800-34 Revision 1) describes the BIA as the step that links a system to the business processes it supports and works out the consequences of a disruption. It has three steps: identify the processes and how critical their recovery is, identify the resources they need, and set recovery priorities. The outputs are numbers.

MTBF is the most misread number in the room. Seagate's data sheet for its Exos X18 enterprise drives lists an MTBF of 2,500,000 hours, about 285 years, next to a five-year limited warranty. Nobody expects a drive to last 285 years. The same sheet gives the honest translation: an annualized failure rate (AFR) of 0.35 percent when running around the clock. MTBF describes a large population during the long, flat middle of its working life. The NIST/SEMATECH engineering statistics handbook models that stretch with a constant failure rate, and the MTBF is its reciprocal. For a data center with 400 such drives, 0.35 percent a year means one or two failures a year. That is why spares and RAID exist even for excellent drives.

MTTR is the number documentation changes most. With the inventory, labels, manual and runbook, her repair takes about three hours; without them, the same repair is a search. Over time, availability is roughly MTBF divided by the sum of MTBF and MTTR. A server that fails on average every 8,000 hours and takes 8 hours to restore is up about 99.9 percent of the time. Halve the recovery time and it approaches 99.95 percent.

Service level agreements (SLAs) turn these figures into promises. Inside the company, IT's SLA with the warehouse promises 99.9 percent uptime each month. Outside, vendors make their own promises, and reading one closely is instructive. Amazon's SLA for its EC2 compute service (last updated May 2022) commits to 99.99 percent monthly uptime only for workloads spread across two or more availability zones. A single instance gets 99.5 percent. When Amazon misses, the remedy is a service credit against future bills, which the customer must claim. It is not compensation for lost business.

The arithmetic is simple. A year has 8,760 hours, and 0.1 percent of that is 8.76 hours; a 30-day month has 720 hours, and 0.1 percent of that is 43.2 minutes. Her three-hour repair alone breaks that month's 99.9 percent promise. That is why uptime requirements drive design: no repair process, however well documented, gets a single server to five nines. Lesson 18's clusters exist for that reason.

Bottom line: the numbers in an SLA are only as good as the inventory, diagrams and procedures behind them. Documentation shortens MTTR, and MTTR is the part of availability an administrator controls.

Where are the documents? Availability and secure storage

One problem nearly sank the night. The current runbook lived on a wiki that ran on a virtual machine stored on the same array that had failed. She found last year's printed copy in the operations binder. It was close enough, but only just.

NIST's contingency planning guide covers both sides of this problem. For availability, copies of the plan go to recovery staff, and a copy is kept at the alternate site and with the backup media, so the plan survives the disaster it describes. NIST also suggests storing vendor contracts, SLAs, software licenses and manuals with the plan. For security, the plan contains sensitive operational and personnel information, so its distribution should be marked and controlled, with a record of who holds copies. The same goes for network diagrams, configuration records and inventories, which are a map for an attacker as much as for an administrator. Store them with access controls and encryption, never write passwords into them, and keep an offline or off-site copy of whatever recovery depends on.

After the fix: change management and updates

By 5:10 a.m. the server is back, but her work is not done until the documents match reality. The emergency repair goes into the change management system after the fact, as the company's change policy allows for emergencies: what changed, when, why, who did it and how it was tested. NIST SP 800-128 describes the normal path. Changes are formally identified, proposed, reviewed, analyzed for security impact, tested and approved before they are made, usually by a change control board.

Then come the updates. The new drive's serial number goes into the inventory, the controller's firmware version into the configuration record, and the old printed runbook is replaced. NIST recommends reviewing a contingency plan at least once a year and whenever significant changes occur, and keeping a record of changes in the plan itself. A document that is not updated after a change is worse than none, because people trust it.

Common misconceptions

• "An MTBF of 2.5 million hours means the drive lasts for centuries." It describes a population; the same drive's AFR is 0.35 percent a year.
• "RTO and RPO measure the same thing." RTO limits downtime; RPO limits data loss, measured back in time from the failure.
• "A leased server's data is the lessor's problem." Media returned from a lease leaves your control, so the data must be dealt with first.
• "An SLA guarantees uptime." It sets a target and a remedy, often a service credit that must be claimed.
• "Documentation is finished once the server is built." It is accurate only until the next change.
• "Network diagrams are harmless reference material." They are sensitive, and their distribution should be controlled.

What to remember

• Inventory every asset with make, model, serial number, asset tag, location, owner and warranty, and label the hardware to match.
• Track the life cycle from procurement through usage and end of life to disposal, and know whether each asset is leased or owned.
• Keep service manuals, infrastructure, architecture and workflow diagrams, configuration records, baselines and recovery processes.
• BIA sets the RTO (downtime) and RPO (data loss); MTBF describes failure rates, and MTTR is recovery time that good documentation shortens.
• 99.9 percent uptime allows 8.76 hours a year, or about 43 minutes a month.
• Keep documents updated through change management, available off-site and with backups, and stored securely.

Sources

1. Swanson, M., Bowen, P., Phillips, A. W., Gallup, D., and Lynes, D. (2010). Contingency planning guide for federal information systems (NIST Special Publication 800-34 Rev. 1). National Institute of Standards and Technology. csrc.nist.gov
2. Johnson, A., Dempsey, K., Ross, R., Gupta, S., and Bailey, D. (2011, updated 2019). Guide for security-focused configuration management of information systems (NIST Special Publication 800-128). National Institute of Standards and Technology. csrc.nist.gov
3. Chandramouli, R., and Hibbard, E. A. (2025). Guidelines for media sanitization (NIST Special Publication 800-88 Rev. 2). National Institute of Standards and Technology. csrc.nist.gov
4. Center for Internet Security. (n.d.). CIS Critical Security Control 1: Inventory and control of enterprise assets. cisecurity.org; Stone, M., Irrechukwu, C., Perper, H., Wynne, D., and Kauffman, L. (2018). IT asset management (NIST Special Publication 1800-5). csrc.nist.gov
5. Seagate Technology. (2020). Exos X18 data sheet. seagate.com; NIST/SEMATECH. (n.d.). e-Handbook of statistical methods, section 8.1.7.1: Homogeneous Poisson process. itl.nist.gov
6. Amazon Web Services. (2022). Amazon compute service level agreement. aws.amazon.com
7. Microsoft. (n.d.). Windows Server 2016 lifecycle and Fixed Lifecycle Policy. Microsoft Learn. Windows Server 2016; Fixed Lifecycle Policy
8. U.S. Environmental Protection Agency. (n.d.). Certified electronics recyclers. epa.gov

The problem: "reuse the existing licenses"

A company is retiring six aging Windows Server machines and moving their workloads onto one new virtualization host with two 24-core processors. The project plan has one line about licensing: "reuse the existing licenses." Three other items ride along. An engineering application is licensed to one of the old servers' network cards. Developers want to modify an open source tool and offer it to customers. And the test team plans to restore last night's production database backup onto an older test server.

Every one of those plans has a licensing trap in it. Licensing is part of a server administrator's job because the administrator decides how many cores, sockets, VMs and users a system has, and those are exactly what licenses count. This lesson works through the problem one trap at a time. Licensing terms change and the contract always governs; Microsoft itself says its licensing guide is not a legal use rights document. Treat the vendor examples here as illustrations, and check current terms before you buy.

What gets counted: licensing models

A license grants the right to use software under conditions, and the main condition is a count. Vendors count different things.

Models are often combined. Windows Server is per-core for the server software plus per-user or per-device CALs for everyone who accesses it. Microsoft's pricing page states that CALs are required for every user or device accessing a server.

Physical versus virtual

Virtualization is where counts go wrong, because one physical host can run many licensed systems. Microsoft offers two ways to license Windows Server. Licensing by physical cores covers the host: Standard edition then allows two VMs, and each additional two VMs require relicensing all the host's cores again, which Microsoft calls stacking. Datacenter allows unlimited VMs. Licensing by virtual machine, added in October 2022 for customers with subscriptions or active Software Assurance, counts each VM's virtual cores instead, with at least 8 core licenses per VM.

Movement matters too. Microsoft's virtualization guidance says licenses assigned by physical cores cannot be reassigned on a short-term basis. So a host must be licensed for the peak number of VMs it might run, including VMs that could migrate onto it. Licenses assigned by VM can move freely between servers in the same server farm. Red Hat draws the same line differently. Each RHEL subscription covers one physical node with up to two sockets, or two virtual nodes, and an unlimited-guest model is sold per physical socket-pair.

Working the numbers for the new host

The new host has 2 processors of 24 cores each: 48 physical cores, comfortably above the 16-core minimum. The team expects to run ten Windows Server VMs on it.

The cheapest choice depends on current prices, which change, so compare quotes rather than memorizing a break-even point. The method does not change: count physical cores, apply the minimums, then multiply by what the edition allows. The old licenses raise two more questions. Licenses for an older version do not cover a newer one unless they carry Software Assurance, whose new version rights allow upgrading to the most recent version. And the CALs must match too, as the next section shows.

Key idea: licenses count things the administrator controls. Every time you add cores, sockets, VMs or users, or upgrade a version, you may be changing what you owe.

Versions and compatibility

A license is for a version. Microsoft grants downgrade rights, so a Windows Server 2025 license may run an earlier version, but an older license does not stretch upward. CALs follow the same rule. Microsoft defines a user CAL as allowing access to the corresponding version of the server software or earlier versions. So when the file servers move to Windows Server 2025, the Windows Server 2019 CALs no longer cover the people using them.

Compatibility uses the same two directions. Software is backward compatible when a newer version works with older data, clients or formats. It is forward compatible when an older version can handle what a newer one produces, which is much rarer. SQL Server shows both. Microsoft's documentation says that a database restored from an earlier version is upgraded automatically, which is backward compatibility. It also says no SQL Server backup can be restored to an earlier version than the one that created it. The test team's plan fails: a backup from production's newer version will not restore on the older test server. Upgrade the test server, or move data a different way.

A license is not maintenance and support

A license is the right to use. Maintenance and support is a separate purchase: updates, new versions and someone to call. With a perpetual license, letting maintenance lapse usually leaves you the right to run what you have, without new versions or help. Microsoft's Software Assurance shows the split: its benefits include new version rights, the right to upgrade each covered license to the most recent version. NVIDIA's licensing FAQ for its legacy PGI compilers shows it inside a license key. The key carries a support service end date, and the license works with every release made on or before that date, but not with later ones.

A subscription bundles the right to use with updates for as long as you keep paying. Microsoft's Enterprise Agreement offers a subscription option in which you subscribe to the rights to use products instead of owning them. Red Hat sells RHEL entirely by subscription. Even its self-support level includes software updates and the knowledgebase, but not phone or web support; the Standard and Premium levels add support under different response-time SLAs. Red Hat states that every RHEL subscription provides 10 years of support for each major release.

Node-locked licenses and signatures

A node-locked license works only on one machine, identified by a host ID. NVIDIA's FAQ explains how this works for its FlexNet-style keys. The host ID is the network card's MAC address, and validation fails if it changes. Moving a license to another machine means requesting a new key. A floating license instead lets a license server hand seats to any machine, up to the count in the key.

That is the trap for the engineering application. Moving it into a VM gives it a new MAC address, so the old key will not validate. Request a new key for the VM, and fix the VM's MAC address in the hypervisor so a later clone or rebuild does not break it again.

Vendors protect license data with signatures, so that keys and activation data cannot be forged or edited to raise a count. Microsoft's description of Active Directory-based activation is a clear example: a domain-joined server queries the directory for an activation object, checks the digital signatures in it, and only then activates.

Open source is a license too

The Open Source Initiative's definition makes clear that open source means more than access to the code. The license must allow free redistribution and modification, and must let derived works be distributed under the same terms. Open source licenses still carry obligations, and they differ. The GNU General Public License version 3 lets you run and modify a program without conditions as long as you do not convey it. It also says that interaction with users over a network, with no copy transferred, is not conveying. Once you convey a modified version, you must license the entire work under the GPL. The GNU Affero GPL, used by some server software, closes the network gap: a modified version must offer its source to users who interact with it remotely.

So the developers can modify the tool and run it internally. Offering it to customers changes their obligations, and the license of each component they use decides how. Keep open source components and their licenses in the inventory from Lesson 21.

Volume licensing, activation and the true-up

Volume licensing programs sell many licenses under one agreement. Microsoft's Enterprise Agreement is aimed at organizations with 500 or more users or devices, for a minimum of three years. Users, devices and products added during the term are accounted for through the annual true-up, the point at which deployments are reconciled with what was bought.

Volume licenses are activated in bulk. A Key Management Service (KMS) host activates systems on the local network once it reaches an activation threshold: at least five Windows Server computers, or 25 Windows client computers. Smaller organizations that cannot meet that threshold commonly use a Multiple Activation Key (MAK), which allows a preset number of activations. Microsoft's Volume Activation Management Tool (VAMT) can track how many remain. Active Directory-based activation is the third method.

License count validation is the habit that keeps a true-up from becoming a surprise. Compare entitlements with reality: physical cores per host against core licenses, VMs per Standard host against stacked licenses, users or devices against CALs, and MAK activations used against those purchased. Do it after every major change and before every true-up or audit.

Common misconceptions

• "A 6-core server needs 6 core licenses." Windows Server licensed by physical cores needs at least 8 per processor and 16 per server.
• "Windows Server Standard covers any number of VMs." It covers two per full licensing of the host's cores; Datacenter covers unlimited VMs.
• "Open source means no obligations." Licenses such as the GPL and AGPL attach conditions when you convey or offer modified versions.
• "Stopping maintenance means losing the software." With a perpetual license you usually keep your version but lose upgrades and support.
• "Existing CALs cover a server upgrade." CALs cover their own version or earlier, not later versions.
• "A backup restores anywhere if the file is intact." SQL Server backups cannot be restored to an earlier version.

The takeaway

• Know what each license counts: cores, sockets, servers, instances, named users or devices, concurrent users or sites.
• In virtualization, license hosts for their peak VM count, or license per VM where the terms allow.
• A license is the right to use; maintenance and subscriptions buy updates, new versions and support.
• Node-locked licenses follow a host ID; signatures stop license data being forged or edited.
• Open source licenses carry obligations that depend on whether and how you convey the software.
• Validate counts before each true-up, and check version compatibility before upgrades and restores.

Sources

1. Microsoft. (n.d.). Windows Server 2025 licensing guidance and Windows Server virtualization technologies licensing guidance. Microsoft Licensing. Windows Server 2025; Virtualization technologies
2. Microsoft. (n.d.). Windows Server pricing. microsoft.com
3. Microsoft. (n.d.). Plan for volume activation and Enterprise Agreement. learn.microsoft.com; microsoft.com
4. Microsoft. (n.d.). RESTORE statements (Transact-SQL). Microsoft Learn. learn.microsoft.com
5. Red Hat. (n.d.). Red Hat Enterprise Linux subscription guide. redhat.com
6. NVIDIA. (n.d.). Legacy PGI frequently asked questions (licensing). developer.download.nvidia.com
7. Open Source Initiative. (n.d.). The Open Source Definition; GNU General Public License version 3; GNU Affero General Public License version 3. Definition; GPLv3; AGPLv3

From the street to the rack

A contractor named Dana has a ticket to replace a failed drive in a customer's server at a colocation data center. The ticket names a building, a cage and a rack. Between the street and that rack she will pass through several layers of physical security, and each layer exists because the one before it can fail. Following her in shows what each control is for. It also shows why the Server+ objectives put physical security next to encryption and access control: a person with hands on a server can pull its drives, boot it from their own media, or carry it away.

The layers here follow the pattern Microsoft describes for its own datacenters: the perimeter, the building entrance, the inside of the building and the datacenter floor. NIST's catalog of security controls, SP 800-53 Revision 5, supplies the vocabulary in its physical and environmental protection family.

Layer 1: the perimeter

The building is hard to pick out. It has no logo, no sign and no hint of what it holds. That is data center camouflage, and Microsoft describes its own datacenters as nondescript buildings. A steel fence surrounds the site, with a single gate. Bollards, short posts of steel or concrete, stand between the road and the walls. NIST lists bollards, concrete slabs, jersey walls and hydraulic vehicle barriers as physical barriers; their job is to stop a vehicle, not a person. At the gate, a camera records Dana's car, and a security guard checks her name against the day's approved visitors.

Layer 2: the building

The lobby windows are reflective glass, which looks like a mirror from outside in daylight, so nobody can watch the room or photograph a screen through it. The data hall itself has no windows. Its walls are an architectural reinforcement: solid construction, with partitions that run from the structural floor to the structural ceiling, so nobody can climb over through the space above a drop ceiling.

Some rooms add signal blocking: shielding in walls, windows and ducts that stops radio signals from passing through. NIST explains both motives. Entry points near systems raise the risk of eavesdropping with wireless sniffers, and information can leak through electromagnetic emanations. Shielding is passive, and it must stay that way in the United States. The FCC states that operating a radio jammer violates federal law, with no exceptions for commercial use.

At the desk, a guard checks Dana's government-issued ID and gives her a visitor badge marked for escorted access only. Microsoft describes the same steps for its visitors, including escort-only badges that expire automatically.

Layer 3: inside the building

Dana's escort opens the next door with a badge at a card reader. Most modern badges carry a radio frequency identification (RFID) chip that the reader detects at close range. Because every badge is electronic, the system can log each door opened and revoke a lost badge at once. The following door adds a biometric reader, which checks a fingerprint, palm or iris. A badge plus a biometric is two-factor authentication: something you have and something you are. Microsoft requires two-factor authentication with biometrics inside its datacenters, and AWS describes multi-factor checks at its server room doors, with alarms if a door is forced or held open.

The data hall entrance is an access control vestibule, sometimes called a mantrap: a small space between two interlocking doors. The second door will not open until the first has closed and the person inside has been verified. NIST describes vestibules as a defense against piggybacking or tailgating, the common trick of following an authorized person through a door. A badge reader alone cannot tell whether one person or three walked through on a single swipe. A vestibule can.

Layer 4: the data hall and the rack

Security cameras cover the aisles; Microsoft monitors the front and back of every rack. Cameras deter and record, but they do not stop anyone. NIST pairs them with intrusion alarms, such as motion sensors, door contacts and glass-break sensors, which trigger a response by guards. Alarms work together with the barriers and locks.

The customer's cage has its own lock, and so do the rack doors. NIST's guidance on locks is mostly about keys. Secure them, inventory them, and change locks or combinations when keys are lost or when people who hold them leave. Microsoft keeps physical keys in an electronic key system linked to the access system, and keys never leave the site.

Dana swaps the drive. The failed one does not leave in her pocket: the customer's policy, as in Lesson 21, is to keep failed drives, so it goes into a locked bin for sanitization. Backup tapes, master keys and sealed emergency passwords live in a safe, the innermost layer, which protects them from theft and fire.

Why this matters: every software control in this course assumes the attacker cannot touch the hardware. Physical security makes that assumption true, layer by layer, so that one failed control, such as a propped door or a borrowed badge, is caught by the next.

The environment: fire, heat, water and sensors

Not every physical threat is a person. For facilities that concentrate computing equipment, such as data centers and server rooms, NIST asks for fire detection and suppression backed by an independent energy source. It also asks for monitored environmental controls and protection from water damage.

Carbon dioxide systems need a special warning. The EPA notes that at its minimum design concentration for flooding a room, 34 percent, carbon dioxide is lethal. The NFPA standard for these systems relies on safeguards: pre-discharge alarms and time delays so people can get out, and a supervised lock-out that prevents a discharge while people unfamiliar with the system are inside. Any room with gas suppression calls for the same discipline: know where the abort switch is, and leave when the alarm sounds.

Sensors connect the environment to the monitoring from Lesson 16. Temperature sensors at rack inlets, humidity sensors, leak detection under raised floors and near cooling units, smoke detectors and door contacts all feed alerts. NIST asks for environmental monitoring that raises an alarm or notification when conditions change in ways that could harm people or equipment. The DOE's guide adds a practical warning: humidity sensors drift. If they are not recalibrated, neighboring cooling units can end up fighting each other, one humidifying while the other dehumidifies.

Common misconceptions

• "Physical security belongs to the facilities team." Anyone who can touch a server can pull its drives or boot it from other media, so administrators depend on it too.
• "A badge reader stops tailgating." One swipe can admit several people. A vestibule or a guard is what stops it.
• "A signal jammer is a good way to protect a secure room." In the United States, operating a jammer is illegal; use passive shielding.
• "Cameras prevent break-ins." They deter and record. Alarms, guards and barriers stop intruders and trigger a response.
• "Any gas suppression is safe for people." Carbon dioxide is lethal at flooding concentrations; every gas system needs alarms, delays and evacuation.
• "Tighter humidity control is always safer." The DOE says over-controlling humidity brings no real operational benefit.

What to carry forward

• Physical security works in layers: perimeter, building, interior, data hall, rack and safe.
• Perimeter: camouflage, fencing, bollards, guards and cameras. Building: reflective glass, reinforced walls and passive signal blocking.
• Entry: card readers with RFID badges, biometrics, and vestibules against tailgating, with escorts for visitors.
• Locks matter only as much as key control: inventory keys and change locks when keys go missing or people leave.
• Fire: automatic detection, pre-action sprinklers or clean agent gas, and safety alarms and delays before any discharge.
• Environment: inlet air at 18 to 27 degrees Celsius, humidity at 8 to 70 percent, leak detection, and sensors wired to alerts.

Sources

1. Microsoft. (n.d.). Datacenter physical access security. Microsoft Learn. learn.microsoft.com
2. Joint Task Force. (2020). Security and privacy controls for information systems and organizations (NIST Special Publication 800-53 Rev. 5), physical and environmental protection family (PE-3, PE-6, PE-13 to PE-15, PE-18, PE-19). National Institute of Standards and Technology. csrc.nist.gov
3. Amazon Web Services. (n.d.). Data center controls. aws.amazon.com
4. U.S. Environmental Protection Agency. (n.d.). Carbon dioxide as a fire suppressant: Examining the risks; Substitutes in total flooding agents; Halons program. CO2 risks; Total flooding agents; Halons
5. U.S. Department of Energy. (2024). Best practices guide for energy-efficient data center design. energy.gov
6. Federal Communications Commission. (n.d.). Jammer enforcement. fcc.gov

Three questions about every piece of data

Take one file server holding payroll exports, project files and ten years of old invoices. Three questions decide how to protect it. Who could reach the data, and at what moment: someone who steals a drive, someone on the network, or someone signed in to the running server? How long must each kind of data be kept, and where should the copies live? And what is each kind of data worth compared with the cost of protecting it? Encryption answers the first question, retention and storage rules the second, and business impact the third. This lesson compares the options for each, because the right choice depends on the threat.

Data at rest: what each kind of encryption protects

NIST's guide to storage encryption (SP 800-111, 2007) describes three families: full disk encryption, volume and virtual disk encryption, and file and folder encryption. Many servers add a fourth, self-encrypting drives. The table compares them.

The third column is the one people forget. NIST states it plainly: when a device is booted, full disk encryption provides no protection, because the operating system becomes responsible for the decrypted data. Red Hat's documentation says the same of LUKS: it protects data only when the system is off. A server runs around the clock, so disk encryption protects it against a stolen drive, a stolen server, a failed drive sent back under warranty and hardware that leaves at end of life. It does nothing against an attacker who signs in to the running server. NIST adds a second gap: volume and file encryption leave swap and hibernation files unprotected, and those can hold copies of data that was in memory.

Microsoft describes BitLocker in the same terms: it encrypts entire volumes to address data theft or exposure from lost, stolen or inappropriately decommissioned devices. It works best with a TPM, which lets BitLocker check that the device was not tampered with while offline. A startup PIN adds another factor, but someone must then type it at every boot. Servers that must restart unattended need another way to unlock. Red Hat's policy-based decryption can unlock LUKS volumes automatically using a TPM or a network server called Tang, so a disk unlocks only on the right hardware or the right network. LUKS also offers several key slots, so a backup key or recovery passphrase can be added and stored safely.

What matters here: "Is it encrypted?" is the wrong question. Ask which threat the encryption stops, and at which moment the data is decrypted.

Data in transit: protecting what crosses the network

SMB Encryption shows how quickly the defaults improve. Microsoft notes that SMB 3.1.1 negotiates AES-128-GCM by default, and Windows Server 2022 added AES-256 cipher suites. Encryption in transit protects the path between two systems. It does not protect the data at either end, which is why it is paired with encryption at rest and with access control.

Firmware and boot loader passwords

Encryption protects the disks, but whoever controls the firmware and the boot loader controls what runs before the operating system can defend itself. Three passwords guard that ground.

Lenovo's ThinkSystem documentation describes the first two. With a power-on password, the server cannot be used until a valid password is entered at every power-on. An administrator password blocks the Setup Utility, and if both are set, only the administrator password can change settings. Red Hat describes the third: a password on the GRUB boot menu prevents unauthorized users from modifying boot parameters. On Rocky Linux and other RHEL-family systems, the grub2-setpassword command sets one.

All three have the same limit. Lenovo documents clearing CMOS to erase lost firmware passwords, and Lesson 9 showed how easy that is with the cover off. Firmware passwords slow down a person at the console; physical security (Lesson 23) keeps that person away, and BMC security (Lesson 8) keeps the same settings safe from the network.

Retention: how long, and where

A retention policy sets how long each kind of data is kept and what happens afterward. The Federal Trade Commission's guide for businesses puts it simply. Keep only what you need. If you must keep information for business or legal reasons, write a records retention policy: what must be kept, how to secure it, how long to keep it and how to dispose of it securely. Retention has a minimum and a maximum.

• The minimum comes from law and business need. The IRS says to keep records supporting a tax return for 3 years in general, 7 years for certain loss claims, and employment tax records for at least 4 years. The HIPAA Security Rule requires covered organizations to retain their security documentation for 6 years.
• The maximum comes from risk. Data kept past its purpose can still be stolen, leaked or demanded in a lawsuit (Lesson 26), so disposal on schedule is part of the policy.
• Retention covers every copy. Backups and archives hold copies too, so their rotation and expiry must match the policy.

The physical location of storage is the other half. On-site copies restore fastest. Off-site copies survive the fire, flood or theft that takes out the main site. NIST's contingency planning guide calls off-site storage of backups good business practice, and lists what to weigh when choosing a facility.

Business impact: value, life cycle and cost

Data value prioritization comes first, because a team cannot protect everything equally. NIST's FIPS 199 rates the potential impact of losing confidentiality, integrity or availability as low (a limited adverse effect), moderate (serious) or high (severe or catastrophic). On the example file server, the payroll exports are high for confidentiality, the project files are moderate, and the old invoices matter mostly for integrity, until their retention period ends.

Life-cycle management means the protection follows the data through every stage: creation, use, storage, backup, archive and destruction. A record protected on the server but copied unencrypted to a backup tape is only as safe as the tape.

The cost of security should be weighed against the risk and the cost of replacement. A rough method: multiply how often a loss is expected per year by what one loss would cost, and compare that with the control's yearly cost. Suppose a lost, unencrypted backup tape is expected about once in 50 years and would cost 500,000 dollars in notification and recovery. The expected loss is about 10,000 dollars a year, while turning on the backup software's encryption costs almost nothing, so it is an easy decision. A 20,000-dollar yearly control protecting a test server that holds no sensitive data and could be rebuilt for 3,000 dollars fails the same test. Hardware is usually the cheapest thing to replace; the data and the downtime are what cost money.

Common misconceptions

• "Disk encryption protects a running server from attackers." It protects data only while the system is off or the volume is locked.
• "Encrypting a folder protects every copy of its data." Swap and hibernation files can still hold unencrypted copies.
• "A power-on password is good practice for servers." It stalls unattended restarts. Use an administrator password plus physical security.
• "Firmware passwords cannot be bypassed." Clearing CMOS erases them, which is why physical access must be controlled.
• "Keeping data forever is the safe choice." The FTC advises keeping only what you need and disposing of the rest securely.
• "Any off-site location will do." NIST also weighs distance from shared disasters, retrieval time, security, environment and cost.

Recap

• At rest: full disk, volume, file and SED encryption protect data until it is unlocked, so they defend against theft and disposal, not a running server's attacker.
• Servers that restart unattended need TPM-based or network-bound unlocking, plus recovery keys stored safely.
• In transit: TLS 1.2 and 1.3 for web traffic, SSH for administration and SMB Encryption for file shares.
• Set a firmware administrator password and a GRUB password; avoid power-on passwords on servers; remember that CMOS clearing defeats them.
• Retention policies set minimums (law) and maximums (risk) for every copy, including backups; store copies off-site using NIST's criteria.
• Rank data by impact, protect it through its whole life cycle, and spend in proportion to the risk.

Sources

1. Scarfone, K., Souppaya, M., and Sexton, M. (2007). Guide to storage encryption technologies for end user devices (NIST Special Publication 800-111). National Institute of Standards and Technology. csrc.nist.gov
2. Red Hat. (n.d.). Security hardening, Red Hat Enterprise Linux 9: chapters 10 (LUKS) and 11 (policy-based decryption). LUKS; Automated unlocking
3. Microsoft. (n.d.). BitLocker overview and SMB security enhancements. Microsoft Learn. BitLocker; SMB security
4. McKay, K. A., and Cooper, D. A. (2019). Guidelines for the selection, configuration, and use of Transport Layer Security (TLS) implementations (NIST Special Publication 800-52 Rev. 2). csrc.nist.gov
5. Lenovo. (n.d.). ThinkSystem ST45 V3: Firmware (BIOS) passwords. pubs.lenovo.com; Red Hat. (n.d.). Managing, monitoring, and updating the kernel: Securing the boot menu with a password. docs.redhat.com
6. Federal Trade Commission. (n.d.). Protecting personal information: A guide for business. ftc.gov; Internal Revenue Service. (n.d.). How long should I keep records? irs.gov; 45 C.F.R. section 164.316. ecfr.gov
7. Swanson, M., Bowen, P., Phillips, A. W., Gallup, D., and Lynes, D. (2010). Contingency planning guide for federal information systems (NIST Special Publication 800-34 Rev. 1), section 3.4.2. csrc.nist.gov
8. National Institute of Standards and Technology. (2004). Standards for security categorization of federal information and information systems (FIPS 199). csrc.nist.gov

A new server, and everyone who wants into it

A database server goes live next week. Five kinds of people and programs need access to it: database administrators, the application that stores its data there, a backup operator, the help desk and the auditors. Identity and access management (IAM) is the discipline of giving each of them exactly the access they need, proving who they are when they use it, and keeping a record of what they did. This lesson sets up IAM for that server as a procedure in six steps, with Windows and Linux examples at each step. Do the hands-on parts in your lab VMs.

Step 1: One account per person, plus admin and service accounts

Every person gets their own user account. Shared accounts make auditing useless, because a log entry that says "admin did it" names nobody. People with administrative duties get a second, separate account for that work. NIST's least privilege control (SP 800-53, AC-6) asks that users with privileged access use non-privileged accounts for everything else, so that reading email or browsing the web never happens with administrator rights.

Programs get service accounts, each with only the rights its service needs. Their passwords are a classic weak spot, because nobody wants to change a password that might break the application. Windows Server's group Managed Service Accounts (gMSAs) solve this: Microsoft explains that Windows manages the password itself, and one gMSA can serve the same application across several servers, such as a load-balanced farm. Accounts nobody uses should be disabled, because an unused account is one nobody notices being misused.

Step 2: Grant permissions to groups, not people

Create user groups that match the jobs, such as DB-Admins, DB-ReadOnly and Backup-Operators. Grant permissions to the groups, then add and remove people. When someone changes jobs, one group membership changes instead of dozens of individual permissions. Red Hat's sudo rules work the same way: entries in /etc/sudoers or in drop-in files under /etc/sudoers.d can apply to groups as well as individual users. Group changes are also easy to audit, as Step 6 shows.

Step 3: Set the password policy: length, lockout and enforcement

NIST's current guidance on passwords, SP 800-63B-4 (July 2025), overturns several old habits.

Lockout is a trade-off. Microsoft's guidance says Windows security baselines recommend a threshold of 10 invalid sign-in attempts as a starting point. That prevents accidental lockouts and limits guessing, but it does not prevent a denial-of-service attack: anyone who knows a username can lock that account by guessing wrong on purpose. A very low threshold makes that attack easier.

Enforcement is where policy meets software. On Windows, password and lockout policies are account policies set through Group Policy. One detail matters for NIST's 15 characters: Windows limits the minimum password length to 14 unless the "Relax minimum password length limits" setting is enabled, which raises the limit to 128. Microsoft advises auditing first, because some older software cannot handle passwords longer than 14 characters. On Linux, the PAM module pam_faillock keeps a record of failed attempts for each user and locks the account after too many consecutive failures, with settings in /etc/security/faillock.conf.

Step 4: Add multifactor authentication and single sign-on

Multifactor authentication (MFA) combines different kinds of factors. Something you know is a password or PIN. Something you have is a smart card, a security key or an authenticator app on a phone. Something you are is a fingerprint or other biometric. Two passwords are still one factor. NIST adds two refinements. A biometric is not an authenticator by itself; it must be paired with a physical authenticator. And passwords are not phishing-resistant, so NIST requires federal staff and contractors to use phishing-resistant authentication. For server administrators, whose accounts are the most valuable to an attacker, MFA is the single most useful addition.

Single sign-on (SSO) means signing in once and reusing that proof across many services. In an Active Directory domain, Kerberos provides it: Microsoft explains that a client obtains credentials for a server once and reuses them throughout the logon session. SSO means fewer passwords to steal and type. It also means one compromised sign-in opens many doors, which is why SSO and MFA belong together.

Remember: authentication proves who someone is; authorization decides what they may do. Steps 1 to 4 handle the first. Step 5 handles the second.

Step 5: Decide who gets what: role, rule and scope

Three principles shape every grant. Least privilege, NIST control AC-6, allows only the access needed for assigned tasks. Segregation of duties, NIST control AC-5, splits sensitive work so that no single person can abuse it without collusion. NIST's own example fits servers well: the people who administer access control should not also administer the audit logs. Otherwise one person could grant themselves access and then erase the record.

Delegation hands out specific administrative tasks without full administrator rights. Red Hat's sudo lets an administrator grant non-root users the right to run particular administrative commands without the root password. On Windows, Microsoft's gMSA documentation describes domain administrators delegating service management to service administrators. The backup operator, for example, gets the right to run backups, not to administer the server.

Step 6: Audit logins, activity, group changes and deletions

Auditing turns access into evidence. The objectives name four things to record, and both operating systems can record them.

Sudo adds its own trail: Red Hat notes that when a user runs a command they are not authorized for, the journal records that the user is not in sudoers. Red Hat also notes that Audit adds no protection by itself; its value is in discovering policy violations. So someone must review the records, or feed them to the log analysis tools in Lesson 26. A sudden addition to the DB-Admins group at 2 a.m. is worth an alert.

Common misconceptions

• "Complex passwords that change every 90 days are best practice." NIST now favors length and blocklists, and forbids periodic changes unless there is evidence of compromise.
• "A lower lockout threshold is always safer." It also makes it easier for anyone to lock out real users on purpose.
• "A password plus a security question is multifactor." Both are something you know: one factor, twice.
• "Administrators should use one account for everything." NIST asks for non-privileged accounts for everyday work.
• "Single sign-on weakens security." It reduces password sprawl; pair it with MFA so one stolen sign-in is not enough.
• "Turning on auditing secures the server." Audit records only help when someone or something reviews them.

What you now know

• Individual accounts for people, separate admin accounts for privileged work, and managed service accounts for programs.
• Permissions go to groups; people move between groups.
• NIST 800-63B-4: at least 15 characters for single-factor passwords, no composition rules or forced expiry, blocklists, and capped failed attempts. Windows needs the relaxed length setting for more than 14.
• MFA combines know, have and are; SSO signs in once and must be protected by MFA.
• Role-based, rule-based and scope-based access, applied with least privilege, segregation of duties and delegation.
• Audit logins, activity, group changes and deletions (events 4624, 4625, 4728, 4732, 4756, 4720 and 4726; Linux Audit and sudo logs), and review what you collect.

Sources

1. Temoshok, D., Fenton, J., Choong, Y.-Y., Lefkovitz, N., Regenscheid, A., Galluzzo, R., and Richer, J. (2025). Digital identity guidelines: Authentication and authenticator management (NIST Special Publication 800-63B-4). National Institute of Standards and Technology. csrc.nist.gov
2. Joint Task Force. (2020). Security and privacy controls for information systems and organizations (NIST Special Publication 800-53 Rev. 5), controls AC-5 and AC-6. csrc.nist.gov
3. Microsoft. (n.d.). Account lockout threshold; Minimum password length auditing and enforcement on certain versions of Windows. Lockout threshold; Password length
4. Microsoft. (n.d.). Group Managed Service Accounts overview; Understand scope for Azure RBAC; What is Conditional Access?; Kerberos authentication overview. gMSA; Azure RBAC scope; Conditional Access; Kerberos
5. Microsoft. (n.d.). Security auditing events 4624, 4625, 4720, 4726 and 4732, and Audit Security Group Management. 4624; 4625; 4720; 4726; Group management
6. Red Hat. (n.d.). Configuring basic system settings, RHEL 9: Managing sudo access; Security hardening, RHEL 9: Auditing the system. sudo; Audit; Linux-PAM project. (n.d.). pam_faillock(8) manual page. man7.org

An argument in the risk meeting

Last week a contractor with legitimate access copied the customer database to a personal cloud drive so that he could work from home. Nothing seems to have leaked, but the monthly risk meeting turns into an argument with three disputes.

The security lead says attackers are the biggest risk and wants data loss prevention and a SIEM. The operations lead says the company loses more data to dead disks and corrupted files than to hackers, and wants the money spent on backups. Then legal counsel mentions that a former supplier is threatening to sue, and says to keep everything. The privacy officer points out that the retention policy says customer data must be deleted after three years. Finally, the chief executive asks whether anyone needs to be told about the copied database, and counsel warns that a legal clock may already be running.

Each side has part of the truth. This lesson settles the three disputes with the evidence. One note before starting: what follows is general information about U.S. rules, not legal advice. When data, lawsuits or breaches are involved, administrators work with the organization's lawyers, and the lawyers decide.

Dispute 1: attackers or accidents?

Both leads are right, because the risks to server data are broader than either one's list.

The contractor's copy fits CISA's definition exactly. CISA defines an insider threat as the potential for an insider to use authorized access or understanding of an organization to harm it, and counts malicious, complacent and unintentional acts. Good intentions do not make the copy safe. Malware deserves its own note. CISA's ransomware guide warns that many ransomware variants search for accessible backups and delete or encrypt them. That is why it calls for offline, encrypted backups that are tested regularly. Outside attackers are changing tactics too. Verizon's 2026 Data Breach Investigations Report says software vulnerabilities have overtaken stolen passwords as the top way attackers get in.

The resolution is the method from Lesson 24: rank risks by likelihood and impact, and fund controls in proportion. Most organizations need both the backups and the monitoring.

The mitigations, defined

Data loss prevention (DLP) watches for sensitive data leaving where it belongs. Microsoft describes its Purview DLP policies as identifying, monitoring and automatically protecting sensitive items. Those policies can cover on-premises file shares and endpoint devices as well as cloud services, so a DLP policy could have flagged or blocked the contractor's upload. DLP covers only the locations and channels its policies target, so it complements access control rather than replacing it.

Data monitoring and log analysis find what prevention missed. NIST's glossary defines a security information and event management (SIEM) tool as an application that gathers security data from system components. It presents that data as actionable information through a single interface. NIST's log management guide (SP 800-92) describes SIEM servers receiving, normalizing and analyzing logs from many hosts. Correlation is the point. A contractor's sign-in at 2 a.m., followed by a 40 GB upload to an unfamiliar cloud service, is two dull log lines on two systems, but one alert in a SIEM.

Three controls take the decision away from any single person.

• Two-person integrity requires at least two authorized people to be present, each able to detect incorrect or unauthorized procedures. NIST's glossary takes the term from cryptographic key handling. On servers it fits rare, dangerous actions, such as exporting a full customer database.
• Split encryption keys apply the same idea to secrets. NIST's term is split knowledge: a key is divided into shares, none of which reveals anything about the key, and a set number of shares rebuilds it. A backup master key split three ways, with any two shares able to rebuild it, survives the loss of one officer and resists the misuse of one.
• Separation of roles, NIST's separation of duties from Lesson 25, keeps conflicting duties apart. Developers do not deploy their own code to production, and access administrators do not manage the audit logs.

In short: no single control covers every risk. For each risk, pick something that prevents it, something that detects it and something that recovers from it.

Dispute 2: keep everything, or delete on schedule?

Counsel's instinct comes from two federal rules of civil procedure. Rule 45 lets a subpoena command a person to produce documents, electronically stored information or tangible things in their possession, custody or control. Rule 37(e) addresses electronically stored information that should have been preserved in the anticipation or conduct of litigation. If it is lost because a party failed to take reasonable steps to preserve it, the court can impose measures. Deleting the supplier's emails on schedule after a lawsuit is expected could therefore cause serious trouble, even though the retention policy says to.

The privacy officer is also right. Lesson 24's retention policy, and the FTC's advice to keep only what you need, still apply to everything else. The resolution is a litigation hold, also called a legal hold, scoped by counsel to the relevant data: the supplier contracts, emails with the supplier and related records. Administrators carry it out by excluding that data from deletion and rotation jobs, preserving any backups that hold the only copies, and documenting what was preserved and when. Scheduled deletion continues for everything outside the hold.

Regulatory constraints: whose data, and which rules

Dispute 3: when must someone be told?

Breach identification comes first: monitoring raises the alarm, then investigation establishes what data was involved, whose it was and how many people are affected. That work starts legal clocks, so it cannot wait for certainty.

• States: the National Conference of State Legislatures reports that all 50 states, the District of Columbia, Guam, Puerto Rico and the Virgin Islands require businesses to notify individuals of breaches involving personally identifiable information.
• Health data: HHS requires notice to affected individuals without unreasonable delay and no later than 60 days after discovery of a breach of unsecured protected health information. A breach affecting more than 500 residents of a state or jurisdiction also requires notice to prominent media there.
• Public companies: since the SEC's July 2023 rules, a cybersecurity incident that a company determines to be material must generally be disclosed on Form 8-K, Item 1.05, within four business days of that determination.

So the chief executive's plan to wait until everyone is sure does not hold. Administrators have three jobs from the first alert. Preserve evidence such as logs and disk images, keep a written timeline, and escalate at once to the incident response team and counsel, who decide on disclosure.

Common misconceptions

• "Most data loss comes from hackers." Hardware failure, corruption and insiders, including honest mistakes, cause it too.
• "It is not an insider threat if the person meant well." CISA's definition includes complacent and unintentional acts.
• "A litigation hold means keeping everything forever." It covers the relevant data, for as long as counsel requires.
• "We can wait to disclose until we know everything." HIPAA counts from discovery, and the SEC from the materiality determination.
• "PCI DSS is a federal law." It is an industry standard enforced through the payment card industry.
• "Two-person integrity means two admins share one password." It requires two authorized people, each accountable and able to spot an improper action.

The short version

• Risks to data include hardware failure, corruption, malware, insiders (deliberate or careless), theft by duplication or publication, backdoors and social engineering.
• DLP blocks sensitive data leaving; monitoring, log analysis and SIEM correlate events into alerts.
• Two-person integrity, split keys and separation of roles stop any one person acting alone.
• Subpoenas can demand electronically stored information, and data relevant to expected litigation must be preserved: use a scoped litigation hold, not "keep everything".
• PII, health information and card data carry their own rules: state breach laws, HIPAA, SEC disclosure and PCI DSS.
• From the first alert, preserve evidence, record a timeline and escalate, because disclosure deadlines start early.

Sources

1. Cybersecurity and Infrastructure Security Agency. (n.d.). Defining insider threats; #StopRansomware guide. Insider threats; Ransomware guide
2. National Institute of Standards and Technology. (n.d.). CSRC glossary: backdoor; two-person integrity; split knowledge; security information and event management tool; personally identifiable information. Backdoor; Two-person integrity; Split knowledge; SIEM; PII
3. Kent, K., and Souppaya, M. (2006). Guide to computer security log management (NIST Special Publication 800-92). csrc.nist.gov
4. Microsoft. (n.d.). Learn about data loss prevention. Microsoft Learn. learn.microsoft.com
5. Legal Information Institute. (n.d.). Federal Rules of Civil Procedure, Rule 37 and Rule 45. Cornell Law School. Rule 37; Rule 45
6. U.S. Department of Health and Human Services. (n.d.). Breach notification rule; Summary of the HIPAA Privacy Rule. Breach notification; Privacy Rule summary
7. U.S. Securities and Exchange Commission. (2023, July 26). SEC adopts rules on cybersecurity risk management, strategy, governance, and incident disclosure by public companies (Press release 2023-139). sec.gov
8. National Conference of State Legislatures. (n.d.). Security breach notification laws. ncsl.org; PCI Security Standards Council. (n.d.). PCI DSS. pcisecuritystandards.org
9. Verizon. (2026). Data breach investigations report. verizon.com

A server that failed its review

A new server, FS-WEB-01, was built to host the company's intranet site and a file share. Before it could go into production, the security team reviewed it and returned eight findings. This lesson debugs them one at a time. For each, it asks what is wrong, why it matters and what fixes it. The findings fall into layers: the operating system, the applications, host security, the hardware and the patching process. NIST's Guide to General Server Security (SP 800-123, 2008) is the main reference, because its advice has not changed: a server should run only what its job requires, and be patched before anyone can reach it.

Finding 1: too many open ports

The review found ports 21 (FTP), 23 (Telnet), 80, 443, 445 (SMB) and 3389 (Remote Desktop) all reachable from the user network. Start by listing what is actually listening. On Linux, ss -tulpn shows listening TCP and UDP ports with the process behind each. On Windows, Get-NetTCPConnection -State Listen does the same in PowerShell, and its OwningProcess property names the process.

The server's job needs 443 for the intranet and 445 for the file share, from the user network only. Remote Desktop is needed only from the administrators' network, and nothing needs FTP, Telnet or plain HTTP. The fix has two parts. First, stop or remove the services behind the unneeded ports (Finding 2). Then configure the host firewall to allow only what remains, from only the networks that need it: Windows Defender Firewall rules on Windows, or firewalld zones on Rocky Linux (Lesson 13). A firewall is the second line, not the first. A rule can be changed by mistake, but a service that is not installed cannot answer.

Finding 2: services the server does not need

Telnet and FTP servers were running, and so was Print Spooler on a server that never prints. NIST says to remove all services, applications and network protocols that are not required, and to disable whatever cannot be removed. It prefers removal, because an attacker who changes settings cannot reactivate a component that is gone, and a disabled service can also be re-enabled by human error. NIST gives three more reasons. Every added service is another avenue of attack. Fewer components mean fewer defects. And fewer services mean fewer log entries, so unexpected behavior is easier to spot.

Removing services takes judgment. Microsoft's guidance on Windows Server system services rates each one. For Print Spooler, the recommendation is that it is OK to disable it if the server is not a print server or a domain controller, because domain controllers use it to prune stale printer objects from Active Directory. Check the vendor's guidance before removing a service you do not recognize.

Finding 3: software nobody asked for

The build included a web browser, a compiler, sample web applications and a set of diagnostic tools. Install only the software the server's role requires. NIST specifically names sample content as something to remove from server applications, because sample pages and scripts are well known to attackers. The installation option matters too. Microsoft describes Server Core as a minimal installation with a smaller disk footprint and a smaller attack surface, because there is less code. Lesson 10 showed how to administer it remotely.

Finding 4: missing updates, for the OS and the drivers

The operating system was three months behind on updates, and the network card driver was two years old. Both count: the objectives list driver updates alongside OS updates, because drivers run with the highest privileges. Apply OS updates through the operating system's update channel, and driver and firmware updates through the server vendor's update packages (Lesson 9).

The finding also revealed a process failure. The server had been built on the production network and left there while unpatched. NIST says a new server should stay disconnected, or on an isolated build network or a tightly restricted VLAN, until all patches and configuration steps are done.

Finding 5: an application with its defaults on

The web server role was installed with every optional feature, including directory browsing, which lists a folder's files to anyone who asks, and WebDAV, which nobody uses here. Application hardening repeats the OS steps at the next layer up. Install the application's latest patches, and remove or disable the roles, features and services the application does not use. The same goes for a database server's sample databases or a file server's unused protocols.

Finding 6: host security turned off

Microsoft Defender Antivirus had been disabled "for performance", and nothing watched the system files. Microsoft states that Defender Antivirus is installed and functional by default on Windows Server 2016 and later. If an organization uses another product, Defender can be set to passive mode instead of being switched off. NIST counts antivirus and broader anti-malware software as commonly needed extra controls on servers.

NIST also names host-based intrusion detection and prevention software, the HIDS/HIPS of the objectives. A HIDS watches a single host and alerts on suspicious activity; a HIPS can also block it. One form NIST mentions is file integrity checking, which spots changes to critical system files. On Rocky Linux, AIDE (Advanced Intrusion Detection Environment) builds a database of files and checks the system against it. NIST's IDPS guide (SP 800-94) treats host-based IDPS as one of four classes, alongside network-based, wireless and network behavior analysis.

Finding 7: hardware left wide open

The firmware allowed booting from USB before the system disk, had no administrator password, and left two unused onboard network ports and a serial port enabled. Hardware hardening closes these gaps.

• Disable unneeded hardware, such as unused NICs and controllers, in the firmware setup.
• Disable unneeded physical ports, devices and functions, such as front USB ports and serial ports, where the server's job allows it.
• Set the boot order to the system disk only, so nobody can start the server from their own media.
• Set a firmware (BIOS/UEFI) administrator password, as Lesson 24 described, so these settings stay changed.

A locked data center does not make these steps optional. Lessons 23 and 26 showed that contractors and other insiders are inside the locks.

Finding 8: patching with no process

The team's practice was to patch "when someone remembers". NIST's patch management guide (SP 800-40 Revision 4, 2022) treats patching as preventive maintenance, a cost of doing business, and defines enterprise patch management as identifying, prioritizing, acquiring, installing and verifying patches. The objectives break it into three stages.

Not every patch is equally urgent. CISA maintains the Known Exploited Vulnerabilities (KEV) catalog, the authoritative list of vulnerabilities exploited in the wild, and urges organizations to use it to prioritize. For federal agencies, CISA's Binding Operational Directive 26-04 replaced its earlier KEV directive in June 2026. It sets remediation urgency by whether a vulnerability is known to be exploited, how automated the exploitation is, its technical impact, and whether the asset is exposed to the internet. Those are useful questions for any patch queue.

The upshot: almost every finding was something present that did not need to be there, such as a port, a service, a program, a feature or a boot device. Hardening is mostly subtraction, followed by the discipline to keep the server that way.

Common misconceptions

• "Disabling a service is as good as removing it." NIST prefers removal, because disabled services can be re-enabled by an attacker or by mistake.
• "The firewall makes unneeded services safe." Firewall rules change; remove the service, then filter what is left.
• "Production servers should install patches automatically as soon as they appear." NIST advises testing on a matching server first, then deploying under change management.
• "Antivirus slows servers, so turn it off." Configure it, or run Defender in passive mode alongside another product, rather than leaving the host unprotected.
• "Hardening is done once, at build time." Patches, changes and drift mean it must be checked against the baseline again and again.
• "USB ports do not matter in a locked room." Insiders and contractors work inside the locks.

Putting it together

• OS: remove unused services, close unneeded ports, install only required software, apply OS and driver updates, and configure the host firewall.
• Applications: install the latest patches; remove unneeded roles, features, services and sample content.
• Host security: antivirus and anti-malware on, plus HIDS/HIPS such as file integrity checking with AIDE.
• Hardware: disable unused devices and ports, boot only from the system disk, and set a firmware administrator password.
• Build new servers on an isolated network until they are patched and configured.
• Patching: test, deploy in waves under change management, and prioritize known exploited vulnerabilities.

Sources

1. Scarfone, K., Jansen, W., and Tracy, M. (2008). Guide to general server security (NIST Special Publication 800-123). National Institute of Standards and Technology. csrc.nist.gov
2. Souppaya, M., and Scarfone, K. (2022). Guide to enterprise patch management planning: Preventive maintenance for technology (NIST Special Publication 800-40 Rev. 4). csrc.nist.gov
3. Scarfone, K., and Mell, P. (2007). Guide to intrusion detection and prevention systems (IDPS) (NIST Special Publication 800-94). csrc.nist.gov
4. Cybersecurity and Infrastructure Security Agency. (n.d.). Known exploited vulnerabilities catalog; (2026). BOD 26-04: Prioritizing security updates based on risk. KEV catalog; BOD 26-04
5. Microsoft. (n.d.). Guidance on disabling system services on Windows Server with Desktop Experience; What is the Server Core installation option in Windows Server?; Microsoft Defender Antivirus on Windows Server. System services; Server Core; Defender on Windows Server
6. Red Hat. (n.d.). Security hardening, RHEL 9: Checking integrity with AIDE. docs.redhat.com; Linux man-pages project. (n.d.). ss(8). man7.org

The last job a server needs

FS-OLD-03 has served the finance department for seven years. Its replacement is running, and the old server is due to leave next month. Decommissioning sounds like unplugging, but it is a procedure with real risks. Switch off a server something still depends on, and an outage follows. Ship drives that still hold data, and a breach follows. Leave live cables and switch ports behind, and the next person in the rack inherits a mess and a security hole. This lesson walks through decommissioning in seven steps. NIST's Guidelines for Media Sanitization (SP 800-88 Revision 2, September 2025) is the main reference for the data steps.

Step 1: Start from policy and a change request

Company policies set the rules: who approves a decommission, which sanitization method each category of data needs, who may perform and verify it, and where hardware may go afterward. The work itself is a change (Lesson 21), so open a change request that names the server, its owner, the planned dates, the rollback plan and the approvers. Rollback matters, because Step 2 may prove someone still needs the server.

Step 2: Verify non-utilization

Never take anyone's word that a server is unused; check the evidence.

• Connections: run ss -tulpn and ss -tnp on Linux, or Get-NetTCPConnection on Windows, at several times of day to see who is still connecting.
• Sign-ins: look for recent logon events (4624 on Windows, from Lesson 25), or run last on Linux.
• Scheduled work: check cron jobs and systemd timers, or Windows scheduled tasks. A month-end job may run only once a month.
• References elsewhere: search DNS records, load balancer pools, firewall rules, backup jobs and the architecture diagrams (Lesson 21) for the server's name and addresses.
• People: confirm with the application owners named in the inventory.

Then use a quiet period: shut the server down, but leave it intact and racked for an agreed time, such as one full monthly cycle. If something breaks, power it back on and investigate. Only when the quiet period passes without complaint does the data work begin.

Step 3: Meet media retention requirements first

Sanitization is irreversible, so check retention before anything is wiped. Lesson 24's retention schedule may require keeping some of the data, such as seven years of invoices, and Lesson 26's litigation holds may freeze other data entirely. Archive retained data to its retention storage, preserve held data as counsel directs, and confirm what happens to the backups that cover this server. Record what was kept, where and why. NIST notes that organizations should involve their records retention officers so that sanitization does not break record-keeping rules.

Step 4: Remove it from service and from the records

Take the server out of everything that knows about it: DNS, monitoring, backup jobs, load balancers, the directory (its computer account), certificate inventories and licensing, where freed licenses can be reassigned (Lesson 22). Update the asset inventory with its new status, and record each action in the change request. Documentation is what proves later that the server and its data left properly.

Step 5: Cable remediation

Unplugging is not the same as remediating. Power: remove the power cords, free the PDU outlets and update the power budget so the capacity can be reused safely. Networking: remove the cables, then disable the switch ports, clear their VLAN assignments and descriptions, and update patch panel labels and network diagrams. A live port in a production VLAN with nothing attached is an invitation to plug in an unauthorized device. A stale cable labeled with a server that no longer exists will mislead the next person troubleshooting the rack.

Step 6: Sanitize the media, then verify and document it

NIST defines three sanitization methods. Clear uses the normal read and write commands, such as overwriting all user-addressable storage, and leaves the media usable. Purge makes recovery infeasible even with state-of-the-art laboratory techniques, while keeping the media potentially reusable; NIST says purge should be used instead of clear when possible. Destroy makes recovery infeasible and the media unusable. Which to use depends on the data's confidentiality category (FIPS 199, Lesson 24) and where the media is going.

Disk wiping deserves a correction. Multi-pass patterns such as the old DoD 5220.22-M overwrite are habits from the past. NIST warns that on SSDs with overprovisioning they give very little confidentiality protection, and points instead to dedicated, standardized sanitize commands described in the IEEE 2883 standard.

Physical destruction has its own vocabulary. NIST lists disintegrating, incinerating (burning to ash), melting, pulverizing and shredding. It warns that as data density rises, pulverizing and shredding storage devices should be avoided for anything but the lowest categories of data. Bending, cutting or drilling a hole may only partly damage a device, leaving portions readable in a laboratory. Crushing, named in the objectives, raises the same question: destruction counts only if recovery is infeasible.

After sanitizing, verify: check the tool's completion status and errors for clear and purge, or inspect the remnants after destruction. Then document each device on a certificate of sanitization. NIST lists the fields, including manufacturer, model, serial number, property number, media type, method, technique, tool and version, verification method and who performed the work.

Worth holding on to: match the technique to the media type, and the strength to the data and its destination. Then prove it with a record for every device.

Step 7: Repurpose or recycle

NIST frames this as a key sanitization decision: whether the media will be reused, inside or outside the organization. Repurposing, or internal reuse, keeps working hardware in service, such as moving the old server to a test lab after sanitization. Electronics recycling has two routes. Internal recycling uses the organization's own IT asset disposal team. External recycling hands hardware to a vendor, and NIST treats media sent to a recycler as leaving organizational control, so sanitize it first. Choose a recycler certified to one of the two accredited standards the EPA names, R2 or e-Stewards (Lesson 21). If media will not be reused at all, NIST notes that destroying it can be the simplest and most cost-effective method.

The purposes of media destruction pull these threads together. Destruction is used when media leaves your control with sensitive data, when it is too damaged or obsolete to sanitize any other way, and when laws or contracts require it. The FTC's Disposal Rule, for example, requires any business that uses consumer reports to take reasonable measures to dispose of information derived from them.

Common misconceptions

• "Deleting the files or reformatting the drive is enough." Those leave data recoverable; use clear, purge or destroy as NIST defines them.
• "More overwrite passes are always safer." On SSDs, overwriting cannot reach all cells; use the device's sanitize commands.
• "A degausser works on any drive." Never on flash, and often not on modern high-coercivity hard drives.
• "A hole drilled through the platters destroys the data." NIST says it may only partly damage the device.
• "If nobody complains, the server is unused." Check connections, logons, jobs and references, then hold a quiet period.
• "Unplugging the cables finishes the job." Disable the switch ports and update the records too.

Where this leaves us

• Decommission under policy and a change request, with a rollback plan.
• Verify non-utilization from evidence, then hold a quiet period with the server off but intact.
• Satisfy retention schedules and litigation holds before any sanitization.
• Remove the server from DNS, monitoring, backups and the directory, update the inventory, and remediate power and network cabling.
• Clear, purge or destroy according to media type, data category and destination; verify, and complete a certificate for every device.
• Repurpose internally or recycle externally through a certified recycler, after sanitization.

Sources

1. Chandramouli, R., and Hibbard, E. A. (2025). Guidelines for media sanitization (NIST Special Publication 800-88 Rev. 2). National Institute of Standards and Technology. csrc.nist.gov
2. U.S. Environmental Protection Agency. (n.d.). Certified electronics recyclers. epa.gov
3. Federal Trade Commission. (n.d.). Disposing of consumer report information? Rule tells how. ftc.gov
4. Linux man-pages project. (n.d.). ss(8) and cryptsetup-luksErase(8). ss; luksErase
5. Microsoft. (n.d.). 4624(S): An account was successfully logged on; Get-NetTCPConnection. 4624; Get-NetTCPConnection

The problem: Thursday, 10 a.m.

Ransomware has encrypted the data volume of a file server. The last full backup ran on Sunday night, and some kind of backup ran every night since. The questions arrive at once. Which backup sets are needed to get back to Wednesday night? Where are they, and can the equipment still read them? Should the data go back onto the infected server? And will the restore actually work? Every answer depends on choices made weeks or years earlier. This lesson works backward from that Thursday to the choices that decide it.

Backup methods: what each one copies

NIST's contingency planning guide (SP 800-34 Revision 1) describes the three classic methods, and backup software adds the others.

Back to Thursday. With incrementals, the restore needs Sunday's full plus Monday's, Tuesday's and Wednesday's incrementals, and one unreadable tape breaks the chain. With differentials, it needs only Sunday's full and Wednesday's differential, but each nightly differential grows larger through the week. NIST's example combines the two strengths: a full backup at the weekend with differentials each evening. A synthetic full cuts the chain short. Veeam's documentation describes it as identical in content to a regular full, but synthesized from backup files already in the repository, so production storage is not read again.

Open file backups solve a real problem. Microsoft explains that data usually has to be backed up while applications are running, so files may be open or inconsistent. The Volume Shadow Copy Service (VSS) coordinates applications, backup software and storage to create a consistent shadow copy, also called a snapshot or point-in-time copy, without taking the applications offline.

Snapshots need a warning label. Microsoft notes that a Hyper-V standard checkpoint is not a full backup, and that it can cause consistency problems for systems that replicate data between nodes, such as Active Directory. Production checkpoints use VSS, or a file system freeze on Linux guests, to be data-consistent. Even then, a snapshot stored on the same array fails with that array. A snapshot becomes a backup only when it is copied somewhere else; Microsoft notes that exported checkpoints can be used for backup.

Frequency and rotation

Backup frequency follows from the recovery point objective (Lesson 21). NIST says the business impact analysis helps determine the type and frequency of backups. If the warehouse can lose at most 15 minutes of orders, a nightly backup cannot meet that RPO, and log backups or replication (Lesson 30) must fill the gap.

Media rotation decides how long each set is kept and when its media is reused. The common scheme is grandfather-father-son (GFS). Daily sets (sons) are kept for a week or two, weekly sets (fathers) for a month or two, and monthly sets (grandfathers) for a year or more. Veeam describes GFS as long-term retention that keeps backups for weeks, months and even years. Rotation must also honor the retention schedule and any litigation holds (Lesson 26).

Two rules shape the copies themselves. CISA's backup guidance gives the 3-2-1 rule: keep three copies of any important file (the primary and two backups), on two different media types, with one copy off-site. And Lesson 26's ransomware warning adds a fourth idea: at least one copy should be offline or immutable, because ransomware hunts for backups it can reach.

Media types

NIST's selection factors explain the warning in the tape row. The backup device must be compatible with the systems you restore to (equipment interoperability), and each medium has a storage life beyond which it cannot be relied on (media life). A shelf of LTO-9 tapes is only as good as a working drive that can still read LTO-9 media, and an LTO-10 drive cannot.

File-level or system-state?

A file-level backup protects files and folders, and suits Thursday's data volume. A system-state backup protects the operating system's own configuration, so that a server's identity and settings can be restored, not just its files. Windows Server Backup creates one with wbadmin start systemstatebackup. Microsoft notes that it does not include users' registry hives (HKEY_CURRENT_USER). Most servers need both, plus a plan for rebuilding the OS itself, such as the golden images CISA's ransomware guide recommends keeping.

Restoring: inventory first, then choose a method

Media inventory before restoration comes first. List exactly which sets the restore needs (for incrementals, the full and every link in the chain), confirm each is present, labeled and readable, and start retrieving off-site media early. NIST's off-site criteria (Lesson 24) include how long retrieval takes. Discovering a missing Tuesday tape halfway through a restore wastes hours.

Then choose the restore method. Windows Server Backup's wbadmin start recovery command shows all three.

For Thursday's problem, the answer is an alternate location: restore to a clean, rebuilt server, not onto the infected one, so the malware does not encrypt the restored data again.

Validation: a backup is only as good as its last restore

• Media integrity: use the backup software's verification and checksums, and replace media that report errors or reach the end of their rated life.
• Equipment: keep working drives and software that can read every set you still retain, and test them on older media.
• Regular testing intervals: restore real data on a schedule and time it against the RTO. NIST says backups should be routinely conducted, stored off-site, rotated and periodically validated. CISA calls for regularly testing the availability and integrity of backups.

So what?: a backup job that reports success has proved only that it wrote something. Only a timed test restore proves you can meet the RPO and RTO you promised.

Common misconceptions

• "Snapshots are backups." Microsoft says a standard checkpoint is not a full backup, and a snapshot on the same storage fails with it.
• "RAID is a backup." RAID survives a drive failure (Lesson 6), not deletion, corruption or ransomware.
• "Incrementals are always better." They are fastest to make, but a restore needs every link in the chain.
• "Cloud backups are safe from ransomware." Reachable backups are targets; use immutability and an offline copy.
• "A new tape drive reads all our old tapes." LTO-10 drives read only LTO-10 media.
• "The job said success, so the backup is fine." Only a test restore proves it.

Pulling it together

• Full, incremental and differential backups trade backup time against restore complexity; synthetic fulls shorten the chain, and open file backups use VSS for consistency.
• Snapshots and checkpoints are not backups until copied elsewhere.
• Set frequency from the RPO, rotate with GFS, follow 3-2-1, and keep one copy offline or immutable.
• Choose tape, disk, cloud or print by restore speed, cost, isolation and compatibility; back up both files and system state.
• Inventory media before restoring, and choose overwrite, side by side or an alternate location deliberately.
• Validate media and equipment, and test restores on a schedule against the RTO.

Sources

1. Swanson, M., Bowen, P., Phillips, A. W., Gallup, D., and Lynes, D. (2010). Contingency planning guide for federal information systems (NIST Special Publication 800-34 Rev. 1), section 5.1. csrc.nist.gov
2. Microsoft. (n.d.). Volume Shadow Copy Service (VSS); Using checkpoints to revert virtual machines to a previous state; wbadmin start recovery; wbadmin start systemstatebackup. Microsoft Learn. VSS; Checkpoints; Recovery; System state
3. Veeam Software. (n.d.). Synthetic full backup and Long-term retention policy (GFS). Veeam Backup & Replication user guide. Synthetic full; GFS
4. Ruggiero, P., and Heckathorn, M. A. (2012). Data backup options. US-CERT, published by CISA. cisa.gov; Cybersecurity and Infrastructure Security Agency. (n.d.). #StopRansomware guide. cisa.gov
5. LTO Program. (n.d.). LTO generation compatibility. lto.org; Amazon Web Services. (n.d.). Locking objects with Object Lock. Amazon S3 user guide. docs.aws.amazon.com

The night the river came up

At 1:40 a.m. on a Saturday, a water main bursts under the street outside a regional credit union's head office. By dawn the basement is flooded, utility power is cut, and the fire department has closed the building for at least three days. The primary data center is in that basement. Backups (Lesson 29) can rebuild data, but not quickly enough for a bank whose members expect cash machines and online banking to work. That is the job of disaster recovery (DR): restoring whole services at another site within the recovery time and recovery point objectives that the business impact analysis set (Lesson 21).

The credit union's plan worked that weekend, and the reasons it worked were decided long before the flood. This lesson follows its recovery to show the three decisions every DR plan makes: where to recover, how data gets there, and how the plan is proved.

Where to recover: site types

NIST's contingency planning guide (SP 800-34 Revision 1) describes the classic alternate sites, from least to most ready.

Cloud recovery sites come in the same range of readiness. Amazon's disaster recovery guidance describes four strategies, in rising order of cost and speed. Backup and restore keeps only data in the cloud. Pilot light keeps core data replicated and a minimal set of systems ready to switch on. Warm standby runs a scaled-down copy of the whole system. Multi-site active/active runs full capacity in more than one place.

The credit union matched each system to its tier. Core banking, with a four-hour RTO, recovers at a hot site in a city 400 kilometers away. File shares, with a 24-hour RTO, use a warm site. The reporting archive, which can wait a week, uses cloud backup and restore.

The most important word in the table is not in the table: separate geographic locations. NIST's controls call for alternate storage and processing sites separated from the primary so that they are not susceptible to the same hazards. Two years earlier, the credit union's warm site had been three kilometers away, on the same river floodplain. It would have flooded the same night. A tabletop exercise caught it, as the testing section explains.

How data gets there: replication

Backups move data in batches; replication copies changes continually to the recovery site, so the RPO can be minutes or zero. The objectives name several properties.

• Synchronous or asynchronous. Microsoft's Storage Replica documentation draws the line clearly. Synchronous replication mirrors data within a low-latency site and ensures zero data loss at the file-system level, because each write completes at both ends before it is confirmed. Asynchronous replication mirrors data across longer distances and higher-latency links, but without a guarantee that both sites hold identical copies at the moment of failure. Distance decides: the credit union's hot site, 400 kilometers away, is replicated asynchronously, and its RPO is a few minutes, not zero.
• Constant or background. Constant replication sends each change as it happens. Background replication queues changes and sends them later, by schedule or throttled to spare the network. Microsoft notes that DFS Replication, a background design, waits for files to close and throttles itself, so its latency can be hours or days, and the newest, busiest files are the least likely to have replicated.
• Application-consistent. Microsoft's Azure Site Recovery documentation explains the difference. A crash-consistent snapshot captures only what was on disk, while an application-consistent snapshot also includes data in memory and transactions in progress, using VSS to make applications flush to disk first. Databases need application-consistent recovery points to restart cleanly.
• Mirroring. A continuously maintained duplicate, from RAID 1 inside a server to NIST's mirrored sites between data centers.
• Bidirectional. Both sites accept changes and replicate to each other. DFS Replication works this way; Microsoft describes it as a multi-master design.

Bidirectional replication brings the file locking problem. Each Windows server knows only about locks on its own files, so DFS Replication cannot stop two users on different servers from editing the same file. When they do, Microsoft explains, the last writer wins, and the losing copy is moved to a hidden ConflictAndDeleted folder. The credit union met this on Monday, after failing back: a loan officer and an auditor had both edited the same spreadsheet, one at each site. One version had vanished into ConflictAndDeleted until an administrator recovered it. Microsoft advises against DFS Replication where users edit the same files at the same time on different servers.

How the plan is proved: testing

A DR plan that has never been exercised is a theory. NIST describes a ladder of tests, and the objectives use the same rungs.

The distinction between production and non-production testing runs through every rung. Non-production tests, such as tabletops and isolated simulations, can run often and safely. Production tests, such as live failovers, prove more but need a change window, a rollback plan and business approval. The credit union's calendar used both: a tabletop each spring, which found the floodplain problem, a simulated failover each autumn, and one planned live failover a year for core banking. On the night of the flood, the team had already done every step at least once.

The core of it: disaster recovery is decided before the disaster. Choose the site your RTO requires, the replication your RPO requires and a location the same hazard cannot reach, then rehearse until failover is routine.

Common misconceptions

• "A backup is a disaster recovery plan." Backups restore data; DR restores whole services at another site within an RTO.
• "Synchronous replication works at any distance." It needs low latency; long distances call for asynchronous replication and a non-zero RPO.
• "Replication protects against ransomware and mistakes." It copies bad changes too, quickly. Keep point-in-time recovery points and backups.
• "A DR site across town is fine." NIST calls for separation from the same hazards, and floods, storms and grid failures cover whole districts.
• "Bidirectional file replication handles simultaneous edits." Without distributed locking, the last writer wins.
• "A tabletop exercise proves the systems will fail over." It tests decisions and roles. Only simulated and live failovers test the technology.

Summing up

• Cold, warm and hot sites trade cost for recovery speed; mirrored sites and active/active cloud designs approach continuous availability.
• Cloud DR strategies range from backup and restore through pilot light and warm standby to multi-site active/active.
• Recovery sites must be far enough away that one hazard cannot reach both.
• Synchronous replication gives zero data loss at short range; asynchronous covers distance with some lag.
• Use application-consistent recovery points for databases, and beware last-writer-wins conflicts in bidirectional replication.
• Test with tabletops, simulated failovers in non-production, and planned live failovers in production.

Sources

1. Swanson, M., Bowen, P., Phillips, A. W., Gallup, D., and Lynes, D. (2010). Contingency planning guide for federal information systems (NIST Special Publication 800-34 Rev. 1), sections 3.4 and 3.5 and appendix F. csrc.nist.gov
2. Microsoft. (n.d.). Storage Replica overview. Microsoft Learn. learn.microsoft.com
3. Microsoft. (n.d.). DFS Replication FAQ and Understanding (the lack of) distributed file locking in DFSR. FAQ; File locking
4. Microsoft. (n.d.). Azure to Azure disaster recovery architecture. Azure Site Recovery documentation. learn.microsoft.com
5. Amazon Web Services. (n.d.). Disaster recovery options in the cloud. Disaster recovery of workloads on AWS whitepaper. docs.aws.amazon.com

Monday, 8:15 a.m.

The help desk forwards a stream of complaints: the company's intranet application is "broken". Some users say pages fail with an error. Others say it works if they press refresh. The application runs on two web servers, WEB01 and WEB02, behind a load balancer (Lesson 18), and a patch window ran on Sunday night. An administrator who guesses can lose a morning. One who follows a method usually finds the cause within an hour, and can prove it.

The Server+ objectives set out an eight-step method. This lesson works through Monday's outage one step at a time. Google's book on site reliability engineering, which describes troubleshooting as an application of the hypothetico-deductive method, adds practical advice: observe, form a hypothesis about the cause, test it, and repeat.

Step 1: Identify the problem and determine the scope

Scope comes before solutions. The objectives list the moves.

• Question users and stakeholders. Who is affected, doing what, since when? Answers: all departments; about half of page loads fail; it started before 8 a.m.
• Identify changes to the server or environment. The change calendar (Lesson 21) shows Sunday's patch window, which rebooted both web servers. Google's SRE book calls this "what touched it last": working systems tend to keep working until something changes them, so recent changes are a productive place to start.
• Collect additional documentation and logs. The load balancer's log shows that every failed request went to WEB02. WEB02's event log shows the reboot at 1:12 a.m.
• Replicate the problem, if possible. Requests sent straight to each server, bypassing the load balancer, succeed on WEB01 and return 503 Service Unavailable on WEB02 every time.
• Back up before making changes, if possible. A fresh backup of WEB02's configuration, plus a checkpoint to roll back to quickly (Lesson 29), means any fix can be undone.
• Escalate, if necessary. If the evidence had pointed at application code, the application team would take over.

Scoping also means triage. The SRE book's advice is to make the system work as well as it can first, for example by diverting traffic away from a broken component, while preserving logs as evidence for the later analysis. Here that means draining WEB02 from the load balancer pool, so every user reaches WEB01 while the diagnosis continues.

Step 2: Establish a theory of probable cause

The objectives add two instructions: question the obvious, and look for a common element or symptom behind multiple problems. The common element is plain: every failure involves WEB02. The obvious question is whether the application service on WEB02 is even running after the reboot. The SRE book's version of the same advice is borrowed from medical training: when you hear hoofbeats, think of horses, not zebras. All else being equal, prefer simpler explanations.

Step 3: Test the theory to determine the cause

On WEB02, Get-Service shows the application's service as Stopped, with its startup type set to Manual. The event log confirms it never started after the reboot. The theory is confirmed, so the next step is to plan the fix.

Had the service been running, the theory would have failed, and the method says to establish a new one, such as a full disk, an expired certificate or a configuration change. The SRE book makes the same point: a clear negative result still tells you something certain. It also warns against hunting spurious correlations. Correlation is not causation, and in a large, heavily monitored system some events will line up purely by coincidence.

Step 4: Establish a plan of action

The plan: with WEB02 already drained, start the service, set its startup type back to Automatic, test WEB02 directly, then return it to the pool, with the checkpoint as the rollback. Notify impacted users: the help desk posts a message that the intranet is working through one server, and that a fix is in progress, with the time of the next update.

Step 5: Implement the solution or escalate

The objectives' key rule: make one change at a time, and test after each. First, start the service and test WEB02 directly; it now returns the application's pages. Second, set the startup type to Automatic. Third, return WEB02 to the load balancer pool and watch the error rate. If a change does not fix the problem, reverse it where appropriate before trying the next one. Otherwise, changes pile up and nobody can say which one mattered, or which one broke something else.

Bottom line: one change at a time, each tested and written down, is what turns a lucky fix into a known fix, and makes every mistake reversible.

Step 6: Verify full system functionality, and prevent a repeat

"The error went away" is not verification. Test the whole application through the load balancer, including sign-in, search and file upload. Confirm both servers are serving traffic, and check that monitoring is green. Then ask what would stop a repeat, and implement the preventive measures that fit.

Step 7: Perform a root cause analysis

The service was stopped, but why? Asking "why" repeatedly finds two contributing causes. First, three months earlier, during unrelated troubleshooting, someone set the service to Manual startup and never changed it back, and no change record existed. The server had drifted from its configuration baseline (Lesson 21). Second, the load balancer's health check tested only whether port 443 accepted a connection, and WEB02's web server software kept accepting connections while answering every request with an error. So the load balancer kept sending WEB02 half of the traffic. The preventive actions follow from the causes: a health check that requests a real application page, configuration drift detection against the baseline, and a post-patch checklist that tests each server directly.

Google's SRE book defines a postmortem as a written record of an incident, its impact, the actions taken, the root causes and the follow-up actions to prevent recurrence. It insists that postmortems be blameless, a learning opportunity rather than a punishment. The goal is a better system, not a scapegoat.

Step 8: Document findings, actions and outcomes, throughout

Documentation is the step that runs alongside all the others. The ticket's timeline should record each observation, theory, test, change and result as it happens, with times. That record lets a colleague take over mid-incident, feeds the postmortem, and becomes the first thing searched the next time a server returns 503 after a patch window.

Common misconceptions

• "Start fixing straight away." Scope first. Triage to stop the damage, but keep the evidence.
• "Change several things at once to save time." Then nobody knows which change worked, or which one broke something.
• "The cleverest theory is probably right." Think horses, not zebras: simple, common causes first.
• "Two things happened together, so one caused the other." Correlation is not causation; test the theory.
• "It works now, so close the ticket." Verify everything, add preventive measures and find the root cause.
• "Root cause analysis finds who to blame." Blameless postmortems find what to fix.

Looking back

• Identify the problem and scope: question stakeholders, find changes, collect logs, replicate, back up, escalate if needed.
• Theorize from the obvious and from common elements; test; if the theory fails, form a new one.
• Plan the fix with a rollback, and notify impacted users.
• Implement one change at a time, testing each and reversing changes that do not help.
• Verify full functionality and add preventive measures, then run a blameless root cause analysis.
• Document observations, actions and outcomes throughout, with times.

Sources

1. CompTIA. (2019). CompTIA Server+ certification exam objectives, exam number SK0-005 (version 5.0), objective 4.1. comptiacdn.azureedge.net
2. Jones, C. (2016). Effective troubleshooting. In B. Beyer, C. Jones, J. Petoff, and N. R. Murphy (Eds.), Site reliability engineering. O'Reilly Media; free online edition from Google. sre.google
3. Lunney, J., and Lueder, S. (2016). Postmortem culture: Learning from failure. In B. Beyer, C. Jones, J. Petoff, and N. R. Murphy (Eds.), Site reliability engineering. O'Reilly Media. sre.google
4. Johnson, A., Dempsey, K., Ross, R., Gupta, S., and Bailey, D. (2011, updated 2019). Guide for security-focused configuration management of information systems (NIST Special Publication 800-128). csrc.nist.gov

A week of hardware tickets

Hardware rarely fails silently. It logs, blinks, beeps, crashes or overheats first, and the skill is reading those signals before the failure becomes an outage. This lesson debugs six tickets from one week in a small data center, using the method from Lesson 31. Each ticket follows the same pattern: the symptom, the evidence, the cause and the fix. Keep two habits from earlier lessons in mind throughout. The BMC (Lesson 8) is usually the first place to look, because its event log records hardware events even when the operating system is down. And replacement parts must come from the server's compatibility list (Lesson 4).

Ticket 1: "Predictive failure" alerts

Symptom. The BMC raises two warnings on a file server: a drive in bay 3 reports a predictive failure, and one DIMM shows a rising count of corrected memory errors. Nothing has stopped working.

Evidence. For the drive, smartctl -H checks its SMART health. The smartctl manual says a failing health status means the drive has either already failed or is predicting its own failure within the next 24 hours, and advises getting the data somewhere safe as soon as possible. For the memory, the Linux kernel's documentation on reliability, availability and serviceability explains the categories. A correctable error (CE) was detected and fixed by ECC. An uncorrected error (UE) exceeded what ECC could fix. If a UE hits a critical component, the only reliable way to avoid data corruption is to hang or reboot the machine. Windows records the same kind of events through the Windows Hardware Error Architecture (WHEA).

Fix. Treat predictive failures as scheduled maintenance, not emergencies. Hot-swap the drive (Lesson 9) and watch the RAID rebuild (Lesson 6). Replace the DIMM in a planned window, before its corrected errors become uncorrected ones.

Ticket 2: three kinds of crash

Symptom. Three hosts crashed this week, each in its own style.

Evidence. The crash screen and the dump point toward software or hardware. Broadcom's guidance on purple screens offers two useful tests. If the screen mentions a Machine Check Exception, ESXi reports that the system has encountered a hardware error and advises contacting the hardware vendor. And if the error message and stack vary widely between crashes, that may indicate a hardware problem, although Broadcom calls it inconclusive. The same reasoning applies on Windows: a different stop code every time, with no common driver, points toward memory, CPU or power.

Fix. Prepare before the crash. Make sure dumps are configured and have space, such as the page file on Windows, a reserved crashkernel and a /var/crash target on Linux, and a core dump target on ESXi. Red Hat notes that a kernel crash dump can be the only information available after a failure.

Ticket 3: a server that will not finish POST

Symptom. After a technician added memory, a rack server will not finish the power-on self-test (POST). Its front panel shows an amber fault LED, and the LCD panel reads a memory configuration error.

Evidence. The fault and device indicators are the first clues, and the server's maintenance manual decodes them.

• LEDs: amber or blinking fault lights on the chassis, drives, power supplies and sometimes beside each DIMM slot.
• LCD panel readouts and POST codes: text or numeric codes that name the failing stage or part.
• Auditory cues: beep patterns during POST, and the sound of fans running flat out.
• Olfactory cues: a burning or hot-electronics smell. Power the server off at once and investigate before restoring power.

Cause and fix. The likely culprits are improperly seated DIMMs, modules in the wrong slots for the population rules, or parts that are not on the compatibility list. Reseat the modules following the manual, check the slot order, and confirm the part numbers. If POST still fails, the BMC's event log names the slot.

Ticket 4: afternoon lockups

Symptom. A server suffers random lockups, always on hot afternoons.

Evidence. The BMC's sensor history shows CPU temperature climbing to its limit just before each lockup, and one fan reporting a fault. Inside, the fan has failed and the heat sink's intake is matted with dust.

Cause and fix. This is a chain of technical causes (a malfunctioning fan, then CPU overheating) and environmental causes (dust and high room temperature; Lesson 23 gives the recommended ranges). Replace the fan, clean the intake with compressed air away from the running equipment, and check the room's cooling. If the heat sink is ever removed, it must be reseated with fresh thermal material; an improperly seated heat sink causes the same overheating. Lenovo's guidance on static-sensitive devices applies to every part: keep components in their static-protective packages until installation, and handle them with an ESD wrist strap or other grounding system.

Ticket 5: a clock stuck in the past

Symptom. After a weekend power cut, a server boots with its clock set years in the past and its boot order reset to the defaults.

Cause and fix. This is a classic CMOS battery failure. The coin cell keeps the firmware settings and real-time clock while the server is unplugged (Lesson 9). Replace the battery, restore the firmware settings from the configuration record (Lesson 21), and correct the time. Lesson 34 shows why a wrong clock breaks domain sign-in.

Ticket 6: a VM that "keeps failing"

Symptom. A database VM runs out of memory and crashes every few days, and its owner blames the hardware. The host shows plenty of free RAM.

Cause and fix. These are misallocated virtual resources: the VM was given 4 GB for a workload that needs 16 (Lesson 19). The opposite case is common too, where a host is overcommitted and its VMs starve at peak times. Compare guest and host metrics before blaming hardware, then adjust the allocation.

Key idea: hardware announces trouble through logs, sensors and indicators long before it stops. Read the BMC first, collect the evidence a crash leaves behind, and change one part at a time.

Causes and tools, collected

Most vendors also ship hardware diagnostics that run before the operating system loads, and can test memory, drives and other components in isolation. Running them is often the fastest way to prove whether a part is at fault.

Common misconceptions

• "Corrected memory errors are harmless." A rising count is a warning; replace the module before errors become uncorrectable.
• "A predictive failure can wait until the drive dies." smartctl's manual says to get the data safe as soon as possible.
• "Every crash is a software bug." Machine check exceptions and constantly changing crash signatures point to hardware.
• "Crash dumps can be set up after the crash." Without a configured dump target, the evidence is gone.
• "A burning smell can wait until the next maintenance window." Power off and investigate at once.
• "If a VM crashes, the host hardware is failing." Check the VM's resource allocation first.

What to remember

• Predictive failures (SMART health, rising corrected memory errors) mean planned replacement before an outage.
• Blue screens, purple screens and kernel panics leave evidence in dumps; configure dump targets in advance.
• Read the indicators: LEDs, LCD readouts, POST codes, beeps and smells, decoded with the server's manual.
• Technical causes (power, fans, heat sinks, seating, compatibility, firmware, backplanes, overheating) often combine with environmental ones (dust, humidity, temperature).
• Tools: event logs, firmware updates or downgrades, vendor diagnostics, compressed air, ESD protection, and reseating or replacing parts.
• For VMs, check virtual resource allocation before blaming hardware.

Sources

1. The Linux kernel development community. (n.d.). Reliability, availability and serviceability (RAS). Linux kernel documentation. docs.kernel.org
2. smartmontools developers. (n.d.). smartctl(8) manual page. Ubuntu Manpage Repository. manpages.ubuntu.com
3. Microsoft. (n.d.). Bug checks (stop code errors); Overview of memory dump file options for Windows; Windows Hardware Error Architecture (WHEA) design guide. Microsoft Learn. Bug checks; Memory dumps; WHEA
4. Red Hat. (n.d.). Managing, monitoring, and updating the kernel, RHEL 9: Installing kdump; Configuring kdump on the command line. Installing kdump; Configuring kdump
5. Broadcom. (n.d.). Interpreting an ESXi host purple diagnostic screen (PSOD) (Article 343033); Machine check error (MCE) output decoding after ESXi panic (Article 367928). PSOD; MCE
6. Lenovo. (n.d.). ThinkSystem ST45 V3: Handling static-sensitive devices. pubs.lenovo.com

The problem: four storage complaints on one Monday

A file server called FS02 generates four tickets before lunch. Users say opening and saving files is painfully slow. Last night's backup job failed. After a weekend reboot, a Linux server that stores scanned documents will not mount its data volume. And users of a third system say their mapped drive Z: is "not available". Storage problems often arrive in bundles like this, because one underlying fault surfaces in several places.

The way through is to work from the bottom of the storage stack up, checking each layer before trusting the one above it.

The point: a symptom near the top, such as a slow share, often has its cause near the bottom. Check the physical layer and the array before you rebuild a file system or blame the network.

Complaint 1: slow file access and slow I/O

The bottom layers explain FS02's slowness. A visual inspection finds an amber fault LED on one drive. The controller's log, read with its RAID management utility, shows two events from Saturday. A drive failed, and the hot spare began rebuilding the array. And the cache battery reported as degraded.

Both events cost performance. An array rebuild reads every surviving drive to reconstruct the missing one, competing with normal I/O for hours (Lesson 6). The battery matters even more for writes. Dell's documentation for its PERC controllers explains that when the cache battery is degraded or faulty, the controller switches its write policy from write-back to write-through to protect data integrity, which reduces system performance. Every write must then reach the disks before it is acknowledged. A cache battery failure therefore looks exactly like a mysterious slowdown, and so does a cache turned off by mistake. Replace the battery or cache module, and confirm the write policy returns to write-back.

Check the remaining causes too. Disk space utilization above roughly 90 percent slows many file systems and leaves no room for growth, so monitoring tools (Lesson 16) should alert well before a volume fills. A full volume holding the page, swap or scratch file causes its own failures, from out-of-memory errors to applications that cannot write temporary data (Lesson 15).

Complaint 2: the unsuccessful backup

The backup log names the cause: insufficient space on the backup target, because the rebuild delayed the previous night's cleanup job. Free the space, re-run the job, and add a capacity alert for the target. An unsuccessful backup is never just an annoyance; until it succeeds, the RPO is not being met. The mirror-image problem, a restore failure, usually traces back to the media inventory and validation steps in Lesson 29.

The rebuild also carries a quiet danger. If a second drive fails during a RAID 5 rebuild, the array is lost, and the only path is restoring from backup. That is a multiple drive failure, and it is why a degraded array deserves priority. Replacing the failed drive brings its own rule. Lenovo's guidance says the drives in a single RAID array must be the same type, size and capacity, so a smaller or different kind of spare from the shelf will be refused. Those are mismatched drives, an easy mistake under pressure.

Complaint 3: a volume that will not mount

On the Linux server, the boot log reports that the data volume failed to mount. Running lsblk shows the disk is present, but it is now called /dev/sdc, because a new disk added last week took the name /dev/sdb. The file /etc/fstab still says /dev/sdb, so this is misconfiguration, not hardware. The fstab manual explains the lasting fix: refer to file systems by LABEL or UUID instead of device names, because device names follow hardware detection order and can change when disks are added or removed. After editing fstab, run mount -a to mount everything it lists and confirm there are no errors before the next reboot. For non-critical volumes, the nofail option tells the system not to report errors if the device does not exist, so a missing disk does not stop the boot.

If lsblk had not shown the disk at all, the cause would sit lower in the stack. When several drives or LUNs vanish at once, check what they share. That could be a backplane failure in one drive cage, or, for SAN storage (Lesson 7), an HBA failure, loose connectors or cable problems on this server alone.

Complaint 4: "drive not available"

On Windows, the same idea applies at the top of the stack. Users' Z: drive maps to a share that was moved from \\fs02\projects to \\fs03\projects. The net use command connects a computer to a shared resource or disconnects it, and controls persistent connections, so the fix is to delete the stale mapping and create a new one, ideally through Group Policy rather than by hand. When a local volume cannot be accessed, open Disk Management. Microsoft describes it as showing every drive and partition, with details and space used. A disk may be offline, or a volume may have lost its drive letter.

The deeper failures: boot, sectors and corruption

• Boot errors and "OS not found": check the firmware boot order first, since a CMOS reset (Lesson 32) can change it. Then check for a corrupt boot sector or boot loader, or an improper disk partition, such as a missing EFI system partition. Microsoft's bcdboot can repair the boot environment on the system partition. Red Hat documents reinstalling GRUB to fix problems caused by missing files or a broken installation.
• Sector and block errors, read and write errors: these point to bad sectors or media failure. Microsoft's chkdsk /r locates bad sectors and recovers readable information, with the disk locked. Then check SMART and plan to replace the drive.
• Partition errors and a corrupt file system table: repair the file system's metadata with chkdsk /f on Windows. On Linux, the xfs_repair manual says the file system to be repaired must be unmounted. If repair fails, restore from backup.
• Data corruption: find the cause first, such as a failing drive, a controller fault or an unclean shutdown with write-back cache and no battery. Then restore the damaged data from backup, and verify it with checksums where the file system supports them (Lesson 12).
• Cache failure: a failed cache module can force the controller into write-through mode or take its arrays offline; replace it, and check the array's consistency afterward.

Common misconceptions

• "Slow storage means the disks are too slow." Look for a rebuild, a degraded cache battery or a disabled cache first.
• "Any spare drive will do for a rebuild." Arrays need drives of the same type, size and capacity.
• "Device names like /dev/sdb never change." They follow detection order; use UUIDs or labels in fstab.
• "File system repair tools can run on a mounted volume." xfs_repair needs it unmounted, and chkdsk /r locks the disk.
• "A degraded RAID array can wait." A second failure during a RAID 5 rebuild loses the array.
• "If the share is missing, the server is down." Check the mapping, the share's new location and permissions first.

The takeaway

• Work bottom-up: physical, controller and array, partition, file system, then mount or share.
• Rebuilds and cache battery failures (write-through mode) are prime suspects for slow I/O.
• Failed backups often come from insufficient space; multiple drive failures mean restoring from backup.
• Replacement drives must match the array in type, size and capacity.
• Mount by UUID or label, test with mount -a, and use net use or Group Policy for Windows mappings.
• Repair boot problems with bcdboot or a GRUB reinstall, and file systems with chkdsk or xfs_repair; restore from backup when repair fails.

Sources

1. Dell Technologies. (n.d.). Data Domain: Faulty PERC battery causes degraded performance (Knowledge base article 000038067). dell.com
2. Lenovo. (n.d.). ThinkSystem ST550: Install a hot-swap drive. pubs.lenovo.com
3. Microsoft. (n.d.). chkdsk; bcdboot; Overview of Disk Management; Net use. Microsoft Learn. chkdsk; bcdboot; Disk Management; net use
4. Linux man-pages project. (n.d.). fstab(5), mount(8), lsblk(8) and xfs_repair(8). fstab; mount; lsblk; xfs_repair
5. Red Hat. (n.d.). Managing, monitoring, and updating the kernel, RHEL 9: Reinstalling GRUB. docs.redhat.com

The week after the update weekend

A company patched forty servers over a weekend. By Wednesday, the help desk had six tickets that all seemed to blame "the update". Some did come from the update, some only surfaced because of the reboots, and one had nothing to do with it at all. This lesson debugs each one with the method from Lesson 31, starting from the evidence rather than the rumor. One theme runs through them all: most operating system and software problems after a change are caused by the change, so always know what changed, and always keep a way back.

Ticket 1: nobody can log on to APP3

Symptom. Domain users cannot sign in to the server APP3, but its local administrator account works.

Evidence. APP3's clock is nine minutes ahead of the domain controller's. Microsoft explains that Kerberos uses time stamps to prevent replay attacks, so the client and domain controller clocks must agree within a maximum tolerance, which Microsoft advises setting to 5 minutes. Nine minutes of clock skew breaks every domain sign-in.

Cause and fix. The reboot reset APP3's time source to an unreachable server. Point it back at the domain time hierarchy, then confirm synchronization with w32tm /query /status on Windows. On Linux, chronyc tracking reports the state of chronyd, and Red Hat also notes that Kerberos relies on synchronized clocks. If the time had been correct and sign-ins still failed with trust errors, the next theory would be that the server is not properly joined to the domain. PowerShell's Test-ComputerSecureChannel tests, and with -Repair repairs, the secure channel between a computer and its domain.

Ticket 2: a service that will not start

Symptom. On a Rocky Linux server, the application's service failed after the update.

Evidence. systemctl status shows the service failed, and the journal says a shared library it needs is missing. A missing dependency: the update removed a package the application relied on, which is a downstream failure due to updates. Dependencies also exist between services. systemctl list-dependencies shows a unit's dependency tree, and on Windows, Get-Service with -RequiredServices or -DependentServices shows what a service needs and what needs it.

Fix. Install the missing package with the package manager, or roll the update back. Red Hat documents dnf history undo, which reverts a single DNF transaction, and dnf history rollback, which reverts everything after a chosen transaction. Red Hat also warns that downgrading system packages such as the kernel, glibc and selinux-policy this way is not supported. For those, a snapshot taken before patching (Lesson 29) is the safer way back. Then fix the process: test updates first (Lesson 27), and use version management, such as holding a known-good version of a library, until the application supports the new one.

Ticket 3: the update that would not install

Symptom. One Windows server reports a patch update failure every night and is now missing two months of security updates.

Evidence and fix. The update log shows corrupted system components. Microsoft's guidance for system file corruption is to run DISM first, with DISM.exe /Online /Cleanup-image /Restorehealth to supply repaired files, and then the System File Checker, sfc /scannow. After the repair, the updates install. Missing updates are a security problem in themselves, so track patch compliance (Lesson 27).

A related problem has no repair at all. A server whose operating system is at end of life or end of support, such as Windows Server 2016 after its extended support ends in January 2027 (Lesson 21), will stop receiving fixes. The only real fix is an upgrade.

Ticket 4: slow, then frozen

Symptom. A server grows slower each day until the application hangs and the console freezes; a reboot fixes it for about a week.

Evidence. The monitoring graphs (Lesson 16) show one process's memory climbing steadily from boot until the server starts paging heavily. That is a memory leak: the program keeps claiming memory and never releases it. A second, smaller problem shows up in the same data. A nightly report job runs at the highest priority, pinned to the same CPU as the database, and the database stalls every night.

Fix. Until the vendor ships a fix, scheduled reboots or service restarts keep the leak from reaching the danger point. Report the leak, and patch it when the fix arrives. For the report job, correct its CPU affinity and priority. On Linux, taskset sets or retrieves a process's CPU affinity, and nice or renice adjust its niceness, from -20 (most favorable) to 19. On Windows, Task Manager sets both.

What matters here: after a change, the change is the first suspect. Before any change, make sure a way back exists: a snapshot, the package history or a previous kernel.

Ticket 5: cannot write logs, cannot reach files

Symptom. A web application cannot write to its logs, and users report they are unable to access files it serves.

Evidence. During the update, the application's service account was replaced, and the new account has no write permission on the log folder or read permission on the content folder. These are inappropriate application-level permissions. On another server, the same symptom had a different cause: the volume holding the logs was full (Lesson 33).

Fix. Someone proposed running the service as root. That fixes the symptom by breaking least privilege (Lesson 25). Instead, grant the new account exactly the folder rights it needs. Administrators should follow the same rule for themselves, through proper privilege escalation. Windows' runas runs a specific program with different permissions than the current sign-in. On Linux, sudo runs a single command with administrative rights and logs it, while su switches to another account, such as root, for a whole session. Prefer the narrowest tool that works.

Ticket 6: a server that will not boot

Symptom. After a new storage driver was installed, a Windows server crashes during startup. A Linux server shows the same symptom after a new kernel module.

Cause. Incompatible or unstable drivers or modules, often ones not on the server's compatibility list (Lesson 4). Similar causes include incompatibility in general, such as software built for a different processor architecture (the Rocky Linux 10 processor baseline in Lesson 1 is one example), and improperly applied patches.

Fix with the recovery tools. On Windows, Microsoft's startup settings include safe mode, which starts Windows in a basic state with a limited set of files and drivers, so the new driver can be removed. On Linux, choose the previous kernel from the GRUB menu, or boot to the rescue target. The systemd documentation describes rescue.target as administering the system in single-user mode, with file systems mounted but no services running. If a snapshot exists, reverting to it may be fastest. As a last resort, reload the OS from the build automation in Lesson 11 and restore the data.

Preventing the next round

• Software firewalls: after updates, confirm the firewall still allows the service's ports in the right zone or profile, adding or removing ports as needed (Lesson 13). A Windows server that decides its network is Public instead of Domain applies a different rule set.
• Insecure dependencies and buffer overruns: a library with a known vulnerability, or a bug that writes past the end of a buffer, is fixed by updating, so keep dependencies inside the patch process.
• Configuration management stops drift and makes fixes repeatable. Microsoft Configuration Manager (the SCCM of the objectives) deploys applications, software updates and operating systems. Group Policy Objects enforce Windows settings. Tools such as Ansible, Puppet and Chef describe the desired state of Linux and Windows servers in code. Ansible's documentation says it lets you automate virtually any task.
• The compatibility list: check drivers and firmware against the server vendor's and OS vendor's lists before installing them.

Common misconceptions

• "Sign-in failures always mean a wrong password." Clock skew beyond Kerberos tolerance, or a broken secure channel, produce the same symptom.
• "Any update can be rolled back with the package manager." Red Hat does not support downgrading the kernel or glibc that way; use snapshots.
• "Run sfc first, and it fixes everything." Microsoft says to run DISM first, then SFC.
• "Running the service as root fixes permission errors." It hides them by breaking least privilege. Grant the exact rights needed.
• "A weekly reboot is a fix for a memory leak." It is a workaround until the software is patched.
• "Safe mode and rescue mode are only for desktops." They are standard server recovery tools when a driver or module breaks the boot.

What to carry forward

• Sign-in failures: check clock skew (5-minute Kerberos tolerance) and domain secure channels.
• Service failures: check status, logs and dependencies; fix missing packages or roll back, testing updates first.
• Patch failures and corruption: DISM, then SFC; track missing updates and end-of-support dates.
• Performance: memory leaks (scheduled restarts until patched) and CPU affinity and priority (taskset, nice).
• Permissions: grant applications exact rights; escalate administrators with runas, sudo or su, not standing root access.
• Boot failures: safe mode, a previous kernel, rescue.target, snapshots, then reloading the OS; prevent repeats with configuration management.

Sources

1. Microsoft. (n.d.). Maximum tolerance for computer clock synchronization; Test-ComputerSecureChannel; Get-Service. Microsoft Learn. Clock tolerance; Secure channel; Get-Service
2. Microsoft. (n.d.). Use the System File Checker tool to repair missing or corrupted system files; Windows startup settings; Runas. SFC and DISM; Startup settings; runas
3. Red Hat. (n.d.). Managing software with the DNF tool, RHEL 9: Handling package management history; Configuring basic system settings, RHEL 9: Configuring time synchronization. DNF history; chrony
4. Linux man-pages project. (n.d.). systemd.special(7), su(1), taskset(1) and nice(1). systemd.special; su; taskset; nice
5. Microsoft. (n.d.). What is Configuration Manager? Microsoft Learn. learn.microsoft.com; Red Hat. (n.d.). Introduction to Ansible. docs.ansible.com

"The server is down"

Monday's ticket says the order server, APP4, is down. Is it? Perhaps the server is fine and its network cable is not. Perhaps it answers on its address but not by name, or a firewall blocks just one port. "Down" describes the user's experience, not the cause. The fastest way to the cause is a procedure that climbs the network from the bottom: the physical link, the interface's configuration, the local network, the routes, the names and finally the service's port. Each step either clears a layer or finds the fault, and no step is skipped because someone is sure it is "obviously DNS". This lesson walks through the procedure with Windows and Linux commands side by side. Use the commands only on your own lab and networks you are responsible for.

Step 1: The physical layer

• Check link lights. No light on the NIC or switch port means no physical link at all.
• Confirm power. Is the switch, transceiver or NIC actually powered? A tripped circuit takes out a whole rack's switches (Lesson 3).
• Verify cable integrity. Swap in a known-good cable, or use a cable tester. Bad cables also cause intermittent errors, not just dead links.
• Check cable selection. The wrong fiber type, transceiver or copper category for the speed will not link, or will link at a lower speed (Lesson 3).

APP4's port showed no link light after its network card was replaced. The switch's log explained it: network port security. Cisco's documentation describes the default response to a violation, such as a new MAC address on a port locked to the old one. The port becomes error-disabled and shuts down immediately, and its LED turns off. The network team re-enabled the port after approving the new card's address. A dark link light is not always a cable.

Step 2: The interface and its IP configuration

Next, look at what the server believes about itself. On Windows, ipconfig /all lists each adapter's addresses, mask, default gateway, DNS servers and DHCP details. On Linux, ip addr shows addresses and ip link shows link state. Compare them with the server's documented configuration (Lesson 21).

A misconfigured NIC belongs here too, such as a team set up differently from the switch (Lesson 18) or a forced speed that does not match the other end. After fixing DHCP problems, ipconfig /release and /renew request a fresh lease.

Step 3: Local reachability

Microsoft describes ping as verifying IP-level connectivity by sending ICMP echo Request messages. Ping the default gateway first, then another host on the same subnet. "Destination host unreachable" means the packet could not be delivered: either the local host found no path to the destination, or a router reported it unreachable. One caution applies to every ping test. Many firewalls block ICMP, so a failed ping alone does not prove a host is down. A successful ping, on the other hand, proves the path works up to IP.

Step 4: Routes beyond the local subnet

If the local subnet works but remote subnets are unreachable, check the routing table: route print on Windows, which displays and modifies the local IP routing table, or ip route on Linux. Typical faults are a wrong default gateway, a missing static route, or a multihomed server with two default gateways competing. Then trace the path. Microsoft explains that tracert sends echo requests with a time to live that rises by one each time, so each router along the path reports itself with an ICMP Time Exceeded message. The last router that answers shows where the path breaks. Linux's traceroute does the same job.

If the trace stops at your own edge router, and nothing on the internet answers while everything internal works, suspect a service provider failure. Check the provider's status and your second provider, if you have one (Lesson 3).

Step 5: Names

If the server answers by address but not by name, the problem is name resolution. "Unknown host" or "cannot reach the server by its FQDN" points to a DNS failure, a missing or wrong record, the wrong DNS server in the client's settings, or a missing DNS suffix for short names (Lesson 13). Test DNS directly with nslookup, which Microsoft describes as diagnosing DNS infrastructure, or with dig on Linux.

A classic trap: nslookup returns the right address, yet the application still connects to the old one. nslookup asks DNS servers directly. The operating system, however, also consults the hosts file, a static table of host names. On Linux, the hosts line in /etc/nsswitch.conf sets the order of those sources. A stale hosts file entry quietly overrides DNS. On Windows, ipconfig /displaydns shows the resolver cache and ipconfig /flushdns clears it. For old NetBIOS short names, nbtstat displays NetBIOS name tables and the name cache.

Remember: climb one layer at a time and change one thing at a time. A ping failure, a DNS answer or a green link light each proves only its own layer.

Step 6: The service's port

The host is reachable and the name resolves, but the application still cannot connect. Now test the port itself.

• On the server, netstat shows listening ports and active connections: -a for all, -n for numbers instead of names, and -o for the owning process. On Linux, ss -tulpn does the same (Lesson 27). If nothing listens on the port, the service is the problem (Lesson 34).
• From a client, test the port with a telnet client (telnet server 5432), or with nc. The ncat manual describes its -z option as zero-I/O mode that reports connection status only. A refused or timed-out connection to a listening service points to a firewall between them: a misconfigured rule, or a failed firewall appliance or host firewall service.

APP4's real problem, once the port was re-enabled, turned out to be this layer. The new network card had been placed in the Public firewall profile, which blocked the application's port (Lesson 34).

Common misconceptions

• "No ping means the host is down." Firewalls often block ICMP; test the service's port as well.
• "A dark link light is always a bad cable." Port security, power and transceivers can all cause it.
• "If nslookup works, name resolution works." The hosts file and resolver cache can still override DNS for applications.
• "A 169.254 address means the network is fine." It means the DHCP client could not reach a DHCP server.
• "Wrong DHCP settings must be the DHCP server's fault." A rogue DHCP server may be answering first.
• "It's always DNS." Sometimes it is. Prove it at Step 5, not by assumption at Step 1.

Recap

• Physical: link lights, power, cable integrity and cable selection; port security can error-disable a port.
• Interface: ipconfig /all or ip addr; APIPA means a DHCP failure, unexpected settings suggest a misconfigured or rogue DHCP server, and VLAN and IPv6 errors show here too.
• Local: ping the gateway and neighbors, remembering that ICMP may be blocked.
• Routes: route print or ip route, then tracert or traceroute to find where the path stops; a stop at your edge may be the provider.
• Names: nslookup or dig, the hosts file, the resolver cache and nbtstat for NetBIOS.
• Ports: netstat -ano or ss on the server, telnet or nc -z from the client; refused ports point to firewalls.

Sources

1. Microsoft. (n.d.). ipconfig; ping; tracert; nslookup; netstat; route; nbtstat; telnet. Windows Commands, Microsoft Learn. ipconfig; ping; tracert; nslookup; netstat; route; nbtstat; telnet
2. Linux man-pages project. (n.d.). ip-address(8), ip-route(8), traceroute(8), hosts(5), nsswitch.conf(5) and ncat(1). ip-address; ip-route; traceroute; hosts; nsswitch.conf; ncat
3. Internet Systems Consortium. (n.d.). dig(1) manual page. Ubuntu Manpage Repository. manpages.ubuntu.com
4. Cisco Systems. (n.d.). Security configuration guide, Cisco IOS XE Dublin 17.12.x (Catalyst 9300 switches): Port security. cisco.com
5. Cheshire, S., Aboba, B., and Guttman, E. (2005). Dynamic configuration of IPv4 link-local addresses (RFC 3927). Internet Engineering Task Force. rfc-editor.org

The problem: five security tickets on a Friday

Late on a Friday, the security queue for one Windows file server and one Rocky Linux web server holds five tickets. Users cannot open files on a share they used yesterday. A web application stopped loading after its files were moved. The file integrity tool flagged a changed system binary. An unknown process is listening on a high port. And an access review found a contractor in the Domain Admins group. Each could be an attack, or a misconfiguration that only looks like one.

Security troubleshooting uses the same method as Lesson 31, plus two extra rules. Preserve evidence before changing anything that might be part of an incident (Lesson 26). And use security tools only on systems you own or are authorized to test. Nmap's own documentation describes it as a tool for network exploration and security auditing, and pointed at someone else's network, it is an intrusion.

Ticket 1: files and shares that will not open

Evidence first. On the Windows share, the user's NTFS permissions say Modify, but the share permissions grant Everyone only Read. Both sets of permissions apply to network access, so the user gets only what both allow: read. This is misconfigured permissions, and the usual fix is to set share permissions broadly and control access with NTFS permissions. Other causes produce the same symptom, so rule them out.

• Improperly configured firewall rules blocking SMB on port 445 (test the port as in Lesson 35).
• Improperly configured local or group policies. The gpresult command displays the Resultant Set of Policy, the settings actually applied to a user and computer, and /r gives a summary.
• Data loss prevention blocking a legitimate copy (Lesson 26). The DLP console records which policy acted, and why.
• On Linux, the mode bits and owners, or an SELinux label, as in Ticket 2.

Ticket 2: an application that will not load

The web application's files were moved from an administrator's home directory into /var/www/html on Rocky Linux, and the server now returns access-denied errors. The audit log holds AVC denials, SELinux's record of refused access. Red Hat's troubleshooting guide finds them with ausearch -m AVC, explains them with sealert, and names this exact case as a common labeling problem. Files moved with mv keep their old label, here the type used for home directories, so the web server is not allowed to read them.

The fix is restorecon -R -v on the directory, which applies the correct labels. Red Hat suggests switching briefly to permissive mode with setenforce 0 only as a test: if the problem continues while SELinux merely logs, something else is the cause. It also warns against reaching first for audit2allow to write new policy. Disabling SELinux is not a fix; it removes a layer of defense to hide a labeling error.

On Windows, applications that will not load often run into UAC instead. Microsoft explains that both standard and administrator users run applications in the security context of a standard user by default. An administrator receives two access tokens, and uses the full one only after elevating. A tool that fails with "access denied" for an administrator may simply need to be run elevated. Anti-malware configurations cause the third common version of this ticket: a quarantined program file, or an overly broad exclusion. Check the quarantine and the detection name, restore only what the vendor confirms is a false positive, and never solve it by excluding a whole drive.

Ticket 3: a file integrity alert

AIDE (Lesson 27) reports that the SSH server binary changed overnight, and no updates were installed. File integrity tools work through checksums: compute a file's hash, store the known-good value, and compare later. On Linux, sha256sum computes and checks SHA-256 digests. On Windows, PowerShell's Get-FileHash computes a file's hash. Compare the changed file with the vendor's published hash, or with a copy from trusted installation media. File integrity tools escalate in four stages. Monitoring watches files, detection flags changes, alerts tell people, and enforcement can block or revert unapproved changes. An unexplained change to a security-critical binary is treated as an incident: isolate the server, preserve evidence, and escalate (Lesson 26).

Ticket 4: an unknown listener

A port scan of the lab network from another lab host, Nmap's job, shows TCP 4444 open on the web server. On the server, ss -tulpn (or netstat -ano on Windows) maps the port to its process. The tickets in this category sort into a few kinds of service and process.

The listener on 4444 was rogue: a reverse shell left by a compromised plug-in. A sniffer confirms what it talks to; tcpdump's manual describes it as printing a description of the contents of packets on a network interface. Anti-malware and antivirus scans then find the files: Microsoft Defender on Windows, or ClamAV, an open source anti-virus toolkit, on Linux. If a rogue process keeps returning after reboots, find its persistence. Microsoft's Sysinternals Autoruns shows what is configured to run during startup and sign-in. Remember the IDS too. If intrusion detection configurations had alerted on the new listener or the outbound shell, this ticket would have arrived on Tuesday instead of Friday, so review which rules are disabled or suppressed.

Ticket 5: too much access

The contractor was added to Domain Admins "temporarily" six months ago. That is both improper privilege escalation, rights granted outside the approval process, and excessive access, rights beyond what the work needs. The fix is Lesson 25's: remove the membership, grant a narrowly delegated role, and audit the group changes (event 4728) so the next "temporary" grant raises an alert. On Linux, the equivalent finding is a sudoers rule that lets a group run every command as root.

In short: security problems look like ordinary failures, and ordinary failures can look like attacks. Read the evidence (permissions, policies, labels, hashes, processes and logs) before deciding which one you have, and preserve it if you might be wrong.

Security tools, collected

Exam readiness: from course to test

You have now met every SK0-005 objective. Lesson 1 described the exam: up to 90 questions in 90 minutes, multiple choice plus performance-based questions (PBQs), with 750 needed on a 100 to 900 scale. The domains are weighted 18 percent hardware, 30 administration, 24 security and disaster recovery, and 28 troubleshooting. A sensible final plan has five parts. It cannot guarantee a result, but it puts your effort where the exam does.

• Check the version. Confirm on CompTIA's Server+ page which exam you will sit before booking. A draft SK0-006 exists (Lesson 1), and the objectives change between versions.
• Rate yourself against the objectives. Score each of the 25 objectives from 1 (shaky) to 3 (confident), and spend review time in proportion to both the weakness and the domain weight.
• Practice PBQs in the lab. PBQs ask you to do things: place components, choose RAID levels, read logs, configure settings. Your VMs are the best rehearsal. CompTIA's PBQ explainer says they can be skipped and revisited, so do not let one consume your time.
• Read for the qualifier. Many questions ask for the best, first or most likely answer. Troubleshooting questions usually reward the method from Lesson 31: scope and gather information before changing anything.
• Rehearse under time. Take this course's final exam and the lesson quizzes again without notes, then review every miss against the lesson's sources.

Common misconceptions

• "If NTFS says Modify, the user can modify the file over the share." Share permissions apply too; the user gets only what both allow.
• "Disable SELinux and the app will work." Fix the label or the policy; Red Hat uses permissive mode only as a brief test.
• "Administrators always run with full rights in Windows." UAC runs their apps as a standard user until they elevate.
• "A zombie process is malware." It is a dead process its parent has not cleaned up; a rogue process is the real threat.
• "A port scan of a partner's servers is harmless research." Scan only systems you own or have written permission to test.
• "Excluding the application's folder from anti-malware is a safe fix." Restore only confirmed false positives, with the narrowest exclusion.

What you now know

• File and share access: share and NTFS permissions both apply; also check firewall rules, gpresult, DLP and Linux labels.
• Applications that will not load: SELinux denials (ausearch, sealert, restorecon), UAC elevation and anti-malware quarantine.
• File integrity: checksums with sha256sum or Get-FileHash, and monitoring, detection and enforcement; unexplained changes are incidents.
• Processes and ports: active, inactive, orphan and zombie services or processes versus rogue ones; find persistence and review IDS rules.
• Access: remove improper escalations and excessive rights, and audit group changes.
• Use port scanners and sniffers only with authorization; then prepare for the exam by objective, by weight and by timed practice.

Sources

1. Red Hat. (n.d.). Using SELinux, RHEL 9: Troubleshooting problems related to SELinux. docs.redhat.com
2. Microsoft. (n.d.). How User Account Control works; gpresult; Autoruns; Get-FileHash. UAC; gpresult; Autoruns; Get-FileHash
3. Lyon, G. (n.d.). Nmap reference guide. Nmap Project. nmap.org; The Tcpdump Group. (n.d.). tcpdump(1). tcpdump.org
4. Cisco Talos. (n.d.). ClamAV documentation. docs.clamav.net; Linux man-pages project. (n.d.). ps(1) and sha256sum(1). ps; sha256sum
5. CompTIA. (n.d.). CompTIA Server+; Performance-based questions explained. Server+; PBQs explained