OPS Building Operations and Service Management

The operations manager shall define minimum staffing per shift and per discipline (electrical, mechanical/thermal, controls/BMS, NOC/IT infrastructure, security) derived from an emergency task analysis that lists the time-critical manual actions in each EOP (for example manual generator start or transfer, CDU failover, leak isolation, load shedding) and compares their completion time with the time-to-impact from MCF-DEP-003. The operations manager shall roster to that minimum, record every shift that falls below it and the compensating measures, and shall define on-call response times for specialists not on site.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, EMERGENCY, MAINTENANCE.

The operations manager shall maintain a responsibility and authority matrix for every Required Function and material dependency across NORMAL, MAINTENANCE, EMERGENCY and RECOVERY states, naming the role and current holder (and deputy) for operation, maintenance, monitoring, change approval, alarm response and emergency decisions (for example authority to transfer to generator, shed load, power-cap or drain a cluster, isolate a cooling loop, or invoke an EOP). The matrix shall be consistent with GOV-02 accountability and with provider shared-responsibility records (MCF-DEP-018).

Evidence required: E1 Documentary and declarative; E2 Configuration and technical state. Optional: E3, E4, E5. Operating states: NORMAL, MAINTENANCE, EMERGENCY, RECOVERY.

The shift lead shall conduct and record a structured handover at each shift change using a defined template covering: current operating state per system and redundancy level, active and standing alarms, open permits to work and isolations, temporary configurations and overrides, work in progress and planned work, deferred maintenance with risk, provider or tenant events, and expected events (deliveries, tests, high-load periods). Outgoing and incoming leads shall both sign, and the incoming lead shall verify critical items at the BMS/EPMS/NOC consoles.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, MAINTENANCE, DEGRADED.

The operations manager shall maintain a competence framework defining required knowledge, skills and assessment for each task class, and an authorisation register recording each person's authorisations (for example authorised person and competent person for HV/LV systems, refrigerant handling certification where legally required, liquid-cooling and CDU work, BMS/EPMS engineering), their scope, expiry and assessor. The operations manager shall verify contractor competence and authorisation before work and shall ensure permits (MCF-OPS-007) are issued only to authorised persons.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, MAINTENANCE, EMERGENCY.

The operations manager shall maintain a controlled procedure set comprising SOPs for routine operation, alarm response procedures for critical alarms, and EOPs for each credible emergency scenario (including utility loss, generator failure to start, UPS failure, cooling loss at sustained AI load, CDU or secondary-loop failure, liquid leak, fire alarm and suppression interfaces, BMS/EPMS loss, and loss of NOC). Each procedure shall have an owner, technical reviewer, version and review date, shall be validated against the as-built system by walkthrough before issue, and shall be available at the point of use including when the BMS and IT systems are unavailable.

Evidence required: E1 Documentary and declarative; E2 Configuration and technical state. Optional: E3, E4, E5. Operating states: NORMAL, MAINTENANCE, EMERGENCY, RECOVERY.

The maintenance manager shall require an approved MOP for every non-routine task on critical infrastructure, including: scope and affected assets; the operating state and redundancy level during each step (for example N during work on one of N+1 UPS modules); preconditions (for example job checkpoint or workload drain agreed with the platform owner, spare on hand, weather); step-by-step actions with hold points and expected indications; isolation and LOTO points; back-out steps; communications plan; and approvals by engineering, operations and change control (MCF-OPS-018). The MOP shall be executed as written with each step signed off and deviations stopped and escalated.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: MAINTENANCE, TRANSITION.

The facility operator shall operate a permit-to-work system for hazardous work, including electrical (HV and LV), battery and DC systems, hot work, confined spaces (for example tanks, pits, trenches, plenums), work at height, pressurised and liquid-cooling systems, refrigerants and fire-suppression systems. Each permit shall identify the work, hazards, isolations (cross-referenced to LOTO), precautions, issuer and receiver (both authorised under MCF-OPS-004), validity period, and cross-references to other live permits in the same system. Permits shall be suspended at shift change unless re-validated, and closed only after hand-back confirms the system is safe to return to service.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: MAINTENANCE, EMERGENCY, NORMAL.

The facility operator shall define LOTO procedures for each equipment class that identify every energy source (including generator, UPS and battery back-feeds, BESS, capacitors, CDU pumps, pressurised secondary loops and facility water), the isolation points, locking devices, and the method to prove isolation (test-before-touch with a proven tester, pressure and temperature verification). Each worker shall apply a personal lock, group lockboxes shall be used for multi-worker jobs, and restoration shall follow a verified sequence. The facility operator shall audit LOTO practice at the profile interval.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: MAINTENANCE, EMERGENCY.

The maintenance manager shall define and approve a maintenance strategy for each critical asset class (including MV/LV switchgear and protection, transformers, UPS and batteries, generators and fuel systems, busways and PDUs, chillers, CRAH/fan walls, CDUs and secondary loops, pumps, cooling towers or dry coolers, BMS/EPMS, fire detection and suppression, and IT infrastructure hardware) stating tasks, intervals, basis (OEM, recognised practice, reliability analysis), and the operating state and redundancy level during each task. The capability owner shall confirm that each maintenance configuration leaves capacity sufficient for the declared load, or that maintenance windows are coordinated with workload reduction.

Evidence required: E1 Documentary and declarative; E7 Derived and analytical. Optional: E3, E4, E5. Operating states: MAINTENANCE, NORMAL.

The maintenance manager shall operate condition monitoring for critical assets using techniques appropriate to their failure modes, for example infrared thermography of electrical connections, partial discharge monitoring on MV equipment, dissolved gas and oil analysis on transformers, battery impedance or conductance and temperature monitoring, vibration analysis on pumps, fans and chillers, coolant quality monitoring (conductivity, pH, inhibitor and biocide levels, particulates) on liquid-cooling loops, and filter differential pressure. Each technique shall have defined thresholds and response actions, and exceedances shall generate work orders.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, MAINTENANCE.

The maintenance manager shall maintain a calibration register for critical instruments, including sensors used in BMS/EPMS/CDU control loops and alarms, protection relays (secondary injection testing), revenue and sub-meters, leak detection, and portable test equipment, stating interval, method, traceability and tolerance. Out-of-tolerance results shall trigger an impact assessment of decisions and evidence that relied on the instrument since its last good calibration.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, MAINTENANCE.

The maintenance manager shall record every deferral or overdue item of maintenance on critical assets in a deferred maintenance register with reason, risk assessment, compensating measures, approver with authority, and expiry date. Deferrals beyond the profile limit shall require re-approval by the accountable owner. Deferred safety-critical maintenance shall be reflected in the operating state (for example the system declared DEGRADED) and in handover (MCF-OPS-003).

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, DEGRADED, MAINTENANCE.

The control-system owner shall maintain an alarm philosophy and master alarm database for in-scope monitoring systems in which each alarm has a documented cause, consequence, priority (derived from consequence and time-to-impact per MCF-DEP-003), setpoint basis, and response procedure. The control-system owner shall measure alarm performance (alarm rates per operator in normal and upset conditions, standing alarms, chattering alarms, shelved or suppressed alarms) at the profile interval, act on exceedances, and control alarm suppression and shelving through MCF-OPS-019.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, DEGRADED, EMERGENCY.

The operations manager shall operate an incident management process that classifies incidents by Service Outcome impact and safety, defines escalation timescales and authorities, establishes a joint bridge between facilities, NOC, platform and provider teams for cross-domain incidents, links to EOPs and to RES-06 crisis command, records a timeline, and requires a post-incident review for incidents above the profile threshold.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: DEGRADED, EMERGENCY, RECOVERY.

The operations manager shall require a root cause analysis using a recognised method for each incident or near miss above the profile threshold, completed within the profile time, identifying direct, contributing and root causes (including procedure, competence, alarm, design, supplier and organisational factors), propagation beyond expected containment, and corrective and preventive actions with owners. RCAs of provider-caused events shall be obtained from the provider and assessed.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, RECOVERY.

The operations manager shall track problems and corrective/preventive actions to closure, shall verify effectiveness of closed actions by evidence that the failure mode has not recurred over a defined period or by test, shall maintain a known-error record with workarounds for unresolved problems, and shall review recurrence trends at the profile interval.

Evidence required: E2 Configuration and technical state; E3 Operational and observational; E4 Validation and test. Optional: E5, E6, E7. Operating states: NORMAL.

The configuration manager shall maintain an approved configuration baseline covering, at minimum: as-built single-line diagrams and P&IDs; protection settings and coordination study revision; BMS/EPMS/CDU setpoints, sequences of operation and PLC logic versions; UPS, ATS and PDU settings; firmware and software versions of facility and IT infrastructure; network and fabric configurations; storage and cluster-manager configuration. The baseline shall be versioned, access-controlled, reconciled to the live state at the profile interval and after each change, and backed up outside the fault domain it describes.

Evidence required: E1 Documentary and declarative; E2 Configuration and technical state. Optional: E3, E4, E5. Operating states: NORMAL, MAINTENANCE, RECOVERY.

The change manager shall operate a change process that requires, for each change to in-scope infrastructure: classification (standard, normal, emergency) with criteria; impact assessment that identifies affected Required Functions using the asset-to-function and dependency maps, redundancy and fault-domain exposure during the change, safety implications, and workload coordination (for example checkpoint or drain); a linked MOP for non-routine work; approval by a change authority with both facilities and IT representation for cross-domain changes; and conflict checking against other scheduled changes and maintenance. Emergency changes shall be retrospectively reviewed.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: MAINTENANCE, TRANSITION, NORMAL.

The shift lead shall record every temporary configuration of critical infrastructure in a temporary configuration register with description, reason, risk assessment, approver, expiry, compensating measures and removal verification. Temporary configurations shall be visible at the operator console where technically possible, included in handover, reviewed at each expiry, and any extension shall be re-approved. Safety-system overrides (fire, leak, gas detection, emergency power off) shall require approval at the level set in the capability profile.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: MAINTENANCE, DEGRADED, EMERGENCY, NORMAL.

The change manager shall require each material change to define back-out criteria, a back-out procedure tested or rehearsed where practicable, and post-change validation (functional checks, telemetry review, alarm review, redundancy restoration check, and for IT infrastructure job or health checks) with evidence attached before closure. Failed validations shall trigger back-out or an incident, and post-implementation review shall be held for failed or backed-out changes.

Evidence required: E2 Configuration and technical state; E3 Operational and observational; E4 Validation and test. Optional: E5, E6, E7. Operating states: MAINTENANCE, TRANSITION, NORMAL.

The capacity manager shall review at the profile interval rated, available, reserved, consumed and headroom capacity for each capacity-bearing element (utility contract, transformers, UPS systems, busways and PDUs, generator capacity, chiller and heat-rejection plant, CDU and secondary loops, air cooling, floor loading, network ports and optics), considering N and N-1 (maintenance) states and transient AI load behaviour (for example synchronous training power swings). New deployments shall pass a capacity admission check against these values before installation.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, MAINTENANCE.

The operations manager shall define and measure indicators for each Service Outcome, with measurement method, source, and data quality checks, shall display them to NOC and BMS operators in an integrated view, and shall report performance against tolerance at the profile interval.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, DEGRADED.

The capacity manager shall define placement rules and balance limits (phase imbalance, per-PDU and per-UPS load, per-CDU flow and heat load) for in-scope infrastructure, shall monitor balance continuously or at the profile interval, and shall coordinate with the platform owner on workload controls (scheduler placement, power capping) so that facility limits are respected. Imbalance or exceedance shall be corrected through change control.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL, MAINTENANCE.

The operations manager shall issue operational reports to accountable owners at the profile interval containing the listed indicators with data sources and lineage, trends and exceptions, and shall record decisions or actions arising.

Evidence required: E2 Configuration and technical state; E3 Operational and observational. Optional: E4, E5, E6. Operating states: NORMAL.