Co-location and Condos
MARCC Condos and Colocation Guidelines:
Purpose: MARCC provides colocation services at its facility at 5400 E. Lombard St, Baltimore Md 21224. The facility is highly secure and has fast connectivity for housing mission critical servers. This document describes guidelines and requirements for physical hosting of computer equipment in a shared facility for the benefit of the Johns Hopkins and UMCP research communities. The colocation service will reduce costs of designing, building and maintaining HPC infrastructure.
Physical security: There are closed circuit video cameras recording all activity inside and outside the data center. Only authorized personnel with card access are allowed in the facility and the data center. Guests are allowed in the facility per request in advance to the director of MARCC.
Cybersecurity: Customers must follow best industry practices to ensure the proper functioning of the data center. JHU IT/Network and the security office will coordinate security, firewalls and access rules. Customers should utilize the Hopkins firewall as much as possible. All exceptions must be clearly justified and documented.
Racks: All servers must fit on standard 42U racks (78.74in x 23.62in x 43.30in). MARCC will provide racks, network and power trays and cables based on a cost recovery model. Contact MARCC at email@example.com for more information on cost.
Rack Space: Customers will be allowed to deploy equipment that has been approved by MARCC. Customers are expected to utilize only the rack space that is allocated to them and respect space allocated to other users.
Power: Equipment placed at MARCC must be compatible with the standard power and cooling provided by the facility. The colocation space is powered by BGE. MARCC provides generator backup power. Specific information regarding equipment power requirements is required by MARCC. Customers should provide information on the power consumption (idle, peak and standard loads) as well as the plug type. Each rack should consume no more than 12 kW/H (average 300 Watts per server). Customers with high density racks (about 25 kW/h) should consult with MARCC about additional service charges. The standards are: 110V/120V or 208/220V, 30 amp circuits.
Hardware: All hardware to be housed at MARCC should be under warranty and should not be older than 5 years. The equipment should be able to function properly for a cooling set point of 75F (+/-3°).
Cooling: The facility has adequate cooling (N+1 redundant) to ensure continuous operation. However, customers are strongly encouraged to follow these guidelines to ensure the proper cooling environment is maintained at all times.
Cables: Should rest on cable trays (provided). There is no under the floor space for cabling. Cable management should be done in accordance with local, state and national electrical, fire and safety standards and law.
Network: MARCC has a redundant 10Gb/s connection to Internet2, and 100Gb/s connectivity to the Homewood, East Baltimore and to UMCP campuses. Connections to different research buildings vary according to the Hopkins Research Network (HORNET). Customers are required to purchase network hardware to connect to the MARCC network via 10Gb/s or 40Gb/s. Customers should also plan for network technology refresh cycle of about 5 years. All network switches, routers, firewalls will be managed by IT@JH. Customers should discuss with MARCC and JHU IT any network needs, security and options to implement them.
Access: Remote access, as well as physical access, to the facility is provided for authorized personnel. Automated monitoring for environmentals and security is available on a 24×7 basis.
Inventory: Customers agree to provide all necessary information about their equipment to keep proper inventory. This includes hardware type, power and cooling requirements, weight.
Warranty: All equipment should have at least a five (5) year warranty or maintenance contract. Equipment can be operated up to five (5) years and customers should plan for refresh cycles. Customers must provide proper information when servers are replaced.
Safety: Customers must follow industry standards to safely operate equipment. Any safety issues or concerns should be reported to the director of MARCC.
End of Life: Customers must replace equipment that is over 5 years old or opt to donate it to MARCC. Proper procedures should be followed to surplus old equipment. It is the PI’s responsibility to surplus the old hardware or move it to another location.
Server Administration: Professional administration is required. Servers that are maintained by students will not be allowed in the colocation area. If there is no professional administration, MARCC might be able to provide system administration for a fee, if resources are available. This service will depend on the size and complexity of the system. An MOU will be provided.
Support: MARCC will provide initial support for deployment, installation, network access, facility access.
Cost: There is no cost to the customer for housing servers or utilities (the respective customer’s schools will pay for space and utilities). The only cost to the PI will be for common infrastructure: racks, cable trays, electrical work, and any other needs to be determined after discussions between customers, IT@JHU and MARCC.
Conflict: The Deans (vice-dean for Research and Sr. Associate Dean for finances) will be resolving any potential conflicts. However, every customer will agree and sign an MOU that describes all requirements and responsibilities. This document will drive this process.
Requests: Please contact MARCC at firstname.lastname@example.org for more information or to start this process.
MOU: Every customer will agree and sign an MOU that describes requirements, responsibilities and fees according to the hardware to be housed by MARCC.
Audits: All hardware must pass security audits provided by IT@JHU.
As of June 29,2020 these additional policies apply:
- Highest priority for space will be given to large shared clusters like Bluecrab. The bigger the user base the greater the priority.
- Priority for collocation space should be higher for larger clusters that cannot be placed in other locations or have a compelling need for communication to other clusters at MARCC.
- Machines placed at MARCC will be allowed to stay there for the lifetime of their service contract or 5 years from new purchase date – whichever is shortest.
- Machines that are out of warranty or over five years old will be allocated space on an annual basis and owners will be responsible for removing them if the space is needed for higher priority use.
- Provide adequate cyberinfrastructure (space, power and cooling)
- Provide basic support for initial setup
- Conduct an audit of the equipment to be collocated. Start inventory.
- Provide restricted physical and virtual access to the facility
- Racks are metered to keep track of power and cooling consumed. MARCC will provide detailed information on power consumption on a quarterly basis to schools
- Approve equipment, if necessary
- Provide power backup
- Communicate with sufficient time scheduled downtime and systems affected to the systems administrators responsible for their management
- Promptly communicate with (best efforts) system administrators about unplanned events that may impact the standard functioning of servers in the colocation area
- Provide equipment (hardware) with warranty or maintenance support for at least five (5) years. Any equipment 5+ years old will need to be replaced or surplussed
- Pay for work that needs to be done to have equipment functioning. For example, power whips connections, any network fees, etc.
- Administration. Professional administration is required. If professional administration is not available, MARCC might be able to provide it at a cost depending on the size of the system. Students performing systems administration is not allowed.
- Implement security measures as needed (discuss with MARCC and IT networking/security).
- Provide and setup any firewalls (discuss with MARCC and IT networking/security).
- Arrange with the IT network group for IP addresses or domains.
- Provide serial numbers to MARCC to be added to the inventory list.
- The group in charge of the system administration is responsible for providing secure access to any user of these resources.
- Provide power and cooling information to MARCC staff with respect to any servers to be added to the colocation area.
- Request any necessary quotes from vendors. Use the MARCC address for delivery: 5400 E. Lombard St. Baltimore Md 21224
- Provide tracking numbers to MARCC staff and ETAs for delivery
- Process all necessary information to be submitted to Fixed Assets
- Submit shopping carts. Please note that carts may be rejected if there is any missing information
- Pay for utilities (power and cooling) used by the equipment located at the data center. Schools will be billed on a monthly basis.
PIs are strongly encouraged to procure funding to integrate resources (condos) to the main cluster. Condos may consist of compute nodes, GPU nodes, sub-racks, management and Infiniband switches, and any necessary software licenses. PIs should discuss configuration with the MARCC director in order to maintain a homogeneous system, as allowed by advances in technology. There will not be a charge to add condos to the existing shared cluster (Bluecrab) but in exchange for the administration of the condo PIs will allow the use of their idling nodes. PIs will have an allocation in walltime hours equivalent to their purchase. Any condo user should be able to use as many cores as possible, up to their allocation. PIs should purchase hardware with a minimum 5-year warranty. Hardware will be kept for a period of five (5) years and will be placed on surplus if it cannot be easily repaired. Schools will cover the additional cost of power and cooling based on utilization.
Please contact us directly for more information on condos and co-location requests.