# Contract manufacturing organization - Case study ## Leading CDMO prevents batch failures and saves $1 million with OpManager With over 5,000 employees and a global presence, this organization is one of the largest Contract Development and Manufacturing Organizations (CDMOs), partnering with biopharmaceutical and biotech companies to manufacture medicines, advanced therapies, and vaccines. The company is committed to a culture of trust and prioritizes exceptional customer experience and timely delivery to meet the needs of its clients. **Industry type** Pharma manufacturing **Experience in Industry** 15+ years **Employees** 5000 ## A closer look at the production process The production process involves receiving detailed drug formulations from leading pharmaceutical firms to manufacture medicines for the market. The organization follows a batch manufacturing process that involves a workflow of multiple steps including gathering raw materials, mixing ingredients, processing the mixture, conducting quality testing, and shipping the product to the packaging facility. This entire manufacturing process is highly automated, with Programmable Logic Controllers (PLCs) managing and controlling each stage until a batch of medicine is successfully completed. To establish optimal environmental conditions crucial for the manufacturing process, the organization relies on a Building Management System (BMS). The BMS monitors and manages critical factors such as temperature, humidity, and airflow, ensuring compliance with regulatory standards throughout production. Additionally, the company employs a Manufacturing Execution System (MES) to track the supply, quality, storage, and utilization of raw materials. These systems are integrated to ensure operational efficiency and maintain the highest standards of quality in the pharmaceutical manufacturing process. ## IT infrastructure: The backbone of production The IT infrastructure is the foundation of the production cycle. The organization's IT encompasses seven data centers spread across the United Kingdom, United States, and Denmark. The servers in these data centers support the collection and storage of all production and quality control data generated during the manufacturing process. IT teams monitor the batch manufacturing process through applications hosted on these servers, ensuring that operations run smoothly and efficiently. The company has deployed Cisco UCS in its data centers for server needs, while for storage requirements, the IT teams utilize devices from leading vendors such as HPE, Dell, NetApp, and Synology. To safeguard its systems and maintain compliance with regulatory standards, the organization has implemented firewalls from Palo Alto. Given the complexity of its multi-vendor, distributed IT network, the company recognized the need for a centralized and robust monitoring solution. The principal engineer at the company shared valuable insights on the transition from other vendors, including SolarWinds, to OpManager. This shift enabled the company to maintain peak performance across its network, achieving 24/7 availability of applications, which is crucial for its business operations. ## Key challenge: Maximizing the availability of Operational Technology The principal engineer highlighted that the company’s network encompasses two separate systems: the Enterprise System and the Operational Technology (OT) system. The Enterprise System includes Tableau servers, HR application servers, and end-user devices like laptops. On the other hand, the OT system comprises mission-critical components such as the BMS, MES, PI System, and PLC machines. The applications connected to the OT category are essential for production, and any downtime could halt operations entirely, resulting in significant financial losses. When asked how an outage could impact its operations, the principal engineer explained, “It depends on the system that goes down. If the Enterprise System experiences downtime for a few minutes, we can manage it. But, when an OT system fails, it triggers organizational panic, as it can halt the production completely, which can cost us millions of dollars.” The key challenge for the CDMO is to ensure that the OT systems are operational and performing optimally 24/7. To address this challenge, the CDMO utilizes OpManager to monitor both environments. ## Navigating the challenge with OpManager Enterprise Edition With multiple data centers located across the globe, the CDMO required a unified monitoring solution to oversee its geographically distributed sites and provide a centralized view of network health and performance. OpManager Enterprise Edition was the ideal fit for its needs. It utilizes a Central-Probe architecture, with Probes deployed to monitor each data center network, which is a combination of routers, firewalls, storage devices, and servers. These Probes gather data, raise alerts for metric violations, and generate comprehensive reports. The data collected by the Probes is periodically sent to a central server installed at the headquarters, which consolidates this information to provide a holistic view of the network. ### Mitigating troubleshooting time Troubleshooting was another major challenge for the organization, particularly when issues like application lag were noticed. The principal engineer frequently faced a challenge that frustrated him: the recurring lag in the plant information system application. Whenever app team members complained about sluggish performance, the IT team sprang into action, determined to resolve the issue swiftly. The app team initially focused its efforts on diagnosing potential network problems, believing that the lag must be a connectivity issue. Hours ticked by, but it was unable to detect the exact cause. Just when it seemed like the team might not uncover the issue, an IT admin decided to check the application server drive—a step originally overlooked. To its surprise, the app team discovered that the drive was nearly full. It took about three hours to identify the root cause before it could add the necessary space to resolve the issue. Reflecting on this experience the principal engineer noted, “With OpManager, we could quickly pinpoint the anomalous metrics, reducing our troubleshooting time from three hours to just thirty minutes.” With the help of OpManager, the organization was able to greatly optimize the mean time to repair value. ### Simplifying network configuration management with the NCM add-on When asked about the Network Configuration Manager (NCM) add-on used alongside OpManager, the principal engineer responded enthusiastically, “It works perfectly; I love it.” The NCM add-on enabled the IT team to easily and efficiently back up 300 switches, a task that previously required an extraordinary amount of time due to complications involving scripting. “Previously, we used scripts to maintain device backups, which could lead to human error. With Network Configuration Manager, we simply enter the credentials for a device, and we can back it up effortlessly,” the principal engineer observed. Applying configuration updates to multiple devices across its network has also become much simpler utilizing the NCM add-on. ## Tangible results The decision to transition to OpManager proved highly beneficial and demonstrated that the investment quickly delivered a positive impact for the organization. OpManager addresses the company's specific challenges by providing: ### Financial benefits: An estimated $1 million in savings With OpManager, the IT team gained multiple functionalities from a single interface at an affordable price, resulting in significant cost savings. The principal engineer noted, “A major benefit of using OpManager is the licensing costs; we now pay one-tenth of what we previously spent on other products.” Monitoring applications that oversee batches is crucial, as any unresponsiveness from the associated application could lead to batch failure. These failures can pose a serious financial threat, with a single incident potentially costing up to $1 million as estimated by the principal engineer. However, the CDMO is now able to efficiently prevent such incidents utilizing OpManager. “With OpManager's alert notifications, we can identify sluggish performance much earlier, prevent crashes, and proactively address issues before they lead to a batch failure,” the principal engineer explained. The troubleshooting costs also decreased significantly with the implementation of OpManager. Before using the tool, the company relied on external consultants to diagnose issues on-site. The principal engineer estimates that hiring a consultant could cost about $150 to $200 per hour, with a minimum of five hours required to troubleshoot an outage, observing: “You can calculate the costs and it comes to at least $1,000 just for the consultant to analyze the situation and identify the issue, which adds financial burden to our organization.” Even when the company did not hire external consultants and sent an in-house employee to troubleshoot issues, it still faced potential productivity losses. The principal engineer explained, “When we sent an in-house employee to troubleshoot, we not only incurred travel costs, but the employee also had to invest a significant amount of time on the road to reach the site and to diagnose the issue. This time could have been spent on more meaningful tasks and strategic initiatives that drive our business forward.” ### Faster adoption rate The user-friendly interface facilitated a quicker learning curve for the team. The principal engineer emphasized that even those not from the IT team found it easy to use: “OpManager was simple to navigate, making it easy for us to train members from application team, and they were able to use it effectively without any struggle.” ### Exceptional support experience Before adopting OpManager, the organization relied on various open-source tools, which often lacked the necessary support. With OpManager, the CDMO experienced timely and effective support services. “We used many open-source tools before, but they offered no support. With OpManager, we receive prompt assistance. Additionally, we also have the option to request features, and they often appear in upcoming releases.” The principal engineer mentioned the Dell VxRail HCI feature as an example, observing, “We were already using the VxRail environment, and when we requested to extend monitoring support for VxRail, it was included in the next major release.” ## OpManager: The trusted partner for contract pharma manufacturers ManageEngine OpManager is a powerful network monitoring software that provides deep visibility into the performance of your routers, switches, firewalls, load balancers, wireless LAN controllers, servers, VMs, printers, and storage devices across a geographically distributed network. It is an easy-to-use and affordable network monitoring solution that empowers you to drill down to the root cause of an issue and eliminate problems quickly. ## Related Products - [Network Monitoring](https://www.manageengine.com/network-monitoring/?relPrd) - [Bandwidth Monitoring & Traffic Analysis](https://www.manageengine.com/products/netflow/?relPrd) - [Network Configuration Management](https://www.manageengine.com/network-configuration-manager/?relPrd) - [Switch Port & IP Address Management](https://www.manageengine.com/products/oputils/?relPrd) - [Firewall Management](https://www.manageengine.com/products/firewall/?relPrd) - [Network Monitoring Software for MSPs](https://www.manageengine.com/network-monitoring-msp/?relPrd) - [IT Operations Management](https://www.manageengine.com/it-operations-management/) - [Application Performance Monitoring](https://www.manageengine.com/products/applications_manager/?relPrd)