“CROs aren’t running your methods, they are running their interpretation of your methods.”
This comment, made to me by an accelerator group, rings true for many of us who have partnered with CROs. This observation might seem benign on its face — after all, why shouldn’t a CRO bring their experience to your methods? However, in reflecting on my previous experiences working with CROs, where I spent a lot of time managing relationships, strategy, and tactics, and became well acquainted with the benefits, challenges, and costs of those relationships, I’ve come to realize that third party interpretation of your methods lies at the heart of the many hidden costs associated with CROs. At first blush, these relationships look very straightforward: scope project, transfer materials, receive requested data/materials, rinse and repeat. However, these interactions are much more complicated (and expensive) than they may appear when you take a more complete and holistic look at the process.
There are a great number of use cases for bringing on a CRO partner — when you need scientific expertise in an area your company has no experience with, when you need capacity to deal with a short term spike in work, or when a technique has become standardized and commoditized in the industry. CROs are not a silver bullet, however, and there are other use cases where CROs are used in ways that can be highly inefficient for organizations. For example, many companies end up using CROs to solve short term capacity problems for critical internal work, never adjusting internal resourcing to meet the new level of demand. Part of this inefficiency is certainly the internal business complexity of research and development organizations, but a large part is that many of the costs of engaging with CROs are either hard to estimate or not obvious until after you start the external work. From my experiences in outsourcing analytical work across a number of external organizations, drug modalities, and engagement strategies (fee for service, external FTE models, and insourcing), I have outlined the four categories that I observed as most impactful and frequently overlooked below, with examples to help give context.
- Hidden Cost #1: Management and Oversight
There is often a significant gap between the expected time cost of managing work at an external partner and the reality of it. When I used to write business cases to evaluate internal and external analytical operations, it was easy to account for lab execution, materials, and deferred capital expenses. It was much harder to account for the amount of time your staff will need to spend on meetings, site visits, tech transfers and technical support. With a sufficient volume of external work it becomes cost effective to hire a dedicated professional to manage the external work, but far more often this responsibility is tacked on to a scientist’s existing role.
The set of skills required to effectively manage a CRO relationship (project planning, communication, operations) presents a challenge in these cases, as they often represent a new domain that scientists have to stretch into and are disparate from the scientific expertise and technical skills for which they were hired. In the most extreme cases, scientists are left juggling internal technically-focused responsibilities with external CRO project management. This can have several downstream impacts on timing, effectiveness, and cost of outsourced work and make the efficiency of outsourcing more perception than reality.
Hidden Cost #2: Scientific Communication
One of the critical lessons I learned early in my time managing work at CROs is that while the science itself may remain constant, every organization talks about and documents their science differently. A well established method may have been transferred across internal technical staff and even to a quality team without errors or questions, but when going through transfer to a CRO, redlines and questions will invariably come up.
Here is an example: A protocol step specifies to filter a sample with a 0.22µm, 13mm disc filter. While this may sound completely specified, the CRO you are working with might have several filters matching that criteria from different manufacturers and made of different materials. Is it a PTFE, PVDF, PES, or Nylon filter? Does it matter if it’s from Whatman or Millipore? This type of omission sounds surprising until you consider the context. The group that developed the method has only one type of filter in their lab, and part of the protocol is to write down the filter product information during execution of that step, so there was no reason for additional information to be included in the method documentation. Fortunately, this is a straightforward gap to address by adding the additional information to the method documentation. Unfortunately, the scale of these types of requests for additional information can frustrate your scientific team and create delays in completion of critical work.
Additionally, many of these gaps are not as straightforward to address. What if the CRO asks how you filter the sample? The process of answering this question could uncover a whole host of unspecified experimental parameters: is it filtered using a pump or pressure? What type of pump? What is the flow rate? Is the filtering temperature dependent? These questions can open new lines of inquiry requiring even more comparability experimentation to set parameters. In this way, a relatively simple and straightforward instruction can require a significant amount of time and energy to resolve and get moving. In the worst case, the gap doesn’t get noticed at all, and the CRO makes an incorrect assumption that materially impacts the results and, because it is such a small detail, goes unnoticed for months or years.
Hidden Cost #3: Sample and Data Acquisition
CROs use different types of sample accessioning forms and have lab information systems that range from Excel and paper lab notebooks to web portals and ELNs for accessing sample results. The key considerations for cost stem from how your organization can interface with a CRO’s systems to send samples out and get data back. This leaves you stuck with the decision of spending a lot of time with formatting and data management for each interaction or investing in logistics and IT infrastructure to make each interaction less painful, but which dramatically increases the effective price in terms of your internal resource commitments and raises the switching costs, reducing flexibility to negotiate or consider other partners in the future.
In my experience, CROs can have rigid sample accessioning procedures, requiring digital copies of the sample form, in addition to copies placed in the box with the samples. This rigidity forces most clients to adapt their own internal sample handling processes to the CRO. Your options are to either take the hit on the time spent to do it manually and have someone transcribe information from your system to their form, or allocate resources to create a report formatted to meet the CRO’s needs in your own system. The actual cost comparison can be pretty easy, but these gaps are usually overlooked until you start working with the CRO, and can leave your group without necessary resources and budget to fix the problem.
CROs can be quite flexible in delivering results back to you, but ancillary data from the experiment is often unavailable or in a static pdf copy of the notebook used. In particular, things like raw instrument output, environmental condition trends, and equipment maintenance records are obtainable, but are typically a manual process on both ends and reserved for investigations of unexpected results. Over time, not having direct access to this data can be detrimental in the sense that the knowledge gained from experimentation is lost. As companies continue to become more savvy in organizing and using their experimental data for more complicated analyses and AI/ML tools, access to this detailed information becomes more important. Similarly to the sample accessioning problem, you could invest in technical solutions that would procedurally gather and organize this type of data, but it would require significant effort and cost from both organizations’ IT departments to set up and maintain, something large CROs have been particularly reluctant to do.
Hidden Cost #4: Setup and Troubleshooting
In many cases, time lost to going through tech transfer and troubleshooting when something goes wrong can be the most significant cost when working with a CRO.
Tech transfer can be a relatively burdensome process, even for well established, robust methods. This process typically involves initial review of the method through a series of meetings with your SMEs, followed by an on-site training session, qualification runs to ensure things are working properly, and continued follow up as needed. It is also likely that retrainings will need to occur, as staff turnover at CROs might mean that the person currently running your method might be several “generations” removed from the person trained by your SME. All of these activities are additional time your team is not working on new or more pressing internal matters, and can delay your project or program timelines.
Troubleshooting with a CRO can be a very different experience than troubleshooting in your own lab, and has significant implications to both FTE costs and timelines, for all the reasons mentioned above. Communication becomes more sluggish and difficult due to organizational norms and the fact that you are trying to isolate a problem over the phone or Zoom. With delays in getting the data or information you need, or executing follow up experiments to gather more information, this can stretch out much longer than expected.
The common theme to these hidden costs is a lack of clear, reliable communication.
There are two factors that make the management, communication, sample and data acquisition, and setup and troubleshooting of work with a CRO so difficult to uncover. The first is that none of them are obvious at the outset of a work engagement, and arise as work is being done. Secondly, the effects can compound, leading to much larger than expected overruns in cost, time, or both. For example, it may be reasonable for a scientist to manage a small study with a CRO, in addition to their in-house lab responsibilities, when everything is going well with the project and methods. However, if there are minor gaps in setting up reliable sample acquisition, and a few missteps in setting up the method, that scientist’s time can quickly be consumed by managing the external partner. For this reason, it is especially important to be realistic and cautious with expectations of time and/or cost savings moving to a CRO.
One of the primary reasons I joined Emerald Cloud Lab is that the remote controlled cloud lab model sidesteps some of the challenges outlined above. ECL connects scientists directly to the lab equipment and, ultimately, their experiments, eliminating the need for additional management or oversight. ECL Command Center, which includes all the features of ELN or LIMS, works with you to ensure that all experimental parameters are defined. It also validates that your experiment will run before anything is started in the lab, confirming that there are no incompatibility issues or reagent/consumable supply problems, largely eliminating the communication problem of working through another person. Setup and troubleshooting also become easier, as you can directly see all of the data associated with a run, and have direct, 24/7/365 access to its on-call systems and instrument teams through the software itself.
As mentioned previously, this is not to say that there are not excellent use cases for working with CROs. When managed effectively and used for the right projects, CROs can be an effective partner, allowing you to leverage your existing team to a greater extent. However, not every use case is right for outsourcing, and it pays off to be more diligent and exhaustive in development of any business case or financial model evaluating external operations.
If you are interested in learning more about any of these topics, feel free to reach me directly at toby.blackburn@emeraldcloudlab.com.
