President Biden shared an encouraging note this week when he spoke to reporters about increasing the incoming administration’s daily Covid-19 vaccination goal from 1 million shots to 1.5 million shots per day.
This newfound optimism is tempered, however, by the continued rocky rollout of the vaccination program across the nation.
In the face of unprecedented demand (and limited vaccine supplies), state and local public health officials are scrambling to get vaccine shots into people’s arms as soon as possible.
The process has been frustrating all around. Many Americans who rejoiced in getting one of the highly sought after vaccination appointments were crushed to find their appointments canceled because the federal government had missed its forecast for available vaccine deliveries to the states.
Even incoming CDC director, Dr. Rochelle Walensky, admitted there is a major problem. Speaking to Fox News, she said, “I can’t tell you how much vaccine we have, and if I can’t tell it to you, then I can’t tell it to the governors, and I can’t tell it to the state health officials… If they don’t know how much vaccine they’re getting, not just this week but next week and the week after, they can’t plan. They can’t figure out how many sites to roll out, they can’t figure out how many vaccinators that they need, and they can’t figure out how many appointments to make for the public.”
Who Are the Primary Stakeholders in the Nation’s Covid-19 Vaccination Program?
To the casual observer, it may seem incomprehensible that we were able to develop, test, and authorize two Covid-19 vaccines from Moderna and Pfizer BioNTech for emergency use in less than a year but, so far, the process of managing the supply chain—from production to delivering shots into people’s arms— has been marked by frustrating delays.
Why is this the case?
Is it a redux of the 2013 healthcare.gov website rollout that crashed and burned in the days after it was launched with great fanfare by President Obama?
Unfortunately, the analogy has some merit.
As was the case with Obamacare, IT system architects working on system integration projects for the Covid-19 vaccination efforts across the country are facing a similar challenge of how to glue together a large number of decentralized database systems operated by distinct stakeholders, including the pharmaceutical industry manufacturers, at least five major federal government departments, approximately 64 state and local vaccination tracking systems, as well as approximately 255 million Americans over the age of 18.
If you are looking for your TLDR moment, this is it.
The sheer number of decentralized database systems handling protected health information (PHI) across multiple levels of government combined with the challenges posed by manufacturing the first-ever approved mRNA vaccines (which must be maintained at incredibly low temperatures and require two injections several weeks apart for each patient) is the kind of nightmare scenario that would wake up the average IT systems engineer in the middle the night screaming—with good reason!
The only good news in this complicated situation is that this program is single-payer; in other words, the federal government is footing the bill, and thus, there is no need to bring in the added complications that bedeviled the healthcare.gov rollout, such as calculating patient subsidies, insurance plan coverage, and CMS reimbursements.
Structure of the “Operation Warp Speed” Vaccination Development Effort
Let’s take a deeper dive into each of these systems, starting with the structure of “Operation Warp Speed” (or OWS) that was launched in March 2020.
The two goals of the OWS program were to
- provide strong, direct financial backing from the federal government for any pharmaceutical company that showed a promising candidate for Covid-19 vaccines or therapeutics
- fund large-scale production of vaccine and therapeutic candidates (as well as critical items such as storage vials, syringes, PPE, etc.) in parallel to the research and development programs. In other words, the federal government foots the bill for ramping up production even before knowing whether clinical trials would pan out or not.
The federal government engaged McKesson as the chief central distribution contractor –a sensible choice since they had a track record since 2006 of managing the federal distribution of more than 1 50 million doses for influenza, chickenpox, and MMR vaccines.
OWS also brought on Palantir to provide “big data” supply chain tracking, inventory management, and data forecasting tools. In keeping with the warp speed Star Trek theme, the Palantir implementation was christened “Tiberius,” after Starship Enterprise’s Capt. Kirk’s middle name.
The company, founded by billionaire PayPal co-founder Peter Thiel, has not been without controversy, particularly among privacy advocates, who argue that the company’s data collection efforts have gone too far. Nonetheless, Palantir successfully leveraged this and other federal contracts to take the company public in late 2020.
Extraordinary Efforts to Track Deep Freeze Temperatures for Moderna and Pfizer BioNTech MRNA-based Vaccines
Two of the Covid-19 vaccines in development worried public health officials, the ones from Moderna and Pfizer BioNTech.
These concerns multiplied when it became apparent these two vaccines would be the first to receive FDA emergency authorization.
The first concern was that the FDA had never approved an mRNA vaccine before. The technology was new and unproven in human populations.
The second concern was that the two candidates— from Moderna and Pfizer BioNTech— both required maintaining the vaccines in extraordinarily cold subzero temperatures, roughly equivalent to that of dry ice (e.g. frozen nitrogen) – a storage requirement that would be difficult to manage outside major medical facilities.
And the third concern was that both these vaccines required a second inoculation to achieve a full immunization response against the Covid virus, complicating efforts to streamline a national vaccination program.
Nevertheless, plans for creating a sophisticated “cold chain” distribution system proceeded, leaning heavily on logistics giants UPS and FedEx to build cold storage facilities as well as unique freight packaging to adhere to the ultra-cold temperature requirements.
Each freight package was equipped with a so-called “Sentinel” that tracked GPS location, altitude, and internal temperature; the data was broadcast back to Palantir Tiberius, which allowed logistics managers to keep constant tabs on each vaccine package as it was flown or driven from the factories to each of the state’s local cold storage facilities.
States Try to Leverage the Infrastructure of their Existing Vaccination Data Systems
Delivery of the first vaccines to the states began in December 2020.
And, unlike past Federal vaccination rollouts, the Trump administration made it very clear that each state would be responsible for developing and managing its vaccination program, as well as determining the priority lists for who get the new vaccines first.
In response, most states chose to leverage their existing vaccination systems to handle the new Covid vaccination challenges.
All states (except New Hampshire), many major metropolitan areas, as well as US territories, have an immunization registry, which, to make things simple here, we will refer to it as the State Immunization Registry.
Most of these states’ immunization registries work fairly well, meaning they get the job done, but there can be hiccups, especially when the systems are pressed to speed up their data collection and processing times.
And speed has been the critical issue with Covid-19 vaccines.
The public wants these vaccines now, but, unfortunately, most state immunization tracking systems were not designed for real-time reporting.
Let’s take the case of Texas, for example, which has faced a difficult time in the past few weeks.
In the Spring of 2020, as the pandemic began to hit the Lone Star State hard, Texas state health officials were caught in the middle of an ill-timed database software upgrade to their “ImmTrac2” immunization tracking system. Texas has also been hamstrung by a pre-Covid law that made participation in the state’s immunization tracking system optional; as a result, many Texans are not in the system, which only maintains records for those that “opted-in” via a signed form.
The Texas Tribune also reports that once the Moderna and Pfizer BioNTech vaccines arrived, many Texas healthcare providers found their internal EHR (electronic health records) systems could not “talk” to the state’s ImmTrac2 system, forcing some desperate organizations to employ medical students to laboriously hand-code the required data into the state’s immunization tracking system to meet the 24-hour reporting deadline.
If you have ever been involved in a just-in-time inventory implementation, you will know the pain that states like Texas are experiencing right now. The existing system worked fine when there were large inventories of flu, chickenpox, or MMR vaccine sitting in the warehouse that could serve as a buffer to spikes in supply and demand.
But with Covid-19 vaccines, the federal government is trying to push out vaccines as fast as the factories can make them – and they are asking for local governments to report back within 24 hours after each patient has been vaccinated.
And, as we’ll see in the next section, without timely data, the federal government cannot make accurate vaccine delivery forecasts for the states.
Inside the Federal Vaccine Tracking and Ordering Systems
Each of the state immunization registries connects directly to the CDC’s “IZ Gateway.”
It’s a two-way communication channel that allows states to retrieve the vaccination records of citizens who may have already been vaccinated in another state. (Useful for “snowbirds” who may live in the north but spend the winters in Florida, for example.)
The individual patient vaccination data then moves to the CDC’s “IZ Clearing House,” where it’s combined with data available from other sources, such as the Federal Bureau of Prisons or vaccination records sent directly from pharmacies.
The data is next passed to VTrckS (short for Vaccine Tracking System), which provides the data used by McKesson to order vaccine products.
Finally, information from the IZ Clearing House and VTrckS databases is copied (minus any personal health information) into the CDC’s “Data Lake” – an aggregate database where public health officials can manage and track orders.
The Palantir Tiberius system also has direct access to each of these data sources above, allowing both state and federal public health officials access to weekly Covid-19 production and supply forecasts, as well as other “big data” modeling tools that allow officials to perform various “what if” analyses, such as predicting the effect of changing which groups have priority access to the vaccine, or where to best locate a major vaccination hub, which is our next topic.
Getting to 5,000 Vaccination a Day at Vaccine Hubs
Each health department is under tremendous pressure from its local state government and citizens to get their fair share of federally supplied vaccine allotments.
Meanwhile, the federal government is using a carrot and stick approach to get each state to report vaccinations as fast as possible – so that it can make accurate forecasts in the weeks ahead. For example, in some cases, it appears the feds are informally telegraphing to the states that they better “use it or lose it” – in other words, supplies will be redirected away from them to states who are implementing their vaccination programs quicker and not letting stocks sit on the shelves.
For this reason, some states, such as Texas and Florida, are moving to a new vaccination program model, the vaccine hub, which is designed for the maximum throughput of individuals – possibly at the expense of rural populations.
The first generation of these hubs are located in major urban areas, often in the parking lots of major arenas (other large venues such as parks) that have vast parking facilities that can host dozens of drive-through vaccination lanes and stalls. Many of these hubs are trying to work up to immunizing as many as 5,000 people per day.
A major consideration for moving toward these vaccination hubs is the need to speed up the transfer of data from the state immunization registries to the CDC’s IZ Gateway. Unlike many smaller hospital systems, doctors’ offices, and clinics, which have experienced trouble transferring data between systems, the vaccination hubs have been pre-vetted to ensure they can transfer the data correctly and quickly, allowing each state to report data to the federal government to ensure they are in line for the prompt delivery of the next available vaccine supply.
Standing up a new front-end registration system is a major new challenge for many state IT departments, and some have found to their chagrin that some people have been caught cheating, for example, by sharing confirmation codes with their friends and family. As a result, many vaccination hubs are switching to single-use-only barcodes or QR codes commonly employed by major concert venues or sports franchises.
New Federal VAMS Web-based Front End is in Development
Meanwhile, the federal government has recognized that many of the states need help developing a front end system that can step each patient through the process of scheduling an appointment (and a second one for the follow-up shot if needed), recording any adverse reactions, and logging the vaccination data to the state and federal databases.
The VAMS initiative, if successful, will provide states with an alternative pathway to manage vaccination programs. It’s quite likely this new system will be adopted by some states and not others, mimicking the way that the earlier federal healthcare.gov insurance exchange system was adopted by some states, while others chose to create a home-grown system or contract out system development to a third-party vendor.
Time will tell if the VAMS system can be available quickly enough to make a difference in the rush to vaccinate as many Americans as possible – an effort that has become even more important now that new variants to the Covid virus are emerging, which worsen the pandemic before enough people are vaccinated.
Formaspace Is Your IT Facility Partner
Formaspace can help you make your IT department more efficient and productive.
We build custom solutions for computer rooms, data centers, QA testing and repair facilities, tech warehouse and distribution centers, and much more.
All our furniture is manufactured here in the USA at our Austin, Texas factory headquarters.
Find out how we can work together to make your next IT project a success.
Contact your dedicated Formaspace Design Consultant today.
