Diagram of pooling method for serum samples from blood donors. The ...

Source: Gibran Horemheb-Rubio et al, 2017

 

A curious thing happened about a week ago. The Government of Ghana, which has been touting its investments into Covid-19 testing capacity to universal acclaim, as the country galloped up the continental league tables, suddenly found itself in the dock. Before long, prominent health leaders in the country were accusing the Government of massaging testing numbers for PR benefits.

Ghana’s Magic

By the time the dust settled, it had become clear that it wasn’t massive investments into diagnostic assays and reagents, not to talk of RT-PCR testing machines, that had catapulted Ghana to its celebrated position of number two in the league table of African countries that have carried out the most tests. Rather, it was the ingenuity of scientists at the country’s preeminent biomedical institution, the Noguchi Memorial Institute of Medical Research that was largely responsible.

Without waiting for the usual bureaucracy of national ethical review approvals or situation-specific peer-reviewed studies, they had decided to deploy the well-known process of “pooled sampling”, thereby expanding testing capacity severalfold literally overnight. I chronicle these fascinating matters here and here.

With countries like Nigeria struggling to obtain enough reagents and other testing consumables in the international market, and with their testing output stuck at 0.0055% of the population (compared to Ghana’s 0.328%, which is nearly 60 times higher on a proportional basis), Ghana’s feats had seemed like pure magic.

A Matter of Some Urgency

The Africa Centers for Disease Control (Africa CDC), the lead continental agency coordinating the regional Covid-19 response, in its 23rd April situation report, gave the total number of tests conducted on the continent as 415,000, of which Ghana alone was responsible for at least a sixth. By the 27th of April, Ghana had completed over 100,000 tests, whilst Nigeria was hovering around 12,000 tests. The Head of the Nigerian CDC continued to open up  about the challenges the country was having in securing supplies of reagents because of severe global shortages.

Recognising that Ghana’s ten-fold multiple of Nigeria’s testing capacity is almost entirely down to a clever algorithm – testing based on pooled samples – one immediately begins to wonder why other African countries should not immediately adopt this algorithm and accelerate their testing coverage dramatically. Especially bearing in mind that with the worldwide testing average now in the region of 1.5%, Africa’s less than 0.01% is no longer tenable. Should Africa not embrace pooled testing continentally right away?

Perhaps, we first need to break down the principles of pooled sampling. When I touched on the subject in my earlier articles, many people reached out to me on social media and through my blog to challenge my thinking and ask clarification questions about my analysis. Clearly, it is the most critical short-term decision African public health authorities would need to make regarding how to alleviate the constraints holding back mass testing.

Pooled Testing: The Basics

Supposed you had 100 items you needed to test. You know that a couple of these items have a particular defect but you don’t know which specific items. You could certainly test each item one after the other. But knowing that only a few of these items have the defect, that would seem to be a waste of time and resources, especially if each test consumes significant time and resources.

Supposed the items could be mixed up in such a manner that the resulting mixture was generally similar in properties to the individual items? If so, then it would make sense to group the 100 items into smaller pools of ten items each (leaving a reserve amount from each sample) and proceed to test each pool. If any of the pools test positive for the defect in question, then the reserve samples from only that pool can be tested one after the other to find out which sample or samples from that pool might contain the defect that is triggering the positive result.

This algorithm can be summarised very simply, if also crudely, as follows.

Number of Samples 100
Test Round 1
Number of Pools 10
Number of Tests 10
Pools Testing Positive 2
Tests “Wasted” 8
Test Round 2
Number of Pools 0
Number of Tests 20
Tests Returning Positive 3
Positive Case Rate 3%
Tests “Wasted” 17
Total Number of Tests 30
Tests Saved 70

 

In effect, assuming simplistically that one homogenous test kit is required for each testing exercise, 30 test kits have been used to do what would have required 100 test kits in the conservative one-by-one model.

More Math Than Biology

The first thing that may have jumped at the reader is the statistical nature of the decision-making here. It is not molecular virology that determines, fundamentally, the issues at play. True, that important discipline plays a role in addressing some of the concerns about sensitivity (a measure of how effective a testing method is in fishing out genuinely or truly defective samples) and specificity (a measure of how effective a testing method is in establishing that no defect exists) that the rest of this article will grapple with, but the primary logic derives from statistical analysis. Not surprising then that the first person to rigorously publish about the process was a political economist writing for a statistical journal.

Sensitivity

Only a tiny modicum of imagination is needed to spark the most obvious concern: might mixing the ten samples into one pool not dilute it to the point where it falls below the threshold of detection? In lay terms, and in the context of Covid-19, could it not happen that when a small quantity from one or more original samples is scooped out and mixed with others to create a new single sample (“the pool”), that smaller sub-quantity might lack the viral RNA? This is the not-so-difficult-to-fathom issue of dilution. Because the tube or well in which each test reaction takes place inside the PCR machine is finite in size, pooling samples necessarily means reducing the amount of sub-sample taken from each original sample.  RT-PCR tests may be the gold standard for detection now, but they are certainly not perfect. There is a roughly 10% chance anytime a PCR test is conducted that the result may be invalid. Will dilution not compound matters?

Luckily, there has been quite a steady stream of experimental research work that has established that modern assays are now so sensitive that the dilution risk is not necessarily elevated by pooling samples, though considerable optimisation is required for different infection scenarios.

So, all good then, we can effectively expand capacity by up to ten times across Africa by instituting pooling as a continental protocol. Except that there are some very important caveats to consider before celebrating this potential bonanza.

Pooling Efficiency

As already indicated, the algorithmic logic of pooling is based on simple mathematics, but one that can become very complex very quickly. I will spare the reader the exotic combinatorics and focus on the easy matter of “pre-test probability”.

Because of weeks of conducting Covid-19 tests, every country now has a rough idea of what the probability of detecting a positive case might be at every point in time. While this factor is changing all the time, the shifts are fairly steady and programmable. In Nigeria, for instance, the probability that a test will return positive is nearly 9.5%. In Ghana, it is currently around 1.5%. The vast disparity is significant and will be touched on later.

This “likelihood of finding a defect” (or, in the Covid-19 context, an individual testing positive) has a huge effect on the effectiveness of pooled sampling. Let us return to our simple table above. The positive rate in that thought experiment was 3%. But supposed we brought it closer to the Nigerian case, and assumed a 9% pre-test probability of finding a defect (or positive case)?

Number of Samples 100
Test Round 1
Number of Pools 10
Number of Tests 10
Pools Testing Positive 9
Tests “Wasted” 1
Test Round 2
Number of Pools 0
Number of Tests 90*
Tests Returning Positive 9
Tests “Wasted” 91
Total Number of Tests 100
Tests Saved 0

*Each of the nine pools is being rerun for individual tests because of an assumption of uniform distribution of viral RNA through the mixture.

It follows then that as pre-test probability of positivity rises, there arrives a time where there is a theoretical mathematical possibility of saving no test kits or time. In fact, an important element of RT-PCR in general is that the saving in time does not scale linearly with the saving in reagents. This is because there are some time-consuming steps like viral RNA extraction, machine setup, maintenance and the like that can be common to all methods, whether pooled- or individual – based. In that regard, in a situation where the pre-test probability of finding positive cases is high, one can actually lose time doing pooling. There are certainly many optimisation algorithms to evade some of these constraints. The Ghana model for instance could be enhanced by successive division of pools instead of complete retesting of every pool that tests positive. But there is only so much optimisation one can do to push back the theoretical limits.

The insight illustrated above has led many experimentalists to argue that when the confirmed cases are above one or two percent of total tested, pooled testing may not be advisable. The only government that has reviewed the situation extensively and taken a policy and ethical (more on “ethicality” later) stance on pooled sampling/testing so far is the Government of India, which has decided to limit the pooling size to 5 samples and the context of use to “screening”, which is most beneficial during the early stages of community transmission of the disease. Which brings us to the last two important caveats.

Error Replication

Once the testing process moves out of clinics into the broader community, through for instance the community screening and contact tracing models, the whole affair becomes a complicated supply chain puzzle to unknot.

Public health officers have to be equipped with sample collection devices and sent off to various corners of the country to collect samples. A single officer may collect more than ten samples and there could be more than 1000 sample takers. The samples have to be transported in cold boxes, deposited at aggregation points and finally delivered to labs, which now have to manage these samples as one would any inventory process, with planning for storage, handling, batching and tracking.

Even in the most rigorous of mass testing situations, errors will occur, and far more than would be the case in a routine testing situation. Some samples will be poorly collected and transported than others.

Pooling, in these circumstances, can easily obscure traceability by making it highly difficult to identify points within the supply chain that are defective, since for the vast majority of tests, there is no link between a test result and its supply chain history.

Should a sample be cross-contaminated, the effect is not just on that second sample but on an entire pool. Quite apart from interfering with corrective action protocols, pooled sampling can increase workload due to error replication.

In effect, whilst pooling under laboratory conditions can be heavily controlled to remove error effects, additional safeguards are required in mass testing protocols where samples are not collected inside health facilities.

But by far the most critical issues are ethical in nature.

Ethical Clearance

In the pooled sampling and testing methodology, pools that test negative are not retested, for obvious reasons: the whole point is to save time and consumables.

In some cases, this is not an issue, but in others it is. During an epidemic or pandemic, such as the current one, testing is not a monolithic affair. We must look at the mass testing protocol in every country as a multi-layered one involving different cohorts of testing subjects.

First, there are those who were ill or felt ill and reported to a clinic or other health setting. The clinician suspected Covid-19 and sent off samples for testing, a process often referred to as “routine surveillance”. The pre-test probability in such cases (what we might also call the “index of suspicion”) is relatively high. In some cases, despite a negative test result, considering the inherent imperfection of all tests, re-testing is imperative. An example would be a situation where the individual’s symptoms and travel history are so similar to those who typically test positive for the disease. In these circumstances, pooling can be harmful to such patients because negative results are not reviewed individually.

Second, there are those who come into contact with persons who have been confirmed to have the disease. In a country with a sophisticated contact tracing system, the risk level for such persons can be considered elevated if they have been in contact with multiple positively tested individuals in settings that heighten exposure. Here too, despite a negative result, it might be necessary to re-test but in a pooling scenario that would be impractical.

Lastly, we have those individuals who fit some theoretically determined risk profile (for example, living in a neighbourhood where many people have tested positive). Testing in such “community screening” scenarios is driven largely by the evolving picture of the outbreak and by the quality of data, modelling and analysis. In countries where public health institutions are weak, the data, models and analyses involved may well also be weak.

In Ghana’s case, the empirically observed situation accords cleanly with the above descriptions. Using the 21st April 2020 Covid-19 situational report for the Greater Accra Region (the capital city region that is presently the epicenter of the outbreak in Ghana, accounting for more than 85% of all infections), we computed a positive case rate of 8.5% for test subjects selected through routine surveillance and 8% for those selected through contact tracing. Community screening results on the other hand yielded positive cases of less than 0.1% of total number of individuals tested.

Ghana’s fascinatingly low 1.5% rate could thus be the result of a major dilution effect from potentially weak screening. If the community screening results are removed, the Ghanaian observed prevalence rate becomes almost identical with that of neighbours such as Nigeria and Ivory Coast.

Pooled Sampling Should be Used Selectively

But even if the community screening exercise is based on perfectly sound models, and I do admit that the current positive rate from routine surveillance nationwide in Ghana is around 2.5% (most likely due to loosening surveillance standards especially outside the capital) and thus less dramatically divergent from the community screening results, individuals screened under this method have different risk profiles and therefore clinical needs.

Those tested under the routine surveillance route, clearly because they are more likely to be sick, and some of those tested through the contact tracing route, because their risk is higher, may all have a right to a standard of care involving re-testing where necessary. Grouping of test subjects into cohorts based on various indices of suspicion and vulnerability seem therefore like an elementary ethical requirement.

In summary, whilst pooled testing could considerably boost testing capacities around the African continent, it should only be implemented in each country following peer-reviewed studies to establish the right optimisation models and after national ethical clearance has established the right of patients to differential diagnosis.

It is probably best then that such methods are reserved for community screening (typically more than 60% of the overall testing volume anyway) only. The evidence shows clearly that this category of testing has the lowest risk, at least before there has been widespread community infection. In the same vein, it may be prudent to avoid using the pooled technique on samples from individuals or patients identified through routine surveillance and contact tracing.

On 11th April, 2020, I wrote a blogpost warning that if the government of Ghana fails to get its data management under control, it will start to lose public trust, regardless of how well the actual management of the Covid-19 outbreak itself was going.

On 19th April, 2020, the President of Ghana announced to Ghanaians that he was, effective the 20th of April, lifting the “partial lockdown” imposed on the country since 30th March, 2020.

He cited the “robustness of data” and the “constancy” of the situation as the basis for the decision. In an earlier article, I have discussed why the President’s assurances did not go down well with many of the country’s health leaders.

The government has left the metrics and indicators that will trigger specific actions (such as the loosening of restrictions) much too loose and ad hoc. The trade-off of this flexibility is second-guessing by experts outside the government’s team.

In this shorter article, I shall be listing, for those who could not read any of the earlier posts, the outstanding data-related issues that will continue to fuel controversy if not comprehensively addressed.

  1. The GHS’ public bulletins are too data-thin and patchy to serve researchers.

The Ghana Health Service (GHS) releases periodic updates on a portal to inform the public about new developments. Compared to the “situational reports” prepared at regional and district level for health administrators, the data on the portal (though visualisation has been improving) is not very useful to researchers as it lacks granularity. None of the reports circulating around is in a format that can be exported to spreadsheets for analysis anyway. The inability of independent researchers to build models to explain features of the disease phenomena is forcing them to speculate and rely on the grapevine. Experts without the help of data are rarely more insightful than ordinary joes.

  1. Beyond data on the spread of the disease, there is also no information on the protocols guiding the response itself.

The Covid-19 response in Ghana is, according to the authorities, based on three main pillars: trace, test and treat (including isolation if necessary). Each of these strategies has many underpinning operational elements. And none of them are being guided by published, widely available, national protocols and standard operating procedures. In the absence of documentation, speculations are rife about all manner of things. In this short post, we will deal primarily with the first two strategies: trace and test.

  1. The “supply chain” for delivering tests to people needs work.

The country’s mass testing protocol has many gaps and lags. Clinical referrals for testing during routine surveillance (where people with Covid-19-like symptoms are identified by clinicians and sampled at a health facility) are currently not automated. The performance of the different tracing teams (public health personnel who actively search for cases in the community) differ considerably. Samples need aggregation before they are sent off to the various labs (virtually all of them in the country’s two major cities of Accra and Kumasi), but the inefficiencies can affect the quality of the sample and thus testing integrity.

Per WHO and US CDC standards, samples need a cold chain at all times. If a sample will take more than 5 days before reaching the lab, dry ice (-70 degrees celsius) is required. Ideally, samples should be transported in protein-antibiotic complexes called viral transport media (VTM). Some laboratory scientists complain of some delivered samples lacking even basic saline buffers, arriving unsealed, or having such small sample volumes as to interfere with viral RNA extraction.

Whilst Ghana’s laboratory scientists are consummate professionals doing their best in trying circumstances, there is a limit to what their ingenuity only can achieve.

  1. The government’s “aggressive tracing regime” has been faltering.

Given all these supply chain and resource constraints, it is not too surprising then that the “aggressive tracing” promised as a partial substitute for the lockdowns appears to be slackening.

In the first week following the lockdown, tracing surged from 635 contacts reached to 5308. Sample collection rose from 589 to 4969. By 19th April, the day the lockdown was lifted, contact tracing figures were down to 2049 and samples collected were as low as 1018. Considering that infection stats are highly sensitive to overall levels of tracing and sample collection, this apparent slackening is worrying.

  1. The interpretations being given by the government about the ratio of positive cases to overall tested results are statistically loose.

At the time of the lifting of the lockdown, much was made of the fact that only 1042 out of 68,591 (ergo, 1.52%) tested subjects were positive. However, that analysis involves a bit of mixing apples and oranges. Some of the testing protocols are so different in their quality that their results should not be allowed to dilute the overall picture.

To illustrate this point, I shall focus solely on the central hotspot of the epidemic in Ghana, Greater Accra. I was lucky enough to get a hold of the Greater Accra Covid-19 situational report of 21st April 2020, just around the period the lockdown was lifted.

The high-level breakdown of the aggregated numbers in Greater Accra as at that date was as follows:

In Accra, people who are being referred for testing because they are showing Covid-19 symptoms (i.e. through “routine surveillance”), at the time the lockdown was lifted, had an 8.5% chance of testing positive. People who were identified for testing because they had come close to someone confirmed as infected had an 8% chance of being positive too.

These two categories of people are being targeted for testing using very established and grounded epidemiological methods. The people who are being randomly tested based on the GHS model of which communities are at risk tend to have a much lower probability of testing positive. The interpretation the government’s advisors have given to this fact is that community spread is low. The more likely answer is that the GHS’ model is weak. Since they refuse to publish and defend it before independent analysts, most biostatisticians I have discussed this issue with dismisses the model out of hand.

  1. Even the higher ratio of positive cases in routine surveillance may be underestimating true spread.

Ghana’s routine surveillance programs have considerable weaknesses. In 2010/2011, and again in 2017, they failed to detect H1N1 outbreaks till very late. In the 2017 episode, four KUMACA students who died of H1N1 at the Komfo Anokye Teaching Hospital were diagnosed only after death. According to the present Auditor-General, the Veterinary Services Department failed or neglected to set up an Asian Influenza pandemic preparedness system in 2010, opting instead to devote the money to workshops. When the epizootic crisis hit, over 400,000 livestock belonging to poor Ghanaian farmers perished from H1N1. The 8.5% positive testing rate recorded in Accra for Covid-19 suspected cases at the time of lifting the lockdown may thus have been lower than the true situation.

I do note that, in more recent days, the national-level routine surveillance ratio of positive cases has fallen to as low as 2.5%. Since situation reports for other regions are hard to come by, it is unclear if suspicion parameters and case definitions are identical nationwide. For instance, when composing the national picture, the GHS, unlike the regional directorates, lumps the community screening activities (based on its proprietary model) with the enhanced contact tracing activities thereby obscuring the fact that it is virtually not detecting any cases through the so-called “community screening” exercises, most likely due to weak modelling.

  1. There is concrete evidence that the GHS risk-based model for community screening is weak.

Using this proprietary model, the government decided to concentrate its efforts in Ayawaso West. At one point, it was even suggested that screening in this district will be universal and compulsory, only for the proclamation to be withdrawn later without explanation or ceremony.

It soon became clear that the transmission dynamics were far more complex. In a few days, the virus penetrated deeply into Ayawaso Central, an extremely high-density, inner city enclave of the city, where suburbs such as Nima, Maamobi and Kanda are clustered. Then it stormed Accra Central (Jamestown, the High Street, the Central Business District etc) before making the most fascinating move of all: turning Korle Klottey (Osu, Ridge, North Adabraka, Odorna etc) into the fastest growing hotspot.

A simple biostatistical model based on covariance analysis of how trends in positive case confirmations across economically connected zones align should have shown clearly that commuter patterns of informal labour pools criss-crossing the Maamobi, Tudu, CBD, Odorna and North Adabraka inner-city rings were the primary features of interest. Some serious urbanography, not just epidemiology, should have been deployed immediately. The lack of open data prevented urban researchers from joining the fray.

  1. Urbanographic analysis is clearly critical in anticipating the worst.

Ayawaso East, Ayawaso West and Korle Klottey are the areas where Covid-19 related hospitalisations are likely to increase due to the growth trend of routine surveillance results. Accra Central and Ayawaso Central appear to be harbouring fast-growing numbers of asymptomatic individuals. How the trends will move from here on require an urbanographic, not just epidemiological, lens. Covid-19 always seems containable until it finds a vulnerable population or highly susceptible community in some cluster and then starts wreaking havoc. We can only hope that we can race ahead of the virus to identify such communities and ring-fence them before Covid-19 does.

  1. The government’s attempt to push the narrative that it was investing heavily into testing resources created the confusion about testing capacity.

It turns out that it was the clever scientists at Noguchi that had found a workaround: pooled sampling, not massive injections of resources into testing infrastructure as the country was being told. Pooled sampling refers to the consolidation of multiple samples from different individuals for a single thermocycling run (i.e. single test).

  1. Pooled sampling has rescued the country but it has important limits.

India is one of the few countries in the world to have commissioned a detailed efficacy and ethical review of whether to update the national protocol on testing by allowing pooled samples. It did so just a little over a week ago but added many caveats, which should be of concern to Ghana too.

The India Council of Medical Research’s (ICMR’s) decision to impose a cap of five samples, a threshold Ghana initially adopted before “escalating” to 10 samples per well, speaks to the fear of overestimating diagnostic sensitivity thresholds. They also went further to permit pooled sampling only if the pre-test probability of positivity is lower than 2%.

Stanford’s Benjamin Pinsky, a clinical virologist, recently led a team to conduct mass community screening for Covid-19 (especially at sub-clinical level) in San Francisco. His team determined that a pre-test probability of 1% is the reasonable threshold to allow pooled sampling. These precautions put in place in other epidemiological contexts raise important issues for Ghana’s continued use of pooled sampling.

Firstly, a pooled sampling protocol is highly responsive to the specifics of the test kit in use, the epidemiological background, and the goals of screening. Hence, the protocols must be submitted to peer review and national-level ethical clearance, as has been done in India.

The ethical issues are compounded because differential diagnosis remains the standard of care in a context like Covid-19 where observed symptoms can be highly non-specific. Many respiratory pathogens could be implicated in the clinical presentation. (Even some health workers have taken to calling SARS-COV-2, the microbe that causes Covid-19, a “flu virus”, but it belongs to a completely different family of viruses).  In that regard, re-sampling for further tests could be warranted even if a negative result ensues. In a pooled testing scenario, this situation is complicated, especially in the absence of patient consent.

In light of this, only the mass screening exercises appear ethically suited for mass sampling. Routine surveillance and enhanced contacts tracing cases, with their high pre-test positivity rates, on the other hand, are best not confirmed through pooled sampling.

  1. The issue of whether the denominator used in determining the positive case ratio is being overestimated remains unresolved.

The different testing labs for Covid-19 in Ghana at present maintain separate indexes and case investigation form-coding procedures. There is currently no efficient way to harmonise and consolidate multiple tests performed on different samples from the same individual, especially also as the case investigation forms map to a unique sample ID but not to a unique patient ID. Multiple cases submitted to different labs would automatically count as separate cases.

Multiple cases submitted to the same lab can be harmonised against a single patient if the data is de-identified. In a pooled sampling regime, it is problematic to de-identify the samples constituting every pool to check for history without defeating the original goal of saving time.

These issues need to be clarified properly in an open and transparent manner before the 1.45% total positivity rate reported by the GHS on 22nd April 2020 can be accepted at face value.

12. Open Data is NOT the enemy.

Open Data is clearly not the enemy here. If anything it is the scorned friend waiting on the sidelines to save the day.

 

On 19th April, 2020, the President of Ghana announced to Ghanaians that he was, effective from 20th April, lifting the “partial lockdown” imposed on the country since 30th March, 2020.

He cited the “robustness of data” and the “constancy” of the situation as the basis for the decision, but he did not elaborate if by this he meant that the lockdown had achieved the generally cited purpose of that draconian form of epidemic control: reducing the number of new infections (called “incidence” in epidemiology).

In later explanations to the media, the Presidential Advisor on Health would add some nuances to the effect that the purpose of the lockdown was to gather more insights into the nature of the disease’s spread so as to pave the way for more targeted measures. Such a step, in his view, was critical to preventing the possibility of a worse humanitarian emergency in poorer communities for whom the lockdown was exacting a frightening economic toll. The inference here must be that the Administration had evaluated the state of the pro-poor economic reliefs it had rolled out at the beginning of the lockdown and determined that they couldn’t do the job of abating suffering, and possible social unrest, should the lockdown continue.

Throughout all these explanations, the country was nevertheless not served with any data-backed models to enable independent researchers and other critical observers analyse the totality of the situation for themselves.

Hardly surprising then that before long senior public health figures outside the government, including a former Head of the Ghana Health Service, the country’s preeminent primary and secondary health agency, decided to break ranks and take serious issue with the quality and integrity of both the data and the decisions purportedly based on them.

I couldn’t suppress the wry smile on my face. On April 11th, at the peak of the partial lockdown, I had pleaded with the government to be more transparent and far more diligent than it had been to date in showing people in advance what specific metrics would prompt which specific actions. To avoid the charge of data manipulation and afterthought justifications of conclusions predetermined on the basis of factors other than the public interest, the public, I argued, has to be educated comprehensively about how different status indicators shall serve as triggers for specific actions. That required, of course, that said status indicators were themselves clearly defined, solidly rigorous, well thought through, and reasonably comprehensive.

I am afraid to say that the government did not listen. Its status indicators remain hard to pin down and what can be gleaned from its interpretation of the data leads to ambiguity on the most important issues; such as whether to intensify or loosen lockdowns, whether to change tack or stay the course in its approach to “mass/community screening”, and whether to admit to an acceleration of incidence or continue to insist on this “constancy” theory.

The government’s earnest and vivacious “communicators” keep pushing one theme: everyone should keep calm, trust government-appointed and favoured “experts”, and have faith that all decisions are being driven by “science” and a dogged commitment to the public interest.

Unfortunately for our good friends in the government’s various PR brigades, science and expertise are not hymn sheets from which a harmonised tune can be blared from loudspeakers on the high and imperious walls of Jubilee House.

“Science”, whether in a pandemic or in normal times, is a highly variegated domain of knowledge, replete with competing worldviews among disciplinary specialists of many stripes. That is why almost every university and think tank in the world worth its salt in every sophisticated country has its own set of theories and models about the virus, complete with different, sometimes contradictory, trajectories and forecasts. If Ghana were a more sophisticated country, there would have been more not less of the disputes amongst different types and grades of health “experts” witnessed in the aftermath of the President’s decision to lift the blockade.

Another obvious truth in this matter is that an expert not armed with data is rarely more effective than the ordinary joe in making sense of emerging phenomena. By failing or refusing to publish sound models and put out enough data to aid serious analysis, the government consigned experts to the grapevine with the rest of us. As snippets of information and anecdotal evidence swirled around, contextless documents added a veneer of depth to opinions, until eventually things came to a head when some of the country’s most eminent health thinkers openly accused the government of “massaging” the Covid-19 incidence and “observed prevalence” statistics. The two main labs in the country handling the surge in testing found themselves in the cross-fire, accused of misrepresenting their testing capacity.

The government was on the verge of squandering the trust it had built up in the early days of the pandemic when citizens rallied behind their leaders as predicted by the sociology of emergencies.

In this brief article, I shall be shedding light on the two main data-related controversies – a) is the incidence rate of Covid-19 “stable” in Ghana? And (b) is “pooled sampling” a satisfactory rebuttal to those who question the effective testing capacity in Ghana? I shall be doing so with a view to reiterating my earlier points about the extreme importance of “trustworthy data in decision-making” during a crisis as we have now.

From the onset of the pandemic in Ghana, the Ghana Health Service (GHS) has published periodic bulletins to inform the public about major developments. Researchers have complained about the unavailability of deeper and broader data sets in a format that can be exported into spreadsheets to enable them chart trend curves and develop elaborate and on the fly models to explain various facets of this unprecedented phenomenon. They have humbly requested for these data sets to be released promptly and consistently to allow for dynamic modelling and analysis, yet the GHS refuses to budge.

A part of the reason may well be capacity. The country’s mass testing protocol has many gaps and lags. Clinical referrals for testing during routine surveillance (where people with Covid-19-like symptoms are identified by clinicians and sampled at a health facility) are currently not automated. The performance of the different tracing teams (public health personnel who actively search for cases in the community) varies. Collected samples have to be aggregated and sent to the various labs (virtually all of them are in the country’s two major cities of Accra and Kumasi) using an ad hoc supply chain.

Per WHO and US CDC standards, samples need a cold chain (between 2 and 8 degrees celsius) and if delays in dispatch will last beyond 5 days, dry ice is required for storage and transport. Ideally, samples should be transported in viral transport media (VTM), protein-antibiotic based mixtures, that some GHS tracing teams don’t have. Some laboratory scientists complain of the occasional sample coming in without even basic saline medium, or having been improperly sealed, or in such small volumes as to interfere with viral RNA extraction. In short, all the messiness of real-world supply chains. In a country with such weak health infrastructure, it is a miracle how the fine professionals working in the reference labs are still keeping things together. The end result, however, is that there are inefficiencies in the data collection process, as I alluded to in my initial article. Until capacity is ramped up at all levels, the data will continue to be patchy. The delays and lumpiness in the release schedule, despite widespread perception, are not all caused by the Ministry of Health and the Ghana Health Service’s insistence on “validating” the data before public release.

But all this shouldn’t be a problem if the entire mass testing protocol was transparent to researchers and critical observers. Statistical methods exist to clean and fix data weaknesses and gaps. The real problem is disinterest on the part of the government’s Covid-19 response team in engaging candidly and openly with the research community on the data question.

Even as the government’s communicators were justifying the decision to lift the lockdown on the grounds of accelerated tracing, tracing figures were actually declining in Accra. In the first week following the lockdown, tracing surged from 635 contacts reached to 5308. Sample collection rose from 589 to 4969. By 19th April, the day the announcement was made to lift the lockdown, contact tracing figures were down to 2049 and the daily tally of collected samples was as low as 1018. Obviously, the facts on the ground did not align with the view that lockdown measures were being replaced with aggressive contact tracing. Considering that infection stats are highly sensitive to overall levels of tracing and sample collection, the discrepancy is curious.

Which brings us to the issue of the positive case ratio per population of the virus. Apart from the fact that testing positive for the virus does not imply a clinical diagnosis of Covid-19, there is also the fact of our present misunderstanding of the role of viral load or concentration on both symptoms progression and detectability. These complexities require care and diligence in executing a mass testing protocol.

Thus, when the President’s advisors conclusively argued in the days following the decision to lift the lockdown that because only 1042 out of 68,591 individuals (ergo, 1.52%) tested were positive, the degree of community spread was somewhat restrained, they were missing some very critical granular facts. The insight is not in these global numbers, made up as they are of apples and oranges forced into a mix. It can rather only be found by careful disaggregation and analysis.

To illustrate, let me use the central hotspot of the epidemic in Ghana. I was lucky enough to get a hold of the Greater Accra Covid-19 situational report of 21st April 2020, just around the period the lockdown was lifted.

The high-level breakdown of the global numbers as at that date was as follows:

Even at this high level, it can quickly be seen that the “positivity profile” of the tested individuals in the different groups listed above (those referred for testing because they were showing some symptoms; those identified for testing because they had come into contact with someone who tested positive; and those who were randomly tested because they live in an area considered as lying within a perimeter or cordon of risk as determined by the prevailing hotspot models of the GHS) vary considerably. Whilst those referred by clinicians for presenting Covid-19-like symptoms had a roughly 8.5% chance of testing positive, those who were identified because they had been in contact with a positive case had a slightly less than 8% chance. More crucially, those earmarked for testing simply because they were caught in the GHS risk-based net had virtually no chance of testing positive.

Different biostatisticians and epidemiologists may draw different conclusions. But one of the most plausible would be that the GHS risk-based community screening model is broken and should not be lumped together with the more epidemiologically grounded models of routine surveillance and primary contact tracing.

True, Ghana’s routine surveillance programs have their own weaknesses. In 2010/2011, and again in 2017, the country’s capacity to detect outbreaks of H1N1 in time to avert mass casualties was tested and found wanting. In fact, in the 2017 wave, four KUMACA students who died of H1N1 at the Komfo Anokye Teaching Hospital were diagnosed post-mortem. Eventually 96 people would be detected but well past when a functional surveillance program should have done so.

According to the present Auditor-General, the Veterinary Services Department failed or neglected to set up an Asian Influenza pandemic preparedness system in 2010, opting instead to devote the money to workshops and other such trifles. When the epizootic crisis hit, over 400,000 livestock belonging to poor Ghanaian farmers perished from H1N1. But all that is merely to say that the positivity rate for routine surveillance is very likely to be higher than 8.5%. Letting the unproven “community screening” numbers dilute the overall positive case ratio amounts to a failure of sound analysis. Even more so since the GHS refuses to publish any document to explain the logic driving it.

Furthermore, enhanced granularity only deepens anxiety. The preliminary “hotspot perimeter designation model” orally described by the President’s coordinators of the Covid-19 response effort, recommended a concentration of effort in Ayawaso West. At one point, it was even suggested that testing there should be universal and compulsory, only for the proclamation to be withdrawn without explanation or ceremony.

It soon became clear that the transmission dynamics were far more complex. In a few days, the virus charged into Ayawaso Central, an extremely high-density, inner city enclave of the city, where suburbs such as Nima, Maamobi and Kanda huddle tightly together. Then Accra Central (Jamestown, the High Street, the Central Business District etc) took its turn. Before the most fascinating development of all, the emergence of Korle Klottey (Osu, Ridge, North Adabraka, Odorna etc) as the fastest mutating hotspot of all. Ethnographic economic data shows trends tied to the commuting habits of specific pools of informal labour that reside in one enclave and ply particular types of trades in other enclaves.

Rudimentary covariance analysis of the interrelationships among trends in shifting hotspots would have highlighted the spatio-economic links driving the spread of the virus from one high-density spot to the other. But serious urbanography, beyond pure epidemiology, would be required to deepen the insight, once again making a strong case for both a wider sharing of prompt and complete data and for maintaining a multidisciplinary stance.

The social relief strategy advised by such a stance would not have prioritised warm rations over invasive but respectful broader public health interventions beyond just disease surveillance. Communal toilets, communal baths, communal pipe stands and similar locations would have seen re-engineering to enhance a modicum of social distancing however difficult that prospect might be in places like Old Fadama, where human density at certain seasonal peaks exceeds 3000 per hectare. The distribution of hygiene products would have received the same attention as the sharing of warm rations.

Knowing that Ayawaso East, Ayawaso West and Korle Klottey are the areas where Covid-19 related hospitalisations are likely to increase due to the growth trend of routine surveillance results, mobile screening facilities should have been deployed more aggressively in a kind of shifting sentinel strategy. Accra Central and Ayawaso Central having become a source of worry because of the fast growing number of asymptomatic individuals observed during contact tracing should have been earmarked for limited serological surveys as part of the post-lockdown measures.

In short, the main point here is not to quibble over the President’s decision to lift the lockdown per se. The focus here is on highlighting what a truly data-driven set of measures would have look like.

And the other controversy over testing numbers?

Here too, much of the hoopla could have been avoided had politicians not attempted to take credit for Ghana’s supposedly high ranking on testing league tables in Africa. The impression was created that the country’s performance was as a result of massive investments into RT-PCR platforms, reagents, and diagnostic assays.

People with connections in the scientific community knew however that these resources were yet to be made available in any quantity capable of effecting such a dramatic transformation of the country’s testing capacity. Judging from the experience in other countries, the recently donated test kits from the Jack Ma Foundation were yet to be put to use because compatible reagents were not available. No wonder then that these PR narratives triggered protests, most famously from the former Director-General of the Ghana Health Service itself.

It turns out that the clever scientists at Noguchi had found a workaround: pooled sampling. It is true that the original algorithms and supporting logic for how to maintain experimental validity when running tests in aggregates date all the way to the work of political economist, Robert Dorfman, notably in his 1943 paper in the Annals of Mathematical Statistics (another toast to multidisciplinary thinking). But the sheer boldness to respond in this manner to the national call for a surge in testing even without the corresponding resources deserves respect.

Contingent ingenuity in the laboratory does not however excuse incoherence at the level of national policy. As Noguchi’s leadership freely admits, pooled sampling shall only remain effective if infection rates in Ghana are subdued. It is not for nothing that many reference labs around the world have not got around to accepting pooled sampling for routine Covid-19 case confirmation. Whilst India did so just a little over a week ago, the many caveats added by the country’s apex health authority, the India Council of Medical Research (ICMR), show clearly that concern over the very real risk of false negatives due to dilution remains a concern for many laboratory quality assurance and bioethics experts.

The ICMR’s decision to impose a cap of five samples, a threshold Ghana initially adopted before “escalating” to 10 samples per well, speaks to the fear of overestimating diagnostic sensitivity thresholds. The ICMR also went further to permit pooled sampling only if the pre-test probability of positivity is lower than 2%. Stanford’s Benjamin Pinsky, a clinical virologist, recently led a team to conduct mass community screening for Covid-19 (especially at sub-clinical level) in San Francisco. He would peg the suspected positivity ratio at 1%. These precautions put in place in other epidemiological contexts raise important points for our continued use of pooled sampling.

Firstly, a pooled sampling protocol is highly responsive to the specifics of the test kit in use, the epidemiological background, and the goals of screening. Consequently, the protocols tend to be submitted to peer review. In Ghana, the protocol has not even been published. Secondly, there are some important ethical issues that arise when human subject testing protocols are changed midstream. Institutional Review Committee approval is typically required. In a public health emergency, national-level ethical clearance, as was the case in India, become important. Given that members of the medical fraternity in Ghana appeared unaware, I would reckon that this is yet to be done. At any rate, in India, the process was openly announced.

It is fair to wonder if this is all mere bureaucracy. Alas, it isn’t. In routine surveillance referrals for testing, clinical outcomes for the individual remain important notwithstanding the prioritisation of public health. Differential diagnosis remains the standard of care in medical situations like Covid-19 where observed symptoms can be highly non-specific. Many respiratory pathogens could be implicated in the clinical presentation. (Even some health workers have taken to calling SARS-COV-2, the microbe that causes Covid-19, a “flu virus”, yet it belongs to a completely different family of viruses).  In that regard, re-sampling for further tests could be warranted even in the event of a negative test result. In a pooled testing scenario, this situation is complicated, especially in the absence of patient consent.

Moreover, because best practice also recommends the use of double probes to target different gene/ORF regions of viral RNA to increase sensitivity, a negative result can, theoretically, involve discordance between the two probes, which again may require retesting.

More crucially, in a routine surveillance or enhanced contact tracing situation, molecular tests are contributory but not determinative in every instance. A high index of suspicion due to travel history, clinical observations and other factors might warrant re-confirmation. A protocol which conserves resources (and to a somewhat lesser extent, time) by eliminating individual retesting of negative samples in all cases risks bumping up against ethical norms.

How to resolve the conundrum of reconciling the national demand for surged testing with the rights of the individual patient? Simple: segment the testing pool into its original cohorts. Mass community screening, which is the main bulk of current testing, has a limited connection to clinical management and should probably not generate any serious ethical issues in a pooled sampling regime. Routine surveillance and enhanced contacts tracing cases, on the other hand, present a clear challenge and are best not confirmed through pooled sampling. Especially considering the higher pre-test probability of positivity going by Ghana’s Covid-19 testing data, well above what stringent review boards elsewhere have determined.

To set minds at ease, the reference labs may consider independent IRB evaluation of the claims that even in the case of samples collected from asymptomatic individuals, presumably with very low viral loads, cDNA concentration techniques and additional amplification time are not required during thermocycling to bring the probability of false negatives within acceptable limits. I acknowledge the steady stream of papers in the protocol investigation literature providing reassurance about the merits of pooled testing, but those testimonials are precisely the kind of evidence that ethical committees are best qualified to weigh.

The last strand of this particular controversy concerns the possibility of “multiple counting” of the same individual as a result of multiple tests per individual. A claim has been made that compartmentalisation of results for recovering patients who are tested a total of three times to resolve a case addresses all the issues. Unfortunately it does not.

The different testing labs at present maintain separate indexes and case investigation form coding procedures. There is currently no efficient way to harmonise and consolidate multiple case investigations of the same individual, especially also as the case investigation forms map to a unique sample ID but not to any patient ID at all. Multiple cases submitted to different labs would automatically count as separate cases. Multiple cases submitted to the same lab can be harmonised against a single patient if the data is de-identified. In a pooled sampling regime, it is problematic de-identify the samples constituting every pool to check for history without defeating the original goal of saving time. So, here too, it is best for limitations to be openly acknowledged and solutions widely canvassed through open and sincere national conversations.

On the whole, Ghanaians seem quite impressed by the enthusiasm with which members of the country’s long neglected scientific community have embraced their role in responding to the public health emergency. The constraints being imposed by politicians’ lack of “data candour” are however slowly undermining the confidence some sections of the society have in the respectful coexistence of science and politics that marked the early days of the government’s response.

 

 

 

 

 

 

 

The Government of Ghana has announced a three-prong strategy for comprehensively responding to the Covid-19 crisis: Testing, Tracing & Treatment.

Of those three dimensions,  many observers feel that the first two are the most critical in the current phase of the crisis as they are more visible and more closely linked with prevention, which given the country’s limited resources, is far more critical than curing.

There is no doubt that tracing and testing are critical, but the strategy for doing both well is even more important.

In addition to early detection, effective tracing and testing also enable responders to use the number of *confirmed cases* to project/predict the pattern of *true cases*. In every epidemic, there are always many people with the disease whose condition is not known and has therefore not been recorded by the official health system. Hence, confirmed cases always lag and underrepresent true cases. What is important is for the confirmed cases to track the true situation on the ground reasonably faithfully.

For that to happen though, confirmed cases must constitute a *representative sample* of true cases.

Crudely, C —> xT, where “x” is a common or constant ratio, “C” is a measure of the confirmed cases and “T” is a measure of true cases. If you consider each daily count/announcement of the infection rate/level as a term in a series, that term should be expressed as closely as possible by the crude mathematical mapping relationship indicated above: C —-> xT.

The reason why one needs a roughly stable relationship between confirmed cases and true cases is because that is the only way one can use the official count of confirmed cases for any kind of policy management.

If the trend in confirmed cases does not reflect the underlying trend of true cases, then the official count becomes useless. No one can tell, in those circumstances, if any policy, such as lockdowns, are working or not.

For the confirmed counts to be representative of the true level of prevalence, the total number of tests doesn’t really matter as much as usually supposed unless the number of tests cover a very large proportion of the overall population, i.e. is in the millions. In Ghana’s case, tests underway are about 44,000, of which 15,000 have been completed (Presidential Advisor on Health, April 11th, 2020). The condition of overwhelming proportionality does not therefore apply.

What matters more than anything else then is how public health authorities determine and secure a *representative sample* of the likely exposed populations for testing without overestimating the true extent of the spread.

So far, I haven’t seen any clear analytical logic explained clearly by officialdom in Ghana as to how that high bar is being aimed for.

And given the logistical challenges in pooling samples, running tests, batching results, sending them to the Ministry of Health, which then releases the data to the GHS strongroom, before proceeding to inform the public, the actual data points in any global number announced on any particular day could be coming from any of the preceding days spanning a two- or even three- week period.

So, when the Ghana Health Service (GHS) announces that 30 more infections have been recorded between, say, the 10th and 11th of April, the breakdown of that “30” figure could easily be something like this:

A. 10 out of the 30 people announced as positive for that 24-hour cycle may have been tested 3 weeks ago.

B. 11 people tested 17 days ago.

C. 3 people tested 3 days ago.

D. 6 people tested on 10th April.

Thus, one is not looking at some kind of realtime dashboard of a consistently evolving situation. One is, in fact, looking at a mixed reality, composed of different snapshots across time. A lagging, composite, picture; not a sequential reel.

It is thus meaningless to say that infections are growing, slowing, growing faster or slowing sluggishly etc etc by simply relying on these global numbers. The current structure of data collection and delivery does not really allow a mere observer to say that.

The Government itself, on the other hand, has better insight into which tests came from which batches etc and therefore has better official intelligence to make those determinations. The general public unfortunately does not.

When the Government wishes to change the tone of policy, however, it would need to align its private picture of the epidemic with the public picture it has painted over time. That process, currently, is a work in progress as the authorities are now in the process of bringing more testing capacity on board by activating other laboratories in the veterinary services, the Tamale Teaching Hospital, the CSIR, the Food & Drug Authority and even, as I have recently heard, Korle Bu Teaching Hospital.

This will make the official counts (example: 408 infections as of 11th April) a truly dynamic picture of the true trends.

Aligning the *public trend picture* with the *official trend picture* is however only one of the two critical things that have to happen to make government policy more reflective of the supporting data.

The second task is what I mentioned earlier: aligning the confirmed cases picture with the true cases picture by ensuring something as close to a constant/common ratio in the daily progression of announced counts. That is to say, work must be done to increase confidence that the confirmed case count for day one is roughly consistent with the true, unknown, case count on day one and the confirmed case count for day two is roughly consistent with the true case count for day two.

In simple terms, if on day one, there were 200 confirmed cases but the true number of infected individuals is 4000, then if the number of confirmed infections move to 220 on day two, the true level of infections must also shift close to 4400. Note that this is more critical than the absolute number, whether 200 or 220. And therein lies an important distinction between the alignment point canvassed in this brief note and other concerns swirling around about what the true prevalence level might be.

These two alignments would then enable the Government to make forward-looking policy based on whether previous policies are having a statistically significant effect or not.

Until those alignments are in place, policy is merely provisional.

Naturally, I have had to severely simplify epidemiological statistics to a great degree in order to make the quick point I intended to make here. But the core points are valid. Refinements using standard biostatistical methods and techniques won’t change the fundamental insights too much.

How can these alignments be achieved then?

Aligning the public and official trend pictures would require improved logistics for sampling and increased capacity, which the Government is already working on. The strategy there is quite clear.

Aligning the confirmed case count more uniformly with the true level of prevalence requires serious modelling of the spatial distribution of the Covid-19 burden in Ghana presently using historical data of where people from overseas usually disperse among the population. And then conducting mass randomised testing that omni-axially tracks infection dynamics along certain key radial pathways. But it also requires deliberate validation of “control sites”. One does not want to overestimate prevalence anymore than one wishes to underestimate it.

In connection with this second angle on alignment, the government’s plans are vague. What has been said publicly suggests considerable gaps in process design since the entire enhanced tracing regime has been based on direct tracing of returnees and attempts to identify and test their direct contacts.

At any rate, the distribution of contact tracers in the current process does not follow a statistically rigorous distribution pattern. Well noted returnee hotspots like Asante Akim and the Techiman area have seen very limited tracing and limited risk-based sampling for mass testing. Nor is any attempt being made to validate assumptions about “non-hotspots” in order to reduce “data anisotropy”.

Part of the challenge arose from the initial skewing introduced by designing contact tracing around the 1030 international arrivals placed under mandatory quarantine. Depending on which day of the week, the cohort of such arrivals would not be adequately representative of international arrivals since the outbreak intensified. The Government’s decision to extend the coverage to most of the entirety of March helps matters but does not entirely dispel the data challenges since the training and effective distribution of trackers nationwide takes time to build up, during which period case contact trails become more convoluted.

The most critical issue of all, though, is the lack of public awareness, even at elite levels, of these gaps and the timeline for fixing them. This makes political milestone management lax since critical observers don’t know how to measure the progression of the health authorities towards this all critical point of alignment.

The media, in these times that civil society is taking a backseat to give Government space to focus on relief, needs to better understand the statistical aspects of the pandemic so that they can nudge the government towards delivering and communicating more effectively on the *twin alignments* discussed here.

It is absolutely imperative that Government assessments of whether the country is doing well or not be sufficiently transparent and logically easy to follow so that the roadmap to success is not hijacked by distrust, morbid partisanship and confusion.

When the time comes for the Government to loosen restrictions and actively kickstart the resumption of economic activities, the collaboration of the citizenry shall be vital. Much better if the logical journey to making those decisions has been made clear from the outset to the larger part of the population for the most part.