Hot Topics in Public Health with Laura Ferguson

View all blog posts under Videos

Join us for a Hot Topics in Public Health webcast with Dr. Laura Ferguson, MPH Online Assistant Professor of Preventive Medicine. Learn why mixed-methods approaches are beneficial to research and how they differ from solely qualitative and quantitative methods. Gain insight into mixed methods research being implemented in sub-Saharan Africa. Understand the challenges and positive outcomes from a mixed-methods approach to maternal and child nutrition and health in The Gambia. And, learn about the USC MPH Online Program.

Produced by: The University of Southern California’s Master of Public Health, the only online MPH delivered by a top-ranked medical school with world-renowned faculty.

Transcript

hello everyone and welcome to the master

of Public Health online programs faculty

spotlight webinar with dr. Laura

Ferguson presented by the Keck School of

Medicine at the University of Southern

California my name is Kiana Carter I am

an enrollment advisor for the Master of

Public Health online program and I like

to thank you for taking time out of your

busy schedule to join us today before we

begin I’d like to review what you can

expect during the presentation to cut

down on background noise please mute

your phone lines so as not to disturb

the presenters if you have any questions

for our speaker please type them into

the Q&A box in the lower right hand

corner of your screen and hit Send feel

free to enter your questions as you

think of them and we’ll answer as many

as time allows at the end of the

presentation a copy of this presentation

will also be available shortly after now

here’s a quick look at what we’ll be

covering first we will hear from our

program director dr. Shobha Kumar who

will talk about her background and share

some information on the Keck School of

Medicine of USC then we will hear from

one of our students William Jardel who

will introduce our speaker dr. Laura

Ferguson lastly we will end the

presentation with a brief Q&A session

again ask your questions type them in

the Q&A box and we will answer as many

as time allows now let’s begin hello dr.

Kumar hi Thank You Kara and thank you

everyone for being here today I’m very

happy to be presenting this webinar with

  1. Ferguson who I work with here at USC

in the department of preventive medicine

as well as with the Institute on any

qualities in global health a little bit

about my background so my back

also in global health like dr. Ferguson

and I did my mph and my PhD looking at

really management issues in the global

health context how NGOs are managed as

well as how financing flows

accountability issues in global health

and really kind of that aspect of

monitoring devaluation of programs

that’s that’s really my background I

used to work primarily in the NGO world

before academia and that’s where a lot

of my research as well sits in terms of

the Keck School of Medicine at USC

we were actually established in 1885

we’re the oldest Medical School in

Southern California and the university

itself is established in 1880 so we’re

over a hundred years old and dynamic

place for activity in terms of research

patient care education we have several

institutes and centers leading research

whether it’s you know clinical

translational science if global health

is epidemiology and all kinds of matters

central to public health and within our

department of preventive medicine we

have over a hundred faculty who are

doing research within disease prevention

bioinformatics five statistics the

genealogy cancer environmental health

health behavior and many other topics

and many of these folks teach in the mph

program both on campus and online so

that’s that’s a little bit of background

about USC and about our program at this

point I’d like to actually turn it over

to will Joe Dell will is one of our

fantastic students in the online mph

program and he is also the student

ambassador for the Master of Public

Health student association group on

campus he is the online student

ambassador and you know it’s really

great because we’ve had a lot of

engagement and interaction with our

students online and this particular

initiative this faculty series webinar

is driven by our students they were the

ones who came up with the ideas and I

really wanted to hear more about what

our faculty are doing what kind of

research topics they’re engaged in and

just understand what

in the world of public health in an

informal way and so this initiative was

really driven by the students and now

like to turn over to well hi everyone

I’m excited to introduce dr. Laura

Ferguson today for a talk on mixed

methods evaluation pitfalls and promises

in the real world he is an assistant

professor of preventive medicine at the

University of Southern California and

the director of the program on global

health and human rights at the USC

Institute on inequalities in global

health her research focuses on

understanding and addressing health

system and societal factors affecting

the uptake of health services and

developing the evidence base of how

attention to human rights can improve

health outcomes it collaborates with the

range of United Nations agencies as well

as foundations universities and

non-governmental organizations in

different countries

she has been extended periods of time in

low-income countries primarily

sub-saharan Africa collaborating with

partners helping to build the capacity

of her local colleagues and designing

and managing research and projects to

tackle a range of issues including HIV

and AIDS sexual and reproductive health

in child health please join me in

welcoming dr. Laura Ferguson hi everyone

and thank thanks well for that

introduction I’m going to talk today

about mixed methods evaluation but

before I really jump into the substance

I’ll give you a little background as

part of who I am and why I want to talk

about this my background really I

started off working for nonprofit first

in Central America actually but quite

soon after that I moved her in Africa

and was really involved in

community-based health in those in those

places until I realized that she didn’t

know anything about health at all so I

decided to go to grad school we thought

I’d go straight back to Africa after

that I got a little bit involved in

advising around global policy for the

United Nations system that that could

have captured my interest I did that for

a few years before doing my PhD and that

then obviously brought me into a

research space and and an academic

setting and and what I try and do in in

this role is that I work still doing the

advising to the UN system to what Health

Organization UN aids UN Development

Programme and others but I also still

work with nonprofits and I find it

really beneficial to have a foot in both

of those worlds because the nonprofit’s

we tied into the real world and what’s

actually happening and those seem like

really important lessons to feed into

people who are making decisions around

global policies and then I can bring

whatever global policy decisions are

occurring back down to the grassroots

and make sure that they’re appropriate

for implementation and follow-up on that

so I’m very much in an applied research

space where I work with this different

range of partners to try and make sure

that that lessons we’re learning are

being applied appropriately in both of

these different spaces um the terms of

mixed methods research I have to say I

think I’m a qualitative researcher at

heart still but when I had what I felt

was my first really good research idea

and that was when I was living in Kenya

and I said to the folks running the

health system there that they were

testing all pregnant women for HIV and

and diagnosing a lot of lots of women

but I could see that they weren’t

turning up to the HIV clinic so all

these people were being diagnosed with

HIV but then disappearing from the

system and so I said to them that this

this is this is great right you

diagnosing people because you want them

to treatment let me do a study to figure

out why they’re not turning up and they

wouldn’t let me do it because they

didn’t believe it was a problem I had to

redesign the study

first do some quantitative research to

make them see that it actually was a

problem and then as soon as I could do

that they’re like wow that’s really

interesting that’s real problem can you

find out why so I could then do the

study that I wanted to do and so all

that to say really like that different

types of data answer different questions

and speak to different people and it

seemed really important in trying to

affect change in the world to be able to

play in both of those spaces and to

bring them together in ways are

appropriate for speaking to the

different audiences that that you have

to work with to do this work and and

really it’s about becoming sufficiently

well-versed in both quantitative and

qualitative that you understand how they

fit together and that you can supervise

teams to do that kind of work so I don’t

want to stay in the abstract today even

I love talking about methods most people

don’t quite as much as I do

what I wanted to do was to talk really

to frame it around a study that I’m

running in the Gambia and this is a

mixed-methods research study and so I’m

going to talk you through why we chose

the methodologies that we did what we

actually did I’ve mentioned some of the

findings but really I’m doing all of

this to illustrate why I think that

mixed methods worked is is important and

and how how it matters I press the wrong

button there we go so the project that

I’m going to talk about is the project

itself is called improving maternal and

child nutrition and health in the Gambia

and I’m going to talk about the study

design for the evaluation of of this

project just to give a little bit of

background this is a map of the Gambia

which is on the coast of West Africa the

project itself is about maternal and

child health and nutrition it started in

2005 begin of 2015 following a pilot

test than the earlier

it’s funded by the World Bank and

implemented by the government and

they’re implementing in three out of the

seven health regions in the country and

these regions were were chosen because

they have porous health and nutrition

indicators and so it was felt that we

could have most impact in these areas

and we’re actually covering a third of

the population of The Gambia so that’s

about 600,000 people and you can see

it’s a small country and yet covering

600,000 people with an intervention

still feels like quite a large

undertaking but as I said I’m not

involved in implementing the project the

World Bank gives money to the government

of the Gambia specifically the Ministry

of Health and the National nutrition

agency and they implement all I’m doing

is trying to fit a study around what

they’re doing so I could tell them at

the end of the day if it has the desired

impact or not

so the overall impact is they want to

improve maternal and child nutrition and

health but the goal of the project so

bringing things down a little bit is to

increase the utilization of community

nutrition and primary maternal and child

health services in selected regions in

The Gambia okay so that’s what it is

that the government has said that they

want to do through this work question

that I was then given was well how would

you measure this and now before we can

answer that we need to look a little bit

more at the project and say can what are

they actually doing before we can decide

whether or not if they’re being

successful so here’s an overview of the

project itself it’s quite a complex

project and keeping this at fairly high

level just general intervention

information I’m really happy to go into

the details in the Q&A if you want but

right now I just want to give you enough

context you can understand the study

design around it and so this this

project is a result based financing

project and for those of you who aren’t

familiar

traditionally health systems are

financed by you know every health

facility if you like is given a budget

and told to go and deliver services

results based financing flipped that on

its head a little bit and says go and

deliver services and show us that you’re

meeting targets to certain services and

then we’ll give you money as a reward

for having done that so the more

services you deliver the more money you

get and there is an aspect of quality

measurement in there as well so that we

don’t totally lose that within less but

that’s actually the intervention – on

this slide the supply side so the

delivery of selected primary health care

services is incentivized through results

based financing or performance based

financing within the health facility so

health facilities where this

intervention is taking place are getting

extra monies for the delivery of these

selected health services intervention

one is on the demand side because as as

we know in public health even if the

services are there if people don’t want

to access them they won’t and so we

decided that it was important to

incentivize demand for services and so

that’s done by two things there’s a

community results based financing

component through which groups within

the community get incentives for them

carrying out health education activities

for them referring people to health

facilities as a whole series of

behaviors for which there they’re then

given money as long as they can show how

many people they’ve referred and all of

that kind of stuff they get subsidized

for that and there’s a conditional cash

transfer for pregnant women the pregnant

women who show up for antenatal care

within the first trimester will get

given a payment if they turn up for

another three routine antenatal care

visits they’ll get another payment so

this is really about asking people to

behave in a certain way on the promise

that if they do they will get financial

records

for it that that’s the entire project so

it’s really about how do you change

people’s behavior within the community

and within the health facility by giving

them financial incentives so I’m a big

believer in theory and in conceptual

frameworks so what I wanted to know is

if we do these interventions here what

what chain of events do we set in motion

that actually going to lead to improved

health outcomes at the end of it so we

created a conceptual framework for the

project we’re not the first people to

try and improve maternal and child

nutrition and health we’re not the first

people to implement results based

financing but we wanted to figure out

what can we learn from previous project

about how these things are meant to work

this is a super complicated IDE and we

don’t need to go into all of the details

but what we try to do here is to say if

we do some of the things on the left and

the organizational changes and the

behavioral changes what are the pathways

that will lead us to improved health

outcomes and we’ve tried to sketch some

of them out on this slide and I’m not

going to go through it all but I think

it’s really important to highlight

because we need to understand the

mechanisms through which we think our

project are going to work and this will

help us when we come to measuring and

evaluating them and I’ll come back to

that in a bit this is our conceptual

framework for the supply-side

intervention we had another conceptual

framework here for what was the work

that was being done in communities that

basically serves the same purpose of if

we implement our set of interventions

through what pathways will that improve

uptake services I’m not going to dwell

on that that hugely interesting that’s

important to have for in any project

that you’re doing or even any research

study it’s about having your hypothesis

and your hypothesized pathways of change

so here’s a list of indicators of the

World Bank prioritized for this project

they decided like if the goal remember

the goal is to increase the uptake of

maternal and child nutrition and health

services this is how they want to

measure that they want to know that

there’s an increase in the percentage of

children aged 0 6 months are exclusively

breastfed they want the number of

deliveries attended by certified

midwives to increase they want the

number of children are appropriately

supplemented with vitamin A to increase

in want more women using modern methods

of family planning and the last

indicator which we’re going to just put

this out there be super honest fake I

don’t think is very useful tool is

something that the World Bank has in all

of their projects they’re trying to

measure how many people they reach

through the funding that they give from

an evaluation perspective I don’t think

it helps but important to understand

that donors sometimes have these things

in there that we go along with ok so

this is what we’re trying to measure

then let’s look at what the study design

was and it’s a very it’s a complex study

design we did a 2×2 phased in randomized

control trial what that means is first

of all important word is randomized so

we randomly assigned health facilities

and communities to intervention groups

and control groups but we did this in a

phased-in way by the end of the project

period

everybody had received the intervention

but the way that we fade it we phased it

in created these for comparison groups

you can see in the little table here a b

c and d where a is a pure control group

where they don’t have either the demand

side or the supply side intervention

going on b shows communities that

receive the demand side intervention but

they’re not in catchment areas of health

facilities with high side interventions

and leslie c show

is there are health facilities where

interventions occurring but communities

around them that do not have the demand

side intervention and then finally D are

places that have both the demand side

and the supply side intervention this

was really important for us because we

hypothesize that if you do the supply

side on its own you might have some

impact if you do the demand side on its

own you might have some impact but if

you can do them together that’s probably

where you’re going to have most impact

so we needed to be able to quantify that

in our study findings that was how we

chose side um oh we going the wrong way

let’s try again

there we go so we included there were

times there are 24 health facilities in

these three regions and two intervention

communities and to control communities a

health facility remember I said that we

randomized this literally meant that we

did we went out into communities and

held lotteries where everybody was

present so everybody could see that this

was a random assignment and that was how

we determined when the different

facilities and communities would enter

into the intervention and it’s again

just show that our timeline for the data

collection and this is just on the

supply side but you can see the orange

boxes is when that’s when the

intervention begins in these different

facilities and we have some evaluation

before anything we have midline

evaluation which helps us capture a time

when half of the facilities had an

intervention and half didn’t and then by

the end line everybody has the intern

again that’s what when I’ve talked about

the phased in nature of the study that

that was that’s really what that was

like okay let’s get to to method a

little bit and that is not the method

slide hang on yes the order of these has

got a little bit confused but I’ll jump

around I can I can do this okay I want

to start with the quantitative data

Colette

and you can see on the left there’s some

household data letters and some

community data but there are I believe

seven different questionnaires there

each of these is administered at

baseline at midlife and nline and you

can see the sample size there from

baseline and midline the household

questionnaire is a huge sample of over

2,000 health hold each round the other

questionnaires are all much smaller and

a question that comes out a lot is why

on earth do you need seven different

questionnaires to evaluate a single

project and the reason that we wanted to

do this is that each of these captures

the perspectives of different people

when the households we were talking to

women who used health services or maybe

they didn’t use health services but

there were women who had been pregnant

or delivered recently and so we wanted

to know about their experiences and

decisions then we talked to health

facility managers and we also did some

direct observation of what stuff look

like in health facilities we talked to

individual health workers about their

motivations on the project we did exit

questionnaires for clients who had just

attended antenatal and child health

services to like really see what just

happened in that consultation that you

were just at and then we talked to our

folks in the community the village

development committees and village

support groups which are kind of

responsible for health education in the

communities the idea was we wanted to

get loads of different perspectives so

we could try and piece together what it

is that was actually happening and just

to take a step back for a minute I

wanted to say something about the

logistics of this this really is large

large scale work my team in The Gambia

was aware I think I had slightly over a

hundred people involved in the

quantitative data collection and I have

an analysis team of about five or six

statisticians who are all around the

world who are helping crunch these now

so coming back to the fact that I do

mixed-method I could not crunch all of

these numbers and I certainly couldn’t

collect all of the data but I know

enough about the methodologies and can

train all of these people that they they

can then do the day-to-day to make is

that’s the quantitative piece of this

then there’s the qualitative piece which

we did alongside again you can see on

the Left all of these different groups

of people that we talked to and we did a

mix of focus group discussion and key

informant interviews now the question is

well why do that why do some focus

groups and some interviews well

different topics and different

respondents it’s better with one or

other of these for example we did focus

group discussions with women again women

who just have kids like find out about

community norms about childbearing about

you know general experiences of having

having children seeking care in within

their communities but then separately we

did individual interviews with what

we’re calling vulnerable women and these

are women who are for example

adolescents who just had children or

widows or people out of wedlock so

they’re generally more marginalized

they’re the kind of people who are not

going to speak up in a group setting but

we still really want their perspectives

because they’re a constituency that

we’re trying to reach so that’s kind of

like one of the factors that goes into

figuring out the methodology that we use

even within the qualitative space to

collect data and again really try to max

out the number of different perspectives

that we could get on on all of these

issues and again I’ve got a whole team

working on this about five or six local

researchers in my Gambia who did a lot

of the data collection and then a team

of about eight or ten analysts who are

working with me on on the data analysis

in terms of data analysis again I don’t

want to go into this usually invest but

just so you understand roughly the the

overall approach to analysis for both

quantitative and qualitative the

quantitative is based essentially on

multivariable linear linear regression

models again I’m really happy to go into

the nuts and bolts of that if there’s

like Quan 22 people here who want to do

that but for now that’s I think enough

enough to know and the qualitative we

transcribed all of our interviews and

focus groups and coded those data based

on a code tree that was initially

derived from the literature and from our

conceptual frameworks that I showed you

earlier so we’re really looking to see

if people had anything to say about the

pathways we were hypothesizing but as

people talked about other random things

as well we added in those codes that we

didn’t lose any of the data that we were

we were getting but again okay so this

is back to the main project indicators

as per the World Bank priorities just to

show how they set things up is that the

indicator shown the unit of measurement

a baseline value is given of what what

things are up at the very beginning of

the project and then all of these

targets for what they want to happen

over the five year lifespan of the

project and again this is diving for us

as to we need to make sure that we’re

capturing this type of information in

our study so this had a small and

confusing slide is a screenshot I took

actually earlier today and we set up our

questionnaires in Excel and just look at

the number of tabs this is from the

household questionnaire and you can see

like all the questions running along the

top of the spreadsheet and and there’s

many many more columns in this in this

tab alone and then you see the number of

tabs which I think barked at at 18 or 19

in the end this is a huge questionnaire

and remember that this is only one the

six other questionnaires as well so

again to note the the scale of it and

also the detail in terms of having to

provide the different response options

and patterns and this kind of big

question areas is a huge attention to

detail process and then again to take a

step back on the logistics we actually

administered the questionnaires using

tablets which may sound super logical

for here but no one had ever done that

in The Gambia before so first of all we

had to train everybody on how to use a

tablet then we had to deal with issues

of connectivity because there’s really

not good internet coverage in some

places – how are we going to do that we

contracted a company in Vietnam the

hosts all of the data in the cloud and

do data checks and stuff like that so

imagine how we’re working across this

range of time zones from The Gambia

through here Vietnam and actually Mike

OPI is in India so we were kind of

working around the clock in weirdly

efficient ways as as data were coming in

so we had like real-time data checks

going on but I just wanted to give you a

sense of what the questionnaire looks

like ok so here are a few of our

findings and again like I don’t want to

spend a ton of time on this but I do

think it’s important to look at

different arms of the study and what we

could tell so I’m going to talk you

through how to interpret this table

you’ll see that I’ve just put four

indicators in there and we can look at

exclusive breastfeeding discuss is

actually let’s look at skilled delivery

because the second one down and you can

treat all the all of the numbers are

essentially proportions so what this

that first column shows you is that 41

point three percent of women reported

that their last delivery was attended by

skilled personnel at baseline okay so we

know that their self starting point 41

percent of these women were attending

delivery then if you look at the midline

you can see that in the control arm that

had gone up to 46% while in the supply

side arm it’s actually gone down to 38%

in the demand side arm it was up to 47%

and in supply and demand it was up to

48% and then if you keep looking over

the next three columns show you the

program impact to the change relative to

the control group in each of their study

arm and really importantly the final

three columns are the P values you can

see that third skill delivery none of

them are below our magic threshold of

point zero five so none of those changes

are actually statistically significant

what we can tell from this is that if

we’re talking about skilled delivery

there was no difference between baseline

and midline okay so that kind of seems

like sad news from our project’s

perspective right and so we needed to

figure out what’s what’s going on so

there are some other indicators that we

collected data on that we could look at

again I’ve replicated that skill

delivery line at the top here that’s

what we just looked at underneath that

you’ll see referrals to delivery

accompaniment to delivery and women who

have actually were transported to

delivery and if you look at the program

impact columns anything that’s bolded

there is statistically significant

consider even though there wasn’t a

change in skilled attendance at skill

delivery more women were being referred

to and accompanied to and even

transported to delivery in the supply

and demand side arms and so this is

promising because all of these are

intermediate variables towards

ultimately increasing skilled delivery

we can say that although at midline we

didn’t see an increase in skill delivery

we did see signs to suggest that we

might get there in the end and we hope

that we’ll see that in our headline data

which is something that we’re going to

look at what we’re actually looking at

right now we don’t have the results yet

so a few things to point out here the

importance of intermediate indicators in

understanding

outcomes of interest right it’s not

enough for me to say look there’s no

difference in skill delivery we can say

okay there’s not absolutely but we see

that some things are shifting and that

could helpful the other thing that I

think is really interesting from this

table is that we do see that synergy

that we hypothesize would exist between

the supply and demand side interventions

the we’re in communities where both of

those were implemented in general we see

the largest positive impact from the

interventions of course is intuitive but

you never quite nervous if it’s going to

play out like that or not okay so what I

want to know is why are women not yet

turning up to delivery why are more

women being referred and more women

being accompanied and so these are the

kinds of issues that we delved into in

our qualitative data collection and this

is really where you can use qualitative

data to really try and explain the

numbers and these were some of the

topics that had come up that we thought

might help explain why it was that

people may not be turning up for

delivery

even if they were being referred and

what came out or we’ll look at some of

the findings in a minute

but these beef questions were informed

by the findings from our baseline as to

what we thought and that some of the

barriers might be but our idea was let’s

use these women’s stories as a way to be

able to explain what’s going on with the

quantitative and so we found that some

things were working really well and you

see here quotes from village committee

members saying that health seeking

behavior for women to go to the health

center delivery is happening that wasn’t

happening before all pregnant women used

to be delivered by the traditional birth

attendant at home before the project

that that has changed now somebody’s

reporting that change is happening and

again these are health workers saying

deliveries most of them are now

delivering at the health facility it

they deliver on the way coming ah that’s

interesting

so maybe there are those who are late to

inform their husbands or in-laws or the

traditional birth attendants those might

be the ones delivering on the way but

otherwise they’re all coming so again

the health workers are saying people are

coming and then the final quote as well

is essentially the same thing saying

make women understand that they should

deliver in a health facility and they’ll

do it so this does not reflect our data

on uptake a skilled delivery but it does

reflect some of those intermediate

indicators that we look there but we

also wanted to figure out well what’s

not working so you see here some

different perspectives and you see some

health workers who say there are some

women here who’d rather die at home than

come here during during childbirth

because there’s no female with midwife

and super-interesting a woman who had

delivered recently said if your husband

doesn’t have a horse car or cannot hire

one you have to deliver in the village

even if you want to go to the health

facility the issue there is around

transportation then that somebody is the

man in the community talking about

quality of care and said the way I was

received when I asked or my wife who was

in labor was not the least satisfactory

to me that said they were having their

breakfast and didn’t have time for me

when my wife called for assistance I

found her a full labor and she delivered

in my hands so perhaps a disincentive

for attending health facility and then

from a health worth but worker

perspective and saying that this is

actually really challenging because we

don’t have the delivery beds we don’t

have the facilities to do this but the

reason to look at these qualitative data

is we you know we’ve got some insight

into like people saying there are

aspects of this that are working really

well and we know that it’s that people

are saying that change change is

happening but also I think even more

critical especially the midline point is

looking at what’s not working if there’s

an issue that there’s no female midwife

and and culturally that unacceptable

then you can put in in place whatever

incentives you want but people are still

unlikely to go and

access care so these data are really

important not just for understanding my

data from from my midline surveys but

this has to be used for course

correction for the project and that was

sort of one of the primary aims was to

help them figure out how to improve the

project for the remaining implementation

period by learning from from these data

and so the qualitative data provide all

these insights you just can’t get from

from the numbers alone okay but I just

wanted to take a step back before

wrapping up which is about the

real-world challenges of doing this work

and this is partly about doing mixed

methods work and it’s partly about is

doing work in the real world and it’s

not like when you have a lab and you run

a controlled experiment the one during

the controlling right that’s me seems

like a learner’s incredible luxury what

I was doing was trying to fit around a

government implemented project and so

inevitably there are delays and any

delays that happened with the project

meant that I had to delay what I was

doing with the evaluation and that had

huge implications for staffing like when

what the right moments are for

recruiting people and and all of those

things so that’s just something you just

you know is going to happen you have to

deal with politics is huge and the

government of the Gambia was essentially

overthrown halfway through this project

everybody left the president was ousted

as he left the country he stole

everything so really there was no money

then to do anything new government had

never been in government before they had

no idea what to do so we went through

all these kind of macro political

changes that of course have an impact on

what’s happening on the ground for four

months there were literally no drugs in

many of these health facilities again

like incentivize as much as you like but

if there’s no treatment of a

of all people don’t go to health

services and so from a study point of

view you have to document that you have

to understand why it is that you’re

doing all this staffers still not seeing

that the results that you anticipate and

linked in to that is this idea of

contamination when we started this work

we were literally the only project doing

maternal and child nutrition and health

in the Gambia so it meant that any

changes that we were seeing were a

result of the work we were doing when

the new government came in lots of new

donors came in and started up this type

of project all over the place so we

suddenly found that in communities and

facilities where we were working there

were other people working on the same

issues and that brings in problems of

attribution in terms of any study how

did we know suddenly that any changes we

were seeing were down to ask what if it

was down to stuff that other people were

doing in the same places again we’ve

been really conscientious in trying to

document all of that and we can’t

account for statistically but we can try

and explain it in our in our write-ups

and then the final challenge that I want

to mention conflicting data and this

really is a challenge of mixed methods

research particularly and especially

when as I showed you we tried to capture

all of these different perspectives both

quantitatively and qualitatively and

what we found quite often was that

everybody would say one thing but then

we’d look and see something totally

different

or we’d collect some quantitative data

and we see something totally different

and and then you come up with a

challenge of how do you how do you do

that do you trust something that’s

written down more than what somebody

says now is this a problem with recall

or is it a problem with record-keeping

and and you can’t always tease that

apart totally and again it’s something

you just have to be very transparent

with in your research and I kind of I’m

not going to say I like it but it feels

like a bit of a mystery to be solved

every time a hat

and I kind of get into detective mode

and dig up as much detail as I can and

try and teach things apart that way but

it’s definitely something that that

comes up every time of like things just

don’t ever add up simply you’ve got to

try and figure out actually what’s going

on so I didn’t there were some really

strong features of this study design

that helped with that one was

randomization we know that there was no

difference between our intervention and

control samples at the beginning and we

checked that balance really carefully we

also the triangulation across multiple

data sources really helps with trying to

create some sort of confidence in data

and like I just said stuff conflicts and

you’re never going to iron it all out

and make it all add up but if you can

look across various different sources

you may be able to get a sense of what

it is that’s going on and so that’s that

I find incredibly helpful and really I

think that for me the ultimate strength

of this is by combining the quantitative

and qualitative methods had an impact

evaluation but with this embedded

process evaluation that really allowed

us to assess not just do these

interventions work or not but how do

they work or not and why do they work or

not and that’s where the richest

learning comes from implementation

science so for me that really what

what’s critical about bringing these

different types of methods together so

just to wrap things up these studies can

be like incredibly complex because

everything’s out of your control and you

just have to learn to live with that and

roll with it

it can be really difficult to isolate

the effect of a given intervention even

in a context of randomization if there’s

all this other work going on at the same

time and that mixed methods really help

try to to figure out how and why things

work or don’t but these kinds of studies

are really really expensive and and I

say that because it means just that you

have to be very rigorous about doing

them and very serious about the

questions that you’re

to answer but I also want to say they’re

really really fun and to have I have a

mountain of data over the last five

years and that feels like a really

exciting treasure trove to kind of delve

into and really there’s an awful lot

that we can say now about maternal and

child nutrition and health in The Gambia

on the basis of all these data that’s

that we’ve collected I’m going to stop

there and let you guys ask any questions

that you might have anyone can can jump

in Thank You dr. Ferguson for sharing

your work we will definitely try to

answer your questions I submit a

question please type it into the Q&A box

don’t be shy she is definitely here to

answer those questions for you don’t

have many of them right now but I would

love to ask a quick question to you dr.

Ferguson you know how would how would

students in our online program

interested in working with you for any

of your upcoming projects or initiatives

get involved yeah that’s a great

question I mean the tricky thing about

it is that I’m not in control of my

timelines and students need a kind of

time bound research experience because

everybody wants to graduate right and so

things are kind of last-minute often I

think that the first thing students can

do is email to indicate interest and

also to indicate why why they’re

interested like I believe that I believe

in student involvement in faculty

research but I believe it has to be for

a purpose obviously like extra hands to

my research is always great but what is

it that the student is looking for in

terms of developing new skills or

experience and that helps me try and

find a match to an appropriate project

because some of my work is mixed methods

and some of its not and so if somebody’s

particularly interested in

and this this kind of approach then that

would help me narrow down okay well what

projects might might be relevant and so

I think certainly I’m not going to speak

for faculty but for me it is about

knowing that students are interested in

but but knowing why and what they want

from it and then us working up between

us what that could look like and how to

make how to find something that’s going

to work both for what I need in terms of

support but also for what a student

wants to get out of it in terms of

learning okay thank you so much for

sharing our next question is do you see

more studies being conducted with the

mixed methods approach an interesting

question I do I think that there’s

increasing recognition of the value of

mixed methods approaches I think that

one of the challenges is that that quite

a lot of our educational programs and

and actually like faculty positions and

research positions are still siloed into

quantitative and qualitative work and so

I think that there are definitely

impediments to people doing

mixed-methods work because when you

think about even tracks in the mph some

of them are very quantitative right and

they don’t bring in qualitative methods

at all and it’s fine I’m not saying

everybody should be doing mixed methods

research but I think in many ways it’s

more difficult to get a solid training

across both quantitative and qualitative

and so you really have to be determined

to make that happen

but I yeah I do think I do think there’s

more and more of it and and it may be

that that’s about more teamwork working

with with people so if you come from

more from one side than the other

reaching out to people who come from the

other and figuring out how you work

might fit together so yes I think it’s

increasing and I think it could increase

more

okay thank you

are your next question is how do you

recommend students gain confidence

handling so much complicated data like

what she discussed other students that

have seemed so overwhelming yeah great

point baby step I think the way that I

I’ve done this with I mean I’ve had a

bunch of students actually working on

this data set with me some of them have

only done qualitative work because

that’s what they’ve wanted to do and and

everybody starts with with really a

bite-sized piece of the puzzle I would

never expect anyone to come in and get

their head around how to manage all of

these data and analyze them bring them

all together into some something

coherent that’s my job but to have

people either say look I only want to

look at what health workers said or I

only want to look at qualitative data or

I’ve got so student night right now

working on quality of care and picking

that that is a component across across

different sources so there’s different

ways of slicing and dicing the data into

something that that fits individual

interests that is manageable and so it

could be about looking at five

transcripts rather than 60 right or it

could be about looking at six

qualitative critics quantitative

indicators and to seeing what we could

say about that

there’s you can you can make things as

have smaller as large as you want in

this and so that if I’ve got postdocs

working on these data I’m going to throw

bigger pieces at them but if I’ve got

people who are in the mph program it’s

going to be a much more manageable size

that we carve out for people to work on

and then and then just one final

thoughts right and ask questions like

when you’re new to stuff like this it’s

of course it’s overwhelming but that you

have to ask questions to learn so I

think this where you need confidence

is to be able to do that to be able to

to say look I’ve got no idea about this

or that or or any of this please walk me

through this please move you through

this again please walk me through this

in two different ways that I can

understand it but I think that that’s

that’s where you’ve got to really put

yourself out is if things aren’t clear

and of course they’re not going to be at

the beginning you just got to be

persistent in asking questions in them

as many different ways as you need to to

get an answer that make sense to you

absolutely absolutely

thank you our next question is what are

possible ways to dis do to decrease

expenses in methods and that’s tricky

here um so the tricky thing with

quantitative data collection is that if

you want to be able to detect change in

a statistically meaningful way you need

a relatively large sample size and

because of we wanted to be able to look

at effect between control arms supply

only demand only and supply and demand

together arms we needed a particularly

large sample we could have just said

look we’re going to stand solid and say

we think the way to do this is the

supply and the demand side intervention

together and it’s not even worth testing

use of the others on their own and we

could have done a much smaller sample

size so that’s one way of doing it is to

narrow down the number of different

things that you’re looking at and that

might be able that might mean you can

decrease your your sample size and to be

honest is that quantitative that’s more

expensive than the qualitative because

if you think about hiring a team of over

a hundred three times over to go out

around half the country to do this

that’s really where a lot of the costs

come in the only the other way that we

could have done it and from a study

perspective would have been absolutely

fun

main would have been to not do a midline

evaluation so we could have done a

baseline and then just come back five

years later and done an endline the

reason I didn’t want to do that is

because I really think that we learnt

useful things for informing the

intervention at the midline point and

because nothing like this had ever been

done in The Gambia before we wanted to

be able to learn those lessons and and

feed into implementation if you don’t if

that’s not a concern and if you’re

really looking at this from a research

perspective we could have cut our cost

by 1/3 right there just by cutting out

one one round of data collection and

then as orders there’s always

compromises that you make and it depends

on what your what your priority learning

objectives are how you go about doing

that I mean we we could have done this

on a much much larger scale with a lot

of sub studies embedded and from a

research perspective I’d have loved it

but it just wasn’t practical from a

budget perspective and so we prioritize

here are the questions that the country

wants answers here are the questions

that the world bank wants answered and

then here are the questions that I

managed to squeeze in there alongside

that that I thought were particularly

interesting and that’s one of the

wonderful things of being a p.i is you

get to do that alright thank you so much

for sharing that again you know she is

available to answer questions please

don’t be shy you can type your question

in the Q&A box and we will definitely

answer them for you

I believe dr. Kumar had a question for

you at this time as well dr. Kumar yes

thank you can thank you dr. Ferguson I

was really interesting I wanted to ask

what would you say is the most rewarding

part of your work a great question I

think the most rewarding part is seeing

change happen

you know I’ve been going to the Gambia

now multiple times a year for

five years and things really have

changed visibly in that time in a lot of

the facilities where this project is

working and in a lot of the communities

where this project is working so there’s

something exciting about that because

you realize that this has an impact on

people’s lives

and that’s ultimately why we’re doing it

is that we want to be improving people’s

lived experiences and when you get to be

involved over this this period of time

you get to see things like that and and

we’ve also been able to use our findings

to feed into not only government policy

which obviously affects the two million

people in The Gambia but also donor

policies so we’ve been like shaping

World Bank prioritization in other

sectors like some of their

infrastructure development projects now

targeted health facilities because we

pointed out that actually none of them

had electricity and this was this maid

service delivery hard USAID and actually

the Global Fund as well have also asked

us for certain bits of data to inform

how they prioritize their giving in the

country and so it’s to me I mean

obviously as a researcher being able to

[Music]

create data that are relevant and

because it can have immediate impact in

terms of what’s happening at a country

at the macro level of policy of donors

donor giving and the very individual

level of ultimately we know that this is

changing people’s lives I think that’s

what gets me excited about the work

wonderful thank you so much for sharing

that we have one more question for you

  1. Ferguson in general do students who

apply for grants to conduct

mixed-methods studies get funding versus

those who choose quantitative versus

qualitative methods only that’s a really

hard question to answer in the abstract

because it depends very much on where

the funding is coming from there

I mean different thunders have different

interests and I think it’s really

important when figuring out your

research that you there’s a balance you

have to start in my mind with a research

questions relevant and the appropriate

methods that go with it you have to

that’s you’ve got to be your starting

point and then you look for potential

funders who will who are interested in

those methods and if you can’t find them

and you have to start making compromises

around your methods because you want to

do mixed methods but you can’t find

anyone who’s going to fund that then you

need to go back and think say well if I

did do only a quantitative study on this

can I still answer my research question

sufficiently from my perspective or if I

only do qualitative is that sufficient I

think that there’s a very careful line

that you have to tread between doing

meaningful work and falling into donor

priorities because that’s what you get

money for and I get from a practical

perspective that we all need funding to

do our research but I think there’s

something really important about

maintaining the integrity of what it is

that you’re trying to do and really and

fighting for it with donors it was a lot

of donor education to be done about why

mixed methods are important so I don’t I

don’t think mixed methods makes it

impossible to get star funded I just

think and again it depends on what the

donors interests are but if somebody’s

coming to me and what I want to know is

is this study solid are these the

appropriate methods to answer the

research question and if that’s purely

quantitative fine if it’s mixed methods

fine

so it’s really about making the argument

for why it is you’ve chosen a particular

set of method to answer your research

question of interest thank you all right

if we don’t have any more questions I

definitely want to thank you

  1. Ferguson for sharing with us today

thank you to our our speakers on dr.

Kumar William Jardel and to everyone who

participated today if you have any

questions regarding the enrollment

process or you feel like you’re ready to

apply you can definitely reach out to

one of our admissions advisors the

contact information is on the screen

right now again a copy of this recording

and slide presentation will be available

shortly after this concludes today’s

webinar thank you again for joining us

and everyone have a wonderful wonderful

rest of the day

thank you