The purpose of the surveys was to estimate prevalence with useful precision. This means that you will not have an optimal design for comparing two proportions (e.g. in terms of statistical power). This does not mean that you cannot do it. The simplest approach is to look for non-overlapping CIs. If the two CIs do not overlap then this is evidence against the null hypothesis that the prevalences are the same. With some arithmetic this method can be extended to do things like estimate the difference between two prevalences or perform a hypothesis test. Kevin Sullivan has done this in this [url=http://www.sph.emory.edu/~cdckms/compare%202%20proportions.htm]this web-based calculator[/url]. I hope this helps.

From Bradley Woodruff: First of all, the sampling method influences only the precision. If done correctly, the sampling method does not change the actual point estimate. For example, let's say you do one survey of nutrition status in pre-school age children in a population using cluster sampling of households, and you include every eligible child in selected households. In another survey of the same population, you use simple random sampling of eligible children. If the prevalence of some nutrition outcome, for example, wasting, has not changed between the surveys, the 2 surveys should produce similar estimates of prevalence. The only thing different between the results will be the precision obtained. If the prevalence of wasting obtained by the 2 surveys was 10%, the first survey might have 95% confidence intervals of 6% and 14%, while the second survey might have 95% confidence intervals of 8% and 12%. This is because cluster sampling usually results in a loss of precision. But if your sampling is unbiased, the point estimates should be similar. The best way to compare the results of surveys is to put the data from both surveys into one dataset. Be sure each child is identified with a cluster number (in the case of non-cluster sampling, each child will have his/her own cluster number because the cluster size is 1). Also assign to each child some code to identify if the child is from survey #1 or survey #2. Then do crosstabulations of survey number by nutrition outcome. The computer, if you are using an appropriate computer program which can account for cluster sampling, should give you a p value for the difference between the results of the 2 surveys. Of course, often you do not have the raw data to put into the same dataset. In this case, many programs will allow you to enter for the 2 surveys the estimates of prevalence along with some measure of precision for each survey, such as standard error or confidence intervals. The program will then compute a p value or some other measure of the statistical significance of the difference between the 2 surveys. Do NOT just compare the confidence intervals of the 2 surveys. This is a very common mistake. Overlap of the confidence intervals from the 2 surveys does NOT mean that the difference is not statistically significant. However, lack of overlap does mean that the difference IS statistically significant. If you rule out statistical significance because confidence intervals overlap, you are underestimating the precision of your comparison. The statistical reason for this is that in calculating the p value for the difference between 2 survey results, you need to use the weighted pooled estimate of the variances for the 2 surveys because under the null hypothesis that there is no actual difference between the surveys, you assume that the variances for the 2 surveys is the same. Nonetheless, as a quick screen, you can compare the confidence intervals, but you must keep in mind that if they DO overlap, then you can make no conclusions about the statistical significance of the difference and you must calculate the specific p value for this difference, either by hand or by using a computer program.

Woody is correct. The simple approach of looking for an absence of overlap of two CIs is a conservative (i.e. low power) test. This means that it has a tendency to not rejecting the null hypothesis when differences are not very large. The link I sent above takes the (more correct) approach outlined by Woody (above).

Kevin, Just checking on previous posts and find that this link: http://www.sph.emory.edu/~cdckms/compare%202%20proportions.htm appears to be dead. Is there a new link? Mark

From Kevin: Hi Mark- our IT group changed the address - the new address is: [url]Web1.sph.emory.edu/cdckms[/url]

Just fixing the link to Kevin's calculator. It is now [url=http://web1.sph.emory.edu/cdckms/compare%202%20proportions.htm]here[/url].

Dear All 

Now i am contended with replying a reviewer comment about change in survey prevalence estimated between two years. I have cited the weblink on this post. The webcalculator is not working anymore. Can anyone help me provide the link that works.

Thank you in advance

Chanadrshekhar Sreeramareddy

Hello,

I got in touch with Kevin Sullivan. He is, I think, no longer at Emory and cannot provide an working link. He sent me an Excel file that does teh calculations needed. I have put it here.

Dear Moderator,

Many thanks for helping me get the excel sheet. It helps a lot.

Regards

Chandrashekhar

Goob. I am happy to have neem able to help.

BTW ... I used to be a moderator here nut I am now just a member.