Results

Table 2 shows the results for the run submitted. The first and second column indicates the number of topics evaluated and the test type. The last column shows the Mean Reciprocal Rank (MRR) obtained for each test. Additionally, the average success at different number of documents retrieved is shown; for instance, the third column indicates the average success of our search engine at the first answer.

Table 2: Evaluation results

		Average Success at
#Topics	Test	1	5	10	20	50	Mean Reciprocal Rank
1939	All	0.0093	0.0217	0.0294	0.0371	0.0464	0.0157
1620	Auto	0.0025	0.0049	0.0086	0.0117	0.0160	0.0040
319	Man	0.0439	0.1066	0.1348	0.1661	0.2006	0.0750
810	A. bi.	0.0037	0.0062	0.0099	0.0123	0.0148	0.0049
124	M. new	0.0323	0.0968	0.1129	0.1613	0.2339	0.0657
810	A. uni.	0.0012	0.0037	0.0074	0.0111	0.0173	0.0031
195	M. old	0.0513	0.1128	0.1487	0.1692	0.1795	0.0810

The best overall results by using the new topic set is presented in Table 3. The results are reported on all topics (all), the automatic (auto) and manual (manual) subsets of topics. The average of the submitted run is shown in the fifth column.

Table 3: Results by using the new topic set

Run	all	auto	manual	average
allpt40bi	0.0272	0.0080	0.0790	0.0435