About this project
Description from the program archive
<p><strong>EvalAI</strong> is an evaluation server that will host AI challenges like Visual Question Answering, Image Captioning. In recent years, it has become increasingly difficult to compare an algorithm solving a given task with other existing approaches. These comparisons suffer from minor differences in algorithm implementation, use of non-standard dataset splits and different evaluation metrics. By providing a central leaderboard and submission interface, we make it easier for researchers to reproduce the results mentioned in the paper and perform reliable & accurate quantitative analysis.<br> The task includes <strong>implementing RESTful web services for EvalAI</strong> which consists of <strong>building a news feed for the portal</strong>, <strong>implementing webhooks</strong>, <strong>creating dashboards</strong> for the participants as well hosts , <strong>setting up continuous deployment</strong>, <strong>writing unit tests</strong> to maintain its integrity and consistency and <strong>scaling it to large number of audience</strong>.</p>