Online political advertising has grown significantly during the last years. To be able to monitor sponsored political discourse online companies such as Facebook, Google and Twitter have put in place Ad Libraries that contain the political ads that have ran on their platforms. The problem is that there is no global consensus on what is a political ad, each platform provides its own definition, and these definitions can be interpreted in different ways by different people. In this paper we investigate whether there are significant differences between ads labeled as political by advertisers, and ads labeled as political by a group of volunteers. Our results show that advertisers are underreporting ads about social issues while volunteers are underreporting ads from news organizations and ONGs. We analyze how variations in labeling strategies impact political ad classifiers. Finally, we devise a set of experiments to study how political ad definitions and experimental settings are impacting how users decide when an ad is political or not.