Even though they are pillars, they are indeed blocks that another player placed. Breaking these blocks can easilly be considered griefing and will be accounted for. I totally agree with your statement though, it does make the world and surrounding structures look ugly...
I guess we should wait for a mod/admin to weigh in.