Nearly all WordPress blogs are hosted on the Apache server. So much so in fact that WordPress plugins just assume that is the only server in town. They are right about 95% of the time, but InstaBlogs are hosted using the Nginx server. Why? Because it is way faster and more scalable. So why doesn’t everyone host on Nginx? Because it is marginally more work to setup and operate, plus there is some real work required to fully support WordPress.
One of the issues is access to the robots.txt file. WordPress plugins assume the Apache server is configured to provide direct file system access to this file. That’s a horrible idea, no matter what server is used, and we specifically deny for all the best security reasons.
Instead, we auto-create a ‘permissive’ robots file for all Instablogs. This is not modifiable, but if you think it needs to be, please let us know and we’ll look into it.
