Help! Googlebot Cannot Access CSS And JS files On My WordPress Site!

Googlebot cannot access CSS and JS files on...

If you received this message this morning from Google Search Console (formerly Google Webmaster Tools), you’re not alone. Google has recently been pushing harder for webmasters to allow crawlers full access to all Javascript and CSS so that they can render a site to determine whether or not it meets mobile standards (among other things). Many WordPress sites (along with the other major CMSs) received an ominous warning that their site was blocking assets, and it could affect rankings. While technically true, many of the sites were only blocking /wp-admin/, where absolutely no public-facing assets should live. For those of you that are in that situation, relax.

However, a number of sites are also blocking /wp-includes/. While it doesn’t seem obvious, there are a number of things that live here which would need to be accessed by users (i.e., crawlers) to render pages properly. For example, Dashicons, the small icons you generally associate with the admin side of WordPress, can often be called by themes for front-end usage. Another major thing that can hinder proper rendering by crawlers is jQuery. Sometimes, themes will enqueue a different version, but by default, it lives in the /wp-includes/ folder. If we dive even further into the issue, we’d see that the built-in emojis and comment reply handling would also be affected.

So, what can be safely blocked? At this point, here’s what a “compliant” WordPress robots.txt would look like (as far as what is safe to block). Of course, you’d want to add in your own sitemap directives and other special cases, but this is a good starting point.


User-agent: *
Disallow: /cgi-bin/
Disallow: /trackback/
Disallow: /wp-admin/
Disallow: /xmlrpc.php

One big word of warning: DO NOT block anything under /wp-content/. You can assume anything blocked here is going to hit you pretty hard eventually, since everything to do with site rendering exists here; plugins, themes, images, and so on. It was popular at one point to block crawlers (like Googlebot) from plugin folders, but given the amount of Javascript and CSS many plugins employ now, it’s a dangerous proposition. If you’re blocking anything here, open it back up immediately.

Questions or comments? Leave them below!

Adding Your Plugin To The WordPress Plugin Directory

(originally posted on Bring Your Own Design)

Recently, I pushed my first plugin to the official WordPress Plugin Directory called Simple Wistia Embed. In the past, adding your plugin to the directory has been no small feat, but with the streamlined process, it’s now easier than ever to get your plugin in the official plugin repository.

  1. Build your plugin. Simple enough, right? Even if your plugin is extremely simple and lightweight, there are still some things that you’ll want to keep in mind. For example, having a README.TXT is required; it helps to populate information about your plugin in the WordPress Plugin Directory. There are also a number of other requirements and suggestions for your plugin that are spelled out in the Codex. Read them carefully, since you don’t want to go through all the work to get rejected for something simple as a formatting issue!
  2. Submit your plugin. Once you’ve done all the preliminary work of documenting your plugin according to the formatting requirements, the actual submission process itself is quite easy. Simply compress your plugin into a zip file, and visit the Add A Plugin page over at WordPress.org (login required). You’ll need to upload the actual plugin zip file to your own hosting account, so be sure to complete that first. Give your plugin a unique name, describe the method for using your plugin, and paste the link to the zip file.
  3. Once you have approval, you’ll upload to the WordPress SVN. Once your WordPress plugin is approved, you’ll get an email with instructions on how to upload your completed plugin to the WordPress SVN. This is where you’ll eventually upload updates and patches to your plugin in the future, so take some time to become very familiar with the process if you’re not experienced with tools like Git or Subversion. Most IDEs support one or both, but there are plenty of standalone clients if you need them.
  4. Promote your plugin! Now that you’ve got a plugin out in the universe, get people to use it! Be sure to take problems and suggestions seriously, since absent development support can quickly destroy a plugin’s credibility. Updates are critical, since WordPress displays how often you’ve updated and if you’re indicated whether or not you’ve tested with the current version. Outdated plugins get abandoned in favor of up-to-date ones, even if the updated plugin might be a better one.

Hopefully this short post has shed some light on the process of submitting your plugin. It’s always a great feeling to see your work being used by many people to accomplish a task that previously may have been difficult or impossible, so take ownership of your plugin seriously. If you have questions or problems submitting your plugin (or need a custom built plugin), get in touch!