eCommerce Marketing Agency | Visiture

One of the first things we do when auditing a website is check the robots.txt file. Robots.txt is a text file that the search engine spiders visit every time they crawl your site. Within the file you can designate specific instructions for which URLs to index and follow on your website.

The robots.txt file is a great way to stop search engine spiders from indexing cart, catalog, and checkout-type pages. Here’s an easy way to look at it: if you don’t want the URL found naturally in organic search, it should probably be included in the robots.txt with a no-index attribute.

While we know every Magento store has it’s own site structure, we wanted to share a basic robots.txt file template that we use with our Magento clients. It’s important to edit the file based on your specific needs before uploading, but we’d like to provide a good working example:

# Crawlers Setup
User-agent: *

# Allowable Index
Allow: /*?p=

Allow: /media/

# Directories
Disallow: /404/
Disallow: /app/
Disallow: /cgi-bin/
Disallow: /errors/
Disallow: /downloader/
Disallow: /includes/
Disallow: /js/
Disallow: /lib/
Disallow: /magento/
# Disallow: /media/
Disallow: /pkginfo/
Disallow: /report/
Disallow: /skin/
Disallow: /stats/
Disallow: /var/

# Paths (clean URLs)
Disallow: /index.php/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /catalogsearch/
Disallow: /checkout/
Disallow: /control/
Disallow: /contacts/
Disallow: /customer/
Disallow: /customize/
Disallow: /newsletter/
Disallow: /poll/
Disallow: /review/
Disallow: /sendfriend/
Disallow: /tag/
Disallow: /wishlist/

# Files
Disallow: /cron.php
Disallow: /
Disallow: /error_log
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /STATUS.txt

# Paths (no clean URLs)
Disallow: /*.js$
Disallow: /*.css$
Disallow: /*.php$
Disallow: /*?p=*&
Disallow: /*?SID=


For more info on setting up a robots.txt file, check out Magento’s Knowledge Base.

Ever wish you could “Undelete” a keyword in your Google Adwords account? Maybe it was a previous account manager that for no apparent reason deleted a bunch of old keywords with tons of historical value (Whyyyyyyyyyy????).  Maybe the client did it as they fumbled through the account over the weekend. Or, lets not forget the last, most highly unlikely, yet theoretically possible scenario… maybe YOU did it.

Regardless of how the keyword was deleted, we’ve all come across this situation, and wished we could go back in time to save it.

adwords back to the future

If you’ve been around Adwords for any significant length of time, you know how valuable an old keyword can be.  Keywords gather strength over time, and Google rewards this with low cpc’s and what I believe to be some intangible magical power to perform. The older the better.  From lower CPA, to better return on ad spend, the benefits of having old keywords are real. So if you’ve ever deleted a keyword by accident or poor judgment, you know the desperation to undo that act. Especially considering Google makes us think that you can’t undo deleting a keyword. If you try to enable it as if it was paused, you get nothing. Cue the desperation.

Well, the good news is you can save your keywords life! The bad news is you won’t get to drive a supersonic flying DeLorean to make it happen. Instead, you’ll get to play doctor, and revive that old dead conversion machine.

drT 2

And even better – it’s quick and easy. No invasive surgery necessary. When you’re done, you’ll pity the fool that doesn’t know this trick. Just follow these steps:

1) Find the keyword you want to revive within the Google Adwords Interface

Screen Shot 2014-01-10 at 4.42.34 PM

2) Click the green button to add a keyword within the ad group of the deleted keyword you wish to enable

Screen Shot 2014-01-10 at 4.40.43 PM

3) When the box pops up prompting you to enter keywords, enter that keyword EXACTLY. You may want to highlight the deleted version and copy-paste it into the box. Make sure the appropriate symbols are in place according to match type. Click Save.

Screen Shot 2014-01-10 at 4.43.42 PM

4) And voila!- Watch the keyword down below come back to life! It should retain all historical data & quality score as the status changes from deleted to enabled (or paused).

Screen Shot 2014-01-10 at 4.44.29 PM


Congratulations, you’ve done it! It’s Alive!

dr Frank

And that’s that – crisis averted!

A new campaign type is now available for AdWords advertisers and it is called, Search Network with Display Select (SNDS).  It has replaced the old campaign setting: Search & Display Networks, which is no longer available to advertisers creating new campaigns.

In typical Google fashion, the new option is somewhat ambiguous upon first look, so we’d like to lay it out and also explain what types of advertisers could benefit from its utilization.

What is SNDS?

According to Google, this new campaign option allows advertisers to expand the reach of their search campaigns to the GDN with greater confidence that their ads will be shown to relevant users.

How, Google?

Search Network with Display Select uses improved signals and methods of predicting when and where your ads are likely to perform best, and sets a higher bar for when to show them. That means ads are more likely to be shown to a smaller number of prospective customers, who are more likely to be interested in your offerings.”

Ok, sounds cool, but how does it work?

Google will utilize the following data to determine where your ad will show on the Display Network.

1) Search campaign data (quality score, keyword CTR, etc.)

2) GDN historical webpage CTR within your category

3) Mined organic search data matching Google pages to user queries.

Who should use this new campaign type?

1) Advertisers who are currently running campaigns with Search/Display Combined who are unhappy with the budget ratio and performance on the GDN. Expansion will reduce volume/spend in the Display portion of your campaign and initial tests show that advertisers, on average, could see a 35% higher click-through-rate, and a 35% lower cost-per-customer purchase.

2) Advertisers who are running search-only campaigns and want to test performance on the GDN. If you are happy with results after testing, however, the best practice is to separate Google search and display campaigns.


How do I select this campaign type when creating a new campaign?                                           

Screen shot 2013-12-10 at 12.50.56 PM 

How do I upgrade if I am running the older version of Search/Display Combined Campaigns?

Go to your Settings tab within the campaign in question. You will see the below alert from Google. Follow prompts and upgrade.

Screen shot 2013-12-10 at 12.51.02 PM

How do I expand a Search campaign to the Display Select Network?

Go to your Settings tab within the campaign in question. Expand the Campaign Type option and select “Search Network with Display Select”

Screen shot 2013-12-10 at 12.51.08 PM

Again, it is both Visiture’s & Google’s stance that the best way to have control over your Search/Display campaigns is to separate them. Our account managers are certified in both Google Search and Display advertising and suggest knowledge of both channels to achieve optimal results.

In Part 1 of the PLA Series we jumped right into the nuts & bolts of creating product level ad groups within your Product Listing Ads campaign. For this installation we’re going to take a step back, and address some Google Merchant Center basics. Specifically, we’ll discuss how to check for problems in your Google Shopping feed.

Screen shot 2013-11-12 at 6.29.38 PM

If you’ve been working with product feeds in the Google Merchant Center, then chances are you’ve encountered some issues. Don’t worry, we all have.  Errors, warnings, suggestions… what do we make of all this? Well, we’re going to give a basic rundown, so at least the next time you encounter issues, you’ll be able to quickly understand what they mean.


Let’s start with how to get around.  When you sign into the Google Merchant Center, you’ll arrive at the dashboard. Here you’ll see a graph illustrating how many products in your shopping feed are approved/disapproved.

Screen shot 2013-11-12 at 4.40.10 PM

Data Feeds

To understand the disapproved products in your data feed, click on ‘Data Feeds’ in the left sidebar.

Screen shot 2013-11-12 at 4.42.34 PM

Once on the Data Feeds tab, click on ‘view warnings’ next to the feed you’re using for PLA’s

Screen shot 2013-11-12 at 5.39.39 PM

Here you’ll see errors and warnings pointing out problem areas in your feed.

First off, lets distinguish the difference between Errors vs. Warnings.

Warnings– These are suggestions from Google, letting you know that a line item could be improved. However, it shouldn’t be the type of issue that would prevent that particular product from showing up in Google search results as a product listing ad.

Errors– These are more serious and could mean that a particular product is actually disapproved & won’t show up in Product Listing ads. In some cases a whole feed could be shut down. Usually a line item is missing a required attribute.

*Special Note: Our Google Rep has informed me that sometimes errors show up as warnings. So I wouldn’t necessarily assume that just because it’s a warning it’s not ‘serious.’

Ok, so once you’re looking at errors & warnings, you can see what Google doesn’t like.  You can even click on examples of the problem and see what the line item looks like.

Screen shot 2013-11-12 at 4.55.44 PMYou’ll get a pop-up window where you can scroll down to the attribute highlighted in red.

Screen shot 2013-11-12 at 5.06.44 PM

This is a great way to get a quick understanding of what you may need to change in your data feed.



Another way to shed some light on which products aren’t working properly is to click into the ‘Products’ tab.

Screen shot 2013-11-12 at 5.01.57 PM

This tab shows which products are showing on Google Shopping, and alludes to which products may have problems in the feed – namely, the ones with a red circle/backslash symbol.

Screen shot 2013-11-12 at 5.02.25 PMThe products tab is not as revealing as other tabs in the Merchant Center, but I sometimes jump in there to take a quick look. It’s also a way to see a newly uploaded feed as it processes.


Data Quality

The Data Quality tab is yet another place to see problems and how to fix them. In the account below you can see there’s a critical error involving 11% of the feed. So approximately 139 out of 1232 items have landing pages that Google cannot find.

Screen shot 2013-11-12 at 5.16.44 PMThis is a critical problem, and Google has disapproved these products from showing in Product Listing Ads.  You can even click the download button to get a spreadsheet of the disapproved products. Also, you can click on the arrow left of “Product pages cannot be accessed” to get a drop down with further explanation.

Below the critical errors, you’ll see suggested optimizations. As implied, these aren’t serious, and are simply opportunities to improve data quality.

Screen shot 2013-11-12 at 5.22.39 PMIn this example, Google is letting us know that 12 products have a title longer than the recommended 70 characters.

So that’s a basic run down of Google Merchant Center data feed trouble-shooting! There’s obviously much more to know about the Merchant Center and creating effective data feeds for your Product Listing Ad campaigns, but hopefully this helps provide some basic guidance. For more info, you can refer to Google Merchant Center Help. Please feel free to send questions and feedback our way.

Last Tuesday, Google announced a change to their AdWords algorithm. To many, this came as a shock as Google has relied on their ranking system for years now. That said, over the past few months, Google has been “shaking things up”. On the heels of enhanced campaigns, sitelink upgrades, and mobile required inclusion, we at Visiture have developed a thicker skin to announcements such as these.

What’s changing?

In the past, ad rank has been determined by the following equation:

Screen shot 2013-10-31 at 11.19.28 AM

The new algorithm will take into account the expected impact from extensions and ad formats.

Screen shot 2013-10-31 at 11.26.00 AM

In their release, Google used the example that if an two ads with the same bid & quality score were competing for a top position, Google would then rank the ad with the extensions most likely enhance a searcher’s experience higher.

What does “Extension Experience” Mean?

Per Google, to achieve a “positive expectant impact from extensions” said extensions must:

1) Have a Good CTR

2) Be Relevant

2) Hold Prominence in Search Results Pages

Given the fact that extensions are only available to ads with position 1 and 2, we foresee necessary bid increases to compete in this new market. To counter this obvious impact, Google states:

“You may see lower or higher average CPCs in your account. You may see lower CPCs if your extensions and formats are highly relevant, and we expect a large positive performance impact relative to other competitors in the auction. In other cases, you may see higher CPCs because of an improvement in ad position or increased competition from other ads with a high expected impact from formats.” 

What Next?

Give your ad extensions the attention they deserve.  Extensions offer more “real estate” for ad copy, which is, as we all know, a hot commodity. Note that with the algorithm change, Google will show extensions with the best performance and in the best combination to heed positive results.  No longer will PPC managers be able to select and order in which their extensions appear.  While the tedious task is gone, remain aware that Google could potentially optimize out of an important sitelink.

Also, make sure you are utilizing call extensions as well. To several of our clients, a call is solid gold. Google recognizes this and now is rewarding those who do too.

Now, get to it! Thank goodness the new editor makes it somewhat easier to bulk edit extensions!